Saturday, May 8, 2021

Listing docker containers according to fields

Command like 'docker ps' is a good tool to check on your running container/s. This is visually pleasant if you do not have many containers. What if you have hundreds of containers, and you just want to print just the names of the containers, or even better the names and id of the containers?


We can use --format flag in this situation. This flag is available on pretty much every docker commands that produce some kind of output to stdout, so that you can filter what you want to see as the output.

To use this flag, you just need to follow below format:
$ docker ps --format '{{json .Names}}'

"hardcore_carson" 


whereby ".Name" is the field that you want to be displayed. For example, you want to list out just the ID of all the running containers, you can use:
$ docker ps --format '{{json .ID}}'

"e914bd4963d4" 


You can see that the field is different from the displayed field name without the --format flag.
$ docker ps
CONTAINER ID   IMAGE     COMMAND     CREATED          STATUS          PORTS     NAMES
e914bd4963d4   alpine    "/bin/sh"   29 minutes ago   Up 29 minutes             hardcore_carson

To know which flag is available to be used:
$ docker ps --format '{{json .}}'

{"Command":"\"/bin/sh\"","CreatedAt":"2021-05-08 11:32:12 +0800 +08","ID":"e914bd4963d4","Image":"alpine","Labels":"","LocalVolumes":"0","Mounts":"","Names":"hardcore_carson","Networks":"bridge","Ports":"","RunningFor":"33 minutes ago","Size":"0B (virtual 5.61MB)","State":"running","Status":"Up 33 minutes"} 


To list 2 (or more) fields:
$ docker ps --format '{{json .ID}} {{json .Names}}'

"e914bd4963d4" "hardcore_carson" 


You can also use the --format without the json keyword, the only different is the output would not be double quoted (which is not easy on the eyes if you have many fields)

$ docker ps --format '{{.ID}} {{.Names}}'

e914bd4963d4 hardcore_carson

 

Thursday, April 29, 2021

Checking if our password is based on a dictionary word on linux command line

There is a package in linux that can be used to check the our password, if it is based on a dictionary word, and that is libcrack (or libcrack2 in debian based).


To install the package:
  • In redhat family:
# yum install cracklib -y
  • In debian family:
# apt install libcrack2

To use, just run cat and pipe it against cracklib-check command, and supply your password after that. Do not forget to type ctrl-d or ctrl-c to get back to the shell, once done.
$ cat | cracklib-check

password 

password: it is based on a dictionary word
$ cat | cracklib-check

Xeir3oongex*

Xeir3oongex*: OK

Thursday, April 22, 2021

Adding Rules to Firewalld before Starting It

This is really useful if we want to start firewalld via ssh. If we start firewalld without allowing ssh, we will be locked out from the machine.

The solution is, to use a command called firewall-offline-cmd. This tool acts similarly with firewal-cmd, except it works during the daemon is dead.

To avoid being locked out of a remotely accessed, we should first allow ssh in firewalld

$ sudo firewall-offline-cmd --add-service ssh

We are now safe to start firewalld

$ sudo systemctl start firewalld

Once started, we can make the rule permanent on firewalld restart 

$ sudo firewall-cmd --add-service ssh --permanent

Make firewalld start automatically on every server boot 

$ sudo systemctl enable firewalld


Wednesday, April 21, 2021

Changing Kernel on Next Boot

To choose which kernel version you want to boot into on next reboot, below are the steps


1. Check what kernel version is available

# grep ^menuentry /etc/grub.cfg

2. Choose which kernel that you want to boot from, remember that the list from the above command start from 0. Let's say we want to choose the second kernel

# grub2-set-default 1

3. Rebuild grub.cfg

# grub2-mkconfig -o /boot/grub2/grub.cfg

4. Reboot the server

# reboot 


You server will reboot to the kernel version that you choose above. 

Sunday, April 18, 2021

Running php-fpm and Nginx in Docker

Php-fpm is an advanced and highly efficient processor for php. In order for your php files to be viewable in a web browser, php-fpm needs to be coupled with a web server, such as nginx. In this tutorial we will show how to setup php-fpm and nginx is docker.

1. Create a directory for your files 

$ sudo mkdir phpfpm

2. Create a network for the containers to use. This makes sure that we can use container's name in the configuration file.

$ docker network create php-network

3. Create nginx config file

$ cd phpfpm

$ cat > default.conf <<EOF

server {

    listen  80;    

# this path MUST be exactly as docker-compose.fpm.volumes,

    # even if it doesn't exist in this dock.

    root /complex/path/to/files;

    location / {

        try_files $uri /index.php$is_args$args;

    }

    location ~ ^/.+\.php(/|$) {

        fastcgi_pass fpm:9000;

        include fastcgi_params;

        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; 

    }

}

EOF

4. Create an index.php file with some random php code (we are using phpinfo() to make it easier)

$ cat > index.php <<EOF

<?php phpinfo(); ?> 

EOF

5. Run a php-fpm container, in detached and intaractive mode, using php-network, and we mount /home/user/phpfpm to /var/www/html in container

$ docker run -dit --name fpm --network php-network -v /home/user/phpfpm:/var/www/html

6. Run an nginx container in detached and intaractive mode, using php-network, and we mount /home/user/phpfpm/default.conf to /etc/nginx/conf.d/default.conf in container

$ docker run -dit --name nginx --network php-network -v /home/user/phpfpm/default.conf:/etc/nginx/conf.d/default.conf -p 80:80 nginx

7. Open a browser, and browse to http://localhost, you should now be able to see the PHPinfo page. 

Of course, there is an easier way to set this up using docker-compose. We will cover that in another post.


Saturday, April 17, 2021

Hiding Apache2 (httpd) Version in HTTP Header

One of the basic concept of cybersecurity, is to hide as much information about your system from the public view. For apache2 (httpd), this is pretty easy to do.

1. First, open /etc/httpd/conf/httpd.conf

$ sudo vi /etc/httpd/conf/httpd.conf

2. Then, append below lines to the file

...

ServerTokens Prod

ServerSignature Off

3. Save the file

4. Test the configuration, to make sure no typo error that can cause httpd to fail to start
$ sudo httpd -t

5. Restart httpd to activate the settings

$ sudo systemctl restart httpd

6. Finally, you can verify the visibility of the webserver's version number using curl or wget 

$ curl --head http://www.mydomainname.com

...

Server: Apache

... 

 

Thursday, April 15, 2021

Checking Web Server Version Using Command Line

We usually use these methods to verify what is being displayed in our HTTP header to the public. There are 2 tools that can be used, curl and wget.


To use wget:
$ wget --server-response --spider http://www.mydomain.com
...
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_fcgid/2.3.9
...

To use curl:
$ curl --head http://www.mydomain.com
...
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_fcgid/2.3.9
...

Wednesday, April 14, 2021

Accessing MTP Mounted Device Via Command Line

When you connect your android phone to a linux box using usb cable, the storage of the phone will appear in your file manager (thanks to automount). It is easily accesible from there, but what if you want to access it via command line? Where is it located?

To know the location of the MTP mounted storage, you need to know your user id

$ id

uid=1000(user) gid=1000(user) groups=1000(user),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),114(lpadmin),134(sambashare)

From the above command, the user ID is 1000. The MTP mounted device can be accessed at /run/user/<user ID>/gvfs

$ ls /run/user/1000/gvfs/

'mtp:host=Android_Android_28f5d3440504'

Just go to the 'mtp:host=Android_Android_28f5d3440504' directory (the name might differ), and you will see your phone's storage.

 

 

 

 

Thursday, April 1, 2021

Getting Access Denied Error when Using systemctl as root

 I got this error in one of our server, when trying to restart nginx

# systemctl status nginx

Failed to get properties: Access denied


Does not make sense, I am a root user. After some searching, a few suggestions came. 

The first suggestion was to restart systemctl daemon:

# systemctl daemon-reexec


That did not work for me. Another solution is to disable selinux temporarily, but this also did not work for me:

# setenforce 0 


The last thing that I tried (that actually worked) was to sending sigterm to systemd, and it will restart by itself:

# kill -TERM 1


If you guys happened to encounter this sort of error, you can try all the above. Some might suit you better than the other.

Installing elinks in CentOS 8

Elinks is a text based web browser, and it is now available in powertools repository. Powertools repository is not enabled by default, thus elinks is not available to be installed just by using standard yum install command.


List all available repositories

$ sudo yum repolist --all

...

powertools

...


Install elinks while enabling powertools repository temporarily

$ sudo yum install --enablerepo=powertools elinks -y

...

Installed:

  elinks-0.12-0.58.pre6.el8.x86_64                                                     gpm-libs-1.20.7-15.el8.x86_64                                                    

Complete!


Congratulations, you can now use elinks to view any website, but only in text based mode.

Wednesday, March 10, 2021

Fixing Wrong Timezone in Docker Logs

I had this issue of my running container logs is using UTC as its timezone, which will make troubleshooting process quite troublesome.

After some experiment, I found that we have to set the timezone early when we run the container, to make the logs recorded in the timezone we prefer.

For currently running container, the safest way is to commit it to image, and rerun a new container based on that image.

1. My current container's logs

$ docker logs mycontainer

...

2021-03-09 12:01:08.939 UTC [1283819] WARNING:  terminating connection because of crash of another server process

...

2. Back up the current container to an image called mycontainer-backup:20210310

$ docker commit mycontainer mycontainer-backup:20210310

3. Stop the current container, to avoid port clash (if any)

$ docker stop mycontainer

4. Run a new container, based of the backup image, this time we need to specify the timezone variable

$ docker run -dit --name mynewcontainer -e TZ=Asia/Kuala_Lumpur mycontainer-backup:20210310

5. Done, we can verify the log's timezone, just to be sure

$ docker logs mynewcontainer

...

2021-03-10 08:03:14.429 +08 [1] LOG:  listening on IPv4 address "0.0.0.0",

...

Monday, March 8, 2021

Postgresql 13 Streaming Replication using Docker

In this setup, we will create a 2 nodes postgresql streaming replication, in docker. This article will make use of postgresql image version 13.

1. Create a network, and take note of the network ip range
$ docker network create mynet

$ docker network inspect mynet 

2. Make 3 directories for pgmaster, pgdata and pgbootstrap. The bootstrap directory will be used to setup pgslave the first time.

$ sudo mkdir -p /dockervol/{pgmaster,pgslave,pgbootstrap}
3.  Create a container called pgmaster
$ docker run -dit -v /dockervol/pgmaster/:/var/lib/postgresql/data -e POSTGRES_PASSWORD=abc -p 5432:5432 --restart=unless-stopped --network=mynet --name=pgmaster postgres 
4. Backup and edit pgmaster's postgresql.conf with below settings
$ sudo cp /dockervol/pgmaster/postgresql.conf 
/dockervol/pgmaster/postgresql.conf.ori
# cat > postgresql.conf <<EOF
listen_addresses = '*'
port = 5432
max_connections = 50
ssl = off
shared_buffers = 32MB
# Replication Settings - Master
wal_level = hot_standby
max_wal_senders = 3
EOF

$ sudo cp postgresql.conf /dockervol/pgmaster/postgresql.conf 

5. Login to pgmaster and create a user for replication
$ docker exec -it pgmaster psql -U postgres -h localhost -d postgres
postgres=# create role replicator with replication password 'abc'
6. Backup and edit pgmaster's pg_hba.conf with ip range from step 1
$ sudo cp /dockervol/pgmaster/pg_hba.conf 
/dockervol/pgmaster/pg_hba.conf.ori
$ echo "host    replication  all  172.16.0.0/16  trust" | sudo tee -a /dockervol/pgmaster/pg_hba.conf 
7. Restart pgmaster container
$ docker restart pgmaster
8. Run bootstrap container
$ docker run -dit --rm -v /dockervol/pgbootstrap:/var/lib/postgresql/data -v /dockervol/pgslave:/var/lib/postgresql/slave-data -e POSTGRES_PASSWORD=abc --network=mynet --name=pgbootstrap postgres

9.  Run backup of master to /var/lib/postgresql/slave-data in pgbootstrap

$ docker exec -it pgbootstrap su - postgres

$ pg_basebackup -h pgmaster -D /var/lib/postgresql/slave-data -U replicator -v -P --wal-method=stream

10. Stop pgbootstrap

$ docker stop pgbootstrap

11. Tell pgslave that it is a slave

$ sudo touch /dockervol/pgslave/standby.signal

12. Edit postgresql.conf in /dockervol/pgslave

$ sudo cp /dockervol/pgdata/postgresql.conf /dockervol/pgdata/postgresql.conf.ori

$ cat > postgresql.conf <<EOF

listen_addresses = '*'

port = 5432

max_connections = 50

ssl = off

shared_buffers = 32MB

# Replication Settings - Slave

hot_standby = on

primary_conninfo = 'host=<master ip> port=5432 user=replicator password=abc@123'

EOF

$ sudo cp postgresql.conf /dockervol/pgdata/postgresql.conf

13. Start pgslave

$ docker run -dit -v /dockervol/pgslave/:/var/lib/postgresql/data -e POSTGRES_PASSWORD=abc -p 15432:5432 --network=mynet --restart=unless-stopped--name=pgslave postgres

14. Check replication state in pgmaster

$ docker exec -it pgmaster psql -h localhost -U postgres -d postgres -c "select usename,state from pg_stat_activity where usename = 'replicator';"

usename   | state  

------------+--------

 replicator | active 

15. Verify the setup by creating a database in pgmaster, and check if the same database appear in pgslave. 

16. To promote to pgslave if pgmaster is down, simply run "pg_ctl promote" command

$ docker exec -it pgslave bash

# su - postgres

# pg_ctl promote

Saturday, March 6, 2021

Configuring Mysql Asynchronous Replication

In this exercise, we will use one master and one slave. The addresses are as follow:

master: 10.0.0.10

slave: 10.0.0.20


1. Make sure mysql-server is installed in both machines

2. In master, check the user_id

$ mysql -u root -p

mysql> show variables like 'server_id';

3. In master, create a user for replication

mysql> create user replicator@'%' identified with 'mysql_native_password' by 'mypassword';

3. In master, grant the user a privilege of slave replication

mysql> grant replication slave on *.* to replicator@'%';

4. In slave, change the server_id to a number different from the master

$ mysql -u root -p

mysql> set global variable server_id = 20;

5. Configure slave with the server's details

mysql> change master to master_host='10.0.0.10',master_user='replicator',master_password='mypassword';

6.  Start slave

mysql> start replica;

7. Check slave status. Make sure slave_io and slave_sql are in running state.

mysql> show slave status\G

8. Test the setup. Create database, add table and some data into master, and check if the data gets replicated to slave. 

Sunday, February 14, 2021

Connecting to GlobalProtect VPN in Linux Mint

I encountered GlobalProtect (GP) vpn while working on a project, and somehow the vpn portal does not have any linux client for me to connect to the server. They have windows and mac though, so I tried searching around for solution.


After a while, I stumbled upon this post and this other post, stating that openconnect client can connect to GP vpn.

To install openconnect is fairly easy. Just fire up your terminal, and use below command to install openconnect client
$ sudo apt install openconnect -y

Once installed, you just have to use below command to connect to your GP vpn
$ sudo openconnect --protocol -u foo vpn.server.com

You will get some warning about "Certificate failed verification", just answer yes
Certificate from VPN server "vpn.server.com" failed verification.
Reason: signer not found
To trust this server in future, perhaps add this to your command line:
    --servercert pin-sha256:1YWmjjGL3wppl245dRc3/p+mytteBnvaVz456DQY+wutt=
Enter 'yes' to accept, 'no' to abort; anything else to view: yes 

It will later ask for you password, just put in your password
Connected to HTTPS on vpn.server.com
Enter login credentials
Password: 

You will know that you are connected if you find something resembles below line
Connected as 192.168.100.72, using SSL, with ESP in progress

Try to access your internal server, and you should be able to.

Rejoice!

Saturday, February 6, 2021

Displaying Text on Remote Desktop over SSH

I used this trick to warn my kids that their computer playing time is almost over. The application name is zenity, and we need to set our DISPLAY environment to :0 beforehand so that the message will appear on their screen.

1. ssh into the machine

$ ssh foo@machine

2. Set the DISPLAY environment

$ export DISPLAY=:0

3. Use zenity to prompt messages to their screen. In this case, I use warning style message, and will be displayed for 2 seconds

$ zenity --warning --timeout=2 --text="Computer will be shut down in 15 minutes"







Monday, January 25, 2021

Replacing Newline with Space

Let's say we have a list of words like below

$ cat animals
cat
fish
zebra
monkey

and we want it to be arranged in one horizontal line, separated by space. We can achieve that by using tr command.
$ cat animals | tr "\n" " " 
cat fish zebra monkey

What happened is, \n (symbol for newline), will be replaced with " ", which is symbol for space. 

Sunday, January 24, 2021

Generating Certificate Signing Request (CSR) for Multi Domain

For multi domain, we have to create a config file for openssl command to refer to, since the interactive mode would not, by default ask for multi domain in a CSR creation.


To create the config file, please follow below command (this example is for mydomain.com)

$ cat >> www-portal.mydomain.conf <<EOF

[req]

distinguished_name = req_distinguished_name

req_extensions = v3_req

prompt = no

[req_distinguished_name]

C = MY

ST = Selangor

L = Cyberjaya

O = MyCompany

OU = Software Development Division

CN = www.mydomain.com

[v3_req]

keyUsage = keyEncipherment, dataEncipherment

extendedKeyUsage = serverAuth

subjectAltName = @alt_names

[alt_names]

DNS.1 = portal.mydomain.com

EOF


Run openssl CSR creation command against the config file

$ openssl req -new -newkey rsa:2048 -nodes -keyout www-portal.mydomain.key -out www-portal.mydomain.csr -config www-portal.mydomain.conf


Once generated, we can send the CSR to the Certificate Authority (usually SSL provider), to get our cert. This one CSR is usable for 2 domains, which are www.mydomain.com and portal.mydomain.com.


Generating a Certificate Signing Request (CSR) for a Single Domain

To generate a certificate signing request (CSR), you need to have openssl package installed. Please refer here for the instruction on how to install it.


Once you have openssl installed, please use below command to create a CSR with key for mydomain.com. 

$ openssl req -new -newkey rsa:2048 -nodes -keyout mydomain.com.key -out mydomain.com.csr

Press Enter and you will need to provide a few information regarding the CSR. The information are as follows:

  1. Common Name: The FQDN (fully-qualified domain name) you want to secure with the certificate. For example: mydomain.com
  2. Organization: The full legal name of your organization including the corporate identifier. For example: MyCompany Co
  3. Organization Unit (OU): Your department such as 'Information Technology' or ‘Website Security.’
  4. City or Locality: The locality or city where your organization is legally incorporated. Do not abbreviate. For example: Cyberjaya
  5. State or Province: The state or province where your organization is legally incorporated. For example: Selangor
  6. Country: The official two-letter country code where your organization is legally incorporated. For example: MY

Once the CSR has been generated, we can provide it to the SSL provider, so that they can use it to provide the SSL for your domain. Please be mindful to keep the key file, because we will need it during our SSL setup.

Friday, January 22, 2021

Installing Openssl Application to Use SSL Functions

The openssl program is a command line tool for using the various cryptography functions of OpenSSL's crypto library from the shell. 


To install openssl in ubuntu or debian:

$ sudo apt install openssl -y


To install openssl in RHEL, CentOS or Fedora:

$ sudo yum install openssl -y


Tuesday, January 19, 2021

How to Redirect HTTP Traffic to HTTPS in httpd on CentOS

The easiest way is to do it using the VirtualHost configuration, if you have control over it. 


Edit your virtualhost configuration for that domain, in CentOS it is usually located in /etc/httpd/conf.d/mydomain.conf:

<VirtualHost *:80>
   ServerName www.mydomain.com
   Redirect / https://www.mydomain.com
</VirtualHost>

<VirtualHost _default_:443>
   ServerName www.mydomain.com
   DocumentRoot /usr/local/apache2/htdocs
   SSLEngine On
...
</VirtualHost>

The most important line is the "Redirect" line which will redirect all http traffic to https.

Once done, save the file.

Do not forget to run syntax test of the configuration files.
# httpd -t

And reload the service
# systemctl reload httpd

Sunday, January 3, 2021

Force Logout Other User from Linux

First, we need to know the username. This can easily be done using w command

# w

12:34:31 up 36 days, 14:14,  2 users,  load average: 0.07, 0.02, 0.00

USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT

ironman   pts/0    ::1              12:34    2.00s  0.01s  0.01s w

root     pts/3    10.29.25.230     12:22    7.00s  0.13s  0.00s bash


Let's say we want to log out ironman from our server. What we have to do is to use a command called pkill against that user.

# pkill -u ironman 


All processes that is owned by that user will be killed. 
# w

12:34:31 up 36 days, 14:14,  2 users,  load average: 0.07, 0.02, 0.00

USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT

root     pts/3    10.29.25.230     12:22    7.00s  0.13s  0.00s bash


The ironman user is no longer logged in. Easy peasy.