How to Install Clustercontrol Using Podman on Ubuntu 22.04

Clustercontrol is a database cluster management system, developed by severalnines. This tool is really useful if you want to install and manage multiple database clusters from one interface, easily. This tool supports many database types such as mysql, mariadb, postgresql, timescaledb and also redis.

This software can easily be installed using docker, just follow the instructions here. For podman users, worry not, below are the detailed instructions on how to do the same using podman. These steps have been tested on ubuntu 22.04, but they should run in any linux that support podman.

1. Make sure you have podman installed, please refer here if you have not install podman

2. Create some directories for clustercontrol's data and configuration

$ mkdir -p clustercontrol/{cmon.d,datadir,sshkey,cmonlib,backups,prom-data,prom-conf}

3. Get the machine's ip address

$ hostname -I

4. Copy ssh key into sshkey directory

$ cp ~/.ssh/id_rsa ~/clustercontrol/sshkey

4.1 If you don't have ssh keys yet, please follow here to generate a pair

5. Run clustercontrol using podman (image 1.9.5-4 is fully working at the time of writing of this article)

podman run -d --name clustercontrol \

-h clustercontrol \

-p 5000:80 \

-p 5001:443 \

-p 9443:9443 \

-p 19501:19501 \

-e DOCKER_HOST_ADDRESS=192.168.10.10 \

-v $PWD/clustercontrol/cmon.d:/etc/cmon.d \

-v $PWD/clustercontrol/datadir:/var/lib/mysql \

-v $PWD/clustercontrol/sshkey:/root/.ssh \

-v $PWD/clustercontrol/cmonlib:/var/lib/cmon \

-v $PWD/clustercontrol/backups:/root/backups \

-v /storage/clustercontrol/prom-data:/var/lib/prometheus \

-v /storage/clustercontrol/prom-conf:/etc/prometheus \

docker.io/severalnines/clustercontrol:1.9.5-4

6. Open a browser, and browse to https://192.168.10.10:5001, and replace the 192.16810.10 to your own ip address that you use in the command above. You should be able to see below page. Register and create user to start using clustercontrol.

Unable to ssh into docker playground virtual machine (Permission denied (publickey) error)

Docker playground is a very useful place to learn how to use docker. However, the web interface is sometimes can be quite difficult to use, especially if we are trying to copy long commands into the virtual machine. 

A good solution to this, is to connect to the virtual machine using ssh. We can copy the link at the ssh column of the virtual machine, and paste it in our terminal. 

One of the issue that we encounter when we are trying to ssh into the virtual machine, is we will get permission denied (publickey) error, like below 

The reason this happened is, the ssh server inside the playground's virtual machine is expecting the client to connect from a machine that owns a ed25519 key. This can be verified by running below command inside the playground's virtual machine

To encounter that, simply create an ed25519 in our machine, using ssh-keygen

$ ssh-keygen -t ed25519

We should be able to ssh into the playground's virtual machine now

Hide Apache Httpd Version in HTTP Header

Hiding software version in any deployment is a basic security practice that we can use to lower the risk of the deployment being breached. In this post, we will see how we can hide the apache httpd version from the http header, and from server generated pages.

To check our header, just use curl. Let's say we have an apache httpd server running on localhost

$ curl --header http://localhost

The version will also showing in the server generated page, like when we tried to access non existent page

$ curl --header http://localhost/error

To hide the version number, we can just add below line into httpd.conf. I usually will put it at the bottom of the configuration file. The location of the httpd.conf will varies depending on how you install httpd. The usual location is at /etc/httpd/conf/httpd.conf:

ServerToken Prod

ServerSignature Off

"ServerToken Prod" will hide apache httpd version from http header, while "ServerSignature Off" will hide the version from server generated pages.

Example is like below

To make sure that our change is syntax error free, test with "apachectl -t"

Once we are satisfied, restart apache httpd

# systemctl restart httpd

Then, we test it back using curl, and we do not see the version anymore

$ curl --head http://localhost

$ curl --head http://localhost/error

Exiting a docker container running in interactive mode

To exit from a docker container while in interactive mode (using the -it option without -d), there are 2 options:

1. Press ctrl-d to exit the shell (if you are in it) and exiting the container

2. Press ctrl-p, then ctrl-q to daemonize the container, making it run in the background without occupying the terminal

Using psql from command line to get data from postgresql

Sometimes we need to get some data from postgresql database, and we want the output to appear on the terminal so that we can further process the output.

Lets say, we want to get a list of actor from a database called dvdrental, we can simply use below command:

$ psql -U postgres -d dvdrental -c "select * from actor;"

The output will be in an interactive mode if the output is very long.

In order to run psql with output that is not interactive, we can use here-document method. The method comprise of a "<<" symbol followed by some text used as ending text for the here-document. For example, in order to get the same result as above using here-document method, we can use below command:

$ psql -U postgres -d dvdrental <<END
select * from actor;

END

The END keyword is a signal to end the here-document, thus executing the command. The output will be like below, which is not interactive, but easy to copy and paste:

Backing Up and Restoring iptables Rules

To backup iptables rules, we can use iptables-save command. This command is to save current iptables rules into a file called iptables.backup

sudo iptables-save > iptables.backup

To restore, simply use iptables-restore. This command is to restore iptables rules from a backup file called iptables.backup

sudo iptables-restore iptables.backup

We can also test the backup file, before committing the ruleset to sytem

sudo iptables-restore -tv iptables.backup

Where -t is for testing, and -v is for verbose.

Flushing and Clearing iptables Rules

Sometimes we need to clear out iptables, and start from scratch in setting up firewall rules. In order to do that, below are the commands to follow

sudo iptables -t nat -F

sudo iptables -t mangle -F

sudo iptables -F

sudo iptables -X

The first command flush out rules in nat table. The second one will flush rules in mangle table. The third command will flush all rules in all chains. The last one will delete all non default chains in iptables.

In order to verify, we can use below command:

sudo iptables -L -n

The above command will print out all rules in all chains, and all port and ip address will be printed in numeric format. 

Cleared out iptables will look like below: 

Turning On Mysql Slow Query Log

Slow query log is a mechanism in mysql which will track and record queries that take long time to complete, for database administrator to analyze and optmize to increase mysql performance.

To check if this feature is already turned on, we can run below command in mysql console:

mysql> show variables like 'slow_query_log';

We will see something like below if it is already turned on: 

The query will be deemed slow, of the query takes longer than 'long_query_time' to complete. We can also check this using below command:

mysql> show variables like 'slow_query_log';

In this case, our long_query_time is set to 10 seconds:

We can check the location of the slow_query_log_file by using below command:

mysql> show variables like 'slow_query_log_file';

In this example, the log file is located in /bitnami/mysql/data/5653c641c26b-slow.log

The above variables, can be set in the runtime using below commands:

To turn off slow_query_log

mysql> set global slow_query_log = off;

To set long_query_time to 5 seconds

mysql> set long_query_time = 5;

All the changes that we made in mysql console, is only for the current runtime, and would not survive mysql service restart. To make the changes permanent, we need to modify /etc/my.cnf, under [mysqld], like below

[mysqld]

slow_query_log=1

long_query_time=5

slow_query_log_file=/var/log/mysql/slow.log

And then restart mysql

$ sudo systemctl restart mysql

Checking Disk Read & Write Speed in Linux

We are going to use 2 tools for this, one is called hdparm to test read speed, and the second one is called dd, to test write speed.

First, install hdparm

$ sudo apt install hdparm -y

Then, get the disk name. The name of the disk in below example is /dev/sda.

$ lsblk

Test read speed using hdparm. -t option is to record the timing for the read process. In below example, the read speed is around 700MB/sec.

$ sudo hdparm -t /dev/sda

To test write speed, we will use dd command. dd will write a 1M file called speedtest.img to disk, and the "oflag=direct" option will make dd use direct IO, skipping any filesystem cache. dd will show the disk write speed after it finishes writing the file to disk. In below example, the write speed is 100MB/s.

$ dd if=/dev/zero of=speedtest.img bs=1M count=1 oflag=direct

Extract Text From Image Using Command Line

The tool to use for this task is called tesseract.

To install tesseract in any ubuntu derivatives, just run below command

$ sudo apt install tesseract-ocr -y

To extract text from an image file called 007.png, run below command

$ tesseract 007.png 007output

007output is the output file, and an extension of .txt will always be put to the output file.

To view the output, just use cat like below

$ cat 007output.txt

Print Column Until End of Line in Linux Terminal

I have a document named mydoc.txt which contains below lines

local STAFF

local STUDENTS

local CONTRACT TEACHERS

I need to print all except the first column. To accomplish that, I can use this command

$ cut -f2- -d ' ' mydoc.txt

Where -d is delimiter, and -f is field number. The hyphen after -f2 is to tell 'cut' to print field number 2 until the end of line.

Remove Defunct (Zombie) Process Linked to Cinnamon in Linux Mint

I encountered this situation before, whereby my Linux Mint (cinnamon) is showing that it has 20 zombie processes, linked to the cinnamon process as parent process.

How to check for zombie processes?

$ ps -ef | grep defunct

Or you can run top, and view the second line for "zombie" keyword

$ top

To exit from top, just press q

Calculate how many zombie processes inside a system

$ ps -ef | grep defunct | grep -v grep | wc -l

One of the way to remove zombie processes is to restart the parent process. How to check which is the parent process of the zombie processes?

$ ps -ef | grep defunct | awk '{print $3}'

Once we get the PID of the parent process, check what is the actual process that is using the PID

$ ps -ef | grep <PIDNUMBER>

For example, if the PID number is 2323

$ ps -ef | grep 2323

Once we know what is the process, we can safely restart the parent process. 

In the case of cinnamon, a restart can be accomplished by simply pressing ctrl + alt + esc. 

If that is not working, kill the cinnamon PID first, then do ctrl + alt + esc.

$ kill 2323

Your desktop will freeze after you kill cinnamon, but you can restart it by pressing ctrl + alt +esc.

Sharing Files Over http Using Nodejs In Docker

Sometimes we just need to share some files over network to some friends, and need a solution that is easy and fast to setup, provided we already have docker installed in our machine.

First, prepare a directory. Then, put all the files that we want to share inside the directory

$ mkdir files

Then, run a container based on nodejs:slim image, and mount the above directory to our container, which is named "fileshare" in this example

$ docker run -dit -p 8080:80 --name fileshare -v $PWD:/files -w /files node:slim

Install http-server inside the container

$ docker exec -it fileshare npm install -g http-server 

Run http-server inside the container

$ docker exec -it fileshare http-server -p 80 .

You should now be able to view your files using a web browser. Just browse to your ip with port 8080 like below

Once you are done, just press control-C on the terminal where the http-server is running, and the http-server will be terminated

Scheduling Tasks With systemd.timer

Systemd.timer is a way to scheduling jobs and tasks in systemd based linux system. The systemd timer units are identified by ".timer" (dot timer) file name extension, compared to ".service" for service units. 

Each timer file requires a service file for it to work. In other word, systemd timers only can schedule systemd services.  

Some of systemd.timer features:

1. timer is managed by systemd similar to other units, using systemctl command
2. timers can be triggered by calendar event, or triggered by specific time elapsed from a certain starting point
3. time units are logged to journal, for easier troubleshooting and monitoring
4. if the system is off during the expected execution time, the timer will be executed when the system is running again

The usage of timer is best explained using an example. Let's say we want to create a timer to run a script called hello.sh.

First, we need to create the script, and get the location of the script. Let's say we created the script inside /usr/local/bin/hello.sh

Next, we need to create a systemd service unit for the above script. Just insert below settings into /etc/systemd/system/hello.service

[Unit]

Description="Hello app"

[Service]

ExecStart=/usr/local/bin/hello.sh

Then, create a systemd timer unit for hello.service. Just add below setting into /etc/systemd/system/hello.timer

[Unit]

Description="Run hello.service 5min after boot, every 24 hours relative to activation time, and everyday at 10:00 am"

[Timer]

OnBootSec=5min

OnUnitActiveSec=24h

OnCalendar=Mon..Fri *-*-* 10:00:*

Unit=helloworld.service

[Install]

WantedBy=multi-user.target

The above example will run the script after 5 minutes of system boot, 24 hours after the timer activation time, and everyday at 10:00am. The format for OnCalendar is as below

Day of Week = Sun to Sat

Date = in yyyy-mm-dd format

Time = in hour:minute:seconds format

For example, if we want to execute the script at 3:00 pm every Monday and Friday

OnCalendar=Mon,Fri *-*-* 15:00:00

Or execute the script at 4:00 pm 20 February 2023

OnCalendar=2023-02-20 16:00:00

For more example on setting the timer, we can type this command to see the manual page of systemd.time

# man 7 systemd.time

Before we use the new systemd unit files for hello.sh, we can verify that they are error free

# systemd-analyze verify /etc/systemd/system/hello.*

If no error detected, then we can start the timer

# systemctl start hello.timer

We can also enable the timer on boot

# systemctl enable hello.timer

To check the status of timer

# systemctl status hello.timer

To list all active timers

# systemctl list-timers 

To list all timers, regardless whether they are active or note

# systemctl list-timers --all

To stop the timer

# systemctl stop hello.timer

Change DNS server using resolvectl

For systemd based system, the configuration of dns resolution is controlled by a service called systemd-resolved.service.

In order to change the dns server that the machine refers to for any DNS request, we can use a command called resolvectl.

To set the current dns server in interface wlp4s0 to 1.1.1.1, use this command

sudo resolvectl dns wlp4s0 1.1.1.1

To set the dns server in interface wlp4s0 to 1.1.1.1 and 8.8.8.8, use this command

sudo resolvectl dns wlp4s0 1.1.1.1 8.8.8.8

We can specify as many dns servers as we like. Just separate them with spaces.

To check our currently set dns server for interface wlps4s0, use this command

resolvectl dns wlp4s0

We can see dns settings for all interfaces in the system by running

resolvectl status

We can also query (get the ip address from dns) record using resolvectl, just like below

resolvectl query www.linuxwave.info

To query different type of dns record, we can add the -t flag. This is an example if we want to query an MX record for gmail.com

resolvectl query -t MX gmail.com

We can also do reverse query, by specifying ip address instead of domain name

resolvectl query 142.250.191.51 

SSH Tunnel That Listens To Non Loopback (Non Localhost) IP

In order for ssh tunnel to listen to non localhost interface, we have to enable "GatewayPorts" in /etc/ssh/sshd_config

1. Open /etc/ssh/sshd_config using your preferred text editor. I use vi

$ sudo vi /etc/ssh/sshd_config

2. Turn on "GatewayPorts" by adding below into /etc/ssh/sshd_config

GatewayPorts yes

3. Save and exit the text editor

4. Restart ssh

$ sudo systemctl restart ssh

5. Now we can create a local/remote tunnel, and the tunnel can be made to listen to non loopback (non localhost). For example, we can forward a port from another machine, and make it to listen to all ipv4 address in our local machine

$ ssh -R 0.0.0.0:1111:localhost:22

6. We can verify it by using ss command

$ sudo ss -tulpn | grep 1111

...

tcp   LISTEN 0      128                             0.0.0.0:1111        0.0.0.0:*                   users:(("sshd",pid=xxxxx,fd=10))