Thursday, May 31, 2018

Creating openstack new security group, and allowing inbound port

To see existing security group
$ openstack security group list

To create a new security group named ssh-allow
$ openstack security group create  ssh-allow



To allow port 22 inbound (ingress)
$ openstack security group rule create --ingress --proto tcp --dst-port 22 ssh-allow

To see if our port allowance is successfully implemented (use --fit-width to format the output properly)
$ openstack security group show ssh-allow --fit-width

Monday, May 28, 2018

Setting up mongodb replication

Mongodb needs at least 2 servers, preferably 3, to setup a proper mongodb replication. In this article, we will use below hostname as our mongodb nodes:

192.168.0.10 mongo-1 (primary)
192.168.0.11 mongo-2
192.168.0.12 mongo-3



Make sure mongodb is installed in all servers.

Set mongodb repo:

mongo-1: $ cat > mongodb.repo << EOF
>[mongodb]
>name=MongoDB Repository
>baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
>gpgcheck=0
>enabled=1
>EOF
mongo-1: $ sudo mv mongodb.repo /etc/yum.repos.d/

Install mongodb:
mongo-1: $ sudo yum install -y mongodb-org


Set /etc/hosts for each server as below:
mongo-1: $ cat > hosts << EOF
>192.168.0.10 mongo-1
>192.168.0.11 mongo-2
>192.168.0.12 mongo-3
>EOF
mongo-1: $ sudo mv hosts /etc/



To ease up this installation, turn off firewall and set selinux to permissive mode, temporarily, in all servers.
mongo-1: $ sudo systemctl stop firewalld
mongo-1: $ sudo setenforce 0


Edit /etc/mongod.conf in every server, to be similar as below (assuming we are using myreplica as our replSet)

mongo-1: $ sudo cat /etc/mongod.conf
logpath=/var/log/mongodb/mongod.log
logappend=true
fork=true
dbpath=/var/lib/mongo
pidfilepath=/var/run/mongodb/mongod.pid
replSet=myreplica


Once editing is done, restart mongodb in each server
mongo-1: $ sudo systemctl restart mongod


On the first server initiate mongo replica:
mongo-1: $ sudo mongo
MongoDB shell version: x.x.x
connecting to: test
Server has startup warnings: 
2018-05-28T04:39:22.580+0000 [initandlisten] 
2018-05-28T04:39:22.580+0000 [initandlisten] ** WARNING: Readahead for /var/lib/mongo is set to 4096KB
2018-05-28T04:39:22.580+0000 [initandlisten] **          We suggest setting it to 256KB (512 sectors) or less
2018-05-28T04:39:22.580+0000 [initandlisten] **          http://dochub.mongodb.org/core/readahead
myreplica:PRIMARY> rs.initiate() 


Add the other server, namely mongo-2 and mongo-3 to the replicaset
myreplica:PRIMARY> rs.add("mongo-2")
myreplica:PRIMARY> rs.add("mongo-3")


Run rs.status() to see the status of our replica
myreplica:PRIMARY> rs.status()
{
        "set" : "myreplica",
        "date" : ISODate("2018-05-28T05:32:10Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongodb-1.novalocal:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 77,
                        "optime" : Timestamp(1527485519, 1),
                        "optimeDate" : ISODate("2018-05-28T05:31:59Z"),
                        "electionTime" : Timestamp(1527485474, 1),
                        "electionDate" : ISODate("2018-05-28T05:31:14Z"),
                        "self" : true
                },
                {
                        "_id" : 1,
                        "name" : "mongo-2:27017",
                        "health" : 1,
                        "state" : 6,
                        "stateStr" : "UNKNOWN",
                        "uptime" : 19,
                        "optime" : Timestamp(0, 0),
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2018-05-28T05:32:10Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : 0,
                        "lastHeartbeatMessage" : "still initializing"
                },
                {
                        "_id" : 2,
                        "name" : "mongo-3:27017",
                        "health" : 1,
                        "state" : 6,
                        "stateStr" : "UNKNOWN",
                        "uptime" : 11,
                        "optime" : Timestamp(0, 0),
                        "optimeDate" : ISODate("1970-01-01T00:00:00Z"),
                        "lastHeartbeat" : ISODate("2018-05-28T05:32:09Z"),
                        "lastHeartbeatRecv" : ISODate("1970-01-01T00:00:00Z"),
                        "pingMs" : 0,
                        "lastHeartbeatMessage" : "still initializing"
                }
        ],
        "ok" : 1
}

In order to rectify the "stateStr: UNKNOWN" and "lastHeartbeatMessage: still initializing", simply add the name of the primary server, as given by mongodb in /etc/hosts of all secondary servers

mongo-2: $ cat /etc/hosts
192.168.0.10 mongo-1 mongodb-1.novalocal
192.168.0.11 mongo-2
192.168.0.12 mongo-3 

mongo-3: $ cat /etc/hosts
192.168.0.10 mongo-1 mongodb-1.novalocal
192.168.0.11 mongo-2
192.168.0.12 mongo-3 


You should be getting "syncingTo : mongodb-1.novalocal:27017", and "stateStr: SECONDARY" when you run rs.status() in primary server

myreplica:PRIMARY> rs.status()
...
{
                        "_id" : 2,
                        "name" : "mongo-3:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 368,
                        "optime" : Timestamp(1527485519, 1),
                        "optimeDate" : ISODate("2018-05-28T05:31:59Z"),
                        "lastHeartbeat" : ISODate("2018-05-28T05:38:06Z"),
                        "lastHeartbeatRecv" : ISODate("2018-05-28T05:38:06Z"),
                        "pingMs" : 1,
                        "syncingTo" : "mongodb-1.novalocal:27017"
                }
...


Your replica is now complete. To test it out:

Create new database in primary server, and fill up with data
myreplica:PRIMARY> use mynewdb
myreplica:PRIMARY> db.stack.save(
... {
...     "name": "myreplica",
...     "description":  "this is my new mongodb replica",
...     "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ],
... })
WriteResult({ "nInserted" : 1 })
myreplica:PRIMARY> show dbs
admin      (empty)
local      2.077GB
mynewdb    0.078GB
myreplica:PRIMARY> show collections;
stack
system.indexes
myreplica:PRIMARY> db.stack.find()
{ "_id" : ObjectId("5b0b97f9aca2dd0afb9d86a5"), "name" : "myreplica", "description" : "this is my new mongodb replica", "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ] }

Login to secondary servers, sync (by running "rs.slaveOk()" ) and check whether the data gets replicated

myreplica:SECONDARY> use mynewdb
switched to db mynewdb
myreplica:SECONDARY> show collections
2018-05-28T05:51:42.601+0000 error: { "$err" : "not master and slaveOk=false", "code" : 13435 } at src/mongo/shell/query.js:131
myreplica:SECONDARY> rs.slaveOk()
myreplica:SECONDARY> show collections
stack
system.indexes
myreplica:SECONDARY> db.stack.find()
{ "_id" : ObjectId("5b0b97f9aca2dd0afb9d86a5"), "name" : "myreplica", "description" : "this is my new mongodb replica", "hosts" : [ "mongo-1", "mongo-2", "mongo-3" ] }


Done :)


Friday, May 25, 2018

Creating image from a running instance

Sometimes we want to spawn off a few new instances, with the same spec and operating systems, but we do not want to go through the hassle of setting up each OS manually, and then update it one by one. In order to do that efficiently, openstack provides a very good way, which is to create an image from a running instance, and this image can be used to spawn off new instances afterwards.

Before we turn any instance to an image, we need to know its instance ID

$ openstack server list

We can then create an image from the above instance ID
$ openstack server image create --name centos7-updated-20180525 21e78f23-8b67-423a-9622-d46c8487f829

To make sure our image is created correctly, check using:
$ openstack image list

To create a new instance from the image, please refer here

Creating a new instance on openstack

In order to create new instance (it is called server in openstack command), you need to know beforehand a few information to feed to the create instance command. Refer below for those information:

check available flavor

$ openstack flavor list

check available images
$ openstack image list

check available network
$ openstack network list

check available security group
$ openstack security group list

check available keypair
$ openstack keypair list


Once you get all the above information, to create the new instance, just use below command, providing the above information as option to openstack server create command
$ openstack server create \
--image centos-7-20180520 \
--key-name my-keypair \
--flavor m1.medium \
--security-group defaults \
--network private-140 \
thenewinstancename


To check whether your new instance has been created and active:
$ openstack server list