Show memory map of a process

How to show memory map of a process? or how to check how much memory a process is consuming?
Use pmap.

How to use pmap:

1. get a pid of a process you are interested in checking, in this example apache:

   [mainuser@serverone ~]$ ps -eaf | grep http 
   root      1759  4493  0 04:02 ?        00:00:00 /usr/sbin/httpd
   apache    1760  4493  0 04:02 ?        00:00:25 /usr/sbin/httpd
   apache    1761  4493  0 04:02 ?        00:00:23 /usr/sbin/httpd
   apache    1762  4493  0 04:02 ?        00:00:21 /usr/sbin/httpd
   apache    1763  4493  0 04:02 ?        00:00:18 /usr/sbin/httpd

   
   
   
2. run the pmap command against the PID number:
   

   [mainuser@serverone ~]$ sudo pmap 1760 | tail 
   97406000     28K r--s-  /usr/lib/gconv/gconv-modules.cache
   97463000     16K rw---    [ anon ]
   97467000 524288K rw-s-  /tmp/apc.tE1RRo (deleted)
   b7467000  11096K r----  /usr/local/zend/lib/libicudata.so.38
   b7f3d000      4K rw---  /usr/local/zend/lib/libicudata.so.38
   b7f3e000     64K rw-s-  /dev/zero (deleted)
   b7f4e000    504K rw-s-  /dev/zero (deleted)
   b7fcc000     32K rw---    [ anon ]
   bfd74000    144K rwx--    [ stack ]
   total  2742152K

3. The process, which is apache is consuming about 2.7GB of memory

That's all folks :)

Ping a list of servers

To do this, you need to put all the hosts that need to be checked in a file. For example, I put all my hosts in a file called ping_list:

$ cat ping_list
cat.myhost.net
dog.myhost.net
tiger.myhost.net
bird.myhost.net

There are a few ways to ping multiple hostnames, I'll list out what I have tried before:

1. Use nmap

$ nmap -sP -iL ping_list
Failed to resolve given hostname/IP: cat.myhost.net.  Note that you can't use '/mask' AND '1-4,7,100-' style IP ranges
Failed to resolve given hostname/IP: dog.myhost.net.  Note that you can't use '/mask' AND '1-4,7,100-' style IP ranges
Host 192.168.0.99 is up (0.00036s latency).
Host 192.168.0.100 is up (0.00061s latency).

where -sP is for ping test and -iL is for inputting from files.

2. One liner for loop

$ for i in `cat ping_list`; do ping -c1 $i; done
ping: unknown host cat.myhost.net
ping: unknown host dog.myhost.net
PING tiger.myhost.net (192.168.0.99) 56(84) bytes of data.

--- tiger.myhost.net ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

PING tiger.myhost.net (192.168.0.100) 56(84) bytes of data.
 
 --- bird.myhost.net ping statistics ---
 1 packets transmitted, 0 received, 100% packet loss, time 0ms

I believe there are other tools or scripts beside those I listed above, but I always these 2 methods to ping multiple hosts. If you have other tools or script, please leave a comment.

Thanks

Download apache directory listings recursively

To apache directory listings recursively, use wget like below:

$ wget -r -np -nH -R index.html http://filestodownload/

where -r is for recursive retrieving, -np is for no-parent option where wget won't get the parent directory when retrieving recursively, -nH equals to no host diretories where generation of host-prefixed directories will be disabled and -R is to omit index.html.

Resetting windows password using linux livecd

The application that we are going to use is chntpw. In this example, we will be using ubuntu livecd.

1. Put the livecd in the cd/dvd drive and boot your windows machine from the livecd.

2. Once booted, open the terminal and check for tool named "chntpw". If not there, you can install it using:

$ sudo apt-get install chntpw

3. After the tool is ready, mount the windows partition. Use "fdisk -lu" to check which partition should be mounted. To mount /dev/sda1 (assuming your windows partition is on /dev/sda1), use below command:

$ sudo mount /dev/sda1 /mnt

4. Find SAM file on the windows partition. Usually it is located in Windows/System32/config. Run the chntpw on the SAM file.

$ cd /mnt/Windows/System32/config

List all user in the SAM record

$ chntpw -l SAM

Interactively edit user credential

$ chntpw -i SAM

5. Follow the wizard of chntpw and clear administrator or any user's password that you want to access.

6. Save changes and restart machine. You can access the windows without password for administrator and users that you have cleared their password. Make sure you take out the livecd, otherwise the machine will boot into it instead of windows.

Getting netbios name in linux

If you are using linux and administering a network full of windows machine, there is a tool called nbtscan to easily scan through the network and list all machines with their netbios name.

To install nbtscan in fedora:

# yum install nbtscan

To use it on single machine(let's say the machine's ip is 10.0.0.100):

# nbtscan 10.0.0.100

To scan the whole class C and list netbios names:

# nbtscan 10.0.0.0/24

To scan ip range:

# nbtscan 10.0.0.10-100

To get all options to use nbtscan:

# nbtscan -h

Running windows cmd from linux

To run windows cmd from linux box, there is one tool you could use, which is winexe. You can download the installer from here. There are 2 ways to install this tool:

1. Use the preinstalled version.

• Download from here
• Unpack the bz2 file: # bunzip2 winexe-static-081123.bz2
• Change mod to allow execute: # chmod +x winexe-static-081123
• Make soft link in your /usr/local/bin: # ln -s winexe-static-081123 /usr/local/bin/winexe

2. Compile from source

• Install necessary packages (gcc, svn, *-devel....)
• Get sources from here
• Unpack the source file: # tar -xvjf winexe-source-081123.tar.bz2
• Compile according to README file:
  • cd to unpacked tar.bz2 sources
  • ./autogen.sh
  • ./configure
  • make proto bin/winexe
• Compiled file will be located in wmi/Samba/source/bin/winexe
• Install winexe:

  install -s wmi/Samba/source/bin/winexe /usr/local/bin/winexe

  
  

To use it is very simple:

# winexe -U foo -W WORKGROUP -n FOO-PC //10.0.0.61 "cmd.exe"

where -U for username, -W for workgroup, -n for target machine netbios name, 10.0.0.61 is the ip address of the target machine and cmd.exe is to start windows command prompt.

Once connected, you will get command prompt like below:

Microsoft Windows [Version 5.2.3790]

(C) Copyright 1985-2003 Microsoft Corp.

C:\WINDOWS\system32>

To quit, just type exit at the windows command prompt.

That's all :)

Labeling linux partition

In linux, to label a partition, there are 3 tools that can be used. The tools are e2label, tune2fs and mke2fs.

To use e2label to label the second partition of the first hardisk with label DATA:
# e2label /dev/sda2 DATA

To use tune2fs to do the similar job as above:
# tune2fs -L DATA /dev/sda2

The third tool, mke2fs is actually a tool to build ext2/ext3 filesystem. So, if you want to build the partition's filesystem as ext2/ext3 and at the same time label it, this command can be used. Be careful though, because it will delete all existing data on that particular partition
# mke2fs -L DATA /dev/sda2

To view the label that you have set, there are 3 ways which are using e2label, blkid and viewing /dev/disk/by-label.

To check using e2label:
# e2label /dev/sda2
DATA

blkid tool is even more useful, because it can list out all the partitions that you have in the machine together with their labels,uuid and filesystem type:
# blkid
/dev/sda1: LABEL="/" UUID="1CC08F13C08EF276" TYPE="ext3"
/dev/sda2: LABEL="DATA" UUID="2063f830-fe5d-438e-b727-571b313cb89e" TYPE="ext3"
/dev/sda3: TYPE="swap" LABEL="SWAP" UUID="3e266b53-42e0-4f09-8fe3-d1cf79cb5d37"

To view the /dev/disk/by-label
# ls -l /dev/disk/by-label
total 0
lrwxrwxrwx 1 root root 10 2009-07-24 05:38 / -> ../../sda1
lrwxrwxrwx 1 root root 10 2009-07-24 05:38 DATA -> ../../sda2
lrwxrwxrwx 1 root root 10 2009-07-24 05:38 SWAP -> ../../sda3

Note that the label will stay with the partition although the disk is moved to another computer.

To use it in /etc/fstab:
LABEL=/ / ext3 defaults 1 1
LABEL=DATA /DATA ext3 defaults 1 2
LABEL=SWAP swap swap defaults 0 0

Hyper terminal for linux

For people that manage hardware devices such as storage, routers and many more using Microsoft Windows, the term hyper terminal is a familiar thing. They use hyper terminal to connect to all the devices mentioned above using serial cable. But what if you have to manage all those devices using linux?

The answer is, linux has 2 alternatives to hyper terminal; one is command line based and the other is GUI based. Let me start with the command line tool first. It is called minicom. You can install this tool using package manager of your linux machine. In fedora/redhat/centos:

# yum install minicom

Running it for the first time requires you to do some settings by running below command as root:

# minicom -s

Picture below shows the screen after command minicom -s


This is where you set the baudrate, serial device you want to use etc. After finish with the setting, save it so that you do not have to do it every time. You can save it to default .dfl file, with the name of .minirc.dfl in your home folder, or you can specify the name and location yourselves. To change the saved setting, just use the above command back.

The second tool is called cutecom, a graphical serial terminal. To install it on fedora,centos or redhat:

# yum install cutecom

It is easier to use since it has GUI. Picture below shows cutecom main screen, where you can set your device, parity, baudrate etc.

DNS lookup

To find an ip address for a domain, we need to do dns lookup. A few tools available for us to use in linux, but in this article I will give brief explanation about 3 most famous tools of dns lookup, which are host, nslookup and dig. Usually if the commands are used without the optional nameserver, then the nameserver entries in /etc/resolv.conf will be used


To find ip address of a domain using host (nameserver is optional):

$ host
example:
$ host www.google.com 208.67.222.222

To find the domain that belongs to an ip address using host (nameserver is optional):

$ host
example:
$ host 202.188.0.133 ns1.tm.net.my

To find ip address of a domain using nslookup (nameserver is optional):

$ nslookup
example:
$ nslookup www.google.com 208.67.222.222

To find the domain that belongs to an ip address using nslookup (nameserver is optional):

$ nslookup
example:
$ nslookup 202.188.0.133 208.67.222.222

nslookup also have interactive mode that you can access by simply typing
$ nslookup


To find ip address of a domain using dig (nameserver is optional):

$ dig @
example:
$ dig @ns1.tm.net.my www.google.com

Searching or finding files owned by user

A few days back, I have to find in my machine files that belong to a user or to a group. The general form are like below:

# find location -user username -group groupname

To find files in / directory owned by user foo:

# find / -user foo

To find files in / directory owned by user in group bar:

# find / -group bar

To find files in directory / owned by user foo and group bar:

# find / -user foo -group bar

You can also use uid or gid
To find files belong to uid 500:

# find / -uid 500

To find files that belong to gid 500

# find / -gid 500

Knowing your hardware

In linux, there are a few ways that you can know you hardware details without opening the chassis of your machine. Below are a few ways that I know and hopefully can help linux users out there;

1. Refer to the /proc directory. This directory contains a few files that can give you information about your hardware such as memory (meminfo), processor (cpuinfo), partitions (partition) and many more
2. Use "lspci" command. This is a command to list all pci devices connected to your machine
3. Use "lshw" command. This command will list out all hardware installed on your system. Available in ubuntu
4. Use "kudzu -p" command. This is redhat/centos hardware probing and installing tool. Use "kudzu -p" to display all the hardware connected to the system
   
5. Use "dmidecode" command. This is a tool for dumping bios information into human readable form
6. Run "lsusb" to list out all usb devices. Thanks to KwangErn Liew for the suggestion in the comment.
   

If anyone have any tool that I didn't list out, please feel free to drop out a comment

Strace - powerful troubleshooting tool

Strace is a tool in linux used for tracing system call and signals. This is very useful for tracking the error occurred when a program is run and we do not have any clue where to start troubleshooting. The usage of strace is very simple. It will record the system calls that happen during the execution of a particular program.

To use strace (below is an example if you want to trace the system call when you run ls):
# strace ls

To use strace and print the output to file
# strace -o outputfile ls

After that, you can analyze the outputfile, to see where is the error that fails the program

sos, machine information collection tool

A few weeks back, I have encountered a problem on one of my redhat server. Since the server is licensed, I send an email to redhat customer support through redhat network in search for the solution. The first thing that redhat reply to me was to run a command named 'sosreport'. What is sosreport?

According to the man page:
Sosreport (formerly known as sysreport) generates a compressed tarball of debugging information for the system it is run on that can be sent to technical support reps that will give them a more complete view of the overall system status.

Sosreport belongs to the sos package:

# whereis sosreport
sosreport: /usr/sbin/sosreport
# rpm -qf /usr/sbin/sosreport
sos-1.7-9.2.el5_2.2
# rpm -qi sos
Name : sos Relocations: (not relocatable)
Version : 1.7 Vendor: Red Hat, Inc.
Release : 9.2.el5_2.2 Build Date: Thu 17 Jul 2008 11:50:34 PM MYT
Install Date: Fri 17 Oct 2008 12:27:17 PM MYT Build Host: js20-bc2-10.build.redhat.com
Group : Development/Libraries Source RPM: sos-1.7-9.2.el5_2.2.src.rpm
Size : 421400 License: GPL
Signature : DSA/SHA1, Thu 28 Aug 2008 08:02:35 PM MYT, Key ID 5326810137017186
Packager : Red Hat, Inc.
URL : http://sos.108.redhat.com/
Summary : System Support Tools
Description :
SOS is a set of tools that gathers information about system
hardware and configuration. The information can then be used for
diagnostic purposes and debugging. Sos is commonly used to help
support technicians and developers.

To use sosreport:

1. type 'sosreport' as root
2. answer a few questions
3. wait for a while
4. check your sosreport output (bz2) at /tmp
5. You can use the sosreport output to troubleshoot remote machine or to ask help from remote technical support personnel
   

That's all folks :)

Saving website on local machine

Sometimes you found a website that is very interesting, but you just do not have enough time to read it on that particular time. You wish you could save it so that you can view it offline without connecting to the internet. This can be done using wget;

$ wget -m -k -K -E http://www.tldp.org/HOWTO/LVM-HOWTO/index.html

where -m for mirror, -k for convert the links so that it will be suitable for local viewing, -K for backup converted files and -E for adding html extension to the files downloaded. This is the result of the above command;

$ ls
www.tldp.org

Use any web browser to view the files offline by opening the .html file inside the above folder

Monitoring hard disk with smartmontools

Monitoring your hard disk health is a very important thing. You do not want to wake up one day, turn on your computer and suddenly your hard disk has crash and all your valuable data has gone with the wind. At that time crying would not get your data back. Like some people always say, prevention is better than cure. Apart from backing up your data regularly, monitoring the health of your hard disk is an essential task. It is to make sure any symptoms of bad sector or any failure can be detected earlier and steps to take care of it can be done sooner. One of the tool that can be used to do the job mentioned before is smartmontools. According to yum description, smartmontools are "Tools for monitoring SMART capable hard disks".

To install smartmontools on fedora:
# yum install smartmontools

Make sure your hard disk is smart capable
# smartctl -i /dev/sda
smartctl version 5.37 [i386-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Model Family: Western Digital Caviar SE (Serial ATA) family
Device Model: WDC WD800JD-60LSA5
Serial Number: WD-WMAM9MA75547
Firmware Version: 10.01E03
User Capacity: 80,026,361,856 bytes
Device is: In smartctl database [for details use: -P show]
ATA Version is: 7
ATA Standard is: Exact ATA specification draft version not indicated
Local Time is: Tue Jul 22 10:05:31 2008 MYT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Smart support is available for this hard disk and enabled

To monitor your hard disk health
# smartctl -H /dev/sda
smartctl version 5.37 [i386-redhat-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

To run test on your hard disk
# smartctl -t short /dev/sda

To see the selftest logs of smartctl
# smartctl -l selftest /dev/sda

See all options for smartctl
# smartctl -h

Manual for smartctl
# man smartctl

mtr, another network diagnostic tool

mtr is a network diagnostic tool that combine both traceroute and ping in one easy to use tool. As mtr starts, it investigates the network connection between the host mtr runs on and HOSTNAME, by sending packets with purposely low TTLs. It continues to send packets with low TTL, noting the response time of the intervening routers. This allows mtr to print the response percentage and response times of the internet route to HOSTNAME. The good thing about mtr is, it will run until we ask it to quit by pressing 'q'. That means we will have a live traceroute and ping that will keep updating until we ask it to stop.

To use mtr, just type:
$ mtr HOSTNAME

example:
$ mtr www.google.com

It will display the a few statistics about ping and packets, enough for us to do some basic network diagnostics work.

 

Using sftp to transfer file through network

There are a lot of ways on how to transfer files through network in linux and open source. One of the solution is to use sftp a.k.a. secure file transfer protocol. The reason this sftp is different from the original ftp is, sftp will do all its operation over encrypted ssh transport. This make sure that your file is safely transferred through network. To use sftp, you can just run command
$ sftp user@servername
For example,
$ sftp foo@server.name or
$ sftp foo@192.168.0.1

To use sftp efficiently, a few important commands one need to know, as listed below:

1. To get help on commands available.
   
• sftp> help
• sftp> ?
  
2. The commands are generally divided into 2 groups: the commands that can be used to manipulate localhost and the commands that can be used to manipulate remote host. The commands that start with 'l' are specially for locahost only. Example, to list all directory listing on localhost:
• sftp> lls
3. To list directories on remote host, use:
• sftp> ls
4. The most important command, how to upload file to the remotehost
• sftp> put /local/path /remote/path
5. To download file from remotehost
• sftp> get /remote/path /local/path

To get more information, use the first step to generate help page where list of commands and how to use it are shown.

Shell scripting built-in variables

When doing scripting in shell, like bash, there are a few built-in variables that we can use to optimize our script. Below are a few useful ones:

• $$ = The PID number of the process executing the shell.
• $? = Exit status variable.
• $0 = The name of the command you used to call a program.
• $1 = The first argument on the command line.
• $2 = The second argument on the command line.
• $n = The nth argument on the command line. n = 0-9
  
• $* = All the arguments on the command line.
• $# = The number of command line arguments.

Hope this can help

Creating banner for ssh server

A banner for ssh server is a few phrase that will come out the time you want to access a server through ssh. By default, this feature is turned off. To turned it on:

1. Login as 'root'
   
2. Create your banner file first. In this example, i will create banner file named /home/banner
   
• # vi /home/banner
• Insert your banner message to the file. I will insert 'Welcome to my pc'
3. After you have finish with the banner file, open /etc/sshd_config
   
• # vi /etc/sshd_config
• Uncomment or add the following line
• Banner /home/banner
4. Restart ssh server
   
• # /etc/init.d/sshd restart
5. When you login, this will be displayed
• # ssh pingu@10.20.20.171
  Welcome to my pc
  pingu@10.20.20.171's password:
  
  

The mystery of quotes

In linux environment, there are 3 types of quotes as far as i know. Each of the quotes bring different meaning and usage.

1. ' a.k.a. single quotes - Everything wrapped in this quote won't be changed (Strong quotes)
   
2. " a.k.a. double quotes - Quotes that doesn't expand meta-characters like "*" or "?," but does expand variables and does command substitution (Weaker quotes)
   
3. ` a.k.a. back quotes - To execute command

Examples of quotes usage (top lines are commands and the output are displayed below the commands):

Example of using back quotes within single quotes. Nothing is changed.
$ echo 'Today is `date`'
Today is `date`

Example of using back quotes within double quotes. The `date` command will be executed
$ echo "Today is `date`"
Today is Mon May 26 09:42:50 MYT 2008