Typescript - Capturing your shell activity

We know about screen capturing and recording our screen activity. There are hundreds of applications for this purpose. But what about your shell activity? Of course you can use the same screen capturing tools if you are working on the GUI window. But what if you are on a remote shell? What if you want to reuse or copy the results/commands as a backup? Obviously the graphical tools will not be of much help. This is where typescripts come into play.

A typescript is a record of whatever is printed in your terminal. That includes stdin,stdout and stderr, which is basically the commands that you type and the output that these commands generate. The command which creates the typescript is "script". You can just type the command "script" and this will start recording the shell activity to a file by the name "typescript" which will be created in the directory from which the command "script" was run. Alternatively you can specify a separate file as an argument to it. When you run the script command it actually opens a new shell, and keeps recording the shell's activity until you exit that shell and return to the parent shell from where you ran the "script"command. Let us check the usage.

safeer@penguinpower:~$ script
Script started, file is typescript
safeer@penguinpower:~$ ls MyDocuments/
Linux Windows
safeer@penguinpower:~$ mkdir MyDocuments/Cisco
safeer@penguinpower:~$ ls MyDocuments/
Cisco Linux Windows
safeer@penguinpower:~$ rm MyDocuments/Cisco
rm: cannot remove `MyDocuments/Cisco': Is a directory
safeer@penguinpower:~$ exit
exit

Script done, file is typescript

Let us examine the contents of typescript.

safeer@penguinpower:~$ cat typescript
Script started on Sunday 03 January 2010 02:05:01 AM IST
safeer@penguinpower:~$ ls MyDocuments/
Linux Windows
safeer@penguinpower:~$ mkdir MyDocuments/Cisco
safeer@penguinpower:~$ ls MyDocuments/
Cisco Linux Windows
safeer@penguinpower:~$ rm MyDocuments/Cisco
rm: cannot remove `MyDocuments/Cisco': Is a directory
safeer@penguinpower:~$ exit
exit

Script done on Sunday 03 January 2010 02:05:55 AM IST

As you can see all that is printed into the terminal is recorded in the typescript. A few useful options to use with the script:

-a : Append to the end of the type script file - if it exists already. Default behavior is to overwrite the file.
-q: Quiet mode, removes the End timestamp from the typescript.
-t: This is an interesting option, it prints the output timing data to standard error. This data can be used to replay the typescript with real-time input output delays. We will see the usage later.

You can specify an alternate typescript file as the last argument.

safeer@penguinpower:~$ script myscript
Script started, file is myscript
safeer@penguinpower:~$ echo HI
HI
safeer@penguinpower:~$ exit
exit

Script done, file is myscript

safeer@penguinpower:~$ cat myscript
Script started on Sunday 03 January 2010 02:10:28 AM IST
safeer@penguinpower:~$ echo HI
HI
safeer@penguinpower:~$ exit
exit

Script done on Sunday 03 January 2010 02:10:44 AM IST

To gather the timing information, we car try using the -t option.

safeer@penguinpower:~$ script -t myscript 2>myscript.time
Script started, file is myscript
safeer@penguinpower:~$ ls MyDocuments/
Cisco/ Linux/ Windows/
safeer@penguinpower:~$ ls MyDocuments/Cisco/
safeer@penguinpower:~$ touch MyDocuments/Cisco/gns3
safeer@penguinpower:~$ ls MyDocuments/Cisco/
gns3
safeer@penguinpower:~$ exit
exit

Script done, file is myscript

Content of time file will look like this:

safeer@penguinpower:~$ head -2 myscript.time
0.500888 27
0.253943 23

The first field indicates how much time elapsed since the previous output. The second field indicates how many characters were output this time. With this information and the typescript file we can use the command "scriptreplay" to replay our terminal activity with the same timing with which you typed on the terminal and the output was displayed. It will give you a feeling of watching a video of your terminal activity.

The usage is like this:

First get the type script to one file and the timing information obtained with "-t" option ( printed to stderr) to another file.

safeer@penguinpower:~$ script -t myscript 2>myscript.time

Now replay the typescript with the format "scriptreplay [timing file] [typescript file] ".
If no typescript filename is given, the name "typescript" is assumed.

safeer@penguinpower:~$ scriptreplay myscript.time myscript

And this will play the typescript with timing.

Changing MAC address in linux

There are occasions when you may need to change your MAC (Media Access Control) address temporarily. This can be used:
1. While connected to a public wifi hot spot - it keeps your original MAC address anonymous.
2. To bypass a router/switch which has MAC address filtering.
3. To obtain a dhcp ip lease that is allocated based on MAC address.

And remember! MAC address is something that is soldered into your network adapter. You cant change it permanently; in other words, the change that you make is not persistent over reboots.

First let us find out the current MAC address of the Ethernet adapter in my machine

safeer@penguinpower:~$ ifconfig eth0|awk '/HWaddr/{print $1" "$5 }'
eth0 00:13:e4:5c:93:0a

Now I am going to change the MAC address of my Ethernet interface.

First, bring down the interface
safeer@penguinpower:~$ sudo ifconfig eth0 down

Then change the MAC addres to whatever you want
safeer@penguinpower:~$ sudo ifconfig eth0 hw ether 00:22:4d:b7:36:01

Bring up the interface
safeer@penguinpower:~$ sudo ifconfig eth0 up

Verify MAC address has changed.
safeer@penguinpower:~$ ifconfig eth0|awk '/HWaddr/{print $1" "$5 }'
eth0 00:22:4d:b7:36:01

The syntax of the command is "ifconfig [interface name] hw [encapsulation type] [MAC address] "

Every argument is self explanatory except the encapsulation type. This is "ether" (Ethernet) for both our Ethernet and wireless interfaces. For finding out encapsulation type for other interface types see ifconfig man pages. To see the encapsulation type of your network interfaces use the following command

safeer@penguinpower:~$ ifconfig -a|awk '/HWaddr/{split($3,enc,":"); print $1" "enc[2];}'
eth0 Ethernet
wlan0 Ethernet

GPRS connection in Linux using wvdial

If you have a mobile with GPRS enabled you can use it to connect your Linux Box to Internet. The program that we can use for this is wvdial. Most of the popular distributions will have it in its repository, if not, get it from here.

I am going to connect using:

Nokia E71
GSM based GPRS connection from Vodafone (India)
Kubuntu 9.04
WvDial 1.60

First connect your mobile phone with USB cable to your PC/Laptop. In the case of Nokia, four options will pop-up on the phone. PC Suite, Media Transfer, Mass Storage and Connect to PC to Web. Though I chose Connect PC to Web, it was not detected as a modem. So changed the mode to PC Suite on the phone, and then a modem was detected at /dev/ttyACM0. I got this message on dmesg - "ttyACM0: USB ACM device". Now we need to cerate a configuration file for wvdial. and save it to "/etc/wvdial-vodafone-gprs.conf" for the time being.

The file should start with:

[Dialer Defaults]

Add the basic settings for the modem.

Modem = /dev/ttyACM0
Modem Type = USB Modem
Baud = 460800

For any gprs connection the most important detail you need is the APN ( Access Point Name ) of the service provider, in the case of Vodafone, it is "portalnmms". This should go in the modem initialization commands as below:

Init1 = ATZ
Init2 = AT+CGDCONT=1,"IP","portalnmms"

Note that the APN is part of the Init2 command. Now we need to add the dialing phone number, username and password. This is same across all GSM providers (CDMA phones have different values).

Phone = *99#
Username = 0
Password = 0

Dirty Secret: GSM don't check the username and password for authentication, it just needs the entries to be present in the configuration - with some junk values.

Save the file and dial with wvdial command using /etc/wvdial-vodafone-gprs.conf as an argument:

safeer@penguinpower:~$ sudo wvdial -C /etc/wvdial-vodafone-gprs.conf

You will get a lot of output, look for the following lines ( the IP's shown may differ)


local IP address 10.16.25.127
remote IP address 10.6.6.6
primary DNS address 10.11.230.2
secondary DNS address 202.144.13.50

This indicate the modem is connected. At this point you will be able to contact your DNS server and resolve names. But you need to set proxy for your browser to start browsing. For Vodafone the proxy and the port are 10.10.1.100 and 9401. Set this on your browser and you can start browsing. When you are done, close the connection by pressing Control+C to terminate wvdial command.

If your connection is CDMA, the phone number will be #777 instead of *99#, and the username and password will be your mobile number ( without country code ). Also the Init2 string will be "ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0" (no APN name).

A few more things to note, depending on your phone, the modem device location may change, it can be /dev/ttyUSB0, /dev/ttySL0 etc... always check dmesg to see how it is detected - use command dmesg|tail after connecting your usb cable.

For good measure, you can add the following settings as well to the wvdial configuration:

Dial Command = ATDT
Stupid Mode = 1
ISDN = 0
New PPPD = yes

To find out your provider's GPRS settings, you can visit Manoj Tivari's blog . I got the Vodafone proxy details from this web site.

FTP macros for automated ftp tasks

FTP macros are nothing but an ordered set of ftp commands grouped with a macro name, similar to a bash function/script. You can use the macro name like another command and execute the commands included with it. You can add a macro while inside an ftp session using the macdef command. Alternatively you can add it into the .netrc file inside your home directory. The usage of .netrc file is explained here.

Let us try creating a macro inside an ftp session. This macro will just change to a directory inside the ftp root and copy a file to local directory.

Each macro starts with a macro name on first line, followed by single ftp command on each line. The macro termination is given by a blank line.

ftp> macdef getmacro
Enter macro line by line, terminating it with a null line
cd /home/safeer/pub/site_backup/
get webdata.tar.gz


Once you create the macro run it from the ftp prompt with $ prepended to the macro name.
ftp> $getmacro
cd /home/safeer/pub/site_backup/
250 Directory successfully changed.
get webdata.tar.gz
local: webdata.tar.gz remote: webdata.tar.gz
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for webdata.tar.gz (0 bytes).
226 File send OK.

A macro defined in an ftp session remains active till the close command is issued. If you want re-usable macros, you need to add them to your netrc file. The above mentioned link has the details on how to set it up. A macro defined by the name "init" (in netrc file) will be run every time you login to the corresponding ftp host.

Macro Arguments

If you want to take arguments inside your macro, use $1,$2,$3 etc... where $1 will be the first argument, $2 will be the second argument and so on. If you want to iterate through each argument of the macro, use the special variable $i. The characters "$" and "\" are special in macros and need to be escaped with "\" if you want them "as it is" in the macros.

Eg:

ftp>macdef 2args
Enter macro line by line, terminating it with a null line
cd /home/safeer/docs
get $1
put $2

Above macro will now download local.txt and upload remote.txt

ftp>$2args local.txt remote.txt
cd /home/safeer/docs
250 Directory successfully changed.
get local.txt
local: local.txt remote: file1.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for local.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (91.1 kB/s)
put remote.txt
local: remote.txt remote: remote.txt
200 PORT command successful. Consider using PASV.
150 Ok to send data.
226 File receive OK.
47 bytes sent in 0.00 secs (454.4 kB/s)

ftp>macdef getmulti
Enter macro line by line, terminating it with a null line
cd /home/safeer/docs
get $i

Above macro will now change to the directory docs in ftp root and recursively download the files given as arguments to macro

ftp> $getmulti file1.txt file2.txt file3.txt
cd /home/safeer/docs
250 Directory successfully changed.
get file1.txt
local: file1.txt remote: file1.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for file1.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (82.3 kB/s)
cd /home/safeer/docs
250 Directory successfully changed.
get file2.txt
local: file2.txt remote: file2.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for file2.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (118.6 kB/s)
cd /home/safeer/docs
250 Directory successfully changed.
get file3.txt
local: file3.txt remote: file3.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for file3.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (154.5 kB/s)
There is an maximum 8 character limit for macro names defined on command line. In netrc file this restriction is not there. Per ftp session, there is a limit of 16 macros and 4096 total characters in all defined macros.

Automating ftp logins with netrc

If you are a regular ftp user, then netrc file can help you a lot in automating your logins and a few tasks you do via ftp.

Let us first consider automating ftp logins.

Create a ".netrc" file in your home directory. The access to this file should be restricted exclusively to you.

cd ~;touch .netrc;chmod u+rw,go= .netrc

Now assume you need to connect to an ftp server "myftp.safeer.in" with username 'safeer' and password 'password#123'. To automate this, add the following line to your .netrc file.

machine myftp.safeer.in login safeer password password#123

Here machine defines the host you want to login to, login is your username for the host and password is the password for this username.

safeer@my-lptp01:~$ ftp myftp.safeer.in
Connected to myftp.safeer.in.
220 (vsFTPd 2.0.7)
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

You can define a line for each of the host you want to login. If you use the same login for many of the ftp hosts, you can use the default entry like this.

default login safeer password password#123

With this entry, if you connect to an ftp host which is not explicitly defined in netrc with a "machine" entry, the login and password from the default line will be used. If you want to disable the autologin feature of netrc while running ftp program, use the option "-n", ie; "ftp -n [ftphost]"

safeer@my-lptp01:~$ ftp -n myftp.safeer.in
Connected to myftp.safeer.in.
220 (vsFTPd 2.0.7)
ftp> pwd
530 Please login with USER and PASS.

Thats all about auto logins, now let us see how to automate tasks. We use ftp "macros" for this. FTP macros are nothing but an ordered set of ftp commands grouped with a macro name, similar to a bash function/script. You can use the macro name like another command and execute the commands included with it.

Creating a macro with name "fmacro"

macdef fmacro
cd /home/safeer/pub/site_backup
put webdata.tar.gz

Now, run the macro:

ftp> $fmacro
cd /home/safeer/pub/site_backup
250 Directory successfully changed.
put webdata.tar.gz
local: webdata.tar.gz remote: webdata.tar.gz
200 PORT command successful. Consider using PASV.
150 Ok to send data.
226 File receive OK.
ftp>

A macro defined by the name "init" (in netrc file) will be run every time you login to the corresponding ftp host. Each macro definition is associated with the login entry preceding it and terminated by a blank line. You can't define global macros in netrc file. Macros associated with the" default" entry will work for all machines that are not explicitly defined in the file. To know more about ftp macros, visit this link.

For each "machine" line you can use one or more of the variables machine,login,password in order. ie: any of the following will work.

machine myftp.safeer.in login safeer password password#123 OR
machine myftp.safeer.in login safeer OR
machine myftp.safeer.in

The last option will be used only if you want to define some macros for this host, but don't want the autologin feature. While defining the default entry, make sure that it is defined as the last entry in the file. This is necessary as netrc is processed sequentially and accepts the first entry that matches the connecting host. For more information on netrc and macros, see man pages for netrc and ftp.

Running Subversion as Windows Service

Download the source package of subversion for windows from http://subversion.tigris.org
Unpack the Zip package to some Directory
Eg: C:\usr\svn-win32-1.4.5\

Now add the subversion binaries to system PATH. Binaries are located in “C:\usr\svn-win32-1.4.5\bin\“. To set this either you can go through the Advanced menu of system applet or use “set” command from command line.

Now you will be able to use the svn commands from command prompt. As the first step, create a root folder for all projects that will run under this server, say C:\svnroot. Now we need to run the svn server.

C:\Temp\>svnserve -d -r C:\svnroot

This will start svn server listening on default port 3690. To verify, start another command window
C:\Temp>netstat -ano|find “3690″
TCP 0.0.0.0:3690 0.0.0.0:0 LISTENING 2820
The process with PID 2820 is listening on port 3690, to verify that the process is svnserve,
C:\Temp>tasklist |find “svnserve.exe”
svnserve.exe 2820 Console 0 3,264 K

This confirms that snvserve is running and listening for connections on port 3690

This method has the drawback that the svnserve command is running in foreground which means,
*A command window will be running in the foreground all the time
*If the windows is closed or the server is rebooted the service will require to be manually started again.
*A user has to be logged in to the server and he cannot log off for another user to use the system.

So to make life easier, we will have to automate the whole thing. This means that the svnserver should be run as a windows service. Subversion has a Windows installer which will install itself as a service, bu I feel it is not that flexible and hence wanted to implement the serive in my way. Svnserve supports running as windows service with the command line switch –service. To make use of this feature we should use the Windows Service Control command line tool(sc.exe).

First, create the service with name SVNSVC
C:\Temp>sc create SVNSVC binpath= “C:\usr\svn-win32-1.4.5\bin\svnserve.exe –service -r C:\svnroot” displayname= “Subversion Server” depend= Tcpip
[SC] CreateService SUCCESS

Configure the service to start at boot time
C:\Temp>sc config svnsvc start= auto
[SC] ChangeServiceConfig SUCCESS

Start service
C:\Temp>net start svnsvc
The Subversion Server service is starting.
The Subversion Server service was started successfully.

If you mis configure the service option or run into problems, you can always remove the service from the system with the command
C:\Temp>sc delete svnsvc
[SC] DeleteService SUCCESS
and create a fresh service.

You can configure, start and stop services from the services MMC console in administrative tools instead of using the command line. But you can’t add or delete service.

Resetting MySQL root password

MySQL root user is the built in user with administrative privileges on MySQL server. Even though this is the default administrator user, you can add any number of new users with varying administrative rights on MySQL server. In many occasions system administrators tend to forget the MySQL root password, but this can be fixed with out much pain.

Stop the MySQL server. If you have start/stop service scripts installed you can do it that way:

[root@linuxbox1 ~]#service mysqld stop OR

[root@linuxbox1 ~]#/etc/rc.d/init.d/mysqld stop or whatever other script you use normally.

In some cases, this may fail if the server is overloaded, in such a case you will have to kill the mysql process

You can get the PID of the process using ps command:

[root@linuxbox1 ~]# ps aux|grep mysqld OR

looking in the mysql pid file ( found usually in MYSQL_DATA_DIRECTORY/mysql.pid or /var/run/mysqld/mysqld.pid MySQL data/installation directory is usually /var/lib/mysql ). Then kill the process

[root@linuxbox1 ~]#kill `cat /var/run/mysqld/mysqld.pid`

Now since the mysql server has stopped, start mysqld with the –skip-grant-tables option

[root@linuxbox1 ~]# /usr/bin/mysqld_safe –skip-grant-tables

Now set new root password

1) Using mysqladmin

[root@linuxbox1 ~]# mysqladmin -u root password ‘my_newpass#123‘

Note that you don.t need to provide a password to use mysqladmin. Now flush the privilege tables in mysql to take the effect of password change

[root@linuxbox1 ~]# mysqladmin -u root flush-privileges

2) Using mysql client

[root@linuxbox1 ~]# mysql -u root mysql

mysql> UPDATE user SET Password=PASSWORD(’my_newpass#123′) WHERE User=’root’;
mysql> FLUSH PRIVILEGES;

Password is reset now. Now you have to restart MySQL service. First kill the mysqld_safe process, use the ps command to get the pid of mysqld_safe. Start MySQL service as usual.

[root@linuxbox1 ~]#/etc/rc.d/init.d/mysqld start

Now you will be able to connect to MySQL with new password ‘my_newpass#123‘

[root@linuxbox1 ~]#mysql -u root -pmy_newpass#123

Configure ISA Server to allow FTP uplod

ISA Server by default allows only ftp read access. Even if you have written access rules to allow FTP communication, that will only let you download files, neither you will be able to upload nor create folders and files in the server.

To allow write permission for FTP, right click on the access rule which you have configured to allow FTP access. From the context menu, chose "Configure FTP". On the pop up window that comes up, untick the option "Read Only". Press OK, and click Apply to save the settings.

Now you will be able to upload files via FTP.

Unix soft link and hard link

To simply put it, a Unix link is a convenient way to access data of a file. It is similar to shortcuts in windows. But in UNIX there are two types of links, soft link and hard link. To understand the difference between these two, we should first know some basics of Unix file structure.

In UNIX file system, a file contains two parts, data part and filename part. Filename part contains the file name and indoe. Data part contains the actual data stored in the file. Inode is a data structure that contains the information about the file like permissions, group, owner, pointer to the data part etc….Each inode (and hence the data it points to) is identified by an inode number. Data part contains the actual data stored in the file. A user accesses a file using its name while the underlying file system access it with its inode number.

Different files can point to the same inode number. These files are called hard links. They will have different names and may be located in different paths, but under the same file system. Hard links can’t be created across different file systems, this is because the inode number is specific to a file system not the OS as a whole. Each file system maintains a table of inodes within that file system.

A soft link is a special file that contains the file system path to another file as its data. But as soft link (symlink) is a special file, any read/write/modify operation on this file will be redirected to the target file. Since symlinks refer to their target using absolute path and file name, it can cross file system boundaries and can be located anywhere. Soft links can point to both files and directories.

Difference b/w soft and hard links

Soft link	Hard Link
Points to file name of original file	Points to inode of original file
Can link between file systems	Can link only within same file system
Both for file & directory	Only for files
Default permission 777	Default permission 644
Can’t access data after original file is deleted	Can access data even after original file is deleted
Can’t change owner/group/permission	Can change owner/group/permission
Inode number not same as original file	Inode number same as original file

Now create a text file

[root@penguin007 links]# touch link-test.orig
[root@penguin007 links]# echo “Original file created” > link-test.orig
[root@penguin007 links]# cat link-test.orig
Original file created

The command to create link is "ln". Let us create a hard link to the file link-test.orig.

[root@penguin007 links]# ln link-test.orig link-test.hard

Now let us create a soft link to the file link-test.orig.
[root@penguin007 links]# ln -s link-test.orig link-test.soft
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig

Now you can make changes to the original file/soft link/hardlink and the change will be reflected on all others. This is because all of these are pointing to the same data location

Edit from soft link

[root@penguin007 links]# echo “Change from soft link” > link-test.soft
[root@penguin007 links]# cat link-test.orig
Change from soft link
[root@penguin007 links]# cat link-test.hard

Change from soft link

Edit from hard link

[root@penguin007 links]# echo “Change from hard link” > link-test.hard
[root@penguin007 links]# cat link-test.orig
Change from hard link
[root@penguin007 links]# cat link-test.soft

Change from hard link

Edit from original file

[root@penguin007 links]# echo “Change from original” > link-test.orig
[root@penguin007 links]# cat link-test.soft
Change from original
[root@penguin007 links]# cat link-test.hard

Change from original

From the output of "ls -li" you can observe following points.

Both the original file and hard link are having same inode (163774475 - column 1 ). In fact both these files are referred to as hard links and you can't distinguish which file is original and which one is link - there is no need to do that either.

Note the permission on the files, hard links are having standard 644 permissions while soft link has 777 permission. Hard links are treated as regular files but soft links are treated as links- note the first character in the 2^nd column of "ls" output above ("-" for regular file/hard link and "l" for soft link)

When you change permission/ownership of a hard link the change is reflected on all hard links. But when you change permission/ownership of a softlink the change is applied to the original file (and its hard links if any) while soft link retains the permission and ownership.

[root@penguin007 links]# chmod 755 link-test.hard

[root@penguin007 links]# ls -li
total 8
163774475 -rwxr-xr-x 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rwxr-xr-x 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig
[root@penguin007 links]# chmod 644 link-test.soft
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig
[root@penguin007 links]# chown safeer link-test.orig
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 safeer root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 safeer root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig
[root@penguin007 links]# chown root link-test.soft
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig

The third column in the output shows the hard links count, ie; how many hard links point to that file (2 in our case).

Links do not cause any additional hard disk usage. We can check it now:

[root@penguin007 links]# du -hs ./*
4.0K ./link-test.hard

0 ./link-test.soft

You may note that only one file (that comes alphabetically first) of the hard links is displayed in disk usage result (this happens if both hard links are in the same directory)

Now let us delete the original file.
[root@penguin007 links]# rm -rf link-test.orig
[root@penguin007 links]# cat link-test.hard
Change from hard link
[root@penguin007 links]# cat link-test.soft

cat: link-test.soft: No such file or directory

As you can see, soft link will not work if the original file is deleted, because it points to the name of the original file. But hard links are pointing directly to the inode, so it can continue accessing the data. When a file has multiple hard links, the data part of the file will not be deleted until all the hard links to the file are deleted. The count of hard links is stored in the inode which will be decremented every time a hard link to that data is deleted.

VNC Desktop in Linux

#su - safeer
[safeer@xmsv03 ~]$ mkdir .vnc
[safeer@xmsv03 ~]$ vi .vnc/xstartup
----------------------
#!/bin/sh

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &

unset SESSION_MANAGER
exec /etc/X11/xinit/xinitrc
------------------------------------------------
[safeer@xmsv03 ~]$chmod a+x .vnc/xstartup
[safeer@xmsv03 ~]$vncpasswd
Password:
Verify:
[safeer@xmsv03 ~]$vncserver -extension XFIXES

New 'xmsv03:2 (safeer)' desktop is xmsv03:2

Starting applications specified in /home/safeer/.vnc/xstartup
Log file is /home/safeer/.vnc/xmsv03:2.log

Note that vncserver for safeer is started at display 2 (xmsv03:2) which will by default listen on port 5902 of remote machine
Now access remote linux desktop via VNC viewer

This will prompt for the password you set using vncpasswd command

To stop the server, use command

[safeer@xmsv03 ~]$ vncserver -kill :2

Setting up your first SVN Repository

This article is based on windows platform, but you can use the same commands on linux also. The only difference will be in the choice of text editor for modifying configuration files- which will be VIM for linux. In windows before setting up an svn repository for your project, make sure that all the svn binaries are available in the system path. For details on installation and configuration of Subversion server, see the related articles in this website.

Find out the SVN root for the projects, let us say C:\svnroot\
Now we can create a test project under svn root.
C:\Temp>svnadmin create C:\svnroot\samplepro
Now setup authentication for this project
Open C:\svnroot\samplepro\conf\svnserve.conf in notepad
Under the section marked [general] find the commented lines:
#anon-access = read
#auth-access = write

This allows anonymous users read access and authenticated users write access to the project source code. This is not desirable in a corporate environment -unless you are working on a project under GPL. So let us deny anonymous users any kind of access while allowing write access to authenticated users.
anon-access = none
auth-access = write

Uncomment the line
#password-db = passwd
This will tell svnserver that the password database to be used is named passwd and is in the same directory as that of svnserve.conf.
* It is a good idea to keep a common password database for all project that has to be accessed by the same users.

Uncomment the line
#realm = My First Repository

Edit the realm name to describe your Project, say

realm = Web Projects
* If you are planning to use the same user database for multiple projects, They all should have the same Realm. This will let the clients use the same cached passwords for all repositories with this realm name.

Now add users to the user database
Open C:\svnroot\samplepro\conf\passwd

Under the [users] section, delete the commented username password pairs
# harry = harryssecret
# sally = sallyssecret

then add your own usernames

safeer = af12@#dr$
murali = oh8$%q23

Now try accessing the repository via svn command line client.

winws07.safeer.in is the host name of this machine

C:\Temp>svn co –username=safeer svn://winws07.safeer.in/samplepro
Authentication realm: <svn://winws07.safeer.in:3690>> Web Projects
Username: safeer
Password for ’safeer’: ******
Checked out revision 0.

Now you can use Any GUI svn clients either as stand alone or as part of your IDE to access this repository.

If you want a stand alone client for windows, go for Tortoise SVN at http://tortoisesvn.tigris.org/
If you are using eclipse, you can get the sub ecliepse pluggin for SVN. This varies with your IDE. Even if your IDE doesn’t support SVN, you can use custom scripts to do the job, depending upon the OS platform you use.

TCP Wrappers

TCP wrappers is used to implement host based access restrictions to network services running on a Linux server. In order for a network service to work with TCP wrappers, it should be compiled with libwrap.a library. To check whether a network service has TCP wrapping enabled use:
strings -f /path/to/service/binary|grep hosts_access, for example to check whether ssh is compiled with TCP wrappers support,

[root@LinuxBox1 ~]#strings -f /usr/sbin/sshd|grep hosts_access
/usr/sbin/sshd: hosts_access

Host access is allowed or denied based on the entries in the files /etc/hosts.allow & hosts.deny ( together known as host access files) . Each of these files contain a list of services and corresponding list of hosts along with certain additional options.

As the name indicates, hosts.allow lists the hosts allowed to access listed services. When a host tries to access a network service on the server, hosts.allow is parsed from top to bottom to see a match for the service-host pair. The parsing will stop with the first match found and the access will be allowed. If no match is found in this file, hosts.deny is parsed. If a match is found access is denied for the host. If neither file contain a match or the files does not exist, access is allowed for the host.

Access files contain a number of rows, each with following format

daemon-list:host-list:[option1]:[option2]:..........

daemon list : Comma separated list of service binary names. Dont take this for the service name, for example, the service name for telnet is 'telnet' but the binary is 'in.telnetd' (/usr/sbin/in.telnetd).

host list : Comma separated list of hosts to be allowed/denied access to the daemon list. Hosts can be specified in any of the following format.

host names : machine1.mydomain.com represents a single machine
host names starting with a period: .mydomain.com represents all host names under this name. Example: machine2.mydomain.com, switch2.sub1.mydomain.com ......
Hostnames containing ? and/or * : Within a hostname or IP you can use this wild cards, '*' can substitute any number of characters while '?' can replace only a single character.
Eg:
*.mydomain.com : mac1.mydomain.com, mac2.sub1.mydomain.com ....
mac?.mydomain.com : mac1.mydomain.com,mac2.mydomain.com,...mac9.mydomain.com,maca.mydomain.com,macb.mydomain.com etc...
IP address ending with period : 192.168. represents all IPs in the network 192.168.0.0/16
Network & Netmasks : 192.168.1.0/255.255.255.128, 10.10.0.0/255.255.224.0 etc...
File path beginning with leading '/' : The specified file contains a list of hosts. Each line of the file will contain a number of host/ip patterns separated by white spaces.

Explicit wild cards: This wild cards has special purposes, even though there are many the popular one is 'ALL' which matches anything ( applicable to daemons as well as hosts)

The 'EXCEPT' operator: This can be used with both hosts and daemons as follows:
a) In the daemon list, a rule that is applicable to all services except vsftpd
ALL EXCEPT vsftpd
b) In the clients list all machine in the network 192.168.0.0 except 192.168.0.15
192.168.0. EXCEPT 192.168.0.15

options : This fields can be used to alter the log behaviour, launch shell commands etc when a rule is matched, but this feature is specific toRedhat version of tcp wrappers.

There are two approaches for any firewall/access control design, mostly closed and mostly open. In mostly closed all accesses arerestricted and only minimum required access is granted. For TCP wrappers mostly closed means a single line ALL:ALL in hosts.deny and necessary services in hosts.allow. In mostly open, all client access is allowed and restrictions applied as and when necessary. InTCP wrappers: ALL:ALL in hosts.allow and necessary restrictions in hosts.deny.

Some examples rules:

Allow telnet from local networks (192.168.0.0./24 & 192.168.1.0/24)
hosts.allow
in.telnetd:192.168.0.,192.168.1.
Suppose, in addition to this, you want to deny telnet access from all other hosts.
hosts.deny
in.telnetd:ALL
Since hosts.allow is parsed first the access rule ther will match request from local networks and allow them. If the request is from an external host, the rules in hosts.allow will not match and will be passed to hosts.deny where the rule is to deny all telnet communication. This rule will be matched and access denied for that host.

Alternatively we can do both this in a single rule in hosts.deny
hosts.deny
in.telnetd: ALL EXCEPT 192.168.0.,192.168.1.

Adding swap space to Linux

The virtual memory used by Linux is called swap space. It is the extension of physical memory in the sense that the total usable primary memory will be the sum of RAM and swap space. The kernel will write the memory blocks of unused programs to the swap space, thus freeing the RAM to be used by active programs. Hard disk access is much slower when compared to RAM access. So swap space can't be considered as a replacement for RAM, it is just a supporting techniques for RAM. Using of swap space imposes overhead on the system, but for systems with less physical memory it is a necessary evil.

When you install the operating system you will add a swap space, most probably up to double of the RAM. This will not be enough if you later add more RAM to your machine and you will have to increase the swap space. You can add swap space to your system in two ways, either by adding a swap partition or swap file.

To use swap partition, you should have unallocated space left on your hard disk. Use fdisk program to create a new partition.
[root@LinuxBox1 ~]#fdisk /dev/hdb
and use "n" command to create a new partition, say /dev/hdb2

If you don't have unallocated space to create a new partition, but have free space on existing partitions, you can create swap files instead. IF you want to create 128 MB of swap space, create a file of that size.

[root@LinuxBox1 ~]#dd if=/dev/zero of=/vmem/swapfile01 bs=1024 count=131072
131072+0 records in
131072+0 records out
134217728 bytes (134 MB) copied, 3.0451 seconds, 44.1 MB/s

In the rest of this document, I will use this swap file as example, but the procedure is same for swap partition also.

To mark the created space usable as swap, a signature is needed to be written into the start of file/partition. This is done withmkswap command.
[root@LinuxBox1 ~]#mkswap /vmem/swapfile01
Setting up swapspace version 1, size = 134213 kB

Now tell the kernel to use this space for swap
[root@LinuxBox1 ~]#swapon /vmem/swapfile01

In order to keep this settings persistent over reboots, add a line to /etc/fstab.
/etc/fstab
/vmem/swapfile01 swap swap defaults 0 0

To list all the swap partitions/files in a system,
[root@LinuxBox1 ~]#swapon -s
Filename Type Size Used Priority
/dev/hda3 partition 1124540 1848 -1
/vmem/swapfile01 file 131064 0 -3
/vmem/swapfile02 file 65528 0 -4

To see the current primary memory status
[root@LinuxBox1 ~]#free -m
total used free shared buffers cached
Mem: 495 470 24 0 3 204
-/+ buffers/cache: 263 232
Swap: 1290 1 1288

You can see that Total Swap reported is the sum of 3 swap sizes as listed in the output of "swapon -s"

If you want to remove the added swap space, turn of swapping for the partition/file
[root@LinuxBox1 ~]#swapoff /vmem/swapfile01

If you want to save hard disk space, remove the file/partition.

Swap files are most useful when you need to run a memory intensive program for a short period of time. You can create a swap file of appropriate size and delete it when the application is finished. If you want to add swap for permanent use, better go for swap partitions- provided your hard disk has unallocated space left.

Blocking Google Talk in your organization

Blocking instant messengers in an organization is a big headache for all system administrators. Not only that IMs waste precious productivity time and bandwidth of your company, it also possess some security threats. The major threat comes with file sharing capabilities of IMs, which can be used to share proprietary information of the company. This will become a night mare for the management if they have signed a non disclosure agreement with their clients. There are a lot of ways for an employee to ship information out of a company, but IM is one of the easiest method to do so. Another big problem is spreading of viruses and worms, unlike emails it is not easy to integrate Anti virus softwares with IMs.

But when you consider blocking IMs, the problem is that most of them are set to work with least resources and under almost any network conditions. One way is to find out the port used by the IM and block it. This works fine for Microsoft Messengers, but most others have work around for this. Here let us specifically take the case of Google Talk.

Google talk initially used Jabber/XMPP protocol which communicates over 5222 & 5223 (for SSL) ports. But blocking these ports alone will not fix the problem. The reason is that if this ports are not available, gtalk will switch to HTTP & HTTPS ports. Obviously we can’t block all HTTP & HTTPS communication. So the only way is to use a combination of protocols and server IPs (of Google talk). Again another problem with this approach is that Gtalk is having a number of servers and if it can’t access one server, it will try another IP and so forth until the connection is established. So we need to find out all the servers(IPs) used by Gtalk and block all of them. This can only be done with a trial and error approach.

Login into you Gtalk account, now the connection will be established with one of the Gtalk servers. Find out the process id of Gtalk application by either using Task Manager or the command line “tasklist.exe”. This is the out put from tasklist.exe command line:

C:\Documents and Settings\Administrator>tasklist /FI “IMAGENAME eq googletalk.exe”

Image Name PID Session Name Session Mem Usage
========================= ====== ================ ======== ============
googletalk.exe 3492 Console 0 4,900 K

So 3492 is the process id of Gtalk application. Now use the netstat.exe command line or any network monitoring application to find out the remote IP and port to which this application is connected. Look at the netstat output

C:\Documents and Settings\Administrator>netstat -ano|find “3492″

TCPÂ 192.168.0.23:4381 72.14.253.125:5222 ESTABLISHED 3492

So we have now one IP of Google talk: 72.14.253.125Â and the port connected to is 5222 -Jabber/XMPP.

Now logout of Google talk and block the communication to this IP through the protocols HTTP,HTTPS, Jabber, Jabber -SSL.ÂLogin to google talk again and repeat the same steps again. After a while you will get all the IPs used by Gtalk. Now block all the IPs as above. This will block not only Google talk but the chat applet inside the Gmail window.

The problem with this approach is that such servers will switch IPs frequently, and you will have to continuously update the list of servers. Bellow is the list of Gtalk Servers that I have found out. As of writing this, this list has been working for our ISA Server 2006 for the past month. I just created a computer set for the Gtalk servers and blocked the HTTP,HTTPS,Jabber Protocols to this Computer set. Also you will have to create Jabber as a custom protocol with outgoing TCP/UDP communication to ports 5222 and 5223.

Gtalk Servers:

216.239.37.125

72.14.253.125

72.14.217.189

209.85.137.125

209.85.163.125

209.85.201.189

216.239.51.125

Protocol - Port

HTTP - 80

HTTPS - 443

Jabber - 5222

Jabber- SSL - 5223

Linux group administration

Group information is primarily stored in /etc/group. There will be some default groups created when operating system is installed. Each line in /etc/group corresponds to a group . Each of these line will have 4 fields separated by colons. an example line is as follows

developers:x:520:baiju,vishin,suraj

Field 1 - developers : Group name

Field 2 -x : The group password was previously stored here. The x here indicate that group password is shadowed(stored in a separate file /etc/gshadow which is only readable by root)

Field 3 - 520 : The numerical group id

Field 4 - baiju,vishin,suraj : A comma separated list of group members. The user will be listed here only if this is his secondary group. Primary group membership will not be shown here.

To add a group use the groupadd command

Syntax:

groupadd [OPTIONS] groupname

OPTIONS:

-r : Create a system account with group id below 4999

-g GID : Numeric group id

To modify a group property use groupmod

Syntax:
groupmod [OPTIONS] groupname

OPTION:

-g GID : change group id

-n new_grp_name : Change group name

To delte a group, use groupdel command

Syntax

groupdel group_name

To add a user to a group you can use usermod or useradd commands with -g/G options. But the limitation with this approach is that only a single user is added at a time. More over this is done from a user perspective. To really administer group memberships, you will have to use gpasswd command.

Every group can have administrators,members and a password. Only root can assign administrators to a group while group admins and root can add members to a group.

The group password is used to restrict access through the command newgrp. The newgrp command is used to switch the primary group of a user. Suppose the user safeer has admin as his primary group, developers and staff as his secondary groups. When user safeer creates a file the owner of the file will be safeer and group of file will be admin (primary group of safeer). In some situations we may need to switch our primary group, either for creating new files with different group membership or access a file to which a certain group has permission. I would better illustrate with an example. Take the case of user safeer in the above example:

Display the groups of which safeer is a member
[safeer@LinuxBox1 ~]$groups
admin developers staff
Of this admin is the primary group and others are secondary groups. Now will create some files with different group membership
[safeer@LinuxBox1 ~]$touch file1
[safeer@LinuxBox1 ~]$ll file1
-rw-r--r-- 1 safeer admin 0 Mar 29 01:07 file1
[safeer@LinuxBox1 ~]$newgrp developers
[safeer@LinuxBox1 ~]$groups
developers admin staff
[safeer@LinuxBox1 ~]$touch file2
[safeer@LinuxBox1 ~]$ll file2
-rw-r--r-- 1 safeer developers 0 Mar 29 01:07 file2

When you use newgrp, you are automatically allotted a new shell. If you type exit from that promt you will be broguht back to the previous shell with old group membership. From the above example
[safeer@LinuxBox1 ~]$exit
[safeer@LinuxBox1 ~]$groups
admin developers staff

The main advantage of newgrp is in access permission to files. Suppose there is a file with the following permissions.
-rw-rw---- 1 root accounts 0 Mar 29 01:07 salary2

As you can see, safeer (or any others except root and members of accounts group) don't have the permission to access this file. Now a situation arises where safeer has to access this file (of course, legally!) . Then the administrator can set a password for group 'accounts ' using gpasswd
[root@LinuxBox1 ~]#gpasswd accounts
Changing the password for group accounts
New Password:
Re-enter new password:

Now root gives safeer the accounts group password.
[safeer@LinuxBox1 ~]$newgrp accounts
Password:
[safeer@LinuxBox1 ~]$groups
accounts developers admin staff
Now safeer can edit the salary2 file.

Let us summarise the use of group password now:

Group password can be set by root or group administrator. Members of a group can use newgrp to set that group as their primary group, they will never be prompted for password. If a non group member tries to newgrp to the group, he will be prompted for group password provided the group password is set, otherwise the system will simply say "Sorry".

You can disable access through newgrp in two ways. First is to remove the group password with -r option. In this case when a user tries to newgrp, he will be returned a "Sorry". Another way is to disable newgrp feature itself for a particular group with gpasswd -R option. If the user tries newgrp and the password is set, the user will be prompted for password, but even if the password he enters is correct the message "Sorry" will be displayed.
[root@LinuxBox1 ~]#gpasswd accounts
Changing the password for group accounts
New Password:
Re-enter new password:
[safeer@LinuxBox1 ~]$newgrp accounts
Password:

This is ok, now disable newgrp feature for this group
[root@LinuxBox1 ~]#gpasswd -R accounts
[safeer@LinuxBox1 ~]$newgrp accounts
Password:
Sorry.

Instead, if you remove the group password
[root@LinuxBox1 ~]#gpasswd -r accounts
[safeer@LinuxBox1 ~]$newgrp accounts
Sorry.

Now let us see how to manage group membership and administration

To add a user baiju to group developers
[root@LinuxBox1 ~]#gpasswd -a baiju developers
To remove the user from group
[root@LinuxBox1 ~]#gpasswd -d baiju developers

Root user can set group administrators and group members
To set the user sreejith administrator of developers group
[root@LinuxBox1 ~]#gpasswd -A sreejith developers
Note that sreejith need not have to be member of developers to administer it.
Now sreejith can use all the gpasswd options on developers group except for -[A/M]

Root can add a number of users to a group with -M option
[root@LinuxBox1 ~]#gpasswd -M sreejith,baiju,vishin developers
Similarly you can add a number of administrators for a group by comma separated list.

You can manually edit /etc/group & /etc/gshadow using the command "vigr".

Per User web hosting in Apache and Tomcat

If there are multiples users on a web server, each user may be allowed to host a web site in his home directory. By this way the to access the web site hosted in the home directory of safeer on the server www.myhostingteam.net use the URL: http://www.myhostingteam.net/~safeer. This is most often used in Universities and research labs, where each staff/scholar will have a user account in the organization's web server.

Apache HTTPD

In apache, this is achieved by using the UserDir directive. This feature will be disabled by default. The web contents will be supplied from the directory public_html inside the users home directory. The directive for this is:

UserDir public_html
This directive will be commented out by default. If the directory path does not start with a leading slash, the path will be considered relative to the user home directory.In this case the user direcory can be anywhere in the file system. Apache will search the user database (/etc/passwd) to find out the home directory of the user and will fetch the web contents from the public_html subdirectory there.

If the path starts with a leading slash, apache will not search the user database, instead the path will be constructed by appending the username to the path given. That is, if
UserDir /home
when you access http://www.myhostingteam.net/~safeer the contents will be supplied from /home/safeer. This setting has a disadvantage that every world readable file and world executable directory in your home directory will be accessible to the public, if they know that such a file or directory exists. Another disadvantage(?) is that apache will treat every directory under /home as a user. That is, if i create a directory "nouser" in the home directory and the public_html stuff under that, the contents can be accessed as http://www.myhostingteam.net/~nouser.

We can solve the first of these issues by using another format for UserDir:
UserDir /home/*/public_html
The asteric (*) will be replaced by the username. In this and the first approach, only the subdirectory public_html will be accessible by apache, and this is a better choice from a security point of view.
For UserDir to work, the file system permissions should be set properly:
The home directory should be world executable ( chmod 711/a+x),
The files/directories under/including public_html should be world readable & executable ( chmod 755/a+rx).

We can restrict which user has permission to use this feature by using additional UserDir directives.

As in any access restriction scheme there are two approaches here, mostly open and mostly closed.

Mostly Open:

All users will be granted access, except for somebody,
UserDir disabled safeer nebu
Here, all users except safeer & nebu will be granted access

Mostly Closed:

All users will be granted access, except for somebody,
UserDir disabled
UserDir enabled safeer nebu
Here, all users except safeer & nebu will be denied access

By default the feature is disabled by the single directive UserDir disabled. If a user is listed in both disabled and enabled, he will be denied access.

Tomcat

In tomcat, the user web application configurations(and many other features) are configured using the Listener element which is included in the Host element. A "className" attribute of the Listener element determines what feature is to be implemented. For per user web applications, the className attribute will be "org.apache.catalina.startupe.UserConfig".

Here we have two approaches , in the first user home directory information is collected from the password database (/etc/passwd) and in the second all directories under a particular directory will be considered as user home directories. The later approach is used mainly in windows and other systems that do not have /etc/passwd user database. This approaches are implemented using the userClass attribute of the Listener element. A "directoryName" attribute is used to specify the name of the directory under user home directory where the user web applications will be deployed.

First Scenario:

<Host name=. . . . . . . . . . >
. . . . . . . .
< Listener className="org.apache.catalina.startup.UserConfig"
directoryName="public_html"
userClass="org.apache.catalina.startup.PasswdUserDatabase" />
. . . . . . . . . . . . < /Host>

Second Scnario:

< Host name=. . . . . . . . . . >
. . . . . . . .
< Listener className="org.apache.catalina.startup.UserConfig"
directoryName="public_html"
homeBase="C:\TomcatHomes
userClass="org.apache.catalina.startup.HomesUserDatabase" / >
. . . . . . . . . . . . < /Host>

In the first scenario, contents for the request to http://www.myhostingteam.net/~safeer will be
rendered from /home/safeer/public_html where as in second scenario it will be from C:\TomcatHomes\safeer\public_html

Tomcat Virtual Hosting

Tomcat virtual hosts are defined inside the Host elements in $CATALINA_HOME\conf\server.xml. $CATALINA_HOME is the tomcat installation directory, hich in my case is /usr/apache-tomcat-5.5.17.
All the Host elements will be enclosed in an "Engine" element. The relevant portion of deafult server.xml is shown below:

<Engine name="Catalina" defaultHost="localhost">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm"
resourceName="UserDatabase"/>
<Host name="localhost" appBase="webapps"
unpackWARs="true" autoDeploy="true"
xmlValidation="false" xmlNamespaceAware="false">
</Host>
</Engine>

Each host element defines a virtual host. To define a virtual host www.myhostingteam.net, we will define a host element as follows.

<Host name="www.myhostingteam.net" appBase="webapps/myhostingteam_net" >
<Context path="" docBase="hostingteam"/>
</Host>

name : The DNS name of the virtual host.

appBase : Application base directory for the virtual host. This directory will hold the web application(s) to be deployed in this virtual host. Path to the directory can be absolute or relative to $CATALINA_HOME.

The "Context" element defines one or more web application contexts defined for this virtual host. There should be atleast one context defined.

path : context path of the web application, which will be matched against the URI request. If left blank, that context will be the default web application for the particular virtual host.

docBase : Document Base/Context Root for the web appliation. This can be either the directiry containing the web application or path to the the web archive (WAR) file for that application. Th e path can be absolute or relative to "appBase" of the virtual host.

Considr the following example:

<Engine name="Catalina" defaultHost="myjavanet.com">

<Host name="www.myjavanet.com" appBase="webapps/myjavanet_com" >
<Context path="" docBase="."/>
</Host>

<Host name="www.myhostingteam.net" appBase="webapps/myhostingteam_net" >
<Context path="" docBase="hostingteam"/>
<Context path="/special" docBase="special_packages"/>
</Host>

</Engine>

Here we have defined two virual hosts, www.myjavanet.com & www.myhostingteam.net

Host www.myjavanet.com: The base directory that holds web applications is /usr/apache-tomcat-5.5.17/webapps/myjavanet_com. There is only one "Context" element, so only one web application is deployed for this host. The "path" attribute is blank indicating that this context is the default context for www.myjavanet.com . The docBAse is ".", means current directory, so the web application is directly deployed inside the appBase Deirectory, which is /usr/apache-tomcat-5.5.17/webapps/myjavanet_com in this case.

Host www.myhostingteam.net : The base directory for web applications is /usr/apache-tomcat-5.5.17/webapps/myhostingteam_net. It holds two web applications, which are deployed in the directories hostingteam and special_packages (docBase) under the "appBase" /usr/apache-tomcat-5.5.17/webapps/myhostingteam_net. The context hostingteam can be accessed with URL http://www.myhostingteam.net, where as special_packages can be accessed with http://www.myhostingteam.net/special (path="/special").

The defaultHost attribute of the Engine element specifies the default virtual host of the tomcat server( if host name in an HTTP request to this server do not match any of the configured virtual host's name attribute, the contents of this virtual host will be supplied instead). This value should match the name attribute of any one of the virtual hosts configured. The default value for this attribute is "localhost".

Multiple DNS names for a single virtual host

Sometimes, you will need to access your site with more than one DNS name, like:
www.myhostingteam.net & hostingteam.net. In such a situation, use the "Alias" element inside the "Host" element.

<Host name="www.myhostingteam.net" ............... >
....................................
<Alias>myhostingteam.net</Alias>
................................
</Host>

You can have any number of <Alias> elements inside a <Host> element.

Running Apache and Tomcat on the same server

Tomcat is a popular servlet container used to host java web applications. By default Tomcat listens on port 8080 (HTTP-Non SSL) and 8443 (HTTP-SSL). If your machine has multiple IPs, Tomcat will automatically bind itself to all the IPs configured.

In a production environement, there are mainly two aproaches to deploy web applications through tomcat. In the first one, the tomcat server will listen on 8080/8443 or any custom ports. All the HTTP request coming to the server (which will default to 80/443) will be received by a web server- most probably "Apache" complied with mod_jk support, or sometimes IIS. All the java related requests will be forwarded to Tomcat. Thus Apache/IIS will act as a proxy for Tomcat. The second approch, which is much more straight forward is to configure tomcat to listen on port 80/443.

With the second aproach, Tomcat will listen on port 80/443 on all configured IP addresses of the server. This is ok if you have a dedicated server for running tomcat. But it may not be the case always. In my case, when I was asked to set up Tomcat(to listen on 80/443) on one of our live servers, it was already running an apache server that serve a bunch of production web sites.

I had two choices now, either to buy a new server for hosting tomcat or to configure both apache and tomcat on the same server. I would have chosen the first option for perfomance reasons, but it was costly. So I decided to go for the later, and bought one additional public IP for the current production server.

I reconfigured apache to listen on one IP address. For this I edited

/etc/httpd/conf/httpd.conf file to change the entry
Listen 80
TO
Listen 61.17.42.50:80

AND
/etc/httpd/conf.d/ssl.conf to change the entry
Listen 443
TO
Listen 61.17.42.50:443

Now apache will listen on the IP 61.17.42.50 only.

Then I configured Tomcat to listen on the newly prcuhased IP, 61.17.42.76

The configuration file for tomcat is CATALINA_HOME/conf/server.xml where CATALINA_HOME is the installation directory for tomcat. In my case it is
/usr/apache-tomcat-5.5.17.

The file server.xml consists of a number of 'elements' enclosed in ankle brackets and a number of attributes specified for each element within those brackets.

The element that is of interest to us is "connector". There is one connector elements each for HTTP-Non-SSL and HTTP-SSL. I will quote the relevant portions of the configuration file:

HTTP Non SSL

<Connector port="8080" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" redirectPort="8443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true" />

HTTP SSL

<Connector port="8443" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" disableUploadTimeout="true"
acceptCount="100" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />

First, change the ports to listen on 80 and 443

<Connector port="80" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" redirectPort="443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true" />

HTTP SSL

<Connector port="443" maxHttpHeaderSize="8192"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" disableUploadTimeout="true"
acceptCount="100" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />

This is not enough, because tomcat will now try to bind on ports 80 & 443 on both the public IPs. To limit tomcat to listen on 61.17.42.76 only, add an attribute "address" to both the connector elements.

<Connector port="80" maxHttpHeaderSize="8192" address="61.17.42.76"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" redirectPort="443" acceptCount="100"
connectionTimeout="20000" disableUploadTimeout="true" />

HTTP SSL

<Connector port="443" maxHttpHeaderSize="8192" address="61.17.42.76"
maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
enableLookups="false" disableUploadTimeout="true"
acceptCount="100" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS" />

It is not important in what order you arrange the attributes inside an element, but leave spaces between each attributes.

Now that I have apache and tomcat running on the 80/443, I can configure the DNS records for the web sites hosted in apache to point to 61.17.42.50 and DNS record for web sites to be served by tomcat to point to 61.17.42.76.

Important Port Numbers

Here is a list of important port numbers categorized by the type of use

Databases
3306	MySQL
1433	MS SQL Server
5432	PostgreSQL
Remote Access - Graphical
3389	MS Remote Desktop
5800	VNC-HTTP
5900	VNC
Remote Access - Console
22	SSH
23	Telnet
512	Rexec
513	Rlogin
514	RSH
Remote Data Transfer & Management
20	FTP Data
21	FTP
69	TFTP
2049	NFS
Directory Service
389	LDAP
636	LDAP Secure
Web
80	HTTP
443	HTTPS
3128	Squid Web Proxy
Mail & Chat
25	SMTP
110	POP3
995	POP3S
143	IMAP
993	IMAPS
5222	Jabber
5223	Jabber SSL
Infrastructure Management
53	DNS
1512	WINS
67	DHCP Server
68	DHCP Client
123/UDP	NTP
161	SNMP