Bash Date Math

There were many occasions when I had to find out something like "one week after current date" or "one month and two days earlier", something like that.  All this lead me to searching for 'date mathematics' in bash.   And here is the result I found. (Warning: this may not work in older systems)

This is achieved using the -d option of date which has a syntax:

date -d STRING

where string defines, the format of date.  Here is were you can implement date math.  As you can see below:

safeer@enjoyfast-lx:~$ date
Wed Dec  3 00:11:06 IST 2008

Now find tomorrows date - just use "+1 day" as format string
safeer@enjoyfast-lx:~$ date -d "+1 day"
Thu Dec  4 00:11:07 IST 2008

So the general format of the string is "+/-N UNIT" where UNIT is any one of "second|minute|hour|day|week|month|year" and N is the number of units.  + or - sign determines whether to add or subtract a date.  A few examples:

15 hours after current time:

safeer@enjoyfast-lx:~$ date;date -d "+15 hour"
Wed Dec  3 00:14:32 IST 2008
Wed Dec  3 15:14:32 IST 2008

3 months earlier:

safeer@enjoyfast-lx:~$ date;date -d "-3 months"
Wed Dec  3 00:18:39 IST 2008
Wed Sep  3 00:18:39 IST 2008

You can also combine different units as needed:

safeer@enjoyfast-lx:~$ date;date -d "+15 week -2 hours"
Wed Dec  3 00:21:21 IST 2008
Tue Mar 17 22:21:21 IST 2009

This way we can get whatever date we need.  Combining this with the output formating option for date will give you good formatted dates to use in your scripts and commands.  For eg:

safeer@enjoyfast-lx:~$ date -d "-2 year +42 hours"
Mon Dec  4 18:23:22 IST 2006

safeer@enjoyfast-lx:~$ date -d "-2 year +42 hours" +%F
2006-12-04

TCP Wrappers

TCP wrappers is used to implement host based access restrictions to network services running on a Linux server. In order for a network service to work with TCP wrappers, it should be compiled with libwrap.a library. To check whether a network service has TCP wrapping enabled use:
strings -f /path/to/service/binary|grep hosts_access, for example to check whether ssh is compiled with TCP wrappers support,

[root@LinuxBox1 ~]#strings -f /usr/sbin/sshd|grep hosts_access
/usr/sbin/sshd: hosts_access

Host access is allowed or denied based on the entries in the files /etc/hosts.allow & hosts.deny ( together known as host access files) . Each of these files contain a list of services and corresponding list of hosts along with certain additional options.

As the name indicates, hosts.allow lists the hosts allowed to access listed services. When a host tries to access a network service on the server, hosts.allow is parsed from top to bottom to see a match for the service-host pair. The parsing will stop with the first match found and the access will be allowed. If no match is found in this file, hosts.deny is parsed. If a match is found access is denied for the host. If neither file contain a match or the files does not exist, access is allowed for the host.

Access files contain a number of rows, each with following format

daemon-list:host-list:[option1]:[option2]:..........

daemon list : Comma separated list of service binary names. Dont take this for the service name, for example, the service name for telnet is 'telnet' but the binary is 'in.telnetd' (/usr/sbin/in.telnetd).

host list : Comma separated list of hosts to be allowed/denied access to the daemon list. Hosts can be specified in any of the following format.

host names : machine1.mydomain.com represents a single machine
host names starting with a period: .mydomain.com represents all host names under this name. Example: machine2.mydomain.com, switch2.sub1.mydomain.com ......
Hostnames containing ? and/or * : Within a hostname or IP you can use this wild cards, '*' can substitute any number of characters while '?' can replace only a single character.
Eg:
*.mydomain.com : mac1.mydomain.com, mac2.sub1.mydomain.com ....
mac?.mydomain.com : mac1.mydomain.com,mac2.mydomain.com,...mac9.mydomain.com,maca.mydomain.com,macb.mydomain.com etc...
IP address ending with period : 192.168. represents all IPs in the network 192.168.0.0/16
Network & Netmasks : 192.168.1.0/255.255.255.128, 10.10.0.0/255.255.224.0 etc...
File path beginning with leading '/' : The specified file contains a list of hosts. Each line of the file will contain a number of host/ip patterns separated by white spaces.

Explicit wild cards: This wild cards has special purposes, even though there are many the popular one is 'ALL' which matches anything ( applicable to daemons as well as hosts)

The 'EXCEPT' operator: This can be used with both hosts and daemons as follows:
a) In the daemon list, a rule that is applicable to all services except vsftpd
ALL EXCEPT vsftpd
b) In the clients list all machine in the network 192.168.0.0 except 192.168.0.15
192.168.0. EXCEPT 192.168.0.15

options : This fields can be used to alter the log behaviour, launch shell commands etc when a rule is matched, but this feature is specific toRedhat version of tcp wrappers.

There are two approaches for any firewall/access control design, mostly closed and mostly open. In mostly closed all accesses arerestricted and only minimum required access is granted. For TCP wrappers mostly closed means a single line ALL:ALL in hosts.deny and necessary services in hosts.allow. In mostly open, all client access is allowed and restrictions applied as and when necessary. InTCP wrappers: ALL:ALL in hosts.allow and necessary restrictions in hosts.deny.

Some examples rules:

Allow telnet from local networks (192.168.0.0./24 & 192.168.1.0/24)
hosts.allow
in.telnetd:192.168.0.,192.168.1.
Suppose, in addition to this, you want to deny telnet access from all other hosts.
hosts.deny
in.telnetd:ALL
Since hosts.allow is parsed first the access rule ther will match request from local networks and allow them. If the request is from an external host, the rules in hosts.allow will not match and will be passed to hosts.deny where the rule is to deny all telnet communication. This rule will be matched and access denied for that host.

Alternatively we can do both this in a single rule in hosts.deny
hosts.deny
in.telnetd: ALL EXCEPT 192.168.0.,192.168.1.

Adding swap space to Linux

The virtual memory used by Linux is called swap space. It is the extension of physical memory in the sense that the total usable primary memory will be the sum of RAM and swap space. The kernel will write the memory blocks of unused programs to the swap space, thus freeing the RAM to be used by active programs. Hard disk access is much slower when compared to RAM access. So swap space can't be considered as a replacement for RAM, it is just a supporting techniques for RAM. Using of swap space imposes overhead on the system, but for systems with less physical memory it is a necessary evil.

When you install the operating system you will add a swap space, most probably up to double of the RAM. This will not be enough if you later add more RAM to your machine and you will have to increase the swap space. You can add swap space to your system in two ways, either by adding a swap partition or swap file.

To use swap partition, you should have unallocated space left on your hard disk. Use fdisk program to create a new partition.
[root@LinuxBox1 ~]#fdisk /dev/hdb
and use "n" command to create a new partition, say /dev/hdb2

If you don't have unallocated space to create a new partition, but have free space on existing partitions, you can create swap files instead. IF you want to create 128 MB of swap space, create a file of that size.

[root@LinuxBox1 ~]#dd if=/dev/zero of=/vmem/swapfile01 bs=1024 count=131072
131072+0 records in
131072+0 records out
134217728 bytes (134 MB) copied, 3.0451 seconds, 44.1 MB/s

In the rest of this document, I will use this swap file as example, but the procedure is same for swap partition also.

To mark the created space usable as swap, a signature is needed to be written into the start of file/partition. This is done withmkswap command.
[root@LinuxBox1 ~]#mkswap /vmem/swapfile01
Setting up swapspace version 1, size = 134213 kB

Now tell the kernel to use this space for swap
[root@LinuxBox1 ~]#swapon /vmem/swapfile01

In order to keep this settings persistent over reboots, add a line to /etc/fstab.
/etc/fstab
/vmem/swapfile01 swap swap defaults 0 0

To list all the swap partitions/files in a system,
[root@LinuxBox1 ~]#swapon -s
Filename Type Size Used Priority
/dev/hda3 partition 1124540 1848 -1
/vmem/swapfile01 file 131064 0 -3
/vmem/swapfile02 file 65528 0 -4

To see the current primary memory status
[root@LinuxBox1 ~]#free -m
total used free shared buffers cached
Mem: 495 470 24 0 3 204
-/+ buffers/cache: 263 232
Swap: 1290 1 1288

You can see that Total Swap reported is the sum of 3 swap sizes as listed in the output of "swapon -s"

If you want to remove the added swap space, turn of swapping for the partition/file
[root@LinuxBox1 ~]#swapoff /vmem/swapfile01

If you want to save hard disk space, remove the file/partition.

Swap files are most useful when you need to run a memory intensive program for a short period of time. You can create a swap file of appropriate size and delete it when the application is finished. If you want to add swap for permanent use, better go for swap partitions- provided your hard disk has unallocated space left.

Blocking Google Talk in your organization

Blocking instant messengers in an organization is a big headache for all system administrators. Not only that IMs waste precious productivity time and bandwidth of your company, it also possess some security threats. The major threat comes with file sharing capabilities of IMs, which can be used to share proprietary information of the company. This will become a night mare for the management if they have signed a non disclosure agreement with their clients. There are a lot of ways for an employee to ship information out of a company, but IM is one of the easiest method to do so. Another big problem is spreading of viruses and worms, unlike emails it is not easy to integrate Anti virus softwares with IMs.

But when you consider blocking IMs, the problem is that most of them are set to work with least resources and under almost any network conditions. One way is to find out the port used by the IM and block it. This works fine for Microsoft Messengers, but most others have work around for this. Here let us specifically take the case of Google Talk.

Google talk initially used Jabber/XMPP protocol which communicates over 5222 & 5223 (for SSL) ports. But blocking these ports alone will not fix the problem. The reason is that if this ports are not available, gtalk will switch to HTTP & HTTPS ports. Obviously we can’t block all HTTP & HTTPS communication. So the only way is to use a combination of protocols and server IPs (of Google talk). Again another problem with this approach is that Gtalk is having a number of servers and if it can’t access one server, it will try another IP and so forth until the connection is established. So we need to find out all the servers(IPs) used by Gtalk and block all of them. This can only be done with a trial and error approach.

Login into you Gtalk account, now the connection will be established with one of the Gtalk servers. Find out the process id of Gtalk application by either using Task Manager or the command line “tasklist.exe”. This is the out put from tasklist.exe command line:

C:\Documents and Settings\Administrator>tasklist /FI “IMAGENAME eq googletalk.exe”

Image Name PID Session Name Session Mem Usage
========================= ====== ================ ======== ============
googletalk.exe 3492 Console 0 4,900 K

So 3492 is the process id of Gtalk application. Now use the netstat.exe command line or any network monitoring application to find out the remote IP and port to which this application is connected. Look at the netstat output

C:\Documents and Settings\Administrator>netstat -ano|find “3492″

TCPÂ 192.168.0.23:4381 72.14.253.125:5222 ESTABLISHED 3492

So we have now one IP of Google talk: 72.14.253.125Â and the port connected to is 5222 -Jabber/XMPP.

Now logout of Google talk and block the communication to this IP through the protocols HTTP,HTTPS, Jabber, Jabber -SSL.ÂLogin to google talk again and repeat the same steps again. After a while you will get all the IPs used by Gtalk. Now block all the IPs as above. This will block not only Google talk but the chat applet inside the Gmail window.

The problem with this approach is that such servers will switch IPs frequently, and you will have to continuously update the list of servers. Bellow is the list of Gtalk Servers that I have found out. As of writing this, this list has been working for our ISA Server 2006 for the past month. I just created a computer set for the Gtalk servers and blocked the HTTP,HTTPS,Jabber Protocols to this Computer set. Also you will have to create Jabber as a custom protocol with outgoing TCP/UDP communication to ports 5222 and 5223.

Gtalk Servers:

216.239.37.125

72.14.253.125

72.14.217.189

209.85.137.125

209.85.163.125

209.85.201.189

216.239.51.125

Protocol - Port

HTTP - 80

HTTPS - 443

Jabber - 5222

Jabber- SSL - 5223

Linux group administration

Group information is primarily stored in /etc/group. There will be some default groups created when operating system is installed. Each line in /etc/group corresponds to a group . Each of these line will have 4 fields separated by colons. an example line is as follows

developers:x:520:baiju,vishin,suraj

Field 1 - developers : Group name

Field 2 -x : The group password was previously stored here. The x here indicate that group password is shadowed(stored in a separate file /etc/gshadow which is only readable by root)

Field 3 - 520 : The numerical group id

Field 4 - baiju,vishin,suraj : A comma separated list of group members. The user will be listed here only if this is his secondary group. Primary group membership will not be shown here.

To add a group use the groupadd command

Syntax:

groupadd [OPTIONS] groupname

OPTIONS:

-r : Create a system account with group id below 4999

-g GID : Numeric group id

To modify a group property use groupmod

Syntax:
groupmod [OPTIONS] groupname

OPTION:

-g GID : change group id

-n new_grp_name : Change group name

To delte a group, use groupdel command

Syntax

groupdel group_name

To add a user to a group you can use usermod or useradd commands with -g/G options. But the limitation with this approach is that only a single user is added at a time. More over this is done from a user perspective. To really administer group memberships, you will have to use gpasswd command.

Every group can have administrators,members and a password. Only root can assign administrators to a group while group admins and root can add members to a group.

The group password is used to restrict access through the command newgrp. The newgrp command is used to switch the primary group of a user. Suppose the user safeer has admin as his primary group, developers and staff as his secondary groups. When user safeer creates a file the owner of the file will be safeer and group of file will be admin (primary group of safeer). In some situations we may need to switch our primary group, either for creating new files with different group membership or access a file to which a certain group has permission. I would better illustrate with an example. Take the case of user safeer in the above example:

Display the groups of which safeer is a member
[safeer@LinuxBox1 ~]$groups
admin developers staff
Of this admin is the primary group and others are secondary groups. Now will create some files with different group membership
[safeer@LinuxBox1 ~]$touch file1
[safeer@LinuxBox1 ~]$ll file1
-rw-r--r-- 1 safeer admin 0 Mar 29 01:07 file1
[safeer@LinuxBox1 ~]$newgrp developers
[safeer@LinuxBox1 ~]$groups
developers admin staff
[safeer@LinuxBox1 ~]$touch file2
[safeer@LinuxBox1 ~]$ll file2
-rw-r--r-- 1 safeer developers 0 Mar 29 01:07 file2

When you use newgrp, you are automatically allotted a new shell. If you type exit from that promt you will be broguht back to the previous shell with old group membership. From the above example
[safeer@LinuxBox1 ~]$exit
[safeer@LinuxBox1 ~]$groups
admin developers staff

The main advantage of newgrp is in access permission to files. Suppose there is a file with the following permissions.
-rw-rw---- 1 root accounts 0 Mar 29 01:07 salary2

As you can see, safeer (or any others except root and members of accounts group) don't have the permission to access this file. Now a situation arises where safeer has to access this file (of course, legally!) . Then the administrator can set a password for group 'accounts ' using gpasswd
[root@LinuxBox1 ~]#gpasswd accounts
Changing the password for group accounts
New Password:
Re-enter new password:

Now root gives safeer the accounts group password.
[safeer@LinuxBox1 ~]$newgrp accounts
Password:
[safeer@LinuxBox1 ~]$groups
accounts developers admin staff
Now safeer can edit the salary2 file.

Let us summarise the use of group password now:

Group password can be set by root or group administrator. Members of a group can use newgrp to set that group as their primary group, they will never be prompted for password. If a non group member tries to newgrp to the group, he will be prompted for group password provided the group password is set, otherwise the system will simply say "Sorry".

You can disable access through newgrp in two ways. First is to remove the group password with -r option. In this case when a user tries to newgrp, he will be returned a "Sorry". Another way is to disable newgrp feature itself for a particular group with gpasswd -R option. If the user tries newgrp and the password is set, the user will be prompted for password, but even if the password he enters is correct the message "Sorry" will be displayed.
[root@LinuxBox1 ~]#gpasswd accounts
Changing the password for group accounts
New Password:
Re-enter new password:
[safeer@LinuxBox1 ~]$newgrp accounts
Password:

This is ok, now disable newgrp feature for this group
[root@LinuxBox1 ~]#gpasswd -R accounts
[safeer@LinuxBox1 ~]$newgrp accounts
Password:
Sorry.

Instead, if you remove the group password
[root@LinuxBox1 ~]#gpasswd -r accounts
[safeer@LinuxBox1 ~]$newgrp accounts
Sorry.

Now let us see how to manage group membership and administration

To add a user baiju to group developers
[root@LinuxBox1 ~]#gpasswd -a baiju developers
To remove the user from group
[root@LinuxBox1 ~]#gpasswd -d baiju developers

Root user can set group administrators and group members
To set the user sreejith administrator of developers group
[root@LinuxBox1 ~]#gpasswd -A sreejith developers
Note that sreejith need not have to be member of developers to administer it.
Now sreejith can use all the gpasswd options on developers group except for -[A/M]

Root can add a number of users to a group with -M option
[root@LinuxBox1 ~]#gpasswd -M sreejith,baiju,vishin developers
Similarly you can add a number of administrators for a group by comma separated list.

You can manually edit /etc/group & /etc/gshadow using the command "vigr".