Test your SMTP server with Telnet

     SMTP ( Simple Mail Transfer Protocol ) is the protocol email servers use to send and receive mails.   Normally we do use email clients like Thunderbird,Outlook or the web-mail to send and receive mails.  But what happens behind the scene is an exchange of SMTP commands and responses between the client and the mail server.  Knowing how to use these commands directly will help you in testing a mail servers functionality.  Here we will examine the basic commands and use them to connect to a mail server and send mail.

     We will use the telnet program to establish a connection to the mail server ( on port 25 - this can be different based on your mail server configuration )

safeer@penguinpower:~$ telnet smtp.safeer.in 25
Trying 10.25.0.1...
Connected to smtp.safeer.in.
Escape character is '^]'.
220 smtp.safeer.in ESMTP Postfix (Ubuntu)
    The first three lines of the output is common for any telnet session, it just says the connection is successfully established.

    The fourth line is a response from the mail server.  Every response from a mail server starts with an SMTP response code followed by further information.  Now in the above line the mail server is displaying its banner which mentions the hostname of the mail server ( smtp.safeer.in ) , the mail server software ( Postfix ) and the operating system on which the server is running ( Ubuntu ).  This banner is configurable in your mail server.

    In order to communicate with the mail server, you should send SMTP request commands which will be in the format of a command followed by arguments.

1) Once you establish a connection with the mail server, to initiate a conversation with it you would greet the server with the command HELO followed by your hostname.  The server will greet you back with response code 250 followed by its hostname.

HELO penguinpower
250 smtp.safeer.in

    Now let us compose a mail

2) The first step is to specify the "from" email address.   The command is "MAIL FROM: <your email id>".  The response code will be 250 followed by the message OK

MAIL FROM: safeer@safeer.in
250 2.1.0 Ok

3) Next specify the recipient address using the command "RCPT TO: <recipient email id>".  The response again will be code 250 followed by message.

RCPT TO: praveen@gmail.com
250 2.1.5 Ok

    This can be repeated multiple times to allow multiple recipients.

4) Once you are ready to type in the message, enter the command "DATA".  Response will be code 354 followed by message.

DATA
354 End data with <CR><LF>.<CR><LF>

    The email body will contain your email message and optionally email headers (which should be added at the beginning of the message).  A few commonly used message headers are Subject,From,To,Cc

     A message header should be of the following syntax:

     <HEADER NAME>: <HEADER VALUE>

    After the headers, add your message then put a period ( . ) on a new line and press enter to finish the message.

     Let us compose our message now.

Subject: Happy New Year!!!
From: safeer@safeer.in
To: praveen@gmail.com
Happy New Year
.
250 2.0.0 Ok: queued as C53E7265CB0

    The response from mail server says the message is accepted for delivery.

    Remember that the delivery of the message is not dependant on the From/To/Cc headers within the message body. MAIL FROM & RCPT TO parameters decides the delivery.  The headers in message is as such delivered to the recipients, and it is the duty of the mail client of the recipient to interpret and present the message headers to the user.

5) If after entering RCPT TO or MAIL FROM you feel that you provided wrong information and want to start over without terminating the existing connection, issue the command RSET ( stands for reset ).

4) We are done.  Say goodbye to server.

QUIT
221 2.0.0 Bye
Connection closed by foreign host.

   Second line above is the response from server while the third line is the output of telnet command terminating.

For more information on smtp protocol, refer to SMTP RFC at IETF

Configuring Dovecot as a POP3 or IMAP server

     Dovecot can be used to retrieve mail using pop3 or imap protocols.  The minimal configuration that will get you started is pretty simple and straight forward.

First install the package, for RHEL based distros, it is dovecot.

yum install dovecot

The main configuration file for dovecot is /etc/dovecot.conf ( location changes based on you installation, file name remains as dovecot.conf )

## Dovecot supports pop3,pop3s,imap and imaps protocols. 
## Enable whichever you require.
protocols = imap pop3

## If you are using conventional unix style mboxes under
## /var/spool/mail/<username> for storing mail
mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u

That is it, you can start using dovecot sevice.  Edit and save the file and restart service.

sudo /usr/sbin/service dovecot restart

To test if dovecot is working, use instructions from following tutorial

Testing your POP3 server with telnet

Managing ISO files in Linux

ISO (International organizations for standardization) format is the archive file format for optical discs. The name is derived from the file system format for optical discs, known as ISO 9660 File System. This format is also known as CDFS (Compact Disc File System). ISO files are used to either create the image of an optical disc, or to write data to it.

Here are a few tricks to create and manage ISO files.

Creating an ISO image out of a CD ROM mounted in your machine.

We can use the dd (Disc Dump) command to take the copy of an optical disc. For this first you have to insert the disc in the drive, then find out the device name of your CD-ROM.

Once the CD is mounted, you can find your CD drive in this way:

safeer@penguinpower:~$ mount|grep -i iso
/dev/sr0 on /media/UBUNTU_10 type iso9660(ro,nosuid,nodev,uhelper=udisks,uid=1000,gid=1000,iocharset=utf8,mode=0400,dmode=0500)

Here my CD device is /dev/sr0. In most of the systems where you have a single CD drive, this will be symlinked to /dev/cdrom, so you can even use that in place of /dev/sr0 ( or whatever your device is).

safeer@penguinpower:~$ ls -l /dev/cdrom
lrwxrwxrwx 1 root root 3 2010-11-06 10:23 /dev/cdrom -> sr0

Now that we know our CD device name, let us create the ISO image of the mounted disc.

safeer@penguinpower:~$ dd if=/dev/sr0 of=/home/safeer/ISO/ubuntu10.iso
1365028+0 records in
1365028+0 records out
698894336 bytes (699 MB) copied, 267.224 s, 2.6 MB/s

Where "/dev/sr0" is my CD device and "/home/safeer/ISO/ubuntu10.iso" is the name of the iso image that will be created from the CD.

Mounting an ISO file as if you are mounting a CD-ROM.

ISO file can be mounted in a UNIX file system just like inserting a CD to your CD drive. We make use of the loop device system for this purpose. Loop device is a pseudo file system available in Linux/UNIX that makes a file accessible as a block device.

First, create a mount point

safeer@penguinpower:~$ sudo mkdir /mnt/iso

Then mount the ISO file as loop device

safeer@penguinpower:~$ sudo mount -o loop -t iso9660 /home/safeer/ISO/ubuntu10.iso /mnt/iso/
safeer@penguinpower:~$ mount |grep iso
/dev/loop0 on /mnt/iso type iso9660 (rw)

Now you can browse the contents of iso /home/safeer/ISO/ubuntu10.iso via the directory /mnt/iso/.

To unmount the device:

safeer@penguinpower:~$ sudo umount /mnt/iso

Creating an ISO file from a directory.

You can use mkisofs command to create an iso. In my system mkisofs is a symlink to the program genisoimage. You can use either command for this purpose.

Here I will create an iso image of all softwares that I downloaded.

safeer@penguinpower:~$ mkisofs -o software.iso /home/safeer/Downloads/
I: -input-charset not specified, using utf-8 (detected in locale settings)
Using VPNC_000.RPM;1 for /vpnc-0.5.3-8.fc13.src.rpm (vpnc-0.5.3-6.fc11.x86_64.rpm)
Using VPNC_001.RPM;1 for /vpnc-0.5.3-6.fc11.x86_64.rpm (vpnc-0.5.3-8.fc13.x86_64.rpm)
1.03% done, estimate finish Sun Nov 7 18:28:59 2010
2.06% done, estimate finish Sun Nov 7 18:28:59 2010
................................
................................
98.68% done, estimate finish Sun Nov 7 18:29:33 2010
99.71% done, estimate finish Sun Nov 7 18:29:34 2010
Total translation table size: 0
Total rockridge attributes bytes: 0
Total directory bytes: 4914
Path table size(bytes): 36
Max brk space used 1b000
486421 extents written (950 MB)

Let us check the iso image by mounting it.

safeer@penguinpower:~$ sudo mount -t iso9660 -o loop software.iso /mnt
safeer@penguinpower:~$ cd /mnt/
safeer@penguinpower:/mnt$ ls
a0654873.tor google_c.deb skype_de.deb torrentd.tor vpnc_000.rpm vpnc_0_5.deb vpnc_con.rpm
adberdr9.deb opera_10.deb skype_ub.deb virtualb.deb vpnc_001.rpm vpnc_0_5.rpm vpnc_scr.rpm

The problem with using the default settings for mkisofs is that it truncates the filenames to be compatible with DOS file system. A few options that will help us create better image is shown below.

safeer@penguinpower:~$ mkisofs -r -R -J -l -o software.iso /home/safeer/Downloads/
I: -input-charset not specified, using utf-8 (detected in locale settings)
1.03% done, estimate finish Sun Nov 7 18:56:33 2010
....................................................
99.71% done, estimate finish Sun Nov 7 18:57:00 2010
Total translation table size: 0
Total rockridge attributes bytes: 5152
Total directory bytes: 11248
Path table size(bytes): 36
Max brk space used 1c000
486433 extents written (950 MB)

Checkout the man pages for more details on mkisofs

Configuring Postfix SMTP as your MTA

     Postfix is one of the most popuar MTAs around.  Postfix has a modular design that improve security,speed and east of configuration.  In this design, a master daemon controls individual light weight processes (modules) that performs specific tasks.  The master deamon binary is "/usr/sbin/postfix" and the daemon configuration file is /etc/postfix/mster.cf.  But this file rarely needs change unless you need to enable / disable some sub processes(modules).  The MTA configuration is in  /etc/postfix/main.cf.  There are other configuration files for specific purposes and all of them lies under the folder /etc/postfix.  Following is a minimal configuration of main.cf that will let you run a postfix mail server for your own domain.

Most of the configuration parameters in main.cf will have a commented example version, you can either uncomment and edit them, or add a new line.  I will just list out important configuration parameters with comments added.  The example configuration is for running an MTA for safeer.in domain.

## The configuration file is in "key = value" format.  The key defined previously in the file
## can be referred to as $key in the coming sections.

## The domain for which you are configuring MTA
mydomain = safeer.in

## FQDN of the mail server where postfix is being configured
myhostname = smtp.safeer.in

## The domain name that locally sent and received mail appears to be from and/or delivered to.
## When you are running mail server for a domain, set as follows, default if unset is to use hostname.
myorigin = $mydomain

## By default postfix listens only on loopback, make it listen an all interfaces.
inet_interfaces = all

## The SMTP server will accept mails to the email ids addressed to the following domains and/or hostnames.
## In our setup, this will include the mail server (smtp.safeer.in) and the mail domain safeer.in
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain

## Define trusted networks from which clients can connect to the smtp server without any restrictions.
mynetworks = 192.168.4.0/24, 10.10.5.0/24

## Allow smtp clients only from $mytnetworks and reject un authorized clients
smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination
    
     This will get you started off with a minimal SMTP server.  Make these changes and restart your postfix server.

sudo /usr/sbin/service postfix

Ensure postfix has started listening by running

sudo netstat -atnp|grep :25|grep LISTEN

To test your server is working from command line, visit this link

Test your SMTP server with telnet

To know more about postfix configuration parameters check the official website of postfix or look at the man page by running "man 5 postconf"

Perl Snippets Part 2 - CSV to XLS

This program will convert a CSV ( Comma Separated Value ) file to an XLS ( Windows Excel ) spreadsheet. The script should be provided with the CSV file name as the first command line argument. It takes an optional second argument as the output XLS file name. If no output filename is given, the output will be saved to out.xls in the current directory.

This script requires Spreadsheet::WriteExcel perl module. On my Ubuntu box, this was installed with the command:
safeer@penguinpower:~$sudo apt-get install libspreadsheet-writeexcel-perl

#!/usr/bin/perl -w

use Spreadsheet::WriteExcel;

if (!defined $ARGV[0]) { die "You must provide a csv file"; }
else {$CSV_FILE = $ARGV[0];}

if ( defined $ARGV[1] ) { $XLS_FILE = $ARGV[1]; }
else { $XLS_FILE = "out.xls"; }

unless ( open ( CSV,$CSV_FILE))
{ die "Unable to open ".$CSV_FILE;}

my $workbook = Spreadsheet::WriteExcel->new($XLS_FILE);
if ( !defined $workbook ) {
             die "Unable to create excel file\n";
         }
$worksheet = $workbook->add_worksheet();
$col = $row = 0;

while (<CSV>)
{
chomp;
my @line = split(",",$_);
for $cell (@line){
      $worksheet->write($row, $col,$cell);
      $col++;
      }
$row++; $col = 0;
}
$workbook->close();
close (CSV);