Managing your IMAP account using Perl

This perl script will give you a basic idea of how your IMAP account can be managed  via perl.  I have been using perl to manager certain tasks with my GMail account ( GMail provides both POP and IMAP services).

We make use of the Mail::IMAPClient perl module for manipulating imap accounts.  This script also prompts for a password from command line so that you don't have to hard code your password in the script.

### Module loading

### Mail::IMAPClient for manipulating IMAP accounts
use Mail::IMAPClient;

### Term::ReadKey is for accepting passwords from terminal
use Term::ReadKey;

### Ask for password
print "Enter Password:";
               ReadMode('noecho');
               chomp ( my $imapPassword = ReadLine(0)) ;
               ReadMode('restore');
print "\n";

### Provide your imap username here
my $imapUser = 'your.imap.user' ;

### Provide your imap server name
my $imapServer = 'your.imap.server.name';

### Provide your imap server port number, default is 993.
my $imapPort = 993;

my $imap = Mail::IMAPClient->new(
    Server   => $imapServer,
    User     => $imapUser,
    Port     => $imapPort,
    Password => $imapPassword,
    Ssl      => 1,
  )

or die "Can't connect to $imapServer as $imapUser: $@";

### You are now logged into the imap account.  Play around a bit

### Print all folders
my $folders = $imap->folders or die "List folders error: ", $imap->LastError, "\n";
foreach ( @$folders )  { print $. , " ) " , $_ ,"\n"; }

### Count messages in the Sent folder
my $msgcount = $imap->message_count('Sent');
print "Count $msgcount\n";

### Process all emails in the folder "Work"

## Start by selecting the Work folder
$imap->select('Work') or
    die("Couldn't select Work folder", $!);

## Load all message ids in Work folder to @msgs
my @msgs = $imap->messages or die "Couldn't get messages: $@\n";

## Backup message to $file

my $TS = time;
my $FILE = "imap_bakup_".$TS;
open MAILBACKUP, ">>", $FILE or die "Couldn't open $FILE"
print MAILBACKUP "Starting backup at time : $TS\n";}

foreach my $msgid (@msgs)
{
 $imap->message_to_file(MAILBACKUP,$msgid) or die "Couldn't backup $msgid: $@\n";
}

print MAILBACKUP "Finished backup";
close BACKUP;

## Delete all messages in "Work" folder
$imap->delete_message(\@msgs) or die "Couldn't delete message in folder Work: $@\n";

### Create new folder "Life"
$imap->create("Life") or die "Couldn't create folder Life: $@\n";

### Delete the folder "Life"
$imap->delete("Life") or die "Couldn't delete folder Life: $@\n";

### Expunge messages marked as deleted in the selected folder
$imap->expunge() or die "Couldn't expunge: $@\n";

### Close the selected folder
$imap->close or die "Couldn't close: $@\n";

### Logout IMAP
$imap->logout

Import functions and variables in bash

     Those who are familiar with programming languages like C/C++/Perl/PHP etc know about importing functions and variables from other file into the program.  This enables the programmer to consolidate reusable code into a file and share it between multiple programs.

     In a similar way, bash is also capable of loading functions and variables from other programs.  It is a lot less used in bash compared to other languages, but there are a couple of places that we know off where bash uses this feature.  One of the is the /etc/profile file.

     There are three ways of doing this.  Let us assume that our functions and variables are in a file called mydefs.inc.  To include it in your code, say myscript.sh, just add one of the following to the top of myscript.sh

 . ./mydefs.inc

source ./mydefs.inc

    I prefer the second form as putting a period before the script may not be noticeable always. The filename should be present in your path if you are giving the filename without a path.  Else you should provide a rlative or absolute path. The include file is just like any other shell script, so it should have proper executable permissions set.

Now a quick example:

safeer@penguinpower:~/DEV$ cat calculator.inc
#!/bin/bash
function sumOf()
{
echo $(($1 + $2));
}

safeer@penguinpower:~/DEV$ cat calculator.inc
#!/bin/bash:~/DEV$ cat testsum.sh
#!/bin/bash
source ${PWD}/calculator.inc
echo $(sumOf 1 2)

safeer@penguinpower:~/DEV$ ./testsum.sh
3

     I defined the function sumOf in calulator.inc file and included it in testsum.sh script at the beginning and then called the function with two arguments.  As expected the function returned the result of 1+2.

Bash Recipes Part 2 - Process a string character by character

For splitting a string character by character, or doing any sort of operation per each character use the following code snippet

safeer@penguinpower:~$ NAME=safeer;for((i=0;i<${#NAME};i++)); do echo ${NAME:$i:1}; done

s

a

f

e

e

r

${#NAME} is the length of the variable $NAME

${NAME:$i:1} - This is a bash parameter substitution technique. U can extract a substring of a variable by using the syntax "${parameter:offset:length}" - This expands to up to "length" characters of parameter starting at the character specified by "offset" - The offset starts at zero.

In our case we use $i as the offset and in every iteration of the for loop we increment $i from zero to length of the string minus one ( which is the list of offsets for every single character in the string ). The length of substring is always 1 ( hence single character ). So ultimately per every iteration we extract one character from the string advancing from the beginning to the end.

Now if you want to extract the character in the reverse order, start the index from -1 and down to negative of string length.

safeer@penguinpower:~$ A=safeer;for((i=-1;i>=-${#A};i--)); do echo ${A:$i:1}; done

r

e

e

f

a

s

Configure mutt as IMAP client

     Mutt is a lightweight yet powerful command line email client.  We will look at a minimal configuration for an IMAP account here.

     Mutt has a global configuration file in either /etc/Muttrc or /usr/local/share/mutt/Muttrc.   This is useful for configuring system wide settings.  But to configure your own account you have to configure your personalized muttrc file in your home directory at ~/.muttrc.

     Here is a sample muttrc file that can get yo start  off with mutt.  Explanations are given as comments.

## Set your imap server name and port here.
set folder=imaps://my.imap.server.name:imapPort/
## Where to look for the mails
## the "+" simbol means, append this to the value of "folder" setting
set spoolfile=+Inbox
## Your imap username
set imap_user=my_imap_user
## Set your imap password - Not preferred for security reasons
## If not provided mutt will prompt you for a password
set imap_pass=my_imap_pass
## Setting sent folder ( note the "+" sign )
set record=+Sent
## Setting draft folder ( note the "+" sign )
set postponed=+Drafts
## To cache downloaded header
## Good for performance
## directory should be manually created
set header_cache=~/.mutt/cache/headers
set message_cachedir=~/.mutt/cache/bodies
## Your preferred editor for composing
set editor=vim
## How to sort your mail
set sort='last-date-received'

A quick primer to GNU Screen - The terminal multiplexer

  Screen is a terminal multiplexer that allows you to open up multiple virtual terminal sessions inside a single terminal.  This enables the user to run multiple programs from a single terminal independently and in parallel.  It also allows switching between them and scrolling through the history as if you were using real independent terminal. 

     Now this may not be an attraction in today's Linux desktop environment where you can open up multiple tabs of the terminal program you want to use ( like gnu-terminal, kde konsole etc).  But imagine the case where your local machine don’t have GUI installed or you are connected to a remote host's terminal via ssh connection.  In the case of local machines terminal, there is a limit to the number of terminals a system can have.  In the case of ssh connection, you can open multiple connections but it takes up network and to manage them you need to open up an equal number of local terminals.  Screen solves this limitation by allowing to spawn virtual shell within a single shell and easily navigate between them.

     This is once advantage of screen but the best part is yet to come.  Any program that you run from a shell is usually spawned as a child of the shell/terminal program.  Hence terminating the shell/terminal session will kill any child process running in it.  Screen has the ability to detach itself from the parent shell process and hook itself as a child of the init process.  This makes screen resilient to logout ( for local terminals ) and network disconnects ( for remote terminals ).  This is a great tool for a power user.

Now let us see how we can use screen.

     First install the screen program using your favourite package manager or download from the screen website.

     Simplest invocation of screen is by just typing the command "screen" on your terminal (local/remote).  You might be shown an optional welcome message, press "Enter" to get rid of the message and drop tot he terminal.  This will start the first virtual terminal session and will look like a regular terminal.  You can run any commands/programs you want from this terminal.  You can finish your work on the terminal and type exit or press Control+D as you do on a regular terminal to exit out of screen.  But this do not add any value to our terminal usage. 

     To make the most out of screen, one should know how to spawn additional virtual terminals, detach the screen session allowing the running program to continue independently and browse through the numerous virtual terminals you open.

     To do all these task you have to sent special commands otherwise known as "key bindings" to the screen application, and all of them start with an escape sequence, followed by one or more characters.  The default escape sequence is "Control+a" ( press control key followed by character a ).  By default screen will pass any input to the current virtual terminal, but sending an escape sequence will make screen aware that it is a special command.  Let us see some useful screen commands/key bindings.

   Control+a c ( Press "Control", then "a" followed by "c" ) - Open up a new virtual terminal ( hereafter called "VT")  in screen. 

     You can create any number of VTs in this fashion, each screen will have a number starting from zero, zero being the number of the VT opened when screen is started.

Control+a " - List all VTs. 

    An example screen session with 13 VTs ( 0-13) is given below.  There will be a selection bar on the VT you are currently on.  You can use up/down arrow keys to select a different terminal or press the terminal number to select it ( this work only for terminals from 0-9 ).  Pressing "Enter" will activate the selected VT.

Num Name

 0  bash
 1  bash
 2  bash
 3  bash
 4  bash
 5  bash
 6  bash
 7  bash
 8  bash
 9  bash
10  bash
11  bash
12  bash

Navigation commands

Control+a p - Move to previous VT ( "Control+a Backspace" can also be used )
Control+a n - Move to next VT ( "Control+a Space" can also be used )
Control+a 0-9 - Move to the numbered VT ( works only for terminals 0-9 )
Control+a Control+a - Toggle between current and previous window

Screen Command Prompt

Screen has a command prompt from which you can input commands for screen.

Control+a : - Activate the command prompt for the current VT

Providing "title TITLE_NAME" from the screen command prompt will rename the current VT (same as control+a A )

Typing help at the prompt will show you a full list of helpful commands.

Usability commands

Control+a C - Clear screen
Control+a Esc OR Control+a [ - Activate copy mode ( in copy mode you can scroll up the output , select and copy text).
Enter to copy, Control+a ] to paste
Control+a ? - Show help/key bindings
Control+a w - Show a list of virtual terminals ( displayed at the bottom of current VT, no option to select VTs )
Control+a M - Monitor for activity
control+a _ - Monitor for silence more than 30 seconds

Control+a A - Rename the current VT. 
    
     As you can see, each of these VTs have a number as well as a name.  While the number increases for each VT, the name by default is the name of the shell/command spawned.  While the numbers are convenient, you might lose track of which terminal is doing what.  Giving it a meaningful name would help you identify the VT.  When this command is given, you will be given a prompt with the default name already in it, erase it with backspace, type in your new name and press enter.

A sample screen session with three renamed VTs

Num Name

 0  Mutt Email Client
 1  Perl Scripting
 2  Screen 3


Detach / Attach

   Now that you know how to operate within a screen session, let us see how to detach/attach to en existing screen session.

Detach : You are running programs under multiple VTs inside a screen session.  Now you want to disconnect from this screen for some reason ( logging out, terminating remote ssh connection etc.. ) and still let the programs under the VTs to continue.  This is where you should use the detach option, which is activated by the key binding "Catrl+a d".  When the key combination is pressed, screen drops you off to the shell from where you initiated the screen session.  Now you can logout / disconnect and the programs will keep running.

This is the sample output you get when you detach from a screen session.
[detached from 9108.maverick]

Now to connect back, you should know the screen session's name ( do not confuse this with the VT names inside the screen session ).  To list all current screen sessions, issue the command "screen -ls"

safeer@maverick:~$ screen -ls
There are screens on:
        9108.pts-0.maverick        (Sunday 20 March 2011 03:49:35  IST)    (Detached)
1 Socket in /var/run/screen/S-safeer.

There is only one screen session open right now and the name is 9108.pts-0.maverick.  The format of the screen name is [PID of screen].[TTY Numebr].[hostname].  This is the same name you saw when you where detatching the screen session.

To re attach to the disconnected session, use "screen -r <SCREEN NAME>", ie screen -r 9108.pts-0.maverick.

When re attaching the PID part is optional and can be reattached this way : "screen -r pts-0.maverick"

With a couple of screen sessions, it is easier to track which screen is what, but not always.  So we can custom name the screen sessions, for that start the screen session with "screen -R <screen name>".  This name replaces the [TTY].[hostname] part, but the pid part will remain.  Since PID is optional when reattaching the screen, you can provide just this screen name while reattaching.  See an

safeer@maverick:/var/run/screen$ screen -ls
There are screens on:  
        21454.SAFEER_TEST       (Sunday 20 March 2011 04:03:52  IST)    (Attached)
        9108.pts-0.maverick       (Sunday 20 March 2011 03:49:35  IST)    (Detached)
2 Sockets in /var/run/screen/S-safeer.


     You can re-attach to an attached session active in one terminal from another terminal after detaching it from the current terminal.

Either do it in two steps as

screen -d <SCREEN NAME>
screen -r <SCREEN NAME>

or together

screen -dr <SCREEN NAME>

Multi user mode

     Multi user mode is one speciality of screen where the same or different users in a system can share the same screen session.  You can control the access using ACLs, which allows restrictions based on usernames, VTs, commands and read/write/execute modes.

     To enable multi user mode, we will have to use the screen command prompt invoked by "Control+a :" which will give you a prompt.  Enter "multiuser on" in this prompt and multi user access will be enabled for this screen session.  Once enabled, this screen session can be accessed by other users from different terminals using the command "screen -x <screen name>"

     A complete discussion of all multi user mode is out of the scope of this article, so I will just list the commands that can be used inside screen command prompt.

multiuser on - Enable multi user mode.
display - Show currently connected users
acladd - add screen access for one or more users
acldel - revoke screen access for one or more users
aclchg - Add/modify permissions ( for commands,VTS,read/write/exec ) to one or more users

     To know more about screen, checkout the man page and the official website of screen.