Friday, December 18, 2009

Changing MAC address in linux

There are occasions when you may need to change your MAC (Media Access Control) address temporarily. This can be used:
1. While connected to a public wifi hot spot - it keeps your original MAC address anonymous.
2. To bypass a router/switch which has MAC address filtering.
3. To obtain a dhcp ip lease that is allocated based on MAC address.

And remember! MAC address is something that is soldered into your network adapter. You cant change it permanently; in other words, the change that you make is not persistent over reboots.

First let us find out the current MAC address of the Ethernet adapter in my machine

safeer@penguinpower:~$ ifconfig eth0|awk '/HWaddr/{print $1" "$5 }'
eth0 00:13:e4:5c:93:0a

Now I am going to change the MAC address of my Ethernet interface.

First, bring down the interface
safeer@penguinpower:~$ sudo ifconfig eth0 down

Then change the MAC addres to whatever you want
safeer@penguinpower:~$ sudo ifconfig eth0 hw ether 00:22:4d:b7:36:01

Bring up the interface
safeer@penguinpower:~$ sudo ifconfig eth0 up

Verify MAC address has changed.
safeer@penguinpower:~$ ifconfig eth0|awk '/HWaddr/{print $1" "$5 }'
eth0 00:22:4d:b7:36:01

The syntax of the command is "ifconfig [interface name] hw [encapsulation type] [MAC address] "

Every argument is self explanatory except the encapsulation type. This is "ether" (Ethernet) for both our Ethernet and wireless interfaces. For finding out encapsulation type for other interface types see ifconfig man pages. To see the encapsulation type of your network interfaces use the following command

safeer@penguinpower:~$ ifconfig -a|awk '/HWaddr/{split($3,enc,":"); print $1" "enc[2];}'
eth0 Ethernet
wlan0 Ethernet

Friday, December 11, 2009

GPRS connection in Linux using wvdial

If you have a mobile with GPRS enabled you can use it to connect your Linux Box to Internet. The program that we can use for this is wvdial. Most of the popular distributions will have it in its repository, if not, get it from here.

I am going to connect using:

Nokia E71
GSM based GPRS connection from Vodafone (India)
Kubuntu 9.04
WvDial 1.60

First connect your mobile phone with USB cable to your PC/Laptop. In the case of Nokia, four options will pop-up on the phone. PC Suite, Media Transfer, Mass Storage and Connect to PC to Web. Though I chose Connect PC to Web, it was not detected as a modem. So changed the mode to PC Suite on the phone, and then a modem was detected at /dev/ttyACM0. I got this message on dmesg - "ttyACM0: USB ACM device". Now we need to cerate a configuration file for wvdial. and save it to "/etc/wvdial-vodafone-gprs.conf" for the time being.

The file should start with:

[Dialer Defaults]

Add the basic settings for the modem.

Modem = /dev/ttyACM0
Modem Type = USB Modem
Baud = 460800

For any gprs connection the most important detail you need is the APN ( Access Point Name ) of the service provider, in the case of Vodafone, it is "portalnmms". This should go in the modem initialization commands as below:

Init1 = ATZ
Init2 = AT+CGDCONT=1,"IP","portalnmms"

Note that the APN is part of the Init2 command. Now we need to add the dialing phone number, username and password. This is same across all GSM providers (CDMA phones have different values).

Phone = *99#
Username = 0
Password = 0

Dirty Secret: GSM don't check the username and password for authentication, it just needs the entries to be present in the configuration - with some junk values.

Save the file and dial with wvdial command using /etc/wvdial-vodafone-gprs.conf as an argument:

safeer@penguinpower:~$ sudo wvdial -C /etc/wvdial-vodafone-gprs.conf

You will get a lot of output, look for the following lines ( the IP's shown may differ)


local IP address 10.16.25.127
remote IP address 10.6.6.6
primary DNS address 10.11.230.2
secondary DNS address 202.144.13.50

This indicate the modem is connected. At this point you will be able to contact your DNS server and resolve names. But you need to set proxy for your browser to start browsing. For Vodafone the proxy and the port are 10.10.1.100 and 9401. Set this on your browser and you can start browsing. When you are done, close the connection by pressing Control+C to terminate wvdial command.

If your connection is CDMA, the phone number will be #777 instead of *99#, and the username and password will be your mobile number ( without country code ). Also the Init2 string will be "ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0" (no APN name).

A few more things to note, depending on your phone, the modem device location may change, it can be /dev/ttyUSB0, /dev/ttySL0 etc... always check dmesg to see how it is detected - use command dmesg|tail after connecting your usb cable.

For good measure, you can add the following settings as well to the wvdial configuration:

Dial Command = ATDT
Stupid Mode = 1
ISDN = 0
New PPPD = yes

To find out your provider's GPRS settings, you can visit Manoj Tivari's blog . I got the Vodafone proxy details from this web site.

Monday, November 16, 2009

Truncating a file

There are situations when you want to adjust the size of a file rather than deleting it. Especially when the file is being used by a process and you want to reduce its size without interrupting the process. Removing a file and creating another file by the same name will not work in such cases as the inode number of the file will change. This happens mainly with log files, where you want to reduce the size of the log - to save space -but don't want to stop the process that is writing to the log.

BSD has an excellent tool for this - the "truncate" command. This command can be used either to reduce/increase the size of a file. Let us consider a live example.

safeer->vps1.safeer.in ~ $ du -hs mysql-slowquery.log
476M mysql-slowquery.log

To reduce the size to 300MB

safeer->vps1.safeer.in ~ $ truncate -s 300M mysql-slowquery.log
safeer->vps1.safeer.in ~ $ du -hs mysql-slowquery.log
300M mysql-slowquery.log

Instead of specifying the exact size, if you just want to free up a 100Mb of disk space, you can do it this way:

safeer->vps1.safeer.in ~ $ truncate -s -100M mysql-slowquery.log
safeer->vps1.safeer.in ~ $ du -hs mysql-slowquery.log
200M mysql-slowquery.log

When you truncate a file by this method, the data is lost from the end of the file.

Truncate also has an option to extend a file's size, but it do not allocate any space for that. So it is not so useful.

safeer->vps1.safeer.in ~ $ truncate -s +400M mysql-slowquery.log
safeer->vps1.safeer.in ~ $ du -hs mysql-slowquery.log
200M mysql-slowquery.log

Another way of specifying size for truncation is to give a reference file. Truncate will calculate the size of the referenced file and truncate the given file to its length.

safeer->vps1.safeer.in ~ $ du -hs httpd.log mysql-slowquery.log
153M httpd.log
200M mysql-slowquery.log
safeer->vps1.safeer.in ~ $ truncate -r httpd.log mysql-slowquery.log
safeer->vps1.safeer.in ~ $ du -hs httpd.log mysql-slowquery.log
153M httpd.log
153M mysql-slowquery.log

truncate do not change the inode of the file.

safeer->vps1.safeer.in ~ $ ls -i mysql-slowquery.log
48980316 mysql-slowquery.log
safeer->vps1.safeer.in ~ $ truncate -s -132M mysql-slowquery.log
safeer->vps1.safeer.in ~ $ ls -i mysql-slowquery.log
48980316 mysql-slowquery.log

As you can see, this is a very useful command, but unfortunately we don't have a similar tool in Linux. So we can use any of the following method to emulate some truncate features.

To empty a file (truncate -s 0 )

safeer@penguinpower:~$ du -hs mysql-slowquery.log
1.1G mysql-slowquery.log
safeer@penguinpower:~$ >mysql-slowquery.log
safeer@penguinpower:~$ du -hs mysql-slowquery.log
0 mysql-slowquery.log

For resizing to an exact size you can use the dd command.

safeer@penguinpower:~$ du -hs mysql-slowquery.log
747M mysql-slowquery.log
safeer@penguinpower:~$ dd if=/dev/null seek=705 bs=1M of=mysql-slowquery.log
0+0 records in
0+0 records out
0 bytes (0 B) copied, 5.1892e-05 s, 0.0 kB/s
safeer@penguinpower:~$ du -hs mysql-slowquery.log
706M mysql-slowquery.log

Here dd argument bs specifies the unit of file size where as seek will specify the number of units to which you want to resize the file. Technically, copying /dev/null to a file will erase the content in the destination file. But with the "seek" option we specify that "seek value" X "block size" data from the beginning of the file should be preserved and the rest of the data should be erased. So in the above example, file was reduced to ~705MB (705 * 1M).

The following commands are equivalent.

truncate -s 0 mysql-slowquery.log
>mysql-slowquery.log
cat /dev/null > mysql-slowquery.log
dd if=/dev/null of=mysql-slowquery.log

Sunday, September 27, 2009

ssh-agent and passwordless ssh

In addition to password authentication, ssh also provides public key authentication. Setup of public key authentication is described here . The summary of authentication is as follows:

* Generate a private key /public key pair:

ssh-keygen -t rsa

This will create a private key named id_rsa and public key named id_rsa.pub under ~/.ssh directory of the user. Here we are using RSA version 2 as the encryption protocol. Alternatively you can specify DSA ( -t dsa ) or RSA version 1 ( -t rsa1 ). If you want to create the key pair with an alternate name/location, specify it with -f option. The above command will prompt you for a passphrase for the private key. You can either give a passphrase or just press enter to have a blank passphrase (alternatively use -P '' option). The problem with having a blank passphrase is that anybody who has access to your private key file will be able to view and use it to impersonate your login. A passphrase for the private key prevents this from happening.

* Copy your public key to destination server and append it to ~/.ssh authorized_keys.

Once this is setup you can login to the destination server without using your account password. If you have provided a blank passphrase while generating public/private key you will be directly logged in. But if you have give a passphrase for you private key, you wil be prompted for that. While this gives a more secure way of logging in, for the end user it is just like replacing one password with another one. To get around this without compromising on security, we can use ssh-agent which will cache your password for repeated use.

ssh-agent is a key management program that can manage all our keys. We just need to unlock/decrypt our private keys - with the passphrase - once and add them into the agent, and for the rest of that session ssh-agent will automatically supply your key credentials whenever a new ssh connection is initiated.

Let us see the usage now.

* Start the agent.

[safeer@fcds ~]$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-EQayeR4473/agent.4473; export SSH_AUTH_SOCK;
SSH_AGENT_PID=4474; export SSH_AGENT_PID;
echo Agent pid 4474;

Now that the agent is running you need to add your keys to the agent.

[safeer@fcds ~]$ ssh-add
Enter passphrase for /home/safeer/.ssh/id_rsa:
Identity added: /home/safeer/.ssh/id_rsa (/home/safeer/.ssh/id_rsa)
Identity added: /home/safeer/.ssh/id_dsa (/home/safeer/.ssh/id_dsa)

here you will prompted to enter the pasphrace for your keys. if all your private keys have the same passphrase, you will be prompted only once.

To see the keys added to ssh:

[safeer@fcds ~]$ ssh-add -l
2048 20:83:de:3d:7c:5f:a4:32:c2:75:c9:12:f6:7f:f2:be /home/safeer/.ssh/id_rsa (RSA)
1024 7a:29:5a:73:6b:56:d0:0d:73:4e:70:67:39:99:80:c8 /home/safeer/.ssh/id_dsa (DSA)

Since the keys are added to ssh-agent you can use private keys without being prompted for passphrase throughout this session.

To remove a specific key from the agent

[safeer@fcds ~]$ ssh-add -d /home/safeer/.ssh/id_rsa
Identity removed: /home/safeer/.ssh/id_rsa (/home/safeer/.ssh/id_rsa.pub)

If you want to remove all the keys, just use "ssh-add -d"

Friday, September 18, 2009

Recovering your kjots books

For those who don't know about kjots, it is a note taking utility that comes with the KDE desktop environment. It allows you to organize your notes into Books. Each book will have pages where you can actually save your notes. It has formating options like font selection,color,alignment etc. It allows you to export your books and/or pages to text/html formats. There are many other handy function like bookmarking searching etc.. It is a very handy and lightweight application.

I was using this for quite some time and had two books full of notes. One fine morning my KDE crashed leaving my system in hung state. When it came up after the reboot, I found that all the notes in Kjots were gone leaving two blank books in it. So I was sitting there scratching my head, suddenly it came to my mind that Kjots has an automatic backup facility and decided to try my luck.

All kde applications store their per user configuration and user data inside the local kde directory under the respective user's home directory. In most of the cases it will be .kde directory under the users home directory. But in KDE4 this wil change to .kde4. So to determine the kde directory for your user account you need to run the following command.

safeer@cms-netbook:~$ kde-config --localprefix
/home/safeer/.kde/

The kjots data is located in the path share/apps/kjots under ~/.kde. Lets examine the contents of this directory:

safeer@cms-netbook:~/.kde/share/apps/kjots$ ls
Oj6443.book Oj6443.book.2 Oj6443.book.4 Oj6443.book.6~ xA4048.book xA4048.book.10~ xA4048.book.3~ xA4048.book.6~ xA4048.book.8~
Oj6443.book.1 Oj6443.book.2~ Oj6443.book.4~ Oj6443.book.7~ xA4048.book.1 xA4048.book.2 xA4048.book.4~ xA4048.book.7 xA4048.book.9
Oj6443.book.1~ Oj6443.book.3 Oj6443.book.5 Oj6443.book.8~ xA4048.book.1~ xA4048.book.2~ xA4048.book.5~ xA4048.book.7~ xA4048.book.9~
Oj6443.book.10~ Oj6443.book.3~ Oj6443.book.5~ Oj6443.book.9~ xA4048.book.10 xA4048.book.3 xA4048.book.6 xA4048.book.8

Kjots by default assigns a random filename to each of your kjots books. This will not be the same as the name you have assigned to the book. Kjots stores its books in xml format. If you want to know which file corresponds to which book, use this command. Here the name I gave to this kjots book is "TrainingNotes".

safeer@cms-netbook:~/.kde/share/apps/kjots$ grep -A 1 -m 1 KJotsBook Oj6443.book.6~|grep Title|cut -d '>' -f2|cut -d '<' -f1
TrainingNotes

Kjots takes backup of your books every time you close it. In addition to the original book, kjots will keep 10 most recent backups of each book. The backup and book naming scheme is as follows.

Book name convention: .book
Eg: Oj6443.book

Backup book name: .book.~ with .book.1~ being the most recent book.
Eg: Oj6443.book.3~
Most recent backup for the above book will have name Oj6443.book.1~

Now that you know about the kjots backup scheme, let us see how I recovered my kjots. First find out the good backup of each books.

safeer@cms-netbook:~/.kde/share/apps/kjots$ ls -l
total 576
-rw------- 1 safeer safeer 0 2009-09-10 16:14 Oj6443.book
-rw------- 1 safeer safeer 0 2009-09-24 18:39 Oj6443.book.1~
-rw------- 1 safeer safeer 19774 2009-09-10 16:52 Oj6443.book.10~
-rw------- 1 safeer safeer 0 2009-09-19 16:37 Oj6443.book.2~
-rw------- 1 safeer safeer 0 2009-09-18 18:16 Oj6443.book.3~
-rw------- 1 safeer safeer 48278 2009-09-17 21:45 Oj6443.book.4~
-rw------- 1 safeer safeer 48278 2009-09-17 21:21 Oj6443.book.5~
-rw------- 1 safeer safeer 48278 2009-09-17 21:21 Oj6443.book.6~
-rw------- 1 safeer safeer 48278 2009-09-16 17:15 Oj6443.book.7~
-rw------- 1 safeer safeer 48278 2009-09-16 17:02 Oj6443.book.8~
-rw------- 1 safeer safeer 19774 2009-09-16 16:59 Oj6443.book.9~
-rw------- 1 safeer safeer 0 2009-09-24 18:39 xA4048.book
-rw------- 1 safeer safeer 0 2009-09-24 18:39 xA4048.book.1~
-rw------- 1 safeer safeer 39850 2009-09-10 16:52 xA4048.book.10~
-rw------- 1 safeer safeer 0 2009-09-19 16:37 xA4048.book.2~
-rw------- 1 safeer safeer 0 2009-09-18 18:16 xA4048.book.3~
-rw------- 1 safeer safeer 42450 2009-09-17 21:45 xA4048.book.4~
-rw------- 1 safeer safeer 42450 2009-09-17 21:21 xA4048.book.5~
-rw------- 1 safeer safeer 42450 2009-09-17 21:21 xA4048.book.6~
-rw------- 1 safeer safeer 42450 2009-09-16 17:15 xA4048.book.7~
-rw------- 1 safeer safeer 39850 2009-09-16 17:02 xA4048.book.8~
-rw------- 1 safeer safeer 39850 2009-09-16 16:59 xA4048.book.9~

As you can see, the most recent good copy for both the books were backed up on 2009-09-17 21:45 (this should be the time when I closed the Kjots on the previous day). All files after this has zero size. So copying this backup files to the corresponding books will get my books back. Make sure to close Kjots before doing this.

safeer@cms-netbook:~/.kde/share/apps/kjots$ cp Oj6443.book.4~ Oj6443.book
safeer@cms-netbook:~/.kde/share/apps/kjots$ cp xA4048.book.4~ xA4048.book

Relaunched Kjots and my books were back with all the contents!

Monday, September 14, 2009

VB script to move files based on creation date

My friend had this issue where he was supposed to move thousands of files sitting in one directory to separate directories based on their creation time.  Right now this script will only check the month and year of creation.  But this can be easily extended to consider up to seconds in creation time.  Explanation for each step is given as comments itself.


SplitAndMove.vbs

'Declare all varibles in advance
Dim curPath,srcDir,dstDir,myDir,objFSO,objFolder,objFiles,objFile

'Create a file sytem object as a base for manipulating files and directories
Set objFSO = CreateObject("Scripting.FileSystemObject")

'Process command line arguments
Select Case WScript.Arguments.Count
'If only one argument is given, take it as source directory
'then create a destination directory in the current working directory
       Case 1
       srcDir = WScript.Arguments.Item(0)
       curPath = objFSO.GetAbsolutePathName(".")
     dstDir = curPath & "\Split_Move_" & Day(Now()) & Month(Now()) & Year(Now()) & Hour(Now()) & Minute(Now()) & Second(Now())
       Case 2
'Two arguments given, first one is source directory
'Second one is destination directory
       srcDir = WScript.Arguments.Item(0)
     dstDir = WScript.Arguments.Item(1)
'In all other cases display Usage and exit script
       Case Else
       Wscript.Echo "Usage: " & Wscript.ScriptName & "<Source Directory> [<Destination Directory>]"
       WScript.Quit 1
End Select

'If source directory does not exist, display error message and exit script
If ( Not objFSO.FolderExists(srcDir) ) Then
   Wscript.Echo "Invalid Source Directory"
   WScript.Quit 1
End If

'If destination directory does not exist, create it.
If Not objFSO.FolderExists(dstDir) Then
   objFSO.CreateFolder(dstDir)
End If

'Get list of files in source directory
Set objFolder = objFSO.getFolder( srcDir )
Set objFiles = objFolder.files

'For each file in source directory, destination should be:
'a directory under destination directory,
'that directory's name should constitute month and year of the file's creation time.
For Each objFile in objFiles
'Construct destination directory name with month and year of file's creation time
myDir = dstDir & "\" & Month(objFile.DateCreated) & "_" & Year(objFile.DateCreated)
'If directory exists, move the file there.
If objFSO.FolderExists(myDir) Then
    objFSO.MoveFile objFile.Path , myDir & "\" & objFile.Name
'Otherwise create the directory and then move the file there.
Else
   objFSO.CreateFolder(myDir)
   objFSO.MoveFile objFile.Path , myDir & "\" & objFile.Name
End If
Next



Saturday, July 25, 2009

Dashes and asterisks in linux filenames

This question was asked in an interview: "How do you delete a file that contain a leading dash/hyphen (-) in its filename ? ".

Well, in real time we rarely encounter such a scenario. But the fact is that dash is allowed to be used in a filename , but we don't usually use it at the beginning of the name. Now the problem with such a filename is that if you try to remove the file with rm command, rm will assume that the letters following the dash are command line options for the command and will throw a syntax either about a wrong option or missing operand. So the solution is to "escape" the dash at the beginning, but how can we do that? The first option is to use "--" which indicates that there are no more command line options to be read and the arguments following it should be considered as operands.

Let us try with an example;

safeer@enjoyfast-lx:~/temp$ ls
-file1
safeer@enjoyfast-lx:~/temp$ rm -file1
rm: invalid option -- 'l'
Try `rm ./-file1' to remove the file `-file1'.
Try `rm --help' for more information.

As you can see it is giving an invalid option error. Let us use the option terminator.

safeer@enjoyfast-lx:~/temp$ rm -- -file1
safeer@enjoyfast-lx:~/temp$ ls
safeer@enjoyfast-lx:~/temp$

By the way, did you notice the second line in the error message given above? Yeah, rm itself is suggesting a solution for your problem, use "./" . Giving an absolute or relative path before the filename prevents rm from interpreting the filename as an option to the command. Let us try it.

safeer@enjoyfast-lx:~/temp$ ls
-file1
safeer@enjoyfast-lx:~/temp$ rm ./-file1
safeer@enjoyfast-lx:~/temp$ ls
safeer@enjoyfast-lx:~/temp$

Now that solves the problem, but how do we create a a file with dash in name at the first place? We will have the same problem as rm with any command for creating a file, like touch,vi etc. We can use the same tricks as above here

safeer@enjoyfast-lx:~/temp$ touch -file1
touch: invalid option -- 'i'
Try `touch --help' for more information.
safeer@enjoyfast-lx:~/temp$ touch -- -file3
safeer@enjoyfast-lx:~/temp$ touch ./-file4
safeer@enjoyfast-lx:~/temp$ ls -l
total 0
-rw-r--r-- 1 safeer safeer 0 2009-07-25 00:01 -file3
-rw-r--r-- 1 safeer safeer 0 2009-07-25 00:01 -file4

Okay, now you have two files, how ill you delete them together? Let us try wild-card

safeer@enjoyfast-lx:~/temp$ rm *
rm: invalid option -- 'l'
Try `rm ./-file3' to remove the file `-file3'.
Try `rm --help' for more information.

This is because "*" expands to "-file3 -file4".So use the same trick again...

safeer@enjoyfast-lx:~/temp$ rm ./*
safeer@enjoyfast-lx:~/temp$ ls
safeer@enjoyfast-lx:~/temp$
safeer@enjoyfast-lx:~/temp$ touch ./-file{3,4}
safeer@enjoyfast-lx:~/temp$ ls
-file3 -file4
safeer@enjoyfast-lx:~/temp$ rm -- *
safeer@enjoyfast-lx:~/temp$ ls
safeer@enjoyfast-lx:~/temp$

Okay, enough fun with dash. Let us try asterisk ( * ) now.

Create a file with name "*" first.

safeer@enjoyfast-lx:~/temp$ touch *
safeer@enjoyfast-lx:~/temp$ ls
*
safeer@enjoyfast-lx:~/temp$ echo "Star" > *;echo "First" > file1;echo "Second" > file2
safeer@enjoyfast-lx:~/temp$ ls
* file1 file2

Now let us print the content of all the files.

safeer@enjoyfast-lx:~/temp$ cat *
Star
First
Second

safeer@enjoyfast-lx:~/temp$ cat * file1 file2
Star
First
Second
First
Second

In both the above cases * expands to the file list "* file1 file2". Now to print the contents of the file "*" let us try the same mechanisms.

safeer@enjoyfast-lx:~/temp$ cat -- *
Start
First
Second
safeer@enjoyfast-lx:~/temp$ cat ./*
Start
First
Second

It is not working because * will always expand unless it is escaped.

safeer@enjoyfast-lx:~/temp$ cat \*
Start

You can delete files the same way

safeer@enjoyfast-lx:~/temp$ rm \*
safeer@enjoyfast-lx:~/temp$ ls
file1 file2

Thursday, June 25, 2009

Bash Tricks - Variable variable name

There are rare situations where you will have to use the value of a variable as another variable name, bash provides a special construct for this purpose.

Create a variable and assign a value

safeer@penguin-power:~/temp$ MY_VAR1="MY_VALUE1"

Assign the variable name "MY_VAR1" to MY_VAR2.

safeer@penguin-power:~/temp$ MY_VAR2="MY_VAR1"

Now use MY_VAR2 to print the contents of MY_VAR1

safeer@penguin-power:~/temp$ echo ${!MY_VAR2}
MY_VALUE1

Let us try with an example where command line arguments ( $1,$2,$3 etc.... ) of a script are parsed using the argument numbers (1,2,3...) taken from another variable.

safeer@penguin-power:~/temp$ cat test.sh
#!/bin/bash
for((i=1;i<=$#;i++))
do
echo ${!i};
done

safeer@penguin-power:~/temp$ ./test.sh first second third fourth
first
second
third
fourth

safeer@penguin-power:~/temp$ ./test.sh first
first

Monday, June 15, 2009

Working with files in perl

Opening a file for Read/Write/Append

You can open a file using the function open(). It takes three arguments: a file handle which is just a symbolic name of your choice for the file, a mode symbol which specifies the file operation mode in which the file should be opened, and third being the file to be opened itself.

Mode operators are as given below.

< : Read - Default if a mode is not specified at all.
> : Write -File is Created if not existing else Overwrites.
>> : Append - creates file if not existing.

If you want to combine read and write, prepend a "+" to the modes.

+< : Read/Write,Don't create file,Overwrites existing.
+> : Read/Write,Create file if not existing, else Overwrite.
+>> : Read/Append,Create file if not existing, no Overwrite

Let us take an example.

open ( MY_FILE_HANDLE,"<","/home/safeer/docs/accounts.txt");

This will open the file "/home/safeer/docs/accounts.txt" for reading and will be accessible through the file handle MY_FILE_HANDLE. We can use this handle to read data from the actual file.

The second and third arguments for open() can be combined into a single string as "< /home/safeer/docs/accounts.txt". In the case of mode, it is OK even if you skip the "<" symbol - since read is the default mode. So,

open ( MY_FILE_HANDLE,"<","/home/safeer/docs/accounts.txt");

open ( MY_FILE_HANDLE,"< /home/safeer/docs/accounts.txt");
open ( MY_FILE_HANDLE,"/home/safeer/docs/accounts.txt");

are all the same.

If you want to open this file for writing, use

open ( MY_FILE_HANDLE,">","/home/safeer/docs/accounts.txt"); OR
open ( MY_FILE_HANDLE,"> /home/safeer/docs/accounts.txt");

Reading data from a file

Open the file in read mode

open ( MY_FILE_HANDLE,"< /home/safeer/docs/accounts.txt");

You can assign the the entire file content to an array variable where each element of the array will correspond to a single line in the file.

@acounts_data = <MY_FILE_HANDLE> ;

But this can be memory intensive if the file you are opening is very large. A better way is to read file line by line and process it as follows.

while ( $line = <MY_FILE_HANDLE> ){
#Process $line here
}

In every iteration $line will contain a single line of the file. Note that the last character of the $line will be newline (\n), so you may want to chomp $line before processing it. The following code will be equivalent to printing the file contents.

while ( $line = <MY_FILE_HANDLE> ){
chomp ($line);
print"$line\n";
}

Write/Append to a file:

We use print command to write/append (depends on the mode in which the file is opened) to a file.

Syntax: print $FILEHANDLE $STRING;

To start writing, first open file in write mode and then print using the file handle obtained from open().

open ( MY_FILE_HANDLE,"> /home/safeer/docs/newaccounts.txt");
$my_account = "safeer:/home/safeer/:usr/bin/bash" ;
print MY_FILE_HANDLE "$my_account\n";

Alternatively, write directly without using variable:

print MY_FILE_HANDLE "safeer:/home/safeer/:usr/bin/bash\n" ;

If the file was opened in append mode ( ">>" ) the string would have been appended to the end of file.

open ( MY_FILE_HANDLE,">> /home/safeer/docs/accounts.txt");
print MY_FILE_HANDLE "safeer:/home/safeer/:usr/bin/bash\n" ;

Closing the file

any file that is opened should be closed once the purpose of open is over.
close (MY_FILE_HANDLE);

Monday, June 1, 2009

Extracting .CAB files in Linux

First of all, what is .cab files?

From Wikipedia:

CAB is the Microsoft Windows native compressed archive format. It supports compression and digital signing, and is used in a variety of Microsoft installation engines: Setup API, Device Installer, AdvPack (for the installation of ActiveX components from Internet Explorer) and Windows Installer.

It was originally called Diamond. Its .CAB file extension and overall file format name comes from word cabinet, internally used by Microsoft tools dealing with CAB files to reference such files.

I got a couple of resume templates from Microsoft Web Site, but the downloaded copies where in .cab format. So googled to find a way to extract them and learned that there is something called "cabextract".

On my Ubuntu box:

safeer@penguinpower:~/Desktop/PPTS$ sudo apt-cache search cabextract
cabextract - a program to extract Microsoft Cabinet files
liborange-dev - development libraries for liborange
liborange0 - library to extract CAB files from self-extracting installers
orange - extracts CAB files from self-extracting installers
safeer@penguinpower:~/Desktop/PPTS$ sudo apt-get install cabextract

RPM based distros can get the package from Dag Wieers' Repositories (if they don't have a native package).

And then, extracted the cab files without errors.

safeer@penguinpower:~/Desktop/PPTS$
cabextract 06199720.cab
06199720.cab: WARNING; possible 6960 extra bytes at end of file.
Extracting cabinet: 06199720.cab
extracting 06199720.dot
extracting content.inf

All done, no errors.
safeer@penguinpower:~/Desktop/PPTS$ cabextract 10168650.cab
10168650.cab: WARNING; possible 1024 extra bytes at end of file.
Extracting cabinet: 10168650.cab
extracting 10168650.dotx
extracting content.inf

All done, no errors.

The extracted files where in good shape and I was able to open them in Open Office.

Saturday, May 16, 2009

Manipulating the bash directory stack

There are occasions when you traverse through so many directories stopping in each directory to work on something. After a few stops you may want to go back to one or all of the direcoties you have traversed. I can't remember a generic example to give you but we all come across this scenario many times. Bash provides a facility for addressing such issues and that is called the "bash directory stack"

In simplest terms directory stack is a list of recently visited directories, like your browser's favorite list. It acts like a conventional stack where you can push and pop items, but you can manipulate it further to reorder and remove items from the middle of the stack. Bash provides three 'directory stack built-in' commands to manipulate this stack, wiz pushd,popd and dirs.

pushd:

With a directory as argument, pushd will save(push) the current directory to the stack and change to the directory given by the argument.

popd:

Without any arguments, popd will remove the top entry from the stack and move to that directory.

dirs:

Displays the stack entries. With optional arguments, it can display selected directories from the stack.


Example:

Am going to add a few directories to the stack first.

safeer@penguinpower:~/bin$ pushd /etc/apache2/
/etc/apache2 ~/bin
safeer@penguinpower:/etc/apache2$ pushd /etc/cvsd
/etc/cvsd /etc/apache2 ~/bin
safeer@penguinpower:/etc/cvsd$ pushd /etc/cups/
/etc/cups /etc/cvsd /etc/apache2 ~/bin
safeer@penguinpower:/etc/cups$ pushd /etc/ldap/
/etc/ldap /etc/cups /etc/cvsd /etc/apache2 ~/bin

In practice pusdhd is done after working in a directory and then moving to another directory. Now let us see the stack contents using dirs command.

safeer@penguinpower:/etc/ldap$ dirs -l -v
0 /etc/ldap
1 /etc/cups
2 /etc/cvsd
3 /etc/apache2
4 /home/safeer/bin

The number associated with each directory shows the index of the corresponding directory in the stack with index zero being the present working directory.

Now let us try moving through the stack with popd.

safeer@penguinpower:/etc/ldap$ popd
/etc/cups /etc/cvsd /etc/apache2 ~/bin
safeer@penguinpower:/etc/cups$ popd
/etc/cvsd /etc/apache2 ~/bin
safeer@enjoyfast-lx:/etc/cvsd$

So I popped two directories out of the stack and now the contents of the stack becomes:

safeer@penguinpower:/etc/cvsd$ dirs -v -l
0 /etc/cvsd
1 /etc/apache2
2 /home/safeer/bin

This will give you a basic idea about stack manipulation. To see the full options for pushd/popd/dirs visit the GNU page for directory stack built-ins
A few more shortcuts and shell variables exists for manipulating the stack.

"$PWD" OR "$DIRSTACK" OR "~+" - Present working directory - Stack index zero
"$OLDPWD" OR "~-" - Previous working directory - Stack index one.
"$HOME" OR "~" - User's home directory

cd : Move to user's home directory. Alternatively use cd ~.
cd - : Move to $OLDPWD - pushd without any arguments will do the same. Or use cd ~-

Examples

safeer@penguinpower:/etc/cvsd$ echo ~{,-,+}
/home/safeer /etc/apache2 /etc/cvsd

which is equivalent to:

safeer@penguinpower:/etc/cvsd$ echo $HOME $OLDPWD $PWD
/home/safeer /etc/apache2 /etc/cvsd

Thursday, May 7, 2009

Cisco kron - the Cisco job scheduler

Cisco provides a job scheduler called "kron" which is similar to the unix crontab service. It allows us to schedule and run certain exec mode commands once or recurringly. The feature was first introduced in IOS version 12.3(1). The prerequisite for using kron is that Cisco clock time must be set - either locally or using an NTP source.

Check your IOS version first:

R1_Balancer#show version | include ^Cisco IOS(.+)Version
Cisco IOS Software, 3700 Software (C3725-ADVENTERPRISEK9-M), Version 12.4(18), RELEASE SOFTWARE (fc1)

The "kron" configuration has two parts, configuring "Scheduler Policy List" and then setting "Scheduler Occurrences". A scheduler policy list is similar to a shell script in unix. Every policy list will have a name and will contain one or more exec cli commands added sequentially. A scheduler occurrence is the setting that tells you when a policy list should be executed. You can associate one or more policy lists to a policy schedule.

Now let us create a kron policy with policy name my_policy_1. The commands that can be added in a policy should not generate any prompt or have the ability to be terminated using keystrokes. The simplest command that I can remember is write, which will save the running config to startup config( you can't use "copy running-config startup-config" since it will ask you to confirm the destination file name )

R1_Balancer(config)#kron policy-list my_policy_1
R1_Balancer(config-kron-policy)#cli write
R1_Balancer(config-kron-policy)#exit

You can view the policy from running configuration.

R1_Balancer#sh run | section include kron
kron policy-list my_policy_1
cli write

Let us create a schedule called schedule_1 now. This is set to run once at next Sunday night 11.45

R1_Balancer(config)#kron occurrence schedule_1 at 23:45 sun oneshot
Clock currently not set it reads 02:58:44 UTC Fri Mar 1 2002

If the clock is not set properly you will get a message as above. So set the time.

R1_Balancer#clock set 22:14:20 may 10 2009
R1_Balancer#show clock
22:17:29.487 UTC Sun May 10 2009

R1_Balancer(config)#kron occurrence schedule_1 at 23:45 sun oneshot
R1_Balancer(config-kron-occurrence)#policy-list my_policy_1
R1_Balancer(config-kron-occurrence)#exit

In the schedule you can optionally specify a user under which the kron should run. Now let us see the full kron configurations.

R1_Balancer#show run | section include kron
kron occurrence schedule_1 at 23:45 Sun oneshot
policy-list my_policy_1
kron policy-list my_policy_1
cli write

To view the schedule you can run this command:

R1_Balancer#show kron schedule
Kron Occurrence Schedule
schedule_1 inactive, will run once in 0 days 01:24:51 at 23:45 on Sun

For detailed information on kron, visit this Cisco link.

Sunday, May 3, 2009

Connect GNS3 to your Linux desktop

This post assumes the reader has basic working knowledge with GNS3 GUI. For those who don't know about GNS3 - the graphical network simulator for Cisco platforms, please visit http://www.gns3.net/.

On windows GNS comes with a script for connecting to local machine, but it don't have a Linux equivalent. Here we will see how to achieve that.

First of all, create a basic network map with a single router. Here I am adding a 3600 series router with two slots, NM-1FE-TX ( one fast ethernet port) and NM-4E (four ethernet ports ). Once added, start the router and go to exec mode to list the router interfaces.

Router(config)#hostname R0
R0(config)#show ip interface brief
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 unassigned YES unset administratively down down
Ethernet1/0 unassigned YES unset administratively down down
Ethernet1/1 unassigned YES unset administratively down down
Ethernet1/2 unassigned YES unset administratively down down
Ethernet1/3 unassigned YES unset administratively down down

Now we need to connect any one of these interfaces to the local desktop. To do this, from the GUI save the map first. I am going to save my router to /home/safeer/GNS/local-connect.net. Now let us examine the contents of the saved network file.

safeer@penguinepower:~$ cat ~/GNS/local-connect.net
autostart = False
[localhost:7200]
workingdir = /tmp
[[3620]]
image = /home/safeer/.gns3/CiscoImages/c3620-i-mz.121-5.T92.bin
idlepc = 0x6012c108
ghostios = True
chassis = 3620
[[ROUTER R0]]
model = 3620
console = 2000
slot0 = NM-1FE-TX
slot1 = NM-4E
x = -29.0
y = -266.0


Now we need to connect this to the local machine using a virtual network interface. To create a virtual network adapter in linux, you need User Mode Linux. On my Ubuntu box, I can get it with apt-get.

safeer@penguinepower:~$ sudo apt-get install uml-utilities

On a Redhat based machines, you can get it via Yum. For more details visit the UML home page http://user-mode-linux.sourceforge.net/

Now add a virtual interface with name tap0 and assign an ip 172.16.1.1 to it.

safeer@penguinepower:~$ sudo tunctl -t tap0 -u safeer
Set 'tap0' persistent and owned by uid 1000
safeer@penguinepower:~$ sudo ifconfig tap0 172.16.1.1 up
safeer@penguinepower:~$ ifconfig tap0
tap0 Link encap:Ethernet HWaddr 72:e7:97:e6:e4:2c
inet addr:172.168.1.1 Bcast:172.168.255.255 Mask:255.255.0.0
inet6 addr: fe80::70e7:97ff:fee6:e42c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:19 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

Now we going to connect this tap0 interface to router R0's fast ethernet 0/0 interface. For this, Close the network map in GUI and open its previously saved network file and add this line to the end of [ROUTER R0] section in the configuration.

f0/0 = nio_tap:tap0

The file will look like:

safeer@penguinepower:~$ cat ~/GNS/local-connect.net
autostart = False
[localhost:7200]
workingdir = /tmp
[[3620]]
image = /home/safeer/.gns3/CiscoImages/c3620-i-mz.121-5.T92.bin
idlepc = 0x6012c108
ghostios = True
chassis = 3620
[[ROUTER R0]]
model = 3620
console = 2000
slot0 = NM-1FE-TX
slot1 = NM-4E
x = -29.0
y = -266.0
f0/0 = nio_tap:tap0

Now save the file and open the network map in GUI. You will see a cloud connected to the f0/0 interface of the router. Now tap0 interface of the host machine is connected to f0/0 of R0 in GNS3. To complete the configuration, start R0 and configure an IP for f0/0 that belong to the same network as tap0.

R0(config)#interface f0/0
R0(config-if)#ip address 172.16.1.2 255.255.0.0
R0(config-if)#no shutdown

Now try pinging the local machine from R0.

R0#ping 172.16.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.1.1, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 4/4/4 ms

That is all, you can access R0 ( and other routers connected to R0) from your local machine now. If you have enabled ip forwarding/routing on your local box, these routers will be accessible in your local network.

Note: Sometimes GNS may have permission issue in accessing tap0 interface, in that case launch GNS3 as sudo.

Saturday, April 11, 2009

FTP macros for automated ftp tasks

FTP macros are nothing but an ordered set of ftp commands grouped with a macro name, similar to a bash function/script. You can use the macro name like another command and execute the commands included with it. You can add a macro while inside an ftp session using the macdef command. Alternatively you can add it into the .netrc file inside your home directory. The usage of .netrc file is explained here.

Let us try creating a macro inside an ftp session. This macro will just change to a directory inside the ftp root and copy a file to local directory.

Each macro starts with a macro name on first line, followed by single ftp command on each line. The macro termination is given by a blank line.

ftp> macdef getmacro
Enter macro line by line, terminating it with a null line
cd /home/safeer/pub/site_backup/
get webdata.tar.gz



Once you create the macro run it from the ftp prompt with $ prepended to the macro name.
ftp> $getmacro
cd /home/safeer/pub/site_backup/
250 Directory successfully changed.
get webdata.tar.gz
local: webdata.tar.gz remote: webdata.tar.gz
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for webdata.tar.gz (0 bytes).
226 File send OK.


A macro defined in an ftp session remains active till the close command is issued. If you want re-usable macros, you need to add them to your netrc file. The above mentioned link has the details on how to set it up. A macro defined by the name "init" (in netrc file) will be run every time you login to the corresponding ftp host.

Macro Arguments


If you want to take arguments inside your macro, use $1,$2,$3 etc... where $1 will be the first argument, $2 will be the second argument and so on. If you want to iterate through each argument of the macro, use the special variable $i. The characters "$" and "\" are special in macros and need to be escaped with "\" if you want them "as it is" in the macros.

Eg:

ftp>macdef 2args
Enter macro line by line, terminating it with a null line
cd /home/safeer/docs
get $1
put $2


Above macro will now download local.txt and upload remote.txt

ftp>$2args local.txt remote.txt
cd /home/safeer/docs
250 Directory successfully changed.
get local.txt
local: local.txt remote: file1.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for local.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (91.1 kB/s)
put remote.txt
local: remote.txt remote: remote.txt
200 PORT command successful. Consider using PASV.
150 Ok to send data.
226 File receive OK.
47 bytes sent in 0.00 secs (454.4 kB/s)


ftp>macdef getmulti
Enter macro line by line, terminating it with a null line
cd /home/safeer/docs
get $i


Above macro will now change to the directory docs in ftp root and recursively download the files given as arguments to macro

ftp> $getmulti file1.txt file2.txt file3.txt
cd /home/safeer/docs
250 Directory successfully changed.
get file1.txt
local: file1.txt remote: file1.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for file1.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (82.3 kB/s)
cd /home/safeer/docs
250 Directory successfully changed.
get file2.txt
local: file2.txt remote: file2.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for file2.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (118.6 kB/s)
cd /home/safeer/docs
250 Directory successfully changed.
get file3.txt
local: file3.txt remote: file3.txt
200 PORT command successful. Consider using PASV.
150 Opening BINARY mode data connection for file3.txt (47 bytes).
226 File send OK.
47 bytes received in 0.00 secs (154.5 kB/s)

There is an maximum 8 character limit for macro names defined on command line. In netrc file this restriction is not there. Per ftp session, there is a limit of 16 macros and 4096 total characters in all defined macros.

Automating ftp logins with netrc

If you are a regular ftp user, then netrc file can help you a lot in automating your logins and a few tasks you do via ftp.

Let us first consider automating ftp logins.

Create a ".netrc" file in your home directory. The access to this file should be restricted exclusively to you.

cd ~;touch .netrc;chmod u+rw,go= .netrc

Now assume you need to connect to an ftp server "myftp.safeer.in" with username 'safeer' and password 'password#123'. To automate this, add the following line to your .netrc file.

machine myftp.safeer.in login safeer password password#123

Here machine defines the host you want to login to, login is your username for the host and password is the password for this username.

safeer@my-lptp01:~$ ftp myftp.safeer.in
Connected to myftp.safeer.in.
220 (vsFTPd 2.0.7)
331 Please specify the password.
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>


You can define a line for each of the host you want to login. If you use the same login for many of the ftp hosts, you can use the default entry like this.

default login safeer password password#123

With this entry, if you connect to an ftp host which is not explicitly defined in netrc with a "machine" entry, the login and password from the default line will be used. If you want to disable the autologin feature of netrc while running ftp program, use the option "-n", ie; "ftp -n [ftphost]"

safeer@my-lptp01:~$ ftp -n myftp.safeer.in
Connected to myftp.safeer.in.
220 (vsFTPd 2.0.7)
ftp> pwd
530 Please login with USER and PASS.


Thats all about auto logins, now let us see how to automate tasks. We use ftp "macros" for this. FTP macros are nothing but an ordered set of ftp commands grouped with a macro name, similar to a bash function/script. You can use the macro name like another command and execute the commands included with it.

Creating a macro with name "fmacro"

macdef fmacro
cd /home/safeer/pub/site_backup
put webdata.tar.gz


Now, run the macro:

ftp> $fmacro
cd /home/safeer/pub/site_backup
250 Directory successfully changed.
put webdata.tar.gz
local: webdata.tar.gz remote: webdata.tar.gz
200 PORT command successful. Consider using PASV.
150 Ok to send data.
226 File receive OK.

ftp>


A macro defined by the name "init" (in netrc file) will be run every time you login to the corresponding ftp host. Each macro definition is associated with the login entry preceding it and terminated by a blank line. You can't define global macros in netrc file. Macros associated with the" default" entry will work for all machines that are not explicitly defined in the file. To know more about ftp macros, visit this link.

For each "machine" line you can use one or more of the variables machine,login,password in order. ie: any of the following will work.

machine myftp.safeer.in login safeer password password#123 OR
machine myftp.safeer.in login safeer OR
machine myftp.safeer.in


The last option will be used only if you want to define some macros for this host, but don't want the autologin feature. While defining the default entry, make sure that it is defined as the last entry in the file. This is necessary as netrc is processed sequentially and accepts the first entry that matches the connecting host. For more information on netrc and macros, see man pages for netrc and ftp.

HT Password Manager

HT Password Manager is a web interface to manage apache's htpasswd files. Manage multiple password files with separate per-file administrator. Administrators can add/delete/search and reset password for all users & users can change their own passwords.


Find the Sourceforge project page here: HT Password Manager

Thursday, April 9, 2009

Running Subversion as Windows Service

Download the source package of subversion for windows from http://subversion.tigris.org
Unpack the Zip package to some Directory
Eg: C:\usr\svn-win32-1.4.5\

Now add the subversion binaries to system PATH. Binaries are located in “C:\usr\svn-win32-1.4.5\bin\“. To set this either you can go through the Advanced menu of system applet or use “set” command from command line.

Now you will be able to use the svn commands from command prompt. As the first step, create a root folder for all projects that will run under this server, say C:\svnroot. Now we need to run the svn server.

C:\Temp\>svnserve -d -r C:\svnroot

This will start svn server listening on default port 3690. To verify, start another command window
C:\Temp>netstat -ano|find “3690″
TCP 0.0.0.0:3690 0.0.0.0:0 LISTENING 2820
The process with PID 2820 is listening on port 3690, to verify that the process is svnserve,
C:\Temp>tasklist |find “svnserve.exe”
svnserve.exe 2820 Console 0 3,264 K

This confirms that snvserve is running and listening for connections on port 3690

This method has the drawback that the svnserve command is running in foreground which means,
*A command window will be running in the foreground all the time
*If the windows is closed or the server is rebooted the service will require to be manually started again.
*A user has to be logged in to the server and he cannot log off for another user to use the system.

So to make life easier, we will have to automate the whole thing. This means that the svnserver should be run as a windows service. Subversion has a Windows installer which will install itself as a service, bu I feel it is not that flexible and hence wanted to implement the serive in my way. Svnserve supports running as windows service with the command line switch –service. To make use of this feature we should use the Windows Service Control command line tool(sc.exe).

First, create the service with name SVNSVC
C:\Temp>sc create SVNSVC binpath= “C:\usr\svn-win32-1.4.5\bin\svnserve.exe –service -r C:\svnroot” displayname= “Subversion Server” depend= Tcpip
[SC] CreateService SUCCESS

Configure the service to start at boot time
C:\Temp>sc config svnsvc start= auto
[SC] ChangeServiceConfig SUCCESS

Start service
C:\Temp>net start svnsvc
The Subversion Server service is starting.
The Subversion Server service was started successfully.

If you mis configure the service option or run into problems, you can always remove the service from the system with the command
C:\Temp>sc delete svnsvc
[SC] DeleteService SUCCESS
and create a fresh service.

You can configure, start and stop services from the services MMC console in administrative tools instead of using the command line. But you can’t add or delete service.

Resetting MySQL root password



MySQL root user is the built in user with administrative privileges on MySQL server. Even though this is the default administrator user, you can add any number of new users with varying administrative rights on MySQL server. In many occasions system administrators tend to forget the MySQL root password, but this can be fixed with out much pain.

Stop the MySQL server. If you have start/stop service scripts installed you can do it that way:

[root@linuxbox1 ~]#service mysqld stop OR

[root@linuxbox1 ~]#/etc/rc.d/init.d/mysqld stop or whatever other script you use normally.

In some cases, this may fail if the server is overloaded, in such a case you will have to kill the mysql process

You can get the PID of the process using ps command:

[root@linuxbox1 ~]# ps aux|grep mysqld OR

looking in the mysql pid file ( found usually in MYSQL_DATA_DIRECTORY/mysql.pid or /var/run/mysqld/mysqld.pid MySQL data/installation directory is usually /var/lib/mysql ). Then kill the process

[root@linuxbox1 ~]#kill `cat /var/run/mysqld/mysqld.pid`

Now since the mysql server has stopped, start mysqld with the –skip-grant-tables option

[root@linuxbox1 ~]# /usr/bin/mysqld_safe –skip-grant-tables

Now set new root password

1) Using mysqladmin

[root@linuxbox1 ~]# mysqladmin -u root password ‘my_newpass#123

Note that you don.t need to provide a password to use mysqladmin. Now flush the privilege tables in mysql to take the effect of password change

[root@linuxbox1 ~]# mysqladmin -u root flush-privileges

2) Using mysql client

[root@linuxbox1 ~]# mysql -u root mysql

mysql> UPDATE user SET Password=PASSWORD(’my_newpass#123′) WHERE User=’root’;
mysql> FLUSH PRIVILEGES;

Password is reset now. Now you have to restart MySQL service. First kill the mysqld_safe process, use the ps command to get the pid of mysqld_safe. Start MySQL service as usual.

[root@linuxbox1 ~]#/etc/rc.d/init.d/mysqld start

Now you will be able to connect to MySQL with new password ‘my_newpass#123

[root@linuxbox1 ~]#mysql -u root -pmy_newpass#123

Wednesday, March 25, 2009

Virtual Box - Resetting Virtual Hard Drive UUID

I wanted to create two RHEL5 VMs with VBox. So installed and setup one virtual machine and then remembering what I used to do with MS Virtual Machine, copied the virtual hard disk of the newly created VM to another location. Then from the VBox GUI I tried to add a new virtual machine by pointing it into an existing hard disk ( which is the newly copied VHD). And then I got this pop-up message:

Failed to open the hard disk D:\VirtualMachines\V-Cent5-3\Cent5-3.vdi.
Cannot register the hard disk 'D:\VirtualMachines\V-Cent5-3\Cent5-3.vdi' with UUID {5eaa4ad6-2742-480b-88f1-364b82c63c3d} because a hard disk 'D:\VirtualMachines\V-Cent5-2\Cent5-2.vdi' with UUID {5eaa4ad6-2742-480b-88f1-364b82c63c3d} already exists in the media registry ('C:\Documents and Settings\safeer\.VirtualBox\VirtualBox.xml').


Result Code: E_INVALIDARG (0x80070057)
Component: VirtualBox
Interface: IVirtualBox {779264f4-65ed-48ed-be39-518ca549e296}


A quick look into the vbox's xml configuration file told me that it keeps a unique id for each hard disk, and this is set within the virtual hard disk. So I started looking for a solution. The first one that came across while googling was to rgenerate the UUID of the newly copied hard disk as follows.

C:\Program Files\Sun\xVM VirtualBox>VBoxManage internalcommands setvdiuuid "D:\VirtualMachines\V-Cent5-3\Cent5-3.vdi"
VirtualBox Command Line Management Interface Version 2.2.0
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.


UUID changed to: 5eaa4ad6-2742-480b-88f1-364b82c63c3d

I just did this and voila, VBox accepted my copied hard disk without complaints. :) That did the trick, but later in another forum post I found out a better method of cloning an existing VHD, it will not only copy the generated hard disk to a specified location, but will also generate a new UUID for the disk.


C:\Program Files\Sun\xVM VirtualBox>VBoxManage clonehd "D:\VirtualMachines\V-Cent5-3\Cent5-3.vdi" "D:\VirtualMachines\V-Cent5-3\Cent5-4.vdi"
VirtualBox Command Line Management Interface Version 2.2.0
(C) 2005-2009 Sun Microsystems, Inc.
All rights reserved.


0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone hard disk created in format 'VDI'. UUID: 7934dd80-0ce6-4530-b471-835af980190a


This looks more straight forward approach. But ofcourse the choice is yours.

Tuesday, March 24, 2009

Configuring NTP client for Cisco devices

Lot many things in Cisco devices depend on time, like certificates, time based ACLs and syslog messages. When it comes to time sources, Cisco devices are pretty much same as Computers. Most of them have a hardware clock as well as a software clock. We are concerned here only about the software clock as that is the time source used in the above mentioned scenarios.

We can manually set the time of hardware and software clocks in Cisco. To know about it, see my post about setting time in Cisco devices.

Here we would see how we can set an NTP server as the time source for a Cisco device.

R0(config)#ntp server 202.55.1.1

This will set the cisco device as a client of the NTP server 202.55.1.1, but there are a few more thing to configure if you want you network to be safe and efficient.

Authentication.

To prevent malicius/accidental setting of incorrect time from an unwanted NTP server, you should use NTP authentication using NTP "keys". There will be keys configured on the NTP server, which are basically string with a maximum length of 8 characters. The encryption scheme used to protect the key exchange is MD5.

Now enable NTP authentication first:

R0(config)#ntp authenticate
Now define the NTP server authentication key(s)
R0(config)#ntp authentication-key 1 md5 SeCrEt
R0(config)#ntp authentication-key 2 md5 TeRcEs

Here 1 and 2 are arbitrary numbers used as identifiers for each different keys. MD5 is the protocol used and the last argument of the command is the key itself ( This will be matching with the keys configured on the NTP server).

R0(config)#ntp trusted-key 1

This tells the device that the key "SeCrEt" that is stored locally is a trusted key which can be used for NTP authentication. Only trusted keys can be associated with an NTP server.

R0(config)#ntp server 202.55.1.1 key 1

This command associates the server 202.55.1.1 with the key "1" (SeCrEt).

This should be enough to set the authentication.

Reliability

What will happen if this NTP server goes down for a while? To deal with such a scenario you need additional NTP server which you can configure using the same command as above ( the key part may differ
though, depending on the key configured on that server).

So we introduce another server 202.55.2.1 which has and MD5 key "TeRcEs" configured.

R0(config)#ntp trusted-key 2
R0(config)#ntp server 202.55.2.2 key 2

Now we have a fail-over NTP server. But what if we need the NTP traffic to be served by 202.55.1.1 as long as it is up and running? and use the other one only if this host is down? You can specify a preferred server, which in this case will be 202.55.1.1

R0(config)#ntp server 202.55.1.1 key 1 prefer

This will make 202.55.1.1 preferred over the other NTP servers.

There are many more security features that you can configure to secure NTP, like enabling NTP messages to be received on selected interfaces and NTP access-group ACLs. I will detail them in another post.

Saturday, March 21, 2009

Perl Regular Expressions - Searching Array Without Loops

Iterating through arrays always require using of loops.  Array elements will be processed for many things but the most common one is searching for a particular element in an array.  If your purpose is to check whether a particular word is in the array elements, you can avoid using loops the following way.

The first thing to learn is a feature of perl arrays, ie;  if you enclose a perl array variable within double quotes it will be equivalent to a string containing all elements of the array separated by spaces.
Like this:

safeer@enjoyfast-lx:~/Scripts$ perl -e '@words=("tom","jerry","atom");print "@words\n"';
tom jerry atom

Now we have a string containing all the elements in the array.  All we need to do is match it against the search string.  If your seach term can be a substring of an array element,things are pretty easy.  You can directly match it agains the string converted array.
Like this:

if ( "$@words" =~ "jer" )
This will match the element jerry and the search will be successfull.  But if you are looking for an exact match of your search string with an array element, you will have to use a bit of regular expression techniques.  In this case we will have four different patterns.

1)  Array has multiple elements and the search string is the firrst element, in this case left hand side of the element will be begining of string ( denoted by "^") and the right hand side a space ( denoted by "\s").
2) Array has multiple elements and the search string is the last element, in this case left hand side of the element will be a space ( denoted by "\s") and  the right hand side be end of the string ( denoted by "$").
3) Array has multiple elements (>2)and the search string is somwhere in the middle of the string, in this case both side of the element will be space ("\s")
4) If the array has a single element, LHS of element will be start of string ("^") and RHS will be end of string ("$").

So a matching expression will look like:

if ( "@words" =~ /(^|\s)$searchterm(\s|$)/ )

Lets write a full script and try some options.  This script will accept search pattern from command line.

safeer@enjoyfast-lx:~/Scripts$ cat regex-search.pl
#!/usr/bin/perl -w
@words=("tom","jerry","atom");
if (defined $ARGV[0])
{
if ( "@words" =~ /(^|\s)$ARGV[0](\s|$)/)
{
print "Match!! \n";
}
else
{
print "No Match \n";
}
}
else
{
print "Provide a Search Term\n";
}

safeer@enjoyfast-lx:~/Scripts$ ./regex-search.pl tom
Match!!
safeer@enjoyfast-lx:~/Scripts$ ./regex-search.pl jerry
Match!!
safeer@enjoyfast-lx:~/Scripts$ ./regex-search.pl atom
Match!!
safeer@enjoyfast-lx:~/Scripts$ ./regex-search.pl jerr
No Match
safeer@enjoyfast-lx:~/Scripts$ ./regex-search.pl to
No Match


The flip side of this approach is that the search will fail if an array element has whitespace included in it.

Setting time in Cisco devices

Cisco devices have two clocks, one hardware and one software. By default the software clock is synced from the hardware one. But there are many things thing that we can modify for our environment.

To view the current time configured:

R0#show clock
*23:21:08.907 UTC Sun May 24 200

R0#show clock detail
*23:21:26.111 UTC Sun May 24 2009
Time source is hardware calendar

Set the software clock time:

R0#clock set 11:26:30 24 MAY 2009
R0#show clock
11:26:32.839 GMT Sun May 24 2009

To change your timezone:

R0(config)#clock timezone IST +5 30
R0(config)#do show clock
11:39:08.535 IST Sun May 24 2009

If you want to set the daylight savings, use command "clock summer-time" with correct options. See "clock summer-time ?"

To sync hardware and software clocks:

R0#clock read-calendar - Read the hardware calendar into the clock
R0#clock update-calendar - Update the hardware calendar from the clock

To view and set the hardware clock:

R0#show calendar
00:40:37 GMT Mon May 25 2009
R0#calendar set 11:45:30 24 MAY 2009

To set time via NTP, see my post about Cisco NTP client configuration