Registering DLLs with wine

If you want to add a DLL to your wine installation, you should register that dll just as you do in the windows OS's.

First, download the DLL and copy it to your wine installation's system32 directory :
safeer@linux-lp-01:~/downloads$ cp devenum.dll /home/safeer/.wine/drive_c/windows/system32/

Now register this DLL with your wine
safeer@linux-lp-01:~/downloads$ regsvr32 "C:\windows\system32\devenum.dll"
Successfully registered DLL C:\windows\system32\devenum.dll


Done. :)

Installing MSI files with wine

Wine is widely used to install windows applications in Linux. The common procedure to install a windows application using wine is to issue the command "wine /path/to/exe". For example:

safeer@linux-lp-01:~$/usr/bin/wine ~/downloads/MP10Setup.exe

But this will fail if the package is an MSI package.

safeer@linux-lp-01:~/downloads$ wine msxml6.msi
wine: could not load L"Z:\\home\\safeer\\downloads\\msxml6.msi": Bad EXE format for

To manage MSI files we have to use "msiexec"

safeer@linux-lp-01:~/downloads$ msiexec /i msxml6.msi

This will work, for more details on msiexec, try "/usr/bin/msiexec /h"

Firefox extension installation error

I was trying to install firebug on my FF2 and got this error:

"installLocation has no properties
file:///usr/local/firefox2/components/nsExtensionManager.js"

After searching the firefox bug tracker, found out the solution.

Go to your firefox profile directory, backup and remove the file "extensions.rdf". Restart firefox and you will be able to install the extensions without errors. But I am not sure about what will happen to your existing extensions, as I had no other extensions installed. So this is all I did:

safeer@myubuntu:~/.mozilla/firefox/qkvt609s.default$ mv extensions.rdf extensions.rdf-bak

safeer@myubuntu:~/.mozilla/firefox/qkvt609s.default$ pkill firefox

safeer@myubuntu:~/.mozilla/firefox/qkvt609s.default$ firefox&

and start installing your favorite extensions :)

Renaming files with space in name

Working on a file with spaces within its name is is always a headache in Linux. The best way is to rename them to eliminate spaces. For a normal user, even this is guaranteed to give pain if there are a lot of files. Here is the shortcut to do it:

I had a directory with songs that I ripped from a CD which had spaces in name. So i decided to replace the spaces with underscores.

safeer@enjoyfast-lx:~/Music/ONV-Shahbaz-Venugopal/Sahayathrikey$ ls -l
total 57284
-rw-r–r– 1 safeer safeer 6124954 2008-11-06 01:29 01 - Track 1.ogg
-rw-r–r– 1 safeer safeer 5647099 2008-11-06 01:29 02 - Track 2.ogg
-rw-r–r– 1 safeer safeer 5954106 2008-11-06 01:30 03 - Track 3.ogg
-rw-r–r– 1 safeer safeer 5783244 2008-11-06 01:30 04 - Track 4.ogg
-rw-r–r– 1 safeer safeer 5951001 2008-11-06 01:30 05 - Track 5.ogg
-rw-r–r– 1 safeer safeer 6212981 2008-11-06 01:31 06 - Track 6.ogg
-rw-r–r– 1 safeer safeer 5580947 2008-11-06 01:31 07 - Track 7.ogg
-rw-r–r– 1 safeer safeer 5519893 2008-11-06 01:31 08 - Track 8.ogg
-rw-r–r– 1 safeer safeer 5092362 2008-11-06 01:32 09 - Track 9.ogg
-rw-r–r– 1 safeer safeer 6653648 2008-11-06 01:32 10 - Track 10.ogg

Replace space with underscore is done by “tr” utility.

safeer@enjoyfast-lx:~/Music/ONV-Shahbaz-Venugopal/Sahayathrikey$ ls|while read FILE;do mv “$FILE” `echo $FILE|tr ‘ ‘ ‘_’`;done
safeer@enjoyfast-lx:~/Music/ONV-Shahbaz-Venugopal/Sahayathrikey$ ls -l
total 57284
-rw-r–r– 1 safeer safeer 6124954 2008-11-06 01:29 01_-_Track_1.ogg
-rw-r–r– 1 safeer safeer 5647099 2008-11-06 01:29 02_-_Track_2.ogg
-rw-r–r– 1 safeer safeer 5954106 2008-11-06 01:30 03_-_Track_3.ogg
-rw-r–r– 1 safeer safeer 5783244 2008-11-06 01:30 04_-_Track_4.ogg
-rw-r–r– 1 safeer safeer 5951001 2008-11-06 01:30 05_-_Track_5.ogg
-rw-r–r– 1 safeer safeer 6212981 2008-11-06 01:31 06_-_Track_6.ogg
-rw-r–r– 1 safeer safeer 5580947 2008-11-06 01:31 07_-_Track_7.ogg
-rw-r–r– 1 safeer safeer 5519893 2008-11-06 01:31 08_-_Track_8.ogg
-rw-r–r– 1 safeer safeer 5092362 2008-11-06 01:32 09_-_Track_9.ogg
-rw-r–r– 1 safeer safeer 6653648 2008-11-06 01:32 10_-_Track_10.ogg

If you want to eliminate spaces in the file name use:

ls|while read FILE;do mv “$FILE” `echo $FILE|tr -d ‘ ‘ `;done

String reversing in bash

I had a requirement where I needed to extract the third level sub domain name from a set of hostnames. "cut" alone was not enough to do the job as the hostnames where at different levels.

What I had was a set of hostnames like this ( This is not the real one that I worked on :P )

safeer@enjoyfast-lx:~/temp$ cat hosts.txt
www.safeer.in
dev.safeer.in
www.dev.safeer.in
mail.dev.safeer.in
mail.safeer.in
svn.dev.safeer.in
pop.mail.safeer.in
smtp.mail.safeer.in
blog.safeer.in
techlog.safeer.in
vm1.alpaha.dev.safeer.in
vm2.alpaha.dev.safeer.in

As you can see the hostnames are at different subdomain levels. So the only way is to extract the third field from right of the hostname. For this I made use of the "rev" command - which basically reverses a string and extracted the third filed as follows.

safeer@enjoyfast-lx:~/temp$ cat hosts.txt |rev|cut -d '.' -f3|rev
www
dev
dev
dev
mail
dev
mail
mail
blog
techlog
dev
dev

Did a sort to eliminate the duplicate entries. That did the trick and I got the result I needed

safeer@enjoyfast-lx:~/temp$ cat hosts.txt |rev|cut -d '.' -f3|rev|sort -u
blog
dev
mail
techlog
www

Changing Your Wordpress URL

Well, I was thinking about changing my blog URL to a nicer one. Added the new DNS sub domain and pointed it to my server. Created a new entry inapache configuration and moved all the files to the new document root( optional - only if you want to move it into a new location). This is enough to change the URL for a normal website. But for a CMS/blogging software that stores all the data and meta data in databases, this is not enough.

For wordpress, there are two database entries that affects the url. Both resides in the table wp_options. So need to update them as well.

mysql> update wp_options set optionvalue='http://techlog.safeer.in' where option_name='home';

mysql> update wp_options set optionvalue='http://techlog.safeer.in' where option_name='siteurl';
That is it. All set now.

Ubuntu Display Tips

Found out a couple of useful shortcuts/tricks while working on my Ubuntu laptop (8.04/Hardy Heron).

Press "Windows" key and move mouse scroll button downwards to get an enlarged display of your desktop.  Move the cursor to the borders of the screen to travel from side to side/top to bottom of the enlarged screen.  To zoom out scroll in the reverse direction

Place cursor on any active window/panel, press "Alt" and move scroll button downwards to make that window/panel transparent.  To turn it back to opaque scroll in the reverse direction.

My "Visual Effects" setting is set to Normal in Appearance Preferences. (System->Preferences->Appearance->Visual Effects)

Useful tips to turn your Wordpress into a CMS - 3

WP Pluggin - List Category Posts

List Category Posts allows you to list posts from a category into a post/page using the [catlist] shortcode. This shortcode accepts a category name or id, the order in which you want the posts to display, and the number of posts to display. You can use [catlist] as many times as needed with different arguments

My requirement was this.My site has a knowledge base for which a lot of articles are created as posts. All these posts come under the category "articles". I downloaded the plugin, copied it into the wp-content/plugins directory and activated it form admin panel. Then I created a page with title "Articles" (no need to be the same as the category, it was my choice) where I wanted to list all the posts under this category. So I just added this line to the page content:

[catlist name="articles" numberposts=-1 orderby=title]
name - category name
numberposts - number of posts to be displayed per page
orderby - sorting order

This worked fine for me as all my posts were within the "article" category, but as soon as I created some posts in another category called "faq" and created a sperate page with shortcode [catlist name="faq" numberposts=-1 orderby=title] it listed all my post that come under both articles and faq. So I checked back the articles page and now it is also listing all the posts form all categories. So I decided to check the plugin code for the file wp-content/plugins/list-category-posts/list_cat_posts.php. I found out an if condition with a wrong test in list_category_posts() function:

if($atts['name']!='default' && $atts['id']!='0') which will never be satisfied as we are giving either category name or category id but not both. So I just changed the "&&" joiner to "||" and everything started working fine. I am going to notify the author of the plugin. But apart from this issue, it is a very nice one. Thanks to him.

You can see this in action here or here

Useful tips to turn your Wordpress into a CMS - 2

WP Plugin - Dagon Design Sitemap Generator

If you want to create a sitemap for your site which will automatically update whenever you add a new post or page, this plugin is a good candidate.

This plugin creates a sitemap for your WordPress powered site. This is not just another XML sitemap plugin, but rather a true sitemap generator which is highly customizable from its own options page in the WordPress admin panel. Some of its features include: support for multi-level categories and pages, category/page exclusion, multiple-page generation with navigation, permalink support, choose what to display, what order to list items in, show comment counts and/or post dates, and much more.

So., I created a top level page, in the content editor chose the HTML editor and added a single line.
<!-- ddsitemapgen -->
This will give you a basic site map. To further tune this go to:
Settings->DDSitemapGen

My settings are:

Items per page: 50

Sitemap page slug: sitemap (if you are using permalinks)

What to show: Pages and posts (You can chose to show either your pages or posts or both)

If showing both, which first: Pages first (Pages is defining my site layout)

Page sort order: By menu order (ascending) - ( This will give the structure of pages and sub pages.)

Excluded categories: 1 (Excluding "Uncategorized" category)

Other setting of interest are: Sitemap navigation method,Show sitemap navigation etc...

See this in action here

Useful tips to turn your Wordpress into a CMS -1

Worpress is a wonderful blogging tool which i have using for last two years. When I planned to create a new website recently, my first consideration was a good CMS and I chose joomla as the one. Joomla is one of the best open source CMS's in the world and I did design a moderate(not bad at least) web site. But still I didn't feel the comfort that I had when I was using wordpress - nothing technical, just my inertia to changes. At last I decided to go back to wordpress and tweak it for my needs. With some basic tweaking and many plugins, I made a website easily. I will post them here for anybody want to make use of this.

Disabling comments and pings

One of the primary requirement when converting a blog software to cms is to disable comments and pings (this depends on how you want your site to be) . To do this we will alter the WP database a little.

If you have already created lot of pages, you will have to first disable them. This is what I did for my site.

mysql> update wp_posts set ping_status='closed',comment_status='closed' where post_type in ('post','page');
Query OK, 36 rows affected (0.00 sec)
Rows matched: 55 Changed: 36 Warnings: 0

Now we should set the default value for these settings to closed.

mysql> alter table wp_posts alter column ping_status set default 'closed';
Query OK, 0 rows affected (0.21 sec)
Records: 0 Duplicates: 0 Warnings: 0

mysql> alter table wp_posts alter column comment_status set default 'closed';
Query OK, 0 rows affected (0.00 sec)
Records: 0 Duplicates: 0 Warnings: 0

This alone will not help to disable comments/pings because the editor to create page/post by default will have the options to allow comments/pings as enabled. To change it

Go to: Settings -> Discussion -> Default article settings and clear all settings

You are all set. Comments will be disable in your web site now.

Flushing linux memory cache

I was looking for a fix to free the unused RAM in Linux. After searching for sometime and doing some research, I found this solution.

root@enjoyfast-lx:~# free -m
total used free shared buffers cached
Mem: 1002 984 18 0 89 386
-/+ buffers/cache: 508 494
Swap: 2047 0 2047
root@enjoyfast-lx:~# sync
root@enjoyfast-lx:~# echo 3 > /proc/sys/vm/drop_caches
root@enjoyfast-lx:~# free -m
total used free shared buffers cached
Mem: 1002 600 402 0 0 112
-/+ buffers/cache: 487 515
Swap: 2047 0 2047

root@enjoyfast-lx:~# echo 0 > /proc/sys/vm/drop_caches

The sync command will write the contents of cache into hard disk. But the cached pages will still remain in memory for quick access. This will actually speed up the execution of programs that are currently accessing this pages. But if you are running out of memory and need to free up some RAM, you can remove this pages from cache permanently bye changing the value of /proc/sys/vm/drop_caches to 3 . Now a quick analysis of what happens when we do this can be done by using the /proc/meminfo file.

Before running the above fix:

root@enjoyfast-lx:~/temp# cat /proc/meminfo > mem.b4

After running the fix

root@enjoyfast-lx:~/temp# cat /proc/meminfo > mem.af8r

root@enjoyfast-lx:~/temp# diff -y mem.b4 mem.af8r|tr -d '|'|awk '{ print $2"\t"$5"\t"expr $5 - $2"\t"$1 }'|awk '{if($3 != 0)print $0}'|sort -k4
474136 395276 -78860 Active:
346992 347096 104 AnonPages:
96172 452 -95720 Buffers:
396076 115380 -280696 Cached:
953836 954052 216 Committed_AS:
316 80 -236 Dirty:
1608 69064 67456 HighFree:
365060 67608 -297452 Inactive:
13992 342888 328896 LowFree:
15600 411952 396352 MemFree:
2928 2932 4 PageTables:
35480 15528 -19952 Slab:
25088 6308 -18780 SReclaimable:
10392 9220 -1172 SUnreclaim:

As you can see, its freeing up buffers and cache. For a complete reference of this values, go to:

http://www.redhat.com/advice/tips/meminfo.html

Unix soft link and hard link

To simply put it, a Unix link is a convenient way to access data of a file. It is similar to shortcuts in windows. But in UNIX there are two types of links, soft link and hard link. To understand the difference between these two, we should first know some basics of Unix file structure.

In UNIX file system, a file contains two parts, data part and filename part. Filename part contains the file name and indoe. Data part contains the actual data stored in the file. Inode is a data structure that contains the information about the file like permissions, group, owner, pointer to the data part etc….Each inode (and hence the data it points to) is identified by an inode number. Data part contains the actual data stored in the file. A user accesses a file using its name while the underlying file system access it with its inode number.

Different files can point to the same inode number. These files are called hard links. They will have different names and may be located in different paths, but under the same file system. Hard links can’t be created across different file systems, this is because the inode number is specific to a file system not the OS as a whole. Each file system maintains a table of inodes within that file system.

A soft link is a special file that contains the file system path to another file as its data. But as soft link (symlink) is a special file, any read/write/modify operation on this file will be redirected to the target file. Since symlinks refer to their target using absolute path and file name, it can cross file system boundaries and can be located anywhere. Soft links can point to both files and directories.

Difference b/w soft and hard links

Soft link	Hard Link
Points to file name of original file	Points to inode of original file
Can link between file systems	Can link only within same file system
Both for file & directory	Only for files
Default permission 777	Default permission 644
Can’t access data after original file is deleted	Can access data even after original file is deleted
Can’t change owner/group/permission	Can change owner/group/permission
Inode number not same as original file	Inode number same as original file

Now create a text file

[root@penguin007 links]# touch link-test.orig
[root@penguin007 links]# echo “Original file created” > link-test.orig
[root@penguin007 links]# cat link-test.orig
Original file created

The command to create link is "ln". Let us create a hard link to the file link-test.orig.

[root@penguin007 links]# ln link-test.orig link-test.hard

Now let us create a soft link to the file link-test.orig.
[root@penguin007 links]# ln -s link-test.orig link-test.soft
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig

Now you can make changes to the original file/soft link/hardlink and the change will be reflected on all others. This is because all of these are pointing to the same data location

Edit from soft link

[root@penguin007 links]# echo “Change from soft link” > link-test.soft
[root@penguin007 links]# cat link-test.orig
Change from soft link
[root@penguin007 links]# cat link-test.hard

Change from soft link

Edit from hard link

[root@penguin007 links]# echo “Change from hard link” > link-test.hard
[root@penguin007 links]# cat link-test.orig
Change from hard link
[root@penguin007 links]# cat link-test.soft

Change from hard link

Edit from original file

[root@penguin007 links]# echo “Change from original” > link-test.orig
[root@penguin007 links]# cat link-test.soft
Change from original
[root@penguin007 links]# cat link-test.hard

Change from original

From the output of "ls -li" you can observe following points.

Both the original file and hard link are having same inode (163774475 - column 1 ). In fact both these files are referred to as hard links and you can't distinguish which file is original and which one is link - there is no need to do that either.

Note the permission on the files, hard links are having standard 644 permissions while soft link has 777 permission. Hard links are treated as regular files but soft links are treated as links- note the first character in the 2^nd column of "ls" output above ("-" for regular file/hard link and "l" for soft link)

When you change permission/ownership of a hard link the change is reflected on all hard links. But when you change permission/ownership of a softlink the change is applied to the original file (and its hard links if any) while soft link retains the permission and ownership.

[root@penguin007 links]# chmod 755 link-test.hard

[root@penguin007 links]# ls -li
total 8
163774475 -rwxr-xr-x 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rwxr-xr-x 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig
[root@penguin007 links]# chmod 644 link-test.soft
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig
[root@penguin007 links]# chown safeer link-test.orig
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 safeer root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 safeer root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig
[root@penguin007 links]# chown root link-test.soft
[root@penguin007 links]# ls -li
total 8
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.hard
163774475 -rw-r–r– 2 root root 22 Jul 19 13:38 link-test.orig
163774476 lrwxrwxrwx 1 root root 14 Jul 19 13:47 link-test.soft -> link-test.orig

The third column in the output shows the hard links count, ie; how many hard links point to that file (2 in our case).

Links do not cause any additional hard disk usage. We can check it now:

[root@penguin007 links]# du -hs ./*
4.0K ./link-test.hard

0 ./link-test.soft

You may note that only one file (that comes alphabetically first) of the hard links is displayed in disk usage result (this happens if both hard links are in the same directory)

Now let us delete the original file.
[root@penguin007 links]# rm -rf link-test.orig
[root@penguin007 links]# cat link-test.hard
Change from hard link
[root@penguin007 links]# cat link-test.soft

cat: link-test.soft: No such file or directory

As you can see, soft link will not work if the original file is deleted, because it points to the name of the original file. But hard links are pointing directly to the inode, so it can continue accessing the data. When a file has multiple hard links, the data part of the file will not be deleted until all the hard links to the file are deleted. The count of hard links is stored in the inode which will be decremented every time a hard link to that data is deleted.

VNC Desktop in Linux

#su - safeer
[safeer@xmsv03 ~]$ mkdir .vnc
[safeer@xmsv03 ~]$ vi .vnc/xstartup
----------------------
#!/bin/sh

[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
xterm -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &

unset SESSION_MANAGER
exec /etc/X11/xinit/xinitrc
------------------------------------------------
[safeer@xmsv03 ~]$chmod a+x .vnc/xstartup
[safeer@xmsv03 ~]$vncpasswd
Password:
Verify:
[safeer@xmsv03 ~]$vncserver -extension XFIXES

New 'xmsv03:2 (safeer)' desktop is xmsv03:2

Starting applications specified in /home/safeer/.vnc/xstartup
Log file is /home/safeer/.vnc/xmsv03:2.log

Note that vncserver for safeer is started at display 2 (xmsv03:2) which will by default listen on port 5902 of remote machine
Now access remote linux desktop via VNC viewer

This will prompt for the password you set using vncpasswd command

To stop the server, use command

[safeer@xmsv03 ~]$ vncserver -kill :2

Setting up your first SVN Repository

This article is based on windows platform, but you can use the same commands on linux also. The only difference will be in the choice of text editor for modifying configuration files- which will be VIM for linux. In windows before setting up an svn repository for your project, make sure that all the svn binaries are available in the system path. For details on installation and configuration of Subversion server, see the related articles in this website.

Find out the SVN root for the projects, let us say C:\svnroot\
Now we can create a test project under svn root.
C:\Temp>svnadmin create C:\svnroot\samplepro
Now setup authentication for this project
Open C:\svnroot\samplepro\conf\svnserve.conf in notepad
Under the section marked [general] find the commented lines:
#anon-access = read
#auth-access = write

This allows anonymous users read access and authenticated users write access to the project source code. This is not desirable in a corporate environment -unless you are working on a project under GPL. So let us deny anonymous users any kind of access while allowing write access to authenticated users.
anon-access = none
auth-access = write

Uncomment the line
#password-db = passwd
This will tell svnserver that the password database to be used is named passwd and is in the same directory as that of svnserve.conf.
* It is a good idea to keep a common password database for all project that has to be accessed by the same users.

Uncomment the line
#realm = My First Repository

Edit the realm name to describe your Project, say

realm = Web Projects
* If you are planning to use the same user database for multiple projects, They all should have the same Realm. This will let the clients use the same cached passwords for all repositories with this realm name.

Now add users to the user database
Open C:\svnroot\samplepro\conf\passwd

Under the [users] section, delete the commented username password pairs
# harry = harryssecret
# sally = sallyssecret

then add your own usernames

safeer = af12@#dr$
murali = oh8$%q23

Now try accessing the repository via svn command line client.

winws07.safeer.in is the host name of this machine

C:\Temp>svn co –username=safeer svn://winws07.safeer.in/samplepro
Authentication realm: <svn://winws07.safeer.in:3690>> Web Projects
Username: safeer
Password for ’safeer’: ******
Checked out revision 0.

Now you can use Any GUI svn clients either as stand alone or as part of your IDE to access this repository.

If you want a stand alone client for windows, go for Tortoise SVN at http://tortoisesvn.tigris.org/
If you are using eclipse, you can get the sub ecliepse pluggin for SVN. This varies with your IDE. Even if your IDE doesn’t support SVN, you can use custom scripts to do the job, depending upon the OS platform you use.

Output buffering in PHP

I was cleaning up the code of a small php application that I wrote long time back. The application was working fine on an old hosting account where it was sitting untouched for almost two years. I was running this application on my local LAMPP installation when I encountered this error:

"Warning: Cannot modify header information - headers already sent by (output started at /opt/lampp/htdocs/sec-app/index.php:35) in /opt/lampp/htdocs/sec-app/index.php on line 86"

Now I was a bit surprised (I don't have much experience in PHP) since this was running fine in that old server and both this and my lampp instance were in PHP5/Apache 2. After searching for sometime I found out that this was due to multiple calls to the php header() function even after the script started outputting html. The remedy was to enable PHP output buffering which I did in the php.ini file. Before going into the configuration, something about output buffering:

Normally, session, cookie or HTTP header data in a PHP script must be sent before any output is generated by the script. If this is not possible in your application, you can enable what PHP calls output buffering, with the output_buffering variable. With output buffering on, PHP stores the output of your script in a special memory buffer and sends it only when explicitly told to do so. This allows you to send special HTTP headers and cookie data even in the middle or at the end of your script;but thsi can have an adverse effect in performance.


output_buffering = Off

You can also pass the output_buffering variable a number indicating the size of the buffer, like:

output_buffering = 1024

I set the value in my php.in as:

output_buffering =4096

That is the default value advised in php.ini. If you want to set this on your production server, you should think about the performance issues. If you don't have access to php.ini on your web server, you can set the same in your .htaccess file as

php_flag output_buffering on

OR

php_value output_buffering 4096

You can add the same setting under a "Directory" directive in apache's httpd.conf file. If even this is not possible or you are not sure about the environment were the script will run, you can enable output buffering in your script itself. Immediately after your php tag opens add the line ob_start(); and immediately before the closing php tag add ob_end_flush(); - or you can enclose the code that causes the error alone within this function duo.

A complete reference of the php.ini variables can be found at: http://www.php.net/manual/en/ini.php

Running a linux command in absolute silence

Just wanted to get the exit status of a command while all its output was to be suppressed. Redirecting both stdout and stderr of a command to a file is achieved by "command &>file". To suppress both the outputs, we just need to send them to /dev/null. My real problem was to create a list of DNS resolvable hostnames from a list of hosts. The "host" command's exit status is used to check whether the host is resolvable and build the host list.

for host in `cat all-hosts `;do host -t A $host &>/dev/null && echo $host >> real-hosts;done

Changing timezone in Linux

The timezone in Linux is defined in the file /etc/localtime. To change this setting you should modify the contents of this file to reflect the desired zone information. By default, the operating system will keep the zone information files for all timezones under /usr/share/zoneinfo. To set a timezone, you can either copy the file as /etc/localtime or link /etc/locatime to the target file. The preferred method is to use soft links.

I will show you how I changed the timezone in my server from PDT to IST.

Current date and timezone:

[root@www ~]# date
Sat Aug 9 09:32:11 PDT 2008
[root@www ~]# ls -l /etc/localtime
lrwxrwxrwx 1 root root 39 Aug 2 07:15 /etc/localtime -> /usr/share/zoneinfo/America/Los_Angeles

As you can see, the timezone of the system is PDT (Pacific Daylight Time) . Now I want to change this to Indian Standard Time (IST). So I am going to change the link to /etc/localtime. My timezone file will be at the location /usr/share/zoneinfo/Asia/Calcutta.

[root@www ~]# ln -sf /usr/share/zoneinfo/Asia/Calcutta /etc/localtime
[root@www ~]# date
Sat Aug 9 22:10:23 IST 2008
As you can see, the timezone is IST now.

VPS hosting @ Linode.com

It is one week since I signed up for the Linode VPS package. So far so good is my impression about their service. It is a very short time to evaluate a hosting service. Especially as I am running only a couple of low traffic websites on top of apache. But still I should say something about my initial experience.

The signup process was very quick. After I filled up my personal details, I got a mail with a link to continue signup process. I filled up remaining information - my choice was the basic package, Linode 360 for 19.95$ PM. Immediately afterwards I got an invoice mail followed by a payment receipt. Five more minutes and I got the account activation mail. All these took only ten minutes. I logged into the control panel (Linode manager), set my root password, partitioned the hard disk and installed a CentOS image. Within twenty minutes my server was up and running. Now that is cool, right? Your own server in thirty minutes. I was really impressed.

I will post some features of linode manager soon.

Bash Date Math

There were many occasions when I had to find out something like "one week after current date" or "one month and two days earlier", something like that.  All this lead me to searching for 'date mathematics' in bash.   And here is the result I found. (Warning: this may not work in older systems)

This is achieved using the -d option of date which has a syntax:

date -d STRING

where string defines, the format of date.  Here is were you can implement date math.  As you can see below:

safeer@enjoyfast-lx:~$ date
Wed Dec  3 00:11:06 IST 2008

Now find tomorrows date - just use "+1 day" as format string
safeer@enjoyfast-lx:~$ date -d "+1 day"
Thu Dec  4 00:11:07 IST 2008

So the general format of the string is "+/-N UNIT" where UNIT is any one of "second|minute|hour|day|week|month|year" and N is the number of units.  + or - sign determines whether to add or subtract a date.  A few examples:

15 hours after current time:

safeer@enjoyfast-lx:~$ date;date -d "+15 hour"
Wed Dec  3 00:14:32 IST 2008
Wed Dec  3 15:14:32 IST 2008

3 months earlier:

safeer@enjoyfast-lx:~$ date;date -d "-3 months"
Wed Dec  3 00:18:39 IST 2008
Wed Sep  3 00:18:39 IST 2008

You can also combine different units as needed:

safeer@enjoyfast-lx:~$ date;date -d "+15 week -2 hours"
Wed Dec  3 00:21:21 IST 2008
Tue Mar 17 22:21:21 IST 2009

This way we can get whatever date we need.  Combining this with the output formating option for date will give you good formatted dates to use in your scripts and commands.  For eg:

safeer@enjoyfast-lx:~$ date -d "-2 year +42 hours"
Mon Dec  4 18:23:22 IST 2006

safeer@enjoyfast-lx:~$ date -d "-2 year +42 hours" +%F
2006-12-04

TCP Wrappers

TCP wrappers is used to implement host based access restrictions to network services running on a Linux server. In order for a network service to work with TCP wrappers, it should be compiled with libwrap.a library. To check whether a network service has TCP wrapping enabled use:
strings -f /path/to/service/binary|grep hosts_access, for example to check whether ssh is compiled with TCP wrappers support,

[root@LinuxBox1 ~]#strings -f /usr/sbin/sshd|grep hosts_access
/usr/sbin/sshd: hosts_access

Host access is allowed or denied based on the entries in the files /etc/hosts.allow & hosts.deny ( together known as host access files) . Each of these files contain a list of services and corresponding list of hosts along with certain additional options.

As the name indicates, hosts.allow lists the hosts allowed to access listed services. When a host tries to access a network service on the server, hosts.allow is parsed from top to bottom to see a match for the service-host pair. The parsing will stop with the first match found and the access will be allowed. If no match is found in this file, hosts.deny is parsed. If a match is found access is denied for the host. If neither file contain a match or the files does not exist, access is allowed for the host.

Access files contain a number of rows, each with following format

daemon-list:host-list:[option1]:[option2]:..........

daemon list : Comma separated list of service binary names. Dont take this for the service name, for example, the service name for telnet is 'telnet' but the binary is 'in.telnetd' (/usr/sbin/in.telnetd).

host list : Comma separated list of hosts to be allowed/denied access to the daemon list. Hosts can be specified in any of the following format.

host names : machine1.mydomain.com represents a single machine
host names starting with a period: .mydomain.com represents all host names under this name. Example: machine2.mydomain.com, switch2.sub1.mydomain.com ......
Hostnames containing ? and/or * : Within a hostname or IP you can use this wild cards, '*' can substitute any number of characters while '?' can replace only a single character.
Eg:
*.mydomain.com : mac1.mydomain.com, mac2.sub1.mydomain.com ....
mac?.mydomain.com : mac1.mydomain.com,mac2.mydomain.com,...mac9.mydomain.com,maca.mydomain.com,macb.mydomain.com etc...
IP address ending with period : 192.168. represents all IPs in the network 192.168.0.0/16
Network & Netmasks : 192.168.1.0/255.255.255.128, 10.10.0.0/255.255.224.0 etc...
File path beginning with leading '/' : The specified file contains a list of hosts. Each line of the file will contain a number of host/ip patterns separated by white spaces.

Explicit wild cards: This wild cards has special purposes, even though there are many the popular one is 'ALL' which matches anything ( applicable to daemons as well as hosts)

The 'EXCEPT' operator: This can be used with both hosts and daemons as follows:
a) In the daemon list, a rule that is applicable to all services except vsftpd
ALL EXCEPT vsftpd
b) In the clients list all machine in the network 192.168.0.0 except 192.168.0.15
192.168.0. EXCEPT 192.168.0.15

options : This fields can be used to alter the log behaviour, launch shell commands etc when a rule is matched, but this feature is specific toRedhat version of tcp wrappers.

There are two approaches for any firewall/access control design, mostly closed and mostly open. In mostly closed all accesses arerestricted and only minimum required access is granted. For TCP wrappers mostly closed means a single line ALL:ALL in hosts.deny and necessary services in hosts.allow. In mostly open, all client access is allowed and restrictions applied as and when necessary. InTCP wrappers: ALL:ALL in hosts.allow and necessary restrictions in hosts.deny.

Some examples rules:

Allow telnet from local networks (192.168.0.0./24 & 192.168.1.0/24)
hosts.allow
in.telnetd:192.168.0.,192.168.1.
Suppose, in addition to this, you want to deny telnet access from all other hosts.
hosts.deny
in.telnetd:ALL
Since hosts.allow is parsed first the access rule ther will match request from local networks and allow them. If the request is from an external host, the rules in hosts.allow will not match and will be passed to hosts.deny where the rule is to deny all telnet communication. This rule will be matched and access denied for that host.

Alternatively we can do both this in a single rule in hosts.deny
hosts.deny
in.telnetd: ALL EXCEPT 192.168.0.,192.168.1.

Adding swap space to Linux

The virtual memory used by Linux is called swap space. It is the extension of physical memory in the sense that the total usable primary memory will be the sum of RAM and swap space. The kernel will write the memory blocks of unused programs to the swap space, thus freeing the RAM to be used by active programs. Hard disk access is much slower when compared to RAM access. So swap space can't be considered as a replacement for RAM, it is just a supporting techniques for RAM. Using of swap space imposes overhead on the system, but for systems with less physical memory it is a necessary evil.

When you install the operating system you will add a swap space, most probably up to double of the RAM. This will not be enough if you later add more RAM to your machine and you will have to increase the swap space. You can add swap space to your system in two ways, either by adding a swap partition or swap file.

To use swap partition, you should have unallocated space left on your hard disk. Use fdisk program to create a new partition.
[root@LinuxBox1 ~]#fdisk /dev/hdb
and use "n" command to create a new partition, say /dev/hdb2

If you don't have unallocated space to create a new partition, but have free space on existing partitions, you can create swap files instead. IF you want to create 128 MB of swap space, create a file of that size.

[root@LinuxBox1 ~]#dd if=/dev/zero of=/vmem/swapfile01 bs=1024 count=131072
131072+0 records in
131072+0 records out
134217728 bytes (134 MB) copied, 3.0451 seconds, 44.1 MB/s

In the rest of this document, I will use this swap file as example, but the procedure is same for swap partition also.

To mark the created space usable as swap, a signature is needed to be written into the start of file/partition. This is done withmkswap command.
[root@LinuxBox1 ~]#mkswap /vmem/swapfile01
Setting up swapspace version 1, size = 134213 kB

Now tell the kernel to use this space for swap
[root@LinuxBox1 ~]#swapon /vmem/swapfile01

In order to keep this settings persistent over reboots, add a line to /etc/fstab.
/etc/fstab
/vmem/swapfile01 swap swap defaults 0 0

To list all the swap partitions/files in a system,
[root@LinuxBox1 ~]#swapon -s
Filename Type Size Used Priority
/dev/hda3 partition 1124540 1848 -1
/vmem/swapfile01 file 131064 0 -3
/vmem/swapfile02 file 65528 0 -4

To see the current primary memory status
[root@LinuxBox1 ~]#free -m
total used free shared buffers cached
Mem: 495 470 24 0 3 204
-/+ buffers/cache: 263 232
Swap: 1290 1 1288

You can see that Total Swap reported is the sum of 3 swap sizes as listed in the output of "swapon -s"

If you want to remove the added swap space, turn of swapping for the partition/file
[root@LinuxBox1 ~]#swapoff /vmem/swapfile01

If you want to save hard disk space, remove the file/partition.

Swap files are most useful when you need to run a memory intensive program for a short period of time. You can create a swap file of appropriate size and delete it when the application is finished. If you want to add swap for permanent use, better go for swap partitions- provided your hard disk has unallocated space left.

Blocking Google Talk in your organization

Blocking instant messengers in an organization is a big headache for all system administrators. Not only that IMs waste precious productivity time and bandwidth of your company, it also possess some security threats. The major threat comes with file sharing capabilities of IMs, which can be used to share proprietary information of the company. This will become a night mare for the management if they have signed a non disclosure agreement with their clients. There are a lot of ways for an employee to ship information out of a company, but IM is one of the easiest method to do so. Another big problem is spreading of viruses and worms, unlike emails it is not easy to integrate Anti virus softwares with IMs.

But when you consider blocking IMs, the problem is that most of them are set to work with least resources and under almost any network conditions. One way is to find out the port used by the IM and block it. This works fine for Microsoft Messengers, but most others have work around for this. Here let us specifically take the case of Google Talk.

Google talk initially used Jabber/XMPP protocol which communicates over 5222 & 5223 (for SSL) ports. But blocking these ports alone will not fix the problem. The reason is that if this ports are not available, gtalk will switch to HTTP & HTTPS ports. Obviously we can’t block all HTTP & HTTPS communication. So the only way is to use a combination of protocols and server IPs (of Google talk). Again another problem with this approach is that Gtalk is having a number of servers and if it can’t access one server, it will try another IP and so forth until the connection is established. So we need to find out all the servers(IPs) used by Gtalk and block all of them. This can only be done with a trial and error approach.

Login into you Gtalk account, now the connection will be established with one of the Gtalk servers. Find out the process id of Gtalk application by either using Task Manager or the command line “tasklist.exe”. This is the out put from tasklist.exe command line:

C:\Documents and Settings\Administrator>tasklist /FI “IMAGENAME eq googletalk.exe”

Image Name PID Session Name Session Mem Usage
========================= ====== ================ ======== ============
googletalk.exe 3492 Console 0 4,900 K

So 3492 is the process id of Gtalk application. Now use the netstat.exe command line or any network monitoring application to find out the remote IP and port to which this application is connected. Look at the netstat output

C:\Documents and Settings\Administrator>netstat -ano|find “3492″

TCPÂ 192.168.0.23:4381 72.14.253.125:5222 ESTABLISHED 3492

So we have now one IP of Google talk: 72.14.253.125Â and the port connected to is 5222 -Jabber/XMPP.

Now logout of Google talk and block the communication to this IP through the protocols HTTP,HTTPS, Jabber, Jabber -SSL.ÂLogin to google talk again and repeat the same steps again. After a while you will get all the IPs used by Gtalk. Now block all the IPs as above. This will block not only Google talk but the chat applet inside the Gmail window.

The problem with this approach is that such servers will switch IPs frequently, and you will have to continuously update the list of servers. Bellow is the list of Gtalk Servers that I have found out. As of writing this, this list has been working for our ISA Server 2006 for the past month. I just created a computer set for the Gtalk servers and blocked the HTTP,HTTPS,Jabber Protocols to this Computer set. Also you will have to create Jabber as a custom protocol with outgoing TCP/UDP communication to ports 5222 and 5223.

Gtalk Servers:

216.239.37.125

72.14.253.125

72.14.217.189

209.85.137.125

209.85.163.125

209.85.201.189

216.239.51.125

Protocol - Port

HTTP - 80

HTTPS - 443

Jabber - 5222

Jabber- SSL - 5223

Linux group administration

Group information is primarily stored in /etc/group. There will be some default groups created when operating system is installed. Each line in /etc/group corresponds to a group . Each of these line will have 4 fields separated by colons. an example line is as follows

developers:x:520:baiju,vishin,suraj

Field 1 - developers : Group name

Field 2 -x : The group password was previously stored here. The x here indicate that group password is shadowed(stored in a separate file /etc/gshadow which is only readable by root)

Field 3 - 520 : The numerical group id

Field 4 - baiju,vishin,suraj : A comma separated list of group members. The user will be listed here only if this is his secondary group. Primary group membership will not be shown here.

To add a group use the groupadd command

Syntax:

groupadd [OPTIONS] groupname

OPTIONS:

-r : Create a system account with group id below 4999

-g GID : Numeric group id

To modify a group property use groupmod

Syntax:
groupmod [OPTIONS] groupname

OPTION:

-g GID : change group id

-n new_grp_name : Change group name

To delte a group, use groupdel command

Syntax

groupdel group_name

To add a user to a group you can use usermod or useradd commands with -g/G options. But the limitation with this approach is that only a single user is added at a time. More over this is done from a user perspective. To really administer group memberships, you will have to use gpasswd command.

Every group can have administrators,members and a password. Only root can assign administrators to a group while group admins and root can add members to a group.

The group password is used to restrict access through the command newgrp. The newgrp command is used to switch the primary group of a user. Suppose the user safeer has admin as his primary group, developers and staff as his secondary groups. When user safeer creates a file the owner of the file will be safeer and group of file will be admin (primary group of safeer). In some situations we may need to switch our primary group, either for creating new files with different group membership or access a file to which a certain group has permission. I would better illustrate with an example. Take the case of user safeer in the above example:

Display the groups of which safeer is a member
[safeer@LinuxBox1 ~]$groups
admin developers staff
Of this admin is the primary group and others are secondary groups. Now will create some files with different group membership
[safeer@LinuxBox1 ~]$touch file1
[safeer@LinuxBox1 ~]$ll file1
-rw-r--r-- 1 safeer admin 0 Mar 29 01:07 file1
[safeer@LinuxBox1 ~]$newgrp developers
[safeer@LinuxBox1 ~]$groups
developers admin staff
[safeer@LinuxBox1 ~]$touch file2
[safeer@LinuxBox1 ~]$ll file2
-rw-r--r-- 1 safeer developers 0 Mar 29 01:07 file2

When you use newgrp, you are automatically allotted a new shell. If you type exit from that promt you will be broguht back to the previous shell with old group membership. From the above example
[safeer@LinuxBox1 ~]$exit
[safeer@LinuxBox1 ~]$groups
admin developers staff

The main advantage of newgrp is in access permission to files. Suppose there is a file with the following permissions.
-rw-rw---- 1 root accounts 0 Mar 29 01:07 salary2

As you can see, safeer (or any others except root and members of accounts group) don't have the permission to access this file. Now a situation arises where safeer has to access this file (of course, legally!) . Then the administrator can set a password for group 'accounts ' using gpasswd
[root@LinuxBox1 ~]#gpasswd accounts
Changing the password for group accounts
New Password:
Re-enter new password:

Now root gives safeer the accounts group password.
[safeer@LinuxBox1 ~]$newgrp accounts
Password:
[safeer@LinuxBox1 ~]$groups
accounts developers admin staff
Now safeer can edit the salary2 file.

Let us summarise the use of group password now:

Group password can be set by root or group administrator. Members of a group can use newgrp to set that group as their primary group, they will never be prompted for password. If a non group member tries to newgrp to the group, he will be prompted for group password provided the group password is set, otherwise the system will simply say "Sorry".

You can disable access through newgrp in two ways. First is to remove the group password with -r option. In this case when a user tries to newgrp, he will be returned a "Sorry". Another way is to disable newgrp feature itself for a particular group with gpasswd -R option. If the user tries newgrp and the password is set, the user will be prompted for password, but even if the password he enters is correct the message "Sorry" will be displayed.
[root@LinuxBox1 ~]#gpasswd accounts
Changing the password for group accounts
New Password:
Re-enter new password:
[safeer@LinuxBox1 ~]$newgrp accounts
Password:

This is ok, now disable newgrp feature for this group
[root@LinuxBox1 ~]#gpasswd -R accounts
[safeer@LinuxBox1 ~]$newgrp accounts
Password:
Sorry.

Instead, if you remove the group password
[root@LinuxBox1 ~]#gpasswd -r accounts
[safeer@LinuxBox1 ~]$newgrp accounts
Sorry.

Now let us see how to manage group membership and administration

To add a user baiju to group developers
[root@LinuxBox1 ~]#gpasswd -a baiju developers
To remove the user from group
[root@LinuxBox1 ~]#gpasswd -d baiju developers

Root user can set group administrators and group members
To set the user sreejith administrator of developers group
[root@LinuxBox1 ~]#gpasswd -A sreejith developers
Note that sreejith need not have to be member of developers to administer it.
Now sreejith can use all the gpasswd options on developers group except for -[A/M]

Root can add a number of users to a group with -M option
[root@LinuxBox1 ~]#gpasswd -M sreejith,baiju,vishin developers
Similarly you can add a number of administrators for a group by comma separated list.

You can manually edit /etc/group & /etc/gshadow using the command "vigr".