ONLamp.com: Hacking BSD, Part 1

Published on ONLamp.com (http://www.onlamp.com/)
http://www.onlamp.com/pub/a/onlamp/excerpt/BSDHacks_chap1/index.html
See this if you're having trouble printing code examples

O'Reilly Book Excerpts: BSD Hacks

Hacking BSD, Part 1

by Dru Lavigne

Editor's note: BSD Hacks offers 100 practical tips, tricks, and tools for administrators and power users of BSD systems. To give you a taste of the kinds of "hacks" you'll find in the book, we've excerpted two here. Check back to this space next week for another sample hack on how to get the most out of FTP using macros and scripts.

Hack #13. Find Things

Finding fles in Unix can be an exercise in frustration for a novice user. Here's how to soften the learning curve.

Remember the first time you installed a Unix system? Once you successfully booted to a command prompt, I bet your first thought was, "Now what?" or possibly, "Okay, where is everything?" I'm also pretty sure your first foray into man find wasn't all that enlightening.

How can you as an administrator make it easier for your users to find things? First, introduce them to the built-in commands. Then, add a few tricks of your own to soften the learning curve.

Finding Program Paths

Every user should become aware of the three w's: which, whereis, and whatis. (Personally, I'd like to see some why and when commands, but that's another story.)

Use which to find the path to a program. Suppose you've just installed xmms and wonder where it went:

% which xmms
/usr/X11R6/bin/xmms

Better yet, if you were finding out the pathname because you wanted to use it in a file, save yourself a step:

% echo `which xmms` >> somefile

Remember to use the backticks (`), often found on the far left of the keyboard on the same key as the tilde (~). If you instead use the single quote (') character, usually located on the right side of the keyboard on the same key as the double quote ("), your file will contain the echoed string which xmms instead of the desired path.

The user's current shell will affect how which's switches work. Here is an example from the C shell:

% which -a xmms
-a: Command not found.
/usr/X11R6/bin/xmms

% which which
which: shell built-in command.

This is a matter of which which the user is using. Here, the user used the which which is built into the C shell and doesn't support the options used by the which utility. Where then is that which? Try the whereis command:

% whereis -b which
which: /usr/bin/which

Here, I used -b to search only for the binary. Without any switches, whereis will display the binary, the manpage path, and the path to the original sources.

If your users prefer to use the real which command instead of the shell version and if they are only interested in seeing binary paths, consider adding these lines to /usr/share/skel/dot.cshrc [Hack #9] :

alias which     /usr/bin/which -a
alias whereis   whereis -b

The -a switch will list all binaries with that name, not just the first binary found.

Finding Commands

How do you proceed when you know what it is that you want to do, but have no clue which commands are available to do it? I know I clung to the whatis command like a life preserver when I was first introduced to Unix. For example, when I needed to know how to set up PPP:

% whatis ppp
i4bisppp(4)              - isdn4bsd synchronous PPP over ISDN B-channel network driver
ng_ppp(4)                - PPP protocol netgraph node type
ppp(4)                   - point to point protocol network interface
ppp(8)                   - Point to Point Protocol (a.k.a. user-ppp)
pppctl(8)                - PPP control program
pppoed(8)                - handle incoming PPP over Ethernet connections
pppstats(8)              - print PPP statistics

On the days I had time to satisfy my curiosity, I tried this variation:

% whatis "(1)"

That will show all of the commands that have a manpage in section 1. If you're rusty on your manpage sections, whatis intro should refresh your memory.

Finding Words

The previous commands are great for finding binaries and manpages, but what if you want to find a particular word in one of your own text files? That requires the notoriously user-unfriendly find command. Let's be realistic. Even with all of your Unix experience, you still have to dig into either the manpage or a good book whenever you need to find something. Can you really expect novice users to figure it out?

To start with, the regular old invocation of find will find filenames, but not the words within those files. We need a judicious use of grep to accomplish that. Fortunately, find's -exec switch allows it to use other utilities, such as grep, without forking another process.

Start off with a find command that looks like this:

% find . -type f -exec grep "word" {  } \;

This invocation says to start in the current directory (.), look through files, not directories (-type f), while running the grep command (-exec grep) in order to search for the word word. Note that the syntax of the -exec switch always resembles:

-exec command with_its_parameters {  } \;

What happens if I search the files in my home directory for the word alias?

% find . -type f -exec grep "alias" {  } \;
alias h                history 25
alias j                jobs -l
Antialiasing=true
Antialiasing arguments=-sDEVICE=x11 -dTextAlphaBits=4 -dGraphicsAlphaBits=2 
-dMaxBitmap=10000000
(proc-arg 0 "antialiasing" "Apply antialiasing (TRUE/FALSE)")
(proc-arg 0 "antialiasing" "Apply antialiasing (TRUE/FALSE)")

While it's nice to see that find successfully found the word alias in my home directory, there's one slight problem. I have no idea which file or files contained my search expression! However, adding /dev/null to that command will fix that:

# find . -type f -exec grep "alias" /dev/null {  } \;
./.cshrc:alias h                history 25
./.cshrc:alias j                jobs -l
./.kde/share/config/kghostviewrc:Antialiasing=true
./.kde/share/config/kghostviewrc:Antialiasing arguments=-sDEVICE=x11 
-dTextAlphaBits=4 -dGraphicsAlphaBits=2 -dMaxBitmap=10000000
./.gimp-1.3/pluginrc:        (proc-arg 0 "antialiasing" "Apply antialiasing (TRUE/FALSE)")
./.gimp-1.3/pluginrc:        (proc-arg 0 "antialiasing" "Apply antialiasing (TRUE/FALSE)")

Why did adding nothing, /dev/null, automagically cause the name of the file to appear next to the line that contains the search expression? Is it because Unix is truly amazing? After all, it does allow even the state of nothingness to be expressed as a filename.

Actually, it works because grep will list the filename whenever it searches multiple files. When you just use { }, find will pass each filename it finds one at a time to grep. Since grep is searching only one filename, it assumes you already know the name of that file. When you use /dev/null { }, find actually passes grep two files, /dev/null along with whichever file find happens to be working on. Since grep is now comparing two files, it's nice enough to tell you which of the files contained the search string. We already know /dev/null won't contain anything, so we just convinced grep to give us the name of the other file.

That's pretty handy. Now let's make it friendly. Here's a very simple script called fstring:

% more ~/bin/fstring
#!/bin/sh
# script to find a string
# replaces $1 with user's search string
find . -type f -exec grep "$1" /dev/null {  } \;

That $1 is a positional parameter. This script expects the user to give one parameter: the word the user is searching for. When the script executes, the shell will replace "$1" with the user's search string. So, the script is meant to be run like this:

% fstring word_to_search

If you're planning on using this script yourself, you'll probably remember to include a search string. If you want other users to benefit from the script, you may want to include an if statement to generate an error message if the user forgets the search string:

#!/bin/sh
# script to find a string
# replaces $1 with user's search string
# or gives error message if user forgets to include search string
if test $1
then
   find . -type f -exec grep "$1" /dev/null {  } \;
else
   echo "Don't forget to include the word you would like to search for"
   exit 1
fi

Don't forget to make your script executable with chmod +x and to place it in the user's path. /usr/local/bin is a good location for other users to benefit.

Hack #21. Manage Temporary Files and Swap Space

Add more temporary or swap space without repartitioning.

When you install any operating system, it's important to allocate sufficient disk space to hold temporary and swap files. Ideally, you already know the optimum sizes for your system so you can partition your disk accordingly during the install. However, if your needs change or you wish to optimize your initial choices, your solution doesn't have to be as drastic as a repartition - and reinstall - of the system.

TIP: man tuning has some practical advice for guesstimating the appropriate size of swap and your other partitions.

Clearing /tmp

Unless you specifically chose otherwise when you partitioned your disk, the installer created a /tmp filesystem for you:

% grep tmp /etc/fstab
/dev/ad0s1e    /tmp    ufs    rw    2    2

% df -h /tmp

Filesystem    Size   Used  Avail Capacity  Mounted on
/dev/ad0s1e   252M   614K   231M     0%    /tmp

Here I searched /etc/fstab for the /tmp filesystem. This particular filesystem is 256 MB in size. Only a small portion contains temporary files.

TIP: The df (disk free) command will always show you a number lower than the actual partition size. This is because eight percent of the filesystem is reserved to prevent users from inadvertently overflowing a filesystem. See man tunefs for details.

It's always a good idea to clean out /tmp periodically so it doesn't overflow with temporary files. Consider taking advantage of the built-in periodic script /etc/periodic/daily/110.clean-tmps [Hack #20] .

You can also clean out /tmp when the system reboots by adding this line to /etc/rc.conf:

clear_tmp_enable="YES"

Moving /tmp to RAM

Another option is to move /tmp off of your hard disk and into RAM. This has the built-in advantage of automatically clearing the filesystem when you reboot, since the contents of RAM are volatile. It also offers a performance boost, since RAM access time is much faster than disk access time.

Before moving /tmp, ensure you have enough RAM to support your desired /tmp size. This command will show the amount of installed RAM:

% dmesg | grep memory
real memory  = 335462400 (319 MB)
avail memory = 320864256 (306 MB)

Also check that your kernel configuration file contains device md (or memory disk). The GENERIC kernel does; if you've customized your kernel, double-check that you still have md support:

% grep -w md /usr/src/sys/i386/conf/CUSTOM

device        md    # Memory "disks"

Changing the /tmp line in /etc/fstab as follows will mount a 64 MB /tmp in RAM:

md /tmp mfs rw,-s64m 0 0

Next, unmount /tmp (which is currently mounted on your hard drive) and remount it using the new entry in /etc/fstab:

# umount /tmp
# mount /tmp

# df -h /tmp
Filesystem    Size   Used  Avail Capacity  Mounted on
/dev/md0       63M   8.0K    58M     0%    /tmp

Notice that the filesystem is now md0, the first memory disk, instead of ad0s1e, a partition on the first IDE hard drive.

Creating a Swap File on Disk

Swap is different than /tmp. It's not a storage area for temporary files; instead, it is an area where the filesystem swaps data between RAM and disk. A sufficient swap size can greatly increase the performance of your filesystem. Also, if your system contains multiple drives, this swapping process will be much more efficient if each drive has its own swap partition.

The initial install created a swap filesystem for you:

% grep swap /etc/fstab
/dev/ad0s1b    none     swap    sw    0    0

% swapinfo

Device          1K-blocks     Used    Avail Capacity  Type
/dev/ad0s1b        639688       68   639620     0%    Interleaved

Note that the swapinfo command displays the size of your swap files. If you prefer to see that output in MB, try the swapctl command with the -lh flags (which make the listing more human):

% swapctl -lh
Device:       1048576-blocks      Used:
/dev/ad0s1b          624          0

To add a swap area, first determine which area of disk space to use. For example, you may want to place a 128 MB swapfile on /usr. You'll first need to use dd to create this as a file full of null (or zero) bytes. Here I'll create a 128 MB swapfile as /usr/swap0:

# dd if=/dev/zero of=/usr/swap0 bs=1024k count=128

128+0 records in
128+0 records out
134217728 bytes transferred in 4.405036 secs (30469156 bytes/sec)

Next, change the permissions on this file. Remember, you don't want users storing data here; this file is for the filesystem:

# chmod 600 /usr/swap0

Since this is really a file on an existing filesystem, you can't mount your swapfile in /etc/fstab. However, you can tell the system to find it at boot time by adding this line to /etc/rc.conf:

swapfile="/usr/swap0"

To start using the swapfile now without having to reboot the system, use mdconfig:

# mdconfig -a -t vnode -f /usr/swap0 -u 1 && swapon /dev/md1

The -a flag attaches the memory disk. -t vnode marks that the type of swap is a file, not a filesystem. The -f flag sets the name of that file: /usr/swap0.

The unit number -u 1 must match the name of the memory disk /dev/md1. Since this system already has /tmp mounted on /dev/md0, I chose to mount swap on /dev/md1. && swapon tells the system to enable that swap device, but only if the mdconfig command succeeded.

swapctl should now show the new swap partition:

% swapctl -lh
Device:       1048576-blocks      Used:
/dev/ad0s1b          624          0
/dev/md1             128          0

Monitoring Swap Changes

Whenever you make changes to swap or are considering increasing swap, use systat to monitor how your swapfiles are being used in real time:

% systat -swap

The output will show the names of your swap areas and how much of each is currently in use. It will also include a visual indicating what percentage of swap contains data.

OpenBSD Differences

You can make this hack work on OpenBSD, as long as you remember that the RAM disk device is rd and its configuration tool is rdconfig. Read the relevant manpages, and you'll be hacking away.