Please consider a donation to the Higher Intellect project. See or the Donate to Higher Intellect page for more info.

Getting a Bang Out of Unix

From Higher Intellect Vintage Wiki
Jump to navigation Jump to search
Open Computing ``Hands-On'': ``Wizard's Grabbag'' Column: January 94

Getting a Bang Out of Unix

Automatically generating temporary files, saving money by sending e-mail in
batches, and revisiting the rounding error in ap

By Becca Thomas

With the start of the new year and the premiere of UnixWorld's Open Computing,
we are taking this opportunity to make some improvements to ``Wizard's
Grabbag.'' This month we begin a new experiment: selecting and presenting items
from O'Reilly & Associates' Unix Power Tools, a book that presents tips,
scripts, and techniques like those published in this column. From time to time
I'll publish interesting items useful to most readers and try to locate related
material that didn't make it into the book. For our first selection we present
a script that automatically generates temporary files for command operations
requiring file arguments.

In the ``From the Wizard's Den'' section I provide a tip that can help some
site administrators configure their system to save money by batching outgoing
electronic mail.

Finally, we have ``Closing the Feedback Loop,'' which contains selected
contributions related to recent columns. Harvey Davies provides some feedback
regarding his ap program that was originally published in my August 1992

Bang Around the Command Line

Dear Editor:

Here's a little tool that should prove useful to anyone who works from the
shell command line. Some commands or command modes require you to provide an
argument that names a file instead of supplying that data on the standard input
or the command line. Such files are often created just for the command
operation and then erased. The !, or ``bang,'' script [Listing 1A] can be used
to name a temporary file that is populated by running a specified command.

I'll start with an example that can be simplified with ! [Listing 1B]. Here, we
manually create two temporary files that contain sorted versions of the input
data. With the comm program, we compare the contents of these two sorted files.
The default comm report--no invocation options specified--consists of three
columns: unique items in the first file, the second file, and common to both
files. The next example [Listing 1C] shows how to collapse the four command
lines [used in Listing 1B] into a single command using !.

We use ! to expand tab stops in files before they are compared by diff [shown
in Listing 1D]. The diff output from files containing tabs may appear
misaligned because of the extra characters (>, <, +, and so forth) that diff
prefixes to the output line. If you expand the tabs to spaces before using
diff, proper alignment of the output can be maintained.

Some popular paginators like more can't back up when they're reading from a
pipe. Perusing the output from a pipe can be painful if you can't reverse
direction to reread something you saw earlier. For instance, ls -l | more
displays a screenful at a time from ls - l output, but if something of interest
has scrolled past, you can't review it unless, of course, you're using a
display system with reverse- scroll capability. However, if you invoked more
using more `! ls - l`, you can review the result as much as you wish by moving
around the temporary file with more file- perusal commands.

The main problem with the ! script published in Unix Power Tools was that it
didn't remove the temporary file it created. The script can't remove this file
as it exits because then the temporary file might not be found when needed by
the ``driver'' command [like comm in Listing 1C]. The new version in Listing 1A
waits 30 seconds for the ``driver'' command to start and then erases the
temporary file. Your readers could take this approach farther by adding an
option that changes the delay time. For instance, ! -600 would wait 10 minutes
(600 seconds) and ! -0 would never remove the file.

Jerry Peek / O'Reilly & Associates Inc. / Sebastopol, Calif

Explanation of ! script. Line 7 defines the parent directory, and line 9
defines the full path name of the unique temporary file. Here, $$ will be
replaced by the process ID number of the shell running the instance of the !

Line 12 sets a trap to erase the temporary file when the script is terminated
by one of the specified signals: hangup (1), interrupt (2), or software
termination (15). The quit (3) signal is not trapped; it will abort ! without
removing the temporary file.

Lines 15-19 process the command line. If no invocation arguments were
specified, line 16 reports the correct usage to standard error, line 17 outputs
the path name of the temporary file, and the script terminates. Hopefully, when
the driver command processes the (missing) temporary file, it will exit

Line 22 runs the specified command line, which is contained in the [email protected] variable,
and redirects its output to the temporary file. Line 23 reports the temporary
file name on the standard output for use by the driver command.

Line 26 is important. This line closes the standard output so the shell that's
``filling the backquotes'' with the output of the ! program won't wait for more
output. Without this line, the calling shell would wait until the subshell on
line 27 exits and closes its standard output.

Line 27 runs a subshell in the background to erase the temporary file after 30

Big Bang Contest!

We're starting a new contest this month. Send in your shell command line
examples for using the ! program discussed here. The most clever or useful
contributions will be featured in a future column.

From the Wizard's Den: Batch UUCP Mail and Save

When Mail User Agents (MUAs) such as mailx realize that a message is destined
for a remote machine connected via UUCP, it invokes uux to queue a request to
run the rmail command on the next machine. The rmail command will deliver the
mail message to a subsequent machine or place it in the mailbox for the
recipient if the next hop is the final destination machine.

When uux finishes queuing the request, it attempts to contact the next machine
by invoking uucico. Unfortunately, the time it takes to establish and sever a
UUCP connection to a remote machine often takes longer than sending the message
itself! Thus, it would be more efficient to collect all mail requests during an
appropriate time interval and then send them all at once. To do this, we need
to have the mail program run uux with the -r option, which will queue the
request but not invoke uucico.

If you can't configure your MUA to run uux -r instead of uux, you could install
a shell script named uux, which in turn would run the actual uux binary with
the -r option. Listing 2A shows one way to set up this configuration.

We change to the directory containing the uux binary. Then we rename the binary
to something else; UUX is a convenient choice. Next we create a simple one-line
shell script containing the directive to run the UUX binary with the -r option,
followed by any arguments passed to the script. We give the script 755
permission modes so it's executable by anyone on the system. Finally, we check
our work with a long directory listing.

The queued mail will be sent when uucico is invoked. Generally, the cron daemon
runs an ``hourly'' script-named /usr/lib/uucp/uudemon.hour with HoneyDanBer
UUCP--periodically, say once or twice an hour. This script in turn invokes
uusched, which will find any queued mail messages and invoke uucico to contact
the next machine on their way to their final destination.

Now that we have a shell script that's invoked to run UUX , we can add other
useful auditing functions. For instance, we can log the actual command line to
see where the message is destined. This feature is provided automatically with
System V Release 4 HoneyDanBer UUCP, which logs all uux as well as uucp
invocations in a file named /var/spool/uucp/.Admin/command.

I've found it useful to display the environment of the shell running the uux
script. In particular, if mail is passing through our site, uux will be invoked
by the ``uucp'' user. Examine the UU_USER environment variable for this user to
see where the e-mail originates. I've found this tactic useful for tracing
excessive e-mail destined for users who no longer use our site yet still
receive mail forwarded by our site, such as automatic messages from mailing
lists. Also, I like to display the complete shell environment-by running the
System env or BSD printenv command--for system accounts that run commands using
cron, such as ``root'', ``adm'', and ``sys''. If any of our readers find any
other interesting applications, let me know.

Listing 2B shows a sample script that supports the additional features I've
described. Caution: Make sure the script works correctly before you replace the
uux binary. Otherwise, a syntax error could cause the script to abort so the
UUX binary would not be run. Also, some commands that you call from the uux
script might have unanticipated side effects. For instance, if you include a
command to mail an error report to a remote user, uux could be called
recursively until some system resource is exhausted, such as all the free
blocks or inodes in the file system containing the UUCP spooling directory.

Listing 2C shows some lines to use in the uudemon.cleanup script to mail the
uux log to the UUCP administrator and then shuffle the log, keeping one backup
copy. We copy instead of moving the log to the Old directory so the subsequent
step will truncate the original copy without changing its ownership or
permission modes. All uux users will need to write to this log--but they don't
need to read it--so you could install it as owned by the ``root'' or ``uucp''
account with mode 622.

Closing the Feedback Loop: Correcting the Correction

Dear Editor:

Your August 1992 column featured my ap program, and you published a ``bug'' fix
in February 1993. However, I wish to point out a rounding-error problem caused
by the modification proposed by David Bacon. For instance, the command line ap
-n 10 0 3.5 0.14 run on a machine using IEEE floating-point arithmetic displays
a sequence that omits the expected final value of 3.5 [see Listing 3A].

Mr. Bacon suggested that you change the constant on line 90 of the original
program from 1.5 to 1.0 [see Listing 3B]. My value of 1.5 minimizes the effect
of rounding error, but does allow a final value beyond a specified limit that
does not correspond to an element of the arithmetic progression. For instance,
the ap 1 6 2 command prints ``1 3 5 7'' with the original version, and the
modified version prints ``1 3 5''.

My original letter should have pointed out the need to specify a limit within
less than half a step of the final value to be produced. I do not believe this
limit is a significant problem, and as far as I can see, any attempt to solve
it will increase the possibility of the more serious omission problem discussed
above. Furthermore, you could add code to not display values greater than the
limiting value.

My other suggestion is to make the default number of elements per line
infinity--rather than 50--because the program is mainly used with command
substitution, as in contour -h `ap -s , - n 999 0 100`. This change eliminates
the need to specify the number of elements per line in such cases [see Listing

Harvey Davies / CSIRO Division of Atmospheric Research / Victoria, Australia

WANTED: Alternate ap Implementations

Here's a challenge for our programming and numerical analysis buffs: Provide an
alternate implementation of ap that doesn't have the ``rounding-error'' problem
discussed herein. One suggestion: instead of computing the number of elements
to display, display elements until the limiting value is reached or exceeded.
Of course, you would need to accommodate both positive and negative integer and
floating-point values for the starting, limiting, and step values. Also, I'd
recommend support for a repeat value using a command-line option-like -r
count--instead of a special case for input values (step is zero).

Copyright © 1995 The McGraw-Hill Companies, Inc. All Rights Reserved.
Edited by Becca Thomas / Online Editor / UnixWorld Online / [email protected].com

 [Go to Content]   [Search Editorial]

Last Modified: Tuesday, 22-Aug-95 15:46:52 PDT