Hamilton LaboratoriesHamilton C shell 2012User guideTutorials

Tar and mt

Oregon Coast

Tar and mt
Previous | Next

Topics

Choosing a suitable tape drive
Rewinding a tape
Listing the contents of a tar tape
Extracting the contents of a tar tape
If you can't read a tape
Writing a new tar tape
Adding more files to an existing tar tape
Check that you can read the tape you wrote!
Exchanging tapes with a Unix system
ASCII text versus binary files
See also

The tar utility can be used to read and write tar format files, tapes and diskettes. The mt utility lets you check the status of a tape drive, rewind or erase a tape, set various features, etc. Because the use of tar and mt to read or tapes is so important to so many users, particularly for interchange with Unix machines, it deserves a chapter of their own.

Most QIC (quarter-inch cartridge), 4mm DAT and 8mm Exabyte drives are supported by Windows and will work just fine with tar. But note:

  1. If you’re using Vista or Windows 7, which has an enhanced security model, you’ll need to enable access to the tape drive before you can use it. Refer to the tar -h help screen for more information.

  2. Tape device support is included only with current 32 and 64-bit versions of Windows (Windows NT and later), not the old 16-bit Windows 9x systems.

Choosing a suitable tape drive

If you don’t yet own a tape drive and you’re trying to choose the one that might be best for your own application, your first consideration should be the types of drives on any machines with which you want to exchange data. If your colleague’s machine has a QIC and you buy a DAT, this is not going to work!

If you have a choice, we generally recommend 4mm DAT drives. A single $10 DAT tape cartridge, weighing only 1.4 ounces, can store 20 gigabytes or more – no other tape format comes close in terms of cost per megabyte or weight per megabyte for media. If you have to store large amounts of data or if you’re concerned about shipping costs, DAT drives win hands down. Also, DAT drives are better-supported than any other type of drive. For example, on a DAT drive, you’ll have more flexibility in the choice of a blocksize (which could be important if someone else is sending you tapes and you have little control over how they’re written). Also, DAT drives allow rewriting of the last block on the tape; that’s useful with tar because it means you can add files onto the end of a tar tape on DAT; with other types of drives, you may have to just rewrite the whole tape to add even one file.

Some other, more expensive but higher performance choices include DLT, AIT, VXA, LTO and others. Virtually any of these should work.

Even the old QIC and Exabyte drives work quite well. QIC drives using full-size, quarter-inch, streaming cartridges have been around for over twenty years; they’re very reliable.

Rewinding a tape

Tar normally rewinds the tape before and after reading or writing it. You can suppress that with the -N (no rewind) option. You can also manually rewind the tape using the mt utility:

66 D% mt rewind

Listing the contents of a tar tape

To list the contents of a tape using the -L (Long Listing) option so you can see in detail what’s there:

67 D% tar -L \\.\tape0

The tape device is \\.\tape0 under Windows. That name was chosen by Microsoft. If you have more than one tape, the others will be called \\.\tape1, \\.\tape2, etc.

When reading a tape, tar will automatically recognize either tar or cpio formats (including both ASCII and binary versions of cpio) and will automatically do whatever "byte-flipping" is required if the tape was written on machine with a different bytesex. (More about bytesex later.) Also, tar will do its best to determine what blocksize used even if you don’t tell it. Under Windows NT 3.5, the blocksize does have to be a multiple of 512 bytes to be readable; tar will iterate through all the possibilities until it finds the right one. Under Windows NT 3.51 or later, if your tape drive supports “variable block i/o,” tar can directly determine the blocksize just by reading the first record on the tape.

For a little more diagnostic information from tar, you can use the -v (verbose) option instead of -L. You’ll get some additional information about what format (tar versus cpio), bytesex and blocksize tar is using and the offsets from the beginning of the archive at which each file in the archive appears.

Extracting the contents of a tar tape

If you can list the contents of a tar tape, you can extract it. Do this using the same procedure you used to list the contents of the tape, but adding the -x (Extract) option.

68 D% tar -Lx \\.\tape0

By default, tar will extract everything on the tape into the current directory. If all you want is just a particular file, you can specify it on the command line. Wildcards can also be used. But remember that the C shell normally expands wildcards before it starts up the application you’ve asked for. To make sure that any wildcards get passed through to tar so it can do the pattern matching, put single or double quotes around each word that contains any wildcards. For example, to extract all the *.c files:

69 D% tar -Lx \\.\tape0 "*.c"

If you can’t read a tape

Most users will never have any trouble at all reading any tape they’re ever given. But if you do encounter difficulty, it’s likely because of one of the following reasons:

  1. You’re using Vista or Windows 7, but not as an administrator, and you have not enabled access to the tape drive. Vista and Windows 7 have enhanced security and will not allow ordinary users to access the tape drive unless that’s specifically been enabled. You’ll have to either run as administrator or follow the steps outlined in the tar -h help screen to enable access to the tape drive.

  2. You don’t have the tape device driver installed. You should see a message from tar complaining that it wasn’t able to open \\.\tape0. You’ll need to install the driver via the Control Panel’s Add Hardware applet.

  3. The tape itself is just not compatible with your drive. Every tape drive technology is constantly being improved. New generation drives can usually read tapes written on the older drives, but because the newer drives typically use newer, higher-density media and more sophisticated recording formats, that compatibility can be one-way only.

    Examples are trying to read a DDS-2 DAT tape on a DDS-1 drive or a DC-6525 (525MB) QIC tape on a drive that only accepts DC-600’s or an Exabyte 8500 tape on an Exabyte 8200 drive. In each case, you just won’t be able to read anything. The tape will appear to be blank. And because a blank tape is a perfectly legal, albeit empty archive, you won’t even see any messages. Just nothing. That’s what makes this failure mode frustrating.

    Often, it is possible to force the drive to write in an older, lower-density format. With QIC and DAT drives, it’s as simple as just being sure to use low-density media. With an Exabyte, the solution is to set a jumper on the drive to configure it to act like the older models.

  4. The blocksize that was used is not supported with your drive under the release of Windows you’re running.

    For example, to pack the absolute greatest possible amount of data onto a tape, some Unix machines support writing tapes with incredibly large blocksizes, sometimes 200KB or more. This is often a problem trying to read tapes from SGI workstations. Very few if any Windows machines support blocksizes that large.

    Another example would be tape that was written with a blocksize that’s not a multiple of 512 bytes. That’s no problem if you’re running Windows NT 3.51 or later, but it’s not supported by the device drivers that Microsoft shipped with Windows NT 3.5; you’ll need to upgrade the operating system to read that tape.

    Finally, while the blocksize might indeed be a multiple of 512, it might not be a multiple that’s supported by your drive. For example, QIC drives often will support only a limited set of blocksizes.

    If it’s a problem with the blocksize, you should generally get a message from tar that makes that clear. The solution is to ask that the tape be rewritten using a blocksize you can read; some good choices might be 10,240 bytes (the POSIX standard), 1024 bytes (which everything can read and write) or the drive’s default, which you can learn by typing mt status. To rewrite the tape on the Unix machine with a 10,240-byte blocksize, the Unix tar -b (blocking factor) option should be used to specify a blocking factor of 20. (On Unix, you multiply the blocking factor times 512 to get the number of bytes per block.)

  5. You need a firmware update for your tape drive. This is a particularly likely source of trouble if you’ve scavenged your drive from an older system. All tape drives today are microprocessor-controlled and the manufacturers have made a lot changes to the firmware embedded in these drives over the years. Windows depends on that firmware being up-to-date so it can properly handshake with the drive.

    Updating the firmware is generally quite simple. Usually, you put a special update tape from the manufacturer in the drive and the drive will automatically recognize it and do the update. Contact your drive vendor for more information if you suspect a firmware problem.

  6. Your drive has a firmware bug. Some drives may claim to support variable block i/o but not actually implement it properly. If the drive claims to support this mode, tar will use it because it allows tapes be read quickly and easily even if the blocksize is unknown. But if the drive has a firmware bug, the tape may look blank in this mode. The workaround is to use tar’s -V option to tell tar to ignore the drive’s claims of supporting variable block i/o.

You’ll find additional discussion in the section, Problems reading and writing tar bapes. If you find that none of these explanations seems to fit the problems you’re having, it’s time to try writing a scratch tape just to see if your drive can at least read its own tapes.

Writing a new tar tape

To create a new tape with one or more files or directories in the archive, use the -c (create) option:

70 D% tar -Lc \\.\tape0 file1 file2 file3 ... filen

You can list as many files or directories on the line as you like. All the usual wildcards can be used and since it’s okay to let the C shell do the wildcard expansion, you don’t need to put quotes around anything. If one of the items is a directory, the entire contents of that directory will be copied to the tape.

By default, tar will try use a blocksize of 10,240 bytes, which is generally considered standard on most Unix machines. (It’s actually specified in the POSIX standard for tar-format tapes.) If your drive doesn’t support that blocksize, tar will choose something that is supported.

Adding more files to an existing tar tape

To add one or more files or directories to an existing archive, use the -a (append) option:

71 D% tar -La \\.\tape0 file1 file2 file3 ... filen

Not all drives support this function since it requires that tar be able to read the entire archive, then back up to overwrite just the last record before continuing with the new files. If you get a message from tar telling you it wasn’t able to write to archive when you use -a, that’s probably the reason. In that case, you’ll have to use the -c option instead and just plan on writing everything you want on the tape in a single operation.

Check that you can read the tape you wrote!

After writing a tape, do be sure to check that you can read it back. This is just a safety precaution the first time you try using your drive or a new setting for blocksize, etc.

Exchanging tapes with a Unix system

If the Unix machine can read the data from your tape, but it comes out garbled, the problem is probably that you’ve written the data in the wrong bytesex for that machine. Bytesex refers to the order in which the bytes are laid out on the tape. A little-endian machine (which is what all Windows machines are by edict from Microsoft) writes the data out starting with the byte containing the least significant bits (the “little” end). Many Unix machines are big-endian, meaning they start at the other end. Reading a tape from any Unix machine, regardless of bytesex, is no problem since Hamilton tar knows how to detect the bytesex and automatically do any byteswapping that might be required. Unix tar utilities lack this feature and leave it up to the operator to figure out what’s going on and, if necessary, to use a separate dd utility to swap bytes. You can verify that this is the problem if you have a tape written on the Unix machine. Read it using the -v (verbose) option and tar will tell you what bytesex was used. This is easy to fix. Use the tar -b (bytesex) option to specify a different ordering when you write the tape next time. Here’s an example, writing all the .c and .h files in the current directory to a tape in big-endian format:

72 D% tar -LcbB \\.\tape0 *.[ch]

If the tape is simply unreadable or appears blank on the Unix machine, chances are they have an older drive that does not support compression. (Since most Windows machines are fairly new, the drives installed in most of them use hardware compression to pack more data onto a tape. But this is a recent improvement in tape technology; an older Unix machine may have been built before hardware compression was available.) To write tapes with hardware compression turned off, use the -Hoff option:

73 D% tar -LcHoff \\.\tape0 *.[ch]

ASCII text versus binary files

One final consideration is that Unix and Windows differ on their line-end conventions. Unix uses a single newline \n character to mark the end of a line; Windows uses a carriage return-newline \r\n combination. Hamilton tar assumes that because tar is fundamentally a Unix format, that any ASCII files stored in a tar file will probably follow the Unix convention. Consequently, when extracting an ASCII text file, tar will convert from \n to \r\n; when adding a file to the archive, it will do the reverse. Binary files are not converted.

You can override this default behavior by specifying either -r to turn off any conversions. The -R option causes conversions to always be done, even on files that appear to be binary. You can also use the TARBINARY and TARASCII environment variables to list files that should be considered as being one type versus the other based on the filename, regardless of content. For example, database products often create files with a lot of ASCII data but which really should be considered as binary. Postscript files with encapsulated images are another example. These files should never translated as if they were ordinary ASCII files. You can indicate that by setting the TARBINARY environment variable. For example, in the System applet in the Control Panel, you might set TARBINARY = *.ps to make it treat all Postscript files as binary data.

See also

tar
mt
Problems reading or writing tar tapes
Special devices
Wildcarding
Quoting

Previous | Next

Getting started with Hamilton C shell

Hamilton C shell, as it first wakes up.

Getting started with Hamilton C shell

A first few commands.

You can set the screen colors to your taste.

You can set the screen colors to your taste.