|
Go to the previous, next
chapter.
The following information may be obsolete or inaccurate. Please take it with a
grain of salt (or even two :-) for the time being.
This chapter is based heavily on John Gilmore's tar(5) manual page for the public
domain tar that GNU tar is based on.
The following information may be obsolete or inaccurate. Please take it with a
grain of salt (or even two :-) for the time being.
A tar tape or file contains a series of records. Each record contains RECORDSIZE
bytes. Although this format may be thought of as being on magnetic tape, other media are
often used.
Each file archived is represented by a header record which describes the file, followed
by zero or more records which give the contents of the file. At the end of the archive
file there may be a record filled with binary zeros as an end-of-file marker. A reasonable
system should write a record of zeros at the end, but must not assume that such a record
exists when reading an archive.
The records may be blocked for physical I/O operations. Each block of n
records (where n is set by the -b option to tar) is
written with a single write () operation. On magnetic tapes, the result of
such a write is a single tape record. When writing an archive, the last block of records
should be written at the full size, with records after the zero record containing all
zeroes. When reading an archive, a reasonable system should properly handle an archive
whose last block is shorter than the rest, or which contains garbage records after a zero
record.
The header record is defined in C as follows:
Example deleted. See src/tar.h in the distribution.
All characters in header records are represented by using 8-bit characters in the local
variant of ASCII. Each field within the structure is contiguous; that is, there is no
padding used within the structure. Each character on the archive medium is stored
contiguously.
Bytes representing the contents of files (after the header record of each file) are not
translated in any way and are not constrained to represent characters in any character
set. The tar format does not distinguish text files from binary files, and no
translation of file contents is performed.
The name, linkname, magic, uname,
and gname are null-terminated character strings. All other fileds are
zero-filled octal numbers in ASCII. Each numeric field of width w contains w
minus 2 digits, a space, and a null, except size, and mtime,
which do not contain the trailing null.
The name field is the pathname of the file, with directory names (if any)
preceding the file name, separated by slashes.
The mode field provides nine bits specifying file permissions and three
bits to specify the Set UID, Set GID, and Save Text (stick) modes. Values for
these bits are defined above. When special permissions are required to create a file with
a given mode, and the user restoring files from the archive does not hold such
permissions, the mode bit(s) specifying those special permissions are ignored. Modes which
are not supported by the operating system restoring files from the archive will be
ignored. Unsupported modes should be faked up when creating or updating an archive; e.g.
the group permission could be copied from the other permission.
The uid and gid fields are the numeric user and group ID of
the file owners, respectively. If the operating system does not support numeric user or
group IDs, these fields should be ignored.
The size field is the size of the file in bytes; linked files are archived
with this field specified as zero. *Note Extraction Options::; in particular the -G
option.
The mtime field is the modification time of the file at the time it was
archived. It is the ASCII representation of the octal value of the last time the file was
modified, represented as an integer number of seconds since January 1, 1970, 00:00
Coordinated Universal Time.
The chksum field is the ASCII representation of the octal value of the
simple sum of all bytes in the header record. Each 8-bit byte in the header is added to an
unsigned integer, initialized to zero, the precision of which shall be no less than
seventeen bits. When calculating the checksum, the chksum field is treated as
if it were all blanks.
The typeflag field specifies the type of file archived. If a particular
implementation does not recognize or permit the specified type, the file will be extracted
as if it were a regular file. As this action occurs, tar issues a warning to
the standard error.
The atime and ctime fields are used in making incremental
backups; they store, respectively, the particular file's access time and last inode-change
time.
The offset is used by the -M option, when making a
multi-volume archive. The offset is number of bytes into the file that we need to restart
at to continue the file on the next tape, i.e., where we store the location that a
continued file is continued at.
The longnames field of the header belongs with something that is not yet
implemented in tar, and is therefore empty.
The following fields were added to deal with sparse files. A file is sparse
if it takes in unallocated blocks which end up being represented as zeros, i.e., no useful
data. A test to see if a file is sparse is to look at the number blocks allocated for it
versus the number of characters in the file; if there are fewer blocks allocated for the
file than would normally be allocated for a file of that size, then the file is sparse.
This is the method tar uses to detect a sparse file, and once such a file is detected, it
is treated differently from non-sparse files.
Sparse files are often dbm files, or other database-type files which have
data at some points and emptiness in the greater part of the file. Such files can appear
to be very large when an ls -l is done on them, when in truth, there may be a
very small amount of important data contained in the file. It is thus undesirable to have
tar think that it must back up this entire file, as great quantities of room are wasted on
empty blocks, which can lead to running out of room on a tape far earlier than is
necessary. Thus, sparse files are dealt with so that these empty blocks are not written to
the tape. Instead, what is written to the tape is a description, of sorts, of the sparse
file: where the holes are, how big the holes are, and how much data is found at the end of
the hole. This way, the file takes up potentially far less room on the tape, and when the
file is extracted later on, it will look exactly the way it looked beforehand. The
following is a description of the fields used to handle a sparse file:
The sp is an array of struct sparse. Each struct sparse
contains two 12-character strings which represent an offset into the file and a number of
bytes to be written at that offset. The offset is absolute, and not relative to the offset
in preceding array element.
The header can hold four of these struct sparse at the moment; if more are
needed, they are not stored in the header.
The isextended flag is set when an extended_header is needed
to deal with a file. Note that this means that this flag can only be set when dealing with
a sparse file, and it is only set in the event that the description of the file will not
fit in the alloted room for sparse structures in the header. In other words, an
extended_header is needed.
The extended_header structure is used for sparse files which need more
sparse structures than can fit in the header. The header can fit 4 such structures; if
more are needed, the flag isextended gets set and the next record is an extended_header.
Each extended_header structure contains an array of 21 sparse structures,
along with a similar isextended flag that the header had. There can be an
indeterminate number of such extended_headers to describe a sparse file.
- LF_NORMAL LF_OLDNORMAL
These flags represent a regular file. In order
to be compatible with older versions of tar, a typeflag value of
LF_OLDNORMAL should be silently recognized as a regular file. New archives
should be created using LF_NORMAL. Also, for backward compatibility, tar
treats a regular file whose name ends with a slash as a directory.
LF_LINK
This flag represents a file linked to another file, of any type, previously archived.
Such files are identified in Unix by each file having the same device and inode number.
The linked-to name is specified in the linkname field with a trailing null.
LF_SYMLINK
This represents a symbolic link to another file. The linked-to name is specified in the
linkname field with a trailing null.
LF_CHR LF_BLK
These represent character special files and block special files respectively. In this
case the devmajor and devminor fields will contain the major and
minor device numbers respectively. Operating systems may map the device specifications to
their own local specification, or may ignore the entry.
LF_DIR
This flag specifies a directory or sub-directory. The directory name in the name
field should end with a slash. On systems where disk allocation is performed on a
directory basis, the size field will contain the maximum number of bytes
(which may be rounded to the nearest disk block allocation unit) which the directory may
hold. A size field of zero indicates no such limiting. Systems which do not
support limiting in this manner should ignore the size field.
LF_FIFO
This specifies a FIFO special file. Note that the archiving of a FIFO file archives the
existence of this file and not its contents.
LF_CONTIG
This specifies a contiguous file, which is the same as a normal file except that, in
operating systems which support it, all its space is allocated contiguously on the disk.
Operating systems which do not allow contiguous allocation should silently treat this type
as a normal file.
'A' ... 'Z'
These are reserved for custom implementations. Some of these are used in the GNU
modified format, as described below.
Other values are reserved for specification in future revisions of the P1003 standard,
and should not be used by any tar program.
The magic field indicates that this archive was output in the P1003
archive format. If this field contains TMAGIC, the uname and gname
fields will contain the ASCII representation of the owner and group of the file
respectively. If found, the user and group ID represented by these names will be used
rather than the values within the uid and gid fields.
The following information may be obsolete or inaccurate. Please take it with a
grain of salt (or even two :-) for the time being.
The GNU format uses additional file types to describe new types of files in an archive.
These are listed below.
- LF_DUMPDIR 'D'
This represents a directory and a list of files created
by the -G option. The size field gives the total size of the
associated list of files. Each filename is preceded by either a Y (the file
should be in this archive) or an N. (The file is a directory, or is not
stored in the archive.) Each filename is terminated by a null. There is an additional null
after the last filename.
LF_MULTIVOL 'M'
This represents a file continued from another volume of a multi-volume archive created
with the -M option. The original type of the file is not given here. The size
field gives the maximum size of this piece of the file (assuming the volume does not end
before the file is written out). The offset field gives the offset from the
beginning of the file where this part of the file begins. Thus size plus offset
should equal the original size of the file.
LF_SPARSE 'S'
This flag indicates that we are dealing with a sparse file. Note that archiving a
sparse file requires special operations to find holes in the file, which mark the
positions of these holes, along with the number of bytes of data to be found after the
hole.
LF_VOLHDR 'V'
This file type is used to mark the volume header that was given with the -V
option when the archive was created. The name field contains the name
given after the -V option. The size field is zero. Only the
first file in each volume of an archive should have this type.
You may have trouble reading a GNU format archive on a non-GNU system if the options -G,
-M, -S, or -V were used when writing the archive.
In general, if tar does not use the GNU-added fields of the header, other
versions of tar should be able to read the archive. Otherwise, the tar
program will give an error, the most likely one being a checksum error.
The following information may be obsolete or inaccurate. Please take it with a
grain of salt (or even two :-) for the time being.
GNU tar implements an early draft of the POSIX 1003.1 ustar
standard which is different from the final standard. Adding support for the new changes in
a backward-compatible fashion is not trivial.
SunOS and HP-UX tar fail to accept archives created using GNU tar
and containing non-ASCII file names, because they use signed checksums, while GNU tar uses
unsigned checksums while creating archives, as per POSIX standards. On reading, GNU tar
computes both checksums and accept any. It is somewhat worrying that a lot of people may
go around doing backup of their files using faulty (or at least non-standard) software,
not learning about it until it's time to restore their missing files with an incompatible
file extractor, or vice versa.
GNU tar is supposed to computed both checksums, signed and unsigned, and
accept any. However, 1.11.2 has a bug by which signed checksums are incorrectly
initialized, so they do not work. This is corrected in the subsequent GNU tar
versions. However, GNU tar has not been modified to produce
incorrect archives to be read by buggy tar's.
I've been told that when Sun first imported tar on their system, they
recompiled it without realizing that the checksums were computed differently, because of a
change in the default signing of char's in their compiler. So they started
computing checksums wrongly, and stayed compatible with themselves afterwards. It now
falls on the shoulders of SunOS and HP-UX users to get a tar able to read the
good archives they receive.
| 
