QIC-113 file sets

QIC-113 defines what is inside a volume: a directory section that describes every file and directory, and a data section that holds their contents. It has two variants, Basic DOS and extended. Tapewyrm reads both, and turns a volume into a tar file with a report of which files touch damage.

Source: QIC-113 Rev G, 15 Jun 95 (qic113g.pdf). Bench tapes also carry Rev F volumes, which Tapewyrm reads with the same code.

Which variant

On a QIC-40 or QIC-80 tape, a volume is extended if its volume table entry carries the QIC-113 signature (113 at byte 58, Rev G §6). Otherwise it is Basic DOS. See Volume table.

Volume layout

The volume is the decompressed byte stream of the volume's segments. Its order depends on the directory-last flag (volume table byte 56, bit 5; Rev G §7):

Directory-last Layout
Clear Directory section, then data section. The directory is wholly on the first cartridge.
Set Data section, segment gap, directory section. Always set for extended volumes.

The directory section is aligned to a segment boundary. The data section ignores sector and segment boundaries and is densely packed (Rev G §7.1, §7.2).

Basic DOS directory entry

The directory section is a run of variable-length directory entries: fixed portion, optional vendor portion, name portion (Rev G §7.1.3).

Fixed portion (Rev G §7.1.3.1):

Offset Size Field Description
0 1 Size Fixed portion plus vendor portion, not counting this byte. 10 in the standard.
1 1 Attributes See below
2 4 Modified Short date/time
6 4 Data entry size Size of this item's data entry; 0 for a non-empty directory
10 1 Extra info Bits 0–5: 2 = file unreadable at backup time

Attributes:

Bit Meaning
0 Read allowed
1 Write allowed
2 Execute allowed
3 Hidden
4 System
5 Subdirectory
6 Last entry in this directory
7 Last entry in the whole directory table

A vendor portion follows when the size byte exceeds 10 (Rev G §7.1.3.2). The name portion is a 1-byte length and an ASCII name (§7.1.3.3).

Data entry size (Rev G §7.1.3.1) counts the whole data header plus the file's bytes. An empty directory has a header-only data entry, so its size is non-zero; a non-empty directory has no data entry and a size of 0.

Directory order

Rev G §7.1.4 groups entries by directory level: all entries of one directory, the last flagged "last in directory". The root level comes first. After a level come the levels of its subdirectories, left to right, each followed by its own subdirectories before the next sibling. The final entry sets "last in table".

The standard's text calls this "breadth-first, preorder", but its own worked example, and every real tape Tapewyrm has read, is depth-first over levels. Tapewyrm follows the example.

Basic DOS data entry

The data section holds one data entry per item, in directory order, for every file and empty directory (Rev G §7.2.1):

Offset Size Field
0 4 Signature 0x33CC33CC (on tape: CC 33 CC 33)
4 n Copy of the item's directory entry
4 + n m Path entry: 1-byte length, then the path of the directory the item is in, components separated by NUL. Empty for the root.
4 + n + m k File data

The signature is a resync point: if part of a volume is lost, reading resumes at the next signature.

Extended directory entry

Extended volumes (Rev G §8) use Unicode names, 8-byte sizes and per-OS attribute structures.

Fixed portion (Rev G §8.2.0.1.1.1):

Offset Size Field
0 2 Directory entry size, not counting this field
2 8 Data entry size; all ones if unknown
10 2 Path entry size
12 2 Native file system (a data description ID)
14 1 Traversal byte

Traversal byte: bit 0 directory, 1 empty directory, 2 file error during backup, 3 last in directory, 4 last on this cartridge, 5 last in the backup set, 6 root entry. A root entry's name is the source device, for example C: or \\SERVER\SYS.

Data description entries follow, one or more, in priority order (Rev G §8.2.0.1.1.3):

Offset Size Field
0 2 Data description ID
2 8 Data area size
10 2 Structure size n
12 n Structure (attributes for that file system)
12 + n 2 Name size m
14 + n m Name, UTF-16LE

Data description IDs: 0 vendor, 1 UNIX, 2 DOS, 3 Novell 3.x, 4 OS/2, 5 NT, 6 AFP, 7 data, 8 Novell 2.x, 9 Novell 4.x, 10 Windows 95.

Extended data entry

Rev G §8.1.1:

Offset Size Field
0 4 Signature 0x33CC33CC
4 d Copy of the directory entry (its sizes may be all ones)
4 + d p Path entry: UTF-16LE components, each tagged with a file-system ID
4 + d + p … Data areas: signature 0x66996699, 2-byte data area ID, then the data

Data area ID 7 is the file's primary data; ID 6 is an AFP resource fork. UNIX and DOS areas carry no data and take no space.

Dates

Variant Format Source
Basic DOS Short date/time: bits 31–25 year − 1970, bits 24–0 packed seconds Rev G §2.2
Extended 4 bytes seconds since 1970 GMT, then 4 bytes time zone and microseconds Rev G §2.2

All bytes FF means undefined.

Multi-cartridge volumes

A directory-last volume that spans cartridges appends a link sub-section to each cartridge's directory: a 4-byte directory end offset, the directory, LTLT, a count, and the ending uncompressed byte offset of each cartridge (Rev G §7.1.2). Tapewyrm recognises and skips it, and reads one cartridge at a time.

Tapewyrm notes

Modules. qiclib.volume.qic113 reads Basic DOS volumes; qiclib.volume.qic113ext reads extended ones. qiclib.volume.fileset wraps both as the qic113-basic and qic113-extended parsers that volume profiles name. qicsilver tar calls them on a TWVL volume.

Corrections from real Basic DOS volumes:

  • Nine-byte fixed portions. Some software writes a size byte of 9, with no extra-info byte. The parser uses the size byte to find the name, not a constant. See MTN backup.
  • No last-in-table bit. The same tapes end the directory with zero padding.
  • Unknown sizes. A data entry size of FFFFFFFF means unknown; the data then runs to the next signature.
  • False signatures. 33 CC 33 CC occurs inside file data. A data entry is believed only if its size byte is 9–64, its name is printable, and its length is not negative; otherwise the search resumes one byte later.
  • LTLT counts only at the end of the volume, because the four letters also occur in files.

Extended volumes: the directory is the authority. Data-section copies of directory entries may carry all-ones sizes, which the standard allows. Entry k's data starts at the sum of the data entry sizes before it in the directory section. On a bench tape those sums total the volume table's data section size exactly.

Finding a directory-last directory. Rev G §7.1.1 locates it by counting back from the end segment by the directory size. Tapewyrm finds it forward from the data section instead: for an uncompressed volume, at the first segment boundary at or after the data section size; for a compressed volume, at exactly the data section size, because the rebuilt stream has no gap. Both are tried and the first that parses wins. qicsilver extract records the exact offset in the TWVL header when it knows it.

Damage is reported, not hidden. Files the directory lists whose data was lost are still reported, so the damage report names them.

See also