Skip to content

Latest commit

 

History

History
144 lines (120 loc) · 6.11 KB

File metadata and controls

144 lines (120 loc) · 6.11 KB

The .DIF format and how PATCH works inside

File layout

The file is text and can be read by eye, but it is processed as bytes. Service lines are ASCII terminated by CRLF; a lone LF is also accepted on reading. Chunk bodies are raw bytes, counted by a byte counter rather than delimited by lines.

PATCH 1.0
;
FILE PATCH BOOT.ASM
OLD 37469 F5ABDE5F
NEW 37527 7533887B
@ 1195
- 7
TILE_01
+ 6
SYSINI
@ 9098
- 2
ND
+ 2
YS
END BOOT.ASM
;
FILE ADD KEYBRD.ASM
NEW 12444 D8C41527
+ 12444
<12444 bytes as they are>
END KEYBRD.ASM
;
FILE DELETE OLDFILE.ASM
;
END PATCH 8
Line Meaning
PATCH 1.0 Signature, first line of the file
FILE action NAME PATCH, REPLACE, ADD or DELETE
For DELETE the next record follows immediately
OLD length checksum What must be in the directory before the edit
NEW length checksum What must result after it
@ offset The chunk starts at this byte of the source file, counted from zero
- n The next n bytes stand here in the source file; remove them
+ n The next n bytes are to be inserted. In ADD and REPLACE records this single insertion is the entire file
END NAME End of record; the remainder of the source file is carried over unchanged
; text Comment, skipped
END PATCH n End of file, n is the number of records

Lengths and offsets are decimal, checksums are eight hexadecimal digits. Each chunk body (-, +) is followed by exactly one line break, which the parser strips; it is there for looks and does not count towards the byte count.

Actions

  • PATCH — the file is edited chunk by chunk.
  • REPLACE — the file is written out whole. This comes out shorter when there are more edits than there is file.
  • ADD — the file did not exist and is created. If it already exists, that is an error.
  • DELETE — the file is erased. Checksums are not verified, and a file that is already absent counts as success, not as an error.

Checksum

CRC-32, polynomial EDB88320h, initial value and final complement FFFFFFFFh. This is the same CRC used by zlib.crc32 and by PKZIP, so the contents of a patch can be verified with outside tools and no DOS at all:

import zlib; '%08X' % zlib.crc32(open(name, 'rb').read())

How a patch is built

The two files are compared byte by byte. At the first divergence the utility looks for the smallest pair of "how much to remove, how much to insert" after which 24 bytes match again; the search runs in a 600-byte window. If nothing is found, the remainder of both files goes out as a single chunk.

Chunks separated by fewer than MERGE matching bytes are fused into one. Edits in text travel in flocks: neighbouring lines differ by two or three characters, and taken separately that is a dozen chunks cutting words in half. Fused, it is one chunk covering whole lines, which can be read by eye. The threshold is the MERGE constant in PATCH.INC.

The resulting chunk list is verified before anything is written: the stretches carried over without editing must match the new file byte for byte, and the layout must cover both files exactly. If it does not, the record degrades into a REPLACE. The utility therefore cannot emit an incorrect patch, however unlucky the search may have been.

Memory

No file is read into memory in full, including the .DIF itself. It is written and parsed as a stream through a buffer of BUFSIZ bytes; chunk bodies are poured from stream to stream by byte count rather than by line, so the size of a patch is unbounded and may exceed by many times both any file being edited and the whole memory of the machine. This has been exercised: a 1.8 MB patch was built and applied on a 640 KB machine, and a 600 KB file was processed there.

Line parsing uses a single buffer of LINMAX = 200 bytes, and only service lines such as @ 1195 or - 7 ever enter it. Chunk bodies do not pass through it.

The files that MAKE compares are not read into memory whole either. A window of WINSIZ bytes rides over each of the two files; memory use is constant and independent of file size. APPLY and TEST make do with stream buffers alone. The utility needs about forty kilobytes and runs on a machine with a hundred and twenty-eight kilobytes of RAM.

One rule keeps the window boundary from affecting the algorithms: file contents are reached only through VMAP, which is told how many consecutive bytes are needed. It either returns a pointer behind which exactly that many good bytes lie, or rolls the window so that they do. A request longer than VMAXRQ is rejected as an error rather than served by halves. Every request in the utility is shorter than VMAXRQ: the largest are the comparison helping of CHUNKM bytes and the resynchronisation window RSWIN = MAXD + CONF. Any stretch therefore fits the window whole and the boundary is invisible to the algorithm.

This was checked by shrinking the window to 768 bytes with VMAXRQ at 640: the patch over the repository and over a binary set came out byte for byte the same as with a 4096-byte window. How often the window rolls has no effect on the result.

Beyond the stream buffers and the windows, the utility asks DOS for one memory block. It holds the chunk list for MAKE and the per-file state established by the first pass for APPLY; the two never run together, so there is nothing to divide. Each record occupies exactly one paragraph, so a record number is an addend to a segment and no wide arithmetic is needed. If DOS grants no block, both parts fall back to a slower path.

Constants

All of these live in PATCH.INC.

Constant Value Meaning
BUFSIZ 2048 Stream buffer, bytes
LINMAX 200 Service line buffer for the .DIF
CONF 32 Bytes in a row that confirm resynchronisation
MAXD 600 Limit of the resynchronisation search, bytes
MERGE 32 Gap at which chunks are fused into one
WINSIZ 4096 Window size, bytes
VMAXRQ 2048 Largest single request to the window
CHUNKM 2048 Bytes compared per helping
PATHLN 80 Full path buffer
NAMLEN 13 8.3 name with its terminating zero