676 published documents kept their own history. Six kept different words.

A PDF is not a picture of a document. It is a container, and the ordinary way to change one is to leave every byte where it is and append a correction to the end. A pointer at the back of the file names the newest table of contents, that table points past the old bytes to the new ones, and the reader shows you the result. Nothing is removed. If you changed a figure and saved, the old figure is very often still in the file, sitting in front of the new one, and it comes back out with ordinary tools.

That risk is real, it is old, and it is well documented. What I could not find anywhere was a number. How often does it actually happen in documents that are already published, on real government and institutional websites, today?

So on 19 September I collected 838 of them and looked.

The corpus

The files came from four publishers, chosen for their production pipelines rather than their subject matter: 300 US Federal Register notices, 250 gov.uk publication attachments, 147 World Health Organization titles from IRIS, and 141 arXiv preprints. A Word document, a LaTeX paper, an InDesign report and a signing pipeline fail in different ways, and a corpus drawn from one of them would have measured that producer rather than the practice.

850 URLs were collected from the four public APIs, of which 849 were distinct. One arXiv paper appeared in two category feeds, which is the kind of detail that quietly turns a denominator into a lie if nobody looks. 838 documents downloaded, about 969 megabytes. Eleven failed, eight at arXiv and three at the WHO, and they are recorded as failures rather than quietly dropped.

Every one of the 850 URLs is written down. I am making a point of that because the earlier corpus in this repository lost 567 of its own source URLs to a collection script that overwrote its own output, and a measurement whose material cannot be obtained again is an assertion wearing a number’s clothes.

The first number

676 of the 838 documents chain to at least one earlier cross reference section. Four in five published PDFs carry some appended history.

Pointed at the same 838 files, the checker I have had built for this raised its highest severity finding, the one that reads “the file contains earlier version(s) of itself, kept in full alongside the current one”, on 103 of them.

103 of 838 is 12.3 per cent. It is a good headline. One in eight published government and institutional documents still contains an earlier version of itself. I do not think it means very much, and finding out why meant writing the check a second time.

Writing it twice

The second implementation shares no code with the first and was written for this article. It walks the file’s bytes, follows the pointer chain backwards, parses every cross reference section it lands on, and reports what it finds. It handles both the classic cross reference table and the compressed cross reference stream, which matters more than it sounds: Word, Acrobat, InDesign and every linearized government file use the stream, and a check that reads only the old form sees a clean file every time.

On the question of whether a file has appended history at all, the two implementations agree on 838 documents out of 838. There is no disagreement to report. The tool is right.

Right about what, though. “This file contains an earlier version of itself” is a statement about the file’s structure. What a reader hears is “this file contains an earlier version of your words.” Those are not the same claim, and the gap between them is the whole article.

Asking a harder question

An appended section does not have to replace anything. It can add a signature, a tag or a piece of metadata and leave every object that was there before exactly where it was and still in use. So the sharper question is whether any object in the file actually exists in two versions: listed in an older section at one place, listed in the newest at a different place, with the older bytes still sitting there.

336 of the 838 documents have at least one such object.

Of the 103 documents the checker flagged at its highest severity, 26 do. The other 77 have an appended section that replaced nothing whatsoever.

Sharper again. Among the objects that do exist in two versions, 402 are page objects, and a page object is the thing that points at the text. So for each one I read the earlier page dictionary and the current one and compared which content stream each points at. Same stream, and the page’s text never changed; what was appended was structural. Different streams, and the earlier text is a separate object still sitting in the file.

403 page comparisons were examined across the corpus. Nine of them differ, and because a single page can be superseded from more than one earlier section, those nine comparisons are seven distinct pages. The seven sit in six documents.

676, then 103, then 26, then six. Same corpus at every step. The only thing that changes is how precisely I asked.

The part I did not expect

The Federal Register contributed 300 of the 838 documents. Every one of them chains to an earlier section. Every one of them contains objects in two versions. Between them they account for the large majority of the replaced structure in the entire corpus.

Not one of them changed a word.

Every Federal Register PDF here is produced by iText and carries a digital signature. Signing a PDF requires appending a revision, and the signing step rewrites the catalogue, the document information dictionary and the page structure in order to attach itself. It rewrites the shelf the text sits on and never touches the text. So the publisher whose files look worst under a naive count is the publisher with nothing behind them at all, and the checker already knows this: it excuses a signature, which is why none of those 300 documents is flagged at high severity despite every one of them carrying history.

All seven of the replaced pages come from gov.uk and the WHO, which is to say from Word and InDesign, where a person edits a page and saves over the file.

Which of these two situations you are in is decided by your publishing pipeline, not by how careful anybody was.

What is in the six

Structurally, and no further. The tool withholds what it finds by default, for the good reason that the report on a document you are worried about is the thing most likely to get pasted into a chat window, and the same restraint belongs in an article.

In five of the six, the earlier content stream decodes cleanly and differs materially from the current one. One page carries 17,042 bytes of earlier content against 14,082 current, with 62 text showing operators against 124. Another holds 52,300 against 36,188. A third replaces eight separate earlier streams with a single current one. These are not near misses. They are earlier drafts of a page, intact, inside documents that are published and downloadable right now.

The sixth is an honest dud. Its earlier stream is 36,838 bytes against a current 36,886, and the two are almost byte identical. The page was rewritten to no real effect. I am counting it, because excluding it would mean choosing a threshold after seeing the data, but a reader should know that the six is really a five and a technicality.

Three ways this could have been wrong

A number that collapses from 103 to six is exactly the shape of a number that has been measured badly, and the collapse flatters my own scepticism, which is its own kind of bias. So the ways the six could be an undercount were tested rather than asserted. Two of the three tests came back clean. The third found a bug, and it was mine.

The first: a page could keep its object number while the content object it points at is replaced in place. That would never show up in a comparison of page objects. Across all 336 documents with replaced objects, the number where a current page’s content object had itself been replaced is zero.

The second nearly went into this article as a finding. A page deleted outright would not be replaced at all, so it would never enter the comparison, yet its bytes would still be in the file. I asked which object numbers appear in an older section and are missing from the newest, and got 84,583 objects including 9,392 that parse as pages, across 557 documents. For about twenty minutes that was the most dramatic number in this piece.

It is an artifact. A linearized PDF has two cross reference sections that divide the document between them, a small one at the front for the first page and the main one for everything else. Neither is a complete picture of the file, so “in the older section and not in the newer” is the ordinary condition of most of a linearized document and says nothing at all about deletion. My parser had been discarding free entries as uninteresting, which left it unable to tell a deleted object from an object that was simply listed somewhere else.

Deletion has an exact signature: an object in use in one section and explicitly marked free in a later one. Parsing free entries on purpose and asking that question gives zero, across all 676 documents that have more than one section. Not one object in this corpus was deleted and left behind.

The third was a plain bug in my own code, and it is the reason this section exists at all. To decide whether a replaced object was a page, I read a fixed window of bytes after the object’s header and looked for its type. A short object ends well inside that window, so the window ran past the end of the object and into whatever was stored next, and the neighbouring object’s type answered a question asked about this one. Two catalogues and two fonts were counted as replaced pages. Closing the window at the object’s own end marker dropped the count of replaced pages from eleven to seven, and it also revealed that six hundred objects in the Federal Register files alone, which I had been attributing to page trees and fonts, were document information dictionaries.

I caught that one because the classification and the page test disagreed with each other and I went to find out why, rather than because anything failed. If the two had happened to agree, the number in this article would be wrong and I would have had no reason to suspect it.

The tool was wrong about fifteen files, and said so confidently

On the first pass, 823 of the 838 documents produced a report and 15 did not. All 15 were gov.uk files, and every one came back with the same message: the file “starts like a PDF but could not be parsed. It is most likely truncated or damaged.”

They are not damaged. They are AES encrypted with an empty password, which is something gov.uk does routinely, and the underlying PDF library needs an optional cryptography package to open them. That package was not installed, the library raised a dependency error, and the tool read that as a broken file and said so. Installing the package and changing nothing else, all 838 read perfectly.

Every figure above uses the full 838.

That is fixed now, in version 0.3.0. The package declares the encryption dependency, so an ordinary install pulls it in and those fifteen files simply open. The two failures have been pulled apart as well: a document the tool cannot read because the tool is missing a library now gets a message that opens by saying your file is not the problem, names the package and gives the command, while a document that really is truncated is still told it is truncated. The two exit with different codes, so the false accusation is gone from the status a pipeline reads as well as from the sentence a person reads. Install it today and that is the behaviour you get.

This is worth more than a footnote, because of the shape of the mistake rather than its size. A tool whose entire purpose is to tell you what is inside a file told fifteen people, with specifics and in confident language, that their perfectly good document was corrupt. It exited with the code that means unreadable, so a pipeline would have stopped on it. The failure was not in the detection logic, which is the part that gets the tests. It was in a dependency declaration and an error message, which are the parts nobody writes tests for, and it took a corpus of real files from a real publisher to surface it.

What this establishes, and what it does not

It establishes that appended history is close to universal in published PDFs, that almost all of it is structural, and that a warning phrased as “contains an earlier version of itself” will be true far more often than it is useful. If you are deciding what to act on, the distinction between a replaced page object and a replaced content stream is the one that matters, and the second is rare.

It does not establish that your documents are safe. This is 838 files from four institutional publishers, which is a population where, for the most part, nobody was trying to conceal anything. It exercises the case where a document is boring and the checker should stay quiet. It barely exercises the case where somebody redacted a name badly and needs to be told. Those are different questions and this measures the first.

It also says nothing about the other kinds of leftover. The same run reported covered text in 35 documents, embedded files in seven, and an author name in the document properties of 443, and none of those was chased down here.

And it is four publishers. The Federal Register’s perfect record is a property of iText and a signing policy, not of American diligence, and a fifth publisher with a different pipeline could move any of these numbers.

The whole measurement is one command over a directory of PDFs. The script and the list of all 850 URLs with their hashes are in the repository, so the way to disagree with me is to run it on documents I did not choose.

I did not write the checker. AI agents did. My part was deciding what it had to survive before it went out, which documents it had to meet that nobody on this side had written, and which of its true statements were worth anything. Of the four wrong numbers in this article, three came out of that step and one came out of my own code. None of them was a bug in the tool.

https://github.com/Waiga/before-you-send

If you publish PDFs and you want to know which of the two things you have, the most useful thing you can send back is a document where the script says a page’s content was replaced and you know for a fact that it was not.