Draft for the record, 2026-07-08. Scopes the whole arc honestly before any public release.
Witness Lab is a hostile, pre-registered benchmark. It was built to test whether a small "witness" or memory state carries real, load-bearing predictive value, and it was deliberately rigged to be able to say no.
It is not a discovery of a "witness", a decision-maker, or anything consciousness-adjacent. The project grew out of a speculative line of thinking about witness states and the observer loop, and the first thing the benchmark did was kill the grand version of that idea. What remains is a rigorous test harness and one narrow, honestly-scoped engineering result. Nothing more, and we say so plainly.
The underlying science is not ours and not new. Everything load-bearing here is a known framework: the Information Bottleneck, predictive-state representations, causal states, bisimulation, sparse distributed representations, rate-distortion, and the thermodynamics of prediction. The only novel artifacts are (1) this hostile benchmark and (2) the application of its one surviving result to an exocortex design.
The starting hypothesis was ambitious: that a compact "witness" state might be a genuine, measurable object, a low-bit summary that beats the raw present at predicting and recovering under perturbation. If true, it would have been a small piece of new science. We wrote it up as three separated, individually falsifiable claims so that any one could die without dragging the others down.
- Claim A — posterior-sufficiency: a label-free committor-style witness
Wmakes the present conditionally uninformative,I(Y ; C | W) <= epsilon. - Claim B — bit-value: the witness sits on the Information-Bottleneck / Pareto frontier, not strictly dominated by finite-sample contenders.
- Claim C — temporal robustness (H3): a low-bit witness/memory state improves competence-per-bit under perturbation, beating raw one-step, PSR, causal-state, and matched recurrent baselines.
The benchmark bit us, repeatedly and on purpose.
| Claim | Verdict | The honest number |
|---|---|---|
| A | REFUTED | The generated witness leaked ~0.065 bits and lost to a raw-context probe. The tautological first form (H1) was thrown out; the label-free form (H1') still failed. |
| B | NUANCED | The generated witness was not strictly dominated pointwise, but a discriminative witness got lower loss on the overlapping frontier, an oracle IB witness dominated, and decoder calibration slop (~0.148 bits) exceeded the irreducible leak (~0.065 bits). Calibration was the main failure, not information. |
| C (H3) | REFUTED on competence | The minimal temporal witness M_t won 2 of 49 informative regimes against the pre-committed K = 30, aggregate margin -0.074138 versus the frozen delta 0.123672. Memory did not beat memoryless. The hoped-for temporal-recovery edge was never claimed: it was found to resist unbiased measurement, and that too is reported as a real result. |
Two design laws fell out of the wreckage, both worth more than the claims that died:
- Build the witness discriminatively, against the decision, not as a compressed world-summary; and calibrate its decoder. (A generative world-summary leaks and mis-calibrates.)
- The failure was architectural, not a proof that memory is worthless. Post-refutation diagnostics showed real memory value existed in every Claim-C regime, an exact-belief oracle beat raw one-step by at least
0.137221competence bits (median gap0.798025), but the low-bit total-channel construction captured only a signed fraction of~0.080of it. The witness lost because it was forced to be a total predictive bottleneck: the present had to be squeezed through the same limited state as the history, so a memoryless baseline got the present for free and won.
If the failure was that the witness was a total bottleneck, the fix is obvious in hindsight: stop making the present pay memory rent. Give the present its own free full-fidelity channel, and charge only for a small, paid bounded memory that holds what the present cannot see.
Formally: predict with y_hat_t = d(C_t, M_{t-1}), where M_{t-1} is a bounded k-bit memory computed from history excluding the current observation C_t. This was pre-registered (docs/claimC-L3-sidecar-preregistration.md) with a hard null control before the code was written.
The result was POSITIVE, and it is a credible positive because it cleared the controls that would have sunk it:
- Beats present-only. Aggregate log-loss: present-only
0.636190; sidecark=1advantage0.143568,k=2advantage0.191907. - The null holds. Random / wiped memory controls stayed effectively at the present-only baseline, so the gain is real bounded memory, not extra capacity absorbing noise.
- Not drift-only. The advantage survives on the present-corrupting classes,
context_poison(k20.062437) andcontradiction(k20.258855), i.e. memory helps most exactly when the present is being lied to. - Verified: 41 tests pass, frozen-doc diff gate clean, runner hash byte-identical.
This is a scoped engineering claim under a specific perturbation protocol on a synthetic task. It is not a consciousness claim, not a novel-witness claim, and not a real-world deployment result. What it earns is a green light for one design decision, backed by a controlled result instead of a hunch. That is its entire size, and it is worth exactly that much.
- No "witness" as a novel object. No evidence of anything "making decisions".
- No new science. The mechanism is textbook partial observability: memory helps when the present is unreliable.
- No result that generalises beyond this synthetic benchmark on its own authority.
Because the honesty is the point. A benchmark that can only say yes is worthless. This one was frozen before the models existed, caught its own endpoint flattering its hypothesis twice, refuted two of its three headline claims including the one closest to the founding idea, and then validated a single modest redesign with the same unforgiving rig. That is what rigour looks like from the inside, and it is rare enough to be worth putting on the record.
The one surviving result feeds forward into an exocortex design principle: a fast agent should read the present at full fidelity on a free channel and keep a small, paid memory whose job is to cross-check the present when it cannot be trusted. We did not prove that at scale. We proved the design choice is sound on a test that could have said otherwise.
- Claims A, B, C: adjudicated (refuted / nuanced / refuted-on-competence).
- Stage 3 sidecar redesign: positive, scoped engineering validation.
- Repo: pre-registered specs, code, and tests committed; generated
results/are reproducible from source. - Public release: pending a final claims-and-leak pass over this framing.