Status: Implemented · Describes: main @ a9c0b9f · Landed across #118, #119, #121,
#122, #123, #124, from the proposal in #117.
An extension pack is an ordinary Python distribution that offers a prompt, a strategy or a verb to
any repository that installs it. A catalog is a static index.toml in a git repository listing
packs and pointing at receipts committed beside them. Between the two, an extension one team writes
can travel to another. A repository can find out what one would do to it before installing,
measure it before trusting it, and be told when it changes.
The whole design turns on one sentence: installing a pack offers it, and a line somebody wrote is what puts it in force. Everything below is either that property or a consequence of it.
This document described a proposal when it was written. It describes the code now. Where the implementation departed from the proposal, in three places, it says so, because a design note that quietly rewrites its own history is worth less than the argument it lost.
Four sources may write a decision. They are applied in one order and win in the opposite one, and a distribution mechanism is a claim about where in this stack a stranger's code may land.
| Applied | Source | Tier | May set |
|---|---|---|---|
| 1st | shipped defaults | SHIPPED |
bodies, layer stacks, LENSES, default policy |
| 2nd | installed in_lockstep.standards packages |
PLUGIN |
tighten-only policy layers, plugin-tier bindings |
| 3rd | .lockstep/lockstep.py |
EXPLICIT |
anything; outranks every plugin regardless of order |
| 4th | the call site, ctx.do(..., via=tdd) |
call-scoped | one run only; the same capability-keyed gates apply |
Policy is the exception and stays the exception: layers merge tighten-only, so order decides who is printed, not who wins.
Drawn: docs/diagrams/extension-resolution.html.
A pack writes into band three and nowhere else. There is no fifth band, and nothing lets a third
party land at Tier.EXPLICIT on its own. A pack that is installed and not named in lockstep.py
has no effect whatsoever, which GATE-PACK-1 holds.
These are what an adopter extends. They are ordinary subclassing and binding, and packaging one for distribution (§3) changes nothing about how it is written.
Subclass a shipped prompt and set emphasis, or point body at your own .md. Install it by
spreading LENSES into AiReview(lenses=…). House guardrails append via
review_layers().plus(…), which cannot displace the shipped baseline.
What you write is now visible before it runs:
in-lockstep show-prompt security # what a run would send, off the BOUND adapter
in-lockstep show-prompt security --diff # what you changed, against the shipped body
in-lockstep show-prompt security --shipped # the framework's version, ignoring your bindingThat was #118, and it was a defect rather than a feature request: show-prompt imported the
shipped LENSES, PROMPTS and TRIAGE_PROMPTS maps and never loaded the module, so the one
command whose job is answering what was the model actually told showed a team the prompt they had
replaced. Three prompt families (fix, rfe and backport) shipped and were unreachable from it
entirely.
ls prints the same answer in summary: each AI binding's prompts, starred where the class is not
the shipped one, with the guardrail chain underneath.
Subclass ImplementStrategy or FixStrategy, implement invoke, and lockstep.use(Yours).
capabilities is refused at class creation if it declares less than the strategy holds, and use()
fills in the WorktreeRunner wrap and the InvokePolicy.under(...) floor that a hand-written
bind can silently drop.
Verb("benchmark") is first-class: its own telemetry label, step ids, spend accounting, middleware,
kill switch. Everything keyed on the adapter comes free. Five things keyed on the verb are owed
by the author: a strategy and request type, a route and a price, prompts and a layer stack, a
corpus family, and a @workflow (there is deliberately no CLI subcommand for a custom verb). A
verb pack is that set, shipped together.
A pack is an ordinary Python distribution. There is no manifest format for behaviour, because the container is still the registration mechanism.
acme_review_prompts/
├── __init__.py # a docstring, for a data pack
├── pack.toml # kind and summary; nothing that changes what runs
├── prompts/*.md
├── corpus/<family>/<prompt>/*.json
└── cassettes/*.json
Resources live inside the module directory rather than at the distribution root, because files outside the package are not installed by a wheel without extra declaration.
in_lockstep.standards installing IS applying
package ──▶ detect() ──▶ container (Tier.PLUGIN, tighten-only)
in_lockstep.extensions installing only OFFERS
package ──╳── container never binds itself
└─────▶ a line in lockstep.py ─────▶ container
The older group is right to apply itself: a standards package can only tighten, so the real risk is
a repository forgetting one. An extension pack is the opposite risk. It hands a model write and
execute tools and it spends money, so its arrival is a diff somebody read. detect() does not even
query the extensions group.
That is the same structural property #104 and #116 established from the other direction: which
strategy runs is a bind-time code decision, so nothing a ticket carries can steer a run. A registry
resolving --strategy acme/tdd-pro over installed packages would have handed it back.
Two consequences worth stating:
Listing a pack runs no code it ships. installed() never calls entry.load(), and pack.toml,
the corpus, the cassettes and the AST behind imports are all read through distribution metadata
rather than importlib.resources, which would import the package to answer.
A broken pack is quiet, where a broken standard is loud. load_standards raises, because
running without standards somebody installed is a control silently absent. A pack that fails to
parse has applied nothing, so it is listed beside the others rather than hidden behind an exception.
in-lockstep pack describe derives what a configuration does. Every field is read off something
that already declares it:
| Field | Read off |
|---|---|
capabilities |
the bound object: the same frozenset ApprovalGate, the budget refusal and the egress trigger key on |
projection / guardrails_intact |
the composed prompt, via PromptLayers.projection() |
policy |
the stack's own layers with their sources, plus what they merged to |
models[].priced |
the same table_for DOC151 uses |
egress |
the same manifest egress-manifest hands the proxy |
imports |
an AST walk over every .py the distribution ships |
offers |
the imported namespace, through core's vocabulary: verb plus capabilities |
corpus / cassettes |
files on disk |
Two subjects. With no argument the subject is this repository, which is where the format was exercised first and is useful on its own as an audit of your own configuration. With a name it is an installed pack.
--json is canonical: sorted keys, and the digest is over exactly what is printed, excluding
itself. A receipt that hashed its own hash could not be verified by recomputation, which is the
only thing the digest is for.
Two fields carry the argument. guardrails_intact says whether a prompt still opens with
guardrail:baseline. Changing that is legal, and it is the thing a reader of somebody else's
extension most needs told.
corpus is null rather than 0 when a repository has none, and says in words that nothing bound
there has been measured. Borrowing the framework's shipped case count would be the reassuring number
computed from somebody else's evidence. priced gets three states for the same reason: true,
false, and null when the provider is not registered, because a machine that cannot check a route
is not a repository with a broken one.
none, modules, or unknown, derived by walking the AST: none means every module the pack
ships holds a docstring and nothing else, so importing it can run no code of the pack's own.
A tier, where kind = "prompt" refuses any real Python, was considered and rejected. It buys a
category-wide guarantee and immediately meets its awkward case: the pack that wants to ship the
three-line Prompt subclass binding its own body, which is the natural thing for a prompt author to
include and which makes the pack code. Then either the tier lies or the pack is not a prompt pack.
So describe computes the field for every pack, and the rule bites at one boundary: the project's
catalog will not list a pack as kind = "prompt" unless it reports imports: none.
unknown is neither of the others. A distribution nothing could resolve to files has not been
checked, and reporting it as inert would be the reassuring answer computed from nothing.
pack.toml carries a kind and a summary. An unknown key is refused, not ignored: a key silently
accepted arrives, gets documented, and becomes load-bearing. That is how a declaration file turns
into a configuration language. The kind is cross-checked against what describe derived, so a pack
calling itself prose while shipping a strategy says so in its own receipt.
$ in-lockstep add acme-tdd-pro
pack acme-tdd-pro 2.1.0
imports modules (importable code — installing puts it in your import graph)
capabilities reads_repo spends_budget writes_files executes_code
catalog the installed code matches the published receipt
recorded .lockstep/packs/acme-tdd-pro.json
commit it: the record IS the acknowledgement, and doctor reads it.
paste into .lockstep/lockstep.py:
from acme_tdd_pro import AcmeTDD
implement = lockstep.use(AcmeTDD)
until you do, nothing changes: `in-lockstep ls` will not mention acme-tdd-pro.
It never edits .lockstep/lockstep.py, under any flag, which GATE-PACK-4 holds. That file can
rebind any adapter, remove any middleware and grant any tool; it is the first entry in its own
protected-path deny list and it loads from a trusted ref precisely because every line in it was
typed by a person. The failure mode is good too: a pack installed and unbound does nothing at all,
and ls says so, which is a better state than a binding nobody noticed arriving.
It does not install anything either, and that is a departure from the proposal. The first draft
had add pin the dependency itself. Putting a stranger's code on your machine is your package
manager's job, in your dependency diff, and a framework that ran the installer would be the thing
deciding what you trust rather than the thing telling you what you are trusting.
add requires the pack to be installed, reports whether it is pinned, and prints
uv add <distribution> when it is not.
.lockstep/packs/<name>.json holds the receipt as it stood when the repository accepted it. This
replaced the proposal's lockstep.use(AcmeTDD, acknowledged="2.4.0"): a version string in code says
when something was accepted and not what, and the comparison that matters is over capabilities,
offers and the projection. It also keeps bookkeeping a tool maintains out of the file whose value is
that a person wrote it, the same instinct as printing the bind lines.
Widening is the line. A capability the pack did not previously hold is refused until --accept
says otherwise, and a refusal records nothing, so it cannot leave a repository having accepted what
it declined (GATE-PACK-3). Everything else is recorded without a flag and reported: a version, a
new prompt, a corpus that grew. Refusing over those too would teach people to pass --accept by
reflex, which is how the flag that guards agency stops meaning anything.
Comparison runs over a material subset of the receipt, excluding requires and imported: a
receipt published on one framework version compared against one derived on another differs in the
deriving machine and in nothing that matters, and a comparison that called that drift would cry wolf
on every upgrade until people stopped reading it.
$ in-lockstep doctor
DOC170 pack 'acme-tdd-pro' may now do more than this repository accepted: +reaches_network
Read what changed, then accept it in a diff:
`in-lockstep add acme-tdd-pro --accept`, and commit the record. FAIL
DOC171 review/security does not open with the shipped guardrail baseline WARN
DOC172 pack 'acme-tdd-pro' is installed but not pinned WARN
Only DOC170 is an error, and the line it draws is agency. It is a note rather than a failure
for a pack installed and never accepted. Installing offers a pack, so one that nothing binds is
ordinary, and failing over it would make the group's premise read as a problem. DOC171 reads the
bound adapters rather than the installed packs, because what matters is the prompt a run would
actually send.
A static file in a git repository. No service, no accounts, no ranking to defend.
# index.toml
[index]
criteria = true # this catalog claims to apply the criteria in §5.1
[[pack]]
name = "acme-review-prompts"
distribution = "acme-review-prompts"
index = "https://pypi.org/simple"
kind = "prompt"
summary = "A security lens that knows about our ORM"
source = "https://github.com/acme/review-prompts"
receipt = "receipts/acme-review-prompts-1.0.0.json" # derived, not writtenin-lockstep market add acme https://raw.githubusercontent.com/acme/index/main/index.toml
in-lockstep search tdd
in-lockstep market lint index.toml # for whoever publishes one, in their CImarket add writes .lockstep/market.toml, committed, because registering a source decides where
this repository looks for code. https only. A catalog says what to install, so over plain http
that description is whatever the network says it is, and the receipt comparison below would be
checking an attacker's document.
An entry may not carry a key that configures, the refusal pack.toml already makes. An entry's
receipt path may not leave the repository: it is untrusted input naming a file this process opens.
search groups by source and reports a name two catalogs claim rather than resolving it. Guessing
which one somebody meant is how the wrong code gets installed under the right name.
The catalog is an install-time artifact. Nothing reads it during a run, so a run of a repository that installed a pack is identical to a run of one that vendored the same class by hand.
Its receipts are falsifiable. An entry points at a receipt derived by pack describe, so it
records what the author's code did rather than what the author wrote. add re-derives it and
refuses a pack that holds more than the catalog published, outright rather than behind --accept.
That is not a decision to weigh. It is a listing that does not describe the code.
The project's own catalog is neither curated nor a bare tap. It states criteria, all machine-checked off the receipt:
- a receipt at all, derived by
pack describe; - a pack listed as
kind = "prompt"reportsimports: none; - a corpus, so the pack can be measured;
- at least one cassette, so measuring it costs nothing.
A criterion the proposal listed is not among them. The layer projection retains
guardrail:baseline cannot be answered from a pack's receipt: a prompt body has no projection
until something composes it, and which guardrails end up around it is a property of the repository
that binds it. DOC171 is where that question is answerable, and it is asked there against the bound
adapters. The same correction narrowed GATE-PACK-2.
Criteria 3 and 4 are two entries because they are two things: a corpus says what to measure, a cassette says measuring costs nothing, and a pack with cases and no recording can only be measured by somebody who pays.
The wording is load-bearing. Meeting these says a pack keeps the framework's guardrails and can be
measured before it is trusted. It says nothing whatever about whether the code is good, and a catalog
implying otherwise would transfer a judgement nobody made. That is the distinction evaluation/
already insists on when it reports an unjudged rubric as outstanding rather than as a pass.
An organisation's internal tap carries no criteria, because it answers a different question: an internal pack is trusted by the fact that someone inside the company published it.
$ in-lockstep pack try acme-tdd-pro --corpus ./our-cases
trial acme-tdd-pro (replaying — no key, no spend)
cassettes 1 (review)
its own cases (14)
decided 9 pass rate 0.89
outstanding 5 — need a judge, so not passes
your cases (22)
decided 17 pass rate 0.76
unrecorded 5 — no recorded exchange; not failures
Everything else about a pack can be checked and none of it says whether the pack is any good. The
answer to that is a measurement made on your own cases, which is why --corpus exists and why its
results are counted apart.
Four states, and keeping them distinct is the point:
| State | Means | Feeds the rate |
|---|---|---|
decided |
a machine settled it | yes |
outstanding |
a rubric, and no judge has answered | no |
unrecorded |
the cassette holds no answer for this case | no (absence of evidence, not a failure) |
unexercised |
a corpus family a trial cannot drive | no (counted, never dropped) |
Counting unrecorded against a pack would penalise an author's incomplete recording rather than
their code. Dropping unexercised silently would report a pass rate over a corpus the trial never
saw. When nothing was decided there is no pass rate, and the output says which absence produced that
rather than printing a zero.
A trial composes the pack's prompts/<aspect>.md inside the shipped layer stack, paired by
convention with corpus/review/<aspect>-reviewer/. A repository's own guardrails are deliberately
not applied: measuring a pack through your configuration measures your configuration, and two
repositories would get different numbers for the same pack with no way to tell why.
Somebody has to pay once. pack try --record calls the provider and writes the cassette everyone
after replays for nothing. Recording transmits and obeys the same egress rules as any real call;
replaying transmits nothing and AiInvoker.transmits already knew the difference, so no control
needed an exemption for measurement.
A $0 trial ranks on deterministic cases only. Rubrics need a judge and a judge costs money. This
step did not resolve that; it made it visible. A pack whose corpus is mostly rubrics shows a small
decided beside a large outstanding on every run, which is the true shape of its evidence.
No request-time selection. There is no --strategy acme/tdd-pro resolving through a catalog.
Which strategy runs stays a bind-time code decision. The CLI's existing --strategy tdd remains what
it was: a shipped-only convenience for a repository that has bound nothing.
No remote prompt bodies at run time. A pack's markdown is vendored into the environment at
install and read through importlib.resources, like every shipped body. A prompt fetched during a
run is an ungoverned input to a model holding write tools.
No pack-supplied middleware, and no pack policy that loosens. Extension packs offer classes and resources and may not contribute policy at all. A pack that wants to constrain a repository is a standards package, where contributions merge tighten-only and print with their source.
| Command | Answers |
|---|---|
show-prompt <name> [--diff|--shipped|--projection] |
what the model is told, off the bound adapter |
ls |
what is bound, and what each AI binding composes |
pack ls |
which packs are installed: offered, not in force |
pack describe [<name>] [--json|--no-load] |
the receipt, for this repository or a pack |
pack try <name> [--corpus|--record|--json] |
what it scores, on its cases and yours |
add <name> [--accept] |
accept it, record what was accepted, print the lines |
market add|ls|lint |
which catalogs this repository reads; whether one meets its own criteria |
search <query> |
packs across those catalogs, grouped by source |
doctor |
DOC170 widened capabilities · DOC171 displaced baseline · DOC172 unpinned |
Public API: in_lockstep.packs (pack, installed, Pack, pinning),
in_lockstep.receipt (receipt_for, receipt_for_pack, compare, read_record, write_record),
in_lockstep.market (sources, read_catalog, criteria_failures), in_lockstep.trial
(run, Trial). Worked examples: examples/acme-review-prompts/
and examples/lockstep-index/.
Two sections of the master design document described shapes the code does not have, and one of them would have led somebody to rebuild what §3.1 refuses. They are amended in §18 of that document, following the idiom §17 established: an "amends §X" delta rather than an edit in place, so what was designed and what shipped are both readable.
- §18.1 (amends §5.7). The strategy protocol, the registry of named approaches, and
selection by string. The registry is withdrawn permanently: resolving a strategy by name at
request time lets untrusted input name one. The
StrategySelectoris deferred rather than withdrawn, behind an invariant that makes it a quality decision instead of a privilege one. Seedesign/strategy-selection.md. - §18.2 (amends §12). The three entry-point groups that were never built, and the one that
was.
in_lockstep.adapters,in_lockstep.workflowsandin_lockstep.evaluatorsshould be struck rather than built, for the reason in §3.1. - §18.3 (amends §12). Three further §12 claims found unbuilt while checking the first two:
the import-purity lint (still wanted), the
in_lockstep.x.*incubation namespace (not wanted as stated), and the.syncmirror (wanted only when somebody needs it). Recorded rather than quietly dropped, because a design document asserting a mechanism nobody built is the decaydocs/controls-crosswalk.mdexists to catch.
GATE-PACK-1 through GATE-PACK-4 live in design/gates.md with their
assertions and held status, each discharged by a test that names it. They are not restated here: a
gate recorded in two places drifts in one of them.
DOC170 through DOC172 are in doctor, described in §4.2.
Nothing has been dogfooded. No pack has been installed from a real index. The worked example is
exercised by tests and has never been uv add-ed, and no catalog has been published at a real URL.
This is the same gap CONTRIBUTING.md records about GitLab, and the first person to publish a pack
for real will find things these tests cannot.
examples/acme-review-prompts cannot be measured. It ships prose and two corpus cases and no
cassette, so market lint fails it on criterion 4 and pack try says there is nothing to replay.
That is deliberate and documented in both places: fabricating a recorded model exchange to make a
lint green would be inventing the evidence this project exists to refuse to invent. Clearing it is
one real --record run against a real change.
pack try drives one verb. review only: a single model turn with the diff in the prompt,
which is exactly what a cassette replays. implement and fix interleave tool calls whose results a
cassette also holds, so extending it is reachable and is more work than what landed.
Open: should a catalog rank on decided count as well as pass rate? A pack with one decided case at 1.00 currently reads better than one with forty at 0.90, and nothing in the criteria distinguishes them.
Kept because the ordering encodes an argument, not just a schedule: the first two steps are worth having even if the rest is never built, and nothing before step 4 requires deciding what a marketplace is.
| Step | PR | |
|---|---|---|
| 1 | Make an override visible: show-prompt off the bound adapter, --diff, the ls block |
#118 |
| 2 | Derive the receipt: against your own repository first | #119 |
| 3 | Name the container: the pack layout, pack.toml, the entry point that offers |
#121 |
| 4 | Make installing a diff: add, the accepted record, DOC170 through DOC172 |
#122 |
| 5 | Publish the catalog: index.toml, market, search |
#123 |
| 6 | Rank by evidence: pack try |
#124 |
Three decisions were taken while drafting and all three survived implementation: installing prints
rather than writes (§4), imports is a derived fact rather than a tier (§3.3), and the project's
catalog states machine-checked entry criteria that are explicitly not an endorsement (§5.1).