Human Influence Telemetry is a design-science research artifact that operationalizes substantive human influence as observable properties of institutional decision records.
- Current repository release:
0.6.7, published on GitHub and Zenodo; see the publication receipt - Human-result release:
0.6.0 - Software concept DOI:
10.5281/zenodo.21446141 - Exact-version DOI:
10.5281/zenodo.23252877 - Version-specific software DOI for v0.6.6:
10.5281/zenodo.23226713 - Version-specific software DOI for v0.6.5:
10.5281/zenodo.21864224 - Previous version DOI for v0.6.4:
10.5281/zenodo.21446142 - Originating research DOI:
10.5281/zenodo.21204892 - Normative assessment contract:
0.4.0 - Conformance engine:
0.5.0 - Research maturity: Level 2, Applicable
- Active empirical protocol:
HIT-IRP-HIT040-002, candidate, scoring prohibited - Stable public-contract target:
1.0.0, gated candidate, release prohibited
Version 1.0.0 is a future compatibility and public-implementability claim. It does not become an empirical reliability claim unless new eligible independent human evidence separately satisfies the active protocol.
Version 0.6.6 distributes supplementary development work and release-readiness controls. The published 0.6.7 package adds a separate Ofqual author-review supplement, with five bounded substantive findings and both limited integrity states accepted. Counsel and the overall comparison remain unresolved; the original assistant assessment stays preserved. The package also includes qualitative reanalysis, source-binding checks, a synthetic evidence-update rehearsal, and a working manuscript with three reproducible exhibits.
Both assistant responses identified the four injected changes in the synthetic rehearsal, with no scored difference. This small, highly cued exercise establishes no comparative advantage or measured human usefulness. Publication creates no new independent ratings, H3 replication, H9 implementation result, institutional outcome finding, or maturity advancement. The v0.6.5 audit below remains limited to its mapped claims; new interpretations require their own support and publication review.
Version 0.6.5 maps H1 through H9 and four material paper claims to exact evidence locators, five gate states, five evidence-fitness dimensions, dependencies, limitations, reversal conditions, and conclusion eligibility. The generated audit state is PASS_WITH_EXCEPTIONS because every deterministic control passes and every unresolved claim remains blocked.
Conclusion-eligible claims are limited to H1, H2, H3, PAPER-C01, PAPER-C02, and PAPER-C03. H4 through H9 and PAPER-C04 remain provisional, blocked, unsupported, or pending according to the machine-readable map. The state adds research-integrity controls. It adds no E5 evidence.
Current-contract external-rater replication remains separate and unresolved under HIT-IRP-HIT040-002. The candidate does not authorize recruitment, scoring, H3 generalization, or Maturity Level 3.
Can independent reviewers use HIT to distinguish substantive human influence, ceremonial oversight, affirmative absence, and insufficient evidence in AI-mediated institutional decision processes?
| Class | Meaning |
|---|---|
| E1 | Executable evidence from validation, deterministic fixtures, or comparison tooling |
| E2 | Artifact evidence contained in specifications, schemas, records, assessments, or locked protocols |
| E3 | External source evidence supporting a construct or case proposition |
| E4 | Comparative inference connecting HIT to adjacent governance systems |
| E5 | Empirical evidence from independent application, field data, or statistical testing |
| ID | Claim | Current status | Update condition |
|---|---|---|---|
| H1 | HIT can represent six substantive dimensions and split Telemetry Integrity in a machine-readable assessment | Demonstrated structurally by the 0.4.0 schema, catalog, canonical example, and CI | Revise when a required construct cannot be represented |
| H2 | The evidence-state and 0/1/2/IE rules distinguish affirmative absence, ceremonial presence, substantive exercise, and missing evidence |
Demonstrated for 48 synthetic boundary cases; independently applied under the preserved 0.1.0 scorer contract | Requires replication under a compatible current contract and cases with category variation |
| H3 | Independent reviewers can reach the predeclared agreement threshold using the published materials | Supported for one frozen Cigna packet: 7 of 7 exact agreements and zero critical disagreements | Replicate across additional cases, sectors, scorer populations, and a compatible current contract |
| H4 | HIT can expose oversight theater that binary human-in-the-loop labels miss | Supported by design logic, historical cases, and convergent ceremonial findings in the first human exercise | Requires external replication under a compatible contract |
| H5 | Higher HIT findings predict lower harm or better repair | Hypothesis | Requires preregistered population, outcomes, and statistical method |
| H6 | A high HIT finding proves meaningful human control in a causal or legal sense | Unsupported | Would require evidence beyond documentary assessment |
| H7 | HIT evidence may support external oversight, accountability, and contestability obligations | Provisional mapping claim | Requires qualified standards or legal review and DEAS-style applicability analysis |
| H8 | HIT is independently adopted | Not yet | Requires use by an institution outside the author's control |
| H9 | The public HIT package can be installed, operated, and explained without private author interpretation | Pending | Requires a completed clean-room implementation audit under HIT-CRI-V100-001 |
Release 0.5.0 established E1 evidence about internal representation, rejection behavior, determinism, and compatibility controls through:
- one canonical schema-valid synthetic assessment under contract
0.4.0; - 48 deterministic rubric boundary cases;
- one valid and fifteen invalid complete-record vectors;
- reusable semantic checks across actors, claims, findings, Repair triggers, integrity, aggregation, and citations;
- deterministic reports;
- compatibility metadata;
- protected migration planning;
- CLI smoke tests and exact-head validation.
Release 0.6.4 adds a public standalone software archive and version-specific DOI. This is provenance and citation evidence, not new E5 evidence.
Release 0.6.5 adds claim-evidence integrity and paper controls. This is E1 and E2 research-integrity evidence, not new E5 evidence.
Post-0.6.5 readiness work includes candidate controls for the current-contract replication, manual scorer workbooks, the 1.0.0 gate ledger, and the clean-room implementation packet. These are E1 and E2 readiness artifacts. They do not create new E5 evidence.
Release 0.6.0 adds the first E5 result.
| Field | Result |
|---|---|
| Protocol | HIT-IRP-CIGNA-001, locked version 1.0.0 |
| Frozen packet | HIT-IR-CIGNA-PXDX-001 |
| Scorer contract | Specification and schema 0.1.0 |
| Eligible independent scorers | 2 of 2 |
| Verified locked submissions | 2 of 2 |
| Compared items | 7 |
| Exact agreements | 7 |
| Exact-agreement proportion | 1.0000 |
| Required threshold | At least 6 of 7 and zero critical disagreements |
| Critical disagreements | 0 |
| Supplementary Cohen's kappa | null, because all six substantive ratings used one category |
| H3 decision | Supported for this bounded exercise |
| Maturity decision | Level 2, Applicable |
| Result release | 0.6.0 |
Both scorers assigned 1 to all six substantive dimensions and limited to Telemetry Integrity. The scorer submissions used different primary source references and different analytical language, but those differences did not produce categorical disagreement.
The result establishes successful independent application under one frozen packet. It does not estimate general inter-rater reliability.
| Field | Current state |
|---|---|
| Protocol | HIT-IRP-HIT040-002 |
| Contract | 0.4.0 |
| Planned cases | 3 |
| Planned scorers | 3 |
| Planned submissions | 9 |
| Case selection | Pending signed human decision |
| Packet IDs | None assigned |
| Recruitment | Prohibited |
| Scoring | Prohibited |
| Result claim | None |
The candidate protocol, source-audit controls, recruitment contingency, and draft manual workbooks are procedural infrastructure. They do not support H3 replication or Maturity Level 3 until the declared human-evidence conditions are met.
| Case | Unit of analysis | Historical assessments | Current disposition |
|---|---|---|---|
| Dutch childcare-benefits scandal | Belastingdienst/Toeslagen harm-period architecture | 1 | historical_version_bound |
| Obermeyer population-health algorithm | Deploying health systems and manufacturer | 2 | historical_version_bound |
| Cigna PxDx | Cigna claims-review workflow | 1 | protocol_completed_historical_version_bound |
The four author-scored assessments remain immutable 0.1.0 artifacts. The human exercise does not convert the historical Cigna assessment into a 0.4.0 assessment.
The research program uses artifact construction, normative specification, JSON Schema validation, deterministic fixtures, complete-record mutation testing, retrospective public-record case studies, blinded inter-rater testing, comparative boundary analysis, clean-room implementation review, and prospective validation when institutional access becomes available.
The locked inter-rater design predeclared eligibility, blinding, source boundaries, agreement threshold, critical disagreements, supplementary kappa, preservation of original scores, and publication of passing and failing results.
The clean-room implementation design tests whether a technically competent external reviewer can install, operate, and explain the public package without private author explanation. That audit tests public implementability, not inter-rater reliability.
- Institutional records may be incomplete, curated, or controlled by the governed system.
- Recorded reasons may be post-hoc rationalizations.
- One observed exercise may not represent the dominant architecture.
- Retrospective public cases have selection bias.
- Public records may not expose internal access or authority.
- Findings compress variation within each dimension.
- Reviewers may apply normative thresholds differently in other cases.
- Litigation records vary by procedural posture.
- Framework comparisons may overstate equivalence.
- Synthetic fixtures demonstrate deterministic design choices, not empirical correctness.
- Complete-record mutation tests demonstrate rejection behavior, not evidence truth.
- The first human exercise has two scorers, seven compared items, and one retrospective insurance case.
- All six substantive ratings used one category, so Cohen's kappa is undefined and category discrimination was not tested empirically.
- The scorer exercise used the preserved
0.1.0contract, not the current0.4.0contract. - A passing first exercise demonstrates reproducibility under one frozen packet, not general reliability.
- Candidate
1.0.0readiness artifacts have not yet passed a clean-room implementation audit. - Stable semantic versioning cannot substitute for missing empirical validation.
| Level | Name | Entry criterion | Current status |
|---|---|---|---|
| 1 | Defined | Public constructs, machine-readable contract, executable fixtures, citable archive | Complete |
| 2 | Applicable | A stranger can apply the method and the locked human exercise passes its declared independence and agreement rules | Current |
| 3 | Evidenced | Level 2 plus at least three public applications under a compatible contract and the active protocol's evidence rule | Pending |
| 4 | Validated | Preregistered prospective design and pilot result | Pending |
| 5 | Adopted | Independent institutional use and external reference | Pending |
The levels are cumulative. Release 0.6.0 advances research maturity to Level 2 because the locked exercise passed. Releases 0.6.4 and 0.6.5 improve archival provenance and research-integrity controls without changing maturity.
A future 1.0.0 release may remain at Level 2 when the stable-contract gates pass before the current-contract human replication is complete. The release must state the exact maturity and H3 evidence available at publication time.
A numbered release requires:
- passing exact-commit validation;
- synchronized repository metadata and component-version boundaries;
- current limitations and claim status;
- migration or compatibility records;
- accurate citation metadata;
- preserved historical and locked artifacts;
- explicit adjacent-system non-claims;
- public release assets matching the recorded SHA-256 digest.
A future 1.0.0 release additionally requires a completed 0.9.0 release candidate, clean-room implementation review, synchronized stable components, breaking-change review, and every gate in docs/v1-readiness-plan.md.
Candidate documents, branch names, and merged readiness controls do not authorize tagging, release publication, scorer activation, DOI archival, or maturity advancement.