|
| 1 | +# Tier 7 Phase 3 — HirBuilder Class State Extraction Spec |
| 2 | + |
| 3 | +**Status:** ACTIVE (pre-launch). Authored 2026-04-23 per pythia #89 |
| 4 | +2026-04-23 + supervisor 2026-04-23T21:17:25Z commitment ("Phase 3 |
| 5 | +launch GATED on theologian-authored spec doc landing first") + |
| 6 | +supervisor 2026-04-23T22:57:16Z disposition. |
| 7 | + |
| 8 | +**Owner:** theologian (this spec) + generalist (implementation). |
| 9 | + |
| 10 | +**Estimated cost:** 5-10 sessions per Tier 7 spec §2.2 builder.cpp |
| 11 | +estimate. Multi-week unknown. |
| 12 | + |
| 13 | +**Pre-Phase-3 prerequisite:** Phase 1 ORIGINAL SCOPE 100% COMPLETE |
| 14 | +(23 emit-methods/helpers eliminated, builder.cpp 4900→4732, -257L |
| 15 | +cumulative, 16 pushes today). Pythia #89 spawn-vs-drain-on-Phase-3 |
| 16 | +concern addressed by this spec. |
| 17 | + |
| 18 | +--- |
| 19 | + |
| 20 | +## 1. Scope |
| 21 | + |
| 22 | +### 1.1 In-scope |
| 23 | + |
| 24 | +Convert HirBuilder C++ class state + algorithmic methods to pure C |
| 25 | +via state-struct extraction. Preserve external interface (callers in |
| 26 | +compiler.cpp + tests) via thin C++ shim that delegates to |
| 27 | +`hir_builder_state_c` struct + C functions. |
| 28 | + |
| 29 | +### 1.2 Out-of-scope |
| 30 | + |
| 31 | +- Emit-methods (Phase 1 burndown — 100% COMPLETE) |
| 32 | +- Inner Function classes (PreloaderManager, etc.) — separate workstream |
| 33 | +- TranslationContext (callee class, separate scope) |
| 34 | +- compiler.cpp call sites — minimum-touch interface preservation |
| 35 | + |
| 36 | +--- |
| 37 | + |
| 38 | +## 2. Architectural state inventory |
| 39 | + |
| 40 | +### 2.1 HirBuilder class data members (builder.h:603-683) |
| 41 | + |
| 42 | +| Member | Type | Purpose | Extraction class | |
| 43 | +|---|---|---|---| |
| 44 | +| `code_` | `PyCodeObject*` | input code object | A — opaque pointer | |
| 45 | +| `block_map_` | `BlockMap` | block lookup map | B — heavy C++ container | |
| 46 | +| `exception_table_` | `std::vector<ExceptionTableEntry>` | parsed handlers | B — STL container (CLOSED via Batch 2 1343895045 Class B-kept) | |
| 47 | +| `pending_b2_blocks_` | `std::vector<PendingBlock>` | (DEAD per Batch 3 grep — W26-era residual) | DELETED Batch 3 | |
| 48 | +| `preloader_` | `const Preloader&` | preload info | A — opaque ref | |
| 49 | +| `current_func_` | `Function*` | current HIR fn | A — opaque pointer | |
| 50 | +| `temps_` | `TempAllocator` | register allocator | B — value-type C++ class | |
| 51 | +| `func_` | `Register*` | function reg | A — opaque pointer | |
| 52 | +| `kwnames_` | `Register*` | KW_NAMES constant | A — opaque pointer | |
| 53 | +| `static_method_stack_` | `OperandStack` | static method stack | B — value-type C++ class | |
| 54 | + |
| 55 | +**Class A (5 members):** opaque pointers/refs. Direct void* equivalent |
| 56 | +in C struct. ZERO extra bridges. |
| 57 | + |
| 58 | +**Class B (5 members nominally, 4 live post Batch 3 dead-state delete):** |
| 59 | +heavy C++ containers/value-types. Need either (1) keep on C++ side via |
| 60 | +opaque pointer, or (2) full C-port of container to PhxArray/equivalent. |
| 61 | + |
| 62 | +**AMENDMENT per pythia #93 2026-04-23 (§2.1 errata + grep-writers |
| 63 | +discipline):** Batch 3 Step A discovered pending_b2_blocks_ was DEAD |
| 64 | +STATE (zero writers; W26-era residual from emitInlineExceptionMatch |
| 65 | +refactor). The original §2.1 inventory categorized by TYPE |
| 66 | +(`std::vector<PendingBlock>`) without grepping for writers. Standing |
| 67 | +discipline: any Class B candidate MUST grep writers |
| 68 | +(`push_back\|emplace_back\|insert\|operator\[\]` mutation) BEFORE |
| 69 | +Step A scoping. Type-only categorization is insufficient — dead state |
| 70 | +masquerades as Class B until grep-verified. Apply to remaining Class B |
| 71 | +candidates: block_map_, temps_, static_method_stack_. |
| 72 | + |
| 73 | +### 2.2 HirBuilder internal struct types (builder.h:608-636) |
| 74 | + |
| 75 | +- `ExceptionTableEntry` (5 fields, POD: BCOffset×3 + int + bool) |
| 76 | + → C struct trivial |
| 77 | +- `PendingBlock` (2 fields: BasicBlock* + FrameState) |
| 78 | + → C struct + opaque pointers |
| 79 | +- `SimpleExceptInfo` (3 fields: int + PyObject* + BCOffset) |
| 80 | + → C struct trivial |
| 81 | + |
| 82 | +All 3 internal structs are POD-equivalent and can be moved to C as |
| 83 | +plain structs. |
| 84 | + |
| 85 | +### 2.3 HirBuilder algorithmic methods |
| 86 | + |
| 87 | +Multi-method (at least): translate(), checkTranslate(), |
| 88 | +parseExceptionTable(), findExceptionHandler(), |
| 89 | +getSimpleExceptInfo(), emitInlineExceptionMatch(), |
| 90 | +emitCallExceptionHandler(), advancePastYieldInstr(). |
| 91 | + |
| 92 | +Plus public Run() / Compile() entry points (callers in compiler.cpp). |
| 93 | + |
| 94 | +Total: ~10-15 algorithmic methods (post Phase 1 emit-method removal). |
| 95 | + |
| 96 | +--- |
| 97 | + |
| 98 | +## 3. Extraction strategy |
| 99 | + |
| 100 | +### 3.1 State struct definition (proposed) |
| 101 | + |
| 102 | +`Python/jit/hir/builder_state_c.h`: |
| 103 | + |
| 104 | +```c |
| 105 | +typedef struct PhxHirBuilderState { |
| 106 | + void *code; /* PyCodeObject* */ |
| 107 | + void *block_map; /* opaque BlockMap* (Class B kept C++) */ |
| 108 | + void *exception_table; /* opaque std::vector<...>* */ |
| 109 | + void *pending_b2_blocks; /* opaque std::vector<...>* */ |
| 110 | + const void *preloader; /* const Preloader& */ |
| 111 | + void *current_func; /* Function* */ |
| 112 | + void *temps; /* opaque TempAllocator* */ |
| 113 | + void *func; /* Register* */ |
| 114 | + void *kwnames; /* Register* */ |
| 115 | + void *static_method_stack; /* opaque OperandStack* */ |
| 116 | +} PhxHirBuilderState; |
| 117 | +``` |
| 118 | + |
| 119 | +ZERO new C-API surface vs existing builder_emit_c.c bridges. Class A |
| 120 | +members are direct void* fields; Class B members are opaque pointers |
| 121 | +to C++-owned objects (lifetime via C++ ctor/dtor through bridge |
| 122 | +functions). |
| 123 | + |
| 124 | +### 3.2 C++ shim pattern |
| 125 | + |
| 126 | +`builder.h` keeps thin C++ shim: |
| 127 | + |
| 128 | +```cpp |
| 129 | +class HIRBuilder { |
| 130 | +public: |
| 131 | + HIRBuilder(PyCodeObject* code, ...); |
| 132 | + std::unique_ptr<Function> Run(); |
| 133 | +private: |
| 134 | + PhxHirBuilderState state_; // owned, opaque to C++ except via bridges |
| 135 | +}; |
| 136 | +``` |
| 137 | +
|
| 138 | +Method bodies in builder.cpp become 1-line delegations to |
| 139 | +`hir_builder_state_*_c` functions taking `&state_`. |
| 140 | +
|
| 141 | +### 3.3 Bridge inventory (estimate) |
| 142 | +
|
| 143 | +- `hir_builder_state_create(code, preloader)` — ctor |
| 144 | +- `hir_builder_state_destroy(state)` — dtor |
| 145 | +- `hir_builder_state_run(state) -> Function*` — main entry |
| 146 | +- `hir_builder_state_check_translate(state) -> bool` — checkTranslate |
| 147 | +- `hir_builder_state_parse_exception_table(state)` — parseExceptionTable |
| 148 | +- `hir_builder_state_find_exception_handler(state, off) -> entry*` — findExceptionHandler |
| 149 | +- `hir_builder_state_get_simple_except_info(state, handler, &info) -> bool` |
| 150 | +- `hir_builder_state_emit_inline_exception_match(state, ...)` |
| 151 | +- `hir_builder_state_emit_call_exception_handler(state, ...)` |
| 152 | +- `hir_builder_state_advance_past_yield_instr(state, tc)` |
| 153 | +- Plus container ops for Class B members (~5-10 bridges): |
| 154 | + block_map_lookup, exception_table_push, pending_b2_blocks_push, etc. |
| 155 | +
|
| 156 | +Total estimate: ~15-20 new bridges. Within ≤5/batch W25b discipline |
| 157 | +if batched as 4-5 batches. |
| 158 | +
|
| 159 | +--- |
| 160 | +
|
| 161 | +## 4. Test plan / falsifier strategy |
| 162 | +
|
| 163 | +### 4.1 Per-batch differential JIT_DCHECK |
| 164 | +
|
| 165 | +Same pattern as Phase A C-ports (per memory |
| 166 | +`project_wiring_methodology.md`): for each method ported, embed |
| 167 | +JIT_DCHECK comparing C-port output vs C++ baseline output on |
| 168 | +parallel run. Detect divergence at compile time. |
| 169 | +
|
| 170 | +### 4.2 W45 fixtures for new bridges |
| 171 | +
|
| 172 | +Per W45 §2.2: add fixture for each new state-struct bridge to |
| 173 | +`scripts/w45_bridge_drift_falsifier.sh`. Insert-sentinel-param |
| 174 | +mutation must trigger build failure at C++ shim call site. |
| 175 | +
|
| 176 | +### 4.3 Phase-3-specific falsifier (NEW) |
| 177 | +
|
| 178 | +Add `scripts/phase3_state_extraction_falsifier.sh` (analogous to |
| 179 | +W45): |
| 180 | +- Mutate one PhxHirBuilderState field (reorder, type-change) |
| 181 | +- Verify build OR runtime test detects divergence |
| 182 | +- Restore field; build PASS |
| 183 | +
|
| 184 | +### 4.4 Existing gates (preserved) |
| 185 | +
|
| 186 | +- W21 golden trip-wire (codegen output comparison) |
| 187 | +- W44 DO-NOT-USE caller gate |
| 188 | +- 7-test Phoenix JIT suite (controlflow + comparisons + autocompile + |
| 189 | + await + repro_min3 + partial_conversions + W44) |
| 190 | +- ARM64 dual-arch tree-match per CLAUDE.md |
| 191 | +- pydebug refleak gate (per memory `feedback_no_workarounds.md`) |
| 192 | +
|
| 193 | +--- |
| 194 | +
|
| 195 | +## 5. Open questions / decision points |
| 196 | +
|
| 197 | +1. **Class B members C++-kept-vs-C-port choice.** State spec proposes |
| 198 | + keeping (block_map_, exception_table_, pending_b2_blocks_, temps_, |
| 199 | + static_method_stack_) as opaque-C++-owned. Alternative: full |
| 200 | + C-port of containers via PhxArray. Cost vs benefit per per-member |
| 201 | + complexity. Defer per-member call to per-batch Step A. |
| 202 | +
|
| 203 | + **AMENDMENT per pythia #92 2026-04-23:** Batch 2 Class B disposition |
| 204 | + decision is FORCED, not deferred — Batch 1's PhxHirBuilderState |
| 205 | + shape (Class A subset only) becomes precedent. To avoid |
| 206 | + Class-A-only-shape calcification (architecture-by-accretion risk), |
| 207 | + Batch 2 MUST re-shape struct if needed before Batch 5 lands. |
| 208 | + Otherwise per-method Class B disposition (Batches 3-5) becomes |
| 209 | + constrained by an unintended Batch 1 design choice. |
| 210 | +
|
| 211 | + **CLOSURE AMENDMENT per pythia #103 2026-04-24 + supervisor 2026- |
| 212 | + 04-24T00:59:39Z hybrid disposition:** Phase 3 closed with all 5 |
| 213 | + Class B members disposed via keep-bias (4 closed Class-B-kept + |
| 214 | + 1 deleted dead-state). The `_cpp` bridge surface (~6 _cpp + 2 _c |
| 215 | + algorithmic + 1 init = 9 extern surfaces) is the substantive Phase |
| 216 | + 3 product. Class B-kept is **TRANSITIONAL DESIGN CHOICE**, NOT |
| 217 | + FINAL — the keep-arm of §5 forcing-decision was validated; the |
| 218 | + migrate-arm (port to pure C) remains untested by Phase 3 alone. |
| 219 | +
|
| 220 | + Earlier draft framing 'Class B-kept is FINAL disposition for |
| 221 | + HirBuilder state' (theologian 23:35:58Z + 23:49:00Z, supervisor |
| 222 | + 00:01:35Z + 00:14:45Z) is WITHDRAWN per pythia #103 cathedral- |
| 223 | + scaffold critique. The 'FINAL' framing would calcify the |
| 224 | + transitional state and contradict MEMORY.md ZERO-C++ terminal |
| 225 | + goal. |
| 226 | +
|
| 227 | + Replacement framing: keep-bias is design choice for Phase 3 |
| 228 | + tractability (PhxArray-equivalent C-port for std::vector / |
| 229 | + unordered_map / Stack / TempAllocator is multi-session-per- |
| 230 | + container scope); Tier 8 §5 migrate-arm pilot |
| 231 | + (`docs/tier8-class-b-cport-migrate-arm-spec.md`, theologian |
| 232 | + 01:01:50Z, supervisor 01:02:46Z ADOPTED) validates the migrate- |
| 233 | + arm by porting `exception_table_` (POD-equivalent per §2.2) to |
| 234 | + PhxArray. Tier 8 acceptance #10 requires net-subtract ≥ Phase 3 |
| 235 | + foundation cost (+257L) across all Class B containers (full |
| 236 | + migration projects net-negative). |
| 237 | +
|
| 238 | + Phase 3 outcome: foundation laid + bridge surface enumerated + |
| 239 | + §5 keep-arm fully validated. Phase 3 is not end-state for |
| 240 | + HirBuilder; Tier 8 pilot is the substantive next-step toward |
| 241 | + ZERO-C++. |
| 242 | +
|
| 243 | +2. **builder.cpp shim file size.** Phase 3 may leave a thin |
| 244 | + builder.cpp shim (~100-200 lines of C++ delegations). Acceptable |
| 245 | + per Tier 7 spec §1.2 intentional residue framing? Or aim for full |
| 246 | + C migration with compiler.cpp caller-rewrite? |
| 247 | +
|
| 248 | +3. **HirBuilder ownership semantics.** `std::unique_ptr<Function> |
| 249 | + Run()` returns owned Function. C-side equivalent needs ownership |
| 250 | + discipline (PhxArray-style? caller responsibility?). Defer per |
| 251 | + per-batch. |
| 252 | +
|
| 253 | +--- |
| 254 | +
|
| 255 | +## 6. Acceptance criteria |
| 256 | +
|
| 257 | +Phase 3 closure requires ALL: |
| 258 | +
|
| 259 | +1. PhxHirBuilderState struct landed + C++ HIRBuilder class uses it |
| 260 | + exclusively for state. |
| 261 | +2. All ~10-15 HirBuilder methods converted (or honestly framed as |
| 262 | + intentional residue per Tier 7 §1.2). |
| 263 | +3. ZERO new bridges beyond the ~15-20 estimate (per W25b ≤5/batch |
| 264 | + budget across 4-5 batches). |
| 265 | +4. W45 fixtures cover every new bridge. |
| 266 | +5. Differential JIT_DCHECK passes for all C-ports. |
| 267 | +6. Full 7-test Phoenix JIT suite + W44 gate + W22 cluster + dual-arch |
| 268 | + tree-match clean. |
| 269 | +7. compiler.cpp callers unaffected (interface preservation |
| 270 | + verified). |
| 271 | +8. builder.cpp final size ≤500L (thin shim) OR ≤200L (deleted) |
| 272 | + depending on Class B disposition. |
| 273 | +9. **Per-commit benchmark gate** (per CLAUDE.md + supervisor |
| 274 | + 2026-04-23T22:59:54Z addendum): every Phase 3 commit runs |
| 275 | + 4-bench (`--compile=auto --reps=3`) against pinned baseline. |
| 276 | + Geo-mean must stay ≥1.0x parity floor (per Alex hard-floor |
| 277 | + directive 2026-04-14). Phase 3 touches HirBuilder hot path; |
| 278 | + per-commit benchmark gate is non-optional. Block push if |
| 279 | + geo-mean drop >5% (per CLAUDE.md ≥5% BLOCK threshold). |
| 280 | +
|
| 281 | + **AMENDMENT per pythia #92 2026-04-23:** Cross-session ABBA |
| 282 | + variance is ±7% per memory `feedback_abba_cross_session.md`. Phase |
| 283 | + 3 spans many sessions ('multi-week unknown'). Same-session ABBA |
| 284 | + per Phase 3 batch is reliable; cross-session comparison is not. |
| 285 | + At session-start, re-baseline 4-bench against pinned baseline |
| 286 | + before any Phase 3 batch lands; within-session compare commit-to- |
| 287 | + commit. Cross-session geo-mean drift up to 7% is noise, not |
| 288 | + regression — block only if same-session drift exceeds 5%. |
| 289 | +
|
| 290 | + **AMENDMENT per pythia #93 2026-04-23 (per-bench floor):** |
| 291 | + Geo-mean alone can mask single-bench regressions when |
| 292 | + high-multiple benches dominate (Batch 2 fibonacci 2.29x + |
| 293 | + nqueens 1.63x absorb gen_simple-class drops silently). Per-commit |
| 294 | + gate now requires ALL of: |
| 295 | + - Geo-mean ≥1.0x absolute parity floor (per Alex hard-floor) |
| 296 | + - Geo-mean drop ≤5% same-session vs prior commit (CLAUDE.md) |
| 297 | + - **No single bench drops >20% from prior commit (same-session)** |
| 298 | + - **No single bench drops below 0.5x absolute parity** (catches |
| 299 | + workload-class degradation that geo-mean smooths over) |
| 300 | + Per-bench thresholds are ABSOLUTE in same-session ABBA only; |
| 301 | + cross-session per-bench drift is noise per pythia #92 amendment. |
| 302 | +
|
| 303 | +--- |
| 304 | +
|
| 305 | +## 7. Cross-link |
| 306 | +
|
| 307 | +- Pythia framing: pythia #89 2026-04-23 (Phase 3 launch gating) |
| 308 | +- Supervisor commitment: 2026-04-23T21:17:25Z + 2026-04-23T22:57:16Z |
| 309 | +- Pattern precedent: W25b/W26 spec docs (state-struct extraction) |
| 310 | +- Methodology: `project_wiring_methodology.md` (differential |
| 311 | + JIT_DCHECK) |
| 312 | +- Sibling falsifiers: W42 (refcount-correctness verifier), W44 |
| 313 | + (DO-NOT-USE caller gate), W45 (bridge-signature drift) |
| 314 | +- Memory: `feedback_grep_before_counts.md` (count-discipline at |
| 315 | + spec write-time + cite time) |
| 316 | +- Phase 1 close: 16 pushes today, builder.cpp 4900→4732, -257L |
| 317 | + cumulative, all 23 emit-methods/helpers eliminated |
| 318 | +- Tier 7 spec §1: docs/tier7-runtime-burndown-spec.md (in-scope file |
| 319 | + list + 3-class load-bearing-infrastructure framing) |
0 commit comments