Skip to content

Latest commit

 

History

History
243 lines (202 loc) · 10.1 KB

HOST-ENGINE-BOUNDARY.md

File metadata and controls

243 lines (202 loc) · 10.1 KB

Host–Relay Engine Boundary

Status: accepted. The host-facing compatibility contract is engine.ContractVersion == "2".

Decision

Reflect is the product face. It owns the terminal UI, agent loop, tool execution, permissions, conversation history, checkpoints, and product semantics. Relay is the engine. It owns credentials, provider/deployment metadata, model discovery, catalog compilation, selection, transport construction, provider request and stream normalization, resilience, normalized usage, and provider telemetry.

User
  |
  v
Reflect (face: UX, session, tools, permissions)
  |
  | stable DTOs and methods
  v
github.com/GrayCodeAI/relay/engine  [contract v2]
  |
  +--> injected secret store
  +--> injected state paths
  +--> invocation-scoped custom gateways
  +--> catalog, routing, and provider adapters
  |
  v
Provider model APIs

The dependency is one-way: Relay must not import Reflect. Reflect's integration layer may import relay/engine; Reflect command, conversation, and UI packages must not assemble Relay's catalog, provider, config, credentials, router, runtime, or setup packages.

Composition root

Reflect constructs one engine.Engine from host-owned dependencies:

e, err := engine.New(engine.Options{
    SecretStore:        store,
    StateDir:           stateDir,
    CatalogPath:        catalogPath,        // optional StateDir override
    ProviderConfigPath: providerConfigPath, // optional StateDir override
    RemoteCatalogURL:   trustedCatalogURL,  // optional; compiled HTTPS default
    CustomGateways:     gateways,           // snapshotted for this Engine
})

StateDir derives model_catalog.json and provider.json when explicit paths are absent. Explicit paths win. The store, paths, remote catalog URL, and custom gateways belong to the Engine instance; behavior does not depend on ambient Reflect paths or a process-global custom-gateway registry.

Stable contract

The facade exposes provider-neutral request, response, stream, catalog, credential, gateway, selection, health, and preflight DTOs. The principal runtime operations are:

New / StatePaths
Resolve / Generate / Stream
Catalog / RefreshCatalog / ListModels / ListLiveModels / ListPublicModels
ResolveCredential / SaveCredential / RemoveCredential / CredentialStatus
GatewayDefinitions / Gateways / SetGatewayRegion
ActiveSelection / EffectiveSelection / SetSelection / ClearSelection
CatalogHealth / ProviderStateSecurityStatus / PreflightWithOptions
MigrateProviderSecretsContext

Provider-specific wire types, authentication headers, retry behavior, and raw stream events do not cross this boundary. Model keeps distinct Owner, ProviderID, GatewayID, CanonicalID, Source, and LiveMetadata fields so Reflect does not reconstruct catalog meaning.

Frozen engine-internal types

Contract v2 is not closed over engine, llm, graph, and tools. The engine-internal symbols below are reachable from the facade through aliases, Options fields, and re-exported functions, so they are frozen as part of contract v2. Changing their names, fields, method sets, or signatures breaks hosts exactly like changing engine itself and needs a contract-version bump.

Frozen symbol Reached through
credentials.Store Options.SecretStore; SetDefaultStore / DefaultStore signatures
credentials.MapStore alias engine.MapStore (test fixture)
credentials.SetDefaultStore engine.SetDefaultStore (test fixture)
credentials.DefaultStore engine.DefaultStore (test fixture)
operationsgraph.Input alias engine.OperationsGraphInput
operationsgraph.Export alias engine.OperationsGraphExport
provider/resilience.AdaptiveRateLimitConfig Options.RateLimitConfig
provider/resilience.HeaderExtractor AdaptiveRateLimitConfig.HeaderExtractor
provider/resilience.RateLimitHeaders result of HeaderExtractor
provider/cache.CacheConfig Options.CacheConfig

Reflect uses engine.MapStore, engine.DefaultStore, engine.SetDefaultStore, engine.OperationsGraphInput, and engine.BuildOperationsGraph today, and its import-path boundary checks cannot see through the aliases. Relay therefore guards the set itself: engine/host_surface_test.go walks the exported surface of the four contract packages, follows every engine-internal symbol it reaches through struct fields, signatures, and exported methods, and fails when the reachable set differs from this table. To drop an entry, give engine its own type and bump the contract version; to add one, update the test's list and this table in the same change.

Credential-to-conversation flow

paste API key / configure gateway
  |
  v
Engine.ResolveCredential
  |
  v
Engine.SaveCredential --> injected credentials.Store
  |                         (secret material only)
  v
Engine.ApplyCredentials / ListLiveModels
  |
  +--> load provider routing from injected provider.json
  +--> resolve only the selected provider's credential aliases
  +--> pass a provider-scoped environment to its live fetcher
  +--> merge/compile catalog and atomically update model_catalog.json
  v
Engine.ListModels --> Reflect picker --> Engine.SetSelection
  |
  v
Reflect builds provider-neutral GenerateRequest from its conversation
  |
  +--> Engine.Generate
  `--> Engine.Stream --> normalized route/content/thinking/tool/usage events

Secrets never belong in catalog rows, requests, logs, diagnostics, telemetry, or host tool environments. Credential status and gateway DTOs expose only safe identifiers, booleans, and masked presentation values. Live discovery receives only the active provider's credential and routing variables, not the complete secret-store contents.

State and migration contract

provider.json contains routing metadata, never credential material. Engine mutations serialize per provider-state path, reject corrupt, unknown-version, or unknown-field state, sanitize before persistence, and replace files atomically with restrictive permissions. They fail closed instead of silently overwriting malformed state.

Legacy provider files may contain credential-shaped fields. The explicit MigrateProviderSecretsContext flow maps every recognized secret to the injected store before writing a sanitized provider file. An unmapped secret or store failure aborts the migration and restores the original state; no plaintext backup is created. Reflect should run security status/migration before normal provider-state writes.

Selection and generation

Hosts express capabilities (streaming, tools, vision, structured JSON, reasoning, and context size) plus an intent or explicit provider/model. An explicit model is a hard constraint unless the host enables fallback. Relay owns canonicalization, gateway ownership, deployment routing, and capability matching.

Relay generation is stateless from Reflect's point of view:

Reflect owns                         Relay owns
----------                        -----------
conversation history              route decision
tool permission and execution     provider transport
checkpoints and replay            retry/fallback policy
product memory                    stream normalization
session lifecycle                 normalized usage/telemetry

Stream is pull-based, cancellable, and must be closed. It emits the selected route before provider events and normalizes content, thinking, tool calls, usage, retry/continuation, TTFT, and completion. Unknown future event types are additive and must be ignored safely. Relay emits tool requests; Reflect authorizes and executes tools, appends results to its history, and begins the next model turn.

Every stream ends exactly once:

  • done — success; Err() stays nil.
  • cancelled — the request context was cancelled or its deadline passed. The event carries ErrorInfo (canceled or timeout) and the route, and Err() then reports ErrorCancelled wrapping the context's error. Hosts should treat it as the user's cancellation, not as a failure, and must not emit a second terminal for it.
  • no terminal event — Next returns false and Err() reports the provider failure with the code from its ErrorInfo (rate_limited, authentication_failed, context_exceeded, invalid_request, or provider_unavailable) and Retryable. An upstream timeout is such a retryable failure, never cancelled.

Cancelling the context releases the provider request at once, but the terminal event waits for the host to read it, so hosts must still read to the end or call Close.

Readiness

Preflight has two explicit modes:

  • Local preflight validates injected paths, catalog health, provider-state integrity, exact selected provider/model, credential presence, deployment configuration, and local transport construction without requiring network.
  • Live preflight additionally probes the selected custom gateway or performs a provider-scoped live model listing and verifies that the selected model is actually available.

These modes must not be conflated: local readiness is suitable for startup and diagnostics; live readiness is an explicit network operation.

Release and sibling-repository order

The boundary is delivered Relay-first:

1. Change and verify standalone Relay
2. Commit Relay and publish a resolvable release/commit
3. Update Reflect's Relay module version when required
4. Update Reflect's Relay module pin to that exact published commit
5. Verify Reflect integration, boundary checks, and clean-clone/module builds
6. Commit the Reflect module-pin update in Reflect's repository

Reflect must never depend on an uncommitted Relay worktree. The parent workspace uses a sibling checkout for source identity during local development; a resolvable published module version is also required for workflows that build Reflect with GOWORK=off.

Compatibility policy

Lower-level Relay packages remain public for non-Reflect consumers and staged migration, but they are not part of Reflect's product boundary, apart from the frozen symbols listed above. Additive fields and stream events are allowed within contract v2. Removing or changing stable DTO semantics requires a contract-version and semantic-version boundary.