feat: export OpenTelemetry traces and metrics over OTLP - #74

Summary

Every CLI command can now export the harness's spans and metrics over OTLP/HTTP using Effect's built-in exporter (effect/observability). No OpenTelemetry SDK packages are added.

• Configuration is env only. OTEL_EXPORTER_OTLP_ENDPOINT turns it on. The standard variables cover headers, per-signal endpoints, protocol (http/protobuf by default, http/json), OTEL_SDK_DISABLED, OTEL_SERVICE_NAME and OTEL_RESOURCE_ATTRIBUTES. Without an endpoint nothing is exported, and a configuration that fails to build runs the command without export.

• Only high-level spans are exported by default. All spans default to Debug and export starts at Info. The command root, the agent loop (Loop.run → runTurn → runTurnAttempt → settleTools → handleTool), LLM.run, and sandbox create/wake are marked Info. A run exports about 11 spans instead of about 490, which were mostly Effect's sql.execute, file checks and event publishes. CODEWORK_TRACE_LEVEL=debug exports every span again. Metrics are unaffected.

• Spans (LLM.run client span, Loop.handleTool, Loop.run) carry the OpenTelemetry GenAI attributes: provider, model, session (gen_ai.conversation.id), response id, finish reason, token counts and time to first chunk. A failed provider request ends its span with an error status. Prompts, file contents and model output are never exported.

• Metrics use codework's own names, because the GenAI metric conventions are still in development and keep renaming:

  Metric	Type	Labels
  codework.llm.duration	histogram, s	provider, model, error
  codework.llm.ttft	histogram, s	provider, model
  codework.llm.tokens	counter	provider, model, type (input, output, cache_read, cache_write, reasoning)
  codework.llm.cost	counter, USD	provider, model
  codework.agent.duration	histogram, s	error
  codework.tool.duration	histogram, s	tool, error
  codework.sandbox.provision.duration	histogram, s	driver, operation, error
  codework.sandbox.exec.duration	histogram, s	driver, error
  codework.sandbox.active	gauge	driver
  child_fibers_*	Effect's fiber runtime metrics	none

• Resource attributes: service.instance.id, deployment.environment.name (the release channel) and codework.command.

• aikit is unchanged. All data comes from its terminal assistant message.

Notes

• Exporter default. Effect 4.0.1 only exports when OTEL_{TRACES,METRICS}_EXPORTER explicitly lists otlp. The spec default is otlp, so the CLI supplies that default; anything set in the environment still wins.
• Units. Effect reads a metric's unit from its unit attribute, so unit also appears as a data point attribute.
• Unreachable collector. The CLI exits about 3 s later while Effect's exporter retries; OTEL_EXPORTER_OTLP_TIMEOUT shortens that.
• Remote-only metrics. codework.sandbox.provision.duration and codework.sandbox.active only record for remote and managed sandboxes, so the local E2E test doesn't cover them.

Test plan

• New E2E packages/codework/test/telemetry.test.ts:
  • Spawns the real CLI against a local fake OpenAI-compatible provider that is also the OTLP collector.
  • The provider fails the first request with a 503, then returns one bash call, then a text reply.
  • Asserts the auth header, a single consistent resource, and no prompt, command, file or model content in any exported trace.
  • Pins the default exported span tree: only the Info spans, no sql.*.
  • Snapshots spans and per-series metric counts and values to __artifacts__/telemetry.json. Stable over 8 runs.
• Checked by hand: protobuf is the default content type, grpc exports nothing, an invalid endpoint is ignored.
• vp check, vp run typecheck, harness and codework test suites.

🤖 Generated with Claude Code