Skip to content

Keep OTLP configuration out of AWF agent sandboxes - #67242

Merged
pelikhan merged 10 commits into
mainfrom
copilot/fix-otel-exporter-otlp-headers
Oct 10, 2026
Merged

pelikhan merged 10 commits into
mainfrom
copilot/fix-otel-exporter-otlp-headers

Conversation

Copilot AI commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

With OTLP configured, awf --env-all passed collector headers and endpoints into the agent and threat-detection sandboxes. This exposed collector credentials and could cause the Copilot CLI to export duplicate telemetry.

  • Sandbox isolation: Exclude OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, and GH_AW_OTLP_ENDPOINTS from both AWF invocations when OTLP is enabled. The values remain available to host-side exporters.
  • Generated workflows: Regenerate affected lock files so their AWF commands include the exclusions.
  • Regression coverage: Cover agent and detection commands, including workflows without OTLP.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix credential exposure through environment variables Keep OTLP configuration out of AWF agent sandboxes Oct 9, 2026
Copilot AI requested a review from pelikhan October 9, 2026 17:42
@pelikhan
pelikhan marked this pull request as ready for review October 9, 2026 18:41
Copilot AI balanced review requested due to automatic review settings October 9, 2026 18:41
@pelikhan

pelikhan commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

@copilot update compiler threat spec

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Evals sandboxes and legacy AWF pins can still receive OTLP credentials through --env-all.

1 open finding
What changed in this PR

This PR prevents OTLP collector credentials and endpoints from entering AWF agent and threat-detection sandboxes.

Changes:

  • Adds OTLP variables to --exclude-env.
  • Propagates exclusions to threat detection.
  • Adds regression tests and regenerates affected workflows.

Review found unresolved gaps for evals sandboxes and AWF versions older than v0.25.3.

File Description
pkg/​workflow/​awf_command_builder.go Adds OTLP exclusions to AWF commands.
pkg/​workflow/​threat_detection_external.go Propagates exclusions to detection.
pkg/​workflow/​awf_env_test.go Tests agent and detection arguments.
pkg/​workflow/​threat_detection_isolation_test.go Tests compiled sandbox isolation.
.github/​workflows/​workflow-normalizer.lock.yml Regenerates affected AWF commands.
.github/​workflows/​windows.lock.yml Regenerates affected AWF commands.
.github/​workflows/​update-astro.lock.yml Regenerates affected AWF commands.
.github/​workflows/​spec-enforcer.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-github-codex.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-gemini.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-codex-bare-auto.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-codex-auto.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-claude-copilot.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-claude-copilot-auto.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-claude-auto.lock.yml Regenerates affected AWF commands.
.github/​workflows/​smoke-agy.lock.yml Regenerates affected AWF commands.
.github/​workflows/​security-review.lock.yml Regenerates affected AWF commands.
.github/​workflows/​schema-feature-coverage.lock.yml Regenerates affected AWF commands.
.github/​workflows/​schema-consistency-checker.lock.yml Regenerates affected AWF commands.
.github/​workflows/​release.lock.yml Regenerates affected AWF commands.
.github/​workflows/​pdf-summary.lock.yml Regenerates affected AWF commands.
.github/​workflows/​notion-issue-summary.lock.yml Regenerates affected AWF commands.
.github/​workflows/​lint-monster.lock.yml Regenerates affected AWF commands.
.github/​workflows/​issue-arborist.lock.yml Regenerates affected AWF commands.
.github/​workflows/​gpclean.lock.yml Regenerates affected AWF commands.
.github/​workflows/​functional-pragmatist.lock.yml Regenerates affected AWF commands.
.github/​workflows/​firewall.lock.yml Regenerates affected AWF commands.
.github/​workflows/​feature-grower.lock.yml Regenerates affected AWF commands.
.github/​workflows/​example-permissions-warning.lock.yml Regenerates affected AWF commands.
.github/​workflows/​example-failure-category-filter.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-pydantic-ai.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-opencode.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-kiro.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-goose.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-gemini.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-deepseek-harness.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-cursor.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-crush.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-copilot.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-codex.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-claude.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-aider.lock.yml Regenerates affected AWF commands.
.github/​workflows/​engine-conformance-agy.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-token-consumption-report.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-team-evolution-insights.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-semgrep-scan.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-security-red-team.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-max-ai-credits-test.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-malicious-code-scan.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-harness-experiment-proposer.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-go-test-stubs-aider.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-documentation-diagram.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-code-debt-aider.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-cli-tools-tester.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-cache-strategy-analyzer.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-arxiv-researcher.lock.yml Regenerates affected AWF commands.
.github/​workflows/​daily-architecture-diagram.lock.yml Regenerates affected AWF commands.
.github/​workflows/​copilot-centralization-drilldown.lock.yml Regenerates affected AWF commands.
.github/​workflows/​codex-github-remote-mcp-test.lock.yml Regenerates affected AWF commands.
.github/​workflows/​chaos-pr-bundle-fuzzer.lock.yml Regenerates affected AWF commands.
.github/​workflows/​bot-detection.lock.yml Regenerates affected AWF commands.
.github/​workflows/​auto-triage-issues.lock.yml Regenerates affected AWF commands.
.github/​workflows/​artifacts-summary.lock.yml Regenerates affected AWF commands.
.github/​workflows/​agy-conformance-reusable.lock.yml Regenerates affected AWF commands.
.github/​workflows/​agentic-token-optimizer.lock.yml Regenerates affected AWF commands.
.github/​workflows/​ace-editor.lock.yml Regenerates affected AWF commands.
.github/​workflows/​ab-testing-advisor.lock.yml Regenerates affected AWF commands.

🧠 Review effort: Balanced

Comment on lines +539 to +545
if isOTLPEnabled(config.WorkflowData) {
for _, name := range otlpSandboxExcludedEnvVarNames {
if !slices.Contains(sortedExclude, name) {
sortedExclude = append(sortedExclude, name)
}
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 8e413e6: evals now carries the host OTLP exclusions into its minimal WorkflowData. Added Copilot/Claude/Codex enabled, disabled, and dry-run coverage plus compiled agent/detection/evals coverage; regenerated affected workflows, including artifacts-summary, and updated CTR-017. Regression tests and workflow drift checks passed. Thread resolution is unavailable through the exposed tools, so it remains open for manual resolution.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

@copilot update compiler threat spec

Updated the compiler threat specification in 7bd4e0f: CTR-017 now documents OTLP sandbox exclusions, host-side retention, AWF compatibility, and regression coverage. Bumped the spec to 1.0.45 and updated its changelog. Conformance tests and the repository validation gate passed.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (pkg/workflow/awf_command_builder.go:545): OTLP isolation still misses the evals sandbox. Evals builds a minimal WorkflowData without Env or ExcludedEnv (evals_steps.go:204-230), so this condition is false even though workflow-level OTLP variables are inherited by the evals job and passed through its awf --env-all. For example, the updated artifacts-summary.lock.yml still has an evals invocation at line 2079 without any of these exclusions. Propagate the OTLP exclusion state into evals and add evals regression coverage. - Keep OTLP configuration out of AWF agent sandboxes #67242 (comment)

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 7bd4e0f
Sous-chef work: d8a31edbc2e1c3854ff51f8779026c0369c132a5225b1a923c1b5543ed89ea27
Sous-chef state: e7a01196e4c3040ee2d9aa149e71ff3b5af88c1efc50a3356465282f49b14ad1

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 3.71 AIC · ⌖ 6.65 AIC · ⊞ 1K · ◷
Comment /souschef to run again

Copilot AI and others added 2 commits October 9, 2026 19:39
…rter-otlp-headers

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged latest main with make merge-main in 98be14c (no conflicts), then pushed the evals OTLP isolation fix, tests, regenerated workflows, and CTR-017 update in 8e413e6. Replied to the listed review thread; resolution requires a maintainer because no thread-resolution tool or GitHub CLI token is available. Tests, standard Go lint, workflow drift, and secret scanning passed. The final gate flagged a pre-existing unchecked fmt.Fprintf at evals_steps.go:576; CodeQL timed out. No CI was triggered; human review and CI verification remain.

Copilot AI requested a review from gh-aw-bot October 9, 2026 19:53
@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Fix failing check Compile queue phases (FAILURE): https://github.com/github/gh-aw/actions/runs/37982741910/job/114017957896.
  3. Fix failing check impacted-go-tests (FAILURE): https://github.com/github/gh-aw/actions/runs/37982741893/job/114018103312.
  4. Fix failing check lint-go-custom (FAILURE): https://github.com/github/gh-aw/actions/runs/37982741893/job/114018103578.
  5. Fix failing check verify-integration-build (FAILURE): https://github.com/github/gh-aw/actions/runs/37982741893/job/114018103256.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: 8e413e6
Sous-chef work: 41c633c50467b0f2dc87116833139ebf287e1ad59f1a015d8d1dfedd9d1b23f1 42e5d572d9d75b7168fe7d0aeda0717cc07e65ed240f6d299e6fec661214f1ac 57b160f058c0cf0ae8470606e768b98956d9e31df7d52cc35017efdeeaeffab1 cabafb2817b103330ac0d1e2e098f214c40c4ca2be128895a8b14391f7e3306f
Sous-chef state: 4fdd6a172cf7b08bc6f0d2752a319f86e9b7ef93a51b4f5952729056c517a1bd

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 3.15 AIC · ⌖ 9.6 AIC · ⊞ 1K · ◷
Comment /souschef to run again

Copilot AI and others added 2 commits October 9, 2026 21:06
…rter-otlp-headers

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes....

Merged latest main with make merge-main in d15341e, preserving OTLP isolation and the threat spec. Main fixes the missing integration helper, Agy conformance failures, and discarded JSON errors; dacb507 fixes the remaining evals serializer lint error. All four reported checks pass locally, along with repository lint, the complete workflow unit suite, and the final gate (334 workflows in sync). Changes are pushed. No CI was triggered. The already-answered evals thread still needs maintainer resolution because no thread-resolution tool or authenticated CLI is available. Automated review was unavailable (missing model); fallback review found no significant issues, and CodeQL timed out. Human review and CI verification remain.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot address the following outstanding work in one pass:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.

Push the necessary fixes, reply to each listed review thread and resolve it when addressed. Ignore feedback already answered or resolved. Use the pr-finisher skill and stop when only human review or CI remains; do not trigger CI.

Sous-chef head: aa83bea
Sous-chef work:
Sous-chef state: 8a95645d7077526798ac705192a5406aae2a19b4d8793fe4759b714118955bc0

Generated by 👨‍🍳 PR Sous Chef · pi · haiku45 · 3.35 AIC · ⌖ 7.4 AIC · ⊞ 1K · ◷
Comment /souschef to run again

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot resolve the merge conflicts on this branch.

…rter-otlp-headers

# Conflicts:
#	specs/compiler-threat-detection-changelog.md
#	specs/compiler-threat-detection-spec.md

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot resolve the merge conflicts on this branch.

Resolved and committed the merge conflicts in bb22dc2. The merge commit includes latest main (be0a12a), preserves both the OTLP spec updates and main’s audit clarification, and has no remaining unmerged paths or conflict markers.

@pelikhan
pelikhan merged commit 32904a3 into main Oct 10, 2026
35 checks passed
@pelikhan
pelikhan deleted the copilot/fix-otel-exporter-otlp-headers branch October 10, 2026 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OTEL_EXPORTER_OTLP_HEADERS and GH_AW_OTLP_ENDPOINTS reach the agent sandbox through awf --env-all

4 participants