Repository navigation
bug: gemini API key rejected by proxy sidecar despite valid key #25944
Description
Activity
- changed the title
[-]engine: gemini — API key rejected by proxy sidecar despite valid key[/-][+]bug: gemini API key rejected by proxy sidecar despite valid key[/+]on Apr 12, 2026 - locked and limited conversation to collaborators
on Apr 12, 2026 - unlocked this conversation
on Apr 12, 2026 Confirming this is still broken on v0.69.0 / firewall image
0.25.25.What we've verified:
GEMINI_API_KEYsecret is correctly set and confirmed valid — direct curl togenerativelanguage.googleapis.comwith the key succeeds- Upgrading through v0.68.1 → v0.68.3 → v0.68.7 → v0.69.0 and recompiling each time does not fix it
- The proxy now correctly reports
Gemini=true(fixed in v0.68.3), but the key is still rejected by Google withAPI_KEY_INVALID
What we see in logs:
[INFO] API proxy enabled: OpenAI=false, Anthropic=false, Copilot=false, Gemini=true [INFO] API proxy sidecar enabled - API keys will be held securely in sidecar container ... _ApiError: {"error":{"code":400,"message":"API key not valid. Please pass a valid API key.","status":"INVALID_ARGUMENT","reason":"API_KEY_INVALID"}}Our interpretation: the proxy sidecar receives the request but forwards a placeholder key to Google rather than substituting the real
GEMINI_API_KEY. The fix in gh-aw-firewall#1944 resolved the routing half (CLI now reaches Google) but the key injection half is still not working.- locked and limited conversation to collaborators
on Apr 21, 2026 - unlocked this conversation
on Apr 21, 2026 Update: still broken on firewall image 0.25.26
PR gh-aw-firewall#1995 was merged on 2026-04-15 and shipped in v0.69.3 (firewall image
0.25.26). We upgraded and just ran the workflow — same result.Run: https://github.com/cognitedata/cognite-function-apps/actions/runs/24777513108/job/72499620442
Log confirms:
API proxy enabled: Gemini=trueGEMINI_API_BASE_URL: http://host.docker.internal:10003API proxy sidecar enabled - API keys will be held securely in sidecar container--image-tag 0.25.26
But still:
_ApiError: {"error":{"code":400,"message":"API key not valid. Please pass a valid API key.","status":"INVALID_ARGUMENT","reason":"API_KEY_INVALID"}}The fix in gh-aw-firewall#1995 addresses stripping the placeholder header/query-param and injecting the real key — but the proxy sidecar may not be receiving
GEMINI_API_KEYin its environment at all. The in-container pre-flight health check only checksANTHROPIC_BASE_URL,OPENAI_BASE_URL, andCOPILOT_API_URL— there's no equivalent Gemini health check, so failures are silent until the first real API call.- locked and limited conversation to collaborators
on Apr 22, 2026 - unlocked this conversation
on Apr 22, 2026 🔗 AWF tracking issue: https://github.com/github/gh-aw-firewall/issues/github/gh-aw-firewall#2173
Generated by Firewall Issue Dispatcher · ● 1.2M · ◷
🔗 AWF tracking issue: https://github.com/github/gh-aw-firewall/issues/github/gh-aw-firewall#2291
Generated by Firewall Issue Dispatcher · ● 436.3K · ◷
engine: gemini— API key rejected by proxy sidecar despite valid keyWhen using
engine: geminiin an agentic workflow, the Gemini API returnsAPI_KEY_INVALIDeven though the key is valid and confirmed working via directcurlrequests.Evidence
✅ GEMINI_API_KEY: Configuredgenerativelanguage.googleapis.comwith the same key succeeds and returns model dataProxy configuration from logs
The lock file routes Gemini calls through the proxy sidecar:
Suspected cause
The API proxy sidecar at
host.docker.internal:10003appears to not be forwarding the API key correctly togenerativelanguage.googleapis.com, or is stripping/modifying it in transit.Reproduction
engine: geminiGEMINI_API_KEYas a repo secretAPI_KEY_INVALIDEnvironment
node/24.14.1