Conversation
* feat(codex): run on whichever stored Codex subscription has quota An account can now store several ChatGPT Codex subscriptions. The primary stays CODEX_AUTH_JSON; extra ones are CODEX_AUTH_JSON_2, CODEX_AUTH_JSON_3, and so on, minted with `pullfrog auth codex --slot N`. Before the first install, main.ts calls selectCodexAuth(). When at least one extra slot is present, it reads each slot's quota from GET https://chatgpt.com/backend-api/wham/usage (the endpoint the Codex CLI uses, codex-rs rust-v0.153.4) and ranks slots in this order: available, unknown (probe failed or access token stale), limit reached with credits, exhausted, unusable (malformed or refresh-latched). Ties keep slot order: CODEX_AUTH_JSON first, then numeric suffix order. The chosen blob is copied into CODEX_AUTH_JSON, every extra slot is deleted from process.env so no agent or shell subprocess inherits it, and the post-run OAuth writeback PUTs a rotated chain back to the chosen slot's own secret name. With only CODEX_AUTH_JSON present, selectCodexAuth() returns at once and the run is unchanged. * fix(codex): rank a slot without an id_token as unusable The codex CLI refuses an auth.json without tokens.id_token, and installCodexHome() already returns null for one. selectCodexAuth() probed such a slot like any other, so an earlier slot with quota but no id_token could win the tie-break, the complete later slot was deleted from process.env, and a codex-harness run started without subscription auth. The slot now ranks unusable and is chosen only when every slot is. * fix(codex): guard slot names in `secret set` and name the chosen slot in warnings - `pullfrog auth codex --slot` requires a safe integer, so `--slot 1e21` fails at parse time instead of producing CODEX_AUTH_JSON_1e+21, which the secrets API refuses only after device authentication. - `pullfrog secret set` applies the repo-copy shadow guard to every CODEX_AUTH_JSON_<suffix> name, the same guard `auth codex --slot` runs, so both write paths refuse the same shadowed save. - The malformed, rejected, and missing-id_token warnings in installCodexAuth() and installCodexHome() name the slot whose blob was selected instead of always naming CODEX_AUTH_JSON. * fix(codex): rank credits above unknown, stop on a spend cap, keep --slot off auth grok - A slot at its limit that holds credits now ranks above a slot whose probe failed. A dead slot always probes as unknown, so the old order picked it on every run over a primary that could still run on credits. - spend_control.reached ranks a slot exhausted. The Codex TUI treats a reached spend cap as a hard stop regardless of the rate limit. - `pullfrog auth grok` parses with the scope-only parser again. It had shared parseCodexArgs, so `auth grok --slot 2` was accepted and saved GROK_AUTH_JSON, ignoring the slot. * test(codex): cover the slot preference order through pickCodexSlot The selectCodexAuth tests only use slots that rank unusable, so every rank tied and an inverted preference order would still pass. The reduce moves into an exported pure pickCodexSlot(), and its tests assert that free quota beats credits, credits beat an unanswered probe, an unanswered probe beats an exhausted or unusable slot, and equal ranks keep slot order. No fetch stub is needed. * fix(codex): require id_token only when the codex harness can run, and require a primary for --slot - selectCodexAuth() takes requireIdToken. main.ts sets it when the codex harness can run: the repo's codexAgent opt-in, payload.codexArm, or PULLFROG_AGENT=codex. OpenAI models otherwise run on opencode, which materializes a blob without tokens.id_token, so such a slot keeps its real quota rank there instead of losing to an exhausted complete slot. - `pullfrog auth codex --slot N` refuses to save while CODEX_AUTH_JSON is neither stored nor inherited on the target. modelHasStoredAuth() matches CODEX_AUTH_JSON by exact name, so a slot-only account could miss the server's stored-auth check. - The device-auth rerun hint keeps `--slot N`, so following it retries the same slot instead of the primary. * fix(codex): let an explicit PULLFROG_AGENT decide the id_token rule resolveAgent() returns a known PULLFROG_AGENT before it reads the codex opt-in, but the selection's requireIdToken ignored that order, so PULLFROG_AGENT=opencode on an opted-in repo still ranked a slot without tokens.id_token unusable. mayRunCodexHarness() in utils/agent.ts applies the same order: a known PULLFROG_AGENT decides, an unknown one falls through, and the opt-in decides otherwise. * fix(codex): require the primary before `secret set` saves a slot `pullfrog auth codex --slot N` already refuses to save a slot while CODEX_AUTH_JSON is neither stored nor inherited on the target. `pullfrog secret set CODEX_AUTH_JSON_<suffix>` now applies the same check, and counts a CODEX_AUTH_JSON saved in the same batch, so neither write path can leave a slot-only account that modelHasStoredAuth() matches by the exact primary name.
![pullfrog[bot]](https://avatars.githubusercontent.com/in/1768019?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Important
The action-side selection looks correct, but extra slots depend on server behaviour this PR doesn't touch or document: pre-run rotation, the refresh-rejected latch, and whether writeback accepts a CODEX_AUTH_JSON_N name. Without them, a slot can quietly lose its refresh chain after the first in-run refresh.
Reviewed changes
I reviewed the full diff: multi-slot Codex subscription storage in the CLI, and quota-based slot selection at run start.
- Slot storage:
pullfrog auth codex --slot Nandsecret setnow saveCODEX_AUTH_JSON_N. Both refuse to save a slot unless a primaryCODEX_AUTH_JSONis already stored, and both apply the same shadow check as the primary. - Run-time selection:
selectCodexAuthchecks each slot's quota againstchatgpt.com/backend-api/wham/usageand ranks the results. It copies the winner intoCODEX_AUTH_JSONand removes the other slots fromprocess.env. - Writeback target:
installCodexAuthandinstallCodexHomereturn the selected slot's name assecretName. The codex and opencode harnesses pass it to the post-hook writeback, andOAuthWriteback.secretNameis now a plainstring. id_tokengating: the newmayRunCodexHarnessmirrorsresolveAgent'sPULLFROG_AGENToverride and codex opt-in. A slot without anid_tokenis only disqualified when the codex CLI harness could run.- CLI cleanup:
auth groknow parses withparseClaudeArgs, so it no longer accepts--slot.
I ran vitest run utils/codexHome.test.ts utils/agent.test.ts locally and all 36 tests pass.
⚠️ Extra slots may not get the server-side refresh handling the primary gets
The primary chain's safety comes from the server, not the action. According to the header of utils/codexHome.ts, the run-context endpoint calls maybeRotateCodexSecret under a Postgres row lock before it returns dbSecrets. The same server path latches refresh_rejected_at, and PUT /api/runtime/secret persists rotations that happen during a run.
None of that code is in this repo, and this PR doesn't change it. If it only handles CODEX_AUTH_JSON, then:
- A selected slot gets refreshed during the run by the harness, with no lock. OpenAI refresh tokens are single-use, so two concurrent runs that pick the same slot will race, and the loser ends up with a dead chain.
- If the writeback endpoint only accepts the two names the old
OAuthWriteback.secretNametype allowed, the rotated chain is dropped with just a post-hook warning. The stored slot is then dead after its first refresh. - A dead slot is never latched as rejected. Its probe keeps returning 401, which becomes
unknown, so it keeps being probed on every run.
Technical details
# Extra Codex slots depend on server-side rotation/writeback/latch that may be primary-only
## Affected sites
- utils/codexHome.ts:391-412 — `selectCodexAuth` assumes every `CODEX_AUTH_JSON_N` arrives as fresh as the primary
- utils/codexRefreshDetect.ts:6 — `secretName` was narrowed to `"CODEX_AUTH_JSON" | "GROK_AUTH_JSON"`, which suggests the server-side PUT may have a matching allowlist
- agents/codex.ts:897, agents/opencode.ts:1291 — writeback now targets `CODEX_AUTH_JSON_N`
## Required outcome
- Confirm (or implement, in the server repo) that `maybeRotateCodexSecret` rotates and latches every `CODEX_AUTH_JSON_N`, not just `CODEX_AUTH_JSON`
- Confirm `PUT /api/runtime/secret` accepts `CODEX_AUTH_JSON_N` names
- Merge this action change only after (or together with) the server change, so slots are never used without that protection
## Open questions for the human
- Is the run-context rotation keyed by name, or by "any Codex-shaped blob"?ℹ️ Nitpicks
main.ts:274:selectCodexAuthprobes every slot on every run, including proxy/Router runs and runs pinned to a non-OpenAI model. In those cases it adds up to 10s of latency and sends each subscription token to chatgpt.com for no benefit.utils/codexHome.ts:96,313: when a slot is the one selected, the rejected-chain and missing-id_tokenwarnings now nameCODEX_AUTH_JSON_N, but the fix they suggest is stillnpx pullfrog auth codex, without--slot N. That command would overwrite the primary instead of repairing the slot.utils/codexHome.ts:63:selectedCodexSecretis module-level state thatcodexHome.test.tsnever resets. After the "no primary" test it staysCODEX_AUTH_JSON_2for later tests in the same worker.
Claude Opus (free via Pullfrog for OSS) ｜ 𝕏
|
|
||
| export async function selectCodexAuth(params: { requireIdToken: boolean }): Promise<void> { | ||
| const slots = Object.keys(process.env) | ||
| .filter((name) => name.startsWith(`${CODEX_AUTH_ENV}_`)) |
Contributor
There was a problem hiding this comment.
This matches any env var that starts with CODEX_AUTH_JSON_, not just numeric slots. secret set has the same gap (commands/secret.ts:95,101), so a user who stores a backup such as CODEX_AUTH_JSON_OLD will have it probed and possibly selected. Selection also deletes any workflow env: var with that prefix. The PR description says slot names are validated consistently, but only --slot actually checks for N >= 2.
Technical details
# Slot name matching is prefix-only
## Affected sites
- utils/codexHome.ts:393 — `name.startsWith(`${CODEX_AUTH_ENV}_`)`
- commands/secret.ts:95 — shadow guard applies to any `CODEX_AUTH_JSON_*`
- commands/secret.ts:101 — primary-required check applies to any `CODEX_AUTH_JSON_*`
## Required outcome
- One shared definition of a slot name (e.g. `/^CODEX_AUTH_JSON_[1-9]\d*$/` with N >= 2), used by the runtime filter and by `secret set`
- `secret set CODEX_AUTH_JSON_FOO` should be rejected or treated as an ordinary secret, never become a run candidate![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9057aa2. Configure here.
| return codexQuota((await response.json()) as CodexUsage); | ||
| } catch { | ||
| return "unknown"; | ||
| } |
There was a problem hiding this comment.
Stale extra-slot tokens mis-rank quota
Medium Severity
probeCodexQuota sends the stored access_token as-is. Extra slots sit unused, so that token is typically expired (these chains last about an hour, and only the primary is described as server-rotated). A 401 becomes unknown, which loses to credits, so a second subscription with free quota is skipped and the run burns credits on the primary instead.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 9057aa2. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds multi-subscription support for ChatGPT Codex auth and automatically selects the stored account with the best usable quota before each run
Users can store additional Codex subscriptions as
CODEX_AUTH_JSON_2,CODEX_AUTH_JSON_3, etc. viapullfrog auth codex --slot N. When multiple slots exist, Pullfrog probes Codex quota usage and selects accounts in this orderTies preserve slot order, with
CODEX_AUTH_JSONfirstThe selected auth blob is exposed as
CODEX_AUTH_JSONfor the run, while extra slots are removed from the environment so subprocesses cannot access them. OAuth token rotation is written back to the original selected slotThis also tightens Codex auth handling by:
id_tokenas unusable only when the Codex harness may run.PULLFROG_AGENTselection when applying that rule.CODEX_AUTH_JSONbefore saving additional slots.--slotvalues and secret names consistently.--slot Nin auth retry hints.auth grokfrom accepting--slot.Single-slot behavior remains unchanged
Note
Medium Risk
Changes credential selection and OAuth writeback targets at run time, including live quota probes with subscription tokens—wrong slot choice or writeback would affect billing and auth persistence, though single-slot behavior is unchanged.
Overview
Adds multi-subscription Codex auth so teams can store extra ChatGPT accounts as
CODEX_AUTH_JSON_2,CODEX_AUTH_JSON_3, etc. viapullfrog auth codex --slot N, with validation that a primaryCODEX_AUTH_JSONexists first and the same shadow-refusal rules as the main secret.Before each run,
selectCodexAuthprobes ChatGPT usage for every configured slot, ranks them (available → credits → unknown → exhausted → unusable), copies the winner intoCODEX_AUTH_JSON, and drops the other slot env vars so agents only see one chain. OAuth rotation writeback now targets the selected slot name (codex and opencode harnesses), not a hardcoded secret.Run startup calls selection with
requireIdTokentied tomayRunCodexHarness, so missingid_tokenblocks only when the Codex CLI harness might run;PULLFROG_AGENTcan override the repo’s codex opt-in the same way as agent routing. Quota logic also treats spend caps as exhausted.secret setmirrors the slot guards;auth grokno longer accidentally accepts Codex-only--slotparsing.Reviewed by Cursor Bugbot for commit 9057aa2. Bugbot is set up for automated code reviews on this repo. Configure here.