Skip to content

Recognize hosted Pipenv references through one shared grammar (#563) - #572

Merged
Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
arch-refactor/563-pipenv-hosted-url
Oct 2, 2026
Merged

Mikola Lysenko (mikolalysenko) merged 3 commits into
mainfrom
arch-refactor/563-pipenv-hosted-url

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #563

Summary

Hosted and vendored Pipenv each had their own private grammar for deciding whether a Pipfile.lock URL is a Socket-hosted PyPI patch reference. This PR moves both onto one shared recognizer, lock_inventory::pypi::hosted_pypi_reference, and deletes the two copies.

Why (leverage)

  • Issue #563 (p1), register rows E04 and E49 (register), living document Part 5.4, "Hosted-URL recognition is duplicated".
  • B=1, U=0, D=2 (two grammars deleted, net −17 production lines), R=L. Score 5, the top eligible candidate. Its files don't overlap any open arch-refactor/* or agent/fix-* PR. Hosted NuGet mapping reads commented-out package sources #561 ranked close behind but touches redirect/mod.rs, which four open PRs also change.

What changed

  • New: hosted_pypi_reference(url, origins). It applies the redirect::hosted_patch_url_uuids origin policy (patch.socket.dev or a configured origin, no userinfo), then the hosted_artifact_url tail grammar (…/patch/pypi/<n>/<v>/<grant>/<uuid>/<wheel-or-sdist>, matched from the end), and requires a non-empty uuid level.
  • Hosted redirect::pipenv::owned_url = the shared recognizer on the grant's own origin, plus a check that the name and version match the dep.
  • Vendored pypi_pipenv::check_target_guards / wire_pipenv take hosted_origins. vendor_pypi_with_pipenv_version passes the run's VendorServiceConfig.patch_server_url. service_preflight passes &[]: only the verdict matters there, and both refusal branches carry the same code.

Deleted

  • The segment-count grammar in owned_url (≈30 lines).
  • vendor/pypi_pipenv.rs::is_socket_hosted_reference (≈10 lines).
  • Diff (approximate split at #[cfg(test)]): production +31 / −48, tests +174 / −37.

Behavior

These changes are intended. They are what #563 asks for:

  • Hosted Pipenv now treats its own pin on a path-prefixed --patch-server-url origin, and a hosted sdist pin, as ours. Rotation works where it used to return Conflict("Pipenv source for … already exists").
  • Vendored Pipenv gives a foreign host's /patch/pypi/… URL the "user-declared" remedy, where it used to say "HOSTED … run rollback". A hosted sdist or path-prefixed pin on an accepted origin now gets the HOSTED remedy. The error code is pypi_pipenv_source_already_exists in every case, unchanged.
  • Small edge: hosted owned_url used to reject a URL with a ?query. The shared recognizer ignores the query, the same as hosted_patch_uuid. Socket never writes one.
  • No JSON, exit-code or contract changes. The wrappers (npm/, pypi/, gem/) need nothing.
  • Poetry, pdm and uv have no private /patch/pypi/ grammar of their own (checked with grep), so nothing more to unify there.

Test evidence

  • cargo clippy --workspace --all-features -- -D warnings: clean.
  • cargo test -p socket-patch-core --lib: 4756 passed, 4 failed. The 4 fail identically on origin/main because the sandbox runs as root (permission-dependent tests): copy_tree::relax_loop_must_not_traverse_symlinked_root, vlt_heal::an_unremovable_hidden_lock_keeps_every_store_entry, pypi_poetry::wire_write_failure_maps_error_and_leaves_lock_untouched, pypi_requirements::wire_failure_rolls_back_already_written_files.
  • cargo test -p socket-patch-cli --all-features --test in_process_redirect_pipenv --test hosted_memory_engine --test hosted_memory_parity: 6 + 28 + 31 passed.
  • Red→green:
    • redirect::pipenv::tests::owned_url_accepts_path_prefixed_origins_and_sdists fails with the old owned_url swapped back in and passes now.
    • vendor::pypi_pipenv::tests::hosted_reference_remedy_follows_the_shared_recognizer fails with the old is_socket_hosted_reference swapped back in (the evil.example URL was called HOSTED) and passes now.
    • Together these run both former callers through the shared code on the inputs where the copies differed.
  • owned_url_follows_the_grant_origin and the vendored Pipenv suite stay green.
  • CodeQL flagged the new test's assert message for printing the refusal detail. ff3ef3e prints only the URL and the expected phrase, and the pypi_pipenv tests are still green (27 passed).
  • The PDM compatibility cell platform-windows hosted failed once and passed on re-run. It isn't on a path this PR touches (see comment).

Risk

Low. The change is confined to Pipenv ownership and remedy selection, and every refusal keeps its code.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WhtPehwinReW5U3cPmhccc


Note

Low Risk
Scope is Pipenv lock URL classification and error messaging only; no auth, API, or contract changes, with existing refusal codes preserved.

Overview
Unifies how hosted Pipenv lock rotation and vendored Pipenv guards decide whether a Pipfile.lock URL is Socket’s own hosted PyPI patch, by introducing shared hosted_pypi_reference and removing two duplicate grammars.

Hosted owned_url now delegates to that helper (origin policy + tail-matched /patch/pypi/… path) instead of a fixed segment-count wheel-only check, so path-prefixed --patch-server-url deployments and hosted sdist pins count as “ours” and can rotate. Vendored check_target_guards / wire_pipenv take hosted_origins from the run’s patch_server_url; hosted vs user-declared remedy text follows the same recognizer (foreign hosts with /patch/pypi/… are user-declared; trusted origins get the HOSTED rollback message). Error codes are unchanged.

Tests cover path-prefixed rotation, sdists, and remedy wording where the old copies disagreed.

Reviewed by Cursor Bugbot for commit ff3ef3e. Configure here.


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko Mikola Lysenko (mikolalysenko) added refactor Structural change: duplicated code or logic, missing abstraction, layering, dead code arch-refactor PR opened by the scheduled architecture refactor routine labels Oct 2, 2026
Hosted Pipenv refused to rotate its own pin when --patch-server-url
carried a path prefix, and treated its own hosted sdist pins as user
sources. Vendored Pipenv called any https host's /patch/pypi/ URL a
Socket reference and told the user to run rollback for a foreign
source, while a path-prefixed or sdist hosted pin got the
"user-declared" remedy instead.

Both now ask lock_inventory::pypi::hosted_pypi_reference: the
hosted_patch_url_uuids origin policy plus the hosted_artifact_url
tail grammar. The two private segment-count grammars are deleted.
Vendored Pipenv passes the run's --patch-server-url origin.

Fixes #563

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 2, 2026 16:20
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 2, 2026
Assisted-by: Claude Code:claude-opus-5-5

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] CI: PDM patch compatibility / native (ubuntu-latest, 2.0.3) failed on one cell, platform-windows hosted (refusalCodeReported). The scan reported no refusal code, and the cell took 27.0s where its siblings took about 7s. Every other cell in the matrix passed or refused as expected.

This doesn't look like this PR:

  • The diff changes only Pipfile.lock ownership (redirect::pipenv::owned_url) and the vendored Pipenv guard. A PDM project has no Pipfile.lock, so neither path runs.
  • The same job on the same base (b0a32db) passed on #562's run.

I couldn't read the case's result.json, because the artifact host is blocked from this sandbox. I've re-run the failed job once. If it fails again, I'll treat it as real and root-cause it.


Generated by Claude Code

Comment thread crates/socket-patch-core/src/vendor/pypi_pipenv.rs Fixed
The new Pipenv remedy test printed the whole refusal detail on
failure, which CodeQL traces as a patch uuid written to a log.
Print only the URL and the expected phrase.

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit ff3ef3e. Configure here.

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] CI: vlt patch compatibility / native (ubuntu-latest, 1.0.0-rc.14) failed on ff3ef3e. 35 of 36 cells behaved as expected. The one that didn't, hosted-optional, came back safe-refusal where patched was expected. The same job logged "transport failure; retrying" on seven other cells, all of which recovered.

This doesn't look like this PR:

I'll re-run the failed job once when the run completes. If it fails again, I'll treat it as real.


Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[agent] CI: e2e (ubuntu-latest, e2e_safety_pnpm) failed in apply_in_a_does_not_mutate_b_or_store. The test's setup socket-patch get got Connection reset by peer (os error 104) from patches-api.socket.dev/patch/view/…, so it stopped before any patching code ran.

This is a transport error against the patch API. The pnpm code it covers isn't touched by this Pipenv-only diff, and the vlt job's transport retries above suggest the same instability. I'll re-run this failed job once. A second failure gets root-caused.


Generated by Claude Code

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 2, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

[burn-down agent] Labeled Ready for review at ff3ef3e6fb2d11b1ee17841086be10bc7dba3985.

  • CI: 484/490 check runs green on this head (6 skipped by path/matrix filters), 0 failing. Mergeable, merges cleanly into current main. 14 commits behind main, no file overlap.
  • Bugbot: reviewed ff3ef3e6fb, no issues found; no unresolved review threads.
  • Reviewer focus: intended behavior change: hosted Pipenv now recognizes its own pins on path-prefixed patch-server origins and hosted sdists (see Behavior section).

Generated by Claude Code

@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Reviewed ff3ef3e6fb2d11b1ee17841086be10bc7dba3985. Recommendation: ready to merge as-is. No actionable correctness or security regressions found.

Checked shared origin/path/filename recognition, foreign-source refusal, category handling, and VEX/rollback interactions. Path-prefixed origins and hosted sdists rotate correctly; foreign hosts, credentials, coordinate mismatches, empty UUID levels, and trailing slashes remain refused.

Validation: 63 Pipenv core tests, 6 shared-origin tests, 20 PyPI VEX discovery tests, and 2 independent reviewer probes passed. The probes additionally exercised custom HTTP origins, rotation/idempotence and 20 URL boundary cases. Clean merge with current main (203e092b). Exact-head CI: 484 successful checks, 6 skipped; Bugbot found no new issues and no unresolved review threads remain.

@mikolalysenko
Mikola Lysenko (mikolalysenko) merged commit 950bfc5 into main Oct 2, 2026
776 of 779 checks passed
@mikolalysenko
Mikola Lysenko (mikolalysenko) deleted the arch-refactor/563-pipenv-hosted-url branch October 2, 2026 19:15
Mikola Lysenko (mikolalysenko) pushed a commit that referenced this pull request Oct 2, 2026
Assisted-by: Claude Code:claude-opus-5-5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arch-refactor PR opened by the scheduled architecture refactor routine Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review refactor Structural change: duplicated code or logic, missing abstraction, layering, dead code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipenv recognizes hosted PyPI patch URLs with two private grammars that disagree with the shared one

4 participants