Skip to content

Lock-only requirements.txt discovery sends PEP 440-equivalent pins verbatim (six==1.16 → pkg:pypi/six@1.16), so a fresh checkout reports "No patches available" while the same file with a venv is patched #604

Description

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

#478 fixed #475 for projects with an installed venv: the hosted rewriter and the vendored writer now compare == pins under PEP 440. Lock-only discovery (no venv, e.g. a fresh CI checkout) still builds the purl from the version as spelled. six==1.16 is queried as pkg:pypi/six@1.16, six==1.16.0.0 as @1.16.0.0, and six==01.16.0 as @01.16.0. pip resolves all three to the registry release 1.16.0, but the patch is keyed pkg:pypi/six@1.16.0, so discovery finds nothing. scan exits 0 with "No patches available", nothing is rewritten, and pip install -r installs the unpatched release.

This is the follow-up the #478 author flagged on #475 ("on a lock-only checkout with no venv, exact_pin still carries the spelled version into the purl … left for a separate change"). I couldn't find an issue tracking it, so this one does.

Impact

The result depends on whether a venv happens to exist. On a developer machine with a venv, the file is rewritten to the patched wheel. In CI, or on any fresh checkout, the same file is left alone, exit 0, and the unpatched release is installed. Hosted and vendored modes are both affected.

Repro (Linux, pip 24.0 / CPython 3.11, main 045d7ec)

The mock patch API serves one patch for pkg:pypi/six@1.16.0 and matches purls exactly. It has the same shape as the wiremock in crates/socket-patch-cli/tests/mode_migration_pypi.rs. VIRTUAL_ENV points at an empty venv so the host's dist-packages don't confound the result.

mkdir -p empty/lib/python3.11/site-packages proj
printf 'idna==3.7\nsix==1.16\n' > proj/requirements.txt
VIRTUAL_ENV=$PWD/empty socket-patch scan --mode hosted --yes --cwd proj \
  --api-url $MOCK --org test-org --api-token x --patch-server-url $MOCK
# exit 0: "No patches available for installed packages."
# mock received: {"components":[{"purl":"pkg:pypi/idna@3.7"},{"purl":"pkg:pypi/six@1.16"}]}
cat proj/requirements.txt          # unchanged: six==1.16
python3 -m venv proj/.venv && proj/.venv/bin/pip install -r proj/requirements.txt
proj/.venv/bin/pip show six        # Version: 1.16.0 (unpatched)

# Contrast: the same file with six installed in a venv is rewritten (#475 fix)
VIRTUAL_ENV=$PWD/wv/.venv socket-patch scan --mode hosted ...   # → six @ <hosted wheel>#sha256=…

Expected vs actual

  • Expected: lock-only discovery treats an exact == pin like pip does. Per PEP 440, ==1.16 selects the 1.16.0 release, and Fix requirements.txt pins not matched under PEP 440 (#475) #478 already applies that rule to the rewriters. docs/ecosystems.md says requirements.txt lock-only checkouts are discovered in hosted and vendored modes, so the patch should be found and the line rewritten, as it is with a venv.
  • Actual: the purl carries the raw spelling, so the API finds no match. Exit 0, nothing is rewritten, and pip installs the unpatched release.
Pin (lock-only, no venv) Purl sent hosted vendored
six==1.16.0 six@1.16.0 rewritten ✅ —
six==1.16 six@1.16 not found ❌ (2/2 runs) not found ❌
six==1.16.0.0 six@1.16.0.0 not found ❌ (2/2) —
six==01.16.0 six@01.16.0 not found ❌ (2/2) —
six==1.16 with a venv holding 1.16.0 six@1.16.0 (crawler) rewritten ✅ —
OS pip / Python Reproduces
Linux 24.0 / 3.11 yes
macOS, Windows — not probed: discovery is OS-independent string handling

It never worked, so this isn't a regression: lock-only six==1.16 was also unmatched before #478.

Suspect code

  • crates/socket-patch-core/src/utils/requirements.rs:104 (exact_pin returns the spelled version), used by crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs:645 to build the lock-only purl.
  • For comparison, crates/socket-patch-cli/src/commands/scan/discovery.rs:105 (lockfile_only_contains) already bridges version spellings for composer (@3.0.2 ≡ @3.0.2.0), but not for pypi.

Possible directions: query the canonical PyPI release spelling (the JSON API, or the index), or send PEP 440 zero-padded variants and match the API's purl back with pep440::versions_equal.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    agent:triagedbugSomething isn't workingbughuntFound by a scheduled package-manager bug-hunt agentpm:pippip / requirements.txtpriority:p1

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions