Skip to content

Fix/ds proxy via proxy - #4661

Open
Gauravsingh096 wants to merge 3 commits into
mainfrom
fix/ds-proxy-via-proxy
Open

Gauravsingh096 wants to merge 3 commits into
mainfrom
fix/ds-proxy-via-proxy

Conversation

@Gauravsingh096

@Gauravsingh096 Gauravsingh096 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Describe the changes that are made

Two replay fixes found while making DaemonSet proxy-via-proxy record/replay green (a k8s-proxy with the DaemonSet agent recording another k8s-proxy, plus a plain Go control app).

1. fix(http): prefer the mock recorded in the current test window (C1)

pkg/agent/proxy/integrations/http/match.go

  • Problem: two tests make the same call, e.g. POST /v1/authorize for a valid PAT, then for a bad one. The two recorded mocks have the same URL and body and differ only in the Authorization value, which is marked as noise. DeriveLifetime lax-promotes tagged HTTP mocks into the session pool (pkg/models/lifetime.go), so neither mock is consumed or window-filtered, and ExactBodyMatch returns the first byte-identical body. The bad-token test gets the valid token's 200 instead of its recorded 401.
  • Fix: before the body match, preferCurrentTestWindow stable-reorders the schema-matched candidates so that those whose ReqTimestampMock lies inside CurrentTestWindow() come first — the rule mysql/replayer already applies. Only a recorded timestamp inside the window moves a mock. With no active window, a single candidate, or no candidate inside the window, the order and the chosen mock are exactly as before.

2. feat(proxy): add an OnTestWindow hook for parsers (C2)

pkg/agent/proxy/mockmanager.go, pkg/agent/proxy/scoped_mockdb.go, pkg/agent/proxy/integrations/integrations.go

  • New optional interface integrations.TestWindowListener. OnTestWindow(fn) registers a callback that runs each time a real test window is published: SetMocksWithWindow past its BaseTime staging call, or a non-zero SetCurrentTestWindow. It runs synchronously on the setter's goroutine, after every MockManager lock is released and before the setter returns, so whatever the callback writes reaches the app before the replayer sends that test's request.
  • SetMocksWithWindow is now a thin wrapper around the unchanged body (renamed setMocksWithWindow, which reports whether it published a window). scopedMockDb forwards OnTestWindow, the same way it forwards Revision.
  • Consumer: keploy/integrations#305 uses it to push the Kubernetes watch events recorded after a watch-list snapshot down the held-open stream, just before the first test recorded after each event. An informer's state at a test is the snapshot plus every event before that test; without this, replay answers every test from the startup state. Integrations declares the interface by its shape, so it still builds against releases without this hook and then simply behaves as before.

Links & References

Closes: NA — part of the DaemonSet proxy-via-proxy record/replay work.

🔗 Related PRs

  • keploy/integrations#305 — HTTP/2 watch-list snapshots and the post-snapshot events that use the hook above
  • keploy/k8s-proxy#1002
  • keploy/enterprise#2676 — needs a go.keploy.io/server/v3 bump to the release carrying this PR

🐞 Related Issues

  • NA

📄 Related Documents

  • NA

What type of PR is this? (check all applicable)

  • 📦 Chore
  • 🍕 Feature
  • 🐞 Bug Fix
  • 📝 Documentation Update
  • 🎨 Style
  • 🧑‍💻 Code Refactor
  • 🔥 Performance Improvements
  • ✅ Test
  • 🔁 CI
  • ⏩ Revert

Added e2e test pipeline?

  • 👍 yes
  • 🙅 no — unit tests here; the end-to-end scenario needs k8s-proxy + enterprise + integrations together and was verified on a DaemonSet rig (results below)
  • 🙋 no, because I need help

Added comments for hard-to-understand areas?

  • 👍 yes
  • 🙅 no, because the code is self-explanatory

Added to documentation?

  • 📜 README.md
  • 📓 Wiki
  • 🙅 no documentation needed

Are there any sample code or steps to test the changes?

  • 👍 yes, mentioned below
  • 🙅 no, because it is not needed
go test ./pkg/agent/proxy/integrations/http/ -run 'TestMatch_IdenticalSessionMocksPreferCurrentTestWindow|TestPreferCurrentTestWindow'
go test ./pkg/agent/proxy/ -run 'TestOnTestWindow|TestScopedMockDbForwardsOnTestWindow'
go test -race ./pkg/agent/proxy/ -run 'MockManager|Window|Scoped|Wave|Race'
  • TestMatch_IdenticalSessionMocksPreferCurrentTestWindow drives match() with the two authorize mocks: each test window gets its own reply; with no window, or a window matching neither, the first recorded mock still wins (unchanged behaviour).
  • TestOnTestWindow_RunsWithNoLockHeld reads the pool back from inside the listener (GetPerTestMocksInWindow takes swapMu), so a listener run under a MockManager lock fails the test instead of deadlocking silently.
  • Full ./pkg/agent/proxy/ and ./pkg/agent/proxy/integrations/http/ suites pass; go vet clean; golangci-lint v2.13.1 --new-from-rev: 0 issues.

Self Review done?

  • ✅ yes
  • ❌ no, because I need help

Any relevant screenshots, recordings or logs?

DaemonSet rig (kind): the recorder k8s-proxy records target k8s-proxy A and a Go control app, then auto-replays. Images were built from k8s-proxy#1002, enterprise#2676 and integrations#305 with this PR's two commits on v3.6.79 (932fa2d1f, 63b147253 — before the main merge df5461f20, which changes none of this PR's files). Two runs of each:

Run Before After
A over HTTP 21–22 / 25 25 / 25
A over HTTPS 21 / 25 24 / 25 — only step 15 (follow-up 1 below)
Control app, HTTP 34 / 35 (step 11 noisy) 35 / 35
Control app, HTTPS / dedicated replay namespace / cluster mode 8–9 / 9 9 / 9
Runner mode (failed job shows as a failed run) pass pass
  • A's step 3 (bad token must get 401) passes because of C1.
  • A's steps 1 and 24 (informer readyReplicas) and the control app's step 11 (informer pod count) pass because of C2 + integrations#305.

Follow-ups (after this PR)

  1. Step 15 over HTTPS: in the replay copy, a Go app's first TLS call to a mocked dependency right at startup (k8s-proxy's self-discovery, ~20 ms after start) can fail, so an endpoint that depends on it (/proxy/update/status) answers differently from the recording. Not addressed here; next.
  2. Recorder GoTLS attach race: seen in earlier rig rounds; it did not reproduce in the final runs above, but it is not fixed yet. Next.

🧠 Semantics for PR Title & Branch Name

Please ensure your PR title and branch name follow the Keploy semantics:

📌 PR Semantics Guide
📌 Branch Semantics Guide


Additional checklist:

🤖 Generated with Claude Code

Signed-off-by: Gaurav Singh <singhgaurav07122004@gmail.com>
Signed-off-by: Gaurav Singh <singhgaurav07122004@gmail.com>
Copilot AI balanced review requested due to automatic review settings September 29, 2026 08:39
@github-actions

Copy link
Copy Markdown

🚀 Keploy Performance Test Results

Multi-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.

Run P50 P90 P99 RPS Error Rate Status
1 N/A N/A N/A N/A N/A ✅ PASS

Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1%

✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2)

P50, P90, and P99 percentiles naturally filter out outliers

@github-actions

Copy link
Copy Markdown

🚀 Keploy Performance Test Results

Multi-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.

Run P50 P90 P99 RPS Error Rate Status
1 2.91ms 3.77ms 5.32ms 100.02 0.00% ✅ PASS
2 2.81ms 3.63ms 4.83ms 100.03 0.00% ✅ PASS
3 2.77ms 3.59ms 4.8ms 100.02 0.00% ✅ PASS

Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1%

✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2)

P50, P90, and P99 percentiles naturally filter out outliers

@Gauravsingh096 Gauravsingh096 left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staff review — keploy/keploy#4661

Reviewed with the three companion PRs on keploy/enterprise#2659 (enterprise#2676, k8s-proxy#1002, integrations#305).

Verdict: no Blocker or Major found.

This is the smallest of the four but has the widest blast radius: preferCurrentTestWindow sits in HTTP mock selection, which every replay uses, so a reorder that fires when it should not would hand wrong mocks to unrelated suites. The claim is that nothing changes without an active window. I checked it rather than took it, and the guards are complete:

  • len(mocks) < 2 || mockDb == nil → returns mocks untouched;
  • winStart.IsZero() || winEnd.IsZero() → untouched, so no active window means no reorder;
  • len(inside) == 0 || len(rest) == 0 → untouched, so all-inside or all-outside keeps the original order.

Only a genuine mix reorders, and the partition is stable — both groups are appended in their original relative order, so within a group the previous choice is preserved. The boundary check is inclusive on both ends (!reqAt.Before(winStart) && !reqAt.After(winEnd)), and a zero ReqTimestampMock is treated as outside rather than accidentally inside.

One implementation detail worth noting because getting it wrong would be a silent corruption: inside is a fresh slice (make([]*models.Mock, 0, len(mocks))), so append(inside, rest...) cannot alias and overwrite the caller's backing array. Had inside been a re-slice of mocks, this would have scrambled the input. It is not.

The OnTestWindow hook (C2) is additive and declared structurally, so integrations still builds against releases without it — which is what keeps keploy/integrations#305 independently mergeable.

Scope and honesty note. These four PRs total ~5,900 added lines. I did not read every line; I prioritised by risk and verified the specific things most likely to be Blocker/Major, listed above. Per Gaurav's instruction this review carries Blocker and Major only — smaller observations are omitted deliberately, not because none exist. Gates not run: this machine is out of disk and the Go build cannot complete, so I have no local test result for any of these branches and am not implying one. Everything above is from reading the code at the head shown.


🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant