Skip to content

perf(benchmarks): measure actor wake with native Rust SQLite - #5772

Draft
NathanFlurry wants to merge 1 commit into
stack/perf-guard-deliver-waiting-http-requests-with-actor-start-ztuzomnsfrom
stack/perf-benchmarks-measure-actor-wake-with-native-rust-sqlite-kmlxvnqm
Draft

NathanFlurry wants to merge 1 commit into
stack/perf-guard-deliver-waiting-http-requests-with-actor-start-ztuzomnsfrom
stack/perf-benchmarks-measure-actor-wake-with-native-rust-sqlite-kmlxvnqm

Conversation

@NathanFlurry

Copy link
Copy Markdown
Member

No description provided.

@the-company-company the-company-company Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 1 high-severity finding

Reviewed commit 67a3b31.

Comment on lines +202 to +205

async fn measure(&self, id: &str, samples: usize, warmup: usize, cold: bool) -> Result<()> {
let mut generation = self.generation(id).await.context("initial actor startup")?;
let mut times = Vec::with_capacity(samples);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 High · Read the Guard URL from configuration, not the actor response

ActorsCreateResponse contains only actor: rivet_types::actors::Actor, and that type has no endpoint field. Every bench invocation therefore exits here with missing actor endpoint before issuing a sample. Route /request through the configured RIVET_ENDPOINT (as the existing actor E2E clients do with the x-rivet-* headers) instead of trying to extract an endpoint from the create response.

@claude

claude Bot commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Review: perf(benchmarks): measure actor wake with native Rust SQLite

This adds a new opt-in benchmarks/guard-leasing crate: a Rust actor + HTTP harness for measuring cold-start/wake latency for the Guard leasing prototype. Overall this is a self-contained, low-risk addition (new workspace member only, no production code paths touched). A few notes:

Code quality

  • Cargo.toml:4 - the new member entry uses a tab for indentation (\t"benchmarks/guard-leasing",) while every other entry in members = [...] uses two spaces. Worth fixing for consistency (mixed whitespace in this list).
  • benchmarks/guard-leasing/src/main.rs:134 - Api::generation checks if let Some(generation) = body.as_u64() before falling through to the object-shaped parse below. Looking at ColdStart::on_fetch (around line 35), the actor always responds with json!({"generation": ..., "startup": ...}), an object, never a bare number, so this branch looks unreachable/dead. If it is a leftover from an earlier response format, worth deleting so it does not get mistaken for a live code path.
  • main.rs:297 - args.get(1).is_none_or(|s| s != "warm") re-derives the cold bool from raw args inside the "cold-start" | "bench" | "warm" arm, duplicating the branch selection the outer match already encodes. Deriving cold directly per match arm, rather than re-parsing args a second time with slightly different logic, would remove the duplication.

Bugs / correctness

  • measure's percentile closure (times[(times.len() as f64 * p).ceil() as usize - 1]) would underflow if ever called with times empty or p <= 0, but today it is only invoked with the hardcoded 0.50/0.95/0.99 against a samples-sized vec that is already ensure!'d to be greater than 0, so this is not currently reachable. Worth keeping in mind if the percentiles become configurable later.
  • Cleanup ordering in the "cold-start"|"bench"|"warm" arm (result?; cleanup?;) runs the DELETE regardless of whether measure failed (good, avoids leaking the actor), though if both measure and the cleanup request fail, only the measure error is surfaced. Fine for a benchmarking CLI.

Performance / security

No concerns. This only runs as an explicit local opt-in binary against a dev Engine/namespace, driven by env vars (RIVET_ENDPOINT, RIVET_TOKEN, etc.), and is not wired into any production code path.

Test coverage

No automated tests, but that matches this repo's convention for benchmark harnesses (manual, opt-in, requires a running local Engine) rather than something exercised in CI.

Nice README with a clear stage-by-stage comparison table for the rollout flags, which makes the benchmark easy to reproduce.

@NathanFlurry
NathanFlurry marked this pull request as draft September 23, 2026 06:10

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant