中文版 · 日本語

The cost-optimization page estimates what the Concise Output switch saves (a global system prompt that asks models to keep replies short). The estimate is a period total for the selected time range: the output spend and the output tokens of your priced traffic, each times a fixed coefficient of 12.6%. This file records how that 12.6% was measured. The complete raw data is at the bottom.

10 fixed questions (listed below), 3 rounds each, on five models: claude-opus-4-7, deepseek-v4-flash, deepseek-v4-pro, glm-5.1, qwen3.5-flash. Each model ran all 30 questions with the switch off, then the switch was flipped through the console's own setting API and the same 30 ran again. 300 calls in total, none failed, which gives 150 on/off pairs — one per (model, question, round).

The ON runs installed exactly what the console installs when you turn the switch on, verbatim:

回答请保持简洁，去掉客套话和不必要的铺垫，保留完整语法和技术细节。优先复用标准库和平台已有能力，非必要不引入新的抽象或依赖，用最短的可行改动解决问题。

Both sentences, because the switch writes both — the second one, about preferring existing platform capabilities and the smallest viable change, shortens code answers as much as the first one shortens prose. The console writes them in its own language, so an English console installs the English wording; that wording is not what was measured here.

Sampling parameters were whatever each model defaults to. For each pair, r = (off_tokens − on_tokens) / off_tokens, where the token counts are the upstream-reported completion_tokens. On reasoning models this includes the thinking tokens, because those are billed too. The shipped coefficient is the median of all 150 ratios.

Measured on 2026-08-24.

Per-model medians:

Every model came out ahead, but the spread inside each one is wide and it is not noise you can average away by looking harder. 44 of the 150 pairs are negative — the prompt made that particular answer longer. Individual pairs swing hard: the same model on the same question landed at −193% in one round and +46% in another. Expect the figure to hold across a month of traffic, not on any single request.

We ship one global number, not a per-model table. Models change faster than such a table stays useful, and the page labels the figure as an estimate. The number also assumes the switch is left on the prompt it installs; replacing that text through the API with something unrelated voids it.

Frozen; the same set is used every time. The runs were made in Chinese — the questions below are translations, kept here so the mix is readable. Question length and language both move the token counts, so reproducing these numbers means sending the Chinese originals verbatim: they are in the Chinese edition. (Embedded code, SQL and log samples are identical in both.)

1. [codegen] Implement in Go a function WordCount(text string, topN int) that counts case-insensitively how often each word appears in an English text and returns the top topN (word, count) pairs by descending count. Provide complete runnable code including imports and a small main example.
2. [codegen] Write a Python CLI script that recursively walks a given directory, finds files larger than 10 MB modified within the last 7 days, and prints their paths and sizes (human-readable) in descending size order. Provide complete code and usage instructions.
3. [code-explain] Explain what this SQL does and point out possible performance problems and improvements: SELECT u.id, u.username, u.email, COUNT(o.id) AS order_count, SUM(o.amount) AS total FROM users u LEFT JOIN orders o ON o.user_id = u.id AND o.status = 'paid' WHERE u.created_at > '2026-01-01' AND u.status <> 'deleted' GROUP BY u.id, u.username, u.email HAVING COUNT(o.id) > 5 ORDER BY order_count DESC LIMIT 100;
4. [code-review] Review this Go code and list every problem you find (error handling, resource leaks, edge cases): func readConfig(path string) ([]byte, error) { f, err := os.Open(path); if err != nil { return nil, err } b, err := io.ReadAll(f); return b, err }
5. [log-analysis] Given this one-hour sample of API access logs (format: time method path status latency_ms), analyze the traffic pattern, surface anomalies, and give operational recommendations: 10:01 GET /v1/models 200 12; 10:01 POST /v1/chat/completions 200 3421; 10:02 POST /v1/chat/completions 429 5; 10:02 POST /v1/chat/completions 200 5410; 10:03 GET /v1/models 200 9; 10:03 POST /v1/chat/completions 500 78; 10:04 POST /v1/chat/completions 200 4102; 10:05 GET /health 200 1; 10:05 POST /v1/chat/completions 200 6230; 10:06 POST /v1/chat/completions 429 4; 10:07 POST /v1/chat/completions 200 3871; 10:08 GET /v1/models 200 11; 10:09 POST /v1/chat/completions 200 5540; 10:10 POST /v1/chat/completions 502 120
6. [summary] Read the product introduction below and produce a structured summary (target users, core features, pricing model, differentiation — at most three bullets each): “YoloRouter is a developer-facing AI model routing gateway. It aggregates multiple upstream model services behind one OpenAI-compatible endpoint; callers switch models by pointing base_url at the gateway. Provider-level failover is built in: one model can map to several providers and requests degrade automatically to the next one, invisibly to the caller. It also provides API key management, per-account usage analytics and budget caps. Deployment is a single binary with SQLite built in, or PostgreSQL. Pricing is upstream cost + 5%, no monthly fee.”
7. [knowledge] Why does TCP need a three-way handshake rather than two? Explain from the protocol's design goals (preventing historical connections, synchronizing initial sequence numbers, confirming two-way communication) and what breaks with two.
8. [knowledge] Explain the four database transaction isolation levels: what each solves, which anomalies remain (dirty read, non-repeatable read, phantom read), and MySQL InnoDB's default level and how it is implemented.
9. [translate] Translate this English technical documentation into Chinese, preserving terminology: “The gateway normalizes every inbound request into an intermediate representation before dispatching it to an upstream provider. This decouples the ingress protocol spoken by the caller from the egress protocol spoken by the provider, so a new provider can be added without touching any caller-side code. Streaming responses are relayed chunk-by-chunk with backpressure, and usage reported by the upstream is reconciled into the audit log at settlement time.”
10. [rewrite] Rewrite this rambling email to be concise and professional, keeping every key fact (time, place, agenda, prep work): “hi 大家好，是这样的，我们本来定在下周三下午的开会时间，因为会议室被占了，所以现在改到周四上午十点了，地点还是老地方 B 栋 301。这次会议主要想跟大家同步一下二季度的进度，然后讨论一下下个季度的计划，另外呢，麻烦大家提前把自己负责模块的数据准备好，最好是能发我一份，我在会上统一汇总，谢谢大家配合，有什么问题随时找我。”

off / on are the upstream-reported completion_tokens (thinking tokens included); r = (off − on) / off. Positive r means the concise prompt shortened the output.

Run the frozen questions on your own instance — in the original Chinese, as above — 3 rounds each: all of them with the switch off, then all of them again with it on. Take the median of the pair ratios. Your numbers will not match ours — models drift — but the shape should: a positive median with a wide spread around it, and a meaningful minority of pairs coming out negative.

caveman is a widely shared "answer in terse caveman-speak" skill whose description claims a 65% cut in output tokens. To check whether the switch's two-sentence prompt leaves that much on the table, the benchmark was rerun with three arms instead of two:

• off — no system prompt (fresh baselines, not reused from 2026-08-24)
• two-sentence prompt — the exact text the console writes, as above
• caveman — the full body of caveman's SKILL.md (YAML frontmatter stripped, ~6.5 KB of English rules), injected verbatim as the system prompt. Fetched 2026-08-28 from commit b433570.

Same 10 frozen questions, same 5 models, 3 rounds, 450 calls, none failed. One methodology note: the claude-opus-4-7 calls in this rerun were streamed, because the provider endpoint serving it timed out long non-streaming requests; completion_tokens is the upstream-reported usage in every arm either way. r is computed per (model, question, round) pair exactly as before. Token-weighted saving is 1 − (arm total ÷ off total) over each arm's summed completion_tokens (244,619 off, 206,726 two-sentence, 236,818 caveman).

Per-model medians:

How to read it:

• The two-sentence prompt's median of +12.8% replicates the 12.6% factory coefficient measured four days earlier. The coefficient stands.
• The caveman prompt's median is +16.4% — 3.6 points above the two-sentence prompt and nowhere near 65%. That is consistent with caveman's own honest-numbers page, which states that no reviewed aggregate output-reduction result is published for the skill.
• The aggressive style has a much heavier tail. Its worst pair turned a 108-token email rewrite into 1,485 tokens (−1275%); several reasoning-model pairs blew up the same way, plausibly because the model spends thinking tokens deliberating over the 6.5 KB rulebook — and thinking tokens are billed. Weighted by tokens actually paid for, caveman saved 3.2% while the two-sentence prompt saved 15.5%.
• The style prompt itself is input: ~6.5 KB (≈1.5k tokens) added to every request, versus ~75 characters for the two-sentence prompt. On short interactions that overhead alone can exceed the output saving.

off / ours / caveman are the upstream-reported completion_tokens (thinking tokens included); each r is against off in the same row.