Knowledge Base: add a standalone recall test for retrieval tuning #2762

Is your feature request related to a problem? Please describe.

When tuning a knowledge base, users currently need to go through an Agent or Workflow debugging flow to inspect what was actually retrieved. That works for debugging a complete downstream request, but it is relatively heavy when the goal is only to answer a narrower question:

For this knowledge base and this query, which chunks are retrieved under a given retrieval configuration, with what scores and sources?

There are several related community signals:

• In [Roadmap] Coze Studio Q4 2025 Product Roadmap #2218, a user directly suggested adding a recall/retrieval test for evaluating a knowledge base.
• 感觉知识库嵌入和召回的能力好差，有什么改进办法 #2334 and 知识库默认的分段结果和召回效果比较差，请问实践过程中是否有好的替代方案？ #2459 report difficulty diagnosing poor chunking / retrieval quality.
• coze知识库检索出来的数据为什么会凭空少很多？ #1857 reports large differences between direct Milvus retrieval and Coze Workflow retrieval under apparently similar inputs.

The problem is not that Coze has no retrieval observability at all. Agent/Workflow debugging can already show recalled chunks. The missing piece is a lightweight knowledge-base-level retrieval tuning loop that does not require first constructing or entering a downstream orchestration flow.

Describe the solution you'd like

Add a standalone Recall Test / Retrieval Playground on the Knowledge Base detail page.

A minimal version could:

1. Let the user enter a single query.
2. Let the user provide temporary, test-scoped retrieval settings such as search strategy, TopK, and MinScore.
3. Reuse the same production knowledge retrieval path rather than introducing a separate retrieval implementation.
4. Show ranked retrieved chunks with at least final score and source document information.
5. Make zero-result cases explicit together with the active test settings.
6. Keep all test settings ephemeral: running a test should not modify the Knowledge Base itself or any Agent/Workflow configuration.

For a first version, I would intentionally keep the scope small: no batch dataset evaluation, no Recall@K / MRR / NDCG metrics, no experiment history, and no retrieval algorithm changes.

Describe alternatives you've considered

• Use Agent/Workflow debugging only. This is useful for end-to-end debugging, but it mixes knowledge retrieval diagnosis with downstream orchestration and makes quick retrieval tuning slower.
• Improve retrieval quality directly. That is still important, but repeated reports such as coze知识库检索出来的数据为什么会凭空少很多？ #1857 / 感觉知识库嵌入和召回的能力好差，有什么改进办法 #2334 are difficult to diagnose when users cannot isolate query → retrieval result → score → configuration at the Knowledge Base layer.
• Expose more fields only in the Workflow knowledge-retrieval node. knowledge_retrieve add return key #2253 is a useful historical example: maintainers noted that exposing more retrieval data requires coordinated frontend and backend behavior. A Knowledge-level test surface seems cleaner for this specific tuning workflow than only extending downstream node output.

Additional context

The current codebase already appears to have much of the underlying foundation:

• the knowledge retrieval domain path returns retrieved slices with scores;
• the Knowledge Base has its own IDE/detail surface where a test action could naturally live;
• the public frontend still contains an autogenerated RetrieveTestReq/Resp contract and a generated client path for /api/devops/knowledge_platform/v1/retrieve_test originating from the earlier Bytedance code mirror.

I am not assuming that the old generated RetrieveTest contract is intended to be restored in the open-source product; it may be an internal-only historical artifact. I am mentioning it only because it suggests that this workflow has been modeled before.

Before implementing, I would like to align with the maintainers on three points:

1. Is a standalone Knowledge-level retrieval test something you want in the open-source product, or should retrieval inspection intentionally remain inside Agent/Workflow debugging?
2. Is the existing generated RetrieveTest contract still conceptually relevant, or should an OSS-native API be designed around the current knowledge retrieval domain path?
3. If the direction is welcome, which Knowledge UI surface and API boundary would you prefer for a minimal first version?

If this direction fits the project, I would be happy to take ownership of the scoped implementation after we align on the boundary.