Skip to content

fix: allow AutoMergingRetriever to pass through parentless matched documents - #13042

Open
0xamlab wants to merge 2 commits into
deepset-ai:mainfrom
0xamlab:fix-automerging-parentless-doc
Open

0xamlab wants to merge 2 commits into
deepset-ai:mainfrom
0xamlab:fix-automerging-parentless-doc

Conversation

@0xamlab

@0xamlab 0xamlab commented Sep 30, 2026

Copy link
Copy Markdown

Fixes #13029

Problem

AutoMergingRetriever.run() and run_async() raised ValueError: The matched leaf documents do not have the required meta field '__parent_id' when a matched document had no parent (e.g. the root document of the hierarchy or any document outside the tree).

The retriever's merge logic already has a pass-through branch for documents without __parent_id (see _try_merge_level), but _check_valid_documents rejected them before they could reach it.

Change

Remove the truthiness check for __parent_id from _check_valid_documents while keeping the presence checks for __level and __block_size (which can legitimately be 0 for the root).

Parentless matched documents are now returned unchanged; normal leaf-to-parent merging remains intact.

Tests

  • Added sync + async regression tests for a parentless document passing through unchanged.
  • Added sync + async tests mixing parentless and leaf documents to ensure merging still works.
  • Removed the obsolete test_run_missing_parent_id / test_run_mixed_valid_and_invalid_documents tests that asserted the old error.

Ran:

hatch run test:unit test/components/retrievers/test_auto_merging_retriever.py test/components/retrievers/test_auto_merging_retriever_async.py
hatch run fmt

All 33 retriever tests pass and formatting/lint checks pass.


This contribution is from Algenta (thyn-ai).

@0xamlab
0xamlab requested a review from a team as a code owner September 30, 2026 16:25
@0xamlab
0xamlab requested review from julian-risch and removed request for a team September 30, 2026 16:25
@vercel

vercel Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor

@0xamlab is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

CLAassistant commented Sep 30, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @0xamlab, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot
HaystackBot removed the request for review from julian-risch September 30, 2026 17:29
@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Sep 30, 2026
@HaystackBot
HaystackBot marked this pull request as draft September 30, 2026 17:29
@saiprasanth-git

Copy link
Copy Markdown

I independently reproduced #13029 and prepared a local fix before seeing this PR. The production change here matches the minimal fix I tested: remove the parent-ID truthiness guard while retaining the __level and __block_size presence checks. I won't open a competing PR.

One test-coverage suggestion: the new mixed parentless/leaf tests describe verifying that merging still works, but each passes only leaf1 from a parent with two children at threshold=0.6. The ratio is 1/2, so no merge occurs; the assertions verify leaf/root pass-through instead. Could you also add paired sync/async cases passing both siblings alongside an unrelated parentless root, and assert the parent replaces the leaves while the root is preserved?

Additional edge cases I covered locally are real HierarchicalDocumentSplitter roots with absent, null, and empty-string __parent_id, both alone and mixed with leaves. I also retained mixed-valid/invalid input validation coverage by making the invalid document lack __level rather than deleting that behavior test entirely.

My local retriever and hierarchical-splitter suites pass together: 62 tests. That result is for my local patch, not this PR's checkout; I have reviewed your diff but have not run your branch's suite. This is community feedback, not a maintainer approval. The investigation and tests were prepared with Perplexity Computer AI assistance.

@0xamlab

0xamlab commented Sep 30, 2026

Copy link
Copy Markdown
Author

Thanks @saiprasanth-git — those were exactly the coverage gaps. I added sync + async cases where both siblings merge into the parent while a parentless root is preserved, real HierarchicalDocumentSplitter roots with absent/None/"" __parent_id (alone and mixed with leaves), and a mixed valid/invalid case that keeps the __level validation pinned. All 39 retriever tests pass and hatch run fmt is clean.

@HaystackBot
HaystackBot marked this pull request as ready for review September 30, 2026 20:31
@HaystackBot HaystackBot removed the cla-pending PR is in draft until the contributor signs the CLA label Sep 30, 2026
@HaystackBot

Copy link
Copy Markdown
Contributor

Thanks for signing the CLA, @0xamlab! 🎉 This PR is now ready for review again and the reviewer has been re-assigned.

@saiprasanth-git

Copy link
Copy Markdown

Thanks for addressing the requested cases. The added sync/async coverage, splitter-root cases, and mixed-validity validation test cover the gaps I identified. Looks good from my side.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AutoMergingRetriever raises ValueError when a matched document has no parent (for example the root document)

4 participants