fix(docx): preserve field display text (fldSimple, MACROBUTTON) - #2553
Rohan Patnaik (rohan-patnaik) wants to merge 1 commit into
Conversation
Mammoth has no handler for w:fldSimple, so simple fields are dropped along with the child runs holding their cached result, and it never emits field instructions, so MACROBUTTON fields (whose display text is the trailing argument of the instruction, with no result runs) vanish entirely. Text carried in Word fields was silently missing from the Markdown output. Add a _pre_process_fields step for document/footnotes/endnotes XML that unwraps w:fldSimple elements so their cached result runs convert like ordinary content, and rewrites the MACROBUTTON instruction of fields without a separator (simple or complex) into text runs holding only the display-text argument. Other instructions are still never emitted, and complex fields with a separator are untouched since Mammoth already converts their result runs. Fixes microsoft#1247
8b9f76e to
bf7054a
Compare
PRABHU KIRAN VANDRANKI (VANDRANKI)
left a comment
There was a problem hiding this comment.
Community review, does not clear the merge gate.
I read the full _pre_process_fields function in packages/markitdown/src/markitdown/converter_utils/docx/pre_process.py and traced both code paths against the new test fixtures in test_docx_fields.py.
Complex-field path (fldChar/instrText): the field_stack correctly tracks nesting with [has_separator, instr_elements], only collecting instrText runs before a separate marker is seen (if field_stack and not field_stack[-1][0]). On end, a field with a separator is left alone (continue), matching the claim that Mammoth already converts cached result runs correctly. For a separator-less MACROBUTTON, I manually walked the position math: position tracks how far into the concatenated instruction string each instrText run's text starts, and keep = text[max(match.start(1) - position, 0):] correctly slices out only the portion of each run that falls at or after the display-text argument's start. I traced this against the test's two-run case (" MACROBUTTON NoMacro " + "Observation 1: Latency is reduced.") and the math produces an empty first run (removed) and the full second run (kept, renamed to w:t), matching the test's assertions that "Observation 1..." appears but "MACROBUTTON"/"NoMacro" don't.
Simple-field path (fldSimple): a field with existing cached child content ("".join(field.itertext()) non-empty) is unwrapped as-is; a MACROBUTTON fldSimple with no children gets a synthesized w:r/w:t run appended from macrobutton.match(field.get(namespace + "instr", "")).group(1) before the same unwrap logic runs. The unwrap itself (parent.insert(index + offset, child) for each of the field's own children, in original document order) relies on lxml's behavior of re-parenting an element that already has a parent, which is correct lxml behavior, not a bug. The tail-text reattachment (field.tail moved to the last inserted child, or the previous sibling, or the parent's own .text) covers all three cases correctly.
I also checked the wiring in pre_process_docx: _pre_process_fields is added to word/document.xml, word/footnotes.xml, and word/endnotes.xml but correctly left out of word/styles.xml, and the early-exit guard (if b"fldSimple" not in content and b"instrText" not in content: return content) avoids the parse/reserialize cost on documents with no fields, matching test_pre_process_fields_returns_fieldless_content_unchanged.
I did not execute the test suite myself, but I traced the logic by hand against the fixture bytes in test_docx_fields.py and the two match. I don't see a bug in either the complex-field position-slicing logic or the simple-field unwrap/tail logic. This looks like a correct, carefully-scoped fix for issue #1247.
Fixes #1247.
Summary
DOCX conversion dropped visible text from simple fields and separator-less
MACROBUTTONfields. This adds a preprocessing step that:w:fldSimplecached result runs, using theMACROBUTTONdisplay argument only when needed;MACROBUTTONfields without exposing other field instructions; andTests
b8f79c5and pass here.The reporter's currently hosted 3GPP sample no longer contains fields, so this fixes the field-loss mechanism without changing that reprocessed file.