unstructured

620cde78 - fix(hi_res): recover text inside PDF figure overlays (#4363)

fix(hi_res): recover text inside PDF figure overlays (#4363) ## Problem On `hi_res`, text drawn into a figure/XObject overlay (rather than the main content stream) is dropped from the output, even though it is real, selectable embedded text. Root cause: - The text lives as loose `LTChar`s inside an `LTFigure`. `strategy="fast"` returns it fine (it recurses all children), but `hi_res` does not. - `hi_res`'s `process_page_layout_from_pdfminer` only extracts text from page objects exposing `.get_text()` (e.g. `LTTextBox`); an `LTFigure` has none, so it took the image-only branch and the figure's text was discarded. - Even routed to the text branch, `extract_text_objects` only collects `LTTextLine`, and these characters aren't grouped into lines — so they were missed. - The text therefore never reached the layout merge (verified: the extracted layout entering the merge contained none of it). Not OCR, not the layout merge/clean steps. ## Fix - New `extract_text_lines_from_loose_chars()` groups loose `LTChar`s inside a container into text lines, inserting a space where the inter-character gap is wide enough to mark a word/phrase break (so spatially separated phrases are not concatenated), and skipping render-mode-3 (hidden) and rotated characters so hidden OCR layers and rotated watermarks are not resurfaced. - `process_page_layout_from_pdfminer` now also recovers text from `LTFigure`-style containers (previously image-only). `fast` already surfaces this text via `_extract_text`; this brings `hi_res` to parity while keeping its layout/table structure. ## Verification Local `hi_res` recovers the previously-dropped figure text; ingest expected outputs updated accordingly (#4365); CI green. ## Risk `hi_res` now extracts text from figure containers generally — figure-heavy PDFs (charts/diagrams with stray glyphs) may gain text elements. The render-mode/rotated gating limits noise; reviewers may want to sanity-check such PDFs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com> Co-authored-by: qued <qued@users.noreply.github.com>