unstructured
238657f6 - fix: chunking dropping table content (#4352)

Commit
79 days ago
fix: chunking dropping table content (#4352) ## Preserve mixed-content tail text in table HTML ### Summary `HtmlTable.from_html_text` compactification cleared every element's `.tail`, silently dropping any text between inline children of a `<td>`. In practice this deleted real content like `": do not..."` after `<b>NOTICE</b>`, bullet lines following each `<br/>`, and any inter-tag prose in rich table cells — visible whenever a table was chunked (any strategy: `basic`, `by_title`, `by_page`). Pure-whitespace tails are still stripped (those are just pretty-print indent). Tails carrying content are now kept, with internal whitespace runs collapsed and a single leading/trailing space preserved so adjacent text doesn't run together. ### Changes - `unstructured/common/html_table.py`: replace the blanket `e.tail = None` with whitespace-aware normalization. - `test_unstructured/common/test_html_table.py`: two regression tests covering mixed inline markup, `<br/>`-separated lines, and whitespace collapsing. - Bump to `0.22.28` + changelog entry. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Low risk: localized change to table HTML minification that preserves previously-dropped text; main risk is minor whitespace/serialization differences in downstream table chunking output. > > **Overview** > Fixes `HtmlTable.from_html_text()` compactification to **stop deleting meaningful mixed-content tail text** (text between inline children like `<b>foo</b> bar` or between `<br/>` siblings), while still dropping pure formatting/indent whitespace and collapsing internal whitespace runs. > > Adds regression tests for mixed inline markup and `<br/>`-separated cell content, and bumps the library to `0.22.28` with a changelog entry describing the fix. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit fe27d3582a40cf1d27e8617491d9b1214057605e. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
Author
Parents
Loading