unstructured

4fe4097f - fix: preserve attachment elements' own filetype in auto.partition() (#4460)

fix: preserve attachment elements' own filetype in auto.partition() (#4460) ## Summary When a document with attachments is partitioned, every element in the output was labelled with the **containing** file's MIME type. Content extracted from an email's attached PDF reported `filetype: message/rfc822`, so downstream consumers could not filter or route by document type. Not format-specific to Outlook — it reproduces for any email with attachments, on both the `.eml` and `.msg` paths. ## Root cause The `augment_metadata` closure in `partition()` (`unstructured/partition/auto.py`) re-stamped `metadata.filetype` on every element it was handed, unconditionally. Attachments are partitioned by a nested `partition()` call, so each attachment's elements are stamped with their correct filetype at their own recursion depth, then flattened into the containing document's element list. The outermost call then ran its own `augment_metadata` over the whole flattened list and overwrote all of them. ## Fix Skip the filetype re-stamp for elements carrying `attached_to_filename`. This matches guards the codebase already applies for exactly this case in `partition/common/metadata.py` and in two places in `file_utils/filetype.py` — so the convention is established, it just was not used by this hand-rolled closure. ### One deliberate narrowing `url` and `data_source` are still applied to attachment elements; only the filetype block is guarded. Skipping the element outright would have been the broader change, but the nested `partition()` call receives neither of those values, so attachment elements would come back with `data_source = None` — silently dropping source lineage (record locators, source URLs) for attachment-derived content. The defect here is filetype-only, so the guard is too. ## Verification An `.eml` built with a PDF and a DOCX attachment: | elements | before | after | |---|---|---| | email body (1) | `message/rfc822` | `message/rfc822` | | PDF attachment (25) | `message/rfc822` | `application/pdf` | | DOCX attachment (8) | `message/rfc822` | `…wordprocessingml.document` | ## Tests Adds `test_auto_partition_preserves_the_filetype_of_attachment_elements`, parametrized over the `.eml` and `.msg` paths using existing fixtures (no new binaries). Both cases were confirmed to fail without the fix and pass with it. `test_auto.py`, `test_email.py`, `test_msg.py` and `common/test_metadata.py`: 327 passed, 1 xfailed. Lint and format clean. ## Note The CHANGELOG diff includes a one-character trailing-whitespace removal on the pre-existing 0.27.3 entry, applied by the repo's own `trailing-whitespace` pre-commit hook. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4460?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>