unstructured

104b585d - fix: fallback character-set detection for file-like objects (#4438)

fix: fallback character-set detection for file-like objects (#4438) ## Summary `FileTypeDetectionContext.text_head()` applied fallback character-set detection only to file paths. For file-like objects it decoded with `errors="ignore"`, silently stripping undecodable characters — corrupting the text head (and downstream file-type classification) for non-UTF-8 streams such as S3/GCS objects and API uploads. ## Fix Mirror the file-path branch: decode strictly with the declared encoding, and on failure fall back to the existing `detect_file_encoding(file=content)` (charset-normalizer + common-encoding fallback). Also resolves the TODO that flagged this gap. ## Test - Updated `and_it_uses_character_detection_to_correct_a_wrong_encoding_arg_for_a_file_like_object` (previously asserted the silent-empty-string behavior) to expect the corrected text head, matching the file-path case. - Added `and_it_detects_a_non_utf8_file_like_object_instead_of_stripping_its_characters` (latin-1 `BytesIO` regression for #4434). ## Verify ```bash uv run --no-sync pytest 'test_unstructured/file_utils/test_filetype.py::Describe_FileTypeDetectionContext' -q ``` Local results: the two new/updated tests pass; `ruff check`/`ruff format --check` clean on changed files. Note: ~24 `mimetype_magic_detection...` tests fail locally because libmagic is not installed on this macOS box — they fail identically on `main` (pre-existing environmental failures, unrelated to this change). `__version__.py` bumped to 0.26.1 and a CHANGELOG entry added per the contribution checklist. ## AI Disclosure Root-cause analysis and the initial patch were AI-assisted; the fallback approach was verified against the existing `detect_file_encoding()` helper (which already accepted a `file=` argument), the test was corrected to latin-1-encodable content, and the final diff was human-reviewed. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4438?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com>