unstructured

42fa45a4 - fix(csv): read CSV and TSV cells as the file holds them (#4574)

fix(csv): read CSV and TSV cells as the file holds them (#4574) ## Summary `read_delimited_text()`, which reads every CSV and TSV upload, calls `pd.read_csv()` with Pandas' default cell interpretation. That changes what the file says in three ways: | cell | `main` | this PR | |---|---|---| | `N/A`, `NULL`, `None` (and `NA`, `nan`, `n/a`, …) | empty | `N/A`, `NULL`, `None` | | `12` in a column that has an empty cell, with `include_header=True` | `12.0` | `12` | | `007` in a column of codes that has an empty cell, with `include_header=True` | `7.0` | `007` | Probe file `name,qty,status,code` / `Pen,12,N/A,007` / `Ink,,NULL,010` / `Pad,7,ok,`. On `main` with `include_header=True` it reads `name qty status code Pen 12.0 7.0 Ink 10.0 Pad 7.0 ok`, and with `include_header=False` it reads `name qty status code Pen 12 007 Ink 010 Pad 7 ok`. With this PR it reads `name qty status code Pen 12 N/A 007 Ink NULL 010 Pad 7 ok`. Without a header row, the header strings already make each column `object`, so only the `N/A` / `NULL` loss shows there. ## Change Both `pd.read_csv()` calls in `read_delimited_text()` pass `dtype=str, keep_default_na=False`. Every cell is the text the file holds, and an empty field is an empty string. The HTML table still writes an empty cell as empty. Delimiter sniffing, the lone-`\r` reader and the cell-count limit are unchanged. Related: #4505 does the same for XLSX (`keep_default_na=False`), and #4556 keeps XLSX whole numbers (`dtype=object`). Each PR only adds read options to its own reader. ## Testing - New `test_partition_csv_reads_each_cell_as_written` (both `include_header` values) and `test_partition_tsv_reads_each_cell_as_written` fail on `main`. - The rest of `test_csv.py` and `test_tsv.py` passes (97 in total). - The 12 CSV/TSV files in `example-docs` partition to exactly the same element text and HTML as on `main`. - ruff 0.15.10 check and format are clean, and `scripts/version-sync.sh -c` passes with `0.27.34-dev1`. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4574?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="View guided diff" src="https://www.cubic.dev/buttons/review-in-cubic-light.svg"></picture></a> <!-- End of auto-generated description by cubic. -->