unstructured

73b42953 - perf(chunking): bound image payload and original-element serialization memory (#4517)

perf(chunking): bound image payload and original-element serialization memory (#4517) ## Summary A stream of empty images can hold every base64 payload until a later text element even when `include_orig_elements=False` and those image fields are dropped from output. Fold metadata incrementally for contiguous empty-image runs and release excluded image payloads immediately, preserving metadata order and chunk boundaries. Original-element serialization now adjusts and encodes one element at a time, escapes large JSON strings in bounded fragments (values without large strings still use one standard `json.dumps` call), and spills compressed buffers above 1 MiB to temporary disk. Avoid the redundant deep copy of the whole original-element graph in `ElementMetadata.to_dict()`. ## Validation - Chunking, original-element staging and element suites pass locally; the new regression cases also pass in an isolated Linux container. - 120 mixed-image/text/title/table/page-break cases with overlap preserve chunk boundaries and metadata against the identity-preserving path. Original Image identity remains intact with the default `include_orig_elements=True`. - Exact legacy JSON field order is preserved, including continuation metadata and empty originals. - Exact legacy compressed bytes match for empty inputs, multiple elements, long Unicode/control-character strings, metadata precision and sorted JSON keys on the same zlib implementation. - Many streamed large image payloads in a Linux container: chunking peak RSS drops to under a quarter and runs faster; serialization peak RSS drops by roughly an order of magnitude with slightly lower runtime. Output hashes match exactly. - Under a constrained container memory limit, both `main` cases run out of memory; both cases on this branch complete with no OOM events. - Metadata nested as deeply as the standard JSON encoder accepts still serializes: the large-string scan is iterative, not recursive (regression test with 600 nested lists). - Repository Ruff checks/format and the Linux version-sync checker pass. ## Operational impact Compressed original-element buffers above 1 MiB use temporary disk. Final serialized output and metadata actually included in chunks still require memory proportional to output. `include_orig_elements=True` continues retaining original objects; changing that identity contract would require a separate API decision. ## Release Targets **0.27.32** (0.27.31 was taken by #4538, which merged first.) Released together with #4419, #4516, #4517, #4524, #4523, #4562 (XLSX workbook reader) and #4563 (dense worksheets). Each PR takes the next `0.27.32-devN` in that merge order (0.27.32-dev2 here); whichever merges after another rebases and renumbers the CHANGELOG heading, as usual for this repo.