unstructured
47f4728e - mem: reduce PaddleOCR rec_batch_num from 6 to 1 (#4295)

Commit
110 days ago
mem: reduce PaddleOCR rec_batch_num from 6 to 1 (#4295) Reduce PaddleOCR `rec_batch_num` from 6 (default) to 1. Paddle's native inference engine allocates 500 MiB memory arena chunks proportional to recognition batch size. With batch_num=6, four chunks are allocated during text recognition. Setting it to 1 reduces this to one chunk. ![benchmark](https://raw.githubusercontent.com/codeflash-ai/codeflash/pr-assets/images/paddle-rec-batch-num-bench.png) | Setting | Peak memory | |---------|------------| | `rec_batch_num=6` | 7,184 MiB | | `rec_batch_num=1` | 2,684 MiB | | **Delta** | **-4,500 MiB (-62.6%)** | Measured with `memray run` on `layout-parser-paper-with-table.pdf` through `partition()` with hi_res + PaddleOCR table OCR. On CPU, batch processing doesn't parallelize — it's sequential within `predictor.run()`. Smaller batches just allocate less workspace memory. ## Reproduce Requires `unstructured[pdf]`, `paddlepaddle`, `unstructured-paddleocr`, and `memray`. ```bash cat > /tmp/bench_paddle.py << 'SCRIPT' from unstructured.partition.auto import partition elements = partition( filename="example-docs/layout-parser-paper.pdf", strategy="hi_res", pdf_infer_table_structure=True, ocr_agent="unstructured.partition.utils.ocr_models.paddle_ocr.OCRAgentPaddle", ) print(f"Partitioned: {len(elements)} elements") SCRIPT # Baseline (main branch, rec_batch_num=6): git checkout main memray run --native --trace-python-allocators -o /tmp/paddle_baseline.bin /tmp/bench_paddle.py memray stats /tmp/paddle_baseline.bin | grep "Peak memory" # With this change (rec_batch_num=1): git checkout mem/paddle-rec-batch-num memray run --native --trace-python-allocators -o /tmp/paddle_opt.bin /tmp/bench_paddle.py memray stats /tmp/paddle_opt.bin | grep "Peak memory" ```
Author
Parents
Loading