unstructured

37635f74 - feat: derive category_depth from heading level in the v2 (ontology) HTML parser (#4360)

feat: derive category_depth from heading level in the v2 (ontology) HTML parser (#4360) ## Problem The v2 ontology HTML parser (`partition_html(html_parser_version="v2")` → `ontology_to_unstructured_elements`) set `category_depth` from **DOM/container nesting** (Page→Column→block), so it tracked physical layout, not section hierarchy — flat (all depth 1) on single-column pages, bumped by columns elsewhere — discarding the `<h1>`/`<h2>`/`<h3>` levels the upstream producer already emits. Surfaced by a customer. ## Change 1. A shared `category_depth_from_html_tag` helper in `partition/common/metadata.py` (h1→0, h2→1, …; ListItem→list-ancestor depth) now feeds **both** the v1 parser and the v2 converter (v1 delegates to it; behavior unchanged). 2. v2 `category_depth` is now heading-level. 3. `ontology_to_unstructured_elements` is **self-sufficient**: it assigns content elements' heading-based `parent_id` via `set_element_hierarchy` itself (no longer relying on the `@apply_metadata` decorator); layout containers keep their tree `parent_id`, and the reverse converter `unstructured_elements_to_ontology` rebuilds nesting from those + document order, so the round-trip stays exact. `@apply_metadata`'s later `set_element_hierarchy` pass is a verified no-op. ## Tests 367 passing (html partition + ontology round-trip + `set_element_hierarchy`); new tests for heading-level depth, multi-column non-bump, self-sufficient `parent_id`, and round-trip. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic The v2 (ontology) HTML parser now derives `category_depth` from HTML heading level (`h1`->0, `h2`->1, …) and assigns heading-based `parent_id`s; layout nesting no longer bumps depth. Aligns with v1. - **Refactors** - Added shared `category_depth_from_html_tag` used by v1 and v2: `Title` → heading level; `ListItem` → list-ancestor count; others → `None`. - `ontology_to_unstructured_elements` now applies `set_element_hierarchy` to its output: content gets heading-based `parent_id`; layout containers keep tree `parent_id`; inline merge uses a transient nesting depth (not `category_depth`). - `unstructured_elements_to_ontology` rebuilds layout from container elements + document order (ignores content `parent_id`), preserving round-trip. - **Bug Fixes** - Empty input to `unstructured_elements_to_ontology` returns an empty `Document` (no `IndexError`). - A layout container with an unknown `parent_id` nests in the current container instead of popping to root. ^{Written for commit e71e4ead70c6a5891c89d6c898b3f17d7ec8c6b0. Summary will update on new commits.} <a href="https://cubic.dev/pr/Unstructured-IO/unstructured/pull/4360?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>