nvda
36cc63cd - Add ICU sentence segmentation to OffsetsTextInfo (#20603)

Commit
7 days ago
Add ICU sentence segmentation to OffsetsTextInfo (#20603) Part of #18901. Follow up for #20602. ### Summary of the issue: Sentence navigation only works in Microsoft Word and Outlook, which have native sentence support. Browse mode in virtual buffers uses TextInfos based on `OffsetsTextInfo`, and its `_getSentenceOffsets` raised `NotImplementedError`. Working sentence navigation in those documents therefore needs a sentence backend first. ### Description of user facing changes: No direct changes. This PR is the necessary building block: programmatic sentence navigation is now possible in cursor managers and in TextInfos based on offsets. The next PR in the stack routes the browse mode sentence gestures to it. ### Description of developer facing changes: `winBindings.icu` gains `UBRK.SENTENCE`. `textUtils.icu` gains `calculateSentenceOffsets`. `OffsetsTextInfo._getSentenceOffsets` is implemented, so `UNIT_SENTENCE` becomes available on all TextInfos derived from it. ### Description of development approach: Sentence boundaries come from the Windows built-in ICU BreakIterator using UAX#29 default rules. Segmentation runs over the containing paragraph, so sentences can span lines. Results are rebased by the paragraph start offset. The root locale is used: UAX#29 sentence boundaries are driven by the language-neutral Sentence_Break property, and locale only adds abbreviation tailoring. A fixed locale also keeps boundaries a deterministic function of position, which the `move`/`expand` tiling invariant requires. When ICU is unavailable (Windows before version 1703), the method keeps raising `NotImplementedError`. ### Testing strategy: New unit tests in `test_sentenceSegIcu.py` cover the `calculateSentenceOffsets` primitive (English, Japanese, abbreviation splits, surrogate pairs, containment) and the `OffsetsTextInfo` integration, including forward and backward tiling across a block-paragraph fixture. ### Known issues with pull request: Abbreviations such as "Dr." end a sentence. Locale-specific tailoring could address this later. `documentNavigation.sentenceHelper` still uses its own logic to detect sentence boundaries.
Author
Parents
Loading