ruff

a54061e7 - [ty] Fix empty spans following a line terminator and unprintable character spans in diagnostics (#19535)

[ty] Fix empty spans following a line terminator and unprintable character spans in diagnostics (#19535) ## Summary This was previously the last commit in #19415, split out to make it easier to review. This applies the fixes from c9b99e4, 5021f32, and 2922490cb8 to the new rendering code in `ruff_db`. I initially intended only to fix the empty span after a line terminator (as you can see in the branch name), but the two fixes were tied pretty closely together, and my initial fix for the empty spans needed a big change after trying to handle unprintable characters too. I can still split this up if it would help with review. I would just start with the unprintable characters first. The implementation here is essentially copy-pasted from `ruff_linter::message::text.rs`, with the `SourceCode` struct renamed to `EscapedSourceCode` since there's already a `SourceCode` in scope in `render.rs`. It's also updated slightly to account for the multiple annotations for a single snippet. The original implementation used some types from the `line_width` module from `ruff_linter`. I copied over heavily stripped-down versions of these instead of trying to import them. We could inline the remaining code entirely, if we want, but I thought it was nice enough to keep. I also moved over `ceil_char_boundary`, which is unchanged except to make it a free function taking a `&str` instead of a `Locator` method. All of this code could be deleted from `ruff_linter` if we also move over the `grouped` output format, which will be the last user after #19415. ## Test Plan I added new tests in `ruff_linter` that call into the new rendering code to snapshot the diagnostics for the affected cases. These are copies of existing snapshots in Ruff, so it's helpful to compare them. These are a bit noisy because of the other rendering differences in the header, but all of the `^^^` indicators should be the same. <details><summary>`empty_span_after_line_terminator` diff</summary> ```diff diff --git a/crates/ruff_linter/src/rules/pycodestyle/snapshots/ruff_linter__rules__pycodestyle__tests__E112_E11.py.snap b/crates/ruff_linter/src/message/snapshots/ruff_linter__message__text__tests__empty_span_after_line_terminator.snap index 5ade4346e0..6df75c16f0 100644 --- a/crates/ruff_linter/src/rules/pycodestyle/snapshots/ruff_linter__rules__pycodestyle__tests__E112_E11.py.snap +++ b/crates/ruff_linter/src/message/snapshots/ruff_linter__message__text__tests__empty_span_after_line_terminator.snap @@ -1,17 +1,20 @@ --- -source: crates/ruff_linter/src/rules/pycodestyle/mod.rs +source: crates/ruff_linter/src/message/text.rs +expression: value.to_string() --- -E11.py:9:1: E112 Expected an indented block +error[no-indented-block]: Expected an indented block + --> E11.py:9:1 | 7 | #: E112 8 | if False: 9 | print() - | ^ E112 + | ^ 10 | #: E113 11 | print() | -E11.py:9:1: SyntaxError: Expected an indented block after `if` statement +error[invalid-syntax]: SyntaxError: Expected an indented block after `if` statement + --> E11.py:9:1 | 7 | #: E112 8 | if False: @@ -21,7 +24,8 @@ E11.py:9:1: SyntaxError: Expected an indented block after `if` statement 11 | print() | -E11.py:12:1: SyntaxError: Unexpected indentation +error[invalid-syntax]: SyntaxError: Unexpected indentation + --> E11.py:12:1 | 10 | #: E113 11 | print() @@ -31,7 +35,8 @@ E11.py:12:1: SyntaxError: Unexpected indentation 14 | mimetype = 'application/x-directory' | -E11.py:14:1: SyntaxError: Expected a statement +error[invalid-syntax]: SyntaxError: Expected a statement + --> E11.py:14:1 | 12 | print() 13 | #: E114 E116 @@ -41,17 +46,19 @@ E11.py:14:1: SyntaxError: Expected a statement 16 | create_date = False | -E11.py:45:1: E112 Expected an indented block +error[no-indented-block]: Expected an indented block + --> E11.py:45:1 | 43 | #: E112 44 | if False: # 45 | print() - | ^ E112 + | ^ 46 | #: 47 | if False: | -E11.py:45:1: SyntaxError: Expected an indented block after `if` statement +error[invalid-syntax]: SyntaxError: Expected an indented block after `if` statement + --> E11.py:45:1 | 43 | #: E112 44 | if False: # ``` </details> <details><summary>`unprintable_characters` diff</summary> ```diff diff --git a/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLE2512_invalid_characters.py.snap b/crates/ruff_linter/src/message/snapshots/ruff_linter__message__text__tests__unprintable_characters.snap index 52cfdf9cce..fcfa1ac9f1 100644 --- a/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLE2512_invalid_characters.py.snap +++ b/crates/ruff_linter/src/message/snapshots/ruff_linter__message__text__tests__unprintable_characters.snap @@ -1,161 +1,115 @@ --- -source: crates/ruff_linter/src/rules/pylint/mod.rs +source: crates/ruff_linter/src/message/text.rs +expression: value.to_string() --- -invalid_characters.py:24:12: PLE2512 [*] Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:24:12 | 22 | cr_ok = f'\\r' 23 | 24 | sub = 'sub ' - | ^ PLE2512 + | ^ 25 | sub = f'sub ' | - = help: Replace with escape sequence +help: Replace with escape sequence -ℹ Safe fix -21 21 | cr_ok = '\\r' -22 22 | cr_ok = f'\\r' -23 23 | -24 |-sub = 'sub ' - 24 |+sub = 'sub \x1A' -25 25 | sub = f'sub ' -26 26 | -27 27 | sub_ok = '\x1a' - -invalid_characters.py:25:13: PLE2512 [*] Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:25:13 | 24 | sub = 'sub ' 25 | sub = f'sub ' - | ^ PLE2512 + | ^ 26 | 27 | sub_ok = '\x1a' | - = help: Replace with escape sequence - -ℹ Safe fix -22 22 | cr_ok = f'\\r' -23 23 | -24 24 | sub = 'sub ' -25 |-sub = f'sub ' - 25 |+sub = f'sub \x1A' -26 26 | -27 27 | sub_ok = '\x1a' -28 28 | sub_ok = f'\x1a' +help: Replace with escape sequence -invalid_characters.py:55:25: PLE2512 [*] Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:55:25 | 53 | zwsp_after_multicharacter_grapheme_cluster = f"ಫ್ರಾನ್ಸಿಸ್ಕೊ ​​" 54 | 55 | nested_fstrings = f'␈{f'{f'␛'}'}' - | ^ PLE2512 + | ^ 56 | 57 | # https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998106 | - = help: Replace with escape sequence - -ℹ Safe fix -52 52 | zwsp_after_multicharacter_grapheme_cluster = "ಫ್ರಾನ್ಸಿಸ್ಕೊ ​​" -53 53 | zwsp_after_multicharacter_grapheme_cluster = f"ಫ್ರಾನ್ಸಿಸ್ಕೊ ​​" -54 54 | -55 |-nested_fstrings = f'␈{f'{f'␛'}'}' - 55 |+nested_fstrings = f'␈{f'\x1A{f'␛'}'}' -56 56 | -57 57 | # https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998106 -58 58 | x = f"""}}ab""" +help: Replace with escape sequence -invalid_characters.py:58:12: PLE2512 [*] Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:58:12 | 57 | # https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998106 58 | x = f"""}}ab""" - | ^ PLE2512 + | ^ 59 | # https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998256 60 | x = f"""}}a␛b""" | - = help: Replace with escape sequence +help: Replace with escape sequence -ℹ Safe fix -55 55 | nested_fstrings = f'␈{f'{f'␛'}'}' -56 56 | -57 57 | # https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998106 -58 |-x = f"""}}ab""" - 58 |+x = f"""}}a\x1Ab""" -59 59 | # https://github.com/astral-sh/ruff/issues/7455#issuecomment-1741998256 -60 60 | x = f"""}}a␛b""" -61 61 | - -invalid_characters.py:64:12: PLE2512 Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:64:12 | 63 | # https://github.com/astral-sh/ruff/issues/13294 64 | print(r"""␈␛�​ - | ^ PLE2512 + | ^ 65 | """) 66 | print(fr"""␈␛�​ | - = help: Replace with escape sequence +help: Replace with escape sequence -invalid_characters.py:66:13: PLE2512 Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:66:13 | 64 | print(r"""␈␛�​ 65 | """) 66 | print(fr"""␈␛�​ - | ^ PLE2512 + | ^ 67 | """) 68 | print(Rf"""␈␛�​ | - = help: Replace with escape sequence +help: Replace with escape sequence -invalid_characters.py:68:13: PLE2512 Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:68:13 | 66 | print(fr"""␈␛�​ 67 | """) 68 | print(Rf"""␈␛�​ - | ^ PLE2512 + | ^ 69 | """) | - = help: Replace with escape sequence +help: Replace with escape sequence -invalid_characters.py:73:9: PLE2512 Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:73:9 | 71 | # https://github.com/astral-sh/ruff/issues/18815 72 | b = "\␈" 73 | sub = "\" - | ^ PLE2512 + | ^ 74 | esc = "\␛" 75 | zwsp = "\​" | - = help: Replace with escape sequence +help: Replace with escape sequence -invalid_characters.py:80:25: PLE2512 [*] Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:80:25 | 78 | # tstrings 79 | esc = t'esc esc ␛' 80 | nested_tstrings = t'␈{t'{t'␛'}'}' - | ^ PLE2512 + | ^ 81 | nested_ftstrings = t'␈{f'{t'␛'}'}' | - = help: Replace with escape sequence - -ℹ Safe fix -77 77 | -78 78 | # tstrings -79 79 | esc = t'esc esc ␛' -80 |-nested_tstrings = t'␈{t'{t'␛'}'}' - 80 |+nested_tstrings = t'␈{t'\x1A{t'␛'}'}' -81 81 | nested_ftstrings = t'␈{f'{t'␛'}'}' -82 82 | +help: Replace with escape sequence -invalid_characters.py:81:26: PLE2512 [*] Invalid unescaped character SUB, use "\x1A" instead +error[invalid-character-sub]: Invalid unescaped character SUB, use "\x1A" instead + --> invalid_characters.py:81:26 | 79 | esc = t'esc esc ␛' 80 | nested_tstrings = t'␈{t'{t'␛'}'}' 81 | nested_ftstrings = t'␈{f'{t'␛'}'}' - | ^ PLE2512 + | ^ | - = help: Replace with escape sequence - -ℹ Safe fix -78 78 | # tstrings -79 79 | esc = t'esc esc ␛' -80 80 | nested_tstrings = t'␈{t'{t'␛'}'}' -81 |-nested_ftstrings = t'␈{f'{t'␛'}'}' - 81 |+nested_ftstrings = t'␈{f'\x1A{t'␛'}'}' -82 82 | +help: Replace with escape sequence ``` </details>