ruff
d3cd61f8 - Use empty range when there's "gap" in token source (#11032)

Commit
1 year ago
Use empty range when there's "gap" in token source (#11032) ## Summary This fixes a bug where the parser would panic when there is a "gap" in the token source. What's a gap? The reason it's `<=` instead of just `==` is because there could be whitespaces between the two tokens. For example: ```python # last token end # | current token (newline) start # v v def foo \n # ^ # assume there's trailing whitespace here ``` Or, there could tokens that are considered "trivia" and thus aren't emitted by the token source. These are comments and non-logical newlines. For example: ```python # last token end # v def foo # comment\n # ^ current token (newline) start ``` In either of the above cases, there's a "gap" between the end of the last token and start of the current token. ## Test Plan Add test cases and update the snapshots.
Author
Parents
  • crates/ruff_python_parser
    • resources/inline/err
      • File
        node_range_with_gaps.py
    • src/parser
      • File
        mod.rs
      • File
        statement.rs
    • tests/snapshots
      • invalid_syntax@node_range_with_gaps.py.snap
      • invalid_syntax@unterminated_fstring_newline_recovery.py.snap