ruff
Replace LALRPOP parser with hand-written parser
#10036
Merged
Overview
Commits
114
Changes
View On GitHub

Replace LALRPOP parser with hand-written parser #10036

dhruvmanila merged 114 commits into main from dhruv/parser
dhruvmanila
dhruvmanila dhruvmanila added parser
dhruvmanila dhruvmanila marked this pull request as draft 1 year ago
dhruvmanila dhruvmanila force pushed from 3209b854 to d2472423 1 year ago
dhruvmanila dhruvmanila force pushed from d2472423 to a6956720 1 year ago
codspeed-hq
dhruvmanila dhruvmanila force pushed from 96addbfa to b5cc384b 1 year ago
dhruvmanila dhruvmanila force pushed from b07c0089 to 70731cbe 1 year ago
github-actions
dhruvmanila dhruvmanila force pushed from cf89468d to b803d4f8 1 year ago
dhruvmanila dhruvmanila force pushed from 81e8050d to 4f5604fc 1 year ago
dhruvmanila dhruvmanila force pushed from 1cf79546 to 707d62ad 1 year ago
dhruvmanila dhruvmanila force pushed from 236aa4ac to c16f7b77 1 year ago
dhruvmanila dhruvmanila force pushed from 7167eafd to 61b5487b 1 year ago
dhruvmanila dhruvmanila force pushed from 48ce841e to 6d26ec6c 1 year ago
dhruvmanila dhruvmanila force pushed from 6d26ec6c to 183cf6b8 1 year ago
dhruvmanila dhruvmanila force pushed from f19aaf24 to 6e7e2be5 1 year ago
dhruvmanila dhruvmanila force pushed from ef330fbc to d8cd78c9 1 year ago
dhruvmanila dhruvmanila force pushed from df9e7665 to 700ef443 1 year ago
dhruvmanila dhruvmanila force pushed from cefa7534 to 509be822 1 year ago
dhruvmanila dhruvmanila added this to the v0.4.0 milestone 1 year ago
dhruvmanila dhruvmanila force pushed from 9dc43070 to 9f8ea614 1 year ago
dhruvmanila dhruvmanila force pushed from 9f8ea614 to 45d59b19 1 year ago
dhruvmanila dhruvmanila force pushed from 38fb3d43 to c30057ab 1 year ago
MichaReiser
dhruvmanila dhruvmanila added performance
LaBatata101 Replace LALRPOP parser with hand-written parser
1475a4d2
dhruvmanila Fix tests and clippy warnings
e4fc0f49
dhruvmanila Assert the parser is at augmented assign token (#10269)
a09b11cd
dhruvmanila Encapsulate `Program` fields (#10270)
40ad261a
dhruvmanila Rename to include "token" in method name (#10287)
7c021af1
dhruvmanila Remove deprecated parsing list functions (#10271)
804eb0e3
dhruvmanila Improve various assignment target error (#10288)
691d3ebf
dhruvmanila Remove `skip_until` parser method (#10293)
2aa8f4ca
dhruvmanila Remove `Pattern::Invalid` variant (#10294)
1d98cbf1
dhruvmanila Move parser quick test to pretty print code
c6797649
dhruvmanila Fix tests and error recovery token sets (#10338)
cfca1ff2
dhruvmanila Remove `FStringElement::Invalid`, improve parsing logic (#10327)
0dfd4dda
dhruvmanila Introduce `LexicalError::InvalidByteLiteral` (#10328)
1c7c3232
dhruvmanila Merge string parsing to single entrypoint (#10339)
6220e790
dhruvmanila Set new parser for benchmarks
93b9d891
LaBatata101 Remove f-string `UnclosedLbrace` error checking from the lexer (#10372)
5eff87de
dhruvmanila Use string flags to mark it as invalid (#10385)
60fd525a
dhruvmanila Remove `Expr::Invalid` (#10386)
f23a0997
dhruvmanila Use recovery context to decide on trailing comma (#10405)
5b632519
dhruvmanila Fix clippy warnings
5399bce0
dhruvmanila Move parser tests to new valid/invalid framework (#10471)
7e1c1aef
dhruvmanila Fix string parsing error ranges (#10406)
08bfd2c7
dhruvmanila Remove unreachable error case while f-string parsing (#10474)
8da7aba3
dhruvmanila Use `TokenId` to track parser progress (#10486)
1bb22d1f
dhruvmanila Update snapshots from rebase
33d1453c
dhruvmanila Update with item parsing to avoid infinite lookaheads (#10418)
b5ca9572
dhruvmanila Remove `peek_nth`, only allow 1 lookahead (#10443)
44b2ac4b
dhruvmanila Wrap around when incrementing token id (#10544)
bce8784f
dhruvmanila Update `ExprName` ctx, add tests (#10552)
9fd5fc83
dhruvmanila Complete end expression token set (#10553)
8498e27d
dhruvmanila Better error recovery for match patterns (#10477)
e150b1c5
dhruvmanila Implement rules around star expressions with different precedence (#1…
2f9f8582
dhruvmanila Add tests for list expression parsing (#10554)
a1ae0cb7
dhruvmanila Add tests for set expression parsing (#10555)
21f8be91
dhruvmanila Add tests for dictionary expression parsing (#10556)
79db220d
dhruvmanila Limit comprehension as per the grammar (#10633)
6e467f46
dhruvmanila Add tests for list comprehension (#10634)
d77bd9b5
dhruvmanila Add tests for set comprehension (#10635)
c01d4321
dhruvmanila Add tests for dictionary comprehension (#10636)
73455379
dhruvmanila Add tests for parenthesized expressions, tuple, generator (#10641)
21a289aa
dhruvmanila Add tests for subscript and slice expressions (#10642)
a3a0bb0f
dhruvmanila Implement `Display` for `TokenKind` (#10678)
9e822da4
dhruvmanila Add support to write inline tests for the parser (#10681)
d5fcd072
dhruvmanila Remove numbered list for parser contributing doc
393a7eca
dhruvmanila Limit `bitwise_or` precedence for starred expression with error (#10657)
d6ab61d5
dhruvmanila Report error for `yield` and `starred` in bitwise or parsing (#10748)
c5061fff
dhruvmanila Add tests for yield and yield from expressions (#10658)
13fd991e
dhruvmanila Add tests for named expression (#10659)
e5fbf81e
dhruvmanila Add tests for conditional expression (#10660)
cbdb0558
dhruvmanila Add precedence error handling in parsing LHS expression (#10679)
6a8c4f88
dhruvmanila Add tests for await expression (#10680)
17d763e1
dhruvmanila Add tests for attribute expression (#10725)
765dd6d4
dhruvmanila Add tests for unary expressions (#10726)
23943f00
dhruvmanila Add tests for boolean operation expression (#10727)
ed13e67d
dhruvmanila Add tests for compare operator expressions (#10730)
34657944
dhruvmanila Add tests for binary operator expressions (#10731)
883a8c6d
dhruvmanila Add tests for function call arguments (#10733)
b5b20bd4
dhruvmanila Add tests for call expression (#10734)
efd55173
dhruvmanila Add tests for lambda expression body (#10737)
b89068fc
dhruvmanila Add bitflags dependency
115e7319
dhruvmanila Add tests for string and f-string literals (#10757)
1e3a7a2e
dhruvmanila Add tests for `global`, `nonlocal` (#10758)
6f86b6e8
dhruvmanila Add tests for `return` statement (#10767)
f415d5d8
dhruvmanila Add tests for `raise` statement (#10768)
8f4c6182
dhruvmanila Add tests for `assert` statement (#10769)
db3a0b00
dhruvmanila Add tests for `del` statement (#10770)
513b52d2
dhruvmanila Add tests for `import` and `from ... import` statement (#10771)
893f0934
dhruvmanila Remove unreferenced snapshot
bc975545
dhruvmanila Use `test_ok` for inline tests, merge test report (#10801)
51269d50
dhruvmanila Avoid error handling duplication for starred, yield, lambda expressio…
03aabcfb
dhruvmanila Fix with items parsing for yield, starred, named expr (#10811)
18f8b75b
dhruvmanila Add tests for type alias statement (#10785)
b16d586a
dhruvmanila Add tests for all kinds of assignment statement (#10786)
9812eecf
dhruvmanila Add tests for `if` statement (#10789)
1407194e
dhruvmanila Add test cases for `try` statement (#10815)
5a7eec9d
dhruvmanila Rename to include "token" in method name (#10287)
7c021af1
dhruvmanila Remove deprecated parsing list functions (#10271)
804eb0e3
dhruvmanila Improve various assignment target error (#10288)
691d3ebf
dhruvmanila Move parser quick test to pretty print code
c6797649
dhruvmanila Fix tests and error recovery token sets (#10338)
cfca1ff2
dhruvmanila Merge string parsing to single entrypoint (#10339)
6220e790
dhruvmanila Set new parser for benchmarks
93b9d891
LaBatata101 Remove f-string `UnclosedLbrace` error checking from the lexer (#10372)
5eff87de
dhruvmanila Use string flags to mark it as invalid (#10385)
60fd525a
dhruvmanila Remove `Expr::Invalid` (#10386)
f23a0997
dhruvmanila Use recovery context to decide on trailing comma (#10405)
5b632519
dhruvmanila Move parser tests to new valid/invalid framework (#10471)
7e1c1aef
dhruvmanila Remove unreachable error case while f-string parsing (#10474)
8da7aba3
dhruvmanila Update with item parsing to avoid infinite lookaheads (#10418)
b5ca9572
dhruvmanila Wrap around when incrementing token id (#10544)
bce8784f
dhruvmanila Update `ExprName` ctx, add tests (#10552)
9fd5fc83
dhruvmanila Complete end expression token set (#10553)
8498e27d
dhruvmanila Better error recovery for match patterns (#10477)
e150b1c5
dhruvmanila Add tests for list expression parsing (#10554)
a1ae0cb7
dhruvmanila Add tests for dictionary expression parsing (#10556)
79db220d
dhruvmanila Limit comprehension as per the grammar (#10633)
6e467f46
dhruvmanila Add tests for list comprehension (#10634)
d77bd9b5
dhruvmanila Add tests for set comprehension (#10635)
c01d4321
dhruvmanila Add tests for dictionary comprehension (#10636)
73455379
dhruvmanila Add tests for subscript and slice expressions (#10642)
a3a0bb0f
dhruvmanila Implement `Display` for `TokenKind` (#10678)
9e822da4
dhruvmanila Add support to write inline tests for the parser (#10681)
d5fcd072
dhruvmanila Limit `bitwise_or` precedence for starred expression with error (#10657)
d6ab61d5
dhruvmanila Report error for `yield` and `starred` in bitwise or parsing (#10748)
c5061fff
dhruvmanila Add tests for named expression (#10659)
e5fbf81e
dhruvmanila Add tests for conditional expression (#10660)
cbdb0558
dhruvmanila Add precedence error handling in parsing LHS expression (#10679)
6a8c4f88
dhruvmanila Add tests for attribute expression (#10725)
765dd6d4
dhruvmanila Add tests for boolean operation expression (#10727)
ed13e67d
dhruvmanila Add tests for binary operator expressions (#10731)
883a8c6d
dhruvmanila Add tests for function call arguments (#10733)
b5b20bd4
dhruvmanila Add tests for call expression (#10734)
efd55173
dhruvmanila Add tests for lambda expression body (#10737)
b89068fc
dhruvmanila Add tests for `global`, `nonlocal` (#10758)
6f86b6e8
dhruvmanila Add tests for `return` statement (#10767)
f415d5d8
dhruvmanila Add tests for `raise` statement (#10768)
8f4c6182
dhruvmanila Add tests for `assert` statement (#10769)
db3a0b00
dhruvmanila Add tests for `import` and `from ... import` statement (#10771)
893f0934
dhruvmanila Remove unreferenced snapshot
bc975545
dhruvmanila Add tests for type alias statement (#10785)
b16d586a
dhruvmanila Add tests for all kinds of assignment statement (#10786)
9812eecf
dhruvmanila Add tests for `while` statement (#10817)
ca7e8613
dhruvmanila Add tests for decorators (#10818)
5f3dae95
dhruvmanila Add tests for `async` statement (#10819)
a63878dd
dhruvmanila Add tests for function definition (#10829)
af224e67
dhruvmanila Add tests for `match` statement (#10831)
d351b8d3
dhruvmanila fixup! Add tests for `match` statement (#10831) (#10853)
d2aff029
dhruvmanila Move IPython test snapshots
e4c53cd1
dhruvmanila Remove `is_current_token_postfix` (#10855)
d3141e99
dhruvmanila Move remaining tests to new framework (#10856)
a7089323
dhruvmanila Use `Semi` as terminator token for simple stmts (#10859)
5edcf467
dhruvmanila Add tests for top level statement parsing (#10860)
52f36599
dhruvmanila Move arguments and delete target validation in `Parser` (#10871)
7b4a1c08
dhruvmanila Add `display` to `TokenKind` for error message (#10870)
dbb56b96
dhruvmanila Review error enum variants and messages (#10872)
1be2041f
dhruvmanila Improve IPython escape command parsing logic (#10873)
f45d7990
dhruvmanila Fix parameter range if it includes `*`, `**` (#10875)
52aed8d8
dhruvmanila Reject corpus which produces empty range for LALRPOP parser (#10876)
b09a89fd
dhruvmanila Expect `for` after `async` instead of `bump` (#10877)
02e63451
dhruvmanila Pre-compute the module range for LALRPOP parser (#10879)
08a478f0
dhruvmanila Move expr token set, docs improvements (#10887)
8ef7d36f
dhruvmanila Consider exactly `_` for wildcard pattern (#10896)
b156d0bb
dhruvmanila Fix clippy
0780f789
dhruvmanila fixup! Pre-compute the module range for LALRPOP parser (#10879)
d61c6890
dhruvmanila Add tests for simple target in annotated assign stmt (#10897)
dbb0e2f1
dhruvmanila Disallow dotted name in `from ... import` statement (#10903)
f1f7657e
dhruvmanila Expect `,` for parenthesized with items and not at `)` (#10910)
36795359
dhruvmanila Avoid panic in invalid pattern recovery (#10966)
1c1d0ece
dhruvmanila Recover from unexpected token at end of expr (#10981)
f3a90a2c
dhruvmanila Reset parser context in parenthesized expression (#10994)
04a70b4f
dhruvmanila Remove invalid named expr test from formatter (#10995)
f464a429
dhruvmanila Add `if` as sequence pattern list terminator (#10996)
80f5419a
dhruvmanila Preserve comma with starred expr in subscript (#10997)
6b73fea6
dhruvmanila Remove LALRPOP based parser (#10998)
e5e36927
dhruvmanila Remove leftover fuzz target
924ed5ea
dhruvmanila dhruvmanila force pushed from e7b19dd8 to 924ed5ea 1 year ago
dhruvmanila dhruvmanila marked this pull request as ready for review 1 year ago
dhruvmanila dhruvmanila requested a review from MichaReiser MichaReiser 1 year ago
MichaReiser
MichaReiser approved these changes on 2024-04-18
MichaReiser
dhruvmanila Remove leftover `Invalid` enum variants
98a76691
addisoncrump
dhruvmanila dhruvmanila merged 13ffb5bc into main 1 year ago
dhruvmanila dhruvmanila deleted the dhruv/parser branch 1 year ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
performance parser
Milestone
v0.4.0
Terms Of Use Privacy Policy Refund Policy Impressum