Falcon port #24523
Rocketknight1
merged
82 commits into
main
from
falcon_port
miladm
commented
on 2023-07-05
Initial commit
37f084a6
Update src/transformers/models/falcon/configuration_falcon.py
f7008b46
Update src/transformers/models/falcon/configuration_falcon.py
b3278e06
Cleanup config docstring
d0c0db5a
Update src/transformers/models/falcon/configuration_falcon.py
89a24884
Convert to relative imports
3e9f11ef
Remove torch < 1.8 warning
9633df5f
Restructure cos_sin header
a4b91cde
qkv -> query, key, value
f31d0ea6
Refactor attention calculation
97a7aa73
Add a couple of config variables to account for the different checkpo…
31e24c72
Successful merging of the code paths!
88fca168
Fix misplaced line in the non-parallel attention path
92da7fe2
Update config and tests
e5c67d0d
Add a pad_token_id when testing
d4d91bd7
Support output_attentions when alibi is None
8febeb9d
make fixup
40010512
Skip KV cache shape test
8900244c
No more _keys_to_ignore_on_load_missing
eb1de871
Simplify self attention a bit
3609c58e
Simplify self attention a bit
03819d4d
make fixup
625baa73
stash commit
2c42786d
Some more attention mask updates
57e8ad14
Should pass all tests except assisted generation!
5cbec392
Add big model generation test
92484ab7
make fixup
677781a6
Add temporary workaround for test
80d8d201
Test overrides for assisted generation
e53bf96e
Update src/transformers/models/falcon/modeling_falcon.py
aef8be76
Update src/transformers/models/falcon/modeling_falcon.py
025435e8
Update src/transformers/models/falcon/modeling_falcon.py
d06ce2fc
Update tests/models/falcon/test_modeling_falcon.py
3b396073
Test overrides for assisted generation
2f738005
Add generation demo
eb83a309
Update copyright
65207ccc
Make the docstring model actually small
05585536
Add module-level docstring
b88bd8b1
Remove all assertions
e68c77cb
Add copied from bloom
ee9a72f9
Reformat the QKV layer
21763d58
Add copied from bloom
a6ecc913
Update src/transformers/models/falcon/modeling_falcon.py
6620500e
Remove unused line and reformat
8123582b
No single letter variables
96f6c787
Cleanup return names
56f44430
Add copied from line
f1ff143f
Remove the deprecated arguments blocks
339d4653
Change the embeddings test to an alibi on/off test
67e530fd
Remove position_ids from FalconForQA
ae3cfc6a
Remove old check for token type IDs
7623f3a6
Fix the alibi path when multi_query is False
b2facf7c
Update src/transformers/models/falcon/modeling_falcon.py
1c6353bc
Update src/transformers/models/falcon/modeling_falcon.py
b9a2e267
Update tests/models/falcon/test_modeling_falcon.py
5863002b
Update config naming
456b5239
Fix typo for new_decoder_architecture
7a035aed
Add some comments
0e661eb0
Fix docstring
f5709ad3
Fix docstring
a9e0d0e0
Create range in the right dtype from the start
3e0b9784
Review comment cleanup
58dda7e4
n_head_kv -> num_kv_heads
17c4b9da
self.alibi -> self.use_alibi
57fe9f24
self.num_kv -> self.num_kv_heads
302cc955
Reorder config args
b744ccdc
Made alibi arguments Optional
e2f2db1e
Add all model docstrings
3b948311
Add extra checkpoints
ab662380
Add author info for Falcon
94c33e9c
Stop removing token_type_ids because our checkpoints shouldn't return…
131be5f8
Add one hopeful comment for the future
e11ff0cc
Fix typo
0de24ed2
Update tests, fix cache issue for generation
2bdc92df
Use -1e9 instead of -inf to avoid float overflow
e7117195
Recompute the rotary embeddings much less often
d7176e27
Re-enable disabled tests
21f84bd6
One final fix to attention mask calculation, and update tests
33593888
Cleanup targeting falcon-40b equivalency
fb4e5557
Post-rebase docs update
28d55b9f
Update docstrings, especially in the config
c1a3559e
More descriptive variable names, and comments where we can't rename them
5125fb9a
Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub