Extend save_pretrained to offloaded models #27412
muellerzr
merged
197 commits into
huggingface:main
from
blbadger:extended-save
added hidden subset
e9c05dc6
Merge pull request #1 from blbadger/master
9c4b96a3
debugged hidden subset contrastive search
fc521667
Merge pull request #2 from blbadger/master
f16ac72c
added contrastive search compression
3906d608
Merge pull request #3 from blbadger/master
40ebe761
debugged compressed contrastive search
2881aef1
Merge pull request #4 from blbadger/master
4ddf45b3
memory reduction for contrastive search
7d29c550
Merge pull request #5 from blbadger/master
b0b98cb4
debugged mem red
57dfaacb
Merge pull request #6 from blbadger/master
a4192450
added low memory option feature
fd0e19f6
Merge pull request #7 from blbadger/master
fc03ab24
debugged mem optmimization output stack
802cfd4b
debugged mem optmimization output stack
0632f061
Merge pull request #8 from blbadger/master
83189689
debugged low mem
9bad256e
Merge pull request #9 from blbadger/master
8fa17317
added low mem cache
a89bb8e8
Merge pull request #10 from blbadger/master
cdbd070f
fixed 2047 tensor view
f90f9489
Merge pull request #11 from blbadger/master
65feec91
debugged 2042 past key val inputs
e1718c37
Merge pull request #12 from blbadger/master
089a2993
reformatted tensors
3fd54e67
Merge pull request #13 from blbadger/master
6d6ac75c
changed low mem output
12d5aea5
Merge pull request #14 from blbadger/master
89f9b13d
final clean
44a9ec42
removed subset hidden csearch
37bb62d9
fixed hidden device
68c1cd86
fixed hidden device
e199ddc9
changed compressor dtype
8ace5a32
removed hstate compression
1ac80a09
integrated csearch in generate
1c3aae7d
test csearch integration into generation
f18bccd6
fixed csearch kwarg integration with generation
abf0a72e
final wrap and added doc
e517d5f3
Update src/transformers/generation/utils.py
cc1ea6d4
Update src/transformers/generation/utils.py
bd2e36b1
Update src/transformers/generation/utils.py
b59ec6d1
added debug print
a7fb76e6
direct hstate cat
961a1bab
direct hstate cat
882b6d2f
direct hstate cat debug
c3f3db37
direct hstate cat debug
692b5e10
expanded full hidden state stack
349bbf95
expanded full hidden state stack
cd4bed0e
matched dims for hstates
ae41c50c
matched dims for hstates
30baaa65
logits fix
ebc19ffa
equality test
752a488a
equality hidden debug
4f973ba9
debug
b8094156
added prints for debug
9230061c
added prints for debug
2863471b
equality check
e6533536
switched squeeze dim
d790ea58
input format debug
f1942219
tracing top_k_ids
665c323c
removed trace
6259b56d
Merge pull request #16 from blbadger/equal-csearch
55561bbb
Merge branch 'huggingface:main' into main
7f52d877
added test context
6d2734cf
Merge pull request #17 from blbadger/equal-csearch
a873dfd4
added jitter
4033b191
added jitter
e2051a72
added jitter
e8f4cd1e
returned state
6bed1971
rebuilt past key value reconstruction
67946f21
debugged
3dbd7762
cleaned traces
547df692
added selection for pkv
f4b1f284
changed output to dict
d1af0f00
Merge pull request #18 from blbadger/selected-pkv
fbb11b59
cleaned
ee94a31d
cleaned
5cfd4546
Merge pull request #19 from blbadger/selected-pkv
b63ec63e
cleaned up contrastive search test
2fbca358
Merge pull request #20 from blbadger/selected-pkv
29b16f73
moved low_memory kwarg
efcba6f0
debugged
5a3b26c5
Merge pull request #21 from blbadger/selected-pkv
fb337c3e
changed low mem test batch size to 1
cf122309
removed output
60fd1850
Merge pull request #22 from blbadger/selected-pkv
0e4fd99e
debugged test input shape
a3355c1c
Merge pull request #23 from blbadger/selected-pkv
704e9b1d
reformatted csearch test
87be0de5
Merge pull request #24 from blbadger/selected-pkv
85644370
added trace
ab307f98
removed unsqueeze on final forward pass
dfff73d8
replaced unsqueeze with view
0334d12e
removed traces
06dacc03
cleaned
94d6dd9d
Merge pull request #25 from blbadger/selected-pkv
fe78f81d
debugged model kwargs
a2293dd9
Merge pull request #26 from blbadger/selected-pkv
150d1a15
removed special models from test
0deba213
Merge pull request #27 from blbadger/selected-pkv
5237cf0b
Merge branch 'main' into main
05c408e3
Merge branch 'huggingface:main' into main
f9bd670f
ran make quality
1aa72793
Merge branch 'huggingface:main' into main
8129e2a9
Update src/transformers/generation/configuration_utils.py
871cf594
Update src/transformers/generation/configuration_utils.py
ef6bfd67
refactored
bad2d186
refactored
f16f2e7b
refactored
af70bef3
Merge pull request #28 from blbadger/selected-pkv
d82e792c
make fixup
2d21e64c
Merge pull request #29 from blbadger/selected-pkv
f310f83c
renamed flag sequential
bf3a0739
renamed flag sequential
b11c156d
Merge pull request #30 from blbadger/selected-pkv
c619204a
Merge branch 'huggingface:main' into main
1ae9d4a5
Merge branch 'huggingface:main' into main
7d5e65fe
Merge branch 'huggingface:main' into main
75d0f312
iterative onloading
57e73360
black style and test utils
fabd8fee
added traces for integrated test
3a4d0884
debugged
bcb5b66d
added traces
baf2c143
make style
0639dee9
removed traces, make style
9cdfa175
muellerzr
commented
on 2023-11-10
muellerzr
commented
on 2023-11-10
included suggestions and added test
93317418
debugged test
f5166a7f
added offload module check and make style
40beac10
is_accelerate_available and make style
f1d9335a
muellerzr
commented
on 2023-11-15
added test decorator
840cd58e
SunMarc
commented
on 2023-11-15
changed test model and config spec
931f3e47
added offload condition
70755cb2
added lazy loading for each shard
7d4c32e3
debugged
dbc7830d
modified sharding
d569da0f
debugged
a33f25a3
added traces
604b61c2
removed safe serialization
016457f8
no index overload;
de669bc5
trace on safe save ptrs
9873ba57
added ptr condition
dae9ef40
debugged
56943d0c
debugged ptr
e5950324
moved module map init
aa4d143f
remake shard only for offloaded modules
e2a82677
refactored
43e9c10e
debugged
13627301
refactored
721b40e3
debugged
50ab59ae
Merge pull request #31 from blbadger/lazy-sharder
60f465a6
cleaned and make style
c87c9985
Merge branch 'huggingface:main' into extended-save
ef423c3d
Merge branch 'extended-save' of github.com:blbadger/transformers
4be2a46c
cleaned and make style
3d380648
added trace
d1bc1563
sparse module map
58228235
debugged
f3f14e71
removed module map conditional
01bc0d9c
refactored
a90e6856
debug
5c47e6e3
debugged
05d3825b
added traces
2d88621d
added shard mem trace
3b907e46
added shard mem trace
9058abd3
removed underlying storage check
30067c37
refactored
1ce0950f
memory leak removal and make style
9115fed0
cleaned
76cc8cf3
Merge pull request #32 from blbadger/memory-test
cd782281
swapped test decs and make style
7f9725e5
Merge branch 'huggingface:main' into extended-save
4e8ac920
added mem checks and make style
c5559c4f
Merge branch 'huggingface:main' into extended-save
c85ff355
added free mem warning
0d324515
Merge branch 'huggingface:main' into extended-save
df170e93
implemented some suggestions
5ed37df3
moved onloading to accelerate
061973ae
Merge branch 'main' into extended-save
595dd262
refactored for accelerate integration
3e58d2bc
cleaned test
98b9fc80
make style
3effcad1
debugged offload map name
43794719
cleaned and make style
bdf508fb
replaced meta device check for sharding
b568bdd4
cleaned and make style
5a2b10ea
Merge branch 'huggingface:main' into extended-save
51dd1c2a
implemented some suggestions
10743489
more suggestions
38802a2e
update warning
74967d85
more suggestions
ec37d6d3
make style
520b19e7
new make style
7eaa2687
Update src/transformers/modeling_utils.py
f3a4e30a
Update src/transformers/modeling_utils.py
5a79199f
Update src/transformers/modeling_utils.py
19469941
muellerzr
approved these changes
on 2024-06-05
Update src/transformers/modeling_utils.py
0d22bdc4
ArthurZucker
commented
on 2024-06-18
Reviewers
Assignees
Labels
None yet
Milestone
No milestone
Login to write a write a comment.
Login via GitHub