Contrastive Search peak memory reduction #24120
gante
merged
117 commits into
huggingface:main
from
blbadger:main
added hidden subset
e9c05dc6
Merge pull request #1 from blbadger/master
9c4b96a3
debugged hidden subset contrastive search
fc521667
Merge pull request #2 from blbadger/master
f16ac72c
added contrastive search compression
3906d608
Merge pull request #3 from blbadger/master
40ebe761
debugged compressed contrastive search
2881aef1
Merge pull request #4 from blbadger/master
4ddf45b3
memory reduction for contrastive search
7d29c550
Merge pull request #5 from blbadger/master
b0b98cb4
debugged mem red
57dfaacb
Merge pull request #6 from blbadger/master
a4192450
added low memory option feature
fd0e19f6
Merge pull request #7 from blbadger/master
fc03ab24
debugged mem optmimization output stack
802cfd4b
debugged mem optmimization output stack
0632f061
Merge pull request #8 from blbadger/master
83189689
debugged low mem
9bad256e
Merge pull request #9 from blbadger/master
8fa17317
added low mem cache
a89bb8e8
Merge pull request #10 from blbadger/master
cdbd070f
fixed 2047 tensor view
f90f9489
Merge pull request #11 from blbadger/master
65feec91
debugged 2042 past key val inputs
e1718c37
Merge pull request #12 from blbadger/master
089a2993
reformatted tensors
3fd54e67
Merge pull request #13 from blbadger/master
6d6ac75c
changed low mem output
12d5aea5
Merge pull request #14 from blbadger/master
89f9b13d
final clean
44a9ec42
removed subset hidden csearch
37bb62d9
fixed hidden device
68c1cd86
fixed hidden device
e199ddc9
changed compressor dtype
8ace5a32
removed hstate compression
1ac80a09
integrated csearch in generate
1c3aae7d
test csearch integration into generation
f18bccd6
fixed csearch kwarg integration with generation
abf0a72e
final wrap and added doc
e517d5f3
gante
commented
on 2023-06-13
Update src/transformers/generation/utils.py
cc1ea6d4
Update src/transformers/generation/utils.py
bd2e36b1
Update src/transformers/generation/utils.py
b59ec6d1
added debug print
a7fb76e6
direct hstate cat
961a1bab
direct hstate cat
882b6d2f
direct hstate cat debug
c3f3db37
direct hstate cat debug
692b5e10
expanded full hidden state stack
349bbf95
expanded full hidden state stack
cd4bed0e
matched dims for hstates
ae41c50c
matched dims for hstates
30baaa65
logits fix
ebc19ffa
equality test
752a488a
equality hidden debug
4f973ba9
debug
b8094156
added prints for debug
9230061c
added prints for debug
2863471b
equality check
e6533536
switched squeeze dim
d790ea58
input format debug
f1942219
tracing top_k_ids
665c323c
removed trace
6259b56d
Merge pull request #16 from blbadger/equal-csearch
55561bbb
Merge branch 'huggingface:main' into main
7f52d877
added test context
6d2734cf
Merge pull request #17 from blbadger/equal-csearch
a873dfd4
added jitter
4033b191
added jitter
e2051a72
added jitter
e8f4cd1e
returned state
6bed1971
rebuilt past key value reconstruction
67946f21
debugged
3dbd7762
cleaned traces
547df692
added selection for pkv
f4b1f284
changed output to dict
d1af0f00
Merge pull request #18 from blbadger/selected-pkv
fbb11b59
cleaned
ee94a31d
cleaned
5cfd4546
Merge pull request #19 from blbadger/selected-pkv
b63ec63e
cleaned up contrastive search test
2fbca358
Merge pull request #20 from blbadger/selected-pkv
29b16f73
gante
requested changes
on 2023-06-24
moved low_memory kwarg
efcba6f0
debugged
5a3b26c5
Merge pull request #21 from blbadger/selected-pkv
fb337c3e
changed low mem test batch size to 1
cf122309
removed output
60fd1850
Merge pull request #22 from blbadger/selected-pkv
0e4fd99e
debugged test input shape
a3355c1c
Merge pull request #23 from blbadger/selected-pkv
704e9b1d
reformatted csearch test
87be0de5
Merge pull request #24 from blbadger/selected-pkv
85644370
added trace
ab307f98
removed unsqueeze on final forward pass
dfff73d8
replaced unsqueeze with view
0334d12e
removed traces
06dacc03
cleaned
94d6dd9d
Merge pull request #25 from blbadger/selected-pkv
fe78f81d
debugged model kwargs
a2293dd9
Merge pull request #26 from blbadger/selected-pkv
150d1a15
removed special models from test
0deba213
Merge pull request #27 from blbadger/selected-pkv
5237cf0b
Merge branch 'main' into main
05c408e3
Merge branch 'huggingface:main' into main
f9bd670f
ran make quality
1aa72793
Merge branch 'huggingface:main' into main
8129e2a9
gante
approved these changes
on 2023-07-07
Update src/transformers/generation/configuration_utils.py
871cf594
Update src/transformers/generation/configuration_utils.py
ef6bfd67
refactored
bad2d186
refactored
f16f2e7b
refactored
af70bef3
Merge pull request #28 from blbadger/selected-pkv
d82e792c
blbadger
commented
on 2023-07-08
make fixup
2d21e64c
Merge pull request #29 from blbadger/selected-pkv
f310f83c
renamed flag sequential
bf3a0739
renamed flag sequential
b11c156d
Merge pull request #30 from blbadger/selected-pkv
c619204a
Merge branch 'huggingface:main' into main
1ae9d4a5
amyeroberts
approved these changes
on 2023-07-20
Login to write a write a comment.
Login via GitHub