transformers
Substantially reduce memory usage in _update_causal_mask for large batches by using .expand instead of .repeat
#29413

Merged

Commits

try to fix gemma mem use

nqgl committed 2 years ago
fix: handle attention mask dim==2 case

nqgl committed 2 years ago
remove logits=logits.float()

nqgl committed 2 years ago
clean up + add llama

nqgl committed 2 years ago
apply formatting

nqgl committed 2 years ago
readability edit: swap order of items being multiplied

nqgl committed 2 years ago
Merge branch 'huggingface:main' into causal_mask_fix

nqgl committed 2 years ago
revert change unrelated to PR

nqgl committed 2 years ago
revert black autoformat

nqgl committed 2 years ago
Merge pull request #2 from nqgl/remove_repeat_to

nqgl committed 2 years ago
switch to one .to

nqgl committed 2 years ago
Merge pull request #3 from nqgl/remove_repeat_to

nqgl committed 2 years ago
Accept style edits

nqgl committed 2 years ago