DeepSpeed
2a2356b0 - reset cache only if using inferece_cuda module

Commit

2 years ago

reset cache only if using inferece_cuda module

References

#2669 - Reset KV-cache at the beginning of text-generation

Author

Reza Yazdani

Parents

Loading