DeepSpeed
c33bc4fd - use num_kv only when it has positive value

Commit

1 year ago

use num_kv only when it has positive value

References

#4351 - DS-Inference Quantization refresh: Fix several issues and add more features

#4313 - Add the policy to run llama model from the official repo

Author

Reza Yazdani

Parents

Files2

csrc/transformer/inference/csrc
- pt_binding.cpp
- transform.cu