Add support of OPT models (#2205)
* add opt replace policy
* simplify inf. api
* fix opt replace policy
* fix use-cash & add relu
* Add support of custom MLP act. function
* Revert "simplify inf. api"
This reverts commit 9e910fcbd5471dec9b3c92008426f5ba590bf0b6.
* fix the inference API (temp. solution)
* fix code formatting
* add unit tests for OPT models.
* refactor pre-attention layer norm configuration
* add support of opt-350m model
* refactor the HF model config initialization
* fix hf model config issue
Co-authored-by: Reza Yazdani <reyazda@microsoft.com>
Co-authored-by: Jeff Rasley <jerasley@microsoft.com>
Co-authored-by: Reza Yazdani <44502768+RezaYazdaniAminabadi@users.noreply.github.com>
