Add embedding models configurable, from both transformers.js and TEI (#646)
* Add embedding models configurable, from both Xenova and TEI
* fix lint and format
* Fix bug in sentenceSimilarity
* Batches for TEI using /info route
* Fix web search disapear when finish searching
* Fix lint and format
* Add more options for better embedding model usage
* Fixing CR issues
* Fix websearch disapear in later PR
* Fix lint
* Fix more minor code CR
* Valiadate embeddingModelName field in model config
* Add embeddingModel into shared conversation
* Fix lint and format
* Add default embedding model, and more readme explanation
* Fix minor embedding model readme detailed
* Update settings.json
* Update README.md
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update README.md
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Apply suggestions from code review
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Resolved more issues
* lint
* Fix more issues
* Fix format
* fix small typo
* lint
* fix default model
* Rn `maxSequenceLength` -> `chunkCharLength`
* format
* add "authorization" example
* format
---------
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Nathan Sarrazin <sarrazin.nathan@gmail.com>
Co-authored-by: Mishig Davaadorj <dmishig@gmail.com>