Add websearch controls for assistants (#812)
* remove query modifiers from generateQuery
* Add backend for assistant RAG
* Add front-end for updating RAG assistant
* enable web parser to return plaintext directly for matching headers
* Update websearch flow for handling assistant rag preferences
* Add our old blocklist to .env.template
* Enable websearch to run on messages depending on assistant requirements
* reorganized imports
* Rename vars
* use projection
* Add environment variable for assistant rag
* fix assistant rag on runwebsearch
* fix styling if rag is disabled
* make sure we always omit credentials when fetching web pages
* Add new checks for SSRF, with a new env var `ENABLE_LOCAL_FETCH`
* Use DNS to check if the links are local or not
* Add a websearch indicator
* Add more tags to parser
* Add indicators
* Display RAG options in settings view
* ui
* fix rag detection
* bit more spacing
* fix button position in assistant form
* wording (mainly)
* reduce number of tags
* Bump max URLs from 3 to 10
* add ul and ol to parseWeb
* change splitting string
* link style
* wording
* add feedback link
* Update src/routes/settings/(nav)/assistants/[assistantId]/+page.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update src/routes/settings/(nav)/assistants/[assistantId]/+page.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update src/routes/assistants/+page.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update src/routes/settings/(nav)/assistants/[assistantId]/+page.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update src/lib/components/chat/ChatWindow.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update src/routes/settings/(nav)/assistants/[assistantId]/+page.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* Update src/lib/components/AssistantSettings.svelte
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
* lint
* throw error if not a string
* simplify rag check
---------
Co-authored-by: Mishig <mishig.davaadorj@coloradocollege.edu>
Co-authored-by: Victor Mustar <victor.mustar@gmail.com>
