langchain
c3d4126e - community[minor]: add user agent for web scraping loaders (#22480)

Commit
1 year ago
community[minor]: add user agent for web scraping loaders (#22480) **Description:** This PR adds a `USER_AGENT` env variable that is to be used for web scraping. It creates a util to get that user agent and uses it in the classes used for scraping in [this piece of doc](https://python.langchain.com/v0.1/docs/use_cases/web_scraping/). Identifying your scraper is considered a good politeness practice, this PR aims at easing it. **Issue:** `None` **Dependencies:** `None` **Twitter handle:** `None`
Author
Parents
Loading