unstructured
feat: introduce language detection function for text partitioning function
#1453
Merged
Overview
Commits
20
Changes
View On GitHub

feat: introduce language detection function for text partitioning function #1453

shreyanid merged 20 commits into main from lang-detection-2
shreyanid
shreyanid shreyanid requested a review from amanda103 amanda103 3 years ago
shreyanid shreyanid marked this pull request as ready for review 3 years ago
amanda103
amanda103 commented on 2023-09-19
amanda103
amanda103 commented on 2023-09-19
amanda103
amanda103 commented on 2023-09-19
amanda103
amanda103 commented on 2023-09-19
amanda103
amanda103 commented on 2023-09-19
shreyanid shreyanid force pushed from b69e7c1a to dc2a8c75 3 years ago
amanda103
amanda103 approved these changes on 2023-09-21
shreyanid shreyanid force pushed from a723533f to 5db281ff 3 years ago
shreyanid shreyanid force pushed from 07126be9 to dd770535 3 years ago
shreyanid shreyanid force pushed from dd770535 to 9b51e35f 3 years ago
shreyanid checkout changes from previous branch due to rebasing mess
dc389fb5
shreyanid pip compile and add multi lang test
1e48746b
shreyanid move langdetect seed call into function to ensure it gets called
8c90cb86
shreyanid add docstring
87462bd9
shreyanid update CHANGELOG
3fadb932
shreyanid add todo about chinese variants
05f3c80f
shreyanid responding to feedback
3de4a16d
shreyanid add comment about ignoring other inputs when auto is also provided
20e10a1c
shreyanid lint
2af3eef6
newelh Add languages to metadata (#1470)
eeed0c43
shreyanid lint
a9af7663
shreyanid add warning for ignored extra inputs with auto value
2578f16e
shreyanid bump dev version and update pip compile
6ee25350
ryannikolaidis feat: introduce language detection function for text partitioning fun…
636e9d76
shreyanid shreyanid force pushed from 8697fbdf to 636e9d76 3 years ago
shreyanid add languages as ingest metadata key
1e9b7a03
ryannikolaidis feat: introduce language detection function for text partitioning fun…
1089c529
amanda103
amanda103 commented on 2023-09-26
tabossert Update docker download url to use scarf gateway (#1523)
3244b3c1
Fixed Sphinx warning errors (#1438)
b36f5d3f
shreyanid metadata addition to changelog
64cd8295
shreyanid Merge branch 'main' into lang-detection-2
3a1b966f
shreyanid shreyanid enabled auto-merge (squash) 3 years ago
shreyanid shreyanid merged 32bfebcc into main 3 years ago
shreyanid shreyanid deleted the lang-detection-2 branch 3 years ago

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum