transformers
c61fcde9 - Enhance DataCollatorForLanguageModeling with Configurable Token Replacement Probabilities (#35251)

Commit
338 days ago
Enhance DataCollatorForLanguageModeling with Configurable Token Replacement Probabilities (#35251) * DataCollatorForLanguageModeling class was updated with new parameters that provides more control over the token masking and relacing * DataCollatorForLanguageModeling class was updated with new parameters that provides more control over the token masking and relacing * Addressed review comments, modified the docstring and made a test for the DataCollatorForLanguageModeling
Parents
Loading