transformers
0eae41ad - Add Code World Model (CWM) (#41199)

Commit
348 days ago
Add Code World Model (CWM) (#41199) * [wip][cwm] Code World Model stubs and setup in HF Transformers * [wip] Get other things working * [wip] Working * Tokenizer pad * fix: cwm window attn * temp remove test * temp remove test * Fixes * Temporarily add auto config remapping option until VLLM 0.11 is out * Fix model type and add layer validation * Lint, remove CwmForSequenceClassification * Lint, tests * Remove CwmForSequenceClassification * Lint * Remove intermediary layer expors/doc errorss, fix tests * Lint * run python utils/sort_auto_mappings.py --check_only * Remove Cwm processor mapping, get check_repo passing * Remove CwmTextConfig from test * Add docstring for CwmConfig * remove global_window and window_pattern params from config * Fix docstrings * Revert change to auto docstring util * lint * Fixes minus test improvements * Alter tests to simply check logits * lint * Have slow tests use repo, make CwmPretrainedModel passthrough * Remove decoder layer implementation, use Llama3Decoder + CwmAttetion * Use linear w/o bias for CwmAttention, add token-level integration test * Don't ignore config attention bias * Remove attention bias parameter entirely from config --------- Co-authored-by: galco <galco@meta.com>
Author
Parents
Loading