transformers
[`GPTNeoX`] Flex Attention + Refactor
#34896
Merged
Overview
Commits
20
Changes
View On GitHub
Commits
  • gpt neox flex attention + refactor
    vasqu committed 1 year ago
  • some formatting
    vasqu committed 1 year ago
  • small fix on dropout
    vasqu committed 1 year ago
  • add assertion on flex attn test
    vasqu committed 1 year ago
  • flaky ci :(
    vasqu committed 1 year ago
  • add head mask support
    vasqu committed 1 year ago
  • style
    vasqu committed 1 year ago
  • handle dtype, replace torch where
    vasqu committed 1 year ago
  • fixup flex with output attns
    vasqu committed 1 year ago
  • code review and several other fixes
    vasqu committed 1 year ago
  • Update src/transformers/modeling_utils.py
    vasqu committed 1 year ago
  • style
    vasqu committed 1 year ago
  • remove unnecessary comment
    vasqu committed 1 year ago
  • remove incorrect comment
    vasqu committed 1 year ago
  • make flex attn check more agnostic tor versions and centralized
    vasqu committed 1 year ago
  • change peft input dtype check to value since q and k could be affected by other stuff like RoPE
    vasqu committed 1 year ago
  • i forgor
    vasqu committed 1 year ago
  • flaky
    vasqu committed 1 year ago
  • code review and small fixes
    vasqu committed 1 year ago
  • Update src/transformers/models/gpt_neox/modeling_gpt_neox.py
    vasqu committed 1 year ago
FAQ Terms Privacy Refunds Impressum
Loading