transformers
[WIP][Fix] GLM 5 set `apply_rotary_pos_emb` to `is_neox_style=False` && remove `F.relu()`
#45017
Open
Overview
Commits
7
Changes
View On GitHub

[WIP][Fix] GLM 5 set `apply_rotary_pos_emb` to `is_neox_style=False` && remove `F.relu()` #45017

JaredforReal wants to merge 7 commits into huggingface:main from JaredforReal:fix/glm5
JaredforReal
JaredforReal fix: rope interleave=Ture means is_neox_style=False
8edcfb62
JaredforReal fix comments and masking
d02261e0
JaredforReal get rid of relu
c75f2544
JaredforReal JaredforReal requested a review from copilot-pull-request-reviewer copilot-pull-request-reviewer 2 days ago
github-actions
copilot-pull-request-reviewer
copilot-pull-request-reviewer commented on 2026-03-26
JaredforReal revert mask
dbd35d72
JaredforReal JaredforReal changed the title [Fix] GLM 5 set `apply_rotary_pos_emb` to `is_neox_style=False` && remove `F.relu()` [WIP][Fix] GLM 5 set `apply_rotary_pos_emb` to `is_neox_style=False` && remove `F.relu()` 2 days ago
JaredforReal pre-commit
02cd1f7f
JaredforReal get rid of is_neox_style
df3482f1
JaredforReal pre-commit
2037e155
ArthurZucker
ArthurZucker approved these changes on 2026-03-26
JaredforReal
HuggingFaceDocBuilderDev
ArthurZucker

Login to write a write a comment.

Login via GitHub

Reviewers
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Terms Of Use Privacy Policy Refund Policy Impressum