llm-foundry
Allow parameters with requires_grad=False in meta init
#1567

Merged

Allow parameters with requires_grad=False in meta init #1567

sashaDoubov merged 7 commits into mosaicml:main from sashaDoubov:sasha/allow-requires-grad-false

add requires grad

ebb8b894

Merge remote-tracking branch 'upstream/main' into sasha/meta-init

34b8c803

Merge branch 'main' into sasha/allow-requires-grad-false

3caa43f0

sashaDoubov marked this pull request as ready for review 2 years ago

sashaDoubov requested a review 2 years ago

mvpatel2000 commented on 2024-10-03

Update llmfoundry/models/utils/meta_init_context.py

9dc2de07

add test

d17de6a4

potentially fix formatting

e0dd608a

fix loss fn

bd0d2c7a

mvpatel2000 approved these changes on 2024-10-07

sashaDoubov merged 7c6c9f1f into main 2 years ago

Reviewers

mvpatel2000

Assignees

No one assigned

Labels

None yet

Milestone

No milestone