xla
[FSDP] Replace all .data usages with modern approaches
#4485

Merged

[FSDP] Replace all .data usages with modern approaches #4485

alanwaketan merged 24 commits into master from alanwaketan/fsdp

alanwaketan requested a review from

ronghanghu 2 years ago

alanwaketan requested a review from

JackCaoG 2 years ago

alanwaketan assigned

alanwaketan 2 years ago

alanwaketan commented on 2023-01-20

alanwaketan force pushed from dfe01e8b to 7b71fa0a 2 years ago

alanwaketan force pushed from 7b71fa0a to d2ae6ce0 2 years ago

alanwaketan force pushed from d2ae6ce0 to 05271722 2 years ago

alanwaketan force pushed from 5eaee46a to f1a2bd01 2 years ago

Trying to replace some of .data= with .copy_()

4f6e939a

Use torch.autograd._unsafe_set_version_counter

26e2f15f

Adds .torch_pin

c89c55aa

Change to context manager

587381c3

Replace thee last .data with .copy

6719c9bc

Revert "Adds .torch_pin"

c2bc0b79

Fix linters

d15f048f

Replace even more .data usages with modern alternatives

70e29085

Use set_

2bb9e779

Fix linters

ce255ece

Adds torch.no_grad for eval loops

51b3c42f

Use morre _set

08d5ff78

Add a test case to validate that .set_ also release the data buffer

6dd19ee6

alanwaketan force pushed from 85bcf77b to 6dd19ee6 2 years ago

Improve the test

fc458aba

Improve test_data_handle_release

a38761a8

Remove unnecessary detach

c1d6df0c

Fix linters

7028a239

Reuse _dummy_data_placeholder

cf82f82b

alanwaketan commented on 2023-02-22

Adds epoch to logs

1fbf62ee

Fix a wrong move

43ddd1bc

Fix some comments

b9bc51f5

Fix linters

536a5436

alanwaketan changed the title ~~[FSDP] Trying to replace some of .data= with .copy_()~~ [FSDP] Trying to replace all .data usages with modern approaches 2 years ago

Adds TODO

e02e9332

alanwaketan changed the title ~~[FSDP] Trying to replace all .data usages with modern approaches~~ [FSDP] Replace all .data usages with modern approaches 2 years ago

JackCaoG commented on 2023-02-23

JackCaoG approved these changes on 2023-02-23

Fix typo

49b15e06

ronghanghu approved these changes on 2023-02-23

alanwaketan merged 5e6a32d0 into master 2 years ago

Reviewers

ronghanghu

JackCaoG

Assignees

alanwaketan

Labels

None yet

Milestone

No milestone

xla [FSDP] Replace all .data usages with modern approaches #4485 Merged

[FSDP] Replace all .data usages with modern approaches #4485

xla
[FSDP] Replace all .data usages with modern approaches
#4485

Merged