transformers
Enhancing SFT Training Efficiency Using Packing and FlashAttention2 with Position IDs
#31629

Merged

Enhancing SFT Training Efficiency Using Packing and FlashAttention2 with Position IDs #31629

ArthurZucker merged 13 commits into huggingface:main from RhuiDih:dev/fa_packing_posid

ArthurZucker commented on 2024-07-10

RhuiDih force pushed from cf6271fa to c3451dbc 1 year ago

ArthurZucker commented on 2024-07-15

ArthurZucker approved these changes on 2024-07-19

fxmarty commented on 2024-07-19

add DataCollatorBatchFlattening

df3c9b29

Update data_collator.py

dfe08de7

change name

8120b3a3

new FA2 flow if position_ids is provided

0598510e

add comments

1ff23436

minor fix

f97ab716

minor fix data collator

08e1f2cb

add test cases for models

48ce3d20

add test case for data collator

6c6b1688

remove extra code

00e7abf6

RhuiDih force pushed to 00e7abf6 1 year ago

formating for ruff check and check_repo.py

3a872933

ruff format

c60b720d

custom_init_isort.py

90305596

ArthurZucker approved these changes on 2024-07-23

ArthurZucker merged 9cf4f2aa into main 1 year ago

Reviewers

ArthurZucker

fxmarty

Assignees

No one assigned

Labels

None yet

Milestone

No milestone