DeepSpeed
62afafe8 - Update falcon fused type order (#5007)

Commit
1 year ago
Update falcon fused type order (#5007) The selection of fused type depends on the order of fused_type_dict. If put “DecoderLayer” in front of “FalconDecoderLayer”, Falcon will still choose glmtype incorrectly, so need to put “DecoderLayer at” the last position of fused_type_dict. --------- Co-authored-by: Michael Wyatt <mrwyattii@gmail.com> Co-authored-by: Logan Adams <114770087+loadams@users.noreply.github.com>
Author
Parents
Loading