[Tutorial] GRPO on Sagemaker using TRL (#2596)
* add: tutorial for GRPO on Sagemaker using TRL
Signed-off-by: DWarez <dario.salvati@huggingface.co>
* change: using ml.p4d.24xlarge instance to enable beta>0.0
change: using zero3 instead of zero2
Signed-off-by: DWarez <dario.salvati@huggingface.co>
* change: revert back to default Sagemaker launcher + sagemaker torchrun
The workaround I did before is totally unnecessary for ml.p4d.24xlarge
therefore I think it was due to some hardware related issue (like high
memory pressure or comms bandwidth). Since we're keeping ml.p4d.24xlarge
as instance type in order to also run the reference model, we can safely
revert back to not use the workaround
Signed-off-by: DWarez <dario.salvati@huggingface.co>
* change: notebook login as auth, better markdown cells
add: references
Signed-off-by: DWarez <dario.salvati@huggingface.co>
* fix: now relying on public image
Signed-off-by: DWarez <dario.salvati@huggingface.co>
---------
Signed-off-by: DWarez <dario.salvati@huggingface.co>
