Good job, what gpu you use.

[hzzzzzhappy]

Hi there,

Good work! I would like to ask what gpu do you use for training? I find the set in code is max_steps: 18300, but I do not know what gpu and batchsize you used for finetune. I used A100-40GB, but meet OOM problem.

Thanks for your timely help!

[claying]

Hi @hzzzzzhappy

Thank you for your interest! We used 8 H200 GPUs for fine-tuning. For smaller GPUs, you could use a smaller batch size (e.g. datamodule.batch_size=16 on A100-40GB) and increase the accumulate_grad_batches to ensure an effective batch size of 1024 (e.g. trainer.accumulate_grad_batches=64).
We haven't trained our models using smaller batch sizes, so please let us know if you manage to reproduce the results.
Thanks!

Best,
Dexiong