what resources are required

[kason-huang]

Hello, what resources are required for this work? Would 8×32G V100 GPUs suffice?

[xingy038]

At least 8 x 80GB graphics cards are required for post-training.

[kason-huang]

At least 8 x 80GB graphics cards are required for post-training.

Thanks for clarifying the post-training requirements. My available hardware is 8 × NVIDIA V100 GPUs with 32 GB VRAM each.
Are there any supported ways to reduce the memory requirements, even at the cost of slower training or some performance loss? For example, would LoRA or other parameter-efficient fine-tuning methods, gradient checkpointing, a smaller per-GPU batch size with gradient accumulation, or parameter/optimizer offloading make this setup feasible?
Could you also clarify whether the 8 × 80 GB requirement is a hard minimum or applies to the default full-parameter post-training configuration? If a lower-resource setup is possible, a recommended configuration or example command would be very helpful.
Finally, are there any V100-specific compatibility limitations, such as BF16 or attention-kernel requirements, that I should consider?
Thank you!

[xingy038]

Thank you for your interest. We haven't tried training on V100s. You could try reducing the batch size and increasing gradient accumulation to see if that helps. Currently, all configurations only support 8 x 80G configurations.

[dwipddalal]

How many GPU hours does it take on the 8x80GB?

[xingy038]

How many GPU hours does it take on the 8x80GB?

It depends on the type of GPU you have and the dataset you fine-tune. Libero 8xA800 takes about one day, Robotwin's c2r with 8xA800 also takes around one day, while Robodojo requires approximately 32xA800 to complete in about a day.

[dwipddalal]

Sounds good, thanks!