Reinforcement learning (RL) after training is now common for building language model agents. It helps models learn to reason and act across multiple steps. Running RL at scale needs a strong cluster infrastructure. Amazon SageMaker HyperPod provides this with features like node health monitoring and automatic replacement of faulty nodes. Paired with checkpointing, jobs can resume from the last saved step after failures, saving hours of work.
HyperPod runs on Amazon Elastic Kubernetes Service (EKS). It supports creating Ray clusters for distributed training. You can submit jobs remotely and monitor them with Amazon Managed Grafana dashboards. This makes managing large RL jobs easier and more transparent.
SkyRL is an open-source RL framework used in this setup. It trains a vision-language model called Qwen3-VL-8B to navigate mazes. The training uses Group Relative Policy Optimization (GRPO), which compares multiple runs to improve performance. This method does not need a separate critic or value model.
The training runs on a cluster with three GPU nodes and one CPU node. The GPU nodes have 6 NVIDIA RTX PRO 6000 Blackwell GPUs in total. The CPU node manages the training and monitoring tools. The model and rollout engines are colocated on the GPUs, sharing storage via Amazon FSx for Lustre. This setup speeds up training and makes it more reliable.
To get started, you need a SageMaker Studio domain, the right cluster setup, and the necessary software packages. The process includes preparing the environment, launching the cluster, running the training job, and monitoring progress. The trained model can then be hosted for inference or further use.
This new capability allows running large, complex RL jobs more efficiently. It reduces the risk of hardware failures disrupting training. It also makes scaling RL training easier for vision-language models.
