llm-d’s Co-op Time-Slicing Boosts GPU Duty Cycles to 70%

Reinforcement learning (RL) post-training for LLMs is a hungry beast—it gobbles up GPUs but leaves them idle 40-60% of the time. That’s like owning a swamp full of caimans that only hunt half the day. The llm-d project just threw a life preserver: co-operative time-slicing that interleaves independent RL jobs onto shared hardware, pushing accelerator duty cycles from ~40% to 70% without hurting model convergence.

The Idle Accelerator Problem

Distributed RL alternates between sampling rollouts and gradient training. In synchronous setups, one phase waits for the other, leaving GPUs twiddling their thumbs. Asynchronous variants help but still suffer from trainer starvation while waiting for fresh rollouts. The result? Millions in wasted compute.

How Time-Slicing Works

llm-d treats RL phases as schedulable entities. When Job A hits an idle phase, the infrastructure checkpoints its GPU state to host DRAM and swaps in Job B’s active phase. The swap uses a checkpoint/restore mechanism via a Snapshot Agent, so only one job’s state occupies the accelerator at a time—no OOMs, no framework changes.

Three-Layer Architecture

• Workload-scoped: Your code calls acquire() and yield() to signal phase boundaries. The ML framework runs unmodified.
• Cluster-scoped: A time-slice orchestrator maintains a lock queue for jobs sharing the same GPUs. Only the head of the queue runs; others wait.
• Node-scoped: The Snapshot Agent (a DaemonSet) performs the actual checkpoint/restore on each node, using CUDA checkpointing.

Developer Experience

Minimal client-side changes. If you use Ray or similar, just wrap your accelerator phases with the @orchestrator.on_accelerators decorator. The orchestrator handles the rest.

Roadmap

• Faster checkpoint/restore backends and selective memory snapshotting (e.g., LoRA adapters).
• Automated scheduler to profile and pair jobs with complementary idle phases.
• Support for TPUs and custom accelerators.

Ready to stop your GPUs from napping? Check out the user guides and join the #sig-rl channel on llm-d Slack. For more on cloud infrastructure efficiency, see our Google Cloud review.

Original announcement published on Google Cloud.