PipeFill runs other work during pipeline gaps, raising reported utilization by as much as 63% while slowing the main job by less than 2%.
ORIGINAL PAPER
PipeFill: Using GPUs During Bubbles in Pipeline-parallel LLM Training
AI training’s idle time may be worth thousands of unused GPUs
Pipeline bubbles are typically 15-30% and can exceed 60% of allocated GPU time in the studied configurations.
PipeFill increased utilization by up to 63% with less than 2% main-job slowdown.
Production deployment at multi-thousand-GPU scale, recovered workload quality, queue availability, energy cost, and capex avoided rather than simulated.
What the paper supports
Supported findings
Pipeline bubbles are typically 15-30% and can exceed 60% of allocated GPU time in the studied configurations.
AbstractPipeFill increased utilization by up to 63% with less than 2% main-job slowdown.
Abstract; p. 11, ConclusionThe 8,000-GPU simulation equates the recovered work to as many as 2,600 GPUs.
Abstract and evaluationLimitations
The largest scale result relies on simulation calibrated from a smaller physical cluster.
Benefits depend on bubble duration, available memory, and a queue of compatible fill jobs.
Fill jobs can run materially slower than on dedicated GPUs because of memory limits and interruptions.
Companies with a traceable link
Alibaba Group9988 HK · BABA UN
Supports LongBetter cluster utilization can improve cloud capex productivity.
Value-chainTencent700 HK
Supports LongMore useful work per installed GPU can reduce the cost of model training.
Value-chainGDS Holdings9698 HK · GDS UQ
MixedHigher cluster productivity can support tenants but delay incremental capacity.
Value-chainVNET GroupVNET UW
MixedColocation demand depends on whether added AI work grows faster than released effective capacity.
Value-chainDell TechnologiesDELL UN
Supports ShortA durable utilization lift could defer some incremental hardware purchases, subject to demand elasticity.
Value-chain