A RoCE network for distributed AI training at scale

Meta showcased their RoCE network infrastructure at ACM SIGCOMM 2024, aimed at large-scale distributed AI training. Authors Adi Gangidi and James Hongyi Zeng detail the design, implementation, and operation of one of the world's largest AI networks, enabling training large models like LLAMA 3.1 405B. The infrastructure includes backend and frontend networks, specialized routing, and congestion control techniques essential for handling extensive GPU workloads for AI applications.

• Meta shared a RoCE network design at ACM SIGCOMM 2024.
• The network supports tens of thousands of GPUs.
• It's part of the infrastructure for training AI models.
• Authors discussed challenges like congestion control.
• They aim to optimize distributed AI training at scale.