Inside Colossus: The Power, Cooling, and RoCE Fabric Behind xAI's 100k Cluster

In mid-2024, the frontier AI training narrative centered on a fatal bottleneck: megawatt availability and substation waitlists stretching out to 2028.

Then, in a matter of 122 calendar days, xAI took delivery of, rack-mounted, liquid-cooled, and brought online Colossus in Memphis, Tennessee — a single-site supercluster comprising over 100,000 Nvidia H100 and H200 Tensor Core GPUs.

1. Velocity as an Engineering Primitive

Industry standard timelines for deploying a 100MW+ Tier-4 data center environment run from 24 to 36 months.

xAI compressed this cycle by more than 80% using a playbook borrowed directly from Tesla Gigafactory and SpaceX Starbase construction sprints:

• Direct Natural Gas Generation: Installing on-site mobile gas turbines to provide bridge power before the local Tennessee Valley Authority (TVA) completed permanent substation upgrades.
• Liquid Cooling Retrofits: Deploying high-density liquid-to-liquid heat exchangers directly adjacent to racks rather than waiting for conventional chilled-water facility piping.
• Unified RDMA RoCE Fabric: Leveraging ultra-high density Ethernet switching fabrics with customized congestion control rather than purely proprietary interconnects, reducing lead-time friction.

2. The Multi-Company Flywheel in Action

Colossus exemplifies the cross-pollination defining the Muskonomy:

1. Tesla Software Synergies: Tesla engineers with experience in high-voltage industrial battery systems (Megapacks) contributed to transient power spike stabilization for the GPU racks.
2. X Infrastructure & Training Ground: Real-time multi-modal conversational feeds from X provided instantaneous RLHF feedback loops and conversational pre-training datasets for Grok.
3. SpaceX Starlink Telemetry: Remote monitoring and distributed disaster-recovery bandwidth links utilizing high-throughput Starlink business terminals.

3. What It Means for the AI Frontier

With Colossus operational and expanding toward 200,000 GPUs, xAI has eliminated the compute deficit that once separated it from legacy hyperscalers like Microsoft, Google, and Meta.

For investors tracking generative AI valuations, the battle has shifted from “who has access to foundational algorithms” to “who can secure sovereign-level megawatts and run continuous, uninterrupted gradient descent runs at zero downtime.”