Software Ecosystems Are Harder to Build Than Silicon

May 4, 2025

AMD's AI accelerators are increasingly competitive on raw silicon the MI300X matches or exceeds Nvidia's H100 on many benchmarks, and the roadmap through MI350 and MI400 is aggressive. Yet AMD remains a distant second in AI infrastructure. The reason is not hardware but software: CUDA is a decade-deep ecosystem of libraries, frameworks, tooling, debugging infrastructure, and most critically millions of developers who think in CUDA.

CUDA is not a programming language. It is a moat built from documentation, StackOverflow answers, university curricula, and muscle memory.

ROCm, AMD's answer to CUDA, has improved dramatically but still lacks the breadth and reliability that production AI teams require. Key gaps: debugging and profiling tools lag behind Nvidia's ecosystem, multi-node distributed training is less battle-tested, and framework support (PyTorch, JAX) works but requires ongoing effort to maintain parity. Every CUDA kernel hand-optimized by a framework team is a piece of technical debt that AMD must match or work around.

The most promising path to eroding CUDA's lock-in is not ROCm alone but compiler-level convergence. MLIR, OpenXLA, Triton, and similar frameworks create hardware-agnostic intermediate representations that can target both CUDA and ROCm backends. If the AI community standardizes on these higher-level abstractions, the importance of the underlying GPU programming model diminishes. This is the real battleground not benchmarks but compiler infrastructure.

For any country or organization building sovereign AI infrastructure, the lesson is clear: buying AMD hardware is the easy part. The hard part is investing in the software ecosystem, compiler toolchains, and developer training that make the hardware usable at scale. Without that investment, cheaper silicon just means cheaper paperweights.

Takeaway: Hardware capability is necessary but nowhere near sufficient the winner in AI infrastructure will be determined by whoever builds the deepest software ecosystem, not the fastest chip.