Power Grab: DataCenters and the Electric Power Grid
Let’s talk compute: the goal for competitors is to improve the quality of their language learning models to deal with complex logic and increase accuracy, training on lots of data w/increasingly powerful machines. Noted analyst Epoch AI notes annual compute capability has been growing at a rate of 4X. Will this growth will continue at that pace, with what implications for power grids?
Epoch AI examines four factors: 1) power availability (more on that later); 2) global chip capacity; 3) the “latency wall” – delays in increasingly complex computations; and 4) the availability of data. Let’s look at 2 through 4.
Chips – Chips are in high demand. These game processing units rapidly perform highly complex calculations. GPUs are in high demand and expensive. Nvidia’s Blackwell chip cost $10 bn to create, and buyers pay $30 – 40K! per GPU. That chip also draws 700 W – 1.2 kW depending on configuration and cooling strategy.
Nvidia owns 80% of GPU market, followed by AMD, and the industry can’t meet current demand. But Big Four are all developing chips, so that strain may ease.
Latency – It takes time (latency) for AI models to process data, and latency increases as model sizes grow. As future training runs get larger and models get bigger, this may become an issue, limiting future growth rates.
Data – Scientific papers, movies, stupid clips on TikTok, all of it. Here, we must understand tokens – the smallest elements into which text data can be broken down for AI models to process them. One word is a single token. Images or videos are typically broken into smaller patches (one picture or second of video may represent 30 tokens). The web holds about 750 trillion words of unique text by 2030. With images, audio, and video and you might get to 20 quadrillion tokens by 2030. BUT, with faster computers and better algorithms we might actually run out of data by 2026. This uncertainty leads to a critical question for utilities. What if we build all this infrastructure, and at some point, there’s less to do with it? “Stranded assets” should come to mind.
Meanwhile, chips become more efficient. Nvidia says its GPUs have seen a 2000x reduction in energy use over 10 years. Until now, such gains just allowed data centers to do more. But if future gains continue, how does that affect future power needs? Nobody knows. What we do know is the power grab continues unabated; data centers are looking at all supply strategies to get juice wherever they can. That’s our topic for the next sessions.
PS – New data points last week:
Exelon’s CEO says “high probability” datacenter load jumped from 6 to 11 GW this year.
Google’s CEO says over 25% of new code is generated by AI. Substitution for labor is a huge part of AI value prop.
Financial Times says spending on AI by Big Four will exceed $200 bn in 2024.
FERC rejects Amazon Web Services bid to expand power supply contract to data center at Talen Energy’s nuke.
