What Nvidia’s new Blackwell chip says about AI’s carbon footprint problem

Hello and welcome to Eye on AI.

The biggest show in AI this week is Nvidia’s GTC developer conference in San Jose, Calif. One Wall Street analyst quipped the chipmaker’s confab was the equivalent of "AI Woodstock," given all the heavy-hitters present from not just Nvidia, but companies such as OpenAI, xAI, Meta, Google, and Microsoft, and the presence of executives from major companies looking to implement AI, including L’Oréal, Lowe’s, Shell, and Verizon.

At GTC yesterday, Nvidia CEO Jensen Huang unveiled the company’s newest graphics processing unit (GPU), the kind of chips that have become the workhorses of AI. The forthcoming Blackwell GPU will have 208 billion transistors, far exceeding the 80 billion its current top-of-the-line H100 GPUs have. The larger chips mean they will be twice as fast at training AI models and five times faster at inference—the term for generating an output from an already trained AI model. Nvidia is also offering a powerful new GB200 “superchip” that would include two Blackwell GPUs coupled together with its Grace CPU and supersede the current Grace Hopper MGX units that Nvidia sells for use in data centers.

What’s interesting about the Blackwell is its power profile—and how Nvidia is using it to market the chip. Until recently, the trend has been that more powerful chips also consumed more energy, and Nvidia didn’t spend much effort trying to make energy efficiency a selling point, focusing instead on raw performance. But in unveiling the Blackwell, Huang emphasized how the new GPU’s greater processing speed meant that the power consumption during training was far less than with the H100 and earlier A100 chips. He said training the latest ultra-large AI models using 2,000 Blackwell GPUs would use 4 megawatts of power over 90 days of training, compared to having to use 8,000 older GPUs for the same period of time, which would consume 15 megawatts of power. That’s the difference between the hourly power consumption of 30,000 homes and just 8,000 homes.

Nvidia is talking about the Blackwell’s power profile because people are growing increasingly alarmed about both the monetary cost of AI and its carbon footprint. Those two factors are related since one reason cloud providers charge so much to run GPUs is not just the cost of the chips themselves, but the cost of the energy to run them (and to cool the data centers where they are housed since the chips also throw off more heat than conventional CPUs). And both those factors have made many companies reluctant to fully embrace the generative AI revolution because they are worried about the expense and about doing damage to net zero sustainability pledges. Nvidia knows this—hence its sudden emphasis on power consumption. The company has also pointed out that many AI experts working on open-source models have found ways to mimic some aspects of the performance of much larger, energy-intensive models such as GPT-4 but with models that are much smaller and less power-consuming.

Currently, data centers consume just over 1% of the world’s power, with estimates that AI is a fraction of that. Schneider Electric recently estimated that AI consumes about as much power annually as the nation of Cyprus. That number may climb rapidly due to AI. One expert at Microsoft has suggested that just the Nvidia H100s in deployment will consume about as much power as all of Phoenix by the end of this year.

Still, I have always thought the focus on AI’s energy consumption in data centers was a bit of a red herring, since most of the data centers of the cloud hyperscalers, which is where most AI is being run right now, are now powered by renewable energy or low-carbon nuclear power. And the fact that these companies are willing to contract for large amounts of renewable power at set prices has played a key role in giving renewable power companies the confidence to build large wind and solar power projects. The presence of these hyperscalers in the renewables market has meant there is more renewable power available for everyone. It’s a win-win. (Far more troubling is the water consumption needed to keep these data centers cool. Here consuming less power, and generating less heat, would have a more direct impact on sustainability.)

That said, AI is a global phenomenon, and there are some places where there isn’t much renewable power available. And if AI is adopted to the extent many project, and if AI models keep getting larger, it is possible renewable energy demand could outstrip low carbon supplies even in the U.S. and Europe. That’s one reason Microsoft has expressed interest in trying to use AI to speed up the process of getting new nuclear plants approved for construction in the U.S.

It is also true that AI’s energy consumption is among the many areas where our own brains are vastly superior to the artificial ones we’ve created. The human brain consumes about 0.3 kilowatt hours daily by burning calories, compared to about 10 kilowatt hours daily for the average H100. To really make AI ubiquitous without destroying the planet in the process, we may need to find a way to get artificial neural networks to operate with an energy profile that looks a bit more like the natural ones.

That’s essentially what the U.K.’s Advance Research and Invention Agency (Aria, which is the country’s answer to the U.S. Defense Department’s DARPA) is hoping to bring about. Last week, Aria announced it was committing £42 million ($53 million) to fund projects working towards reducing the current energy footprint of running AI applications by a factor of a thousand. It said it would consider radically different ways of building computer chips in order to do so, including chips that rely on biological neurons for computation instead of silicon transistors. (I wrote about one such effort in 2020.)

The effort is pretty sci-fi and may not yield the results Aria hopes. But the very fact the Aria challenge exists and that Nvidia is now putting energy efficiency on center stage at GTC are signs the world is getting serious about tackling AI’s carbon footprint. Hopefully, this means AI won’t destroy our efforts to build a more sustainable world.

There’s more AI news below. But first, if you’re enjoying reading this newsletter, how would you like to participate in a live version—chatting in-person and IRL with me and many of the world’s foremost experts on deploying AI within companies? If that sounds intriguing, please apply to attend Fortune’s Brainstorm AI conference in London on April 15 and 16. I’ll be there cochairing the event and moderating sessions. You will get to hear from Google DeepMind’s Zoubin Ghahramani, Microsoft chief scientist Jaime Teevan, Salesforce chief ethical and human use officer Paula Goldman, as well as Shez Partovi, the chief innovation and strategy officer for Royal Philips, Accenture’s chief AI officer Lan Guan, Builder.ai CEO Sachin Dev Duggal, and many others. Email BrainstormAI@fortune.com to apply to attend. I hope to see you there!

With that, here’s the AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

This story was originally featured on Fortune.com