Is the GenAI Bubble Finally Popping?

Doubt is creeping into discussion over generative AI, as industry analysts begin to publicly question whether the huge investments in GenAI will ever pay off. The lack of a “killer app” besides coding co-pilots and chatbots is the most pressing concern, critics in a Goldman Sachs Research letter say, while data availability, chip shortages, and power concerns also provide headwinds. However, many remain bullish on the long-term prospects of GenAI for business and society.

The amount of sheer, unadulterated hype layered onto GenAI over the past year and a half certainly caught the attention of seasoned tech journalists, particularly those who lived through the dot-com boom and ensuing bust at the turn of the century, not to mention the subsequent rise of cloud computing and smartphones with the introduction of Amazon Web Services and the Apple iPhone in 2006 and 2007, respectively.

The big data boom of the early 2010s was the next tech obsession, culminating with the coronation of Hadoop as The New New Thing, to paraphrase Michael Lewis’ illuminating 1999 book into Silicon Valley’s fixation on continuous technological reinvention. After the collapse of Hadoop–slowly at first, and then all of a sudden in 2019–the big data marketing machine subtly shifted gears and AI was the hot new thing. Several other new (new) things made valiant runs for attention and VC dollars along the way–Blockchain will change the world! 5G will turbocharge edge computing! Self-driving cars are almost here! Smart dust is new oil!–but nothing really seemed to really gain traction, and the big data world made incremental gains with traditional machine learning while wondering what these newfangled neural networks would ever be good for.

GenAI is the newest new thing

That is, until OpenAI dropped a new large language model (LLM) called ChatGPT onto the world in late 2022. Since then, the hype level for neural network-powered AI, and transformer network-based GenAI in particular, has been eerily reminiscent of these previous Big Moments In Tech. It’s worth pointing out that some of these big moments turned out to be actual inflection points, such as mobile and cloud, some had us asking ourselves “What were we thinking (blockchain, 5G), while it took years for the full lessons from other technological breakthroughs to become apparent (the dot-com boom, even Hadoop-style computing).

So the big question for us now is: Which of those categories will we be putting GenAI into in five years? One of the voices suggesting AI may go the way of 5G and blockchain is none other than Goldman Sachs. In a much-read report from the June edition of the Goldman Sachs Research Newsletter titled “Gen AI: too much spend, too little benefit?” Editor Allison Nathan ponders whether AI will pan out.

“The promise of generative AI technology to transform companies, industries, and societies continues to be touted, leading tech giants, other companies, and utilities to spend an estimated ~$1tn on capex in coming years, including significant investments in data centers, chips, other AI infrastructure, and the power grid,” she writes. “But this spending has little to show for it so far beyond reports of efficiency gains among developers.”

Nathan interviewed MIT Professor Daron Acemoglu, who said that only a quarter of tasks that AI is supposed to automate will actually be automated in a cost-effective manner. Overall, Acemoglu estimates that only 5% of all tasks will be automated within 10 years, raising the overall productivity of the United States by less than 1% over that time.

“Generative AI has the potential to fundamentally change the process of scientific discovery, research and development, innovation, new product and material testing, etc. as well as create new products and platforms,” Acemoglu told Nathan. “But given the focus and architecture of generative AI technology today, these truly transformative changes won’t happen quickly and few–if any–will likely occur within the next 10 years.”

Accelerating GenAI progress by ramping up production of its two core ingredients–data and GPUs–probably won’t work, as data quality is a big piece of the equation, Acemoglu said.

GenAI seems to attract irrational exuberance (Roman-Samborskyi/Shutterstock)

“Including twice as much data from Reddit into the next version of GPT may improve its ability to predict the next word when engaging in an informal conversation,” he said, “but it won’t necessarily improve a customer service representative’s ability to help a customer troubleshoot problems with their video service.”

A shortage in chips suitable for training GenAI models is another factor in Goldman’s pessimistic (some would say realistic) take on GenAI. That has benefited Nvidia enormously, which saw revenue grow by more than 260%, to $26 billion, for the quarter ended April 28. That helped pump its market cap over the $3-trillion market, joining Microsoft and Apple as the most valuable companies in the world.

“Today, Nvidia is the only company currently capable of producing the GPUs that power AI,” Jim Covello, Goldman’s head of global equity research, wrote in the newsletter. “Some people believe that competitors to Nvidia from within the semiconductor industry or from the hyperscalers–Google, Amazon, and Microsoft–themselves will emerge, which is possible. But that’s a big leap from where we are today given that chip companies have tried and failed to dethrone Nvidia from its dominant GPU position for the last 10 years.”

The huge costs involved in training and using GenAI act as headwinds against any productivity or efficiency gains that the GenAI may ultimately deliver, Covello said.

“Currently, AI has shown the most promise in making existing processes–like coding–more efficient, although estimates of even these efficiency improvements have declined, and the cost of utilizing the technology to solve tasks is much higher than existing methods,” he wrote.

Nvidia’s fortunes have skyrocketed thanks to GPU demand from GenAI

Covello was semiconductor analyst when smartphones were first introduced, and learned a few lessons about what it takes to actually realize monetary gains from technological innovation. For instance, the smartphone makers promised to integrate global positioning systems (GPS) into the phones, he said, and they had a roadmap that proved prescient.

“No comparable roadmap exists today” for AI, he said. “AI bulls seem to just trust that use cases will proliferate as the technology evolves. But eighteen months after the introduction of generative AI to the world, not one truly transformative–let alone cost-effective–application has been found.”

Finally, the amount of power required to train LLMs and other GenAI models has to be factored into the equation. It’s been estimated that AI currently consumes about 0.5% of the world’s energy, and that amount is expected to increase in the future.

“Utilities are fielding hundreds of requests for huge amounts of power as everyone chases the AI wave, but only a fraction of that demand will ultimately be realized,” says Brian Janous, the Co-founder of Cloverleaf Infrastructure and formerly the VP of energy at Microsoft.

The total capacity of power projects waiting to connect to the grid grew nearly 30% last year, with wait times currently ranging from 40-70 months, Janous said. With so many projects waiting for power, data centers looking for more power to fuel AI training will become “easy targets.”

The US needs to expand its grid to handle expected increase for power demand, but that isn’t likely to be done cheaply or efficiently, he said. “The US has unfortunately lost the ability to build large infrastructure projects–this is a task better suited for 1930s America, not 2030s America,” Janous said. “So, that leaves me a bit pessimistic.”

The enormous electricity demands of AI, and the US’s inabilty to build new power sources, also pose headwinds to AI success (BESTWEB/Shutterstock)

But not everyone is pessimistic about AI’s future. One GenAI optimist is Joseph Briggs, Goldman’s senior global economist. In his article countering Acemoglu, Briggs estimates that GenAI ultimately will automate 25% of all work tasks and raise US productivity by 9% and GDP growth by 6.1% cumulatively over the next decade. What’s more, GenAI will not only automate some existing tasks currently done by humans, but will spur the creation of new tasks, he said.

“…[T]he full automation of AI exposed tasks that are likely to occur over a longer horizon could generate significant cost savings to the tune of several thousands of dollars per worker per year,” he wrote. “The cost of new technologies also tends to fall rapidly over time. Given that cost-saving applications of generative AI will likely follow a similar pattern, and that the marginal cost of deployment will likely be very small once applications are developed, we expect AI adoption and automation rates to ultimately far exceed Acemoglu’s 4.6% estimate.”

Kash Rangan is another GenAI believer. In an interview with the Goldman editor Nathan, the senior equity research analyst said he’s amazed at the pace of GenAI innovation and impressed at the infrastructure buildout of the cloud bigs. He acknowledged that GenAI hasn’t discovered its killer app yet, in the way that ERP dominated the 1990s, search and e-commerce dominated the 2000s, and cloud applications dominated the 2010s.

“But this shouldn’t come as a surprise given that every computing cycle follows a progression known as IPA—infrastructure first, platforms next, and applications last,” Rangan said. “The AI cycle is still very much in the infrastructure buildout phase, so finding the killer application will take more time, but I believe we’ll get there.”

His colleague, Eric Sheridan, joined him in a bullish stance.

“So, the technology is still very much a work in progress. But it’s impossible to sit through demonstrations of generative AI’s capabilities at company events or developer conferences and not come away excited about its long-term potential,” he said.

“So, while I would never say I’m not concerned about the possibility of no payback, I’m not particularly worried about it today, though I could become more concerned if scaled consumer applications don’t emerge over the next 6-18 [months],” Sheridan said.

The promise of GenAI remains high, if unfulfilled at the end of the day. The big question right now is whether GenAI’s returns will go up before the clock runs out. The clock is ticking.

GenAI Hype Bubble Refuses to Pop

When GenAI Hype Exceeds GenAI Reality

The post Is the GenAI Bubble Finally Popping? appeared first on Datanami.