Microsoft, which is already invested in OpenAI to the tune of $1 billion and enjoys an exclusive partnership with it to train its AI models, reportedly is looking to invest $10 billion in the company following the breakout success of OpenAI’s latest large language model, ChatGPT.
ChatGPT, which was released as a research tool by OpenAI on November 30, has taken the Internet by storm thanks to its uncanny capability to generate compelling text responses to questions. The large language model, which is based on an earlier (and much larger) large language model (LLM) called GPT-3.5, has officially gone viral, and is unofficially “the next big thing” on the Internet.
People have compared ChatGPT’s launch with other seminal events in the Internet’s history, such as the debut of the iPhone in 2007 and the first commercial Web browser in 1994. “Everything has changed,” some are declaring. Students will no longer write term papers, programmers won’t need to write code, lawyers won’t need to write briefs, and even hard-working journalists (gasp!) may soon be out of jobs, all thanks to ChatGPT’s ability to string together text in a mostly accurate manner (its 15% to 21% “hallucination rate” notwithstanding).
The LLM iron is hot, and multiple companies are looking to strike by plunking down some cash to own a piece of OpenAI. Headed by Sam Altman, the San Francisco startup is allowing its employees to sell shares at a $29 billion valuation, according to a story last week in the Wall Street Journal. Microsoft appears to be one of the big takers, according to a story in Semafor this week.
Semafor detailed the unusual terms of Microsoft’s $10 billion investment:
“Microsoft’s infusion would be part of a complicated deal in which the company would get 75% of OpenAI’s profits until it recoups its investment, the people said,” write Semafor’s Liz Hoffman and Reed Albergotti. “After that threshold is reached, it would revert to a structure that reflects ownership of OpenAI, with Microsoft having a 49% stake, other investors taking another 49% and OpenAI’s nonprofit parent getting 2%.”
(Ascannio/Shutterstock)
Microsoft already has an exclusive offer to train OpenAI’s models in its Azure cloud data centers. Training the LLMs is no small feat. GPT-3, which debuted in the spring of 2020, utilizes a transformer model with 175 billion parameters. Microsoft described its training investment in a May 2020 blog post:
“The supercomputer developed for OpenAI is a single system with more than 285,000 CPU cores, 10,000 GPUs and 400 gigabits per second of network connectivity for each GPU server. Compared with other machines listed on the TOP500 supercomputers in the world, it ranks in the top five.”
Training such a large model is expensive. By some accounts, it would have taken over a month at a compute cost of more than $5 million to train a model with 175 billion parameters, while developing the model in the first place likely took upwards of 10x that amount.
With each subsequent generation of LLM, the results improve in quality, but the inputs in data, energy, and money grow exponentially. GPT-4, which could debut in 2023 with 100 trillion parameters, would likely require another order of magnitude increase in data, energy, and dollars.
The stakes are high, which is why Microsoft and others are willing to put up so much money into OpenAI. Whether LLMs will define the consumer tech space for the next decade has yet to be determined, but plenty of wealthy and influential folks clearly think there’s a chance they will.
Microsoft reportedly has big plans to incorporate OpenAI’s technology into various products, including its Bing search engine, which trails woefully behind Google’s eponymous search engine. It even plans to bring the various GPT capabilities, such as question-and-answering and text generation, into its Office suite (hold the “Clippy 2.0” jokes, please). A version of an OpenAI model is already being used in Word, according to The Information. 
All of this AI activity between OpenAI and Microsoft has put Google on the defensive. The Mountain View tech giant’s CEO, Sundhar Pichai, reportedly declared a “code red” over the potential for an LLM-infused Bing search engine to cut into its extremely lucrative search business. Google, whose researchers developed many of the core underlying technologies used by OpenAI, is reportedly looking to accelerate plans to use its biggest and best LLM, dubbed LaMDA, with its search engine. Concerns about ethics and potential–not whether LaMDA is sentient, as one Google employee claimed in 2022, are reportedly what’s holding up a more aggresssive LLM roll-out by Google.
While the tech giants position themselves for a AI battle royale, and folks like Altman debate whether we have arrived at the precipice of the big breakthrough — an artificial general intelligence (AGI) — businesses are also left to wonder whether and how the LLMs can improve their own operations. The short answer is yes, but developing a custom conversational AI system that understands the quirks of your particular business is harder than it looks, and today requires skilled engineers who can string together multiple technologies.
At least that’s the case now. In another year or two, your guess is as good as ChatGPT’s.
Related Items:
Large Language Models in 2023: Worth the Hype?
Microsoft Announces ChatGPT-powered Bing, Google CEO Declares ‘Code Red’
The Drawbacks of ChatGPT for Production Conversational AI Systems
The post Microsoft Seeks $10B Investment in OpenAI: Report appeared first on Datanami.
0 Commentaires