For tech giants, AI like Bing and Bard poses billion-dollar search problem

24/2/23, 12:43 PM
For tech giants, AI like Bing and Bard poses billion-dollar search problem | Reuters
Page 1 of 7
https://www.reuters.com/technology/tech-giants-ai-like-bing-bard-poses-billion-dollar-search-problem-2023-02-22/
For tech giants, AI like Bing and Bard
poses billion-dollar search problem
Jeffrey Dastin
6 minute readFebruary 23, 202312:49 AM GMT+1Last Updated a day ago
MOUNTAIN VIEW, Calif., Feb 22 (Reuters) - As Alphabet Inc (GOOGL.O)
looks past a chatbot flub that helped erase $100 billion from its market
value, another challenge is emerging from its efforts to add generative
artificial intelligence to its popular Google Search: the cost.
Executives across the technology sector are talking about how to operate
AI like ChatGPT while accounting for the high expense. The wildly popular
chatbot from OpenAI, which can draft prose and answer search queries,
has "eye-watering" computing costs of a couple or more cents per
conversation, the startup's Chief Executive Sam Altman has said on
Twitter.
Advertisement · Scroll to continue
In an interview, Alphabet's Chairman John Hennessy told Reuters that
having an exchange with AI known as a large language model likely cost
10 times more than a standard keyword search, though fine-tuning will
help reduce the expense quickly.

24/2/23, 12:43 PM
Page 2 of 7
Register for free to Reuters and know the full story
Even with revenue from potential chat-based search ads, the technology
could chip into the bottom line of Mountain View, Calif.-based Alphabet
with several billion dollars of extra costs, analysts said. Its net income was
nearly $60 billion in 2022.
Morgan Stanley estimated that Google's 3.3 trillion search queries last
year cost roughly a fifth of a cent each, a number that would increase
depending on how much text AI must generate. Google, for instance,
could face a $6-billion hike in expenses by 2024 if ChatGPT-like AI were
to handle half the queries it receives with 50-word answers, analysts
projected. Google is unlikely to need a chatbot to handle navigational
searches for sites like Wikipedia.
Advertisement · Scroll to continue
Others arrived at a similar bill in different ways. For instance,
SemiAnalysis, a research and consulting firm focused on chip technology,
said adding ChatGPT-style AI to search could cost Alphabet $3 billion, an
amount limited by Google's in-house chips called Tensor Processing
Units, or TPUs, along with other optimizations.

24/2/23, 12:43 PM
Page 3 of 7
What makes this form of AI pricier than conventional search is the
computing power involved. Such AI depends on billions of dollars of chips,
a cost that has to be spread out over their useful life of several years,
analysts said. Electricity likewise adds costs and pressure to companies
with carbon-footprint goals.
The process of handling AI-powered search queries is known as
"inference," in which a "neural network" loosely modeled on the human
brain's biology infers the answer to a question from prior training.
In a traditional search, by contrast, Google's web crawlers have scanned
the internet to compile an index of information. When a user types a
query, Google serves up the most relevant answers stored in the index.
Alphabet's Hennessy told Reuters, "It's inference costs you have to drive
down," calling that "a couple year problem at worst."
Alphabet is facing pressure to take on the challenge despite the expense.
Earlier this month, its rival Microsoft Corp (MSFT.O) held a high-profile

24/2/23, 12:43 PM
Page 4 of 7
event at its Redmond, Washington headquarters to show off plans to
embed AI chat technology into its Bing search engine, with top executives
taking aim at Google's search market share of 91%, by Similarweb's
estimate.
A day later, Alphabet talked about plans to improve its search engine, but
a promotional video for its AI chatbot Bard showed the system answering
a question inaccurately, fomenting a stock slide that shaved $100 billion
off its market value.
Microsoft later drew scrutiny of its own when its AI reportedly made
threats or professed love to test users, prompting the company to limit
long chat sessions it said "provoked" unintended answers.
Microsoft's Chief Financial Officer Amy Hood has told analysts that the
upside from gaining users and advertising revenue outweighed expenses
as the new Bing rolls out to millions of consumers. "That's incremental
gross margin dollars for us, even at the cost to serve that we're
discussing," she said.

24/2/23, 12:43 PM
Page 5 of 7
The logo for Google LLC is seen at the Google Store Chelsea in Manhattan, New York City, U.S., November 17,
2021. REUTERS/Andrew Kelly
And another Google competitor, CEO of search engine You.com Richard
Socher, said adding an AI chat experience as well as applications for
charts, videos and other generative tech raised expenses between 30%
and 50%. "Technology gets cheaper at scale and over time," he said.
A source close to Google cautioned it's early to pin down exactly how
much chatbots might cost because efficiency and usage vary widely
depending on the technology involved, and AI already powers products
like search.
Still, footing the bill is one of two main reasons why search and social
media giants with billions of users have not rolled out an AI chatbot
overnight, said Paul Daugherty, Accenture's chief technology officer.
"One is accuracy, and the second is you have to scale this in the right
way," he said.

24/2/23, 12:43 PM
Page 6 of 7
MAKING THE MATH WORK
For years, researchers at Alphabet and elsewhere have studied how to
train and run large language models more cheaply.
Bigger models require more chips for inference and therefore cost more.
AI that dazzles consumers for its human-like authority has ballooned in
size, reaching 175 billion so-called parameters, or different values that the
algorithm takes into account, for the model OpenAI updated into ChatGPT.
Cost also varies by the length of a user's query, as measured in "tokens"
or pieces of words.
One senior technology executive told Reuters that such AI remained cost-
prohibitive to put in millions of consumers' hands.
"These models are very expensive, and so the next level of invention is
going to be reducing the cost of both training these models and inference
so that we can use it in every application," the executive said on condition
of anonymity.
For now, computer scientists inside OpenAI have figured out how to
optimize inference costs through complex code that makes chips run
more efficiently, a person familiar with the effort said. An OpenAI
spokesperson did not immediately comment.
A longer-term issue is how to shrink the number of parameters in an AI
model 10 or even 100 times, without losing accuracy.
"How you cull (parameters away) most effectively, that's still an open
question," said Naveen Rao, who formerly ran Intel Corp's AI chip efforts
and now works to lower AI computing costs through his startup
MosaicML.
In the meantime, some have considered charging for access, like OpenAI's
$20 per month subscription for better ChatGPT service. Technology

24/2/23, 12:43 PM
Page 7 of 7
experts also said a workaround is applying smaller AI models to simpler
tasks, which Alphabet is exploring.
The company said this month a "smaller model" version of its massive
LaMDA AI technology will power its chatbot Bard, requiring "significantly
less computing power, enabling us to scale to more users."
Asked about chatbots like ChatGPT and Bard, Hennessy said at a
conference called TechSurge last week that more focused models, rather
than one system doing everything, would help "tame the cost."
Register for free to Reuters and know the full story
Reporting By Jeffrey Dastin in Mountain View, Calif. and Stephen Nellis in
Sunnyvale, Calif.; Additional reporting by Greg Bensinger; editing by
Kenneth Li and Claudia Parsons
Our Standards: The Thomson Reuters Trust Principles.

For tech giants, AI like Bing and Bard poses billion-dollar search problem

Recommended

Recommended

More Related Content

Similar to For tech giants, AI like Bing and Bard poses billion-dollar search problem

Similar to For tech giants, AI like Bing and Bard poses billion-dollar search problem (20)

More from LUMINATIVE MEDIA/PROJECT COUNSEL MEDIA GROUP

More from LUMINATIVE MEDIA/PROJECT COUNSEL MEDIA GROUP (20)

Recently uploaded

Recently uploaded (20)

For tech giants, AI like Bing and Bard poses billion-dollar search problem