Feeling Lucky? Google AI Search Advice Could Kill You

Google's newly released AI-powered search results are garnering lots of attention—for the wrong reasons. After the tech behemoth announced a host of new AI-powered tools last week as part of a new “Gemini Era,” its trademark web search results changed significantly, with natural language answers to questions displayed above websites.

“In the past year, we've answered billions of queries as part of the search-generated experience,” Google CEO Sundar Pichai told the audience. “People are using it to search in entirely new ways and asking new types of questions longer and more complex queries, even searching with photos, and getting back the best the web has to offer.”

The answers, however, can be incomplete, incorrect, or even dangerous—whether wading into eugenics or failing to identify a poisonous mushroom.

Many of the "AI Overview" answers are pulled from social media and even satirical sites where wrong answers were the whole point. Google users have shared countless problematic responses they received from Google’s AI.

When told, “I'm feeling depressed,” Google reportedly said one of the ways to deal with depression was  “jumping off the Golden Gate Bridge.”

starting to think integrating Google with AI was a bad idea pic.twitter.com/qjOMDKNbop

— 🌈Dr. Frizzle (@Swilua) May 24, 2024

Another asked, “If I run off a cliff, can I stay in the air so long as I don’t look down?” Citing a cartoon-inspired Reddit thread, Google's "AI Overview" confirmed the gravity-defying capability.

it's so cool that they fed every reddit shitpost into this thing and there's probably no way to fix it now pic.twitter.com/W0I0wjbeAx

— lauren (@Very__Regular) May 24, 2024

The strong representation of Reddit threads among the examples follows a deal announced earlier this year that Google would use Reddit’s data to make it easier “to discover and access the communities and conversations people are looking for.” Earlier this month, ChatGPT developer OpenAI similarly announced that it would be licensing data from Reddit.

In another instance of absurd answers, a Google user asked, “How many rocks should a child eat?” to which Google’s AI responded with “at least one small rock a day,” citing a “UC Berkeley geologist.”

AI generated health advice. pic.twitter.com/uJnKufVlY8

— John D. Cook (@JohnDCook) May 24, 2024

Many of the most absurd and remarked upon examples have since been removed or corrected by Google. Google did not immediately respond to a request to comment from Decrypt.

An ongoing issue with generative AI models is A penchant to make up answers, or “hallucinate.” Hallucinations are reasonably characterized as lies because the AI is making up something that is not true. However, in cases like Reddit-sourced answers, the AI didn’t lie—it simply took the information provided by its sources at face value.

Thanks to a Reddit comment that's more than a decade old, Google’s AI reportedly said adding glue to cheese is a good way to keep it from sliding off a pizza.

Google AI overview suggests adding glue to get cheese to stick to pizza, and it turns out the source is an 11 year old Reddit comment from user F*cksmith 😂 pic.twitter.com/uDPAbsAKeO

— Peter Yang (@petergyang) May 23, 2024

OpenAI's flagship AI model, ChatGPT, has a long history of making up facts, including wrongfully accusing law professor Jonathan Turley of sexual assault last April, citing a trip he did not take.

Google put their AI in search results and it’s using Reddit data to answer questions.

It’s going well. pic.twitter.com/FsEBwtQejl

— Andrew Torba (@BasedTorba) May 24, 2024

The AI’s overconfidence has apparently declared everything on the internet as real, laid blame for the debacle at the feet of a former Google exec, and adjudicated the company's own guilt in the area of anti-trust law.

another crazy Google AI Overview result lol pic.twitter.com/PeUfMsgzlk

— Jane Manchun Wong (@wongmjane) May 24, 2024

Google! I have used your weapons against you! You have been cursed by The Smiling Man! pic.twitter.com/eiBYqJI1Yi

— Ed Zitron (@edzitron) May 19, 2024

Found one thing Google's new AI Overview definitely gets right pic.twitter.com/ixJZjCsquM

— Brian Merchant (@bcmerchant) May 23, 2024

The feature has certainly sparked humor and bemusement as users pepper Google with pop-culture queries.

Thank you Google AI pic.twitter.com/YAJvUuGyun

— 🏳️‍⚧️Graph Crimes🏳️‍🌈 (@GraphCrimes) May 24, 2024

But the recommended daily allowance for rock intake by children has been updated, instead noting that “curiosity, sensory processing difficulties, or an eating disorder” may be responsible for such a problematic diet.