A couple of weeks ago, I noticed that Google’s AI, Gemini, has been answering the majority of my google searches. The change happened with no fanfare. I hardly even noticed. I looked a little closer, and I noticed that Gemini’s answer occupies my entire screen. If I want to read an answer that (might) be written by a human being, then I have to scroll down, and choose to ignore the cleanly packaged, authoritative answer offered by Google’s Large Language Model (LLM.)

My gut told me that I should not be relying on Gemini to answer my questions. I wanted to examine this, so I have been looking into the use of Large Language Models as search engines, and I don’t like what I have found. I’d like to share with you.

Indeed, Gemini assisted search does not represent the first foret of AI in the Google search experience.

We have gotten used to Googles “snippets” as well. These are short answers to easier, more fact based questions, like “When was King Henry V born?”

Whats really great about snippets is that they pull off of trusted sources like Wikipedia. Snippets are different from Gemini’s “AI overview,” because snippets are not generative. That means unlike Gemini, snippets just quote websites, or pull from Googles “Knowledge Vault”: an extensive database of things that Google accepts as facts. In contrast, generative AI’s like Gemini create new writing.

AI Hallucination

A big topic in the public discourse since GPT 3 was released is “AI hallucination.” Hallucination means when an LLM produces plausible but false information. LLMs are not designed to “know” things. They are designed to say things that are plausible. Hallucination is hard to define though. As best as I can tell, we mean hallucination when we don’t get the result that we want. Its not clear what behaviour was different that causes the bad outcome vs. the desirable outcome where the AI says the truth. It all comes down to what’s called the problem of “mechanistic interpretability.” As of now, we can make LLMs that do some really interesting things. But much like other technological advances like roman concrete, metallurgy, batteries, we are pretty lost as to what’s really happening.

 This generative capacity is what makes an LLM extremely different from a traditional search engine. In the words of Open AI cofounder Andrei Karpathy,

“An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.”

And that’s exactly why we want to use LLMs as search engines! Often, the question that I shoot of to Google is really sideways. There isn’t a straight answer. This is where generative AI is potentially really useful. Instead of me trawling through a bunch of websites, Gemini’s AI overview will do it for me. It will give me a simple answer to a tough question, it will do it in plain English, and it will do it really fast.

Google is offering a solution to the Hallucination problem. Beside each section of an AI overview, Google will cite a source. Look at the screen grab below. On the left is Gemini’s answer, on the right are clickable links that it is claiming are its sources.

However, those sources don’t give me so much peace of mind. I am quite worried that they are giving me a false sense of security that we have gotten around the hallucination problem. In a study about Bing’s AI assisted search, as well as other popular LLM assisted search engines, Shahan Ali Memon at Washington University found

“on average, a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence.”

Memon found that often Gemini would provide a citation that was actually correct, but taking the information completely out of context. “In another example, searching for “beneficial effects of nicotine” generated a list of benefits, including improved mood, improved concentration, and so on (see Image 2). However, the listed source actually pointed to an article discussing why smoking is addictive.”

Bias: and maybe an upshot?

There has been a lot of concern about AI making us double down on racial and gender based bias. A famous example comes from facial recognition software. A study in 2018 found that facial recognition  worked well for white males but failed to accurately recognize women or people with darker skin tones. The study showed error rates of 34% for darker-skinned women compared to 1% for lighter-skinned men.

You are probably thinking, what in the hell is the upshot here? Well, any example of biased AI is coming from a bias in its training data, which is likely human generated. AI are biased because we are. It might be the case that Google Gemini reading through the web will look in places for information that you and I might not. While some of our bias is good, like trusting Wikipedia more than a random blog post, plenty of it is not coming from the right place. Gemini has the chance to change the kind of sources that we trust.

Maybe its better for Grandma?

Have you ever seen your grandma trying to do a google search, using text-to-speech? She begins the search with “Hello”. What follows is something very long winded, probably unclear, and bound to yield no useful results. Now with Google Gemini, grandma doesn’t have to ask a question Googles’ way.

Conclusion

 LLMs show some real promise for being incorporated, somehow, into the search experience. But for now, it doesn't seem wise to trust what they come up with. We might wonder why Google is rolling this out so quickly. In Google's press release in May 2024 eerily entitled, "Generative AI in Search: Let Google do the searching for you", the VP of Google search Liz Reid says that they aimed to reach 1 billion users with Gemini assisted search by the end of 2024. One of their motivations is probably future ad revenue. If I rely on Gemini's AI overview, and I don't click on a website, then I stay on the Google front page. For now, there are no advertisements on Gemini's Ai overview. I will happily eat my words: there will be. 

(This is for an application to Tarbell's AI journalism fellowship. I would really like feedback please!)