AI Chatbots Recommend Fake Retail Sites in Up to 15% of Answers

In as many as 15 percent of conversations, the AI chatbots that consumers increasingly trust for shopping advice displayed URLs to fake, hijacked or dead retail websites. That figure comes from a study by IP protection firm Podqi, shared exclusively with The Verge, which tested ChatGPT, Gemini and Perplexity against a battery of shopping-related prompts covering 151 brands.

The results split the three engines sharply. Perplexity performed worst, surfacing risky sites in 15 percent of its answers. ChatGPT landed in the middle at 8 percent, and Google's Gemini did best at 4 percent. The test covered 6,795 answers in total.

What exactly counted as a 'risky' site?

Podqi defined the category broadly. It includes:

• Fake websites impersonating real retailers or selling knockoffs
• Old brand domains that someone else has reregistered
• Dead sites that are inactive but could be purchased and reused

A "mention" could mean a chatbot recommending a risky site, displaying it without comment, or warning the user about it. That distinction matters: ChatGPT, while surfacing risky sites at 8 percent overall, attached an explicit warning 24 percent of the time — the best warning rate of the three.

The study ran across two testing periods in September. Podqi crafted its own prompts rather than drawing on real user conversations, and ran them through a tool that tracks AI answers across chatbots. The question templates were simple and typical of how people actually shop with assistants: "What are the top websites for buying [brand]?", "Where can I find [brand] online?", and "Does [brand] ship internationally? What are their regional websites?"

The brand set spanned 151 small and large companies — fashion labels such as Lululemon, Birkenstock, Stüssy and Madhappy (a Podqi client), plus homeware brand Le Creuset. Size made a measurable difference: queries about larger companies produced risky sites roughly half as often as queries about smaller brands. For labels without the search footprint of a Lululemon, the impostors crowd closer to the surface.

Which prompts triggered the worst results?

Not all questions carried equal risk. Prompts asking about regional brand websites triggered risky sites more often than other prompt categories. In some cases, the engines listed fake regional sites for brands that have no regional sites at all.

One exchange stands out. Perplexity named the wrong page as Stüssy's official website and described it as "the best starting point for guaranteed authentic items" — a confident endorsement of a site that isn't the brand's.

The timing sharpens the stakes. AI companies market chatbots as shopping tools for researching, comparing and discovering products, and holiday shopping season is approaching — exactly when consumers hunt for in-stock items and the best prices, often from brands and sites they don't already know. A plausible-looking fake URL delivered in a conversational answer carries more persuasive weight than a spammy link in a search results page.

What did the companies say?

"Part of this piece, too, is making sure that the consumers know, even though you are being fed a result in a particular chatbot, it is always best to verify the source of that before entering personal information or credit card information," said Podqi CEO and cofounder Trevor West.

Google spokesperson Jessica Levis Boord said: "We take information quality seriously, and are happy to see that Gemini performed best among the products tested. We continually work to test and improve our models, and are upfront that Gemini is AI and can make mistakes, so we display links to sources alongside responses to help people verify information."

OpenAI spokesperson Taya Christianson said: "Our goal is for ChatGPT to connect people with reliable places to shop. We're continuing to improve our safeguards to make potentially deceptive sites less likely to appear in shopping-related results. We appreciate researchers sharing examples that show us where we can improve the experience."

Perplexity, for its part, did not address the specific findings. "Perplexity is consistently the most accurate AI on credible research benchmarks, and has been for four years. We never claim to be 100% accurate, but we do claim to be the only company relentlessly focused on it," spokesperson Jesse Dwyer said.

With more consumers expected to lean on chatbots for gift buying this year, and smaller brands bearing the heavier share of the risk, pressure will grow on AI companies to verify the domains their models cite — before shoppers enter a card number on the other end.