It’s not something I want to acknowledge, but during the holiday season I have spent quite a bit of money shopping online. Unsurprisingly, some purchases did not meet my expectations. The photobook that I purchased was damaged while in transit. So I sent a couple of pictures to the retailer and they gave me a refund. For years, online shopping platforms have relied on customer photos to verify that refunds are valid. But generative AI Now, the system is beginning to fall apart.
A Pinch Too Suspicious
The Chinese social media app RedNoteWIRED found more than a dozen comments from customer service agents and ecommerce vendors complaining about refund requests that they claim were generated by AI. A customer once complained about a torn bed sheet, but when they looked at the Chinese characters of the shipping label, it was gibberish. Another buyer sent an image of a coffee cup with what looked like tears in paper. “This is a ceramic cup, not a cardboard cup. Who could tear apart a ceramic cup into layers like this?” The seller has written.
Merchants have reported that AI-generated photos of damage are most commonly abused in a handful of product categories: fresh groceries, low-cost beauty productsItems like fragile ceramic cups are also a good choice. The sellers often do not ask the customer to return this item before issuing a return refund. They are therefore more susceptible to return scams.
A merchant selling live crabs in November received a picture from a client that looked like the majority of crabs had already died. Two others, however, were still alive. Even videos were sent by the customer showing dead crabs being poked in the eye with a human hand. There was something wrong.
“My family has farmed crabs for over 30 years. We’ve never seen a dead crab whose legs are pointing up,” Gao Jing was the seller. She said so in a Douyin video. What ultimately revealed the scam was the gender of the crabs. Two males were present in the video while four females appeared in the second. Another had nine legs instead of the usual eight.
Gao then reported the fraud online to the police. The police determined that the videos were indeed fakes and held the buyer in custody for 8 days. It was a case that received widespread coverage on Chinese social networks, partly because this was the first AI refund fraud of its type to prompt a regulatory reaction.
Lowering Barriers
It’s not just a Chinese problem. Forter is a New York fraud detection firm that estimates the use of AI-doctored photos in refund claims has increased more than 15% since the beginning of this year.
“This trend started in mid-2024, but has accelerated over the past year as image-generation tools have become widely accessible and incredibly easy to use.” Michael Reitblat is the CEO and founder of Forter. The AI does not need to be perfect, he says. Frontline workers at retail and the refund review team may not have time to carefully scrutinize every picture.
