Somewhere on TikTok right now, a Black woman named Aliyah is crying about her struggling belt buckle business. She is also not real.
Aliyah is one of dozens of AI-generated personas researchers have identified across TikTok, Instagram, and Facebook — digital characters built specifically to exploit racial empathy and guilt in order to sell mass-produced fast fashion. The belt buckles she supposedly handcrafts? Available on Shein for a fraction of the price. The tears streaming down her face? Generated by software that, frankly, isn't even doing a convincing job.
The tells are there if you look. The voice is flat and robotic, completely disconnected from the emotion on screen. In one clip, she sews leather in a spot where no sewing would ever happen. When she wipes away a tear, the liquid below her finger vanishes too — a classic artifact of AI video generation. And the background, the tabletop, the spool of twine? Identical across multiple accounts featuring entirely different AI characters.
This is not a small operation. Jeremy Carrasco, a researcher at Riddance.ai, an organization focused on AI video detection, told The Verge his team is identifying up to 100 of these accounts every single day. Some are solo schemes. Others appear coordinated, running a single AI avatar across multiple storefronts or cycling through several characters to hawk different product lines — mugs, crochet bags, cardigans, you name it.
What makes this particular scam especially uncomfortable is how deliberately it weaponizes identity. These accounts are not just pretending someone handmade a product. They are specifically constructing Black personas, complete with automated comment responses that attempt to mimic African American vernacular, to trigger a guilt response in potential buyers. The implied message is hard to miss: support a small Black creator or be complicit in her failure. It is manipulative in a way that goes well beyond standard dropshipping fraud.
TikTok's AI disclosure rules require creators to label synthetic content, and some of these accounts do add that label. But a small disclaimer at the bottom of a crying video is doing approximately zero work against the emotional pull the content is engineered to create. The label is there; the manipulation still lands.
The broader problem is one of scale and speed. Generative AI has made it trivially easy to spin up a convincing-enough human face, attach a backstory, link it to a Shopify storefront, and automate the entire customer interaction pipeline. The marginal cost of launching one of these accounts is close to zero, which means the economics work even if conversion rates are low.
Platforms are in a difficult position. Detecting synthetic video is genuinely hard, and bad actors iterate faster than moderation teams can respond. Meanwhile, researchers like Carrasco are essentially running a manual triage operation, flagging accounts one by one while the supply keeps growing.
The scam is gross on multiple levels at once — it defrauds buyers, it appropriates and commodifies Black identity, and it pollutes the creator economy that actual small businesses depend on. And right now, it is scaling faster than anyone is stopping it.