TikTok bans accounts using AI to impersonate Black influencers
What's the story
TikTok has banned nearly 20 accounts for using artificial intelligence (AI) to impersonate Black female influencers. The move comes after the BBC reported a disturbing trend of AI-generated content being used to direct users to adult websites. The investigation found that many of these accounts were also present on Instagram, with names like "Black," "Noir," or "Ebony."
Policy violation
AI-generated content used to direct users to adult websites
The AI-generated avatars were often depicted in revealing outfits, with their skin tone artificially altered. This practice has been condemned as both racist and exploitative. The BBC investigation, conducted in partnership with independent AI publication Riddance, discovered a number of accounts on TikTok and Instagram featuring highly sexualized AI-generated Black avatars and characters. These images were not labeled as AI-generated, violating platform guidelines.
Content links
Accounts manipulated videos and reels of real people
The BBC investigation found nearly 60 accounts, mostly on Instagram, linking to websites with sexually explicit content. While these sites labeled the imagery as AI-generated, the Instagram accounts did not. The accounts were found to have manipulated videos and reels of real people on social media platforms. These altered videos were presented as posts by an AI-generated character and some even gained close to three million followers within weeks of their creation.
Exploitation case
Malaysian model Riya Ulan's videos were manipulated
The investigation revealed that videos from Malaysian model Riya Ulan had been manipulated to create these AI-generated avatars. The altered videos made her look much darker-skinned, while the clothes and background were replicated. A series of Riya's videos had been edited to show her in revealing outfits. Further, the links shared with the content led to sites with explicit adult material.
Information
Meta is investigating the matter
In light of this disturbing trend, Meta (Instagram's parent company) said it was investigating the matter. The company also emphasized that it has rules in place to label AI-generated content. Several accounts violating these rules and hosting such content have already been banned from the platform.