TikTok removes channel full of AI-generated violence towards politicians and migrants

TikTok has removed an account full of AI-generated violence towards politicians, refugees and migrants following a Tell MAMA investigation.

That investigation began after a member of the public flagged an AI-generated video of a white man in a Union Jack t-shirt punching London Mayor Sadiq Khan in the face at a restaurant. It accrued 12,000 likes before its removal. One of the most popular comments read: “One day this will come true”, gained hundreds of likes.

The AI-generated violence in this video depicted a white male in a Union Jack t-shirt violently assaulting Sadiq Khan, gaining thousands of likes and comments wishing for real-world racist violence.

TikTok did not initially remove the content, only doing so after we flagged various concerns about the harmful content across the channel.

The TikTok channel boasted over 17,000 followers and posted a dizzying amount of AI-generated content that reinforced racist tropes about refugees and migrants (linking them broadly to criminality) alongside demographic conspiracies about Muslims, with despicable content linking Muslim men to bestiality, and broadly, to terrorism or as violent cultural threats.

The individual behind the account had links to an earlier banned account that used AI to generate AI materials that depicted the murder of refugees and migrants, including the sinking of dinghies or boats. That channel had created another infamous video of machinery destroying such a boat and leaving those aboard at risk of drowning. In Tell MAMA’s previous report, we drew attention to this example, which appeared on a self-styled “comedy” page, with the video gaining over one million views, which the platform declined to remove. That report also found how AI-generated materials reinforced racist tropes about Black and South Asian men, linking them to rape and violence and directly as a threat to white women and children.

Both pages shared another violent AI-generated video of an assault rifle indiscriminately murdering refugees and migrants, in an article we published last month about how such AI-generated materials normalise, reinforce and perpetuate dehumanising, racialised ideas about Muslims, migrants, refugees and other minorities, whilst normalising violence towards them.

In a wide-ranging report, The Bureau of Investigative Journalism detailed the growing problem of AI-generated videos fuelling racism on TikTok, with some accounts monetising their accounts to sell products.

A Guardian news report revealed earlier this month that OpenAI’s new Sora 2 platform had, within hours, become riddled with content that “depicted copyrighted characters in compromising situations as well as graphic scenes of violence and racism.” 404 Media also raised concerns about Sora 2’s misuse of copyrighted content.

Returning to our investigation, we detailed to TikTok how the user used different AI-generative content platforms to depict violence towards various politicians, notably Muslim politicians like Sadiq Khan and Shabana Mahmood.

The individual behind the account used the software to depict far-right violence towards the prime minister, Sir Keir Starmer, including Stephen Yaxley-Lennon (aka Tommy Robinson) violently punching him.

Other violent examples depicted Nigel Farage punching Starmer in the face and assaulting him with a pint glass.

Other political violence had overt misogyny and racialised misogyny, including one video that depicted a homeless male assaulting a female Labour MP with a wine bottle.

The post TikTok removes channel full of AI-generated violence towards politicians and migrants appeared first on TELL MAMA.

Categories: News, Sadiq Khan, TikTok