|
Getting your Trinity Audio player ready...
|
By Melisa Mong’ina
Nairobi, Kenya: When Aidah Munzatsi posts a video on TikTok, she braces herself for the criticism that often follows posts made by influential creators. Munzatsi, a health and science communicator and sexual and reproductive health rights (SRHR) advocate with more than 1.4 million followers across digital platforms, has built a large audience by sharing information that helps people make informed health decisions.
However, as a woman creating content on a topic considered sensitive, she has experienced online harassment.
“Sometimes people send me inappropriate photos,” she says. “They do not understand the educational point of it.”
Many women using social media platforms to educate, entertain, advocate and earn a living face online abuse. A 2021 study by The Economist Intelligence Unit showed that half of the women surveyed considered the internet an unsafe space to express their thoughts and opinions, while 35% reported mental health impacts after experiencing online violence.
While social media platforms rely on a combination of artificial intelligence (AI) and human moderators to identify and remove harmful content, AI-powered content moderation systems often fail to recognise abuse expressed in local languages, slang and coded insults.
“There are words that the system may not detect, so a human being needs to go through it, and then the platform will take action,” says Munzatsi. “Personally, I feel like TikTok has responded positively to me even though sometimes they take time.”
Ironically, most of the people who harass her in public seek her help in private.
“I look at the insults like a cry for help because once they have your attention, they come to your DMs requesting assistance,” she says.
Rather than reporting every individual, sometimes Munzatsi posts the offensive comments publicly and calls out the behaviour. This prompts her followers to report the offending accounts or comments.
She blocks those who repeatedly target her to protect her work and mental well-being.
Munzatsi’s experience highlights a recurring pattern described by several creators: while AI appears effective at detecting some forms of abuse, it misses abusive comments written in local languages or slang.
Why AI Misses Some Harmful Content
According to Newton Gichuru, Director of Hush Solutions, a Nairobi-based digital agency that trains businesses and professionals across Kenya and East Africa on the use of AI tools, most social media platforms rely on machine learning models to identify and detect harmful content.
“When a comment or video comes in, the system scores it against patterns it has learnt. If the score crosses a certain threshold, the content gets auto-removed or demoted so fewer people see it, or it’s queued for human review,” he explains.
He notes that for women creators, the system identifies sexualised language or threats of violence where one creator is targeted by numerous accounts. However, most abuse goes undetected because the models were mostly trained on English language data from the Western market.
In Kenya, users often mix Kiswahili, Sheng and English in one sentence, and abusive content is frequently disguised through emojis, deliberate misspellings and coded phrases that carry offensive meaning.
“A lot of what women here experience gets through because the machine simply does not recognise it as abuse,” he explains.
Gichuru notes that when reports are sent for human review, there may be delays due to too few moderators, and some reviewers may not understand the local language or cultural context needed to identify the abuse.
“An insult in Sheng that any Nairobi teenager understands instantly will often mean nothing to a model trained mainly on English,” says Gichuru.
He adds that sarcasm, memes and indirect insults are the most difficult for AI systems to interpret because meaning often lies in context rather than the words themselves. As soon as one coded term is identified and flagged, abusers adopt new ones, making it hard for moderation systems to catch up.
Cost of Going Viral
Mirriam Chepkemoi, an actress and nutritionist who posts humorous skits to entertain and educate her followers, has noticed that as her audience grows, so do the attacks.
“I mostly receive verbal abuse in the comments, including insults about appearance and sometimes my content,” she says. “There are also cases of body shaming and sarcastic remarks. It doesn’t happen every day, but it’s frequent enough, especially when a video gets more reach.”
Initially, when she reported, she noticed that only a small number of the abusive comments were taken down.
“Sometimes TikTok removes the content, but in many cases, I get feedback saying the content did not violate guidelines. So nowadays I just ignore them and keep moving,” she explains.
“It makes me hesitant to post particular content or express myself freely,” says Chepkemoi. “However, I try to stay focused on my purpose and the positive impact I want to create.”
Local Language Blind Spot
For Tabby Wothaya, who creates educational content advocating for girls, the abuse is in the form of sexist stereotypes. Comments frequently make false assumptions about her family and personal life.
“They say I belong in the kitchen, that I’m jealous of younger women because I’ve ‘grown old’, that I have trauma from being raised by a single mother, or that I’m a bitter single mum. The funny thing is none of that is true,” she says.
She rarely reports abusive comments, but when she does, TikTok’s response takes time. She has also noticed that the platform’s moderation system struggles with content in local languages.
“It doesn’t understand comments that aren’t in English. I’ve seen it restrict comments that weren’t abusive at all simply because they weren’t in English,” she explains.
“I’ve had multiple videos about femicide and paedophilia flagged, yet abusive comments are left on the platform,” she adds.
The experiences described by the three creators reflect patterns documented by researchers. Data from KICTANet shows that technology-facilitated gender-based violence (TFGBV) – trolling, online harassment, and the misuse of AI to create and spread harmful content – is widespread, with 63 per cent of women in urban areas and 37 per cent in rural areas reporting experiences of online harassment or abuse.
“We have observed countless forms of technology-facilitated gender-based violence through our OGBV tracker. We also receive private complaints from women experiencing different forms of online abuse,” says Cherie Oyier, Program Lead, Women’s Digital Rights at KICTANet.
Similarly, Nendo’s Dada Disinfo study, which analysed social media posts involving 143 content creators, found that sexual harassment accounted for nearly half of abusive posts targeting women content creators. The study also found overwhelming negative sentiment across more than 50,000 posts.
Limited awareness of reporting tools and persistent language barriers undermine efforts to address online gender-based abuse.
“While moderation mechanisms are somehow adequate, the fact that the algorithms are not trained in local languages makes it easy for abuse in those languages to bypass the systems,” explains Oyier.
To address this challenge, KICTANet has developed lexicons in Kiswahili, Dholuo, Kikuyu, Somali, Ateso, and Luhya and is engaging platforms, including TikTok, to incorporate them in content moderation algorithms.
TikTok says it combines automated and human moderation to detect and remove harmful content such as misinformation and hate speech. In an emailed response, the company pointed to its Community Guidelines Enforcement Report, saying that it removed videos that violated community guidelines.
“In the fourth quarter, TikTok removed 820,552 videos in Kenya for violating its community guidelines. Of these, 99.9 per cent were proactively removed before anyone reported them, while 98.4 per cent were taken down within 24 hours of posting,” Pereruan Kanana, TikTok’s Communications Lead for East Africa, wrote in an email.
The company says this reflects its commitment to offering a safe and trusted platform for users. It has partnered with the Centre for Analytics and Behavioural Change, the Association of Media Women in Kenya and the Kenya Editors Guild to deepen understanding of TFGBV.
“These engagements provided valuable insights into emerging risks and language patterns, helping strengthen our moderation efforts and enhance user safety,” says the company.
What Needs to Change
To better protect women creators, Gichuru stresses the need to train the AI models on local and cultural language, assisted by human moderators who also understand context. He adds that creators should be provided with faster and clearer reporting tools to filter or limit the attacks.
“The goal is not to delete everything that offends someone,” he says. “It is to draw a clear line between criticism or disagreement, which people are entitled to, and targeted harassment, threats and dehumanising attacks, which are not speech worth protecting.”
Oyier says that platforms need to be more transparent on how moderation decisions are made.
“The platforms should be held accountable. For this to happen, regulators must put in place appropriate laws and regulations,” she stresses.
For now, Munzatsi, Chepkemoi, and Wothaya continue posting their content. But with every video, they prepare themselves for what the comment section might bring, because they know the moderation system does not detect many abusive comments directed at them.
This article was produced as part of the Gender+AI Reporting Fellowship, with support from the Africa Women’s Journalism Project (AWJP) in partnership with DW Akademie. The journalist used AI-assisted research tools to review and summarise relevant policy and research documents, identify patterns, surface data points for independent verification, and extract key statistics. All reporting, analysis, editorial decisions, fact-checking and final wording were done by the reporter, in line with Talk Africa’s editorial standards.
