Mitigating the impact of online extremism has proven a complicated task for companies that want to protect free expression.
Social media companies have struggled to moderate hate speech and adapt to changes in how white supremacists spread their views on digital platforms. Libby Hemphill, an associate professor at the University of Michigan School of Information, believes machine learning technology might provide an answer.
“We know that white supremacists and other types of extremists use social media to talk to each other, to recruit, to try to get their message to go mainstream,” Hemphill said. “The challenge has been that the platforms haven’t really stepped up to fight hate on their platforms.”
Through a partnership with the Anti-Defamation League, Hemphill set out to teach algorithms to distinguish white supremacist and extremist speech from the typical conversations people are having on social media. Turns out, extremist groups are pretty good at hiding in plain sight but algorithms can become even better at finding them.
“We can’t do content moderation without some machine assistance,” Hemphill said. “There’s just too much content.”
Hemphill started by collecting a sample of 275,000 blog posts from Stormfront, a white nationalist website. The data was fed to algorithms used to study the sentence structure of posts, detect specific phrases and flag recurring topics.
The goal was to train a machine to identify toxic language using the Stormfront conversations as a model. Algorithms compared the Stormfront data to 755,000 Twitter posts from users affiliated with the alt-right movement and another set of 510,000 Reddit posts collected from general users.
The results are still being compiled. Hemphill is hoping to unveil a public tool later this fall.
Big tech companies like Facebook and Twitter have been accused of stifling free speech by removing users who violate community guidelines prohibiting hate speech. Offending posts are identified by algorithms trained to detect hate speech and through reports from users.
However, advocacy organisations aren’t satisfied with enforcement standards.
The Center to Counter Digital Hate found the top five social media companies took no action on 84% of anti-semitic posts reported to them. CCDH, a non-profit group in the United States and the United Kingdom, flagged 714 posts through the platforms’ user reporting tools. The anti-semitic posts were viewed up to seven million times.
“Platforms must aggressively remedy their moderation systems which have been proven to be insufficient,” CCDH CEO Imran Ahmed wrote in a study announcing the group’s findings.
Steps to ban white nationalist content also inspired the rise of new platforms with more relaxed content standards. Gab, MeWe and Parler purport to champion free speech but have been criticised as havens for extremists.
Gab CEO Andrew Torba promotes his website as “the only place you can criticise groups like the American Jewish Congress and the ADL” in emails to Gab’s user base.
Hemphill said there’s significant conflict revolving around what constitutes hate speech and what social platforms should do about it. She also acknowledged concerns that machines can make mistakes and unfairly punish users.
“The challenge is that we as a community don’t share a set of values,” Hemphill said. “We disagree about what is hateful and what ought to be permissible. One of the things that I would like to see come out of work like this is more explicit discussions about what our values are and what is okay with us and what isn’t. Then we can worry about what we ought to teach machines.”
Finding the line can be tricky. Accurately identifying nuanced racial stereotypes and microaggressions is harder than finding slurs, Hemphill said. There’s also a large number of posts containing meme images and videos that are harder to accurately catalogue.
Stormfront was used as a baseline in Hemphill’s research because the group openly identifies as a forum for white nationalists. Alt-right Twitter users were selected from a study commissioned by the Centre for American Progress.
Beyond the use of racial slurs and other kinds of toxic language, Hemphill said subtle differences were found between white nationalists and the average internet user. For example, white nationalists swear less often, possibly a tactic to appear more palatable to mainstream audiences.
“They are sort of politely hateful,” Hemphill said. “If you’ve spent any time on the Internet at all, you know that it’s a pretty profane place, but white supremacists are not profane. They are marked by what they don’t do as well.”
Hate speech makes up a relatively small amount of content on the Internet, Hemphill said, though it makes a large impact on whoever sees it. She plans to collect more data from smaller websites like Gab, 4chan and Parler to continue teaching algorithms to identify hate speech.
Ultimately, the goal is to encourage social media companies to create inclusive spaces for discussion.
“It’s OK to be freaked out about big tech, and it’s OK to be freaked out about academic researchers and what we may be doing,” Hemphill said. “I think the more this conversation happens in the open and the more data is shared among folks who are trying to understand how these decisions get made, the better, more just decisions about what is permissible will come from that openness.” – mlive.com/Tribune News Service
