Deconstructing the Ecosystem of Online Hate in Social Platforms & Games
| Type of Work | Model - DiagramProject |
|---|
I was brought into the Center for Technology and Society at ADL to help the new VP understand the scope and scale of the ecosystem of online hate through a series of diagrams. The project was intended to get the team all on the same page in terms of what we mean when we talk about this ecosystem and to help everyone understand the scope of what we are talking about.
Through several months of research and stakeholder interviews, I developed a series of diagrams and models - some eventually evolved to be released externally as an Online Ecosystem Primer for ADL constituents while others were just used as talking materials within the team. The text for the first three models is text I wrote, that was then edited by our Editorial Director and is on the ADL Primer page.
Perpetrators / Targets / Mitigators / Amplifiers
- a taxonomy of the system
The Perpetrators/Targets/Mitigators/Amplifiers diagram identifies and examines the four primary “players” within the online hate ecosystem and the context in which each acts. The Perpetrators section examines who perpetrates online hate, what types of hate are happening, and what compels people to post hateful content. The Targets section explores why and how people are targeted, as well as the outcomes of being targeted by online hate. These two sections can mirror each other: religious bigotry, for example, can be committed by those of one creed who target those of another, with targeted individuals and communities including Jews, Muslims, Christians, Mormons and other faiths. Outcomes for targets can be subtle (hurt feelings) to devastating (lost jobs, reputations and lives).
The Amplifiers and Mitigators sections explore the people, places and tactics across social spaces through which hate spreads and rhetoric is broadcast and amplified, or, helpfully, can be mitigated and blocked. What are the activities that occur in gaming, on social platforms, and in other media that spread online hate? Who is working to stop online hate? Looking at the Mitigator section, we see system actors—governments, lawmakers, media—as well as individualș—teachers, parents, celebrities and bystanders—all working in various ways to educate, intervene and develop systems for stopping online hate.
The outer ring of this first diagram lists possible reasons why someone might hold a specific ideology or participate in online hate; what might happen as a result of being targeted by online hate; the types of mitigation efforts and policies that can be used to fight online hate; and the areas within social platforms where we most often see online hate amplified.
By then breaking down the higher-level classifications into discrete categories, actions, and outcomes, we can better understand the breadth of each of these classifications and the relevant context, beginning to understand where there are overlaps between areas (e.g., the media can act as a perpetrator, mitigator and amplifier of hate) and strategize solutions based on causalities and other relationships. We also create a shared vocabulary and start to understand the nuances between the contexts.
Relationships Between the Actors
- A simplified concept model
The Relationships Between Actors diagram looks at the problem of online hate from a different lens, examining the deeper relationships between the actors, specific places and opportunities to intervene and how to make changes in the system in the form of a simple concept model.
The diagram walks us through who posts online hate (inputs), where they post and what happens when they participate in hate against their targets (outputs). What are the relationships between the parts, what paths connect them? Where are the perpetrators found, and where the mitigators? It also includes an option to follow along the path that those who fight hate can take and note where it intersects with the inputs and outputs. This helps illustrate possible entry points for those countering hate. For example, one path traces how nonprofits train law enforcement to recognize hate speech – speech that, as we’ve seen so often in recent years, can radicalize and incite offline violence and terror. This is a very high-level visualization of the landscape and is intended to be an overview to prompt discussions and deeper dives into more granular details around each area.
Corporate Value Chain
- Internet and Finance infrastructure supporting bad actors. This diagram helped inform policy colleagues from the agency when giving testimony on capitol hill about some of the issues of tech infrastructure supporting online hate.
The Corporate Value Chain diagram depicts companies that provide infrastructure to online businesses. Who makes money from online hate and by what means? How is online hate supported through site visitors and advertisers? The complexity of the number and types of infrastructure companies -- by which we mean that part of the tech world that provides services such as payment processors, application hosts, and security providers -- is often hidden from the general public. When tech companies choose to not address hate on a platform they have built and managed, it becomes important for infrastructure companies to evaluate whether to provide services or products to that company. These companies have tended to stay neutral when considering online hate content beyond what is strictly prohibited by law, but we are now beginning to see some of them act to deny their services to egregious purveyors of hate. Hopefully through awareness and pressure, more of them can be compelled to reject supporting online businesses that profit off hate.
Deconstructing Online Hate at IXDA Interaction 21
Online Hate Escalation
This diagram is intended to show what an individual's path into the depths of extremist social media might look like - what inputs and confirmations might happen as well as where and when they might be targeted by extremists taking advantage of their vulnerabilities and trauma. The diagram also shows where education, family and platforms can disrupt and intervene in the cycle of radicalization. This is still in progress to become an animated experience for people to walk through.
PDF available upon request.
Complex Ecosystem
- used exclusively for internal discussions and to work through the complex relationships between actors in the system and their behavior within the social platforms. Each color area details out actors, companies and people of interest related to the topic —which is one of the reasons this diagram was internal only. However, without this diagram, the previous super simple—Relationships between actors—concept model could not have been created.
PDF available upon request.
Network Diagram of Extremist Groups
Initial diagram intended to explore relationships of extremist groups to hateful ideologies. This was hand done based on research that implied general sizes of groups and influence. The diagram got unwieldy when trying to account for the various factions of white supremacists and the overlaps with other related ideologies like neo-nazis, the alt-right, anti-government and others. The diagram was ultimately abandoned but the information was useful as I moved into exploring the other diagrams.
