Deen Freelon: Identity factors, which include, you know, not only race, gender, in some cases, sexual identity, national origin, also in some cases religion, really help to get a fuller picture of what’s going on the web and in various digital domains. So that’s something I’d encourage every web science practitioner to do, first of all, to read up on it, to figure out how to integrate that into the work they’re already doing, and then secondly of course, to implement that knowledge.
Noshir Contractor: Welcome to this episode of Untangling The Web, a podcast of the web science trust. I am Noshir Contractor and I will be your host today. On this podcast we bring thought leaders to explore how the web is shaping society and how society in turn is shaping the web.
You just heard our guest today, Deen Freelon, talking about why identity is key to understanding the complex interplay between the Web and society. Deen is an associate professor in the School of Media and Journalism at the University of North Carolina in Chapel Hill. His research covers two major areas of scholarship: political expression through digital media, as well as data science and computational methods for analyzing large digital datasets. He has authored or co- authored more than 30 journal articles, book chapters and public reports, in addition to editing a scholarly book. He has also served as principal investigator on grants from the Knight Foundation, the Spencer Foundation and the US Institute of Peace. Professor Freelon has been at the forefront of research into misinformation, disinformation, hyperpartisan content, ideological asymmetry, identity politics, and personalized information environments. And as a member of the web science community, Deen writes lots of software to analyze data, some of which he releases in open source spaces. Welcome, Deen.
Deen Freelon: Thanks.
Noshir Contractor: I’m so glad that you’re able to join us here today, I have been a huge fan of your work for a long time. Let me first begin by asking you, how did you first get interested in studying the web?
Deen Freelon:I’ve always been a bit of a nerd, my dad was an early adopter of computers, I learned how to do web pages when I was in high school, this is mid-90s. I went to college thinking that I was going to be a computer science major, but I was at Stanford at the time. And I I found that the way they taught it wasn’t quite wasn’t quite my speed. So I sort of pulled back and I majored in psychology. Later, I taught myself how to do PHP in my first job, which was a as a technology trainer at Duke University, which is in my hometown. And at the same time, I was teaching myself how to code, I was also becoming more politically aware, right. So this is around the time, 2000, 2003, start of the Iraq war, and all that. So the code piece and political piece were happening right around the same time. And so it was only later that I realized, wow, I kind of had these two pieces of my eventual scholarly identity, that were percolating and evolving at the same time. And this is actually before the field of Communication Studies, and probably web science as well, starts to become aware of computational methods and data science is a key component of both of those. And so really, it was kind of serendipity that I ended up having those skills and those interests at a time when those fields were starting to value those and starting to promote them.
Noshir Contractor: Well, I think we’re all very lucky for that serendipity, because you really were the right person at the right time. And one of the things that I really admire about your work, Deen, over the years is that you’ve taken issues and been able to capture it in a way that advances intellectual insights, but also speaks to a larger public. And you’ve done this in an amazing way in your scholarship, as well as your public engagement. Talk a little bit about how you began to think about these issues. I’ll throw a couple of recent papers that you’ve written. You have a paper called “False equivalencies: Online activism from left to right.” Tell us a little bit about what this false equivalence is, and why it might be going against the grain of some conventional wisdom that we might be listening to in this area.
Deen Freelon: That paper is really the culmination of a lot of thoughts that I’ve had over the past, I don’t know, probably half a decade at least. And the false equivalency is between the left and right, so you have a lot of work on the left that has really come from and we talked about this in the paper, from kind of the hashtag activism school, right, so let’s look, you know, there’s a lot of work on, you know, Black Lives Matter, there’s a lot of work on, you know, the climate change movement in terms of their use of hashtags. And so there is one view of the the left, and actually, that connects to prior work. That’s not computational or web science in nature, primarily in sociology and communication, that in which the left has overwhelmingly been focused on, when you, when you’re talking about social movements, social activism.
And you’ve got work on the right, that really comes out of the tradition of sort of the right wing media ecosystem, which of course, long predates the web, right, going back all the way back to the 30s. But you know, really intensifies the 1980s, and the sort of mistrust of the mainstream media, that, that dates back decades as well. And so those sort of very divergent research traditions, I thought were really interesting and important to look at in contrast in that piece. And so that’s really what it does, it tries to figure out, you know, how the left does business as far as activism goes, how the right does business? What similarities are there? They’re both online, they both use many of the same social media platforms. What differences are there? The literature tells us that, for example, disinformation is a much bigger problem on the right than it is on the left, the issue that we identify in the piece, or one of the issues we identify, is that there hasn’t been that much research on disinformation on the left. So there’s a couple possibilities. One possibility is the research record reflects reality, right? Disinformation is a bigger problem on the right, than it is on the left. Another possibility is that, because there hasn’t been quite as much research done on disinformation on the left, we simply don’t know.
What we call for in that piece is to try to figure out exactly what is going on as far as disinformation on the left goes. Searching through the literature, we didn’t really find that there were that many attempts to even answer the question. So what we want what we’re, what we’re advocating for is an affirmative answer, to this question of how much disinformation there really is, in terms of left wing left leaning or left oriented, so that we can characterize it against the disinformation that we know is rampant on the right.
Noshir Contractor: Deen, why do you think there hasn’t been more studies that have tried to examine disinformation on the left?
Deen Freelon: That’s a good question. I think some of the disinformation may not be quite as out there. I think, as we saw in terms of the events of January 6, there is a very strong argument to be made that the disinformation on the right, apart from how much of it there is, I think that the character of it is a lot more virulent and more likely to result in injury and harm to bodies, specifically, as well as to democratic norms. And so I think there’s a greater urgency there simply because of that. However, I do think it’s more than a mere scholarly curiosity in terms of characterizing the nature of disinformation that may appeal to the left as compared to that which appeals to the right. We simply haven’t done that work. I think it’s analytically important. I think it has public importance as well.
Some of it may have to do with the political commitments of the people who do the research.I don’t — I’m certainly not going to cast aspersions on anyone who does that kind of work, and I certainly don’t know enough about their political commitments to be able to say definitively, that’s just how, you know, confirmation bias and sort of, you know, motivated reasoning tend to work.
This is something that, again, extends from research tradition that extends, at least until the 60s, you know, the studies of the civil rights movement, being kind of the paradigmatic social movement. And even if you look at some of the definitions of social movement, some of it actually has, almost seems to have left wing politics built into it. And so I don’t think that’s a great idea. But I do think that some of the analytical pieces of this also play a role in determining what gets categorized as a quote unquote social movement, and what is studied as, you know, reactionary politics or, or mainstream politics, because they’re practiced by people of different ideological commitments.
Noshir Contractor: So you’re not making a conspiracy argument, you’re just saying that this is a scientific curiosity that needs to be balanced across the left and the right.
Deen Freelon: Yeah, I really try not, I really try to stay away from any and all conspiracies. I do think that, you know, in that review, I think we’re doing what good reviews do, which is to point out, you know, gaps in the literature to say, we’ve done a really good job over here, we haven’t done quite as much work over here. So let’s, you know, balance the scales a little bit.
Noshir Contractor: One of the things that, obviously, is front and center on many of our minds these days, especially in the United States, is the Black Lives Matter movement. And I want you to talk a little bit about your piece that was titled,”Black trolls matter: Racial and ideological asymmetries and social media disinformation.”
Deen Freelon: Sure. Well, I want to give credit for that title to the wonderful Jeff Hancock of Stanford University. That piece really grew out of my work on Black Lives Matter. I did a report, a public report that came out 2016 and a follow up empirical article a couple of years after that. And so that actually was one of my big entree into the world of online disinformation, because I had this big black lives matter dataset. And when the internet research agency, Russian troll list of handles came out at the end of 2017, I basically just looked into my Black Lives Matter dataset and said, Wow, there’s like 300, you know, some names from this data set represented in my Black Lives Matter data set. So I said, Okay, well, this is definitely something I have to study, because they seem to have some interest in activism, specifically Black activism. And that piece of research that you, that you mentioned, is really the culmination of that investigation.
What we found was that Black-presenting Russian trolls were actually more likely than any other of the categories that we looked at, which included right wing trolls,non-Black left wing troll trolls, and a couple of other ones. They were more likely to pull in retweets, replies and likes on a per tweet basis. And we thought that was quite remarkable, especially because the study design allowed us to disaggregate the influence of ideology from race.
Noshir Contractor: Can you talk a little bit more about that? What does it mean to be able to disambiguate race from ideology? And also, if you could just recap again, what exactly was the asymmetry in the social media disinformation that you found?
Deen Freelon: So we rely on that study on categories that came from a couple of researchers out of Clemson University, They came up with a really great initial typology, they lumped together, Black left wingers and non-Black left wingers, and so based on some theory that we detail in the piece, we made the theoretical argument for disaggregating those. We found out that a substantial amount of the effect for likes retweets and replies that were attributed initially to left-leaning were actually explained by Black-presenting, right. We found was a very, very strong indicator that the Black presentation was actually driving, a lot of, a significant portion of the effect.That’s where the asymmetry comes from, the asymmetry between left and right being more effectively explained by race than by ideology. And also the asymmetry between being sort of non-Black left wing as well as between Black left-leaning.
Noshir Contractor: That is incredibly interesting, because it’s so easy for us to conflate some of these in our stereotypes. I’m going to ask you a more general question, do you make a distinction between disinformation and misinformation?
Deen Freelon: If you look at our piece that ran in political communication last year, “Disinformation is political communication,” we talk about disinformation as being false or misleading content that is intentionally spread to damage a third party. So that is where the person spreading it is aware of the deceptive nature of what they’re spreading. And they’re doing it with a specific goal of damaging some enemy. Misinformation is where content is spread, without knowledge on the part of the spreader that it’s false, or that there is some deceptive element to it. And so what that actually implies is that dis- and misinformation are not necessarily inherent qualities of the content itself, but rather, they are relations between the people who spread them and the content.
Noshir Contractor: And so by that definition, then the two pieces that you wrote about Russia, one titled “The Russian Disinformation Campaign on Twitter” and the other about Russia as internet research agency, appearing in the US News is Vox Populi, tell us a little bit about how you got interested in this particular issue. And what were some of the key takeaways for you?
Deen Freelon: I feel that my interest in disinformation is sort of, you know, charitably achieved through my interest in social movements, and in the way that a lot of the most prominent disinformation including the IRA and others, have really tried to glom on to existing social movements, to be able to spread their falsehoods. And so I think that is something that is a logical outgrowth of outgrowth of the work that I’ve done.A lot of the work that I that I have done in this has really stuck close to the sort of the relationship between disinformation and social movements, because that’s something I’ve been interested in since I was a grad student.
Noshir Contractor: And you find that in the case of the “Russian Disinformation Campaign,” one of the things that you argue, which again, is counter to the conventional wisdom, is that the disinformation campaign on Twitter targeted political communities from across the spectrum, not just from the left, as some in the media would have us believe.
Deen Freelon: The internet research agency, which was a very specific group of paid Russian trolls that were paid by the Russian government, targeted, not only you know, folks in the Black community, or on the left, they also targeted folks on the right. And one of the studies, the study that was published in the Misinformation Review, my my colleague, Tanya Loca, and I point out that the specific identity that the IRA agents took on was the same identity of the people that they actually wanted to reach. So conservative presenting trolls wanted to reach conservatives, Black-presenting trolls, mostly reached Black individuals, left presenting trolls reached out to and actually ultimately reached left-leaning individuals. So in some ways, that’s actually helpful analytically to understand exactly what they’re doing. They’re playing on this, this idea that most of us who study social media, and web science understand, which is like follows like, right, you know, birds of a feather flock together. And so they’re really taking advantage of that specific tendency on the internet and social media, to be able to reach out to folks and have the real individuals who share those political identities to carry forth their disinformation for them. And that’s one of the main ways that they’re able to get traction is to have real people sharing, retweeting and engaging with their content, which gives it that imprimatur of reality.
Noshir Contractor: And what was interesting is that you suggest that the best way to counter that or at least one way to counter the Russian disinformation campaign, would be for people across the political spectrum to collaborate against it? Tell us more about that.
Deen Freelon: Now, that was, in all honesty, a bit of a pipe dream, right. I mean, we’re pretty, we’re pretty polarized, I think in our country right now. But I think if there are, I think if there are opportunities to do that, I think it would be a great thing. I don’t know anybody who openly proclaims that having, you know, foreign agents, infiltrating our political conversations is a good thing. So it seems to be at least in principle, to be something where people from differing sections of the political spectrum could come together and agree at least, that this is a bad thing, and we should find ways to, to combat it. So now, in terms of how likely that is, I don’t really know.
Noshir Contractor: Yes, we are living in rather, hyperpolarized times, as you might put it. You did a project that has been going on for a period of time called the filter map. Tell us more about where that project started and where it is now.
Deen Freelon: “The filter map” is the name of a piece that came out, I was commissioned to write this piece by the Knight Foundation, and it came out in 2018. And in that piece, I sort of take issue with some of the conversations that were occurring around ideas of the echo chamber, and the filter map. The idea at the time was, Oh, well, people really need to engage with content that lies outside of their own bubble, right. So it’s, it’s content that is produced by people who disagree with them, they need to engage across ideological lines. And my contribution to the conversation is, there are certain ideas that it is not fruitful for us to engage with, right. So if you’re talking about, you know, open racism, open sexism, you know, Nazism, things of this nature, these aren’t ideas that we should give the time of day to, so to speak. And so what I tried to do in the piece is I tried to articulate the kinds of ideas that we disagree with, that we may want to give the time of day to, and those kinds of ideas that we may not want to, right.
So the idea behind the filter bubble, is to say, whether you agree with something as sort of one aspect of your relationship to an idea. A second aspect is, if you’ve decided you disagree with something, whether it lies beyond the pale of things that you would at least consider as a second factor. And so that general set of ideas kindly of sat on the shelf for a little bit, until I was lucky enough with three of my colleagues to be able to receive one of the big Knight Foundation Center endowing grants in 2019. And at that time, I realized that I had an opportunity to put the ideas in this filter map into practice.
So I’ve collapsed it into two dimensions. And so one dimension is if you think about this horizontally left versus right, so there’s been a lot of progress in the past few years, a few years in terms of ideologically scaling, media personalities, media outlets, and Twitter handles, things like that. So you can think about that as being scaled in a horizontal axis, as well as on a vertical axis that would look at things like the total number of, you know, ratings that you get on PolitiFact, right. So if you’re high truth, you’re up here, you’re low truth, you’re down here, right? So and now you got two axes that shows left, right, one, high truth, low truth, up and down. And you can actually look at your own social media feed and see how much of each quadrant you actually get. So if you think about above, board where your high truth, that’s where you’re seeing the kind of content that you want to engage with, oh, here’s the high truth, right wing stuff, okay? Let’s think about that. Let’s engage with that. And if it’s low truth, well, it’s low truth, and it’s on my side, that’s maybe disinformation that’s trying to target me, that’s where I’m most vulnerable. And that’s what I want to keep out of my information stream.My hope is that that will help people understand their social media feeds better. And it’ll help put some of this heady, you know, theoretical stuff into practice in a way that ideally makes people’s lives a little bit better.
Noshir Contractor: This is an example of how you make your scholarship very actionable or potentially actionable by individuals in terms of giving them something to look at. You’ve also contributed by way of sharing code and your software tools that you’ve developed it etc. Tell us about why you chose to do that. And what do you see as the challenges and opportunities for people in the web science community to be sharing their code.
Deen Freelon: Sure. Well, I first started sharing my code when I was a grad student. And actually, the very first thing I shared is by far the most popular thing I’ve ever shared. And that is Recall, which is online, in a code of reliability calculator for content analysis. So that’s kind of that’s kind of fun for me, and I think, useful for many people.
In many ways the success of that project, which was really just an offshoot of my master’s thesis, I mean, the the short version of the story was that before Recall, the primary intercoder reliability program with something called Pram, and it only ran on Windows, and I had a Mac. And you know, I did my grad work in Seattle. And, if you know, Seattle, it’s very rainy. And so when I was doing my Unicode reliability tests for, for my master’s thesis, I didn’t want to walk from my apartment all the way to the lab, in the University of Washington Communication Department. So I said, well, I’ll just make one myself and program this thing, literally do the math myself to do this, sort of prettied it up, and made it usable for others, when I put it on my website. So the success of that really led to other you know, sort of forays into writing software for the research community, I think is incredibly important. I think of how I personally have benefited from other people’s software that they’ve created that’s been on an open source basis. And I just want to give back a little bit to that one issue, I find that I think hampers people from from doing this is that, especially outside of computer science, and perhaps information science, the production of open source software for the research community is often not seen as, as, as much of a contribution as it should be.
Noshir Contractor: We’ve talked about sharing code. What about sharing data? You were involved as part of the beta test that Twitter has offered to make all of its data available for free for researchers who apply for it. So tell us about moving from sharing code to sharing data in the web science community.
Deen Freelon: This is a really big topic. Our access to data, especially that which is owned by or stewarded by, for profit corporations is fundamentally tenuous. We’ve seen, you know, the rise of social science, one, which provides application based access and also money to Facebook data, we’ve seen this more recent initiative by Twitter, which allows access all the way back to the first tweet in 2006, to researchers who applied but again, even though I applaud that particular move by Twitter, ultimately, they have a say over who they accept, in terms of this program. Thatstill puts a lot of power in their hands in terms of deciding who gets to access this kind of data, and who gets to do this kind of research. I think that any researcher in the web science area should really have what I consider to be a diversified portfolio in terms of the data streams that they’re working with. So don’t become over reliant on one type of data, to be able to get your work done. So a lot has been written about and said about our field’s over reliance on Twitter data. And so you know, if Twitter data is your only game in town, well, if Twitter decides, you know, that giving this kind of data access is not in their best interest, or if they decide to reject your application for access to this wonderful, you know, time-unlimited stream, then you’re not going to be in a very good position. So having a number of different data sets that can speak to the kind of questions that you’re interested in,whatever they may be, I think is critical for being a web science researcher in 2021.
Noshir Contractor: We talked earlier about polarization, and I’m going to use that as a pivot to a very polarizing concept that I would love to get your take on. And that is, the notion of being able to infer individual level characteristics from digital trace data. You get people on one end of it, who think that that’s the most incredibly powerful way of being able to get to things and others who think that is the scariest idea on the web.
Deen Freelon: Well, this is something I’ve been thinking about for a very long time, and it’s something that I feel like I wish more people paid attention to. Because there are certain norms in certain fields, that don’t really think or are not thoughtful enough about what those traces really mean. For some researchers, it seems that simply studying the trace itself is enough. And there’s not really a whole lot of discussion about what theories this may apply to, and what those traces actually mean.
So I think that under certain circumstances, certain digital traces are really, really great proxies for things that we really care about. In other cases, the fit may not be so great, but what I really want the scholarly community to do, web science and other social sciences, is to really consider carefully the fit between the, the theoretical concepts and research questions of interest and the data to which they have access.
Noshir Contractor: What do you see today, based, either on your own work, or more generally, what do you see as important issues that web science should be addressing moving forward?
Deen Freelon: Again, another really big question. I’ll just sort of beat a drum that I’ve been talking about for a while now. I think that, you know, web science community is in many ways, not unique among social sciences in underestimating the importance of identity more broadly, and race specifically. So, when you’re thinking about any topic that you deal with whether it’s virality or some of the more policy-oriented aspects of this, keeping an identity-focused aspect of this firmly in mind is really important. Identity factors, which include you know, not only race, gender, in some cases, sexual identity, national origin, also in some cases religion, really help to get a fuller picture of what’s going on the web and in various digital domains. So that’s something I’d encourage every web science practitioner to do, first of all, to read up on it, to figure out how to integrate that into the work they’re already doing, and then secondly of course, to implement that knowledge.
Noshir Contractor: Now that’s extremely important especially because in some ways, one can argue that the web conceals some of the normal visual surface level characteristics that we will look closely at many of these identity issues, not all, but some of these identity issues.
Deen Freelon: Yeah, and that actually ties back into the trace data issue, So one of the examples has to do with the underlying concept of gender versus race. So you’ve got a situation in which gender, generally, at least, anglicized names, can be heard with high levels of accuracy from someone’s first name. Then the question is, to what extent does the system support the use of real quote unquote first name. Facebook has its terms of service that you can use your first name so that’s in terms of service level issue, you can use something else but you risk your account being kicked off. Twitter does not require you to do its Terms of Service, and lots of people don’t. So, you would assume that any study that had a bunch of names of individuals on Facebook, and have a lot easier time determining gender than what an equivalent study on Twitter. Now shift to the idea of race, race is a lot harder, especially in the United States, to infer the basis of someone’s first name. In some cases you might be able to do in other cases, you may not be able to do it.
And so that becomes a lot harder to be able to to get. Actually, a better example is that Facebook allows you to indicate your gender, so, the difference in terms of the identity characteristics that you’re able to get out of those systems is baked into the design of the system. So, that means that some identity characteristics are easier to integrate into a research study than others. But I think that the effort is well worth it when you’re trying to figure out how for example, different soci technical systems are used by different people, how they impact different kinds of people, and how different kinds of people see them.
Noshir Contractor: In closing, in 2020, spending almost a year in isolated confined environments and dealing with all kinds of reckonings, cultural, racial, health-related, etc. Can you talk a little bit about what this entire experience might have been how it might have been different, for better and/or for worse, if we didn’t have the web?
Deen Freelon: The image that popped into my mind was, how would a skyscraper be different if you remove the second floor. The second floor goes away, what happens is, floors three through n crash down, and they crush floor 1. So, I think that you know taking the web, that is so, you know, deeply enmeshed into everything we do, would render our society completely unrecognizable. So it’s not like, okay, you take the web out and you go back to the 1980s. It’s everything that relied on that, everything from banking to getting your takeout with a couple of clicks of an app, to your health, to how you relate to others, the fact we can have this conversation remotely.I just don’t think that would really be something that we could imagine, we can’t really put the genie back in the bottle. We have to live with this, as it is. I think there’s certainly ways that people can use the web better, there are choices that I wish people hadn’t made. I think it’s extremely difficult to imagine our society without the web.
Noshir Contractor: I love the metaphor of the second floor of a skyscraper falling apart, I think that is an extremely evocative way of capturing our dependence, if you may, in a very foundational way. Deen, thank you again so much for taking time to talk with us today. I think that your work is extremely important in part because it challenges some conventional wisdoms and does so in a way that really is provocative and advances our understanding and sensibility about many issues related to web science. And I look forward to seeing continued research and insights from you in the years and decades ahead so thank you again.
Deen Freelon: It was really great to be here.