Although everyone’s first thoughts today are undoubtedly on Paris and France, the 2016 U.S. presidential campaign continues nevertheless. What will we learn from tonight’s Democratic debate?
Although focus groups conducted immediately after debates can offer a quick glimpse into the minds of voters, the small and unrepresentative samples can encourage misleading conclusions, such as the notion that Bernie Sanders “won” the last Democratic debate. Alternatively, using social media activity in the form of tweets about the debate can provide a more in-depth look at how audiences (and different segments of the audience) are reacting to the candidates onstage. While tweets are also not fully representative of the population, they do give us a window into the real-time thoughts and feelings of many who are actively following American politics. As we wait for tonight’s proceedings to begin, here are a few takeaways from the last Democratic debate, on Oct. 13.
Recall that according to pundits, Hillary Rodham Clinton was a huge “winner.” But according to various reports of social media mentions, Sanders was the “winner”: More people mentioned him on social media, and there were more Google searches for him after the debate. However, according to a CNN/ORC poll a few days later that asked people “who won,” Clinton was the winner. Without getting into the big picture question of what it really means to “win” the debate, we instead focus on what we can learn about: a) which of the issues being discussed the public thought were important; b) what the public thought about the candidates; and c) whether opinion about the candidates differed based on the ideology of the voters watching the candidates.
To examine these questions, the New York UniversitySocial Media and Political Participation (SMaPP) lab gathered all tweets during the debate that included the hashtag #demdebate (with the unfortunate exception of tweets from approximately 9:00 – 9:10 PM, which we were unable to collect). To make our analysis more meaningful, we consider two groupings of obvious relevance. First, since Clinton and Sanders have clearly staked out different positions on the ideological spectrum, we divide users into liberal Democrats, moderate Democrats, and Republicans (see the explanation for how we do this at the end of this post). We also consider gender, as some analysts expect that Clinton will run exceptionally well among women. Participants are categorized based on a dictionary of names.
First, what issues did people care about? One way to examine this with tweets is to look at a word cloud made up of the most commonly used terms. In Figure 1 we show word clouds based on the respondents identified as either left-leaning Democrats, moderate Democrats, or Republicans. We can see from the word clouds of the top terms used in tweets that most tweets referred to specific individuals (i.e., candidates in the debate). This may suggest that tweeters were not primarily discussing the issues raised in the debates.
Word clouds of Tweets from October 13 Democratic Debate (Data: NYU Social Media and Political Participation lab; Figure: Andy Guess)
To examine this further, we identified several issues raised in the debate that we might expect people to tweet about. First, the promise of free college tuition came up. Thus we considered tweets in our sample containing either “college” or “tuition.” Second, there were several references to banks and Glass-Steagall in the debate. We thus considered tweets containing any of: “steagall,” “bank,” or “wall street.” We also looked at tweets mentioning the Iraq war, as Clinton’s vote on it was an Achilles’ heel for her in the 2008 primaries and may still resonate with more liberal Democratic voters in 2016. And finally as the debate opened with a discussion of guns and the NRA, a subject several candidates returned to later, we examined tweets containing the terms “gun” or “NRA.”
We can look at the proportion of tweets about some of these issues to demonstrate again that issues were a small part of the Twitter stream about the debate. Looking at the table below, only 2.3% of tweets by liberal Democrats were about Clinton’s email (“email”, “e-mail”), and only 1% were about college. We note that this could be a distressing number for Clinton, to have twice as many tweets from this group be about email as about college — but of course they could just be tweeting about what was probably the most reported one-liner of the debate, Sanders’ comment about Clinton’s emails. Not surprisingly, Republicans were more likely to tweet about email than Democrats. But Republicans were also more likely to tweet about college than were Democrats, though we do not know what they were saying about it.
Data: NYU Social Media and Political Participation lab; Table: Andy Guess
If we consider mentions of candidates, we see that women were more likely to tweet about Sanders than to tweet about Clinton. And this does not seem to represent the secular tendency to tweet more about Sanders than Clinton, because we see that men are equally likely to mention Sanders or Clinton. And when we look at this by party (not reported in the Table) it is even stranger since Republican women are exceptionally likely to mention Clinton. Sanders claimed several times during the debate that when turnout is high, it is good for Democrats. One of the implicit assumptions of Clinton’s candidacy that her electability rests on is the notion that women will turn out for her. But if they will not even tweet about her, are they likely to turn out for her?
Data: NYU Social Media and Political Participation lab; Table: Andy Guess
We also examined mentions of each candidate by each of the ideological groups we established. We show the proportion of mentions of each candidate attributable to each of our groups throughout the entire debate. While liberal and moderate Democrats tweeted about Clinton and Sanders at very close rates, we can see that Republicans were much more likely to tweet about Clinton than about Sanders.
Mentions of Candidates by Ideology on Twitter (Data: NYU Social Media and Political Participation lab; Figure: Andy Guess)
By graphing mentions of issues over time by different groups we can see if the different segments of the electorate watching the debate responded more to particular issues as they were mentioned. We can see for instance that liberal Democrats were more likely than either of the other two groups to mention banks or Wall Street when the issue was raised during the debate.
Mentions of Issue by Ideology on Twitter (Data: NYU Social Media and Political Participation lab; Figure: Andy Guess)
And we can see that liberal Democrats were also more likely to respond to mentions of Iraq than were other groups.
Reaction to Iraq by Ideology on Twitter (Data: NYU Social Media and Political Participation lab; Figure: Andy Guess)
Thus we can see that people on social media are paying attention to issues: As they come up in debate, the public responds with more mentions. And the mentions can vary systematically by the ideology of the debate viewers.
But what about these mentions? We can go one step further and see how the sentiment – a broad gauge of positive or negative valence – of tweets varied by ideology and candidate. We compute sentiment by comparing words in tweets to those in commonly used dictionaries for this purpose. The last figure below plots the average sentiment (where more positive numbers mean more positive sentiment) toward Clinton in tweets about the debate in real time. Interestingly, while the most liberal Democrats tended to swing wildly from negative to positive, the highs were highest among this group (especially around the email exchange) than moderate Democrats. Even Republicans apparently found some positive things to tweet about Clinton – contextualizing the earlier finding about Republican mentions — although the bulk of tweets about her were negative.
Sentiment by Ideology on Twitter (Data: NYU Social Media and Political Participation lab; Figure: Andy Guess)
In short, beware of the conventional wisdom that forms immediately after the debate ends. Many commentators believed (and continue to believe) that Clinton has an inherent advantage with women and that Sanders has the more liberal wing of the Democratic Party locked up. At least as far as social media activity among the most enthusiastic political audience goes, these beliefs may have to be revised.
[Lessons from Twitter from the 3rd Republican Debate.]
Details of the Analysis
Twitter users don’t tell us which political party they support or directly reveal their ideological leanings. But thanks to Twitter’s network structure, we can infer these attributes using a scaling method developed by Pablo Barberá. At the most basic level, people on Twitter who are interested in politics follow relevant accounts: news sources, journalists, parties, and politicians themselves (in Congress and elsewhere). Thanks to the clear partisan affiliation of many of these sources, it is more likely that someone who follows mainly Republican-leaning media outlets and elected officials will be conservative than someone who follows mainly Democratic-leaning outlets and officials.
Our analyses of 426,717 tweets during the debate focus mainly on the subset of users in our sample for whom we have ideological scores — that is, those who followed at least three political or media accounts on Twitter at a time when we scanned political accounts for followers. We are then left with a set of 241,820 tweets, which we label as being posted by a “moderate Republican” (someone with an estimated ideological score to the right of 0 but left of 1), a “conservative Republican” (a score to the right of 1) or a “Democrat” (left of 0). These designations potentially include those who lean to one party or another.
Andy Guess is a postdoctoral researcher at the NYU Social Media and Political Participation Laboratory. Jonathan Nagler and Joshua Tucker are co-Directors of the lab and Professors of Politics at New York University.