Many viewers of last week’s Republican primary debate watched a live broadcast annotated by the instant reactions, snap analyses and hot takes on Twitter.
How closely did real-time reactions correspond to the emerging conventional wisdom on the “winners” and “losers” of the debate? Judging from the media coverage in the days since, it would seem that people quickly coalesced around a view that Marco Rubio’s performance was the most impressive.
But a closer look at publicly available Twitter data shows that while that view gained traction over the course of the night, post-debate impressions were likely also shaped by the emerging consensus of media analysts the morning after.
The distinction between the spontaneous reactions of debate watchers and the narrative imposed by political analysts is important. Real-time Twitter data allows us to potentially disentangle these effects, and to see how different groups of people responded to the same events on screen. Using tweets about the debate that we collected at the New York University Social Media and Political Participation (SMaPP) lab and merged with our unique data on Twitter users’ ideology, we took a closer look at how people responded to the debate — both as it was happening and after it ended.
[The full-length SMaPP lab Data Report on the October 28, 2015 Republican debate.]
From 221,718 tweets posted during the debate, we can see that one piece of the conventional wisdom was definitely true: Republican-leaning viewers (but not Democratic-leaning ones) tweeted a great deal about the moderators of the debate and the questions they asked the candidates. This becomes clear when we look at word clouds of tweet text, with common words, such as the candidates’ names, removed.
Words used on Twitter during October 28 GOP presidential debate by estimated partisanship (Red = Republican; Blue = Democrat; Data: NYU Social Media and Political Participation lab. Image: Andy Guess)
Republican-leaning Twitter users (in red) were much more likely to discuss the “moderators,” the “media,” “CNBC,” which hosted the debate, and other related words (“mainstream,” “answer”) than Democratic ones. Here is a clear case in which a common criticism about the debate was reflected in the postings of thousands of individual debate watchers.
What about the candidates themselves? Many analysts paid attention to the dynamic between Jeb Bush and Marco Rubio, fellow Floridians who have been competing furiously to be the establishment favorite. First, we can look at the proportion of tweets in our sample that mentioned either candidate as the debate progressed.
Proportion of Tweets mentioning Jeb Bush and Marco Rubio by minute during October 28, 2015 Republican debate (Data: NYU Social Media and Political Participation lab; image: Andy Guess)
We break tweeters down into three groups: “Democrats,” or left-leaning users tweeting about the debate; “Moderate Republicans,” who we estimated to be right of center; and “Conservative Republicans,” or users who we estimated to be to the right of the moderates (see below for details on the coding of ideology). In general, the ideological groups respond in parallel to events happening on screen. From 8:36 p.m. to about 8:40 p.m. ET, Bush and Rubio sparred about the latter’s job representing his state in the Senate. The spike corresponds to Jeb’s suggestion that Rubio resign. The later spike at about 9:26 p.m. matches Rubio’s statement that the mainstream media is the “ultimate super-PAC” for Hillary Clinton — a line that seemed to generate somewhat more activity among conservative Twitter users.
Counting tweets can tell us what people were talking about and by whom, but not how they tweeted about it. We can take a step further by using a simple form of “sentiment analysis,” in which we match the text of tweets to standard dictionaries of words labeled as signifying either positive or negative feeling. Below, we can see the average sentiment towards Bush and Rubio as the debate went on.
Average positive vs. negative sentiment towards Jeb Bush and Marco Rubio across conservatives by minute during the October 28 GOP presidential debate (Data: NYU Social Media and Political Participation lab; image: Andy Guess)
Evidently, by the time the debate ended (at about 10:30 p.m.), there was a measurable gap in sentiment among both moderate and more right-leaning Republicans: Rubio was tweeted about in a more positive way than Bush, whose average sentiment was roughly neutral. Rubio takes off in an especially pronounced way among the moderates.
Did these instant reactions carry over into the morning after? The graph below plots a rolling average of sentiment beginning at midnight and into the next morning. This is a slightly more difficult exercise because the volume of tweets dropped off substantially after the debate ended.
Still, we can see that the conventional wisdom about Rubio’s performance as compared to Bush did not immediately register: Sentiment about the two on Twitter on the morning after the debate was roughly equal. As afternoon approached, however, Rubio began to take off again, especially among the more conservative users on Twitter.
Positive and negative sentiment towards Bush and Rubio among Republican-leaning Twitter users on the day after the Oct. 28th GOPpresidential debate (Data: NYU Social Media and Political Participation lab; image: Andy Guess)
It seems plausible that Rubio’s second wind among the Twitterati was driven by post-debate analysis, in which journalists and other pundits gave the Rubio narrative an additional boost. Here we have a case where the consensus eventually — but not immediately — matched what contemporaneous viewers seemed to think about the candidates’ performance.
Beyond simply counting the number of tweets about each candidate, this kind of analysis allows us to use publicly available data to explore what different subgroups on Twitter are saying about various political topics — and how they felt about them. Combining simple sentiment analysis with minute-by-minute social media data lets us see how narratives are shaped both by the people who experience an event as it happens and those who package it afterwards.
[The third Republican debate: Before and after]
Twitter users don’t tell us which political party they support or directly reveal their ideological leanings. But thanks to Twitter’s network structure, we can infer these attributes using a scaling method developed by NYU Moore-Sloan Data Science Environment post-doctoral fellow Pablo Barberá. At the most basic level, people on Twitter who are interested in politics follow relevant accounts: news sources, journalists, parties and politicians themselves (in Congress and elsewhere). Thanks to the clear partisan affiliation of many of these sources, it is more likely that someone who follows mainly Republican-leaning media outlets and elected officials will be conservative than someone who follows mainly Democratic-leaning outlets and officials.
Our analyses of 404,750 tweets during the debate focuses mainly on the subset of users in our sample for whom we have ideological scores — that is, those who followed at least 3 political or media accounts on Twitter at a time when we scanned political accounts for followers. We are then left with a set of 221,718 tweets, which we label as being posted by a “moderate Republican” (someone with an estimated ideological score to the right of 0 but left of 1), a “conservative Republican” (a score to the right of 1) or a “Democrat” (left of 0). These designations potentially include those who lean to one party or another.
Andy Guess is a Postdoctoral Fellow in Data Science at the NYU Social Media and Political Participation (SMaPP) Lab. Jonathan Nagler is a Professor of Politics and a Co-Director of the NYU Social Media and Political Participation (SMaPP) Lab.
