To tell it from opinion surveys, President Trump is failing to make a good impression on the American people in his first month in office. His approval ratings (he’s underwater with 45 percent approval and 49 percent disapproval, according to the latest RealClearPolitics average) are jaw-droppingly lower than those for any other president since polling began.
But can we take polls at face value in the Trump era? After all, election polls in most states failed dismally in November, substantially underestimating how well Trump performed on Election Day. What if this means that current surveys are similarly miscalculating support for Trump and his policies?
That’s the suggestion made by none other than the president himself Monday, when he cast aspersion on recent negative poll findings by evoking the failure of polling in the general election.
Any negative polls are fake news, just like the CNN, ABC, NBC polls in the election. Sorry, people want border security and extreme vetting.
— Donald J. Trump (@realDonaldTrump) February 6, 2017
Why might opinion surveys be selling the president’s popularity short? Trump has faced unprecedented levels of scorn from society’s tastemakers — whether they be newspaper editorial boards, celebrities or his fellow Republican elected officials. The hypothesis is that “shy Trump voters” react to this stigmatization by keeping their support for the president to themselves when being interviewed for an opinion survey.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/02/08/3-lessons-from-republicans-failed-attempt-to-silence-elizabeth-warren/?utm_term=.7385d0a28c4c”]3 lessons from Republicans’ attempt to silence Elizabeth Warren[/interstitial_link]
This in turn would cause polls to be plagued by what’s called “social desirability bias,” depressing the levels of approval they record for Trump and his policies. In experiments in which respondents are randomly assigned to different survey modes, survey researchers have found social desirability bias affecting surveys on sensitive topics such as respondents’ number of sexual partners, their sexual orientation and even their bad grades in college. Respondents are more likely to disclose potentially stigmatizing information when surveys are not conducted by human interviewers.
Here’s how I investigated whether the “shy” Trump supporter actually exists
A good way to see if the “shy” Trump supporter actually exists, then, is to compare responses about Trump from polls conducted with live telephone interviews to those done in a more anonymous fashion via the Internet or automated phone. We don’t yet have enough polls on Trump’s approval ratings to make this comparison.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/02/07/unlike-all-previous-u-s-presidents-trump-almost-never-mentions-democratic-ideals/?utm_term=.1343415926a6″]Unlike all previous U.S. presidents, Trump almost never mentions democratic ideals[/interstitial_link]
But we can do the next best thing: HuffPost Pollster collected a total of 240 national election polls from the 2016 presidential campaign conducted from July 1 to Election Day. Somewhat more than half of these surveys were administered via the Internet or automated phone and the remainder were done over the phone by live interviewers. If social desirability bias were present, we would expect support for Trump in the Internet surveys to be consistently higher than in live phone polls.
The graph above plots these election surveys by date, displaying Democratic nominee Hillary Clinton’s lead over Trump recorded by each survey in percentage points, with negative values assigned to polls that had Trump ahead. Surveys conducted over the Internet are plotted in green; live-phone polls are in purple. Trends in both sets of polls are displayed with smoothed lines.
Until October, live-phone polls gave Clinton a slightly bigger lead than Internet surveys, although the difference was never more than about two percentage points. The pattern reversed itself in the final weeks of the campaign, with Clinton performing worse on live-phone polls than Internet polls.
In the end, these live-phone polls were better predictors of the actual result (Clinton’s popular-vote margin of 2.1 percentage points) than Internet surveys.
So where was the “shy Trump voter”?
All of this casts a fair amount of doubt on the “shy Trump voter” hypothesis. Live-phone interviews, in which voters would presumably be most reluctant to disclose support for Trump, never gave Clinton much of a substantial boost compared to Internet surveys. And when things really counted — in the surveys conducted just before Election Day — no evidence of social desirability bias was present.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2016/10/25/shy-trump-supporters-this-new-evidence-says-no/?utm_term=.d185fa9304c9″]Shy Trump supporters? This new evidence says no[/interstitial_link]
This is an imperfect test, particularly because of the substantial differences in how live-phone and Internet surveys obtain their samples. But it fails to corroborate the claim that Trump supporters were reluctant to disclose their intention to vote for him.
All of this suggests that the record low level of approval for the president we’re seeing in current polls is very likely real. That’s problematic for Trump’s attempts to claim popularity for himself and his policies.
Patrick J. Egan is an associate professor of politics and public policy at New York University. He specializes in public opinion, political institutions and their relationship in American politics. Find him on Twitter @Patrick_J_Egan.