“The state of our union is . . . dumber” — or is it? A 2013 Guardian report found that presidential addresses were using progressively simpler language. During the 2016 campaign, various readability analyses claimed that Donald Trump’s speeches were given at the level of a fifth– or fourth– or even third-grader.
We decided to analyze a wider range of political texts, including Supreme Court decisions and executive orders, to see how concerned we should be about dumbing down in U.S. politics.
Here’s what we found. Yes, the State of the Union address is arguably getting less sophisticated, but this trend was not visible in the broader set of political writings. And we figured out why the State of the Union appears less cerebral. In the past, presidents delivered this as a written document, while now they primarily address Congress by giving a speech. This itself is probably not a cause for deep anxiety about the intellectual status of U.S. politics.
Executive orders became harder to read
There was an unexpected finding. We noticed a far more striking trend in another series of political texts: Presidential executive orders have become way harder to read. Executive orders — the president’s declarations that set policy for federal agencies — have recently attracted a lot of attention, particularly regarding the frequency with which presidents have used them.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/01/30/most-of-trumps-executive-orders-arent-actually-executive-orders-heres-why-that-matters/?utm_term=.3c292890b58a”]Most of Trump’s executive orders aren’t actually executive orders. Here’s why that matters.[/interstitial_link]
We used the Flesch Reading Ease (FRE) score to analyze these political texts. (See Figure 1.) We agree with other reports that this measure is imperfect, but it remains the most prominent readability measure — and the basis of dumbing-down claims. A FRE score simply measures the average number of words per sentence and the average number of syllables per word.
Figure 1 — Trends in readability of political texts. This figure traces trends in the ease of reading three types of political texts: State of the Union addresses, executive orders and Supreme Court decisions. Data: Justia; the American Presidency Project. Figure: Kevin Munger.
Beginning in the early 2000s, the FREs for executive orders take a nosedive. The FRE score was designed to model what percentage of grade-school students were able to read a text and correctly answer comprehension questions about it. For reference, the median article in the student newspaper at Harvard University has a FRE score of 54, which corresponds to an 11th-grade reading level.
A score of 100 means an average fifth-grader would have no trouble comprehending the text. By 2009, the average executive order actually had a negative score.
To dig further, we decomposed the FRE score into its two component parts: the mean number of syllables per word and the mean number of words per sentence, as shown in Figure 2. Shorter words and shorter sentences correspond to higher FRE scores and more readable texts.
Figure 2 — Trends in mean number of syllables per word in political texts. This figure traces trends in the mean number of syllables per word in three types of political texts: State of the Union addresses, executive orders and Supreme Court decisions. Data: Justia; the American Presidency Project. Figure: Kevin Munger.
Executive orders have indeed tended to use longer words over time, but the upward trend has been fairly constant since the early 20th century. In contrast, the State of the Union has tended to use shorter words since about the 1960s.
Here’s why — Figure 3 shows what we called “The Great Sentence Shortening” of the 20th century.
Figure 3 — Trends in mean sentence length of political texts. This figure traces trends in the mean sentence length of three types of political texts: State of the Union Addresses, executive orders and Supreme Court decisions. Data: Justia; the American Presidency Project. Figure: Kevin Munger.
In general, we noted that executive orders, State of the Union addresses and texts of Supreme Court decisions all exhibited this trend. Again, we don’t think that the fact that politicians tend to use fewer 40-word sentences is evidence of a crisis of democracy.
Executive orders, however, are now far more complex
The big new trend is more complex executive orders, starting in 2002. The average sentence length skyrockets, and the explanation is simple: Executive orders now tend to be far grander in scope. Like laws passed by Congress, they contain a justification for their existence and detailed and extensive information about implementation.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/02/23/yes-our-flawed-democracy-just-got-downgraded-heres-why/?utm_term=.70089cbd1f3b”]Yes, our ‘flawed’ democracy just got downgraded. Here’s why.[/interstitial_link]
To see this in action, check out Ronald Reagan’s first five executive orders. They contain plenty of minute details, but in keeping with the fact that Reagan was a true conservative, they mostly modify or repeal existing executive orders.
Contrast that with President Trump’s Executive Order 13780 — Protecting the Nation From Foreign Terrorist Entry Into the United States, the “updated travel ban.” Massive, paragraph-length sentences provide arguments and justifications for the order, and there are sections with many subclauses that look like laws. There are nine sentence-length clauses separated by semicolons, for instance.
[interstitial_link url=”https://www.washingtonpost.com/news/monkey-cage/wp/2017/04/13/what-trumps-tweets-tell-us-about-his-tv-viewing-and-his-thinking/?utm_term=.02b4abc0aa2b”]What Trump’s tweets tell us about his TV viewing — and his thinking[/interstitial_link]
Our main finding is that the State of the Union addresses may in fact be getting easier to understand, in large part because of the difference between written and spoken language.
This contrasts with the transformation of executive orders. Some political observers are concerned about the expanded use of these orders to enact policy without congressional approval. The rapid decrease in the FRE scores of executive orders after 9/11 provides descriptive evidence of their changing role in policymaking. Far from getting dumbed down, the presidency has cleverly expanded its authority.
Ken Benoit is professor of quantitative social research methods and head of the Department of Methodology at the London School of Economics and Political Science.
Kevin Munger is a PhD candidate in politics at New York University and a PhD research associate in the NYU Social Media and Political Participation (SMaPP) lab.
Arthur Spirling is an associate professor of politics and data science at New York University and the (interim) deputy director of the Center for Data Science.