Soldiers parade during official celebrations of the 60th anniversary of Vietnam’s Dien Bien Phu victory over France at a stadium in the Northwestern town of Dien Bien Phu on May 7. (Hoang Dinh Nam/AFP/Getty Images)
At the dawn of the 21st century, the Vietnamese were the most authoritarian people in the world, according to the World Values Survey. Ninety-nine percent of the survey’s respondents in Vietnam in 2001 said they favored military rule. So did 96 percent of the survey’s sample in Indonesia and five sixths of respondents in Albania in 1998 and Iran in 2000. In the next round of the survey only a few years later, however, only a third of the samples in Vietnam and Iran supported military rule, and only one eighth in Albania. Of the four most authoritarian countries from the previous round, only Indonesia remained consistent at 95 percent.
What happened to cause such dramatic shifts?
Some scholars looked for causal explanations rooted in economic, social or political change. Ronald Inglehart, a political scientist at the University of Michigan and director of the World Values Survey for more than a quarter century, and Christian Welzel, director of research at the World Values Survey Association, argue that new values emerge as a result of socioeconomic development. “If younger generations are socialized under significantly different conditions from those that shaped earlier generations, the values of the entire society will gradually change through intergenerational replacement.” But there was no dramatic leap in socioeconomic development in the early 20th century that would account for a rapid change in attitudes in Albania, Iran and Vietnam, and no mass die-offs to accelerate intergenerational replacement.
What happened is that the World Values Survey switched translations.
The WVS is the world’s greatest cross-national social science collaboration. It has polled more than a quarter million people in 87 countries over the past generation, working with hundreds of scholars to create a historic data-bank of human attitudes around the world. The project’s commitment to transparency is unmatched – the datasets are available on its Web site for download and online analysis, as are most of the local-language survey instruments. In such a massive undertaking, mistakes are inevitable.
That’s the most reasonable explanation for the sudden shift in Vietnamese views on the military. In 2001, the survey in Vietnam translated the English question about “Having the army rule” as “Vai trò của quân đội,” meaning “The role of the military,” I am told by a Vietnam expert. So respondents who answered “very good” or “good” may have been applauding the role of the military, not the rule of the military. I have not been able to locate the 2006 version used in Vietnam, but it is probably a more accurate translation.
The Vietnamese translation probably changed also for “Having a strong leader who does not have to bother with parliament and elections,” which was asked in 2001 as “Vai trò của một nhà lãnh đao” – the role of a leader, with no mention of parliament or elections – and dropped from 99 percent in favor to 9 percent in 2006. “Having experts, not government, make decisions according to what they think is best for the country,” which was translated as “Vai trò của các chuyên gia trong việc ra quyết định của chính phủ” – the role of experts in decision making by government – fell from 98 to 33 percent. The associated question about democracy – “Vai trò của một hệ thộ́ng chính trị dân chủ,” or the role of a democratic political system – remained steady above 90 percent.
It isn’t just Vietnamese. In Iran, the 2000 Persian-language survey asked about حكومت مقتدر (hokumat-e moqtadar) – strong government (sometimes used pejoratively to mean dictatorship). The 2007 version asked about حكومت نظامى (hokumat-e nezami) – military government. In 1998, the Albanian-language survey asked about “Të kesh rregulla të ushtrisë” – having military rules, not military rule. In 2002, the question was changed to “Të kesh regjim ushtarak – having a military regime. The 2001 survey in Indonesia also asked about military rules, not military rule: “Memiliki peraturan yang jelas tentang angkatan bersenjata” – having clear rules concerning the armed forces. In 2006, the question removed the term “yang jelas” (clear). Indonesian respondents continued to approve of military rules almost unanimously.
Where translations are reliable, cross-national surveys have real value. The country with a legitimate claim to being most favorable toward military rule is Jordan, which ranked fifth-highest in 2001 with 59 percent expressing support for حكم الجيش (hukm al-jaysh), a straightforward translation of army rule. In 2007, that percentage rose to 74 percent with the same question wording.
What about Pakistan, which registered 40 percent approval for military rule in 1997 and only 4 percent in 2001? The local-language questionnaires do not appear to be available online, unfortunately, so we don’t know what to make of this: Did the question wording change, or did Pakistanis change their views in the wake of the 1999 military coup?
I’ve focused on these countries because I’m interested in political attitudes and was motivated to look into countries with exceptionally high rates of support for military rule or exceptionally large shifts in their rate of support. But this is not the only question with translation issues. Trolling for other attitudes that shifted improbably quickly, I came across Peru’s views on unselfishness as an important quality for children to learn at home. In 1996, only 16 percent of respondents in Peru selected this item, when unselfishness was translated as “pensar en los otros, altruismo.” Five years later, 50 percent agreed. The translation had changed to “solidaridad.”
Another example: Political science graduate student Jon Mellon examined nine translations of the word “ideas” in the WVS item, “Progress toward a society in which ideas count more than money.” Two of the nine translations (Polish and Russian) use the word for ideals (ideały, идеалы) rather than ideas, and another (Chinese) uses a word whose primary English usage is “spirit” (精神, jīngshén). Using large-scale internet corpora in each language, Mellon shows that these terms have associations that are distinct from the associations that the word “ideas” has in English.
Perhaps spirit or ideals reflect the concept of a society-not-obsessed-with-money in China, Poland or Russia better than a direct translation of the English word “ideas.” I leave this to scholars who work on issues of equivalency in multilingual surveys, in the tradition of the late Janet Harkness of Gallup and her colleagues at the International Workshop on Comparative Survey Design and Implementation. The latest volume of the Annual Review of Sociology presents a paper by Eldad Davidov and colleagues proposing sophisticated statistical tests to identify non-equivalent variables.
Without minimizing the importance of equivalent translations, I am more concerned about identifying outright mistakes, not just in the WVS but in all cross-national datasets, including projects that do not make the local-language instruments widely available, such as the Pew Global Attitudes Project and the Gallup World Poll.
Will identifying mistaken translations undermine research findings that are based on these surveys? Perhaps, but it is also possible that removing the “noise” of bad data could reinforce existing findings. Inglehart and Welzel, for example, noted that Vietnam’s seemingly extreme support for military rule in 2001 weakened their argument about the importance of self-expression values, rather than pro-democracy values, on political institutions. They reported a weak statistical correlation with pro-democracy values “that only exists because of one single leverage case: Vietnam. Without Vietnam, there would be no relationship at all.” In a footnote, Inglehart and Welzel mulled over the Vietnamese outlier: “The very low percentage of ‘solid democrats’ in the case of Vietnam reflects a very high percentage of respondents expressing support for the army rule. In a country in which the army is a symbol of national liberation, these figures require a different interpretation. However, we display data as they are, not eliminating cases that do not fit into the pattern.” Being wrong about the meaning of the Vietnamese survey helps make them right about the rest of their analysis.
Charles Kurzman is a professor of sociology at the University of North Carolina at Chapel Hill and co-director of the Carolina Center for the Study of the Middle East and Muslim Civilizations.
