Mark Palko comments on the (presumably) well-intentioned but silly Jumpstart test of financial literacy, which was given to 7000 high school seniors Given that, as we heard a few years back, most high school seniors can’t locate Miami on a map of the U.S., you won’t be surprised to hear that they flubbed item after item on this quiz.
But, as Palko points out, the concept is better than the execution:
With the complex, unstable economy, the shift away from traditional pensions and the constant flood of new financial products, financial literacy might be more important now than it has been for decades. You could even make the case for financial illiteracy being a major cause of the economic crisis.
But if the supporters of financial literacy need a good measure of how well we’re doing, they’ll need to find a better instrument than the Jump$tart survey.
The ‘test’ part of the survey consists of thirty-one questions. That’s not very long but that many questions should be sufficient for a tightly focused, well-structured test. Unfortunately the focus of the Jump$tart survey is ridiculously broad, ranging from investments to retirement to credit cards to debt counseling to auto insurance to macroeconomics to really questionable career advice.
Even within the categories the questions have a random, pulled-from-a-hat quality with no apparent effort to prioritize. There are multiple references to credit histories but no mention of credit scores. None of the few questions on credit cards mention teasers or other cases where rates can change on a credit card. There are no questions that refer to charts or tables though the ability to read both is an essential part of financial literacy.
On the individual question level the situation is no better. Most of the questions are either badly written, trivial/irrelevant, open to interpretation, guessable or factually challenged. The test resembles nothing so much as the homework paper a student teacher might turn in when asked to come up with 31 questions on financial literacy.
One of the more ridiculous items on the test is this one, which looks like an item from the final exam in Wally Cleaver’s high-school civics class:
18. Don and Bill work together in the finance department of the same company and earn the same pay. Bill spends his free time taking work-related classes to improve his computer skills; while Don spends his free time socializing with friends and working out at a fitness center. After five years, what is likely to be true?
a.) Don will make more because he is more social.
b.) Don will make more because Bill is likely to be laid off.
c.) Bill will make more money because he is more valuable to his company.
d.) Don and Bill will continue to make the same money.
You’re supposed to pick c.
They get extra credit for hyerprecision in reporting the percentage of respondents who picked each item. (In the above item, it’s 11.5%, 9.8%, 67,9%, and 10.8%.) I’m speaking ironically here. One of the basic principles of statistical literacy is to (almost) always round percentages to the nearest percent. Then again, if political scientists and economists can’t get this right when presenting their regressions, how can we expect anybody else to know better?
What this test really reminds me of is the written test they give you when you get your driver’s license: a mix of obvious questions, trivia, and weird things that look like trick questions, along with some items that actually might be useful.
But . . . we just graded our Intro to Statistics final exams today, and I have to say we’re not much better. Some of our questions seem pretty pointless too. The difference is that we’re using our exam to grade the students, and I have every reason to believe that higher scores on the exam correspond to better understanding of statistics. In contrast, the questions on the quiz above are for . . . what, exactly? I’m not sure.
P.S. My favorite item on the survey is this one:
12. Barbara has just applied for a credit card. She is an 18-year-old high school graduate with few valuable possessions and no credit history. If Barbara is granted a credit card, which of the following is the most likely way that the credit card company will reduce ITS risk?
a.) It will make Barbara’s parents pledge their home to repay Karen’s credit card debt.
b.) It will require Barbara to have both parents co-sign for the card.
c.) It will charge Barbara twice the finance charge rate it charges older cardholders.
d.) It will start Barbara out with a small line of credit to see how she handles the account.
What’s wrong with this item? To start with, I have no idea who “Karen” is (in choice “a”). But there’s one more thing. Check this out:
An 18-year-old named Barbara, huh? What year did they write this question??