Wednesday, October 9, 2019

The Star Gets It Right About Pima County Standardized Test Scores

Posted By on Wed, Oct 9, 2019 at 2:41 PM


Since my last post was a rant about some misleading headlines in Tuesday's Star, let me begin this post by saying, the headline for this Star article is right on the money.
Pima County students lag rest of Arizona in passing standardized math tests
Here's what I like about it.

First, it focuses on Pima County, not TUSD. Star headlines and stories about standardized test scores often focus on TUSD's scores, which are always lower than most other districts in the area. Knocking TUSD sells papers, which is the only reason I can figure that the Star is so fond of going for the district's jugular. But not this time.

Second, by saying the Pima County scores are lower than the state average in math, the headline implies that county schools equalled the rest of the state in English, which you learn in the story is true. I even like the word "lag," which suggests that the math scores trail the rest of the state, but not by much. That's true as well. Statewide, 42 percent of students passed the math test. In Pima County, it was 40 percent.

A good headline is a good headline is a good headline. Let me raise my glass and toast its creators.

The story is even better than the headline. When the Star's annual standardized test stories come out, I often find myself shaking my head and muttering, "Comparing district test scores means nothing unless you factor in family incomes." Then I sit down and write a post picking the article apart.

But as I read this article, I found myself nodding "Yes." After the first three paragraphs summarizing the county scores, the reporter launches into a discussion of why the districts' scores break down the way they do.

Like the state as a whole, race and socioeconomic status factored heavily into test scores — a consideration that many say is a problem with AzMERIT and other standardized tests.
Yes, that's absolutely right. The next few paragraphs are an intelligent discussion of how family income correlates with test scores, going so far as to cite a recent academic study and a Census Bureau report. That. Is. How. It's. Done.

When the story gets down to individual district scores, it relates them back to family income. The three districts with the highest scores, Catalina Foothills, Vail Unified and Tanque Verde, have the county's lowest poverty rates. The districts with the lowest scores, Sunnyside and TUSD, have the county's highest poverty rates. Which makes it clear, you can't say one district is doing a better job than another by looking at standardized test scores alone.

The article cites two outliers to the correlation between scores and poverty rates. Flowing Wells' passing rates are higher and Marana's are lower than one would expect given their poverty rates.

Why do those two districts defy the correlation between test scores and family income? The article doesn't venture an explanation, and I don't have an answer either. But let me consider a few possibilities.

It could be as simple as, Flowing Wells does a better job with its students than other districts and Marana needs to clean up its educational act.

But a more granular analysis of the data might show the two districts' scores aren't the outliers they appear to be. Looking beyond the passing rates and analyzing the way students' scores spread across the entire continuum from lowest to highest might indicate that Flowing Wells' scores aren't as high and Marana's aren't as low as that one data point, the passing rates, indicates. Likewise, looking at detailed socioeconomic data from the two districts, not simply the poverty rate, might indicate that Flowing Wells' family incomes are higher and Marana's are lower than the number of families above and below the poverty line indicates.

I have no idea what a more sophisticated analysis of the two districts would reveal, and I'm not likely to find out any time soon. A task like that is way above my pay grade and far beyond the scope of the Star article. It would probably take an academic crunching the numbers to reach any significant conclusions.

There. That was fun. It's a pleasure writing about a good article on local education rather than picking apart one that doesn't do the job.

Tags: , , ,