Kentucky's NAEP test fiasco
From Kentucky Citizen Digest, July, 1999

The March 4 release of the National Assessment of Education Progress report (NAEP) by the U.S. Department of Education was billed by reform loyalists as a vindication of the 1990 Kentucky Education Reform Act (KERA) — “deliverance at last” from all the criticisms it had received over the last nine years.But a May 14 NAEP report based on a fact-finding study of the test results told a different story.

The initial accolades ignored an important truth that was contained in the first review: the fact that Kentucky eliminated 2 1-2 times the number of “students with disabilities” in the 1998 testing versus those eliminated in 1994.In other words, they cut out far more students who likely would have lowered the test scores.

Factoring out a small number of students isn’t the problem. When evaluating an educational system it is appropriate to drop those few students who are struggling most with their educational progress in order to avoid biasing the overall evaluation.In Kentucky, for instance, with our self-developed KIRIS test, we are inclusive, involving almost all students, even to the extent of having teachers read the questions on the test to students with problems.Assisting students on test questions is not accepted by NAEP, so Kentucky teachers exclude students who have an “Individual Education Plan” (or who are “SD” — “student with disabilities”).

Here is the problem: In 1994 Kentucky dropped only 4 percent of its students as SD; in 1998 it dropped a full 10 percent — 250 percent of its earlier elimination group.So the Kentucky Department of Education boasted about a 1 percent rise in reading scores of fourth-graders and claims vindication of KERA, but, on the other hand, didn’t mention the much-more-than-double exclusion rate.

Troubling facts in the March 4 report showed that only two states had higher rates of exclusion than Kentucky — Louisiana at 13 percent and South Carolina at 12 percent.And, of the six states that increased their exclusion rates to double digit percentages, all raised their state rankings and their raw scores.

So while Kentucky Department of Education officials continued to play down the possibility of any effect of the increased exclusion rate, the NAEP officials released a stunning and contrasting report May 14.I quote:

“It is of particular interest to consider the four states that showed large, and statistically significant, score gains and large increases in exclusion rates from 1994-1998 (Conn., Ky., La., and S.C.).When the adjusted means were used, the gains appeared smaller for all four states. However, increases remained statistically significant for all but Kentucky.” (Emphasis added.)

Tough questions: Why did the Kentucky Department of Education not report this thoroughly to both legislators and the public?Why do they still not come forward and express their concerns about our (yours and my) education system instead of allowing the deceptive “thrill of success” to preside?

Continued reticence to be forthcoming with our education system’s weaknesses has placed them in the role of the Department of Public Relations, adeptly exercising the skills of “damage control,” rather than the Department of Education, faithfully serving the parents of Kentucky with honest assessment — of themselves as well as of the children.

The damage has already been done.Trust has been broken time and time again. . . this time with the premier national test.But let me humor them a bit.Let’s pretend that our scores were “OK” with the NAEP test — that there was indeed “some kind of increase” or even just a “holding steady.”Then I have to ask them to give an account for this:Why in the four short years from 1994 to 1998 — by their own selection process — have we multiplied by 250 percent the number of students who are not qualified to take the test?

Read it and weep.Our fourth-graders have much to learn about reading. . .and our system limits teachers from succeeding in their task.
 

Key Family Foundation Contacts: Kent Ostrander, Executive Director Martin Cothran, Senior Associate Policy Analyst