Mathews' "Teaching to the Test" Gets it Wrong

Charlie likes shoes, big shoes. His new favorite book is The Big Blue Spot. It's good, very good. However, when a reporter turned columnist misrepresents the current state of testing and teaching that is bad, very bad. Yet that’s exactly what occurs in Washington Post’s Jay Mathews’ “Let’s Teach to the Test”. Clearly he hasn’t read my posts, or if he has he doesn’t understand them. In Mathews’ world of education NCLB Assessments are written by teachers, state standards are clear, and a good old-fashioned lesson review is all that’s needed to prepare students for the big day. Mathews claims that teaching to the test does not mean an annual drill and kill fest that use released test items to prepare students for state tests. I beg to differ. In my ten years of experience under Chicago Public Schools Accountability teaching to the test was a drop everything and prep affair across curriculums affair. The same drop everything and prep is alive and well in Chicago—at schools that are insecure about their AYP prospects. First we need to first examine the word “test” before we can get down to the nuts and bolts of “teaching to the test”. There are two types of tests out there—norm referenced (NRT) and criterion referenced (CRT). Mr. Mathews’ failure to distinguish between CRT and NRT highlights a common misunderstanding about NCLB Assessments. Briefly, NRT tests are general—they have a few items for each instructional objective or standard, and are designed to promote variability in scores—think SAT, GRE, ACT. In contrast, CRT tests thoroughly cover a limited number objectives or standards. CRT items reflect the criterion chosen and taught by the teacher(s). Most of us experience CRT tests in the form of teacher designed unit tests. The PSAE in Illinois is a hybrid—6/7 NRT and 1/7 CRT. That means that 6/7 of the tests reflect national norms. The Day 1 exam is the ACT—a NRT by design in which students are intended to fall along a “normal curve”. Two thirds of the day 2 exams are made up of Work Keys Reading and Math—both nationally normed tests written by the ACT for use in the workplace. Only the day 2 science exam is written by teachers. Teachers in Illinois have input on 1/7 of the total state exam. The rest of the test is designed by psychometricians to fit a normal curve. That means that students won’t really budge on the curve. This has been corroborated by a report by the Chicago Consortium that found little movement in scores on the PSAE. Few parents, teachers, administrators, and politicians know that PSAE tests are designed in a way that prevents substantial movement from quartile to quartile. Nevertheless starting in February of each year, teachers are pressured, and many cases mandated to teach to the ACT and Work Keys portions of the test. I call this type of teaching ACTriage. Companies like Kaplan, Princeton Review, and Cambridge are more than willing to offer materials and curriculum to assist in these efforts. Unfortunately these materials are geared to students how possess the requisite skills to do well on the ACT. Teachers in Chicago who participate in ACTriage quickly realize that many students have never been exposed to material at the level it is presented in the ACT. This means that many students don’t read the questions, let alone the passages with any level of competence. The same holds true for math—the math tests on the PSAE are verbal heavy, the current math curriculum in Chicago is not. Even the most resolute teachers begin to panic when they realize how fundamentally ill equipped the students are to take the exam. Each year ACTriage is a bust to teachers and students and a boon to test prep companies. Mathews gets partial credit for his belief that teachers teach to the state standards since they can’t legally get a copy of the state test. This assumes that the standards and tests are in complete alignment. As anyone familiar with teaching knows, only certain standards are actually assessed on the state exam. Many teachers teach to the state standards that are assessed on the state test. In Illinois, the ACT offers teachers the EPAS to assist them in this effort. The EPAS represents the subset of standards that are assessed by the ACT. Not surprisingly, schools that have used the EPAS to design curriculum score quite high on the ACT. It appears that Illinois has outsourced both the testing and standards to a testing company. Let’s be clear, a teacher who teaches to the EPAS is teaching to the test; not teaching to the vast array of standards in Illinois. Mathews’ failure to understand the fundamentals of assessment and curriculum renders his critique toothless. One more note to the uninformed, International Baccalaureate and Advanced Placement represent a set of curriculums and assessments, not just test that are “taught to”. Oh, Mr. Mathews, your contempt for teachers is as palpable as your ignorance of NCLB, curriculum and assessment. Your paper gets an “A” for style and an “F” for substance. All hat, no cattle my friend—just what I expect from someone inside the beltway MSM. Hey, if the shoe fits...