Standardized testing stands in the way of student choice

Dec. 18, 2012

By Dr. Dawson Orr

This article was recently published in the winter issue of INSIGHT, the Texas Association of School Administrator's quarterly magazine.

As Texans ready themselves for a tumultuous legislative session, there is spirited dialogue across the state over how best to prepare students for the future.

Testing dramatically increased this year with the introduction of the State of Texas Assessments of Academic Readiness (STAAR), and parents are concerned about the unintended consequences of the state's cumbersome bureaucratic approach to student and school accountability. Parents are bewildered by the complexity of a high school graduation plan that requires endless state testing starting in the freshman year and continuing until at least the junior year. In the end, the student receives a cumulative score. It's not sufficient to demonstrate proficiency in Algebra II. In the state's eyes, it's not whether you cross the goal line; it's how many yards you gained on the plays before reaching the goal line. You can't be a late bloomer in the state's version of accountability.

Besides the artificial hurdles, parents are concerned over state end-of-course (EOC) tests that students in Advanced Placement (AP) and International Baccalaureate (IB) classes are required to take in addition to the existing tests associated with those curricula. Here's an example: A student challenges himself by enrolling in English III AP instead of the standard English III class. The curriculum for the AP course meets or exceeds the academic rigor of a comparable college course, and it covers different material than the English III course. Yet the student must take two tests: the AP test AND the English III EOC test, which covers material from a completely different class with a different curriculum.

In the meantime, the state is designing its own college readiness benchmark, which will be based on test scores from the Algebra II and English III EOC exams. Why does Texas need its own measure when established, reliable national measures such as the ACT and SAT already exist? While it is not uncommon for states to set higher standards for out-of-state students, it is unusual for a state to erect greater hurdles for its own citizens. This is why we're seeing some of our top high school students leave Texas for college. The top 10-percent admissions rule produces one hurdle, and it is possible that the state-designed readiness benchmark will result in yet another requirement, this one based on STAAR test results.

This state-created maze, which actually works against our students, exemplifies the diminished voice of local communities in the education of their children and youth.

Why the divergence in the state's interest and those of local communities? While not an easy question to answer, I offer the following for consideration.

Customization Versus Standardization

Texas state government has pursued a strategy of bureaucratic standardization to govern and improve K-12 education. It's evident in the over-specification of thousands of discrete curricular standards; a one-size-fits-all 4 x 4 high school graduation plan; and a data collection, testing, and accountability regimen designed for regulatory compliance.

Parents, teachers, and communities want customization, and they successfully create it, despite state obstacles. They know that all children are not alike, and they favor multiple pathways for students to develop their unique talents, skills, and competencies. They recognize that college is a great choice for many, but not all. They know that all children do not learn at the same rate or in the same way. They value some testing, but also value examples of their children's work, such as a well-written paper, a project report, and a conference with their child's teacher to review academic progress.

While there are many examples of the tension between state standardization and community customization, let's return to the much-discussed English III EOC exam. It is a high-stakes accountability measure attached to the individual student and the school. Under present law, it represents a triple jeopardy for students: 15 percent of the final grade of a required course (waived for one more year), a passing score to graduate, and a yet-to-be-determined " higher-than-passing " score that will be combined with an Algebra II EOC score to determine eligibility to attend a four-year college or university in Texas. It assumes that all students study the same curricular standards. English III is English III is English III.

How does this standardized state test fit with the customized reality of what teachers teach to students during the junior year of high school? Not very well! Go into a Texas high school today, and you will find some juniors taking the state's version of English III. Other students may be enrolled in a dual-credit English III course for which they can receive both high school and university credit. Other students are in AP English III. AP courses are developed in cooperation with the College Board and are designed to be the equivalent of a first-year college English course. Students sit for a national Advanced Placement exam to receive college credit. Other students may be in an IB English course designed to prepare them academically for universities in America and across the world. IB students, too, sit for externally developed and scored exams that lead to college credit. These courses all address the domain of English, but there are differences in the specific fluencies, competencies, and skills that are emphasized. The English III EOC exam is a standardized, high-stakes test that can't be responsive to the customized pathways that students choose in their pursuit of learning. This focus on standardization will drive out choice, which, ironically, will have the effect of reducing academic rigor.

A Texas hurdle for a world of choices

Let's go deeper into the notion of Texas developing its own measure for college readiness based on standardized test scores. Let's also acknowledge that the state's college readiness benchmark, when implemented, will become a de facto minimum admission requirement for Texas universities. If Texas were self-contained in the sense that all its high school graduates only attended Texas colleges and universities, it might be logical that Texas would need to give its seal of approval to college-bound students. As we know, that is not the case. Texas students compete for admission to colleges and universities across this great nation and the world. It is doubtful that those institutions look at STAAR test results. Universities tell us that they consider students' course selections, grades, essays, extracurricular activities, and leadership skills. The only standardized tests they strongly consider are the ACT and SAT college entrance exams.

So if the ACT and SAT exams are the standardized measures that colleges use, how well does the Texas English III EOC standard match those of the ACT and SAT? Not very well! TEA's external validity studies highlight the significant differences between English III EOC and the existing, broadly accepted measures used by universities. Let's compare English III EOC and the ACT on two key concepts: content overlap and score correlation. Each of these concepts requires explanation.

Content is key, and the ACT and English III EOC have less than 50 percent overlap. Why do they differ? The ACT is based on a long history of periodically surveying high schools and colleges about the academic knowledge students need to be successful in college. The ACT distills its vast amount of information gleaned from years of work into the core power standards needed in college. The ACT tests address these standards. Why does the state EOC test measure more than twice the content as the ACT? Simply stated, the English III EOC tests many more curricular standards because it measures student performance on a broad state curriculum designed to serve purposes in addition to college readiness. The ACT is a relatively well-honed instrument that assesses a consensus set of learning standards related to college academic readiness. The EOC is a broad-brush assessment of the state's extensive collection of readiness and supporting standards, and it covers a much broader set of standards. The state curriculum is categorized into learning standards that are either "readiness" or "supporting." The state testing blueprints call for 60 to 70 percent of items to address the readiness standards, which are considered the grade curricular standards of greatest importance. That leaves 30 to 40 percent to address supporting standards, which are those standards that contribute to understanding. Supporting standards may have been emphasized in the previous year's instruction or may become a readiness standard in a future year. At the end of the day, the state test will address 22 standards with as few as 25 multiple-choice questions. Some standards will be assessed with a single question. The issue is not whether the state curriculum should be assessed, but how it is assessed and for what purpose.

It logically follows that two tests with great variation in content would also differ in what their scores represent. We would hope to find a very high correlation between the EOC and ACT, but there is only a moderate positive relationship (.59) between the two scores. The magnitude of that correlation is sadly lacking for something as critical to our students as college admission. The content overlap for Algebra II EOC is even weaker with the same moderate level of correlation. The ironic reality is that state English III EOC and Algebra II EOC college readiness scores that are only moderately related to the ACT and SAT could stand in the way of a student going to a Texas college.

In summary, the state's strategy of standardization over customization isn't needed, nor does it do justice to the multiple pathways Texas students follow in pursuit of postsecondary options. As this legislative session approaches, our state leaders have a great opportunity to embrace customization, which will provide the best possible education for the children of Texas. It is only by reinvigorating the state-community partnership that we will accomplish this goal. In the short run, legislative action is needed to limit the role and scope of EOC tests. Parents, educators, and business leaders recognize the need to restore balance to the assessment of children.

We need to thoughtfully design a system that helps our children succeed instead of endlessly adding more tests and measures. That's where the newly appointed Texas High Performance Schools Consortium will play a strategic role. This pilot program, made possible by Senate Bill 1557 during the last legislative session, provides a unique opportunity for Texas to develop a world-class public education system that customizes curriculum, instruction, assessment, and accountability as the primary strategies for school improvement.

Dawson Orr is superintendent at Highland Park ISD, a member of TASA's School Transformation Network Design Team, and steering committee co-chair of the Texas High Performance Schools Consortium. Click here to print this column.

Dr. Orr will host a public forum to address the state's testing and accountability system at 6 p.m., Jan. 10 in the Highland Park High School cafeteria. Click here for more information.

The Texas Education Agency is currently seeking feedback on the its new testing and accountability mandates. Comments can be submitted here through Jan. 18.

Click here to read Dr. Orr's March 2012 column "What is the measure of a child?"