by Sigrid Bathen published July 12, 1998
As director of research and evaluation for the state Department of Education for 12 years, Alexander Law was responsible for the statewide testing of students under the California Assessment Program, or CAP. In retirement, he has continued to observe the twisted history of the state’s efforts to measure student achievement, including the furor surrounding the STAR (Standardized Testing and Reporting) program, which utilized the Stanford 9 as the test. Alarmingly, it is a tale increasingly political, with worrisome consequences for education. It also raises the question of whether a standardized, statewide test can truly tell us anything meaningful about the academic progress of California’s diverse student population.
Law was both a designer and an overseer of CAP, which replaced the standardized tests of the 1950s and ’60s. “The early standardized tests were fairly straightforward, but they took several class periods, which in modern times does not seem inordinate, but then was,” recalls Law. “So we had to develop a test that took a class period or less and was broad enough to assess a variety of commonly taught skills.” The result was a “matrix” system that tested a student’s mastery of subject matter, with students getting different sets of questions from the matrix.
CAP only produced scores in various subject areas. It did not individualize results, although it did offer comparisons among districts and groups of students. “The best part of CAP was that it was tailored to the state’s curriculum,” says David W. Gordon, who worked under Law and is now superintendent of the Elk Grove Unified School District near Sacramento. “We were always confident that we were testing what people expected to be taught. With a national standardized test [like the Stanford 9], by definition, you’ll never get that.”
CAP chiefly fell victim to the political wrangling between then-Gov. George Deukmejian and then-Superintendent of Public Instruction Bill Honig. It also lost out to a growing testing movement that favored more subjective, essay-type questions, which were thought to measure student achievement more accurately. The resulting test, called the California Learning Assessment System (CLAS), debuted in 1992-’93 and was quickly dumped, criticized for, among other things, being too subjective.
California’s latest entry into statewide testing, the STAR program, has been no less controversial than its predecessor. From the outset, the use of a $35-million, “off-the-shelf,” standardized test was opposed by virtually every major education group, in part because of Gov. Pete Wilson’s insistence that the test be administered only in English and in part because state educators have been laboring to devise academic standards that would serve as a basis for a California-specific test. In the absence of such standards and corresponding curricula, the Stanford 9 results are all the more ambiguous, because any relationship the test may have with current curricula is coincidental.
An 11th-hour court ruling blocked release of some district scores, but test results have dribbled out, and reactions to them have ranged from outrage to disbelief, from resignation to disgust, all in an atmosphere of utter confusion. For example, when the Los Angeles Unified School District first released test results, it cheerfully reported that the district’s overall scores, year-to-year, had improved. On second look, however, it soberly announced that the improvement was not as great as previously announced, because the results of limited-English-proficient students had not been included. (Interestingly, in some school districts, scores of bilingual students who have mastered English were higher than their English-speaking counterparts).
Not surprisingly, Stanford 9 scores in well-heeled districts were generally high, while in poverty-stricken ones low. The belief that education in California is patently unequal is nowhere borne out more starkly, and with less explanation of variables, than in standardized testing.
Educators caution that the first test period is a foundation year and that results, accordingly, should not be given too much weight. They shouldn’t be dismissed, either. “It’s important that people ask questions–what is working, what isn’t?–as we begin to change the test, augment it, make it more aligned to the standards,” says Robert L. Trigg, a member of the state Board of Education and former district superintendent.
Yet, does a standardized test, given only in English, make any sense in a state whose student population is as ethnically and culturally diverse as California’s? Law thinks that testing non-English-speaking students in English is “absurd on its face.” Instead, he says, California needs two types of testing. “Level A are tests selected by the individual districts and schools to assess their programs. And you need state-level oversight–a temperature-taking, if you will–not of all grades, not every year, but a consistent, coherent program that would give you state-level information like CAP [did].” Where language differences are great, as in L.A. Unified, he says, the problem “is close to insoluble” using traditional testing methods. “You should not include scores into your aggregate of students who, by definition, cannot take the test.”
Regardless of the bilingual issue, the point is that California’s kids should be able to compete with children across the nation. Ultimately, that may mean dual testing. “We want individual results,” says Trigg, “and we want programmatic results. . . . You need both sets of information.”
Of course, you need a curriculum first, and California’s educators are working to develop one. Meantime, the STAR program is a poor substitute. “There are 5 million kids in California, and 4 million were tested,” says Education Department testing administrator Richard Diaz. “A lot of problems can occur. It was a compressed time period. There may have been some misunderstandings about what was to be tested. All these things are playing into this.”
Testing, moreover, has become an especially potent political tool in the education wars. Beginning with Deukmejian and CAP on down through Wilson’s demand last year that California’s schoolchildren be tested only in English, the issue polarizes to the point that “nobody trusts anybody else to do a plan,” said one prominent state educator. “In the ’70s and ’80s, people pretty much trusted [the state Department of Education] and, as a consequence, they had the space to do some logical planning. Now, there is such polarization between [state Supt. Delaine] Eastin and the board and the governor that everyone is scrambling for political advantage rather than doing what is logical.”
Because of California’s fractured system of education governance, Eastin, who is running for reelection, has no real policy-making power. That authority falls to the state board. Although Eastin ostensibly heads the bureaucracy in the Department of Education, she has virtually no control over the education budget and almost no rapport with the governor and the board he appoints.
Such fragmentation is exacerbated by the fact that individual districts, not the state Department of Education, legally hold the contracts with Harcourt Brace, publisher of the Stanford 9. When problems– including reports of defective magnetic tapes containing thousands of test scores and student scores being posted in the wrong district–arise, there is no central clearing house to vet technical glitches and quality control. As one frustrated superintendent put it: “For what we’re paying Harcourt Brace, we ought to just do our own test, absolutely tailored to our own curriculum. We should have a statewide test, but there is no spirit of cooperation [at the state level]. Nobody is blameable or blameless, really. But it’s a system with a lot of rancor and animosity.”
Sigrid Bathen is senior editor of the California Journal, a monthly magazine about politics and government.
