Tuesday, September 21, 2010

"Scientifically tested tests"

In her latest New York Times Op-Ed piece on what's wrong in education (her third since February), Susan Engel faults No Child Left Behind testing for measuring the wrong things and failing to help schools improve.

Many, myself included, have specific concerns about the NCLB tests: they set too low a bar and too low a ceiling, dumbing down the classroom curriculum; sometimes, correct but unexplained answers to math problems get only partial credit, as do incorrect but explained answers; the tests aren't used to help teachers adjust to the immediate needs of particular students, or to help particular students and their parents know what they need to work on, in the course of a particular school year. 

But Engel dislikes multiple choice, fact-based tests in general:
There are few indications that the multiple-choice format of a typical test, in which students are quizzed on the specific formulas and bits of information they have memorized that year, actually measures what we need to know about children’s education.
Well, that depends on what your opinion is about "what we need to know about children's education."

For Engel, it seems, the priorities are the "higher level" thinking skills that, in Engel's words, are "the qualities of well-educated children":
The ability to understand what they read; an interest in using books to gain knowledge; the capacity to know when a problem calls for mathematics and quantification; the agility to move from concrete examples to abstract principles and back again; the ability to think about a situation in several different ways; and a dynamic working knowledge of the society in which they live.
As for science, Engel mentions understanding the concept of "controlling variables."

Completely absent from Engel's proposals is content knowledge--unless "dynamic working knowledge of the society in which they live" includes things like world geography, American history, and current events in Pakistan. This, despite the fact that the latest cognitive science research indicates that "higher level" skills neither develop, nor apply, independently of structured, information-rich content. 

Also absent are such specific skills as penmanship, decoding, sentence construction, foreign language fluency, balancing chemical equations, and finding the roots to quadratic equations. 

Specific skills; rich, structured, factual knowledge: these are things a decent multiple choice test could assess, assuming you cared about them.

Then there are the specific skills Engel proposes tests should assess. However important these skills may be, it's highly questionable whether they (unlike, say, geography and quadratic equations) can actually be taught by classroom teachers.

For example, Engel proposes randomly sampling student writing to measure vocabulary and grammatical complexity. But as every linguist knows, general vocabulary (as opposed to specific vocabulary words taught in the classroom) and, even more so, grammatical complexity are developmental skills, not academic ones. Neurologically typical native speakers increase their general vocabulary sizes and their grammatical complexity through a combination of incidental exposure and brain development. Neither regular classroom teachers, nor standard curriculum packages, do much to address these skills, and, to the extent that they do, they have very little effect on them.

The same goes for perspective taking skills. Engel suggests having children “Write a description of yourself from your mother’s point of view" in order to "gauge the child’s ability to understand the perspectives of others." Again, it's not clear what purpose this assessment serves--beyond identifying who is and who isn't on the autistic spectrum.

Similarly problematic is Engel's proposal to measure reading comprehension levels by having children do an oral reconstruction of a story to a "trained examiner." What about shy children; what about children were struggle to express themselves orally?  How does this testing not penalize them and their teachers for circumstances beyond their control?

Then there's the cultural bias we see in Engel's proposal to measure literacy levels by "testing a child's ability to identify the names of actual authors amid the names of non-authors." Unless these authors come from some sort of core curriculum--something that Engel does not support--how does this testing not penalize socio-economically disadvantaged children and/or those who teach them?

While there are lots of problems with NCBL testing, there are decent ways to test children that help teachers teach better and children learn more. Unfortunately, Engel's proposals lead us in the opposite direction.

1 comment:

Peter B. said...

I'm intrigued by her use of "an interest in using books to gain knowledge" to assess the quality of a child. I am a highly-educated over-50 engineer who maintains a huge thirst for knowledge. Yet, I have almost no interest in using books to gain that knowledge. I prefer blogs, audio, and other tech-based sources. Cheaper. Faster. Richer. An "interest in using books" could easily surpass "an interest in using mules to plow a field" among important attributes in the quality of a child. Also, how does she know Robert Sponge is not a published author? There are now so many ways to self-publish (both electronically in print) that it is likely that there actually is a published author by that name. Is she suggesting that there is a "correct" list of authors that kids should recognize by name? And that only "better" kids know the names on her list?