Tuesday, May 31, 2016

True Grit?

If we define success in life by public acclaim and genius awards, then one approach is to come up with an idea that is simultaneously ground-breaking and accurate. Short of that, you can always come up with an idea that is, depending on how you characterize it, sometimes ground-breaking, and sometimes accurate.

This is how it works. Pick a startling "new" idea, whether it's, say, that human intelligence consists of Seven Intelligences, or that some non-cognitive factor, say Grit, is one the biggest determiners of success. Proclaim it loud and clear. Give Ted Talks; win public acclaim and Macarthur genius grants.

Then, as the real world (or the eduworld) starts gleefully enacting your ideas, and as certain aspects of those enactments risk looking silly, start qualifying what you said, at least to certain audiences. I never said that there were exactly 7 intelligences; I never said that these intelligences were discrete entities; or, We don't really know how to teach or measure grit; Grit is only just one of many factors that determine success.

Then, even when someone in your field raises empirical challenges, you're exactly where you want to be.

Consider Marcus Crede's recent findings about Angela Duckworth's claims about grit, recently reported on NPR:

"My overall assessment is that grit is far less important than has commonly been assumed and claimed," says the lead author, Marcus Crede, an assistant professor of psychology at Iowa State University. "And it doesn't tell us anything that we don't already know."
 For example, Duckworth states that:
Cadets who scored a standard deviation higher than average on the Grit–S were 99% more likely to complete summer training.
As NPR notes:
That sure sounds like a big win for grit. But, as Crede points out, in fact what happened is that 95 percent of all cadets make it through Beast Barracks, while 98 percent of the very "grittiest" candidates made it through.
In other words:
it's the odds of making it through that improved by 99 percent. Most laypeople, though, would interpret "99% more likely" as meaning that your chances of getting through bounced, say, from 40 percent to 80 percent. Not by 3 percentage points
In contrast to what Duckworth has proclaimed to the public, Crede's meta-analysis finds
the relationship between grit and academic success "only modest." His analysis found an overall correlation of 0.18, looking at papers by Duckworth and others.
This compares to a much higher correlation of 0.50 between, say, SAT scores and performance in college.
Further undermining Duckworth's public proclamations is this:
In the various studies Crede looks at, conscientiousness scores and grit scores are very highly correlated — between 80 and 98 percent. Therefore, he calls grit a case of "old wine in new bottles."
This matters, because a major implication of Duckworth's work is that grit is a skill. Schools and districts around the country are currently working hard on creating curricula for grit, and even accountability tests to measure it.
But, psychologists say conscientiousness isn't a skill. It's a trait — driven by some unknowable combination of genetics and environment. It can change, typically improving as children grow up, but it's not necessarily amenable to direct instruction. Nor would we necessarily wish it to be.
"I think as a parent I would feel uncomfortable if my daughter came home and said, 'My school is changing my personality,' " Crede says. 
Complicating matters, however, is that Duckworth is saying different things to different audiences. First, there's academia and skeptics at NPR:
Duckworth's own numbers [for the correlation between grit and academic success], in a paper published in 2007, are only slightly higher [than Crede's]: 0.20.
Fundamentally, she told NPR Ed, she doesn't disagree with Crede here... She says her findings of the independent impact of grit are what personality psychologists would put in the "small-to-medium" range.
But then there's the general public, as addressed, for example, in Duckworth's 2013 Ted Talk, where:
she presents grit as a powerful, even unique factor: "One characteristic emerged as a significant predictor of success. And it wasn't social intelligence. It wasn't good looks, physical health, and it wasn't IQ. It was grit."
Besides saying different things to different audiences, there's a second trick: let the exaggerations of your supporters go uncorrected:
The cover blurb on Duckworth's book by Daniel Gilbert, author of Stumbling on Happiness, states, "Psychologists have spent decades searching for the secret of success, but Duckworth is the one who found it."
Naturally,
Duckworth says she never tried to oversell her findings, and has always tried to be "clear and honest."
Nor do I want to oversell my ideas. I'm not saying that every sometimes ground-breaking, sometimes accurate idea will result in fame, fortune, and Macarthur genius awards. To really be successful, the idea has to be something the public really wants to hear. If I were to "discover" that the one and only way to achieve communication breakthroughs in autism is through forty plus hours per week of intensive direct instruction in language plus intensive language practice, I doubt anyone would pay attention.

One indication that Duckorth has told the public exactly what it wants to hear is that, as of this writing, only four media outlets show up on Google News as having picked up this troubling counter-narrative. I'm guessing that Duckworth is still exactly where she wants to be, and that the public, and the eduworld in particular, are still hearing--and enacting--exactly what they want to.

Wednesday, May 25, 2016

News flash: how interesting and curious you are also matters!

It’s long been fashionable to downplay the significance of IQ—though, from the breathlessness of each new article about how unimportant IQ is, you’d never guess just how old this news is. Hand in hand with these constant revelations are revelations about the importance of “non-cognitive” skills--emotional intelligence, the ability to cooperative with others, grit. Promoters of these skills seem to feel that it’s news to the rest of us that how hard we work and how well we get along with others significantly affect our success in college, career, and life.

Buried in all of this codified common sense is another assumption: that all skills fall into one of two categories: those measured by IQ /SAT/ACT aptitude tests, on the one hand, and “non-cognitive” skills, on the other. But there are plenty of cognitive skills that aren’t measured by IQ and other aptitude tests.

What most IQ tests measure is how quickly you access basic, non-complex information—e.g., basic facts, single vocabulary words---and how quickly you perform non-complex operations—repeating digits, reversing digits, doing arithmetic, remembering and reproducing geometric configurations, copying and identifying patterns. IQ measures the how well your brain functions as a simple computer: its long term memory storage and access, its processing speed. With most IQ tests, the biggest determiners of your score are how much vocabulary you’ve memorized, how big your short-term memory buffer is, your basic spatial reasoning and pattern recognition skills, and how fast you process information.

While plenty of cognitive skills can be reduced to these factors, many cannot. IQ (and aptitude tests more generally) don’t measure the volume, organization, and connectedness of the facts you know, which vary widely from person to person. Nor do aptitude tests measure how interesting and creative your questions and your ideas are—which also varies widely from person to person. While these skills are partly a function of memory, processing speed, and pattern recognitions, they’re also a function of things like:

--attention and observation: how much of the world around you do you sponge up?

--curiosity: do you notice what you don’t know and care enough to seek answers—asking, listening, reading widely and in depth?

--your patterns of reflecting later on what you learned earlier (regular recall and reflection promotes long term memory)

--the breadth of topics your mind ranges over: does it brood on a narrow range of fixed topics or does it wander widely and to new places?

--the breadth of new connections—logical, analogical, relational—that your mind makes among the things it ranges over.

If we need to be reminded that how hard we work and how well we get along with others affect our success in life, perhaps we also need to be reminded that how knowledgeable, curious, and intellectually creative and interesting we are matter as well.

And perhaps our schools should be exploring not just whether they can promote grit and socio-emotional learning, but also whether they can promote, say, curiosity?

After all, most kids are born curious, and many famously go on to lose that curiosity during their school years—at least during the hours they spend in classrooms. Maybe, just maybe, there’s something our schools could do about that.

Monday, May 23, 2016

Another autism miracle cure!

Here we go again! Yet another child who "has autism"--and a normal mind "locked inside." From this past week's Washington Post:

For the first 14 and a half years of Gordy’s life, Evan and Dara Baylinson had no reason to think their son could comprehend anything they said: He had never spoken, and he couldn’t really emote. They worried aloud about his future, not filtering what they said, because they didn’t think he understood. 
But Gordy, it now appears, was absorbing everything.
“My brain, which is much like yours, knows what it wants and how to make that clear,” he wrote in a letter he sent this month to a police officer. “My body, which is much like a drunken, almost six-foot toddler, resists.” 
He typed each letter one at a time with his right index finger. No one coached him, edited his words or told him what to say, according to his parents and therapist. After two one-hour sessions, he had written a nearly 400-word note.  
“This letter is not a cry for pity, pity is not what I’m looking for,” he wrote. “I love myself just the way I am, drunken toddler body and all. This letter is, however, a cry for attention, recognition and acceptance.” 
Unbeknownst to his parents for so many years, their son was a beautiful writer with a lot to say.
The usual trappings are in place: an expensive, unproven, Guru-driven technique; a specific"therapist" who is apparently present for all the child's communications; a keyboard that for some reason has to be held up for the child instead of being placed on a stand; letters that for some reason have to be read out loud to the child "as he types." And a reporter who assures us that the therapist doesn't "visibly prompt him or move the keyboard."

With no video to watch--despite the fact that the Washington Post often does accompany stories with videos--I can't rule out that Gordy's communications are his. But if they are, then there's something I can rule out:  namely, that Gordy is actually autistic.

That's because, in just a year and a half of practice with productive language, Gordy's verbal exchanges (assuming they're indeed his) are more social, introspective, and empathetic than what we find even among the highest functioning individuals with autism. Gordy--if it's indeed him--is, for example, producing far more socially and emotionally sensitive utterances than what Temple Grandin--one of world's highest functioning individuals with autism--has exhibited after about fifty times as many years of practice:
When Gordy answered several questions from a reporter, he sat quietly, showing no external signs of all that he was feeling. But his answers showed he feels profoundly.
Why did you write your letter?

Meghann [his therapist] suggested it and I’m so glad, it was something my entire being felt compelled to do.

Why did you feel so strongly about it?

I’ve heard too many tragic stories of the mistreatment and mishandling of autistics due to lack of knowledge. It breaks my heart because I know no one is truly at fault.

Are you excited to meet everyone on Friday?

Absolutely, I never expected this but I’m jumping around like a madman inside.

What is your favorite thing to do?

I love learning new things, iPads, I love communicating and typing with this gal on my right.
We'll probably never know what's really going on here. But two things are clear:

1. even if this is a miraculous communication breakthrough, it isn't an autism miracle cure

2. it is highly unprofessional, and highly irresponsible to all who are touched by autism, to report on it as such.

Friday, May 20, 2016

Math problems of the week: Common Core-inspired open-ended test questions

Sample 3rd grade open ended test question from the State of New York's EngageNY.

First, the question and the associated Common Core Goal:


Next, the "Extended Rationale":


Next, five sample answers:








Extra Credit:

The question is worth 3 points. Given the Common Core Goal and the Extended Rationale, provide a score for each of these questions.

(I'll post EngageNY's scores for these in a few days.)

Wednesday, May 18, 2016

Grammar: everywhere--and nowhere

Grammar is dry and soporific; it’s also hip and deep. It is overlooked; and it is found everywhere. Actual grammar gets little publicity; false sightings are rampant.

A recent false sighting: the supposed (hip and deep) “grammar” of emoticons. Then there’s the supposed grammar of prairie dogs, and that of ornamentation.

The fact that something occurs in a sequence does not make it a grammar—particularly if that sequence is fixed:




Actual grammar is structure; it’s a set of iterative rules for grouping and ordering elements. E.g.:
A sentence consists of optional modifiers plus noun phrase plus verb phrase 
A verb phrase consists of a verb plus optional noun phrases plus optional prepositional phrases plus optional modifiers 
A prepositional phrase consists of a preposition plus a noun phrase 
A noun phrase consists of an article plus optional adjective phrases plus a noun plus optional modifiers 
An adjective phrase consists of an optional adverb plus an adjective 
And so on…
Thus even a relatively simple sentence derives from multiple rules and has structure:
[The quick brown fox] jumped [over [the lazy [prairie dog]]].
Grammars aren't limited to human languages; consider math, e.g., simple algebra:
An expression consists of a list of simple or complex terms connected by plus and/or minus symbols 
A simple term consists of a single number or variable 
A complex term can consist of simple terms connected by times and or division symbols, or parenthesized expressions connected by times and division symbols.
A small set of unstructured symbol sequences, or unstructured sequences of animal calls, is only a grammar in the most degenerate sense—however hip an deep it sounds to call it grammar.

Then there’s the dry, soporific version of “grammar”: what is actually punctuation, spelling, or both. Classic examples are its vs. it’s; your vs. you’re; there vs. their vs. they’re.

Real grammar is hipper and deeper than all of these.

Monday, May 16, 2016

Hating Everyday Math--every day

"What makes Everyday Math especially misleading is the fact that, when other programs are blatant about the de-emphasis of skills, Everyday Math camouflages this de-emphasis by the massive onslaught of a super-abundance of skills."
--Prof. Hung-Hsi Wu, U.C. Berkeley Mathematics Department.
This quote was sent to me by a comrad-at-arms in the math wars--along with a petition for the Central Bucks school district in Eastern Pennsylvania to Drop Everyday Math.

Yes, while math education continues to morph-- from Common Core-inspired tests to flipped classrooms to online learning--Everyday Math is still very much with us.

One of the locals who signed the petition provided a link to a 2013 article in Forbes by Emily Willingham. Willigham concludes with this:
“My children like math and play math games at home for entertainment. But they hate Everyday Math, every day.”

Saturday, May 14, 2016

Re-equpping students for "complex texts"

The Edweek article I blogged about earlier offers one other reason for grammar instruction: reading comprehension. Since the Common Core requires students to read “complex texts”

“we need to teach them how to read complex sentences,” said Chris Hayes, a veteran elementary teacher in Washoe County, Nev. And that requires deep knowledge of grammar.
Edweek touts the “Juicy sentences” approach of Lily Wong Filmore:
the technique involves pulling a particularly complicated sentence out of a text that students are reading, and deconstructing it as a class.
But for this, you need a basic vocabulary of terms with which to label the parts of sentences. And that’s where parts of speech come in. Edweek does not connect the dots; here’s my proposal:

First, teach parts of speech—especially noun, verb, preposition, article, adjective, adverb, co-ordinate conjunction, subordinate conjunction, and pronoun. Through in a few linguistic terms like “relative pronoun” (the “which” of “that” of “the book that/which I read”) and “complementizer” (the “that” of “I said that I read the book”).

Then use these terms to teach the syntactic elements of sentences—especially noun phrase, verb phrase, subject, main verb, modifier, restrictive and nonrestrictive relative clause, appositive.

Using these terms, teach students the specific steps for deconstructing and making sense of complex sentences, for example this one I blogged about earlier from Jane Austen:
To know that she had the power of revealing what would so exceedingly astonish Jane, and must, at the same time, so highly gratify whatever of her own vanity she had not yet been able to reason away, was such a temptation to openness as nothing could have conquered but the state of indecision in which she remained as to the extent of what she should communicate; and her fear, if she once entered on the subject, of being hurried into repeating something of Bingley which might only grieve her sister farther. -
Steps:

1. Remove all the parentheticals, appositives, nonrestrictive relative clauses, and sentence-level modifiers:
To know that she had the power of revealing what would so exceedingly astonish Jane, and must so highly gratify whatever of her own vanity she had not yet been able to reason away, was such a temptation to openness as nothing could have conquered but the state of indecision in which she remained as to the extent of what she should communicate; and her fear of being hurried into repeating something of Bingley which might only grieve her sister farther.
2. Find the main verb
“was”
3. Find the subject, noting that it might be many words long, and being sure to include all its modifiers. Note that in older texts, the subject may sometimes occur after the main verb.
To know that she had the power of revealing what would so exceedingly astonish Jane, and must so highly gratify whatever of her own vanity she had not yet been able to reason away
4. Find any direct or indirect objects
such a temptation to openness as nothing could have conquered but the state of indecision in which she remained as to the extent of what she should communicate; and her fear of being hurried into repeating something of Bingley which might only grieve her sister farther.
5. Repeat steps 1-4 with any really long restrictive relative or other subordinate clauses--e.g.:
[what] must so highly gratify whatever of her own vanity she had not yet been able to reason away
and:
[that] nothing could have conquered but the state of indecision in which she remained as to the extent of what she should communicate; and her fear of being hurried into repeating something of Bingley which might only grieve her sister farther.
6. Paraphrase the gist of the subject and object; then paraphrase the gist of the sentence

7. Repeat steps 1-6 with any with really long parentheticals, appositives, nonrestrictive relative clauses, and sentence-level modifiers.

8. Gradually add the gist of these modifiers to the gist of the sentence.

Thursday, May 12, 2016

Math problems of the week: Common Core-inspired open-ended math questions for 5th grade

From the Pennsylvania System of School Assessment (PSSA) Mathematics Item and Scoring Sampler for Grade 5:


Extra Credit:

Does this "complete explanation" show that the student has moved beyond procedural understanding to conceptual understanding?

Monday, May 9, 2016

Fan mail

In my inbox this past week, the following letter:

You’re website is politically incorrect. Students are no longer being labeled “autistic”, but instead labeled as having autism. The word “retarded” is no longer acceptable for use either. If you are priding yourselves on helping students, you need to change your website. It is making me sick.
My first reaction was: what a concise reflection of today's times! The writing skills; the thought-policing; the primacy of how I feel over thoughtful argumentation.

My second reaction was: I'm pretty sure I've studiously avoided the r-word. So I searched my blog, and the only thing that comes up is this: People look at me and assume that I am dumb because I can’t speak and I knowingly contribute to my looking retarded by carrying around a plastic spoon, but spoons are my comfort. There aren't my words, but a quote from the person doing the voiceover for Sue Rubin, an individual who purportedly "has autism" and whose communication is "facilitated."

As portrayed in this video and elsewhere in the media, Sue Rubin is a poster child for "having autism" vs. "being autistic." She is one of those purportedly "normal child locked inside" individuals who, with the help of a facilitator and a keyboard, is supposedly fully aware, expressive, introspective, and empathetic. And the above is what she purportedly says about her attachment to spoons.

According to the clinical (as opposed to the popularized) picture of autism, "being autistic," as I've argued (most recently here), is a much more appropriate characterization, given how deeply entangled the condition is with core cognitive and personality traits.

Consistent with this, many higher functioning individuals in the adult autism community prefer "autistic" to "having autism." In more general terms, like members of the Deaf community (who reject labels like "with deafness" or "having hearing impairments"), they prefer "identity-first" language to "person-like" language.

The counter-argument by self-styled disability advocates is that "indentity-first" language stigmatizes the people; these "advocates" overlook how "person-first" language stigmatizes the condition. "Having autism" sounds like "having shingles"--and it denies what many people think of as intimate to their identity.

As I've noted earlier, the divide between those with special needs and those who purport to speak for them can be quite toxic for the former. In person-first language, those who "have special needs advocacy goals" need to listen more carefully to those who "have special needs, period."

Friday, May 6, 2016

Math problems of the week: Common Core-inspired "complete explanations"

From the 8th grade Pennsylvania System of School Assessment (PSSA) Mathematics Item and Scoring Sampler for Grade 8:




Extra Credit:

1. In what sense is this a complete explanation?

Hint: Consider the terms "even," "exact value," and "never stops." Consider as well the difference between answering a question and rephrasing the question as a statement.

2. Discuss the human element involved in scoring responses to open-ended questions on tests taken by millions of students.

3. Discuss the human element involved in scoring the verbal explanations submitted by tens of thousands of English-impaired students.

Wednesday, May 4, 2016

When old news inspires open-ended assignments

In a recent article in Edweek, Sarah Sparks reports on a new book by affiliates of USC's Brain Creativity Instistute, Emotions, Learning, and the Brain. In this book, Immordino-Yang et al

found that as students learn new rules during a task, such as the most efficient way to answer a math problem or the best deck to choose in a card game, they show emotional and physical responses long before they became consciously aware of the rules or are able to articulate them.  
This emotional response—think of a student having a "gut feeling" that a particular answer was right—was the first sign of a student learning from her experience with the task.
Why the rediscovery of the gut feeling is worth an article in Edweek is thus far unclear. But there’s more:
Immordino-Yang and her colleagues [at UCS’s Brain Creativity Institute] have found that even abstract academic concepts can inspire an emotional connection if people understand their context. For example, mathematicians show the same pleasure response in the brain when they see an efficient equation as others have shown when viewing a beautiful piece of art.
If Immordino-Yang et al had delved a little deeper, they might have found that “abstract academic concepts,” including mathematical ones, have a long history of inspiring emotions--even in the absence of whatever is meant here by “context.” But perhaps, for those who steer clear of abstractions, this finding is nonetheless newsworthy.

Also apparently newsworthy:
…separate studies… suggest that negative emotions can interfere with learning in part because they compete with normal engagement with new concepts.
You mean if I think about death--or taxes, or how anxious I am about math--while I’m trying to learn math, it might actually interfere with my learning process?

Other startling findings:
"What happens when thinking is devoid of emotion is you don't remember it or think deeply about it," Immordino-Yang said.
You mean the stuff I think about the most and remember the best tends not to be stuff I’m completely indifferent to?

What’s not so obvious is Edweek’s takeaway: that schools should “support students’ emotional development.” Why not instead make sure the stuff they’re learning is actually engaging? Engagement, after all, is inherently emotional: a combination of excitement, curiosity, inspiration, and ambition.

Sparks reminds us that
Prior studies have shown children become less positive over the course of elementary school, and new German studies suggest academic engagement and achievement—or the lack of them—could create feedback loops for young students. 
… A student who was anxious in math class in 2nd grade was likelier to have lower math achievement at the end of the year; lower math achievement at the end of 2nd grade made it likelier that the student would be even more anxious in 3rd grade, increasing the risk of even lower math performance, and so on through elementary school. Boredom also produced a negative cycle, while early enjoyment in math created a positive feedback cycle.
But is the answer to tinker with students’ emotions, or with the assignments we subject them to?

Yang’s answer, refreshingly, involves both. First, encourage students to use their cognitive intuitions while learning. Second, make content meaningful rather than distracting students with jokes and prizes. So far so good. But then comes recommendation number three:
Give students open-ended problems that force them to dig into the definition of the task itself.
Where did this come from? What does being forced to "dig into the definition of the task" have to do with positive emotions and learning? Open-ended problems are already all the rage, and many of them, for many students, lead not to positive feelings, but to disengagement and anxiety.

Monday, May 2, 2016

Common Core-inspired grammar fallacies, and why we need paper graders

An article in a February issue of Edweek, Will the Common Core Step Up Schools' Focus on Grammar?, repeats the false choice I blogged about earlier between isolated grammar lessons and unstructured reading and writing assignments:

Determining the best approach for teaching grammar and semantics is now once again a critical conversation topic. Should teachers dedicate time to stand-alone grammar lessons and tasks---diagramming sentences, for instance, or memorizing the differences between adjectives and adverbs? Or can students learn the language system through broad writing and reading?
Later on in the piece, however, author Liana Heitin acknowledges that the Common Core, at least accordingly to some interpretations, has taken a middle ground, appearing to embrace what I’ve called “applied syntax”:
“The standards do, however, focus more on grammar application than most previous state standards, some say—which could encourage more authentic grammar work.
On the other hand, the standards (and their professional interpreters) still fall for Fallacy II: failing to factor out those aspects of grammar that native English speakers already know:
the grammar skills in the content standards don’t differ too much from most previous state standards… For instance, they ask students to ‘use an apostrophe to form contractions’ and ‘form and use regular and irregular verbs’—benchmarks that shouldn’t much surprise teachers. 
“Whereas before it was OK for a kid to identify nouns, now, it’s that they actually have to be able to use them and use them correctly,” said [one teacher].
Native English speakers do need to learn written conventions like when to use an apostrophe; they don’t need to learn how to conjugate English verbs and how to use English nouns. The latter skills belong in Common Core Standards for ESL, not Common Core Standards for ELA.

However willing teachers are to teach native speakers how to conjugate verbs and use nouns, Liana Heitin and the teacher she quotes have reservations about embedding grammar instruction in the feedback-on-multiple-writing-drafts approach I suggested earlier:
The realities of classroom management can make teaching grammar through writing tough as well. “Ideally, you wouldn’t have to teach [basic grammar skills] in isolation—you’d be having students writing a paper and then correcting it,” said Meghan Everette, a 3rd grade teacher…. “But it doesn’t really work out that way.”  
“Young students need a lot of direction in learning new skills, she said. And managing that kind of individualized task with 20 or 30 students is just too time consuming.”
Yes, as I noted earlier, marking up a stack of student papers is one of the most tedious teaching tasks out there. And yet, teachers used to do that regularly. If that’s too much to ask of today’s teachers, then let’s try something similar to the classroom teacher vs. classroom management labor-division I suggested in my last post. In addition to classroom teachers, let’s hire paper graders—akin to those we find in in large college classes. Again, we can safely offset the cost of the added personnel by increasing average class size.

Not all of the student self-correction process needs to be labor-intensive for teachers. Many errors needn’t be explicitly pointed out in order for students to fix them. Sometimes it’s just a matter of requiring students to actually re-read, and proofread, their papers—something that fewer and fewer seem to be in the habit of doing.

Teachers can also motivate careful revisions with minimal time expenditure by harnessing the tremendous editorial power of the Word Limit. As anyone who has faced one of these can tell you, cutting out words without sacrificing content generally involves drastically improving your prose. So why not ask students to reduce their word count, say, by 50%, without eliminating any content, from penultimate to final draft?