Sunday, May 4, 2014

What does complete comprehension of a text mean--for the purposes of measuring understanding?

On a recent post, a commenter took issue with my notion of “full comprehension” as a measure of reading skill. In the age of post-post-modernism, how can anyone still think that full comprehension is possible?

So let me draw a distinction between linguistic comprehension—which is mostly (though not entirely) about a text’s literal meaning—and literary interpretation, which goes far beyond it, and leaves plenty of room for—well—interpretation.

Linguistic comprehension involves basic things like: knowing what the words mean in the given context, knowing what the pronouns (“it”, “they”, “this”, etc.) as well as generic terms like “that idea” and “the statesman” refer to in the context of the text and its assumed background knowledge; identifying where an interrupted clause picks up after an appositive; knowing which parts of a sentence its various modifiers modify; and having the basic background knowledge to make connections elements of the text that, literally, seem disconnected. Without this, the text will remain incoherent, and comprehension will be faulty or incomplete.

Consider this passage from "Gulliver’s Travels to Lilliput," a rather tough passage that I helped my daughter work her way through just this past week. It occurs just after a section describing how, in service of the emperor of Lilliput, Gulliver, a.k.a. "Man-Mountain," dragged fifty men-of-war from the fleet of the empire of Belfuscu over to the shores of the empire of Lilluput.

His Majesty desired that I would take some opportunity of bringing all the rest of his enemy's ships into his ports. And so unmeasurable is the ambition of princes, that he semed to think of nothing less than reducing the whole empire of Blefuscu into a province and governing it by a viceroy; of destroying the Big-endian exiles and compelling that people break the smaller end of their eggs, by which he would remain the sole monarch of the world. But I endeavored to divert him from his designs by many arguments drawn from the topics of policy as well as justice, and plainly protested that I would never been an instrument of bringing a free and brave people into slavery. And, when the matter was debated in council, the wisest part of the ministry were of my opinion. This bold declaration was so opposite to the schemes and politics of his Imperial Majesty, that he could never forgive me. And from this time began an intrigue between his Majesty and a junto of ministers maliciously bent against me, which broke out in less than two months and has like to have ended in my utter destruction.  
About three weeks after this exploit there arrived a solemn embassy from Blefuscu, with humble offers of peace; which was soon concluded on conditions very advantageous to our Emperor. There were six ambassadors with a train of about five hundred persons. When their treaty was finished, wherein I did them several good offices by the credit I now had at court, their Excellencies invited me to that kingdom in the Emperor, their master’s name. I desired they would present my most humble respects to the Emperor, whose royal person I resolved to attend before I returned to my own country. Accordingly, the next time I had the honour to see our Emperor, I desired his license to wait on the Blefuscudian monarchy, which he was pleased to grant me, as I could plainly perceive, in a very cold manner but could not guess the reason, till I had a whisper from a certain person that Flimnap, the High-Treasurer, and Bolgolam, the Admiral, had represented my intercourse with those ambassadors as a mark of disaffection from which I am sure my heart was wholly free.
Here are the questions I would ask to assess whether a child completely understands the text, in the linguistic sense of understanding:

1. Translate the first sentence of the first paragraph into ordinary, modern English (“His Majesty desired that I would take some opportunity of bringing all the rest of his enemy's ships into his ports”).

2. What does it mean to “reduce the whole empire of Blefuscu into a province”?

3. What does “viceroy” mean?

4. Translate “I endeavored to divert him from his designs by many arguments drawn from the topics of policy as well as justice” into ordinary, modern English.

5. What does “his designs” in the above sentence refer to?

6. In “The matter was debated in council,” what does “the matter” refer to?

7. What does “in council” mean?

8. What was the result of the debate?

9. What does “this bold declaration” refer to?

10. Why does the emperor never forgive the Man-Mountain?

11. What do “intrigue” and “junto” and “bent against” mean?

12. Translate “had like to have ended in my utter destruction” into ordinary, modern English.

13. What does “this exploit” refer to?

14. What does “embassy” mean in this text?

15. Translate “with humble offers of a peace; which was soon concluded upon conditions very advantageous to our Emperor” into ordinary, modern English.

16. What does “train” mean in this text?’

17. What does “wherein” mean, and what does it refer to?

18. What does “good offices” mean in the text?

19. What does “the credit I now had at court” refer to?

20. What does “in the Emperor, their master’s name” mean?

21. Translate “I desired they would present my most humble respects to the Emperor, whose royal person I resolved to attend” into ordinary, modern English.

22. What does “his license” mean here?

23. What does “monarch” mean?

24. Who “could not guess” what reason?

25. Translate “I had a whisper from a certain person that Flimnap and Bogolam had represented my intercourse with those ambassadors as a mark of disaffection from which I am sure my heart was wholly free” into ordinary, modern English.

18 comments:

Anonymous said...

I wonder if it strikes you, as it does me, that answering all those questions would take fifty times as long as reading the text in question, and change the nature of her work from a reading to a writing exercise.

I also wonder how old your daughter is, whether you still read to her, and whether you have personal experience with reading literature in a foreign language.

I find that my experience reading literature in other languages influences my conception of the process of acquiring linguistic fluency through reading. I certainly haven't encountered any theorists who would gate reading practice on full prior comprehension the way you suggest.

Katharine Beals said...

Anonymous,

You're reading way too much into this post.

This isn't about my daughter; rather, to requite from the post, what I'm doing is enumerating "the questions I would ask to assess whether a child completely understands the text, in the linguistic sense of understanding."

Nor do these questions need to be answered in writing. They can asked and answered quite quickly orally-assuming the child understands the text.

Nor am I proposing them as way to "gate reading practice."

Rather, my main point is that if a child can't answer these questions, they haven't yet attained complete linguistic understanding of the text.

Katharine Beals said...

As for reading literature in a foreign language, my daughter, as well as myself, regularly read literature in other languages. When my daughter reads in French, I sometimes have her translate the tougher passages. And when I read, say, Kafka, in German (not my best foreign language), I make sure I can translate the longer sentences into English. This is a very fruitful way to monitor comprehension in cases where it isn't second nature.

Katharine Beals said...

Type above: "requite" should be "re-quote."

Anonymous said...

I take your point about linguistic versus interpretational understanding of texts. But would propose that it can take more than one reading, over a span of years, to even achieve full or near-full linguistic understanding. Shakespeare is a good example. The Bible is another.

Anonymous said...

And, I would add, it's frequently OK for students to get a good start on linguistic understanding; pounding a lot of difficult stuff into them when without that level of parsing they can get the gist and understand most of what's going on, is sometimes (not always) a better strategy.

Anonymous said...

One of the wonderful things about homeschooling is that it allows such a diversity of methods and responses to each child's individual needs. That said, I wonder if you've considered that your approach to and understanding of reading isn't universally accepted, and might even appear backwards to some. It seems like you're taking sledgehammer to a flower here; perhaps it's an example of, in your terms, left-braining a right-brain process – similar to when curriculum designers turn math problems into art problems, but the other way around.

There's no need to impute scoffingly academic fashions ("post-post-modernism") to the fundamental fact that people understand words not in an either-or, absolute unfamiliarity vs absolute mastery fashion, but on a scale, e.g. from 'never seen that word before' through 'I think I saw that before' and 'that's similar to another word I know' all the way to 'I use that correctly every day.' This is the way children and adults acquire words, in gradations rather than all at once. Furthermore, reading with partial comprehension is the primary way children acquire vocabulary in general. Don't take an anonymous' word for this; a good read from a professional might be Nagy, "Learning Words Through Context," available online. If you wanted to extend the ZPD metaphor to a text, all vocabulary learning happens when kids don't 'fully comprehend' all the words in a book, and no vocabulary learning is possible when kids do 'fully comprehend' all the words in a book.

When I say backwards, I mean that literally. When you subject kids to such a lengthy inquisition, you not only raise the affective barrier, but interrupt their acquisition of vocabulary, rather than advancing it. If this sort of inquisition is what your children get whenever they read a new book, it's no wonder they don't want to read very much. Interrupting the reading process for a different sort of exercise – writing, oral exams, definition making, questions from the teacher, call it what you will – may do more to inhibit acquisition of vocabulary than stimulate it. Children do not normally learn vocabulary apart from reading and then apply that studied vocabulary to the exercise of reading. They learn vocabulary, bit by bit, through the process of reading itself, as each new word moves up the scale from 'never seen it before' through 'good idea what that means' to usable vocabulary. Vocabulary quizzes are an interruption rather than extension of the vocabulary acquisition process if they substitute for reading.

With second language fluency acquisition, the process is similar, once the basics are learned: words are naturally learned in context, through progressive approximation. Some vocabulary can be gained through study divorced from normal usage, using flashcards and drills and the like, but such translation-based fluency is limited and can provide at most the scaffolding for acquisition of real fluency (or the ability to pass school quizzes). Stopping a child's reading in the second language to insist the language be translated back into the mother tongue does not necessarily assist and may rather impair the language learning progress. If you conceive of a second language as nothing but a funny code for English, you can never approach fluency. You're not fluent because you can translate a sentence back into English; you're fluent to the extent that you don't have to.

Katharine Beals said...

"But would propose that it can take more than one reading, over a span of years, to even achieve full or near-full linguistic understanding. Shakespeare is a good example. The Bible is another."

Can you share some Biblical or Shakespearean passages that illustrate this point?

Katharine Beals said...

It’s interesting how entrenched we are in issues of reading comprehension here.

Anonymous writes:

“It seems like you're taking sledgehammer to a flower here; perhaps it's an example of, in your terms, left-braining a right-brain process – similar to when curriculum designers turn math problems into art problems, but the other way around.”
...
“When you subject kids to such a lengthy inquisition, you not only raise the affective barrier, but interrupt their acquisition of vocabulary, rather than advancing it. If this sort of inquisition is what your children get whenever they read a new book, it's no wonder they don't want to read very much.”

Anonymous writes these things even though I never wrote that I would actually teach anyone this way, let alone interrupt reading in this way, and even though I clarified that fact in an earlier comment on this very thread.

Anonymous, who appears to be an opponent of needless scoffing, also writes:

“There's no need to impute scoffingly academic fashions ("post-post-modernism") to the fundamental fact that people understand words not in an either-or, absolute unfamiliarity vs absolute mastery fashion, but on a scale, e.g. from 'never seen that word before' through 'I think I saw that before' and 'that's similar to another word I know' all the way to 'I use that correctly every day.' This is the way children and adults acquire words, in gradations rather than all at once. Furthermore, reading with partial comprehension is the primary way children acquire vocabulary in general. Don't take an anonymous' word for this; a good read from a professional might be Nagy, "Learning Words Through Context," available online. If you wanted to extend the ZPD metaphor to a text, all vocabulary learning happens when kids don't 'fully comprehend' all the words in a book, and no vocabulary learning is possible when kids do 'fully comprehend' all the words in a book.”

“Children do not normally learn vocabulary apart from reading and then apply that studied vocabulary to the exercise of reading. They learn vocabulary, bit by bit, through the process of reading itself, as each new word moves up the scale from 'never seen it before' through 'good idea what that means' to usable vocabulary. Vocabulary quizzes are an interruption rather than extension of the vocabulary acquisition process if they substitute for reading. “

To linguists, this is very old news--and so we do not need to take Anonymous’ word for it. Nor does it contradict anything written in this post.

“With second language fluency acquisition…”

Ditto.

Anonymous writes:

“Stopping a child's reading in the second language to insist the language be translated back into the mother tongue does not necessarily assist and may rather impair the language learning progress.”

People have been claiming for decades that translation exercises impede language learning. But this is yet another area where empirical studies should outweigh opinions. Please share any empirical studies you are aware of that support your statement.

“ If you conceive of a second language as nothing but a funny code for English, you can never approach fluency.”

No language is a funny code for any other language. Nor does the translation process make any such assumptions.

As for fluency, by far the most promising route is extended immersion—and only rarely is this possible in classroom or homeschool settings.

“You're not fluent because you can translate a sentence back into English; you're fluent to the extent that you don't have to.”

And you’re not a good reader because you can translate letters into sounds; you’re a good reader to the extent you don’t have to. Does that mean we shouldn’t teach phonics?

Anonymous said...

It's all just a big misunderstanding. When you said, "Here are the questions I would ask to assess whether a child completely understands the text," I mistakenly thought that you would actually ask a real child those questions to assess whether she completely understood the text. I'm glad we're in agreement that, in fact, you wouldn't, and it would be quite silly to do so.

Katharine Beals said...

It is not silly to check in with a child from time to time with this sort of question to gauge their understanding and provide appropriate support or (if the child turns out to be struggling) switch to an easier text.

But, yes, the point of this list of questions was to answer the question posed by the title of this blog post, and to demonstrate that it's pretty straightforward to enumerate what "complete linguistic comprehension" means for a particular text.

Auntie Ann said...

Asking some of those question, especially verbally in a one-on-one tutoring-style session is also a good way to *teach* linguistic comprehension. If a student can not follow the text, teaching them how clauses refer to one another, how to find the compliment of a pronoun, and discussing new vocabulary will help improve comprehension. I could easily see a teacher asking a classroom some of those questions as they read through a text, discussing some of them, and putting some on a quiz or test.

I have plans to go through this book: Golden Book of the American Revolution with our 12 year old, in part for the information it contains, and in part to parse selected sentences to show how the language works and improve his knowledge of grammar.

Anonymous said...

Re: Shakespeare and Bible passages that are more likely to be understood over time: pretty much any chapter of the Bible, and pretty much any individual scene in a Shakespeare play. It's not that the individual words can't be learned in relatively short order, but that the cumulative meaning is hard to grasp until the word meanings are embedded in one's repertoire of known words. I should add that I meant the King James version; there are more modern translations that use much easier vocabulary, not just more modern but more widely-used.

SJ said...

Most kids in the middle grades already speak English, so I'm not quite getting the foreign language analogies. Sometimes when I read in a foreign language, I'm trying to get close enough for jazz (newspaper). When I'm attempting to consciously improve my language skills, I'm reading fairly closely. And really, these questions are _nothing_ compared to an upper level language class in Hebrew or Latin. Now _that_ is close reading.

I think we did a terrible disservice in abandoning the upper level readers in favour of "living books." It's much more workable to read an excerpt closely, rather than an entire book.

Katharine Beals said...

Anonymous@9:44pm,

I'm intrigued by what you write about full linguistic comprehension vis a vis the Bible and Shakespeare, but I don't really comprehend it. I wonder if you will indulge me a bit more with this. Below is the KJV translation of the first verses of the book of Ruth. Can you show me how it--or some other passage from Ruth (my daughter will be reading Ruth today, hence this choice)--illustrate what you say about full linguistic comprehension.

"Now it came to pass, in the days when the judges 1ruled, that there was a famine in the land. And a certain man of Bethlehem, Judah, went to 2dwell in the country of Moab, he and his wife and his two sons.
2 The name of the man was Elimelech, the name of his wife was Naomi, and the names of his two sons were Mahlon and Chilion—Ephrathites of Bethlehem, Judah. And they went to the country of Moab and remained there.
3 Then Elimelech, Naomi’s husband, died; and she was left, and her two sons.
4 Now they took wives of the women of Moab: the name of the one was Orpah, and the name of the other Ruth. And they 3dwelt there about ten years.
5 Then both Mahlon and Chilion also died; so the woman survived her two sons and her husband."

Anonymous said...

I'll give it a shot when I get home from work . . .

Anonymous said...

"Now it came to pass, in the days when the judges 1ruled, that there was a famine in the land. And a certain man of Bethlehem, Judah, went to 2dwell in the country of Moab, he and his wife and his two sons.
2 The name of the man was Elimelech, the name of his wife was Naomi, and the names of his two sons were Mahlon and Chilion—Ephrathites of Bethlehem, Judah. And they went to the country of Moab and remained there.
3 Then Elimelech, Naomi’s husband, died; and she was left, and her two sons.
4 Now they took wives of the women of Moab: the name of the one was Orpah, and the name of the other Ruth. And they 3dwelt there about ten years.
5 Then both Mahlon and Chilion also died; so the woman survived her two sons and her husband."

Here are some words/phrases from the passage above that I could be taught, and would remember for a while, but which are foreign enough so that hearing them many times would be necessary before I would read this passage with fluency:

Came to pass
famine
dwell
took wives
survived (in the sense of "lived longer than")

So, not one of the harder passages in the Bible. But for a 7 or 8 year old, not one that can be read without some vocab instruction, and one that will not be simple to read even with the vocab instruction.

Katharine Beals said...

Anonymous,
Thanks for taking time with this. I agree with you about what it takes to read with fluency. And fluency is one route to complete linguistic understanding--the most pleasant one, but, I'd argue, not the only one.