From the “questions to consider” section of LRNT 504 Instructional Design for Technology-Mediated Learning
By now you, as a learner for a great number of years, must have written numerous tests – into the hundreds, for sure. Were most of these criterion-referenced tests? Did you know the criteria (or objectives) being tested? Assuming not, would it have been better if you had?
Criterion-based courses is defined by Wikipedia as: ” one that provides for translating test scores into a statement about the behaviour to be expected of a person with that score or their relationship to a specified subject matter. Most tests and quizzes written by school teachers are criterion-referenced tests. The objective is simply to see whether the student has learned the material”.
It has only been about two years since I really started to consider how a test is created. In a lot of my courses during my undergraduate I had written papers rather than multiple choice tests. Yet, I still clearly remember the Diploma Examinations for grade 12 and the Provincial Achievement Tests (PAT) of grades 3, 6 and 9.
For a lot of my K-12 education I was told you couldn’t study for a multiple choice test for reader comprehension–which isn’t true exactly. You can certainly learn how to answer the multiple choice questions. I would posit that the best way to learn would be to read often, but that’s more opinion than fact.
I’ve recently been reading Nassim Nicholas Taleb’s book, “The Black Swan: The impact of the highly improbable” (Poor Professor Bill Muirhead, he really is losing the battle on impact–but he’s proven right on just about everything else). In the book, Taleb (2010) described a commonality of the study of success. Everywhere you turn you see examples of a variation on a theme, say “ten steps to becoming a millionaire”. In this genre of books, call it self-improvement, business, whatever, there is an overwhelming attention to minding certain steps as though becoming a millionaire can be arrived at through formulaic steps. There are very few books written by people who have tried and failed and yet we may learn more through the study of things that went wrong than we would with things that went right.
To tie these two concepts together in the context of reading comprehension what I am attempting to convince you of is that it may be best to study the areas that need improvement rather than the ways in which students succeed–and to be able to recognize the differences. The objectives of a typical reading comprehension test seem to be discerning whether a learner can encode and store information from a written test, and parse said text for meaning through metaphor. Wooley (2011) states that, “a number of researchers have reported that students with reading difficulties appear to have memory deficits that impact on their ability to effectively comprehend text”. If the problem with a student’s success can be contributed to an lack of cognitive architecture than the adherence to a cognitive-based learning theory would probably suit that learner best.
Lastly, I am not sold entirely on the usefulness of learning objectives. As an instructional designer, I plug them in and quietly agonize over the working of each objective, ensuring that it ties to (in the case of my work at the Safety Codes Council) the multiple-choice assessment that the student will take on completion of the learning material. To this end, we deal almost exclusively within the lower realms of Bloom’s Taxonomy. In contrast, as a student I almost never find them helpful I read them out of dutiful obligation but never feel more enlightened.
So, I conclude that knowing the course objectives and criteria is not helpful to me, at least not in a way I am able to cognitively acknowledge, but that does not mean it is not working for other people. Rather what is most helpful is recognizing the areas that a student struggles with and having appropriate support available to recover in those areas.
Woolley, G. (2011). Reading comprehension: Between facts or strategies. New York: Springer.
Talbeb, T, N., (2010). The black swan – the impact of the highly improbable. (2nd ed.). New York: Random House.