Question

From the “questions to consider” section of LRNT 504 Instructional Design for Technology-Mediated Learning 

By now you, as a learner for a great number of years, must have written numerous tests – into the hundreds, for sure. Were most of these criterion-referenced tests? Did you know the criteria (or objectives) being tested? Assuming not, would it have been better if you had?

 

Criterion-based courses is defined by Wikipedia as: ” one that provides for translating test scores into a statement about the behaviour to be expected of a person with that score or their relationship to a specified subject matter. Most tests and quizzes written by school teachers are criterion-referenced tests. The objective is simply to see whether the student has learned the material”.

It has only been about two years since I really started to consider how a test is created. In a lot of my courses during my undergraduate I had written papers rather than multiple choice tests. Yet, I still clearly remember the Diploma Examinations for grade 12 and the Provincial Achievement Tests (PAT) of grades 3, 6 and 9.

For a lot of my K-12 education I was told you couldn’t study for a multiple choice test for reader comprehension–which isn’t true exactly. You can certainly learn how to answer the multiple choice questions. I would posit that the best way to learn would be to read often, but that’s more opinion than fact. 

I’ve recently been reading Nassim Nicholas Taleb’s book, “The Black Swan: The impact of the highly improbable” (Poor Professor Bill Muirhead, he really is losing the battle on impact–but he’s proven right on just about everything else). In the book, Taleb (2010) described a commonality of the study of success. Everywhere you turn you see examples of a variation on a theme, say “ten steps to becoming a millionaire”. In this genre of books, call it self-improvement, business, whatever, there is an overwhelming attention to minding certain steps as though becoming a millionaire can be arrived at through formulaic steps. There are very few books written by people who have tried and failed and yet we may learn more through the study of things that went wrong than we would with things that went right. 

To tie these two concepts together in the context of reading comprehension what I am attempting to  convince you of is that it may be best to study the areas that need improvement rather than the ways in which students succeed–and to be able to recognize the differences. The objectives of a typical reading comprehension test seem to be discerning whether a learner can encode and store information from a written test, and parse said text for meaning through metaphor.  Wooley (2011) states that, “a number of researchers have reported that students with reading difficulties appear to have memory deficits that impact on their ability to effectively comprehend text”. If the problem with a student’s success can be contributed to an lack of cognitive architecture than the adherence to a cognitive-based learning theory would probably suit that learner best. 

Lastly, I am not sold entirely on the usefulness of learning objectives. As an instructional designer, I plug them in and quietly agonize over the working of each objective, ensuring that it ties to (in the case of my work at the Safety Codes Council) the multiple-choice assessment that the student will take on completion of the learning material. To this end, we deal almost exclusively within the lower realms of Bloom’s Taxonomy. In contrast, as a student I almost never find them helpful I read them out of dutiful obligation but never feel more enlightened. 

So, I conclude that knowing the course objectives and criteria is not helpful to me, at least not in a way I am able to cognitively acknowledge, but that does not mean it is not working for other people. Rather what is most helpful is recognizing the areas that a student struggles with and having appropriate support available to recover in those areas. 

Thoughts? Anyone?

 

 

Woolley, G. (2011). Reading comprehension: Between facts or strategies. New York: Springer.

Talbeb, T, N., (2010). The black swan – the impact of the highly improbable. (2nd ed.). New York: Random House.

Advertisements

Assessment design in an online environment

Recently an assignment got me thinking about what it is to work individually when you are in an online environment. When I was in my undergraduate it was very clearly explained to us during lectures what constituted academic dishonesty. This included having another student read your work prior to submission–but not in all cases. There were also projects that involved peer editing or group work. In some cases collaboration was encouraged in some cases it was not. There was a list of policies presented to us in each syllabus at the beginning of the course. However, nothing prevented students from gathering in the library or computer labs to talk over their assignments. There was also class time or office hours if you needed the support of the instructor.

With online learning things may be a little different. The library is social networking and the teacher’s office is through email. And areas that may be considered dishonest can become a little grey. After all the communication is written, and so copy and pasting sections to which you are referring to…Isn’t that getting help from other students? In order to avoid this I think it is helpful for the instructor or instructional designer to embed the expectations on individual and collaborative assignments, because policy can be overlooked or misunderstood. 

 Tony Bates on cheating: “Cheating is often the result of a poor educational process or experience. Once again, this comes down to the distinction between learning as transferring information vs learning as a developmental process. If, as I do, you believe education is a developmental process, it is the student in the end who loses from cheating, because they have missed the point of the exercise, which is self development and growth.”

I leave you with a very entertaining article written by a man who is said to “be writing your students work”.