Failure analysis and exam design

The basic quantitative reasoning and introductory calculus classes at NYU are (or perhaps were) organized into small teaching teams: a graduate student, a postdoctoral instructor, and a professor would work together, teaching separate lectures, but meeting regularly to agree on common topics, quizzes, homeworks, and exams. The system worked pretty well, and I learned a tremendous amount from Al Novikoff by co-teaching a class on "Understanding the Mathematical Patterns in Nature" with him during my first year as a Courant instructor. Al is very good at designing fair, gradeable quizzes and exams, because he designs toward student failure modes. For each question we wrote, Al would always ask "what are the ways we expect students to get this wrong?" In the discussion that ensued, the group usually figured out what the common wrong answers would be, though there were always a couple surprises that we missed. We could then decide whether the question was okay, too easy, too hard, or just too hard to grade. I still try to predict failure modes when I write my exams, but it's hard to get it right. Part of the intrinsic difficulty --- indeed, part of the intrinsic difficulty of most teaching activities --- is that I am not my students, and I have to constantly revise my mental model of what students will know or not know, what they will find difficult or easy. And sometimes, the really hard part of understanding how students are thinking is understanding that there is something to think about! We get too close to our material, and forget sometimes that just because a student has had multivariable calculus does not mean they understand logarithms as thoroughly as we might take for granted. Teaching is a learning experience in many ways.

Comments

blog comments powered by Disqus
Fork me on GitHub