Having spent a considerable amount of the last month or so invigilating and marking exams, not to mention doing my stint as external examiner for the MSc in Conservation Management at Writtle College I have started to think about what is that we expect to achieve by setting exams. In fact, I have even begun to wonder if we need exams at all. Do we just set exams because that it is the way it has always being or do they tell us something that other assessment methods can’t?
I have used the word easy in my title, which at first sight might seem a little contradictory. After all, as an examiner, there is the annual slog of producing ‘novel’ questions, together with a model or indicative answer, against which to mark, and then there is the curse of marking X number of scripts in a very short time, making sure that you grade the answers according to whichever criteria your department or institution has opted for. Not exactly a stress-free occupation and made worse by the fact that inevitably there is an incredibly short turnaround time required so that the students don’t have to wait too long to hear their fate. Marking is definitely one of my least favourite things.
We should also of course, not forget the stress and anguish that most students will have gone through to produce the scripts that you, with a strict time limit to observe, will inevitably spend less time on than they deserve.
Why do we use exams, and in this case I mean the traditional, sit down and answer questions in a room full of other aspirants, mostly all perspiring nervously and hoping against hope that the topics the revised will be on the face-down script in front of them. The easy answer is that they tell us, the teachers, how much the student has understood of what they
have been taught and how good they are at retaining that information and regurgitating it on to a piece of paper. We would also argue that it tells us which students are also able to interpret and analyse what they have been taught and put it into context in a stressful situation. From that, plus usually some coursework and a dissertation or research project, we can then assign (in the UK at least) an arcane grade (First, Upper Second, Lower Second, Third, Pass or Fail or at MSc level, Distinction, Merit, Pass or Fail) that then determines the future of that student. On the student side, a written exam favours those with good memories, the ability to shrug off stress and write quickly, legibly and coherently. That said, after some 40-odd years of sitting and setting exams, I find that I can pretty much tell what final degree result a student will achieve just from teaching and interacting with them in class, without actually seeing any written work.
So why do we need exams, why not just go for a less stressful approach? Well obviously that would not work, because occasionally, even I am surprised by a result and more importantly, how would you assess if the student had attained the required learning outcomes? So you definitely need some form of assessment. In other words, some form of coursework testing, which arguably reflects the situations we face in real life. Coursework gives students the chance to be analytical, synoptic and reflective, to read around their subject and to meet deadlines. There are also arguments against this form of assessment. These include, marking (coursework actually requires you to give some feedback), direct cheating by copying and pasting from sources; plagiarism software helps, but doesn’t tell you if there has been collusion. In addition, it penalises the bright but lazy. I revelled in exams; coursework, which luckily in my degree, didn’t count towards my final degree, was, sad to say, something I expended only the minimum of effort on. It tended to get in the way of my social life. What are the plus points for exams? They are a good quick test that puts students on the spot, trains them to learn, to cope with stress and to time manage. On the other hand you might argue that what it really tests is memory, stress resilience and the ability to write with a pen. On the negative side, there is a great tendency for students, and I was no exception in my early years as an undergraduate, to analyse past exam papers and reduce the pressure on their memory centres, by question spotting and by avoiding revising any material from lecturers that they didn’t like or who are known to be hard markers. Thus you end up with students who may have passed their exams but have gaping holes in their knowledge base. You can argue that a written exam allows us to test synoptic and analytical thinking, but this could be tested just as effectively with coursework, as synoptic thinking is certainly not restricted to exams. A huge downside is the marking element. Do we do the students justice? Almost certainly not. A typical final year exam might require the student to answer three questions in three hours, so forty minutes per answer. If you are lucky and have a relatively small class, of thirty students, that results in 90 scripts in less than perfect hand-writing. You then probably have two weeks to turn the scripts around, including them being moderated, or second-marked by a colleague. Practices differ between institutions, but, in my experience, there is always some element of moderation/quality review, either by marking a sample of the scripts or in some places looking at very single exam answer. Suppose you allow a mere ten minute per script, that will take you 15 hours if you went at it non-stop. Some of my colleagues have several hundred scripts to mark – how can they possibly do them justice? How much rest do you give yourself between batches, how many scripts can you mark before your analytical ability is eroded? Should you go through and re-read every script after you have finished marking all the scripts?
So what alternatives are there? Some course modules at other universities and here at Harper Adams are assessed entirely by coursework and that raises the collusion/cheating concerns mentioned earlier. We can detect plagiarism using software such as Turnitin™, but how can we tell that someone else didn’t write the essay, prepare the presentation or write the report and of course you still have to mark the work! What about on-line assessment? On the entomology MSc that I run, some modules are tested entirely by Multiple Choice Questions on-line; administered and marked by the computer – how good is that 😉 It is also a good way of testing basic entomological factual knowledge but again it is a memory test and doesn’t really test application and synthesis. We also use short answer questions in conjunction with some on-line testing; this lets us test across a range of the learning outcomes, but again does not really allow much synthesis and analytical thought, and generates piles of marking, albeit somewhat easier, as mainly checking against lists. One of my colleagues is very much in favour of exams, as he has rather a jaundiced view of coursework, although it can give us an opportunity to assess different skill sets; oral presentation, team work, writing skills, analytical thought and in the case of entomology, identification skills. Unfortunately it does produce piles of marking, but hopefully also is good for deep learning which is what I think we want as an end result.
Another interesting method of assessment which I quite like the idea of, is an open book exam, where students are presented with some reading material some days or even weeks before the exam without being told what the questions will be, are allowed to take their annotated document in with them and are then presented with an exam paper with three questions, all of which must be answered. Having observed this in practice, I have now come up with an interesting variant which I hope to be able to get approved by our exam board. Students arrive in a computer room, they are presented with an exam paper, with, say six questions of which they have to answer three, and then allowed to use the internet to bolster their knowledge. They are only allowed to take notes using pen and paper. Two hours later they then move to a secure computer, with no internet access and spend the next two hours writing their exam. This is more akin to real life, where you solve problems that you know some basic background stuff about by checking sources, getting extra information and then synthesising that knowledge into an intelligible report. Of course it doesn’t around the marking problem.
My friend Professor Guy Poppy at The University of Southampton, has gone on record via Twitter, suggesting that we could test students at the end of their university course with a portfolio of work and a 15 minute oral examination to check that what is there is actually their own, work, basically a mini-PhD viva.
Is this a mad idea? Actually the Italians do all their exams orally and the Germans use a mixture of written and oral exams.
An oral exam could be a bit stressful for shy students and might be an advantage for extroverts. You would of course need a panel of at least three people to ensure fair play. A typical British university final year might have 150 students in it. So at 15 minutes a viva, which I think is probably too short, that would be almost 37 hours. So let’s assume that the Department has sixty academics, allocates three staff members per viva, so 20 vivas can be run at once. If we allow five minutes for marking that gives us twenty minutes per student which equals sixty vivas every hour. So even if we decide to let the students have a bit more time to impress the examiners, we can still easily examine the whole final year in a day and the academics will only need to take part in three or four oral exams. We used a similar system as part of the final year research project assessment when I was in the Life Sciences Department at Imperial College and that worked reasonably well. What about consistent standards and available expertise? You need a balanced department and also some people who have to circulate between panels to ensure similar standards, but even so, this could be a viable option. It does not however, do away with marking the portfolio of work during the preceding three or four years, but perhaps if modules are assessed with a mixture of on-line MCQ tests plus informal use of the usual assessment methods, so not requiring detailed marking, it might actually work and encourage deep learning.
So at the end of all this I don’t really have an answer. I find marking really tedious, but students and staff need the feedback to know if learning outcomes are being achieved. Exams are a pain for the students to do but even more of a pain for academic staff to mark! What I do quite like though is the idea of the short, sharp open book exam, despite the fact that it would need marking. I would welcome any ideas and also any examples of novel easy to administer and mark test/exams and alternative assessment methods that have worked for you.
Just as I was going to post this, Meg Dufy over at Dynamic Ecology posted a great post about teaching ideas and their possible assessment methods.