External Examining – what is it and why do it?

I am now in my first year as the external examiner for the BSc Zoology degree programme at University College Dublin (UCD).  Some of you will know what this entails from personal experience, others will not even be aware that the process exists.  For my sins I have been involved in external examining university courses since 1997.  So exactly what does being an external examiner involve and why do I do it?

According to the UCD handbook ….

“Extern (sic) Examiners play a vital role in assisting the University in fulfilling its obligations of assuring the academic standards and integrity of its modules leading to awards.

1.2. Extern Examiners provide an important consultative and advisory function in supporting the development of modules as well as the enhancement of teaching, learning and assessment practices.

1.3. UCD appoints Extern Examiners that are internationally recognised experts in their subject areas and are from institutions with the highest academic reputations.”

Looking across UK universities and reading their descriptions of the duties of external examiners we find a common message that as an external examiner your duties involve the review of  all summative assessment (ie check that the exam papers are fair and understandable) prior to students being assessed and the requirement to submit an annual report, based upon your professional judgement, about  a number of aspects of the programme.  These being answers to the following questions; are the academic standards set for the programme awards appropriate, how rigorous are the assessment processes, are the students assessed fairly and equitably and are the standards and student achievements comparable with those at other similar institutions.  You are also asked to identify examples of good practice and to suggest areas where improvements or changes would be beneficial.

I got involved with external examining purely by accident; back in 1997 I was asked to validate a new degree programme in Life Sciences at what was then The Roehampton Institute of Education, now Roehampton University.   Validating a degree programme involves assessing the course materials, programme specifications, learning outcomes, projected demand, resource availability, goodness of fit to other programmes etc., etc.  Having agreed that there was a case for them to start the new programme I found myself two years later being asked if I would like to be the external examiner!  Obviously, having validated the degree in the first place I felt that it would be churlish to refuse 😉

I have since that first appointment been an external examiner at the University of Leeds (BSc Biological Sciences), York University (Environmental Sciences (Undergraduate and Masters), University College London (BSc Biology), Writtle College (MSc Conservation Management), University of Cumbria (MSc Forest Ecosystem Management and Conservation Biology) and now BSc Zoology at UCD.   I have also validated and re-validated several degrees at other universities but managed to suggest alternative external examiners for those 😉

Appointments are usually made for three years but in many cases extended to four years and very occasionally to five.  So what does it actually involve in practice?  Before your annual, or in some cases biennial visit, you will be sent exam papers to check if the questions are appropriate and to proof-read (you will be amazed at how many spelling and grammatical mistakes can creep into exam papers).  You should also receive module descriptors and the material that students are given before being set assignments and also details of how scripts are graded and marks allocated.  Some places will send you examples of course work and student project dissertations before your visit so that you can make a start on your evaluation ahead of time.  On arrival at the Department, where you will spend the next two to three days,  you should be allocated a space (a secure office/room) where you will be faced with a pile of paperwork to go through.  This can vary greatly, depending on the number of students in a cohort and the number of assessments that count towards the final degree mark.  Those programmes where all years of study contribute to the final grade will have more for you to look at.  Your main duties usually involve examining the paperwork associated with those students graduating in that year, so final year exam scripts, examples of coursework assignments and of course their final year project dissertations/theses.  My UCD pile was about average.

When I first started as an external examiner,  exit vivas for students were fairly wide-spread and in some cases were used to moderate the degree results of individual students.  This practice has pretty much disappeared and in fact it is now rare to actually get to talk to students on a one-one basis. I totally agree that it was unfair to classify a student’s degree on the basis of a twenty-minute interview, but on the other hand feel that you learn a great deal about the course by talking to students about their projects and experiences.  The staff who deliver a course look at things from a very different viewpoint than those who are on the receiving end.  At some universities students are invited along to an informal gathering with the external examiner(s) (drinks and nibbles provided as an inducement) so that at least some idea of what the students think can be gained; unfortunately take-up can be very low and those who do turn up are certainly not a random sample.   At many universities now, final grades are not changeable and final degree outcomes are determined by an algorithm.  The practice of speaking up at the final exam board in the hopes of moving a student from one grade to another is a thing of the past, and perhaps rightly so as it was very subjective and dependent on the student having a passionate advocate.  Now, if you as an external feel that items within a module were over-marked or under-marked, you are there in an advisory position and not in an executive position, and if any changes to marks are made it will be for the module and not for individual students.

At UCD I was able to talk face to face with half the zoology cohort,  an experience that I found interesting, informative and enjoyable and which did  not have any life-changing effects on the students.  Their final marks were not affected.

At the end of your visit you will be invited along to the Final Examination Board where the marks are presented to the assembled faculty and you will be invited to say a few words about general standards and highlight any concerns that you wish to be addressed before your next visit.  You should also highlight the good things that you noticed and say thank you for the hospitality that you will have received.   Incidentally you will also have to provide a written report to the host university before you receive your fee although you do generally get your travel expenses back fairly promptly.

So why do I agree to be an external examiner?  First,  I find it very useful to be able to compare practices across institutions, I often find examples of good practice which I take note of and apply in my own teaching or tell others in my home department about.  Sometimes I come across great exam questions which I shamelessly borrow to inflict on my own students!  Secondly I think that it is something one should do as a ‘ good academic citizen’  and finally,  you are also usually well treated by the staff of the Department you are visiting and get a least one opportunity to be wined and dined and to meet staff members in an informal setting.

So if you are asked to become an external examiner say yes.  It is not that onerous and it is certainly educational.  If you do decide to become an external examiner I have three pieces of advice for you.

Don’t ask for things to be changed just for the sake of it – for example, the way exam questions are posed in a particular way is generally for a reason.  The people who taught the course know what the students are expecting.  I remember one external examiner asking me to reword an exam question which I did, the result of which was that the students did not answer the question because they did not ‘get’ what it was about.

Don’t be antagonistic – I treat external examining much the same way as I do PhD examining – a two-way process that should have positive outcomes for all involved.  You might not agree with the way things are done in the Department, but find out why it is done that way; if there is no reasonable answer then gently point out why you think it would be better done your way.  If you have valid criticisms to make do so, but do make sure to mention the things you thought were excellent.  Talk to staff and find out if there are things that they feel should be changed by the university or Department.  As the external examiner your report is read by the university administrators and usually published on the University’s website.    You do have the power to make a difference.

Finally, talk to the students.  Even if the Department you are visiting does not have student vivas, ask them to arrange a meeting, either with a group or preferably individual students, so that you can hear things from both sides.

Change will not happen instantaneously, even if you feel it should.   Look, listen and learn and it will be a pleasant three or four years for you and your hosts and at the end of it all you will hopefully feel that you have achieved something worthwhile.

 

 

 

Entomological classics – The insect olfactometer

In 1924, Norman McIndoo (1881-1956) an entomologist based at the Fruit Insect Investigation Department in the USDA Bureau of Entomology in Washington DC was instructed by his boss Dr. A.L. Quaintance, to make a study of insect repellents and attractants.   After two years of frustrated experimentation McIndoo invented a piece of apparatus that would revolutionise the study of insect behaviour, the Y-tube olfactometer (McIndoo, 1926) . He freely admitted in his paper that he had borrowed the name from the Zwaademaker olfactometer (Zwaademaker, 1889) a device used to test the sense of smell in humans.  As you can see however, his apparatus bore no resemblance to that of Zwaademaker.

   

McIndoo ‘s apparatus was first used to find out whether Colorado potato beetles (Leptinotarsa decemlineata) responded to the odour of the potato plants. The beetles were placed in a dark bottle in a light-tight box, the bottle being attached to the stem of the Y-tube by a tube through which the beetles were able to move, at first being attracted to the light. Once they reached the junction of the Y they then had to make a choice between the two forks this time using their sense of smell. A pump was used to draw air from the two forks, one of which was connected to a jar containing a potato plant, the other which held the control substance. In theory, once at the fork the beetles were confronted with two streams of air, one smelling of potato, the other being odourless. McIndoo was indeed able to show that about 70% of the beetles responded positively to the odour produced by the potatoes. He also showed that the beetles responded to extracts made from the foliage of a number of different host plants.  He briefly mentions in the paper that the beetles were able to tell the opposite sex by smell and that the males would follow sexually mature females. He had accidentally discovered insect sex pheromones but did not realise it at the time. In the last part of his paper he provides data showing that other insect species, including Lepidoptera, were also able to respond to host plant odours.  The Y-tube olfactometer and the closely related T-tube olfactometers soon became the accepted way to test insect response to odours and are widely used in laboratories around the world to this day, for example http://weslaco.tamu.edu/research-programs/entomology/subtropical/behavior/ and http://sciencebykathy.wordpress.com/

http://openi.nlm.nih.gov/detailedresult.php?img=3422343_pone.0043607.g005&req=4

They do however have some limitations; there is a tendency for turbulence to occur at the junction of the Y- and T-tubes which means that there is some mixing of the test odours and this means that there is not a clearly delineated odour field into which the insects can enter, leave and re-enter if they so wish. In 1970, Jan Pettersson from the Swedish University of Agricultural Sciences at Uppsala, invented the four-way olfactometer with which to test the existence of a sex pheromone in the aphid Schizaphis borealis (Pettersson, 1970).

  

The four-way olfactometer provides a neutral central zone which is surrounded by four very distinct odour boundaries which the test insects can enter, sample the odour and then either stay or leave and move into another area of the apparatus. Louise Vet and colleagues (Vet et al., 1983) from the University of Leiden added some modifications to the original Pettersson version, with which to study the behaviour of aphids and their parasitoids.

 The four-way olfactometer, whether a Pettersson or Vet version, or a modification of the two, is now regarded as the ‘gold’ standard and is used very widely around the world.

http://www.nrcb.res.in/gallery8.html

It is certainly our research group’s favoured version and we use it for testing the responses of aphids, hymenopteran parasitoids, lepidoptera and beetles to a range of odours (Trewhella et al., 1997; Leahy et al., 2007; Pope et al., 2012). We are currently using mini-versions to test the olfactory responses of predatory mites. Watch this space.

References

Leahy, M.J.A., Oliver, T.H., & Leather, S.R. (2007) Feeding behaviour of the black pine beetle, Hylastes ater (Coleoptera: Scolytidae). Agricultural and Forest Entomology, 9, 115-124. http://onlinelibrary.wiley.com/doi/10.1111/j.1461-9563.2007.00328.x/full

McIndoo, N.E. (1926) An insect olfactometer. Journal of Economic Entomology, 19, 545-571

Pope, T.W., Girling, R.D., Staley, J.T., Trigodet, B., Wright, D.J., Leather, S.R., Van Emden, H.F., & Poppy, G.M. (2012) Effects of organic and conventional fertilizer treatments on host selection by the aphid parasitoid Diaeretiella rapae. Journal of Applied Entomology, 136, 445-455. http://onlinelibrary.wiley.com/doi/10.1111/j.1439-0418.2011.01667.x/full

Pettersson, J. (1970). An aphid sex attractant I Biological studies. Entomologia Scandinavica 1: 63-73.

Sanford, E.C. (1891) Laboratory course in physiological psychology. American Journal of Psychology, 4, 141-155, http://psychclassics.yorku.ca/Sanford/course2.htm

Trewhella, K.E., Leather, S.R., & Day, K.R. (1997) The effect of constitutive resistance in lodgepole pine (Pinus contorta) and Scots pine (P. sylvestris) on oviposition by three pine feeding herbivores. Bulletin of Entomological Research, 87, 81-88. http://journals.cambridge.org/action/displayAbstract?fromPage=online&aid=2497592

Vet, L.E.M., Van Lenteren, J.C., Heymans, M., & Meelis, E. (1983) An airflow olfactometer for measuring olfactory responses of hymenopterous parasitoids and other small insects. Physiological Entomology, 8, 97-106. http://onlinelibrary.wiley.com/doi/10.1111/j.1365-3032.1983.tb00338.x/abstract

Zwaademaker, H. (1889) On measurement of the sense of smell in clinical examination. The Lancet, 133, 1300-1302

 

Sitting in judgment – Calling the shots – Peer review – A personal view

I have just finished refereeing four papers from four different journals with different approaches to the peer review system. As a result I now feel moved to share my thoughts about the way journals ask us to review papers. In earlier posts I have written about my experiences as an editor and also about why and how many papers we should referee as good citizens of the scientific community. The four journals for which I have just acted as a reviewer are Ideas in Ecology & Evolution, Physiological Entomology, and Journal of Insect Behavior and Animal Behaviour. Ideas in Ecology & Evolution operates a completely open review process and allows authors, should they wish, to solicit their own reviews in exchange for a reduced processing charge. As a reviewer you can, as well as submitting your review, submit a response article free of charge. As someone who is usually unable to publish in open access journals as my usual funding sources rarely, if ever, provide funding for page and processing charges, I took the opportunity to do so and am now waiting to see if my offering will get published 😉 Animal Behaviour on the other hand operates a double-blind system where the referee is supposedly as blind as the author(s).

This is achieved by removing the names and addresses of the authors. As however, the references and acknowledgements are usually left untouched, it is, in my field, comparatively easy to work out who wrote the paper with some confidence.  The other two journals give you the option of revealing your identity should you so wish.

I have for my sins reviewed a lot of papers over the years; sadly as a data nerd, I have kept a record not only of how many papers I review a year but for which journals (I really should get a life).

In all those years last week was the first time that I have ever unambiguously revealed my identity. I say unambiguously, because, during the 1990s when Oikos used to give you the opportunity to sign your review, I did use to scrawl my totally illegible signature at the bottom of the page, confident that no-one could actually read it! So why do I opt for anonymity when reviewing papers?

I am confident that I would write exactly the same review if I did it openly and in fact sometimes I am sure that the authors can guess that I was the reviewer. I choose to be anonymous because I feel that a generally open review system would tend to make some, if not most reviewers, pull their punches. How many young post-docs would dare to reject a paper by an established academic knowing that they could be on their next interview panel or reviewing their next grant application or be the editor of the journal they submit their next paper to?  We are all supposed to be dispassionate scientists able to take constructive criticism, but even the most laid-back of us will probably remember the name of the young whippersnapper who dared reject our paper.  I am pretty sure that when I was at the beginning of my career I would never have dared criticize, let alone reject, a Southwood or Lawton paper in an open review system.   As an Editor who occasionally submits papers to my own journal I have to take this ‘fear’ factor into account and have rejected my own papers when for other authors I would have allowed a resubmission.  In my experience as an Editor, I rarely see reviewers using the confidential comments to the editor section to say something very different from the review they have provided for the authors to see. So I think that the traditional system works pretty well.

I also think that by revealing our identities to authors we could end up with a you scratch my back I will scratch yours situation in that if A gives B a favourable review then B will in turn give A one back and the whole system will be subverted.

So in conclusion until someone proves otherwise I will continue to remain anonymous except in exceptional circumstances.

 

Post script

For a less personal and more scientific viewpoint see this article

http://blogs.scientificamerican.com/information-culture/2014/04/19/introduction-to-traditional-peer-review/

and for an alternative viewpoint see this from Dynamic Ecology

https://dynamicecology.wordpress.com/tag/peer-review/

Post-post script

In case you are wondering why I was so lazy in 1991, I was desperately applying for jobs in the university sector.