Two seemingly unrelated factors stimulated me to write this post. One print-based, the other ‘ether’ based.
At the back-end of last year (2015) I was reading an article in The Times Higher Educational Supplement (29-10 – 4.11.2015 volume 2227 pp 6-7) about the fall in success rates of grant applications to UK research councils. A sub-heading of the article, Reviewers are stretched, pointed out that the research councils, like journals are struggling to find enough reviewers. A week later I came across this ‘conversation’ on Twitter.
Coincidentally I had just accepted an invitation to review a grant application, having just submitted one the week before and as I was, in my Editorial capacity, inviting Reviewer number 9 to cast judgement on a submission to Annals of Applied Biology, I felt I ought to respond to the invitation. I had tried desperately hard to wriggle out of reviewing the grant application, pointing out that I had been a Co-Investigator with the Principal Investigator on a earlier grant, that I had been the PhD supervisor of one of the Co-Is and published three papers with them and that I had taught the other Co-I. I think you could say that I knew them very well indeed.
This image ‘borrowed’ from the University of Houston http://www.uh.edu/research/compliance/coi/
So full disclosure of the facts and surely, I thought, enough conflicts of interest there to rule me out! To my surprise the Research Council involved, replied saying that as long as I declared this on the review form, they were quite happy for me to referee the proposal. I am used to the Research Councils being very flexible when it comes to the time allocated to do a review, they find it so difficult to get people to agree that they are very willing extend deadlines for several weeks if you promise that you will eventually deliver a report. This response to what I saw as a major conflict of interest was, however, somewhat surprising. To say that I was gobsmacked* is a bit of an understatement, but as this had been my main reason for not accepting the task, I felt honour bound to do the review and make a recommendation.
So what exactly is a conflict of interest and how worried should we be about their potential to influence our responses in a scientific context? Here are a couple of definitions that I gleaned from the web.
A situation that has the potential to undermine the impartiality of a person because of the possibility of a clash between the person’s self-interest and professional interest or public interest.
The real or apparent conflict between one’s personal interest in a matter and one’s duty to another or to the public in general regarding the same matter.
Webster’s New World Law Dictionary © 2010 by Wiley Publishing, Inc., Hoboken, New Jersey.
So pretty clear, if there is a connection, personal or business, with the person(s) that you are asked to comment on, or their work, then you have a potential conflict of interest.
I am, as some of you may know, Editor-in-Chief of Annals of Applied Biology and was, until last year, Editor-in-Chief of Insect Conservation & Diversity, so potential conflicts of interest have been part of my life for many years. How do we handle this as Editors and Handling Editors? In the letter that our Handling Editors at Annals send out to potential reviewers we state:
“If you agree to referee the paper please declare to the handling editor if you have:
published papers, submitted grant proposals, or supervised students with an author. It is essential that you declare this information before reviewing the manuscript.”
So very clear what we mean by a conflict of interest here. But given the shortage of people willing to review papers and note that it is common practice to initially invite four to get the minimum two that we aspire to for a fair and balanced decision, how fussy can we afford to be? On one memorable occasion, I once had to approach thirteen potential reviewers before I found two willing victims! In the UK, if you are in a specialist field such as applied entomology, you are almost certain to know just about everyone who works in that area, either personally or by reputation. Given the virtual insistence these days by the national grant funders on collaborative projects, you also have a fairly high probability of having been in joint grant application with many of them as well. Most journals now ask for suggestions of preferred and non-preferred reviewers when you submit a paper. These are highly likely to be people you know personally, and your preferred reviewers are also unlikely to be people you think will regard your work unfavourably. Is this a conflict of interest? As an Editor you can take notice of these names, often ignoring them because you suspect that the preferred reviewers have been chosen because of the possibility of them delivering a favourable review. You then have a decision to make as to which reviewers to select; do you read through the references and see who has been cited most and pick them, do you resort Web of Science and look for publications in a similar area involving the same systems or use the keywords in your particular Editorial manager system? Whichever way you go you have a high chance of picking people who know the author(s) and/or have worked with them at some stage, but if you want an expert opinion you are pretty much stuck with those choices. It is further complicated by the fact that some people are more likely to respond in the affirmative than others, so your choice is narrowed still further.
As a potential reviewer receiving an invitation from a journal that doesn’t ask you for as much information as the Annals of Applied Biology does (looking back at last year’s 50+ requests to review that I receive, it seems that we at the Annals are much more up-front in this respect than other journals) what constitutes a conflict of interest? Even if you don’t know the author personally, which if they are from the USA** or other country where entomologists are still fairly numerous is quite likely, does the fact that they have cited you a lot constitute a conflict of interest? A favourable review may ensure publication and add to your citation index. On the other hand, if the paper doesn’t cite you when you feel it should, is that also a conflict of interest and what about when it cites you unfavourably, are you sure that you will write your review impartially? Should we also ask if you have received a favourable or non-favourable disclosed review from the author(s) for one of your own papers? I can’t help but think that having had a favourable review from someone, you are, despite how impartial you consider yourself to be, likely to look more kindly on a paper from that person than one from someone who has said that your paper should be rejected. I know there are a number of people who feel that open review is the way forward but I am not the only one that thinks it just adds to the conflicts of interest dilemma.
Leaving those issues aside. What about if you have answered yes to the questions posed by the Annals in that you have published papers, submitted grant proposals, or supervised students with an author? Interestingly I have just noticed that we don’t ask whether the potential reviewer has supervised or taught one of the authors. As someone whom to date, has supervised 50 PhD students, more than 130 MSc project students and about 150 undergraduate project students, not to mention the 1000+ students whose names I learnt when teaching them, this is yet another area of potential conflict of interest. The last time I co-authored a paper with my PhD supervisor was 1989, is that still a conflict of interest in 2016? I freely admit that I have reviewed more than one of his papers and even recommended rejection once or twice (I don’t think he reads my blog :-)).
I am not saying that our current review system is fatally flawed, in fact I think it works quite well and feel that the open reviewing system advocated by some has just as many, if not more, opportunities for potential conflicts of interest to arise. See this post by Dynamic Ecology which puts the case for pre-publication review very clearly.
When does it stop being a conflict of interest to review a paper by one of your former PhD students? Five years, ten years or longer? I will put my hand up now and admit that I have reviewed papers written by former students, but only after what I consider a decent five-year interval since the last co-authorship. What about co-authors who were not students or RAs? Often you end up on multi-author papers arising from working groups, do you apply the same rules in those cases when asked to review a paper or a grant proposal? I don’t but should I?
When it comes to PhD examinations the situation is even more acute, at least in my case. I have examined about 50 PhD students and had 48 of my own examined in return. As far as I can recall, of those that I have examined only three of them have been students of people who I didn’t know personally. Of the students of mine examined, I think three of them were by people I didn’t know personally (these were bird projects), but the co-supervisor knew them. This of course is totally understandable within the UK system, where the usual PhD viva panel consists of an internal examiner, an external examiner and, increasingly more common, an independent chair. You are hardly likely to choose examiners you don’t know to give your students a grilling. You choose someone who is fair and has a good reputation, which generally speaking means someone you know personally. Given the paltry fee paid by UK universities for a PhD examination***, friends are much more likely to agree to do the job than total strangers 🙂
Should we have a system where only examiners who have no personal contact with the supervisors are allowed? This would almost certainly mean that in the UK, all examiners would have to come from overseas; I suspect that the Universities would baulk at the increased costs associated with such a system. They are already very stingy when it comes to travel and accommodation costs for UK examiners so the added cost of getting someone from across the water is unlikely to appeal.
Where does this leave us? Not much further forward I suspect. I think, that as scientists, we all regard ourselves as being able to decide if, and when a real conflict of interest is likely to arise and would, I hope, inform the person(s) requesting the review of the pertinent facts.
*for non-native English speakers or English speakers from other parts of the world, a literal translation would be “like being hit unexpectedly in the mouth” 🙂
**although I have just noticed that a paper I have accepted an invitation to review from an American journal, has, hidden away in the middle of the author list, someone who did their PhD in the same research group as me at the same time, and with whom I have spent many a night drinking pints of Courage beer in The Mitre pub, on the Earlham Road in Norwich 🙂
***approximately £200 if you are lucky – say the viva takes 3 hours minimum, plus you have to read the thesis, say 150 pages, so even if you read very quickly you are looking at another 5 hours minimum, more likely closer to ten hours as you have to take notes as you go along, then add on an hour for the report and four hours travel time, that makes a total of 17 hours or so giving you an hourly rate of about £12, and that is before tax 🙂