Tag Archives: referees

How not to respond to reviewers – even if it is Reviewer #3

I have been an Editor for many years, since 1993 to be precise, and am currently Editor –in-Chief of one journal and a Senior Editor of another as well as being on the Editorial Board of two other journals. On top of that, I review about 40 papers a year so have come across quite a lot of response to reviewers letters.  I have also, as the author of over 200 papers, written my own share of reviewer responses.  Yes, there are some reviewers who have caused my blood pressure to rise and engendered a desire to rend them limb from limb, and I have sometimes been tempted to reply to suggested comments with the phrase “up yours”, but sanity and common sense have prevailed.

Based on responses I have seen over the years, here are a few suggestions of what not to do, and what to do, to maximise the chances of your resubmitted paper being accepted.

First, take a deep breath, close the document, go for a walk and don’t read it again for at least 24 hours. A hastily anger-filled response will almost always result in a rejection. Avoid knee-jerk reactions at all costs.

Do not start your response by saying “Do not send our revised paper back to Reviewer 1 as it is clear that he clearly demonstrates a lack of knowledge or understanding of the study/subject area in general” This is likely to annoy the Editor who has gone to great pains to find a suitable reviewer for your paper and will most certainly annoy the reviewer when it is sent back to him/her as it will almost certainly be.   Much better to begin your response by thanking the Editor and reviewers for taking the time to consider your manuscript and making helpful suggestions.  Then respond carefully, comment by comment, as instructed in the letter from the Editor.

Do not respond to comments by baldly stating I/we disagree; politely state with good reasons, why you disagree.

Do not point out to the reviewer that she/he has made a spelling mistake.

Do not respond to the comment “This section is unclear” by saying “It is perfectly clear to us”. Ask yourself, why is it unclear to the reviewer?  One way to address the problem is by asking a colleague from another discipline if it is clear to them and then rewriting it when they say it isn’t.

If the reviewer challenges your description of random sampling as not being random because you did not use a random number generator do not respond by saying that this is how everyone you know describes it.

If challenged on your statistical analysis do not respond by saying “I/we have always done it this way”.  There may actually be a better way to do it, if you are sure there isn’t then explain why.

If challenged on the quality of your figures do not respond by saying this is the standard output from Excel.

Do not respond by saying “this was not raised as an issue by the reviewers of the previous journal we submitted our paper to”

If the Editor asks you to reduce the length of your Introduction or Discussion at least make some effort to do so, do not respond by saying “No, I/we think that the length is totally justified”.

If you really can’t bear to respond to the comments politely, then there are other journals, but do remember, there are only a finite number of willing expert reviewers and there is a very good chance that one of the reviewers of your paper that you have submitted to Journal Y will be the same as one you had for Journal X, so it makes sense to have made some changes to your original submission.

In the main, reviewers try to be constructive and helpful.  Remember they are unpaid, so are doing this for the good of the community and with a genuine desire to maintain the reputation of their discipline.  They are not doing it to annoy you.

 

 

4 Comments

Filed under Science writing

Conflicts of interest – are there ever any situations where there aren’t?

Conflict1

Two seemingly unrelated factors stimulated me to write this post. One print-based, the other ‘ether’ based.

At the back-end of last year (2015) I was reading an article in The Times Higher Educational Supplement (29-10 – 4.11.2015 volume 2227 pp 6-7) about the fall in success rates of grant applications to UK research councils.  A sub-heading of the article, Reviewers are stretched, pointed out that the research councils, like journals are struggling to find enough reviewers.  A week later I came across this ‘conversation’ on Twitter.

Conflict2

Coincidentally I had just accepted an invitation to review a grant application, having just submitted one the week before and as I was, in my Editorial capacity, inviting Reviewer number 9 to cast judgement on a submission to Annals of Applied Biology, I felt I ought to respond to the invitation. I had tried desperately hard to wriggle out of reviewing the grant application, pointing out that I had been a Co-Investigator with the Principal Investigator on a earlier grant, that I had been the PhD supervisor of one of the Co-Is and published three papers with them and that I had taught the other Co-I.  I think you could say that I knew them very well indeed.

Conflict3

This image ‘borrowed’ from the University of Houston http://www.uh.edu/research/compliance/coi/

So full disclosure of the facts and surely, I thought, enough conflicts of interest there to rule me out! To my surprise the Research Council involved, replied saying that as long as I declared this on the review form, they were quite happy for me to referee the proposal.  I am used to the Research Councils being very flexible when it comes to the time allocated to do a review, they find it so difficult to get people to agree that they are very willing extend deadlines for several weeks if you promise that you will eventually deliver a report.  This response to what I saw as a major conflict of interest was, however, somewhat surprising. To say that I was gobsmacked* is a bit of an understatement, but as this had been my main reason for not accepting the task, I felt honour bound to do the review and make a recommendation.

So what exactly is a conflict of interest and how worried should we be about their potential to influence our responses in a scientific context? Here are a couple of definitions that I gleaned from the web.

A situation that has the potential to undermine the impartiality of a person because of the possibility of a clash between the person’s self-interest and professional interest or public interest.

http://www.businessdictionary.com/definition/conflict-of-interest.html#ixzz3tjt5c5l7

The real or apparent conflict between one’s personal interest in a matter and one’s duty to another or to the public in general regarding the same matter.

Webster’s New World Law Dictionary © 2010 by Wiley Publishing, Inc., Hoboken, New Jersey.
So pretty clear, if there is a connection, personal or business, with the person(s) that you are asked to comment on, or their work, then you have a potential conflict of interest.

I am, as some of you may know, Editor-in-Chief of Annals of Applied Biology and was, until last year, Editor-in-Chief of Insect Conservation & Diversity, so potential conflicts of interest have been part of my life for many years.  How do we handle this as Editors and Handling Editors? In the letter that our Handling Editors at Annals send out to potential reviewers we state:

If you agree to referee the paper please declare to the handling editor if you have:

published papers, submitted grant proposals, or supervised students with an author. It is essential that you declare this information before reviewing the manuscript.”

So very clear what we mean by a conflict of interest here. But given the shortage of people willing to review papers and note that it is common practice to initially invite four to get the minimum two that we aspire to for a fair and balanced decision, how fussy can we afford to be? On one memorable occasion, I once had to approach thirteen potential reviewers before I found two willing victims!  In the UK, if you are in a specialist field such as applied entomology, you are almost certain to know just about everyone who works in that area, either personally or by reputation.  Given the virtual insistence these days by the national grant funders on collaborative projects, you also have a fairly high probability of having been in joint grant application with many of them as well.  Most journals now ask for suggestions of preferred and non-preferred reviewers when you submit a paper.  These are highly likely to be people you know personally, and your preferred reviewers are also unlikely to be people you think will regard your work unfavourably.  Is this a conflict of interest?  As an Editor you can take notice of these names, often ignoring them because you suspect that the preferred reviewers have been chosen because of the possibility of them delivering a favourable review.  You then have a decision to make as to which reviewers to select; do you read through the references and see who has been cited most and pick them, do you resort Web of Science and look for publications in a similar area involving the same systems or use the keywords in your particular Editorial manager system?  Whichever way you go you have a high chance of picking people who know the author(s) and/or have worked with them at some stage, but if you want an expert opinion you are pretty much stuck with those choices.  It is further complicated by the fact that some people are more likely to respond in the affirmative than others, so your choice is narrowed still further.

As a potential reviewer receiving an invitation from a journal that doesn’t ask you for as much information as the Annals of Applied Biology does (looking back at last year’s 50+ requests to review that I receive, it seems that we at the Annals are much more up-front in this respect than other journals) what constitutes a conflict of interest?  Even if you don’t know the author personally, which if they are from the USA** or other country where entomologists are still fairly numerous is quite likely, does the fact that they have cited you a lot constitute a conflict of interest?  A favourable review may ensure publication and add to your citation index.  On the other hand, if the paper doesn’t cite you when you feel it should, is that also a conflict of interest and what about when it cites you unfavourably, are you sure  that you will write your review impartially? Should we also ask if you have received a favourable or non-favourable disclosed review from the author(s) for one of your own papers?  I can’t help but think that having had a favourable review from someone, you are, despite how impartial you consider yourself to be, likely to look more kindly on a paper from that person than one from someone who has said that your paper should be rejected.  I know there are a number of people who feel that open review is the way forward but I am not the only one that thinks it just adds to the conflicts of interest dilemma.

Leaving those issues aside. What about if you have answered yes to the questions posed by the Annals in that you have published papers, submitted grant proposals, or supervised students with an author?  Interestingly I have just noticed that we don’t ask whether the potential reviewer has supervised or taught one of the authors.  As someone whom to date, has supervised 50 PhD students, more than 130 MSc project students and about 150 undergraduate project students, not to mention the 1000+ students whose names I learnt when teaching them, this is yet another area of potential conflict of interest. The last time I co-authored a paper with my PhD supervisor was 1989, is that still a conflict of interest in 2016? I freely admit that I have reviewed more than one of his papers and even recommended rejection once or twice (I don’t think he reads my blog :-)).

I am not saying that our current review system is fatally flawed, in fact I think it works quite well and feel that the open reviewing system advocated by some has just as many, if not more, opportunities for potential conflicts of interest to arise. See this post by Dynamic Ecology which puts the case for pre-publication review very clearly.

When does it stop being a conflict of interest to review a paper by one of your former PhD students? Five years, ten years or longer?  I will put my hand up now and admit that I have reviewed papers written by former students, but only after what I consider a decent five-year interval since the last co-authorship.  What about co-authors who were not students or RAs? Often you end up on multi-author papers arising from working groups, do you apply the same rules in those cases when asked to review a paper or a grant proposal?  I don’t but should I?

When it comes to PhD examinations the situation is even more acute, at least in my case. I have examined about 50 PhD students and had 48 of my own examined in return.  As far as I can recall, of those that I have examined only three of them have been students of people who I didn’t know personally.   Of the students of mine examined, I think three of them were by people I didn’t know personally (these were bird projects), but the co-supervisor knew them.  This of course is totally understandable within the UK system, where the usual PhD viva panel consists of an internal examiner, an external examiner and, increasingly more common, an independent chair.  You are hardly likely to choose examiners you don’t know to give your students a grilling.  You choose someone who is fair and has a good reputation, which generally speaking means someone you know personally.   Given the paltry fee paid by UK universities for a PhD examination***, friends are much more likely to agree to do the job than total strangers 🙂

Should we have a system where only examiners who have no personal contact with the supervisors are allowed? This would almost certainly mean that in the UK, all examiners would have to come from overseas; I suspect that the Universities would baulk at the increased costs associated with such a system.  They are already very stingy when it comes to travel and accommodation costs for UK examiners so the added cost of getting someone from across the water is unlikely to appeal.

Where does this leave us? Not much further forward I suspect. I think, that as scientists, we all regard ourselves as being able to decide if, and when a real conflict of interest is likely to arise and would, I hope, inform the person(s) requesting the review of the pertinent facts.

Conflict4

Footnotes

*for non-native English speakers or English speakers from other parts of the world, a literal translation would be “like being hit unexpectedly in the mouth” 🙂

**although I have just noticed that a paper I have accepted an invitation to review from an American journal, has, hidden away in the middle of the author list, someone who did their PhD in the same research group as me at the same time, and with whom I have spent many a night drinking pints of Courage beer in The Mitre pub, on the Earlham Road in Norwich 🙂

***approximately £200 if you are lucky – say the viva takes 3 hours minimum, plus you have to read the thesis, say 150 pages, so even if you read very quickly you are looking at another 5 hours minimum, more likely closer to ten hours as you have to take notes as you go along, then add on an hour for the report and four hours travel time, that makes a total of 17 hours or so giving you an hourly rate of about £12, and that is before tax 🙂

 

3 Comments

Filed under Uncategorized

The Three Rs of Science – Reading, Writing and Reviewing

And before anyone jumps in and says there are 4Rs in Science i.e. Reading, Research, Writing and Reviewing, I am including research as part of writing as without research you would have nothing to write about.  This post is mainly about writing for publication as I have written about refereeing and reading in earlier posts.  Almost twenty-five years ago I designed and implemented a scientific paper writing class for the undergraduate course that I used to run at Imperial College; later I re-tooled it as part of our postgraduate training programme and it was later rolled out across the university graduate school as part of the Doctoral Training Programme.

The first question I would ask students was “Why do scientists write papers?”  Undergraduates usually responded that scientists wrote to tell the world and their peers about what research they had done and thus advance science and prevent duplication of effort.  My response to this was that if they really wanted to publicise their research and make it accessible to the world they would publish their work in the popular press which has a lot more reach than a scientific journal.  After a bit of prodding they would then decide that perhaps it was for peer recognition and subsequent scientific validation via the review process.  Postgraduate students reached this stage more quickly and also understood that they needed to publish to make their cvs competitive and also of course to stake a claim to a particular research area to help with obtaining funding.

The first step in this journey is to do some good science!  Before setting out on the publication trail I also think that one should ask yourself if your work is important, although of course this is pretty subjective.  I am sure that all of us if asked, would consider that what we do is important enough to be published.  Next ask yourself if the experimental design or methodology is sound and if the work has been done well.  This will save time and remove some of the pain likely to be met during the review process.  Most importantly, at least in my opinion, is to ask yourself if there is a story.  There needs to be a strong narrative if you want to get people to read and cite your paper.

As a first time author you definitely need to ask advice about who does what, where you will send your paper and it is usually a good idea to get some agreement on authorship order earlier and not later.  Even as an experienced author I think that this sort of discussion can be very useful.  At the very least it will help you decide what particular slant your story will have.

Remember, have a clear story to tell and also remember that complexity is not the same as learning; keep your language simple, concise, precise and incisive and even at this early stage, make sure you follow the journal style!

At this point in the course I would put up this table and ask the students what each column represented.

Paper table

They would quickly guess that the first column represented the traditional layout of a scientific paper.  The other two columns took a bit longer, especially for the undergraduates until I asked them how they read papers when gathering material for their assignments and they were then able to identify the third column as how they, and most of us tend to read papers.  If the title seems interesting then we read the abstract, zip down to the results, see what the authors said about them, then check the introduction and then check the references for follow-up literature.  Methods and materials usually trail in at the end and then only if you have some doubts about what the authors have said or if you want to do something similar.  Then you look at the results again and you might look at the acknowledgements to see where they got their funding and to guess how many times they had to revise the paper (how many anonymous referees they acknowledge).

The middle column represents how most of us now write papers especially in these days of cut and paste. We follow the line of least resistance, start with the title to give us a starting point, our methods should have been written already in our lab books, the results come next and then we get on to the harder bits, the Introduction and the Discussion; acknowledgements flow logically from this and then it is a matter of adding the references and perhaps the hardest bit of all, the abstract or summary.    By the time you have done all this, your initial title almost certainly will no longer appeal to you so you come up with something new and more fitting.

Although this tends to be how we write papers I am not sure that it is actually the best way.  In the days before personal PCs some of us had access to typing pools and even if we didn’t, we either wrote our first drafts in long-hand or at a typewriter.  This meant that we got all our material together, had a long think about what we wanted to say and actually started at the beginning and worked our way through the paper in the same order as it would be printed.  Some people argue that this meant that ‘flow’ of those papers was smoother and more coherent.  I don’t think I know anyone who actually writes like that anymore, but I am happy to be contradicted.

Regardless of the fact that most of us live in a cut and paste world I am going to work through the various bits of a paper in the usual printed order.  Remember you are telling a story and there are a lot of rival authors out there competing for space in the top journals and you have to convince the journal editor and two or three referees that your paper is the one that should see the light of day in their journal.

You need a title; ideally it should be short, snappy and very importantly informative, although perhaps not too informative.  In the course I ran, I presented this to the students as a somewhat tongue in cheek example;

The effect of two lodgepole pine (Pinus contorta Douglas ex Loudon) seed origins (South coastal and Alaskan) on the growth, survival and development of larvae of the pine beauty moth, Panolis flammea (Denis & Schiffermuller) in the presence and absence of predators in a Scottish field site.

This although informative is not necessarily going to gain you readers or publication in a high impact journal.  In fact the external pre-REF (Research Excellence Framework – UK academics will know what this is) consultant employed by my university to help decide which papers should go forward for assessment was very clear that titles beginning with The effect of were very unlikely to receive high scores by the external assessors.

As it happened, the work I had done which is very clearly described by the informative title above was actually published as

Leather & Walsh title page

Not very informative but it certainly got a readership.

The abstract is perhaps the least favourite bit of a paper for authors; I certainly find them difficult and invariably save them to last.  They are however, extremely important and according to Wiley-Blackwell, publishers of the journal that I edit (Insect Conservation & Diversity) they are much more important than we as authors realise – they and the title are the ‘hook’ that gets your paper downloaded and hopefully read and then cited.  You should thus not just rush it off in a couple of minutes.  Think hard about what you want to say and what it is that is likely to get someone to download and read your paper.

Next is the Introduction, here you should put your work in context, remembering that it is not a literature review but make sure that you do cite some of the earlier relevant work as well as the more recent literature.  State the problem clearly and indicate who else has tried and failed and why your work is special and how you have succeeded where others have failed.

Now for the Materials and methods section, which to me is the most important part of the paper.  This is where you as a referee or reviewer should go first.  This is the detail that matters.  If the methodology is flawed then it doesn’t matter how great the writing is or how fancy the statistics, the paper should be rejected.  I think it is deplorable that there are now a number of ‘high impact’ journals that have relegated the methods to a subsidiary position, almost hiding them away and placing the results at the front end of the paper.  This is tantamount to telling reviewers that the methods don’t matter, just look at the results.  I have heard however, that some of these journals are now reconsidering this policy after some embarrassing publicity.

My advice to students is that the methods should contain as much detail as would be required for someone else to repeat your work without having to contact you.  So for example, the species involved, cultivars and phenological stage of the plants used, the sample size; for field work, the site details, the equipment used but not necessarily the supplier, unless of course it is very specialist, and the statistical treatment and assumptions.

The results section is your showcase.  Decide which display method is best for the message you want to get across and then pick out the most important points from your tables and graphs and turn them into a commentary, but DO NOT discuss them.  For the figures and tables do make sure that you follow the journal style.  Make sure that the figure and table titles are informative and comprehensive; in the days before Japanese journals published in English, the only English bit of the papers were the figure and table legends and it was possible to get a very good idea of know what the paper was about from them. Keep symbols simple and check line thickness.

The Discussion section is where you discuss YOUR results, highlight the strengths and weaknesses of your approach, underline your most important results, compare them with similar data and interpret your results in the broader view.  It is always a good idea to show how you addressed your initial hypothesis.  This and the methods section are the two sections where you can try and pre-guess the reviewers and get your retaliation in first.  If you can answer the reviewers before the questions are raised in their reports it increases the chances of getting your paper accepted.

Again, DO NOT use convoluted and obscure language and do AVOID jargon and pretentious statements.  As scientists our job is to communicate, not just to our peers, but to a wider audience. Quite often the reason our results are misinterpreted by the popular press is not because they are doing it on purpose but because we have obscured what we have said by using over-complicated language.  Be clear, use simple everyday words where possible, e.g.  laid rather than oviposited and be concise.

Speaking as an Editor I like acknowledgements to be brief, but do appreciate that there are funding agencies and helpful colleagues to thank.  I would advise against too much flippancy as after all you are advertising yourself and some people do read them.

Finally, the references; are they COMPLETE? Do they follow the journal style?  Editors do check and if you have had the misfortune to be rejected by your first choice journal, it does not go down well with the Editor of your second choice journal if you haven’t made the effort to change the formatting!  Do text citations and bibliography agree?  Check and recheck!

So now are you ready to release your pride and joy into the wild to suffer “the slings and arrows of outrageous fortune” more commonly known as editors and referees?  Actually no, that was just the first draft!  DO NOT SUBMIT IT YET.  Pass it around for comments; if you are a PhD student your supervisor definitely needs and wants to see it!  Let colleagues read it too and for communication test, get a non-specialist to read it.  If they can understand what you did and what your central message is then you have cracked the communication barrier.  Do listen to what people say, rewrite it!!  Be brutal in revision!  It is better to revise before submission than to have your paper rejected without the chance to revise.  Pass it around again. Then and only then, log on to the journal site and start the submission process, but do remember to read the guidelines for authors before you press the submit button!

Submit button

9 Comments

Filed under Bugbears, Teaching matters, Uncategorized

Referees – Your Journals Need You!

Editor-in-Chief

I have thought about writing on this subject for a while but it was this Tweet from Britt Koskella http://brittkoskella.wordpress.com/ on the 19th November 2013 that finally stirred me into action.

Britt 1

As an editor (I am for my sins, Editor-in-Chief of Insect Conservation & Diversity  http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1752-4598)  I love people like Britt.  It is such a joy to be able to select their names from the journal data base and assign them a manuscript, knowing that nine times out of ten they will accept my invitation to review a manuscript and that on that tenth occasion they will very kindly suggest an alternative (sometimes two or three) reviewer who will also almost certainly accept my invitation.  Britt Koskella, I love you and those like you 🙂  My reply to Britt was as follows:

Britt 2

You will have noticed that I confessed to doing too many myself; in fact in addition to those manuscripts that I read as an Editor I do on average, forty to fifty reviews for other journals.  Like Britt I have a hard time saying no.  I am getting better though – I actually turned down two this month 😉

There is a lot of debate at the moment about the peer review process in general with a number of journals adopting an open mass review process and other journals opting for the as long as the science is sound it is publishable approach.  We are, however, mainly, despite its many flaws, still operating on the traditional two referees per paper peer review system.

So how many papers should you referee asks Britt?  The general rule of thumb to entitle you to call yourself a good citizen is to agree to referee two papers for every paper that you submit as that is the minimum number of referees that you would expect to look at your own papers.  To be on the safe side and to feel that you are making a real contribution to your community, I would suggest that a 3:1 ratio is very acceptable.  In my experience as an Editor of two journals and as an Associate Editor on three other journals, there are a number of people who referee many more papers than that and a disturbingly large number of prolific authors whom, as far as I can see, never ever agree to referee papers.

As an Editor, what do I want from a referee?  In a nut-shell, someone who reads the paper thoroughly, checks first that the experimental design and statistical analysis are sound; if the experiment is not designed properly then it doesn’t matter how well the paper is written, it is not worth proceeding with; that the appropriate literature is cited (and by this I don’t just mean the referee’s own papers) and that the paper fits the remit of the journal and advances the subject area significantly.  I also do not want the referee to say how good the paper is in the comments to authors section and to tell me in the confidential comments that it is crap.  If you don’t like it then have the guts to tell the author why, don’t leave it up to the poor Editor to try to explain why he/she is rejecting their paper despite the apparently favourable comments they can see in the referee’s reports.  I also expect total impartiality; you might not agree with what you read but unless the methodology is flawed that is not a reason to reject the paper.  Be open-minded and fair above all.  If you are rejecting a paper, be constructive, authors at the start of their career are not as resistant or as resilient as old timers http://dynamicecology.wordpress.com/2013/11/18/are-you-resistant-or-resilient-in-the-face-of-rejection/.  Above all be fair, write your report bearing in mind the sort of review that you as an author would like to receive.  Do unto others as you would have others do unto you and that goes double for those of you who don’t referee as many papers as you should!  I am very tempted sometimes to do an instant reject on authors who have turned down my invitation to review a paper, especially if I have just accepted one of theirs.

Post script

I used to run a course for PhD students about getting published and it always used to amaze them that decisions on whether papers were published or not was dependent on the opinions of two to three people.  My response was that if you think that is bad, decisions about grant funding are often made with just as few opinions and those decisions have even greater implications for career prospects.

7 Comments

Filed under Bugbears