Tag Archives: papers

Keeping up with the literature – an unwinnable battle?

I was going to write about predatory journals* but got distracted by tidying my office 🙂

Pre- and post-annual holiday office tidy

Whilst triaging the many pieces of paper that littered my office floor, table and desks, I realised that my “papers to read pile” had,

One that failed the triage, too fly-blown even for a keen entomologist 🙂

like an aphid, not only multiplied but spread itself across several locations. Once collated and stacked neatly into a single entity I was shocked to see that I was now faced with over 30 cm of interesting and potentially useful literature, to peruse, digest, annotate and add to my filing system.

The paper mountain – an awesome spectacle or should that be awful?

I am, as regular readers will know, no longer in the first flush of youth**.  Forty-five years ago as undergraduates, my generation went to the library to read physical journals, making notes as we went along and, very, very occasionally, spending money for a photocopy.   As a PhD student this was also how we operated, although we had by then learnt the art of reading Current Contents and the appropriate CABI Abstracts and sending a reprint request card to the authors of papers whose titles and abstracts had particularly caught our attention. We also had access to inter-library loans and depending on your supervisor, access to and the budget to enable you to photocopy older papers found while browsing journals in the library.  Once read, the paper was reverentially placed in a filing cabinet, especially if it had been signed by an eminent luminary, and the more organised of

My filing cabinets – all drawers full

D-K; A few of the 33 record card storage boxes that help clutter my office

us, entered the details, including our own keywords, on a record card which were then stored in a record card storage box, or card index system as we called it 🙂

 

Record cards ready for the details to be entered to my EndNote data base and subsequent filing in the appropriate storage box.  I prefer my own keywords as they often differ from those supplied by the authors.

In those days there were far fewer journals, the learned societies that I was, and am still a member of, the Association of Applied Biologists, the Royal Entomological Society, and the British Ecological Society, published one, two, and three journals respectively.  Keeping up with the literature, although in those days without the aid of search engines and email alerts, was fairly easy.  The biggest hindrance being the lack of response from some authors to your laboriously penned postcard reprint request.  Now those three societies publish three, seven and seven journals, all available on-line and two of the British Ecological Society’s journals don’t even have print copies.  On top of that, there are a plethora of commercial publishers producing huge numbers of journals.  You think of a subject and there will be a journal, and then there are the predatory journals to add to the deluge L Things have certainly changed over the last forty years, and the library and Current Contents have been replaced by email alerts, on-line tables of contents with their snazzy graphical abstracts to tempt me to download the full version pdf with every intention of reading it later. Now, as a creature of habit and a great believer in the belts and braces approach to data storage, I keep both physical and electronic versions of all the papers that I download, hence the paper mountain in my office and my guilt complex about being behind with the literature.  I, like many of you, get irritated when I read a paper dealing with stuff I work on and find I have not been cited or as a referee notice that relevant literature has not got a mention.  Now, I’m a great believer in giving credit when it’s due (Leather, 2004, 2014), and have of course blown off steam about it previously on this blog, but even I am starting to have second thoughts about keeping au fait with the literature both past and present, or in the case of some journals, future J

I am sure that like me, when you sit down to write a paper you do a literature search.  In the old days I was pretty confident that I was right up to date and that my trusty card index system computer-based data base would give me all the references I need.   Now though, especially, given the unread paper mountain sitting on my office floor and the pile of record cards still to be input to EndNote™, I know, that I am, sadly, not likely to have all the relevant papers to hand, so like everyone else I hit Google Scholar and Web of Science.

Papers I read to write my last two papers – still to be added to my card index system

This of course generates another pile of papers, albeit ones I have read, but lacking a record card and no presence on my data base L  The latter problem I could solve by using the EndNote™ download function but that goes against my neurotic need to have my own keywords and writing record cards for every paper I read while researching material for the paper in progress would slow the writing process hugely which is already under pressure from my other duties, teaching, student supervision, administration and all the other demands that impinge on the typical academic’s life.

In conclusion, I think and it makes me sad to write this, but the days of putting aside what look like interesting papers to read later, is no longer viable and I have now reached the stage where I can only cope with accessing and reading the literature needed for a work in progress. The battle has been lost L

References

Leather, S.R. (2004) Reinventing the wheel – on the dangers of taxon parochialism and shallow reference trawling! Basic and Applied Ecology, 5, 309-311.

Leather, S.R. (2014) How Stephen Jay Gould wrote Macbeth – not giving credit where it’s due: lazy referencing and ignoring precedence. Ideas in Ecology & Evolution, 7, 30-40.

 

*

I am sure I will eventually get round to writing it 🙂

**

Externally at any rate, internally I am still a youthful 17-year old 🙂

Advertisements

7 Comments

Filed under Science writing

Conflicts of interest – are there ever any situations where there aren’t?

Conflict1

Two seemingly unrelated factors stimulated me to write this post. One print-based, the other ‘ether’ based.

At the back-end of last year (2015) I was reading an article in The Times Higher Educational Supplement (29-10 – 4.11.2015 volume 2227 pp 6-7) about the fall in success rates of grant applications to UK research councils.  A sub-heading of the article, Reviewers are stretched, pointed out that the research councils, like journals are struggling to find enough reviewers.  A week later I came across this ‘conversation’ on Twitter.

Conflict2

Coincidentally I had just accepted an invitation to review a grant application, having just submitted one the week before and as I was, in my Editorial capacity, inviting Reviewer number 9 to cast judgement on a submission to Annals of Applied Biology, I felt I ought to respond to the invitation. I had tried desperately hard to wriggle out of reviewing the grant application, pointing out that I had been a Co-Investigator with the Principal Investigator on a earlier grant, that I had been the PhD supervisor of one of the Co-Is and published three papers with them and that I had taught the other Co-I.  I think you could say that I knew them very well indeed.

Conflict3

This image ‘borrowed’ from the University of Houston http://www.uh.edu/research/compliance/coi/

So full disclosure of the facts and surely, I thought, enough conflicts of interest there to rule me out! To my surprise the Research Council involved, replied saying that as long as I declared this on the review form, they were quite happy for me to referee the proposal.  I am used to the Research Councils being very flexible when it comes to the time allocated to do a review, they find it so difficult to get people to agree that they are very willing extend deadlines for several weeks if you promise that you will eventually deliver a report.  This response to what I saw as a major conflict of interest was, however, somewhat surprising. To say that I was gobsmacked* is a bit of an understatement, but as this had been my main reason for not accepting the task, I felt honour bound to do the review and make a recommendation.

So what exactly is a conflict of interest and how worried should we be about their potential to influence our responses in a scientific context? Here are a couple of definitions that I gleaned from the web.

A situation that has the potential to undermine the impartiality of a person because of the possibility of a clash between the person’s self-interest and professional interest or public interest.

http://www.businessdictionary.com/definition/conflict-of-interest.html#ixzz3tjt5c5l7

The real or apparent conflict between one’s personal interest in a matter and one’s duty to another or to the public in general regarding the same matter.

Webster’s New World Law Dictionary © 2010 by Wiley Publishing, Inc., Hoboken, New Jersey.
So pretty clear, if there is a connection, personal or business, with the person(s) that you are asked to comment on, or their work, then you have a potential conflict of interest.

I am, as some of you may know, Editor-in-Chief of Annals of Applied Biology and was, until last year, Editor-in-Chief of Insect Conservation & Diversity, so potential conflicts of interest have been part of my life for many years.  How do we handle this as Editors and Handling Editors? In the letter that our Handling Editors at Annals send out to potential reviewers we state:

If you agree to referee the paper please declare to the handling editor if you have:

published papers, submitted grant proposals, or supervised students with an author. It is essential that you declare this information before reviewing the manuscript.”

So very clear what we mean by a conflict of interest here. But given the shortage of people willing to review papers and note that it is common practice to initially invite four to get the minimum two that we aspire to for a fair and balanced decision, how fussy can we afford to be? On one memorable occasion, I once had to approach thirteen potential reviewers before I found two willing victims!  In the UK, if you are in a specialist field such as applied entomology, you are almost certain to know just about everyone who works in that area, either personally or by reputation.  Given the virtual insistence these days by the national grant funders on collaborative projects, you also have a fairly high probability of having been in joint grant application with many of them as well.  Most journals now ask for suggestions of preferred and non-preferred reviewers when you submit a paper.  These are highly likely to be people you know personally, and your preferred reviewers are also unlikely to be people you think will regard your work unfavourably.  Is this a conflict of interest?  As an Editor you can take notice of these names, often ignoring them because you suspect that the preferred reviewers have been chosen because of the possibility of them delivering a favourable review.  You then have a decision to make as to which reviewers to select; do you read through the references and see who has been cited most and pick them, do you resort Web of Science and look for publications in a similar area involving the same systems or use the keywords in your particular Editorial manager system?  Whichever way you go you have a high chance of picking people who know the author(s) and/or have worked with them at some stage, but if you want an expert opinion you are pretty much stuck with those choices.  It is further complicated by the fact that some people are more likely to respond in the affirmative than others, so your choice is narrowed still further.

As a potential reviewer receiving an invitation from a journal that doesn’t ask you for as much information as the Annals of Applied Biology does (looking back at last year’s 50+ requests to review that I receive, it seems that we at the Annals are much more up-front in this respect than other journals) what constitutes a conflict of interest?  Even if you don’t know the author personally, which if they are from the USA** or other country where entomologists are still fairly numerous is quite likely, does the fact that they have cited you a lot constitute a conflict of interest?  A favourable review may ensure publication and add to your citation index.  On the other hand, if the paper doesn’t cite you when you feel it should, is that also a conflict of interest and what about when it cites you unfavourably, are you sure  that you will write your review impartially? Should we also ask if you have received a favourable or non-favourable disclosed review from the author(s) for one of your own papers?  I can’t help but think that having had a favourable review from someone, you are, despite how impartial you consider yourself to be, likely to look more kindly on a paper from that person than one from someone who has said that your paper should be rejected.  I know there are a number of people who feel that open review is the way forward but I am not the only one that thinks it just adds to the conflicts of interest dilemma.

Leaving those issues aside. What about if you have answered yes to the questions posed by the Annals in that you have published papers, submitted grant proposals, or supervised students with an author?  Interestingly I have just noticed that we don’t ask whether the potential reviewer has supervised or taught one of the authors.  As someone whom to date, has supervised 50 PhD students, more than 130 MSc project students and about 150 undergraduate project students, not to mention the 1000+ students whose names I learnt when teaching them, this is yet another area of potential conflict of interest. The last time I co-authored a paper with my PhD supervisor was 1989, is that still a conflict of interest in 2016? I freely admit that I have reviewed more than one of his papers and even recommended rejection once or twice (I don’t think he reads my blog :-)).

I am not saying that our current review system is fatally flawed, in fact I think it works quite well and feel that the open reviewing system advocated by some has just as many, if not more, opportunities for potential conflicts of interest to arise. See this post by Dynamic Ecology which puts the case for pre-publication review very clearly.

When does it stop being a conflict of interest to review a paper by one of your former PhD students? Five years, ten years or longer?  I will put my hand up now and admit that I have reviewed papers written by former students, but only after what I consider a decent five-year interval since the last co-authorship.  What about co-authors who were not students or RAs? Often you end up on multi-author papers arising from working groups, do you apply the same rules in those cases when asked to review a paper or a grant proposal?  I don’t but should I?

When it comes to PhD examinations the situation is even more acute, at least in my case. I have examined about 50 PhD students and had 48 of my own examined in return.  As far as I can recall, of those that I have examined only three of them have been students of people who I didn’t know personally.   Of the students of mine examined, I think three of them were by people I didn’t know personally (these were bird projects), but the co-supervisor knew them.  This of course is totally understandable within the UK system, where the usual PhD viva panel consists of an internal examiner, an external examiner and, increasingly more common, an independent chair.  You are hardly likely to choose examiners you don’t know to give your students a grilling.  You choose someone who is fair and has a good reputation, which generally speaking means someone you know personally.   Given the paltry fee paid by UK universities for a PhD examination***, friends are much more likely to agree to do the job than total strangers 🙂

Should we have a system where only examiners who have no personal contact with the supervisors are allowed? This would almost certainly mean that in the UK, all examiners would have to come from overseas; I suspect that the Universities would baulk at the increased costs associated with such a system.  They are already very stingy when it comes to travel and accommodation costs for UK examiners so the added cost of getting someone from across the water is unlikely to appeal.

Where does this leave us? Not much further forward I suspect. I think, that as scientists, we all regard ourselves as being able to decide if, and when a real conflict of interest is likely to arise and would, I hope, inform the person(s) requesting the review of the pertinent facts.

Conflict4

Footnotes

*for non-native English speakers or English speakers from other parts of the world, a literal translation would be “like being hit unexpectedly in the mouth” 🙂

**although I have just noticed that a paper I have accepted an invitation to review from an American journal, has, hidden away in the middle of the author list, someone who did their PhD in the same research group as me at the same time, and with whom I have spent many a night drinking pints of Courage beer in The Mitre pub, on the Earlham Road in Norwich 🙂

***approximately £200 if you are lucky – say the viva takes 3 hours minimum, plus you have to read the thesis, say 150 pages, so even if you read very quickly you are looking at another 5 hours minimum, more likely closer to ten hours as you have to take notes as you go along, then add on an hour for the report and four hours travel time, that makes a total of 17 hours or so giving you an hourly rate of about £12, and that is before tax 🙂

 

3 Comments

Filed under Uncategorized

Referees – Your Journals Need You!

Editor-in-Chief

I have thought about writing on this subject for a while but it was this Tweet from Britt Koskella http://brittkoskella.wordpress.com/ on the 19th November 2013 that finally stirred me into action.

Britt 1

As an editor (I am for my sins, Editor-in-Chief of Insect Conservation & Diversity  http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1752-4598)  I love people like Britt.  It is such a joy to be able to select their names from the journal data base and assign them a manuscript, knowing that nine times out of ten they will accept my invitation to review a manuscript and that on that tenth occasion they will very kindly suggest an alternative (sometimes two or three) reviewer who will also almost certainly accept my invitation.  Britt Koskella, I love you and those like you 🙂  My reply to Britt was as follows:

Britt 2

You will have noticed that I confessed to doing too many myself; in fact in addition to those manuscripts that I read as an Editor I do on average, forty to fifty reviews for other journals.  Like Britt I have a hard time saying no.  I am getting better though – I actually turned down two this month 😉

There is a lot of debate at the moment about the peer review process in general with a number of journals adopting an open mass review process and other journals opting for the as long as the science is sound it is publishable approach.  We are, however, mainly, despite its many flaws, still operating on the traditional two referees per paper peer review system.

So how many papers should you referee asks Britt?  The general rule of thumb to entitle you to call yourself a good citizen is to agree to referee two papers for every paper that you submit as that is the minimum number of referees that you would expect to look at your own papers.  To be on the safe side and to feel that you are making a real contribution to your community, I would suggest that a 3:1 ratio is very acceptable.  In my experience as an Editor of two journals and as an Associate Editor on three other journals, there are a number of people who referee many more papers than that and a disturbingly large number of prolific authors whom, as far as I can see, never ever agree to referee papers.

As an Editor, what do I want from a referee?  In a nut-shell, someone who reads the paper thoroughly, checks first that the experimental design and statistical analysis are sound; if the experiment is not designed properly then it doesn’t matter how well the paper is written, it is not worth proceeding with; that the appropriate literature is cited (and by this I don’t just mean the referee’s own papers) and that the paper fits the remit of the journal and advances the subject area significantly.  I also do not want the referee to say how good the paper is in the comments to authors section and to tell me in the confidential comments that it is crap.  If you don’t like it then have the guts to tell the author why, don’t leave it up to the poor Editor to try to explain why he/she is rejecting their paper despite the apparently favourable comments they can see in the referee’s reports.  I also expect total impartiality; you might not agree with what you read but unless the methodology is flawed that is not a reason to reject the paper.  Be open-minded and fair above all.  If you are rejecting a paper, be constructive, authors at the start of their career are not as resistant or as resilient as old timers http://dynamicecology.wordpress.com/2013/11/18/are-you-resistant-or-resilient-in-the-face-of-rejection/.  Above all be fair, write your report bearing in mind the sort of review that you as an author would like to receive.  Do unto others as you would have others do unto you and that goes double for those of you who don’t referee as many papers as you should!  I am very tempted sometimes to do an instant reject on authors who have turned down my invitation to review a paper, especially if I have just accepted one of theirs.

Post script

I used to run a course for PhD students about getting published and it always used to amaze them that decisions on whether papers were published or not was dependent on the opinions of two to three people.  My response was that if you think that is bad, decisions about grant funding are often made with just as few opinions and those decisions have even greater implications for career prospects.

7 Comments

Filed under Bugbears