Journals of Irreproducible Research – downgrading reproducibility and fact checking

As far as I am concerned, good science is about communication and reproducibility, or, as Stephen Heard argues, at least being able to believe that it is reproducible.  I would argue a bit more strongly than Stephen, in that I think you should, at the very least, be able to be confident that you could reproduce the experiment without having to contact the author(s) and that you can also easily check the cited literature.   In this context, there are two things that really annoy me about some of the so-called ’high impact’ established print journals and their on-line would be rivals.  First, the way in which the methods and materials section is relegated to the end of the paper, often in smaller font, and in some cases to the supplementary material section  In other journals e.g. Nature, the methods section is also very minimal and I defy anyone to repeat those experiments!  My second bugbear is the habit that some journals have, possibly to reduce space, in making you use numbers to denote references, placing them either in parentheses or superscript in the main text.

Perhaps I am alone in this, but I do like to know whose work is being cited without having to constantly refer to the references section.  What  particularly annoys me, are those journals that not only insist on numbered references but then list them in number order and not in alphabetical order!  I once wrote a review paper for Annual Review of Entomology, which has the numbering system, but subverted it by listing my references alphabetically – the editor never noticed 😉

You may say that what all these journals are doing is merely structuring the paper in the order that people tend to read them which is, I admit, a valid point. To me however, they are saying to the scientific community, perhaps not overtly, but certainly subliminally, that methods and materials are something you don’t really need to bother about, somewhat akin to those things that you store in an attic or basement, just in case you might want them at some time in the future, but probably not often, if at all.

This sends a strong and erroneous message to authors that despite the methodology being the most important part of how we do our science, as long as they report the general gist of how they did things it is fine.  To referees the subversion of the methods section sends an equally strong signal; you don’t really need to spend a lot of time reading about the methodology as long as the rationale for the work is justified and that the results are significant and well presented.

As someone who works on insect-plant interactions I constantly come across inadequate methods and materials sections both as a referee and as a reader of published work.  The thing that perhaps causes me the most annoyance are descriptions of plant phenology.   Herbivorous insects have a very intimate relationship with their host plants and the growth stage of their host plant or the age of the plant tissue that they are feeding on can have very marked effects on their development, survival and fecundity (Awmack & Leather, 2002).  I so often came across methods descriptions along the lines of “10 day-old cabbage seedling” “ 3 week old pepper plant”,  “2 week-old wheat plant”, that in desperation I wrote an editorial (Leather, 2010) explaining how important it was to use a measure that didn’t depend on the temperature,  photoperiod, nutrient or water status that the plants were grown at i.e. the BCCH scale.  I also compiled a virtual issue of Annals of Applied Biology, with relevant examples drawn from the journal which has a long and distinguished history in publishing such articles.  If you can’t find your host plant in past issues of the Annals you will find that most plants have a published version somewhere, even if only on Wikipedia.  Despite my efforts however, I still often have to remind authors to describe the phenological stage of their host plants accurately and precisely.

Methods and materials, please come back, we need you!

 

References

Awmack, C. S. & Leather, S. R. (2002). Host plant quality and fecundity in herbivorous insects. Annual Review of Entomology 47, 817-844.

Leather, S. R. (2010). Precise knowledge of plant growth stages enhances applied and pure research. Annals of Applied Biology 157, 159-161.