Waifs and strays – those papers nobody cites (or reads?)

I guess, like most, if not all of us, who publish papers, we hope that not only will our papers be read, but that they will be cited by others, not just ourselves.  From a purely practical point, it is after all, how academics impress promotion boards or prospective employers. From a more personal point of view, the papers we publish represent a lot of effort, not just in gathering the data or having the idea, but also the nightmare of turning it into deathless prose and then the battle with editors and reviewers.  We all have a few papers that we hope will make our name and perhaps become a citation classic, although as Stephen Heard has pointed out not just once, but twice, our favourites are not always everyone else’s, with some papers significantly failing to meet expectations.  A recent article in Times Higher Education, showed that in some disciplines, notably in the arts, 77% of papers were still uncited five years after publication and even in the sciences, about 40% of papers suffered a similar fate.  For ecologists, the hot area is ecological modelling, with only 6% of papers remaining uncited after five years. As someone with an advanced case of imposter syndrome, this is really quite reassuring; although I have twelve papers that are uncited (according to Web of Science), they only represent 6% of my output (and I am not a modeller 😊), of which only one dates back to 2012 (0.5%).  I do, however, have another 34 papers, that although they have been cited, have been cited fewer than five times, 17% of my output, or, if I add in my never been cited papers, 23% of my work has had relatively little impact on the ecological and entomological world.

Despite this, I was curious about what, if anything, these unwanted (=uncited) waifs and strays had in common, and how they differed from my most cited papers; absolutely nothing to with the fact that Stephen Heard only had four zero papers, all of which were recent papers 😊 In Steve’s analysis he looked at time since publication and found a positive correlation, his oldest papers had accrued the most citations.  I have a somewhat larger corpus of work than Steve, so concentrated my analysis on my top twenty papers.  There was absolutely no relationship (Figure 1), all pretty much of a muchness apart from the massive outlier, but even with that removed, still nada.

Figure 1. My top twenty papers.  The massive outlier is my single Annual Review of Entomology paper.

My least cited papers, do however, show a relationship between years since publication and the paltry number of citations that they have received (Figure 2).

Figure 2. The foot of the table papers.

If I combine the two data sets and leave out my star paper, there is a relationship between time since publication and the number of citations gained (Figure 3), so I expect, but I could be wrong, and I am not going to invest the time in finding out, that if I analysed all my papers that there would be a similar relationship as that shown by Steve’s analysis.

Figure 3. Relationship between time since publication and number of citations accrued for my most and least cited papers (excluding the massive outlier).

So, what makes a paper a waif, or reversing the question, a star?  Editors, of which I am one, are great fans of Reviews, believing, usually correctly, that they garner a lot of Impact Factor points, authors perhaps less so, as they tend to take away citations from your other papers. After all, who writes a review without citing themselves? 😊 The other thing that helps a paper get cited is their title, Andrew Hendry over on Eco-Evo Evo-Eco suggests that two main factors come into play. The first is that those papers that have a very good “fill in the box” titles are much more likely to be cited than those with more specific titles. He points out that a paper he and colleagues published in Philosophical Transactions of the Royal Society of Biology paper is the only one in the literature with Eco-Evolutionary Dynamics being the sole words in the title so, any paper writing about eco-evolutionary dynamics can use that citation to “fill in the citation box” after their first sentence on the topic.  The second inflation factor he cites, is that citations beget citations. When “filling in the box”, authors tend to cite papers that other authors used to fill in the same box.  In other words, authors tend to be lazy and use what other people have cited in their introductions.  This is not something to be encouraged, as it can lead to people being wrongly attributed; I have raged against this practice in the past.  Stephen Heard over at Scientist See Squirrel reckons that his most original papers are cited less because they report research from “outside the box” and most people are working “inside the box”.  Dorothy Bishop over at Bishopblog suggests that the best way to bury your work is to put it in a book chapter in an edited book.

So, what about my stars and strays?  My most cited paper is indeed a review, and for an entomologist, being in that most prestigious of review journals, the Annual Review of Entomology, it is no surprise to me that it tops my top ten chart, with just over 1000 cites.  Incidentally, number 4 (Leather et al., 1999) and number 7 (Leather et al., 1989) in my top 10, are also reviews.  My second most cited paper (Leather, 1988), is also, I guess, a review of sorts, albeit very short, although I prefer to think of it as more of a synthesis cum speculation paper.

What about the duds, those that no-one cites, not even me.  If I ignore the most recent papers, those published this year (2018) and last 2017), as being unlikely to have had time to be read, let alone cited, then all my zero papers are either editorials or commentary papers (e.g. Leather, 2014).  Don’t let yourself be fooled by the hope that a commentary paper, even with a sexy title and published in a top-notch journal will get cited.  My effort in Journal of Animal Ecology in early 2015 being a prime example, even the magic words, “climate change” failing to elicit a single citation to date (Leather (2015).

It is hard to see a pattern in my other lesser cited papers, they don’t seem to be markedly different from my more frequently cited papers, being published in my usual journals and covering the same subject matter, aphids, agricultural and forest pests and biological control in the main.  I confess to being very disappointed in the low number of citations to my aphid cannibalism paper (Cooper et al., 2014) especially as it got a lot of media attention, but I guess it falls into the too original box, not many people work on aphid cannibalism 😊

Sadly, it seems that Steve Heard is right, despite the journal blurbs, we don’t value originality, and the message for both journal editors and authors, is clear, if you want citations, publishing reviews and sticking to well ploughed fields is the safest bet.

References

Cooper, L.C., Desjonqueres, C. & Leather, S.R. (2014) Cannibalism in the pea aphid, Acyrthosiphon pisum. Insect Science, 21, 750-758.

Leather, S.R. (1988) Size, reproductive potential and fecundity in insects – things aren’t as simple as they seem. Oikos, 51, 386-389.

Leather, S.R. (2014: Modifying glucosinolates in oilseed rape – Giamoustaris & Mithen (1995): a top-twenty paper in the Annals of Applied Biology. Annals of Applied Biology, 164, 318-319.

Leather, S.R. (2015) Title: Onwards and upwards – aphid flight trends follow climate change. Journal of Animal Ecology, 84, 1-3.

Leather, S.R., Walters, K.F.A. & Dixon, A, F.G. (1989) Factors determining the pest status of the bird cherry-oat aphid, Rhopalosiphum-padi (L) (Hemiptera, Aphididae), in Europe – a study and review.  Bulletin of Entomological Research, 79, 345-360.

Leather, S.R., Day, K.R. & Salisbury, A.N. (1999) The biology and ecology of the large pine weevil, Hylobius abietis (Coleoptera: Curculionidae): a problem of dispersal? Bulletin of Entomological Research, 89, 3-16.

 

Advertisements

2 Comments

Filed under Science writing

2 responses to “Waifs and strays – those papers nobody cites (or reads?)

  1. Thanks for this Simon, an interesting read. I especially liked your point about original publications. Perhaps not curiously, I’ve found original papers are difficult to identify reviewers for, and make an interesting point with funding interdisciplinary research.

    Your blog makes me think about one particular article* which noted the most cited papers seem to show pretty looking figures (less messy, more neat data) and clear (often) controversial** messages. I feel it only takes one person to read your paper for it to have an effect. As another thought, it’s intriguing when we consider beyond metrics how we affect things on a social level.

    *Tried digging it out, but no luck with a quick look. It was maybe a year or so ago, in Nature.

    **The paper also highlighted these highly cited papers could be distorting science (similar to a point you made).

    Liked by 1 person

  2. Interesting analysis Simon! (How do you find the time?). It seems to me that citations often lead to more citations. It is critical to hit on some specific words of interest in the title or keywords, and it seems that the study subject can also be of importance. I have a number of zero-citation papers, some of which I think are fairly decent, but not mainstream. Anyway, thought-provoking blog!

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.