Tag Archives: unconscious bias

Does naming your study animals introduce observer bias?

I think, that most, if not all entomologists, will confess to a bit of funding envy when talking with those of their colleagues who work with the “undeserving 3%”, the large charismatic mega-fauna and the modern dinosaurs. The terminology gives us away, although the evidence is overwhelmingly on our side (Leather, 2009).  As entomologists, particularly those of us working in the field, we are used to reporting numbers collected in the tens of thousands (Ramsden et al., 2014 ), if not the hundreds of thousands (Missa et al., 2009) and  even a short six-week study can result in the capture of thousands of ground beetles (Fuller, et al., 2008).  Naming our subjects, much as we love them, is not an option, even if we wanted to. Even behavioural entomologists counting individual flower visits by pollinators are used to dealing with hundreds of individuals.   In the laboratory, although numbers may be smaller, say tens, we still assign them alphanumeric codes rather than names, even though one might look forward to counting the number of eggs laid by the unusually fecund moth #17 or hope that aphid #23 will be dead this morning as she is becoming a pesky outlier for your mortality data 🙂

Our colleagues who work with mammals in the field, seem however to adopt a different strategy. It appears quite common for them to name their animals as the following examples from Twitter make clear.

OB1

From her dissertation field note book, Erin Kane @Diana_monkey but not yet published.

Published data in McGraw et al., (2016) are from another study where the animals are not named.

OB2

Anthropomorphic judgement values

OB3

Anne being very involved with her cheetahs, although the paper (Hillborn et al., 2012) does not mention them by name.

 

OB4

Another example of subjects with names Hubel et al., 2016), but this time named in the paper.

OB5

Although in the description of methodology and results animals are referred to as subjects, the Table gives it away! (Allritz et al., 2016).

OB6

Another example of named subjects (Stoinski et al., 2003).

OB7

More named subjects (Dettmer & Fragaszy, 2000), but as these were captive the names almost certainly not chosen by the observers.

OB8

In this case (Blake et al., 2016), use no human-based names either in the methods or tables, so exemplary, although of course I have not seen their field note books 🙂

My concern, highlighted by these examples, is that by naming their study animals, the observers are anthropomorphising them and that this may lead them to inadvertently bias their observations. After all, the names have not been chosen at random, and thus could influence the behaviours noted (or ignored). I say ignored, because of two very specific examples, there are more, but I have these two to hand.

Victorians used birds as examples of good moral behaviour, erroneously believing them to be monogamous, probably because of seeing the way they fed their chicks cooperatively. Tim Birkhead (2000)* quotes the Reverend Frederick Morris who in 1853 preached  “Be thou like the dunnock – the male and female impeccably faithful to each other,”  and goes on to point out that despite a hundred years of ornithological science it was not until the late 1960s that  the promiscuous behaviour of female birds was revealed, interestingly enough coinciding with the new moral code of the 1960s.

Descriptions of penguin homosexual behaviour and their penchant for acts of necrophilia so shocked George Levick’s publishers that they removed them from his 1915 report but printed them and privately distributed them to selected parties marked as “Not for Publication” (Russell et al., 2012).  He also transcribed his descriptions of this ‘aberrant’ behaviour in Greek in his notebooks, presumably to make it less accessible.

And finally from me, this recent report about ‘sacred and ritualistic’ behaviour in chimpanzees Kuhl et al (2016),   where, I feel the authors have really allowed themselves to over-anthropomorphise with their subjects, very much to the detriment of scientific  detachment.  I have yet to find an entomologist who agrees with their interpretation. http://www.nature.com/articles/srep22219

AND NOW SOMETHING NEW for my blog, an embedded comment/riposte. I thought that it would be useful to get a response from someone who works on large charismatic mega-fauna and who names their subjects.  Anne Hilborn, whom many of you will know from Twitter as @AnneWHilborn, has kindly agreed to reply to my comments.  In the spirit of revealing any possible conflicts of interest I should say that I taught Anne when she was an Ecology MSc student at Silwood Park 🙂

Over to you Anne…..

“Hello, my name is Anne and I name my study animals.”

Decades ago this might have gotten me jeered out of science, the assumption being that by naming my study animals I was anthropomorphizing them and that any conclusions I drew about their behavior would be suspect. Thankfully we (at least those of us who have the privilege of working on megafauna) have moved on a bit in our thinking and our ways of doing science.

There are two parts to Simon’s concern about naming study animals. One is that naming leads to anthropomorphization, the second is that the anthropomorphizing leads to biased science. I would argue that the naming of study animals doesn’t necessarily increase anthropomorphism. On the Serengeti Cheetah Project we don’t name cheetahs until they are independent from their mother (due to a high mortality rate).  During my PhD fieldwork I spent a lot of time following a young male known as HON752MC (son of Strudel).  Several months after I started my work he was named Boke. My interest in his behavior, my chagrin at his failures and happiness when he had a full belly didn’t change when he was named. Many of us get emotionally attached on some level to our study animals, whether they have names or numbers.

An interesting thing to ponder is that if naming does lead to anthropomorphizing, does it only happen when human names are used? What human characteristics am I likely to attach to cheetahs named Peanut, Muscat, Strudel, Fusili, or Chickpea?

OB9

As to whether anthropomorphism leads to biased science… it definitely can if, as Simon points out, certain behaviors are not recorded because they do not fit the image of the animal the researcher had in their head. I don’t have any data on this, but I suspect this is extremely rare now days. Almost all researchers have had extensive formal training and know the importance of standardized data collection. I study cheetah hunting behavior, and I record how long a cheetahs spends spend stalking, chasing, killing, and eating their prey. I record the number of animals in the herd they targeted, how many second the cheetah spends eating vs being vigilant, and at what time they leave the carcass. No matter my personal feelings or attachments to an individual cheetah, the same data gets recorded.

Research methods have advanced a lot in the past decades and we use standardized methodologies and statistics expressly to prevent bias in our results. Anthropomorphism is just one possible source of bias, others include wanting to prove a treasured hypothesis, the tendency to place plots in areas where you suspect you will get the best results, etc..

As Adriana Lowe (@adriana_lowe ) puts it “Basically, if you’ve got a good study design and do appropriate stats, you can romanticise the furry little buggers until the cows come home and it won’t have a massive effect on your work. Any over interpretation of results would get called out by reviewers when you try to publish anyway.”

Simon points out examples of people being shocked when birds didn’t follow the dictates of contemporary human morality. I would like to think that biologists no longer place human values on animals. I can admire hyenas because the females are bigger bodied and socially dominant to males, but that doesn’t mean I draw parallels or lessons from them to human society (not in the least because the females give birth through their elongated clitoris and the cubs practice siblicide). As scientists we are capable of compartmentalizing, of caring deeply for our subjects, of shedding a tear when Asti turns up with one cub when previously she had five, without that changing the way we record data. In our training as biologists, we are taught not impose our own feelings or values on our study animals. We may find infanticide in lions (Packer and Pusey 1983), extra pair copulations in birds and primates (Sheldon 1994, Reichard 1995), or siblicide in boobies (Anderson 1990) to be repugnant, but we record, analyze, and try to publish on the phenomenon all the same.

To go on the offensive, there are ways naming study animals actually improves data collection.

Again, Adriana Lowe “If you’re doing scan sampling for instance, so writing down all individuals in a certain area every 10 minutes or so, names help. At least for me, it’s harder to remember if someone is M1 or M2 than Janet or Bob, particularly if you have a big study troop/community. So it can improve the quality of the data collected if you’re less likely to make identification errors.”

Because of our own training and peer review, assigning emotions or speculating about the intent on animals rarely makes it into scientific papers. However the situation is very different for those of us who wish to present our results outside of the ivory tower. While fellow scientists might be willing to wade through dry descriptions about how M43 contact called 3 times in 4 minutes when he was no longer in visual contact with M44, the public is not. Effective science communication needs a story and an emotional hook to draw people in. It is much easier to do that when you tell a story about Bradley and Cooper and not M43 and M44.  I will admit this does get into grey areas with the type of language we use outside of scientific papers. I tell stories about the cheetahs in my blog posts and even assign emotions to individuals. But if I am answering questions from the media or the public, I am still very careful not to make any definitive claims about behavior that haven’t been backed up by statistical analysis.

OB10

Here I use language and make assumption in tweets that I never would in a scientific paper.

OB11

There are a lot of issues that negatively affect the objectivity of science ie. the majority of funding going to well established entrenched researchers, papers being reviewed primarily by people from the same school of thought, the increasing pressure to have flashy results that generate headlines, but naming of study animals is not high on the list.

——————————————————————————————————————–

So now, over to you the readers, what do you think? Please comment and share your views or at the very least, please cast your vote.

VOTE NOW

 

 

References

Allritz, M., Call, J. & Borkenau, P. (2016) How chimpanzees (Pan troglodytes) perform in a modified emotional Stroop task. Animal Cognition, 19, 435-449.

Anderson, D. J. (1990) Evolution of obligate siblicide in Boobies. 1. A test of the insurance-egg hypothesis. American Naturalist, 135, 334–350.

Birkhead, T. (2000) Promiscuity: An Evolutionary History of Sperm Competition and Sexual Conflict. Faber, London.

Blake, J.G., Mosquera, D., Loiselle, B.A., Swing, K., Guerra, J. & Romo, D. (2016) Spatial and temporal activity patterns of ocelots Leopardus pardalis in lowland forest of eastern Ecuador.  Journal of Mammalogy, 97, 455-463.

Dettmer, E., and Fragaszy, D. 2000. Determining the value of social companionship to captive tufted capuchin monkeys (Cebus apella). Journal of Applied Animal Welfare Science, 3, 293-304

Fuller, R. J., Oliver, T. H. & Leather, S. R. (2008). Forest management effects on carabid beetle communities in coniferous and broadleaved forests: implications for conservation. Insect Conservation & Diversity 1, 242-252.

Hillborn, A., Pettorelli, N., Orme, C.D.L. & Durant, S.M. (2012) Stalk and chase: how hunt stages affect hunting success in Serengeti cheetah. Animal Behaviour, 84, 701-706

Hubel, T.Y., Myatt, J.P., Jordan, N.R., Dewhirst, O.P., McNutt, J.W. & Wilson, A.M. (2016) Energy cost and return for hunting in African wild dogs and cheetahs. Nature Communications, 7, 11034 DOI:doi:10.1038/ncomms11034

Kühl, H.S., Kalan, A.K., Arandjelovic, M., Aubert, F., D’Auvergne, L., Goedmakers, A., Jones, S., Kehoe, L., Regnaut, S., Tickle, A., Ton, E., van Schijndel, J., Abwe, E.E., Angedakin, S., Agbor, A., Ayimisin, E.A., Bailey, E., Bessone, M., Bonnet, M., Brazolla, G., Buh, V.E., Chancellor, R., Cipoletta, C., Cohen, H., Corogenes, K., Coupland, C., Curran, B., Deschner, T., Dierks, K., Dieguez, P., Dilambaka, E., Diotoh, O., Dowd, D., Dunn, A., Eshuis, H., Fernandez, R., Ginath, Y., Hart, J., Hedwig, D., Ter Heegde, M., Hicks, T.C., Imong, I., Jeffery, K.J., Junker, J., Kadam, P., Kambi, M., Kienast, I., Kujirakwinja, D., Langergraber, K., Lapeyre, V., Lapuente, J., Lee, K., Leinert, V., Meier, A., Maretti, G., Marrocoli, S., Mbi, T.J., Mihindou, V., Moebius, Y., Morgan, D., Morgan, B., Mulindahabi, F., Murai, M., Niyigabae, P., Normand, E., Ntare, N., Ormsby, L.J., Piel, A., Pruetz, J., Rundus, A., Sanz, C., Sommer, V., Stewart, F., Tagg, N., Vanleeuwe, H., Vergnes, V., Willie, J., Wittig, R.M., Zuberbuehler, K., & Boesch, C. Chimpanzee accumulative stone throwing. Scientific Reports, 6, 22219.

Leather, S. R. (2009). Taxonomic chauvinism threatens the future of entomology. Biologist, 56, 10-13.

McGraw, W.S., van Casteren, A., Kane, E., Geissler, E., Burrows, B. & Dsaegling, D.J. (2016) Feeding and oral processing behaviors of two colobine monkeys in Tai Forest, Ivory Coast.  Journal of Human Evolution, in press.

Missa, O., Basset, Y., Alonso, A., Miller, S.E., Curletti, G., M., D.M., Eardley, C., Mansell, M.W., & Wagner, T. (2009) Monitoring arthropods in a tropical landscape: relative effects of sampling methods and habitat types on trap catches. Journal of Insect Conservation, 13, 103-118.

Packer, C. & Pusey, A.E. (1983) Adaptations of female lions to infanticide by incoming males. American Naturalist, 121, 716–728.

Ramsden, M.W., Menéndez, R., Leather, S.R., & Wakkers, F. (2014) Optimizing field margins for biocontrol services: the relative roles of aphid abundance, annual floral resource, and overwinter habitat in enhancing aphid natural enemies. Agriculture Ecosystems and Environment, 199, 94-104.

Reichard, U. (1995) Extra-pair copulations in a monogamous gibbon (Hylobates lar). Ethology ,100, 99–112.

Russell, D.G.D., Sladen, W.J.L. & Ainley, D.G. (2012) Dr. George Murray Levick (1876-1956): unpublished notes on the sexual habits of the Adélie penguin.  Polar Record, 48, 387-393

Sheldon, B. C. (1994) Male phenotype, fertility, and the pursuit of extra pair copulations by female birds. Proceedings of the Royal Society B: Biological Sciences, 257, 25–30.

Stoinski, T.S., Hoff, M.P. & Maple, T.L. (2003) Proximity patterns of female western lowland gorillas (Gorilla gorilla gorilla) during the six months after parturition. American Journal of Primatology, 61, 61-72.

 

Post script

I said that entomologists don’t name their study animals but they do name their pets. Some of our PhD students had an African flower

OB12

Soulcleaver; despite his name he seems quite cute when viewed side-on, perhaps even with a cheeky grin, although as an entomologist I couldn’t possibly say that 🙂

beetle, Mecynorhina ugandiensis, which they named Soulcleaver, and I know that some beekeepers name their Queens https://missapismellifera.com/2016/03/17/the-decay-of-spring/

 

*note that Tim Birkhead also falls into the very trap that he describes by using the word promiscuous in the title of his book, a human judgemental term relating to moral behaviour, multiple mating would have been more appropriate.

 

 

 

Advertisements

12 Comments

Filed under Bugbears, Science writing, Uncategorized

Conflicts of interest – are there ever any situations where there aren’t?

Conflict1

Two seemingly unrelated factors stimulated me to write this post. One print-based, the other ‘ether’ based.

At the back-end of last year (2015) I was reading an article in The Times Higher Educational Supplement (29-10 – 4.11.2015 volume 2227 pp 6-7) about the fall in success rates of grant applications to UK research councils.  A sub-heading of the article, Reviewers are stretched, pointed out that the research councils, like journals are struggling to find enough reviewers.  A week later I came across this ‘conversation’ on Twitter.

Conflict2

Coincidentally I had just accepted an invitation to review a grant application, having just submitted one the week before and as I was, in my Editorial capacity, inviting Reviewer number 9 to cast judgement on a submission to Annals of Applied Biology, I felt I ought to respond to the invitation. I had tried desperately hard to wriggle out of reviewing the grant application, pointing out that I had been a Co-Investigator with the Principal Investigator on a earlier grant, that I had been the PhD supervisor of one of the Co-Is and published three papers with them and that I had taught the other Co-I.  I think you could say that I knew them very well indeed.

Conflict3

This image ‘borrowed’ from the University of Houston http://www.uh.edu/research/compliance/coi/

So full disclosure of the facts and surely, I thought, enough conflicts of interest there to rule me out! To my surprise the Research Council involved, replied saying that as long as I declared this on the review form, they were quite happy for me to referee the proposal.  I am used to the Research Councils being very flexible when it comes to the time allocated to do a review, they find it so difficult to get people to agree that they are very willing extend deadlines for several weeks if you promise that you will eventually deliver a report.  This response to what I saw as a major conflict of interest was, however, somewhat surprising. To say that I was gobsmacked* is a bit of an understatement, but as this had been my main reason for not accepting the task, I felt honour bound to do the review and make a recommendation.

So what exactly is a conflict of interest and how worried should we be about their potential to influence our responses in a scientific context? Here are a couple of definitions that I gleaned from the web.

A situation that has the potential to undermine the impartiality of a person because of the possibility of a clash between the person’s self-interest and professional interest or public interest.

http://www.businessdictionary.com/definition/conflict-of-interest.html#ixzz3tjt5c5l7

The real or apparent conflict between one’s personal interest in a matter and one’s duty to another or to the public in general regarding the same matter.

Webster’s New World Law Dictionary © 2010 by Wiley Publishing, Inc., Hoboken, New Jersey.
So pretty clear, if there is a connection, personal or business, with the person(s) that you are asked to comment on, or their work, then you have a potential conflict of interest.

I am, as some of you may know, Editor-in-Chief of Annals of Applied Biology and was, until last year, Editor-in-Chief of Insect Conservation & Diversity, so potential conflicts of interest have been part of my life for many years.  How do we handle this as Editors and Handling Editors? In the letter that our Handling Editors at Annals send out to potential reviewers we state:

If you agree to referee the paper please declare to the handling editor if you have:

published papers, submitted grant proposals, or supervised students with an author. It is essential that you declare this information before reviewing the manuscript.”

So very clear what we mean by a conflict of interest here. But given the shortage of people willing to review papers and note that it is common practice to initially invite four to get the minimum two that we aspire to for a fair and balanced decision, how fussy can we afford to be? On one memorable occasion, I once had to approach thirteen potential reviewers before I found two willing victims!  In the UK, if you are in a specialist field such as applied entomology, you are almost certain to know just about everyone who works in that area, either personally or by reputation.  Given the virtual insistence these days by the national grant funders on collaborative projects, you also have a fairly high probability of having been in joint grant application with many of them as well.  Most journals now ask for suggestions of preferred and non-preferred reviewers when you submit a paper.  These are highly likely to be people you know personally, and your preferred reviewers are also unlikely to be people you think will regard your work unfavourably.  Is this a conflict of interest?  As an Editor you can take notice of these names, often ignoring them because you suspect that the preferred reviewers have been chosen because of the possibility of them delivering a favourable review.  You then have a decision to make as to which reviewers to select; do you read through the references and see who has been cited most and pick them, do you resort Web of Science and look for publications in a similar area involving the same systems or use the keywords in your particular Editorial manager system?  Whichever way you go you have a high chance of picking people who know the author(s) and/or have worked with them at some stage, but if you want an expert opinion you are pretty much stuck with those choices.  It is further complicated by the fact that some people are more likely to respond in the affirmative than others, so your choice is narrowed still further.

As a potential reviewer receiving an invitation from a journal that doesn’t ask you for as much information as the Annals of Applied Biology does (looking back at last year’s 50+ requests to review that I receive, it seems that we at the Annals are much more up-front in this respect than other journals) what constitutes a conflict of interest?  Even if you don’t know the author personally, which if they are from the USA** or other country where entomologists are still fairly numerous is quite likely, does the fact that they have cited you a lot constitute a conflict of interest?  A favourable review may ensure publication and add to your citation index.  On the other hand, if the paper doesn’t cite you when you feel it should, is that also a conflict of interest and what about when it cites you unfavourably, are you sure  that you will write your review impartially? Should we also ask if you have received a favourable or non-favourable disclosed review from the author(s) for one of your own papers?  I can’t help but think that having had a favourable review from someone, you are, despite how impartial you consider yourself to be, likely to look more kindly on a paper from that person than one from someone who has said that your paper should be rejected.  I know there are a number of people who feel that open review is the way forward but I am not the only one that thinks it just adds to the conflicts of interest dilemma.

Leaving those issues aside. What about if you have answered yes to the questions posed by the Annals in that you have published papers, submitted grant proposals, or supervised students with an author?  Interestingly I have just noticed that we don’t ask whether the potential reviewer has supervised or taught one of the authors.  As someone whom to date, has supervised 50 PhD students, more than 130 MSc project students and about 150 undergraduate project students, not to mention the 1000+ students whose names I learnt when teaching them, this is yet another area of potential conflict of interest. The last time I co-authored a paper with my PhD supervisor was 1989, is that still a conflict of interest in 2016? I freely admit that I have reviewed more than one of his papers and even recommended rejection once or twice (I don’t think he reads my blog :-)).

I am not saying that our current review system is fatally flawed, in fact I think it works quite well and feel that the open reviewing system advocated by some has just as many, if not more, opportunities for potential conflicts of interest to arise. See this post by Dynamic Ecology which puts the case for pre-publication review very clearly.

When does it stop being a conflict of interest to review a paper by one of your former PhD students? Five years, ten years or longer?  I will put my hand up now and admit that I have reviewed papers written by former students, but only after what I consider a decent five-year interval since the last co-authorship.  What about co-authors who were not students or RAs? Often you end up on multi-author papers arising from working groups, do you apply the same rules in those cases when asked to review a paper or a grant proposal?  I don’t but should I?

When it comes to PhD examinations the situation is even more acute, at least in my case. I have examined about 50 PhD students and had 48 of my own examined in return.  As far as I can recall, of those that I have examined only three of them have been students of people who I didn’t know personally.   Of the students of mine examined, I think three of them were by people I didn’t know personally (these were bird projects), but the co-supervisor knew them.  This of course is totally understandable within the UK system, where the usual PhD viva panel consists of an internal examiner, an external examiner and, increasingly more common, an independent chair.  You are hardly likely to choose examiners you don’t know to give your students a grilling.  You choose someone who is fair and has a good reputation, which generally speaking means someone you know personally.   Given the paltry fee paid by UK universities for a PhD examination***, friends are much more likely to agree to do the job than total strangers 🙂

Should we have a system where only examiners who have no personal contact with the supervisors are allowed? This would almost certainly mean that in the UK, all examiners would have to come from overseas; I suspect that the Universities would baulk at the increased costs associated with such a system.  They are already very stingy when it comes to travel and accommodation costs for UK examiners so the added cost of getting someone from across the water is unlikely to appeal.

Where does this leave us? Not much further forward I suspect. I think, that as scientists, we all regard ourselves as being able to decide if, and when a real conflict of interest is likely to arise and would, I hope, inform the person(s) requesting the review of the pertinent facts.

Conflict4

Footnotes

*for non-native English speakers or English speakers from other parts of the world, a literal translation would be “like being hit unexpectedly in the mouth” 🙂

**although I have just noticed that a paper I have accepted an invitation to review from an American journal, has, hidden away in the middle of the author list, someone who did their PhD in the same research group as me at the same time, and with whom I have spent many a night drinking pints of Courage beer in The Mitre pub, on the Earlham Road in Norwich 🙂

***approximately £200 if you are lucky – say the viva takes 3 hours minimum, plus you have to read the thesis, say 150 pages, so even if you read very quickly you are looking at another 5 hours minimum, more likely closer to ten hours as you have to take notes as you go along, then add on an hour for the report and four hours travel time, that makes a total of 17 hours or so giving you an hourly rate of about £12, and that is before tax 🙂

 

3 Comments

Filed under Uncategorized