There is an interesting post (and lots of discussion in comments section) by Brian McGill at Dynamic Ecology blog on excessive usage of fancy, complicated and unnecessary statistics in ecology literature; he calls this statistical machismo (hat tip: Hari Sridhar, a graduate student in our department). It reminded me of an interesting comment on statistics from a referee on one of my manuscripts. I thought of writing about that, and also more generally about statistics and its role in my field of theoretical ecology.
Before I get to my story, let me say that I couldn’t agree more on the basic point McGill is making that ecologists tend to overuse (perhaps, largely in response to demanding referees) statistics when simple methods would yield equally good results. If you dont have fancy named stats in your publication, referees and editors, especially those of high impact factor journals, may object and ask for more, and likely even reject your paper. Your peers may criticize that you haven’t been rigorous enough. Despite recent debates, fancy and complicated statistics continue to flourish in the ecology literature.
What is the role of statistics in theoretical ecology? Being a theoretical ecologist, at least in principle, I don’t need to use any statistics for my publications. Our aim is to show general ecological principles or develop predictive tools using simple mathematical models. We do this either through analytical calculations where we derive relationships between two ecological quantities, or when we can not derive analytical results, we do numerical simulations to find those relationships. For the former case, statistics is not needed at all. For the latter, since we can do as many simulations as we like, we can obtain relationships where the error bars could be smaller than the thickness of the dots or lines on the plot. In other words, we can live without statistics (but see last paragraph).
Here is my unusual story of referee’s comment on stats during a manuscript review. As a physics graduate student, I submitted a paper to an ecology journal developing a quantitative method to ‘assess’ ecosystem dynamics, commenting on how it may offer an useful tool for ecologists. In this largely theoretical paper, we showed our claim by analytical calculations and numerical simulations but we had not done any statistics (and I could not have done anything either). Referees in general liked the idea, and here is a comment by one of the referees that you may find interesting: “Authors are physicists, not ecologists; it would be unfair to ask them to do detailed statistical analyses, and that will make paper too long” [1, 2]. I dont know what I thought of that at the comment at that time, but now I find it amusing at first sight but also very reasonable. I will leave it to you for your own interpretation.
Why would a theoretical paper need statistics when I argued above that we can avoid it entirely. Statistics can be very useful, and perhaps essential, to test whether our principles or tools derived from mathematical models work (i.e., provide significant results) even within typical limitations of real ecological data sets. For example, the ecological data are typically short and not finely resolved. It will have observational errors and we usually lack a full understanding of underlying processes. It was in this context that the referee was talking about statistical analyses in our manuscript. Eventually, after our work and several other related papers suggesting new such tools came out, many papers that exclusively develop and discuss statistical methods for these types of analyses got (and continue to get) published. Clearly, it would have been an overkill to demand statistics for our paper and I am glad referees were with us.
 I have made a slight modification to the actual referee comment to shorten an otherwise long sentence.
 I am not entirely sure if its okay to quote an anonymous referee’s comments. Any ideas?