Ioannidis, J. P. A., Fanelli, D., Dunne, D. D. & Goodman, S. N. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 13, e1002264 (2015).
Google Scholar
Hampton, S. E. et al. The Tao of open science for ecology. Ecosphere 6, art120 (2015).
Google Scholar
Rothstein, H. R., Sutton, A. J. & Borenstein, M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments (John Wiley & Sons, 2005).
Sutton, A. J. in The Handbook of Research Synthesis and Meta-Analysis (eds Cooper, H. et al.) 435–452 (Russell Sage Foundation, 2009).
Nakagawa, S., Koricheva, J., Macleod, M. & Viechtbauer, W. Introducing our series: research synthesis and meta-research in biology. BMC Biol. 18, 20 (2020).
Google Scholar
Nakagawa, S. et al. A new ecosystem for evidence synthesis. Nat. Ecol. Evol. 4, 498–501 (2020).
Google Scholar
Coolidge, H. J. & Lord, R. H. in Archibald Cary Coolidge: Life and Letters 308 (Houghton Mifflin Harcourt, 1932).
Nickerson, R. S. Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175–220 (1998).
Google Scholar
Touchon, J. C. & McCoy, M. W. The mismatch between current statistical practice and doctoral training in ecology. Ecosphere 7, e01394 (2016).
Google Scholar
Begley, C. G. & Ellis, L. M. Raise standards for preclinical cancer research. Nature 483, 531–533 (2012).
Google Scholar
Aarts, A. A. et al. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).
Google Scholar
Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2, 637–644 (2018).
Google Scholar
Fraser, H., Parker, T., Nakagawa, S., Barnett, A. & Fidler, F. Questionable research practices in ecology and evolution. PLoS ONE 13, e0200303 (2018).
Google Scholar
Culina, A., van den Berg, I., Evans, S. & Sánchez-Tójar, A. Low availability of code in ecology: a call for urgent action. PLoS Biol. 18, e3000763 (2020).
Google Scholar
Jennions, M. D. & Møller, A. P. Publication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method. Biol. Rev. Camb. Philos. Soc. 77, 211–222 (2002).
Google Scholar
Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behavior. Behav. Ecol. 14, 438–445 (2003).
Google Scholar
Cassey, P., Ewen, J. G., Blackburn, T. M. & Møller, A. P. A survey of publication bias within evolutionary ecology. Proc. Biol. Sci. 271, S451–S454 (2004).
Google Scholar
Kardish, M. R. et al. Blind trust in unblinded observation in ecology, evolution, and behavior. Front. Ecol. Evol. 3, 51 (2015).
Google Scholar
Jennions, M. D. & Møller, A. P. Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. Proc. Biol. Sci. 269, 43–48 (2002).
Google Scholar
Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016).
Google Scholar
Chalmers, I. & Glasziou, P. Avoidable waste in the production and reporting of research evidence. Lancet 374, 86–89 (2009).
Google Scholar
Altman, D. G. The scandal of poor medical research. BMJ 308, 283–284 (1994).
Google Scholar
Glasziou, P. & Chalmers, I. Is 85% of health research really ‘wasted’? BMJ Opinion (14 January 2016).
Glasziou, P. & Chalmers, I. Research waste is still a scandal. BMJ 363, k4645 (2018).
Google Scholar
Chalmers, I. et al. How to increase value and reduce waste when research priorities are set. Lancet 383, 156–165 (2014).
Google Scholar
Kunin, W. E. Robust evidence of declines in insect abundance and biodiversity. Nature 574, 641–642 (2019).
Google Scholar
Christie, A. P. et al. Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences. Nat. Commun. 11, 6377 (2020).
Google Scholar
Campbell, H. A. et al. Finding our way: on the sharing and reuse of animal telemetry data in Australasia. Sci. Total Environ. 534, 79–84 (2015).
Google Scholar
Koricheva, J. Non-significant results in ecology: a burden or a blessing in disguise? Oikos 102, 397–401 (2003).
Google Scholar
Bennett, L. T. & Adams, M. A. Assessment of ecological effects due to forest harvesting: approaches and statistical issues. J. Appl. Ecol. 41, 585–598 (2004).
Google Scholar
Duval, S. & Tweedie, R. A nonparametric ‘trim and fill’ method of accounting for publication bias in meta-analysis. J. Am. Stat. Assoc. 95, 89–98 (2012).
Brlík, V. et al. Weak effects of geolocators on small birds: a meta-analysis controlled for phylogeny and publication bias. J. Anim. Ecol. 89, 207–220 (2020).
Google Scholar
Hurlbert, S. H. Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54, 187–211 (1984).
Google Scholar
Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behaviour. Behav. Ecol. 14, 438–445 (2003).
Google Scholar
Culina, A., Purgar, M. & Klanjscek, T. Datasets and codes for Purgar et al. 2022: quantifying research waste in ecology. Zenodo https://zenodo.org/record/6566100#.YrLWB-zMIqs (2022).
Ferguson, C. et al. Europe PMC in 2020. Nucleic Acids Res. 49, D1507–D1514 (2021).
Google Scholar
Huang, C.-K. et al. Meta-Research: Evaluating the impact of open access policies on research institutions. eLife 9, e57067 (2020).
Google Scholar
Ross-Hellauer, T. Open science, done wrong, will compound inequities. Nature 603, 363 (2022).
Google Scholar
Smith, A. C. et al. Assessing the effect of article processing charges on the geographic diversity of authors using Elsevier’s ‘Mirror journal’ system. Quant. Sci. Stud. 2, 1123–1143 (2021).
Google Scholar
Christie, A. P. et al. Reducing publication delay to improve the efficiency and impact of conservation science. PeerJ 9, e12245 (2021).
Google Scholar
Desjardins-Proulx, P. et al. The case for open preprints in biology. PLoS Biol. 11, e1001563 (2013).
Google Scholar
Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (2017).
Google Scholar
O’Dea, R. E. et al. Towards open, reliable, and transparent ecology and evolutionary biology. BMC Biol. 19, 68 (2021).
Google Scholar
Culina, A. et al. Navigating the unfolding open data landscape in ecology and evolution. Nat. Ecol. Evol. 2, 420–426 (2018).
Google Scholar
Culina, A., Crowther, T. W., Ramakers, J. J. C., Gienapp, P. & Visser, M. E. How to do meta-analysis of open datasets. Nat. Ecol. Evol. 2, 1053–1056 (2018).
Google Scholar
Roche, D. G., Kruuk, L. E. B., Lanfear, R. & Binning, S. A. Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295 (2015).
Google Scholar
Grainger, M. J., Bolam, F. C., Stewart, G. B. & Nilsen, E. B. Evidence synthesis for tackling research waste. Nat. Ecol. Evol. 4, 495–497 (2020).
Google Scholar
Nørgaard, B. et al. Systematic reviews are rarely used to inform study design—a systematic review and meta-analysis. J. Clin. Epidemiol. 145, 1–13 (2022).
Google Scholar
Webb, J. A. et al. Weaving common threads in environmental causal assessment methods: toward an ideal method for rapid evidence synthesis. Freshw. Sci. 36, 250–256 (2017).
Google Scholar
Collins, A., Coughlin, D., Miller, J. & Kirk, S. The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide (Joint Water Evidence Group, 2015).
Carrick, J. et al. Is planting trees the solution to reducing flood risks? J. Flood Risk Manag. 12, e12484 (2019).
Google Scholar
Nuñez, M. A. & Amano, T. Monolingual searches can limit and bias results in global literature reviews. Nat. Ecol. Evol. 5, 264 (2021).
Google Scholar
Morrison, A. et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28, 138–144 (2012).
Google Scholar
Wu, T., Li, Y., Bian, Z., Liu, G. & Moher, D. Randomized trials published in some Chinese journals: how many are randomized? Trials 10, 46 (2009).
Google Scholar
Vorobeichik, E. L. & Kozlov, M. V. Impact of point polluters on terrestrial ecosystems: methodology of research, experimental design, and typical errors. Russ. J. Ecol. 43, 89–96 (2012).
Google Scholar
Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366 (2011).
Google Scholar
Transforming Our World: the 2030 Agenda for Sustainable Development (United Nations, 2015).
MacCoun, R. & Perlmutter, S. Blind analysis: hide results to seek the truth. Nature 526, 187–189 (2015).
Google Scholar
Parker, T. H. et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 (2016).
Google Scholar
Announcement: reducing our irreproducibility. Nature 496, 398 (2013).
Moher, D. et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet 387, 1573–1586 (2016).
Google Scholar
Smaldino, P. E. & McElreath, R. The natural selection of bad science. R. Soc. Open Sci. 3, 160384 (2016).
Google Scholar
Vrieze, J. Landmark research integrity survey finds questionable practices are surprisingly common. ScienceInsider https://www.sciencemag.org/news/2021/07/landmark-research-integrity-survey-finds-questionable-practices-are-surprisingly-common (2021).
Woolston, C. Impact factor abandoned by Dutch university in hiring and promotion decisions. Nature 595, 462 (2021).
Google Scholar
Directorate-General for Research and Innovation (European Commission). Towards a Reform of the Research Assessment System. Scoping Report (Publications Office, 2021).
Athena Research & Innovation Center, Directorate-General for Research and Innovation (European Commission), PPMI, UNU-MERIT. Monitoring the Open Access Policy of Horizon 2020. Final report (European Commission, 2021).
Kwon, D. University of California and Elsevier forge open-access deal. TheScientist https://www.the-scientist.com/news-opinion/university-of-california-and-elsevier-forge-open-access-deal–68557 (2021).
Vines, T. H. et al. Mandated data archiving greatly improves access to research data. FASEB J. 27, 1304–1308 (2013).
Google Scholar
NPQIP Collaborative Group. Did a change in Nature journals’ editorial policy for life sciences research improve reporting? BMJ Open Sci. 3, e000035 (2019).
Google Scholar
Glasziou, P. et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 383, 267–276 (2014).
Google Scholar
Fecher, B. & Friesike, S. in Opening Science: the Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing (eds Bartling, S. & Friesike, S.) 17–47 (Springer International Publishing, 2014).
Hardwicke, T. E. et al. Calibrating the scientific ecosystem through meta-research. Annu. Rev. Stat. Appl. 7, 11–37 (2020).
Google Scholar
McKiernan, E. C. et al. How open science helps researchers succeed. eLife 5, e16800 (2016).
Google Scholar
Fidler, F. et al. Metaresearch for evaluating reproducibility in ecology and evolution. Bioscience 67, 282–289 (2017).
Google Scholar
Cornwall, C. E. & Hurd, C. L. Experimental design in ocean acidification research: problems and solutions. ICES J. Mar. Sci. 73, 572–581 (2016).
Google Scholar
Fidler, F., Burgman, M. A., Cumming, G., Buttrose, R. & Thomason, N. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology. Conserv. Biol. 20, 1539–1544 (2006).
Google Scholar
Forstmeier, W. & Schielzeth, H. Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner’s curse. Behav. Ecol. Sociobiol. 65, 47–55 (2011).
Google Scholar
Gillespie, B. R., Desmet, S., Kay, P., Tillotson, M. R. & Brown, L. E. A critical analysis of regulated river ecosystem responses to managed environmental flows from reservoirs. Freshw. Biol. 60, 410–425 (2015).
Google Scholar
Haddaway, N. R., Styles, D. & Pullin, A. S. Evidence on the environmental impacts of farm land abandonment in high altitude/mountain regions: a systematic map. Environ. Evid. 3, 17 (2014).
Google Scholar
Heffner, R. A., Butler, M. J. & Reilly, C. K. Pseudoreplication revisited. Ecology 77, 2558–2562 (1996).
Google Scholar
Holman, L., Head, M. L., Lanfear, R. & Jennions, M. D. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS Biol. 13, e1002190 (2015).
Google Scholar
Hurlbert, S. H. & White, M. D. Experiments with freshwater invertebrate zooplanktivores: quality of statistical analyses. Bull. Mar. Sci. 53, 128–153 (1993).
Johnson, W. T.3rd & Freeberg, T. M. Pseudoreplication in use of predator stimuli in experiments on antipredator responses. Anim. Behav. 119, 161–164 (2016).
Google Scholar
Kozlov, M. V. Pseudoreplication in ecological research: the problem overlooked by Russian scientists. Zh. Obshch. Biol. 64, 292–307 (2003).
Google Scholar
Kozlov, M. V. Plant studies on fluctuating asymmetry in Russia: mythology and methodology. Russ. J. Ecol. 48, 1–9 (2017).
Google Scholar
McDonald, S., Cresswell, T., Hassell, K. & Keough, M. Experimental design and statistical analysis in aquatic live animal radiotracing studies: a systematic review. Crit. Rev. Environ. Sci. Technol. 52, 2772–2801 (2021).
Google Scholar
Møller, A. P., Thornhill, R. & Gangestad, S. W. Direct and indirect tests for publication bias: asymmetry and sexual selection. Anim. Behav. 70, 497–506 (2005).
Google Scholar
Mrosovsky, N. & Godfrey, M. H. The path from grey literature to Red Lists. Endang. Species Res. 6, 185–191 (2008).
O’Brien, C., van Riper, C.3rd & Myers, D. E. Making reliable decisions in the study of wildlife diseases: using hypothesis tests, statistical power, and observed effects. J. Wildl. Dis. 45, 700–712 (2009).
Google Scholar
Parker, T. H. What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology. Biol. Rev. Camb. Philos. Soc. 88, 511–536 (2013).
Google Scholar
Ramage, B. S. et al. Pseudoreplication in tropical forests and the resulting effects on biodiversity conservation. Conserv. Biol. 27, 364–372 (2013).
Google Scholar
Sallabanks, R., Arnett, E. B. & Marzluff, J. M. An evaluation of research on the effects of timber harvest on bird populations. Wildl. Soc. Bull. 28, 1144–1155 (2000).
Sánchez-Tójar, A. et al. Meta-analysis challenges a textbook example of status signalling and demonstrates publication bias. eLife 7, e37385 (2018).
Google Scholar
Waller, B., Warmelink, L., Liebal, K., Micheletta, J. & Slocombe, K. Pseudoreplication: a widespread problem in primate communication research. Anim. Behav. 86, 483–488 (2013).
Google Scholar
Van Wilgenburg, E. & Elgar, M. A. Confirmation bias in studies of nestmate recognition: a cautionary note for research into the behaviour of animals. PLoS ONE 8, e53548 (2013).
Google Scholar
Yoccoz, N. G. Use, overuse, and misuse of significance tests in evolutionary biology and ecology. Bull. Ecol. Soc. Am. 72, 106–111 (1991).
Zaitsev, A. S., Gongalsky, K. B., Malmström, A., Persson, T. & Bengtsson, J. Why are forest fires generally neglected in soil fauna research? A mini-review. Appl. Soil Ecol. 98, 261–271 (2016).
Google Scholar
Zvereva, E. L. & Kozlov, M. V. Biases in studies of spatial patterns in insect herbivory. Ecol. Monogr. 89, e01361 (2019).
Google Scholar
RStudio Team. RStudio: Integrated Development for R (RStudio, 2020).
Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48 (2010).
Google Scholar
Source: Ecology - nature.com
