Ioannidis, J. P. A., Fanelli, D., Dunne, D. D. & Goodman, S. N. Meta-research: evaluation and improvement of research methods and practices. PLoS Biol. 13, e1002264 (2015).PubMed
PubMed Central
Article
CAS
Google Scholar
Hampton, S. E. et al. The Tao of open science for ecology. Ecosphere 6, art120 (2015).Article
Google Scholar
Rothstein, H. R., Sutton, A. J. & Borenstein, M. Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments (John Wiley & Sons, 2005).Sutton, A. J. in The Handbook of Research Synthesis and Meta-Analysis (eds Cooper, H. et al.) 435–452 (Russell Sage Foundation, 2009).Nakagawa, S., Koricheva, J., Macleod, M. & Viechtbauer, W. Introducing our series: research synthesis and meta-research in biology. BMC Biol. 18, 20 (2020).PubMed
PubMed Central
Article
Google Scholar
Nakagawa, S. et al. A new ecosystem for evidence synthesis. Nat. Ecol. Evol. 4, 498–501 (2020).PubMed
Article
Google Scholar
Coolidge, H. J. & Lord, R. H. in Archibald Cary Coolidge: Life and Letters 308 (Houghton Mifflin Harcourt, 1932).Nickerson, R. S. Confirmation bias: a ubiquitous phenomenon in many guises. Rev. Gen. Psychol. 2, 175–220 (1998).Article
Google Scholar
Touchon, J. C. & McCoy, M. W. The mismatch between current statistical practice and doctoral training in ecology. Ecosphere 7, e01394 (2016).Article
Google Scholar
Begley, C. G. & Ellis, L. M. Raise standards for preclinical cancer research. Nature 483, 531–533 (2012).CAS
PubMed
Article
Google Scholar
Aarts, A. A. et al. Estimating the reproducibility of psychological science. Science 349, aac4716 (2015).Article
CAS
Google Scholar
Camerer, C. F. et al. Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nat. Hum. Behav. 2, 637–644 (2018).PubMed
Article
Google Scholar
Fraser, H., Parker, T., Nakagawa, S., Barnett, A. & Fidler, F. Questionable research practices in ecology and evolution. PLoS ONE 13, e0200303 (2018).PubMed
PubMed Central
Article
CAS
Google Scholar
Culina, A., van den Berg, I., Evans, S. & Sánchez-Tójar, A. Low availability of code in ecology: a call for urgent action. PLoS Biol. 18, e3000763 (2020).CAS
PubMed
PubMed Central
Article
Google Scholar
Jennions, M. D. & Møller, A. P. Publication bias in ecology and evolution: an empirical assessment using the ‘trim and fill’ method. Biol. Rev. Camb. Philos. Soc. 77, 211–222 (2002).PubMed
Article
Google Scholar
Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behavior. Behav. Ecol. 14, 438–445 (2003).Article
Google Scholar
Cassey, P., Ewen, J. G., Blackburn, T. M. & Møller, A. P. A survey of publication bias within evolutionary ecology. Proc. Biol. Sci. 271, S451–S454 (2004).PubMed
PubMed Central
Article
Google Scholar
Kardish, M. R. et al. Blind trust in unblinded observation in ecology, evolution, and behavior. Front. Ecol. Evol. 3, 51 (2015).Article
Google Scholar
Jennions, M. D. & Møller, A. P. Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution. Proc. Biol. Sci. 269, 43–48 (2002).PubMed
PubMed Central
Article
Google Scholar
Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016).CAS
PubMed
Article
Google Scholar
Chalmers, I. & Glasziou, P. Avoidable waste in the production and reporting of research evidence. Lancet 374, 86–89 (2009).PubMed
Article
Google Scholar
Altman, D. G. The scandal of poor medical research. BMJ 308, 283–284 (1994).CAS
PubMed
PubMed Central
Article
Google Scholar
Glasziou, P. & Chalmers, I. Is 85% of health research really ‘wasted’? BMJ Opinion (14 January 2016).Glasziou, P. & Chalmers, I. Research waste is still a scandal. BMJ 363, k4645 (2018).Article
Google Scholar
Chalmers, I. et al. How to increase value and reduce waste when research priorities are set. Lancet 383, 156–165 (2014).PubMed
Article
Google Scholar
Kunin, W. E. Robust evidence of declines in insect abundance and biodiversity. Nature 574, 641–642 (2019).CAS
PubMed
Article
Google Scholar
Christie, A. P. et al. Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences. Nat. Commun. 11, 6377 (2020).CAS
PubMed
PubMed Central
Article
Google Scholar
Campbell, H. A. et al. Finding our way: on the sharing and reuse of animal telemetry data in Australasia. Sci. Total Environ. 534, 79–84 (2015).CAS
PubMed
Article
Google Scholar
Koricheva, J. Non-significant results in ecology: a burden or a blessing in disguise? Oikos 102, 397–401 (2003).Article
Google Scholar
Bennett, L. T. & Adams, M. A. Assessment of ecological effects due to forest harvesting: approaches and statistical issues. J. Appl. Ecol. 41, 585–598 (2004).Article
Google Scholar
Duval, S. & Tweedie, R. A nonparametric ‘trim and fill’ method of accounting for publication bias in meta-analysis. J. Am. Stat. Assoc. 95, 89–98 (2012).
Google Scholar
Brlík, V. et al. Weak effects of geolocators on small birds: a meta-analysis controlled for phylogeny and publication bias. J. Anim. Ecol. 89, 207–220 (2020).PubMed
Article
Google Scholar
Hurlbert, S. H. Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54, 187–211 (1984).Article
Google Scholar
Jennions, M. D. & Møller, A. P. A survey of the statistical power of research in behavioral ecology and animal behaviour. Behav. Ecol. 14, 438–445 (2003).Article
Google Scholar
Culina, A., Purgar, M. & Klanjscek, T. Datasets and codes for Purgar et al. 2022: quantifying research waste in ecology. Zenodo https://zenodo.org/record/6566100#.YrLWB-zMIqs (2022).Ferguson, C. et al. Europe PMC in 2020. Nucleic Acids Res. 49, D1507–D1514 (2021).CAS
PubMed
Article
Google Scholar
Huang, C.-K. et al. Meta-Research: Evaluating the impact of open access policies on research institutions. eLife 9, e57067 (2020).PubMed
PubMed Central
Article
Google Scholar
Ross-Hellauer, T. Open science, done wrong, will compound inequities. Nature 603, 363 (2022).CAS
PubMed
Article
Google Scholar
Smith, A. C. et al. Assessing the effect of article processing charges on the geographic diversity of authors using Elsevier’s ‘Mirror journal’ system. Quant. Sci. Stud. 2, 1123–1143 (2021).Article
Google Scholar
Christie, A. P. et al. Reducing publication delay to improve the efficiency and impact of conservation science. PeerJ 9, e12245 (2021).PubMed
PubMed Central
Article
Google Scholar
Desjardins-Proulx, P. et al. The case for open preprints in biology. PLoS Biol. 11, e1001563 (2013).CAS
PubMed
PubMed Central
Article
Google Scholar
Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, 0021 (2017).PubMed
PubMed Central
Article
Google Scholar
O’Dea, R. E. et al. Towards open, reliable, and transparent ecology and evolutionary biology. BMC Biol. 19, 68 (2021).PubMed
PubMed Central
Article
Google Scholar
Culina, A. et al. Navigating the unfolding open data landscape in ecology and evolution. Nat. Ecol. Evol. 2, 420–426 (2018).PubMed
Article
Google Scholar
Culina, A., Crowther, T. W., Ramakers, J. J. C., Gienapp, P. & Visser, M. E. How to do meta-analysis of open datasets. Nat. Ecol. Evol. 2, 1053–1056 (2018).PubMed
Article
Google Scholar
Roche, D. G., Kruuk, L. E. B., Lanfear, R. & Binning, S. A. Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295 (2015).PubMed
PubMed Central
Article
CAS
Google Scholar
Grainger, M. J., Bolam, F. C., Stewart, G. B. & Nilsen, E. B. Evidence synthesis for tackling research waste. Nat. Ecol. Evol. 4, 495–497 (2020).PubMed
Article
Google Scholar
Nørgaard, B. et al. Systematic reviews are rarely used to inform study design—a systematic review and meta-analysis. J. Clin. Epidemiol. 145, 1–13 (2022).PubMed
Article
Google Scholar
Webb, J. A. et al. Weaving common threads in environmental causal assessment methods: toward an ideal method for rapid evidence synthesis. Freshw. Sci. 36, 250–256 (2017).Article
Google Scholar
Collins, A., Coughlin, D., Miller, J. & Kirk, S. The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide (Joint Water Evidence Group, 2015).Carrick, J. et al. Is planting trees the solution to reducing flood risks? J. Flood Risk Manag. 12, e12484 (2019).Article
Google Scholar
Nuñez, M. A. & Amano, T. Monolingual searches can limit and bias results in global literature reviews. Nat. Ecol. Evol. 5, 264 (2021).PubMed
Article
Google Scholar
Morrison, A. et al. The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies. Int. J. Technol. Assess. Health Care 28, 138–144 (2012).PubMed
Article
Google Scholar
Wu, T., Li, Y., Bian, Z., Liu, G. & Moher, D. Randomized trials published in some Chinese journals: how many are randomized? Trials 10, 46 (2009).PubMed
PubMed Central
Article
Google Scholar
Vorobeichik, E. L. & Kozlov, M. V. Impact of point polluters on terrestrial ecosystems: methodology of research, experimental design, and typical errors. Russ. J. Ecol. 43, 89–96 (2012).Article
Google Scholar
Simmons, J. P., Nelson, L. D. & Simonsohn, U. False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci. 22, 1359–1366 (2011).PubMed
Article
Google Scholar
Transforming Our World: the 2030 Agenda for Sustainable Development (United Nations, 2015).MacCoun, R. & Perlmutter, S. Blind analysis: hide results to seek the truth. Nature 526, 187–189 (2015).CAS
PubMed
Article
Google Scholar
Parker, T. H. et al. Transparency in ecology and evolution: real problems, real solutions. Trends Ecol. Evol. 31, 711–719 (2016).PubMed
Article
Google Scholar
Announcement: reducing our irreproducibility. Nature 496, 398 (2013).Moher, D. et al. Increasing value and reducing waste in biomedical research: who’s listening? Lancet 387, 1573–1586 (2016).PubMed
Article
Google Scholar
Smaldino, P. E. & McElreath, R. The natural selection of bad science. R. Soc. Open Sci. 3, 160384 (2016).PubMed
PubMed Central
Article
Google Scholar
Vrieze, J. Landmark research integrity survey finds questionable practices are surprisingly common. ScienceInsider https://www.sciencemag.org/news/2021/07/landmark-research-integrity-survey-finds-questionable-practices-are-surprisingly-common (2021).Woolston, C. Impact factor abandoned by Dutch university in hiring and promotion decisions. Nature 595, 462 (2021).CAS
PubMed
Article
Google Scholar
Directorate-General for Research and Innovation (European Commission). Towards a Reform of the Research Assessment System. Scoping Report (Publications Office, 2021).Athena Research & Innovation Center, Directorate-General for Research and Innovation (European Commission), PPMI, UNU-MERIT. Monitoring the Open Access Policy of Horizon 2020. Final report (European Commission, 2021).Kwon, D. University of California and Elsevier forge open-access deal. TheScientist https://www.the-scientist.com/news-opinion/university-of-california-and-elsevier-forge-open-access-deal–68557 (2021).Vines, T. H. et al. Mandated data archiving greatly improves access to research data. FASEB J. 27, 1304–1308 (2013).CAS
PubMed
Article
Google Scholar
NPQIP Collaborative Group. Did a change in Nature journals’ editorial policy for life sciences research improve reporting? BMJ Open Sci. 3, e000035 (2019).Article
Google Scholar
Glasziou, P. et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 383, 267–276 (2014).PubMed
Article
Google Scholar
Fecher, B. & Friesike, S. in Opening Science: the Evolving Guide on How the Internet is Changing Research, Collaboration and Scholarly Publishing (eds Bartling, S. & Friesike, S.) 17–47 (Springer International Publishing, 2014).Hardwicke, T. E. et al. Calibrating the scientific ecosystem through meta-research. Annu. Rev. Stat. Appl. 7, 11–37 (2020).Article
Google Scholar
McKiernan, E. C. et al. How open science helps researchers succeed. eLife 5, e16800 (2016).PubMed
PubMed Central
Article
Google Scholar
Fidler, F. et al. Metaresearch for evaluating reproducibility in ecology and evolution. Bioscience 67, 282–289 (2017).PubMed
PubMed Central
Google Scholar
Cornwall, C. E. & Hurd, C. L. Experimental design in ocean acidification research: problems and solutions. ICES J. Mar. Sci. 73, 572–581 (2016).Article
Google Scholar
Fidler, F., Burgman, M. A., Cumming, G., Buttrose, R. & Thomason, N. Impact of criticism of null-hypothesis significance testing on statistical reporting practices in conservation biology. Conserv. Biol. 20, 1539–1544 (2006).PubMed
Article
Google Scholar
Forstmeier, W. & Schielzeth, H. Cryptic multiple hypotheses testing in linear models: overestimated effect sizes and the winner’s curse. Behav. Ecol. Sociobiol. 65, 47–55 (2011).PubMed
Article
Google Scholar
Gillespie, B. R., Desmet, S., Kay, P., Tillotson, M. R. & Brown, L. E. A critical analysis of regulated river ecosystem responses to managed environmental flows from reservoirs. Freshw. Biol. 60, 410–425 (2015).Article
Google Scholar
Haddaway, N. R., Styles, D. & Pullin, A. S. Evidence on the environmental impacts of farm land abandonment in high altitude/mountain regions: a systematic map. Environ. Evid. 3, 17 (2014).Article
Google Scholar
Heffner, R. A., Butler, M. J. & Reilly, C. K. Pseudoreplication revisited. Ecology 77, 2558–2562 (1996).Article
Google Scholar
Holman, L., Head, M. L., Lanfear, R. & Jennions, M. D. Evidence of experimental bias in the life sciences: why we need blind data recording. PLoS Biol. 13, e1002190 (2015).PubMed
PubMed Central
Article
CAS
Google Scholar
Hurlbert, S. H. & White, M. D. Experiments with freshwater invertebrate zooplanktivores: quality of statistical analyses. Bull. Mar. Sci. 53, 128–153 (1993).
Google Scholar
Johnson, W. T.3rd & Freeberg, T. M. Pseudoreplication in use of predator stimuli in experiments on antipredator responses. Anim. Behav. 119, 161–164 (2016).Article
Google Scholar
Kozlov, M. V. Pseudoreplication in ecological research: the problem overlooked by Russian scientists. Zh. Obshch. Biol. 64, 292–307 (2003).CAS
PubMed
Google Scholar
Kozlov, M. V. Plant studies on fluctuating asymmetry in Russia: mythology and methodology. Russ. J. Ecol. 48, 1–9 (2017).Article
Google Scholar
McDonald, S., Cresswell, T., Hassell, K. & Keough, M. Experimental design and statistical analysis in aquatic live animal radiotracing studies: a systematic review. Crit. Rev. Environ. Sci. Technol. 52, 2772–2801 (2021).Article
Google Scholar
Møller, A. P., Thornhill, R. & Gangestad, S. W. Direct and indirect tests for publication bias: asymmetry and sexual selection. Anim. Behav. 70, 497–506 (2005).Article
Google Scholar
Mrosovsky, N. & Godfrey, M. H. The path from grey literature to Red Lists. Endang. Species Res. 6, 185–191 (2008).
Google Scholar
O’Brien, C., van Riper, C.3rd & Myers, D. E. Making reliable decisions in the study of wildlife diseases: using hypothesis tests, statistical power, and observed effects. J. Wildl. Dis. 45, 700–712 (2009).PubMed
Article
Google Scholar
Parker, T. H. What do we really know about the signalling role of plumage colour in blue tits? A case study of impediments to progress in evolutionary biology. Biol. Rev. Camb. Philos. Soc. 88, 511–536 (2013).PubMed
Article
Google Scholar
Ramage, B. S. et al. Pseudoreplication in tropical forests and the resulting effects on biodiversity conservation. Conserv. Biol. 27, 364–372 (2013).PubMed
Article
Google Scholar
Sallabanks, R., Arnett, E. B. & Marzluff, J. M. An evaluation of research on the effects of timber harvest on bird populations. Wildl. Soc. Bull. 28, 1144–1155 (2000).
Google Scholar
Sánchez-Tójar, A. et al. Meta-analysis challenges a textbook example of status signalling and demonstrates publication bias. eLife 7, e37385 (2018).PubMed
PubMed Central
Article
Google Scholar
Waller, B., Warmelink, L., Liebal, K., Micheletta, J. & Slocombe, K. Pseudoreplication: a widespread problem in primate communication research. Anim. Behav. 86, 483–488 (2013).Article
Google Scholar
Van Wilgenburg, E. & Elgar, M. A. Confirmation bias in studies of nestmate recognition: a cautionary note for research into the behaviour of animals. PLoS ONE 8, e53548 (2013).CAS
PubMed
PubMed Central
Article
Google Scholar
Yoccoz, N. G. Use, overuse, and misuse of significance tests in evolutionary biology and ecology. Bull. Ecol. Soc. Am. 72, 106–111 (1991).
Google Scholar
Zaitsev, A. S., Gongalsky, K. B., Malmström, A., Persson, T. & Bengtsson, J. Why are forest fires generally neglected in soil fauna research? A mini-review. Appl. Soil Ecol. 98, 261–271 (2016).Article
Google Scholar
Zvereva, E. L. & Kozlov, M. V. Biases in studies of spatial patterns in insect herbivory. Ecol. Monogr. 89, e01361 (2019).Article
Google Scholar
RStudio Team. RStudio: Integrated Development for R (RStudio, 2020).Viechtbauer, W. Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48 (2010).Article
Google Scholar More