in

Improving quantitative synthesis to achieve generality in ecology

  • Houlahan, J. E., McKinney, S. T., Anderson, T. M. & McGill, B. J. The priority of prediction in ecological understanding. Oikos 126, 1–7 (2017).

    Article 

    Google Scholar 

  • Lawton, J. H. Are there general laws in ecology? Oikos 84, 177–192 (1999).

    Article 

    Google Scholar 

  • Elliott-Graves, A. Generality and causal interdependence in ecology. Philos. Sci. 85, 1102–1114 (2018).

    Article 

    Google Scholar 

  • Fox, J. W. The many roads to generality in ecology. Philos. Top. 9, 83–104 (2019).

    Article 

    Google Scholar 

  • McGill, B. J. et al. Species abundance distributions: moving beyond single prediction theories to integration within an ecological framework. Ecol. Lett. 10, 995–1015 (2007).

    Article 
    PubMed 

    Google Scholar 

  • MacArthur, R. H. & Wilson, E. O. An equilibrium theory of insular zoogeography. Evolution 17, 373–387 (1963).

    Article 

    Google Scholar 

  • Gurevitch, J., Fox, G. A., Wardle, G. M., Inderjit & Taub, D. Emergent insights from the synthesis of conceptual frameworks for biological invasions. Ecol. Lett. 14, 407–418 (2011).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Borer, E. T. et al. Finding generality in ecology: a model for globally distributed experiments. Methods Ecol. Evol. 5, 65–73 (2014).

    Article 

    Google Scholar 

  • Gurevitch, J., Koricheva, J., Nakagawa, S. & Stewart, G. Meta-analysis and the science of research synthesis. Nature 555, 175–182 (2018).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Anderson, S. C. et al. Trends in ecology and conservation over eight decades. Front. Ecol. Environ. 19, 274–282 (2021).

    Article 

    Google Scholar 

  • Kneale, D., Thomas, J., O’Mara-Eves, A. & Wiggins, R. How can additional secondary data analysis of observational data enhance the generalisability of meta-analytic evidence for local public health decision making? Res. Synth. Methods 10, 44–56 (2019).

    Article 
    PubMed 

    Google Scholar 

  • Aguinis, H., Pierce, C. A., Bosco, F. A., Dalton, D. R. & Dalton, C. M. Debunking myths and urban legends about meta-analysis. Organ. Res. Methods 14, 306–331 (2011).

    Article 

    Google Scholar 

  • Polit, D. F. & Beck, C. T. Generalization in quantitative and qualitative research: myths and strategies. Int. J. Nurs. Stud. 47, 1451–1458 (2010).

    Article 
    PubMed 

    Google Scholar 

  • Cardinale, B. J., Gonzalez, A., Allington, G. R. H. & Loreau, M. Is local biodiversity declining or not? A summary of the debate over analysis of species richness time trends. Biol. Conserv. 219, 175–183 (2018).

    Article 

    Google Scholar 

  • Lundberg, I., Johnson, R. & Stewart, B. M. What is your estimand? Defining the target quantity connects statistical evidence to theory. Am. Sociol. Rev. 86, 532–565 (2021).

    Article 

    Google Scholar 

  • Lawrance, R. et al. What is an estimand & how does it relate to quantifying the effect of treatment on patient-reported quality of life outcomes in clinical trials? J. Patient-Rep. Outcomes 4, 68 (2020).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Findley, M. G., Kikuta, K. & Denly, M. External validity. Annu. Rev. Polit. Sci. 24, 365–393 (2021).

    Article 

    Google Scholar 

  • Pearl, J. & Bareinboim, E. External validity: from do-calculus to transportability across populations. Stat. Sci. 29, 579–595 (2014).

    Article 

    Google Scholar 

  • Westreich, D., Edwards, J. K., Lesko, C. R., Cole, S. R. & Stuart, E. A. Target validity and the hierarchy of study designs. Am. J. Epidemiol. 188, 438–443 (2019).

    Article 
    PubMed 

    Google Scholar 

  • Carpenter, C. J. Meta-analyzing apples and oranges: how to make applesauce instead of fruit salad. Hum. Commun. Res. 46, 322–333 (2020).

    Article 

    Google Scholar 

  • Rohrer, J. M. & Arslan, R. C. Precise answers to vague questions: issues with interactions. Adv. Methods Pract. Psychol. Sci. 4, 1–19 (2021).

    Google Scholar 

  • Breslow, N. E. & Clayton, D. G. Approximate inference in generalized linear mixed models. J. Am. Stat. Assoc. 88, 9–25 (1993).

    Google Scholar 

  • Koricheva, J. & Gurevitch, J. Uses and misuses of meta-analysis in plant ecology. J. Ecol. 102, 828–844 (2014).

    Article 

    Google Scholar 

  • Gonzalez, A. et al. Estimating local biodiversity change: a critique of papers claiming no net loss of local diversity. Ecology 97, 1949–1960 (2016).

    Article 
    PubMed 

    Google Scholar 

  • Konno, K. et al. Ignoring non-English-language studies may bias ecological meta-analyses. Ecol. Evol. 10, 6373–6384 (2020).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Nakagawa, S. et al. Methods for testing publication bias in ecological and evolutionary meta-analyses. Methods Ecol. Evol. 13, 4–21 (2022).

    Article 

    Google Scholar 

  • Rosenthal, R. The file drawer problem and tolerance for null results. Psychol. Bull. 86, 638–641 (1979).

    Article 

    Google Scholar 

  • Leung, B. et al. Clustered versus catastrophic global vertebrate declines. Nature 588, 267–271 (2020).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Rothman, K. J., Gallacher, J. E. J. & Hatch, E. E. Why representativeness should be avoided. Int. J. Epidemiol. 42, 1012–1014 (2013).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Spake, R. et al. Implications of scale dependence for cross-study syntheses of biodiversity differences. Ecol. Lett. 24, 374–390 (2021).

    Article 
    PubMed 

    Google Scholar 

  • Spake, R. & Doncaster, C. P. Use of meta-analysis in forest biodiversity research: key challenges and considerations. For. Ecol. Manag. 400, 429–437 (2017).

    Article 

    Google Scholar 

  • Christie, A. P. et al. Simple study designs in ecology produce inaccurate estimates of biodiversity responses. J. Appl. Ecol. 56, 2742–2754 (2019).

    Article 

    Google Scholar 

  • Nakagawa, S., Noble, D. W. A., Senior, A. M. & Lagisz, M. Meta-evaluation of meta-analysis: ten appraisal questions for biologists. BMC Biol. 15, 18 (2017).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Higgins, J. P. T. & Thompson, S. G. Quantifying heterogeneity in a meta-analysis. Stat. Med. 21, 1539–1558 (2002).

    Article 
    PubMed 

    Google Scholar 

  • Schielzeth, H. & Nakagawa, S. Conditional repeatability and the variance explained by reaction norm variation in random slope models. Methods Ecol. Evol. 13, 1214–1223 (2022).

    Article 

    Google Scholar 

  • Nakagawa, S. et al. The orchard plot: cultivating a forest plot for use in ecology, evolution, and beyond. Res. Synth. Methods 12, 4–12 (2021).

    Article 
    PubMed 

    Google Scholar 

  • Lorah, J. Effect size measures for multilevel models: definition, interpretation, and TIMSS example. Large-Scale Assess. Educ. 6, 8 (2018).

    Article 

    Google Scholar 

  • O’Connor, M. I. et al. A general biodiversity–function relationship is mediated by trophic level. Oikos 126, 18–31 (2017).

    Article 

    Google Scholar 

  • Ojha, M., Naidu, D. G. T. & Bagchi, S. Meta-analysis of induced anti-herbivore defence traits in plants from 647 manipulative experiments with natural and simulated herbivory. J. Ecol. 110, 799–816 (2022).

  • Dodds, K. C. et al. Material type influences the abundance but not richness of colonising organisms on marine structures. J. Environ. Manag. 307, 114549 (2022).

    Article 

    Google Scholar 

  • Dornelas, M. et al. Assemblage time series reveal biodiversity change but not systematic loss. Science 344, 296–299 (2014).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Senior, A. M. et al. Heterogeneity in ecological and evolutionary meta- analyses: its magnitude and implications. Ecology 97, 3293–3299 (2016).

    Article 
    PubMed 

    Google Scholar 

  • Blowes, S. A. et al. The geography of biodiversity change in marine and terrestrial assemblages. Science 366, 339–345 (2019).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Nakagawa, S. & Cuthill, I. C. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. 82, 591–605 (2007).

    Article 
    PubMed 

    Google Scholar 

  • Glass, G. V. Primary, secondary, and meta-analysis of research. Educ. Res. 5, 3–8 (1976).

    Article 

    Google Scholar 

  • Glass, G. V. Meta‐analysis at 25: a personal history. Education in Two Worlds https://ed2worlds.blogspot.com/2022/07/meta-analysis-at-25-personal-history.html (2000).

  • Cooper, H. M. Organizing knowledge syntheses: a taxonomy of literature reviews. Knowl. Soc. 1, 104–126 (1988).

    Google Scholar 

  • Soranno, P. A. et al. Cross-scale interactions: quantifying multi-scaled cause-effect relationships in macrosystems. Front. Ecol. Environ. 12, 65–73 (2014).

    Article 

    Google Scholar 

  • Gerstner, K. et al. Will your paper be used in a meta-analysis? Make the reach of your research broader and longer lasting. Methods Ecol. Evol. 8, 777–784 (2017).

    Article 

    Google Scholar 

  • Hortal, J. et al. Seven shortfalls that beset large-scale knowledge of biodiversity. Annu. Rev. Ecol. Evol. Syst. 46, 523–549 (2015).

    Article 

    Google Scholar 

  • Simons, D. J., Shoda, Y. & Lindsay, D. S. Constraints on Generality (CoG): a proposed addition to all empirical papers. Perspect. Psychol. Sci. 12, 1123–1128 (2017).

    Article 
    PubMed 

    Google Scholar 

  • Yarkoni, T. The generalizability crisis. Behav. Brain Sci. https://doi.org/10.1017/S0140525X20001685 (2020).

  • Lopez, P. M., Subramanian, S. V. & Schooling, C. M. Effect measure modification conceptualized using selection diagrams as mediation by mechanisms of varying population-level relevance. J. Clin. Epidemiol. 113, 123–128 (2019).

    Article 
    PubMed 

    Google Scholar 

  • Campbell, D. T. in Advances in QuasiExperimental Design and Analysis (ed. Trochim, W.) 67–77 (Jossey-Bass, 1986).

  • Spake, R. et al. Meta‐analysis of management effects on biodiversity in plantation and secondary forests of Japan. Conserv. Sci. Pract. 1, e14 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Forest Ecosystem Diversity Basic Survey (in Japanese) (Forestry Agency of Japan, 2019); https://www.rinya.maff.go.jp/j/keikaku/tayouseichousa/index.html

  • Ito, S., Ishigamia, S., Mizoue, N. & Buckley, G. P. Maintaining plant species composition and diversity of understory vegetation under strip-clearcutting forestry in conifer plantations in Kyushu, southern Japan. For. Ecol. Manag. 231, 234–241 (2006).

    Article 

    Google Scholar 

  • Utsugi, E. et al. Hardwood recruitment into conifer plantations in Japan: effects of thinning and distance from neighboring hardwood forests. For. Ecol. Manag. 237, 15–28 (2006).

    Article 

    Google Scholar 

  • Kominami, Y. et al. Classification of bird-dispersed plants by fruiting phenology, fruit size, and growth form in a primary lucidophyllous forest: an analysis, with implications for the conservation of fruit–bird interactions. Ornthological Sci. 2, 3–23 (2003).

    Article 

    Google Scholar 

  • Tsujino, R. & Matsui, K. Forest regeneration inhibition in a mixed broadleaf-conifer forest under sika deer pressure. J. For. Res. 27, 230–235 (2021).

    Article 

    Google Scholar 

  • Spake, R., Soga, M., Catford, J. A. & Eigenbrod, F. Applying the stress-gradient hypothesis to curb the spread of invasive bamboo. J. Appl. Ecol. 58, 1993–2003 (2021).

    Article 

    Google Scholar 

  • Mize, T. D. Best practices for estimating, interpreting, and presenting nonlinear interaction effects. Sociol. Sci. 6, 81–117 (2019).

    Article 

    Google Scholar 

  • Karaca-Mandic, P., Norton, E. C. & Dowd, B. Interaction terms in nonlinear models. Health Serv. Res. 47, 255–274 (2012).

    Article 
    PubMed 

    Google Scholar 

  • Spake, R. et al. Forest damage by deer depends on cross-scale interactions between climate, deer density and landscape structure. J. Appl. Ecol. 57, 1376–1390 (2020).

  • McCabe, C. J., Kim, D. S. & King, K. M. Improving present practices in the visual display of interactions. Adv. Methods Pract. Psychol. Sci. 1, 147–165 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Shackelford, G. E. et al. Dynamic meta-analysis: a method of using global evidence for local decision making. BMC Biol. 19, 33 (2021).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Christie, A. P. et al. Innovation and forward‐thinking are needed to improve traditional synthesis methods: a response to Pescott and Stewart. J. Appl. Ecol. 59, 1191–1197 (2022).

    Article 

    Google Scholar 

  • Haddaway, N. R. et al. EviAtlas: a tool for visualising evidence synthesis databases. Environ. Evid. 8, 22 (2019).

  • Delory, B. M., Li, M., Topp, C. N. & Lobet, G. archiDART v3.0: a new data analysis pipeline allowing the topological analysis of plant root systems. F1000Research 7, 22 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Perkel, J. M. The future of scientific figures. Nature 554, 133–134 (2018).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Weaver, S. & Gleeson, M. P. The importance of the domain of applicability in QSAR modeling. J. Mol. Graph. Model. 26, 1315–1326 (2008).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Sutton, C. et al. Identifying domains of applicability of machine learning models for materials science. Nat. Commun. 11, 4428 (2020).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar 

  • Meyer, H. & Pebesma, E. Predicting into unknown space? Estimating the area of applicability of spatial prediction models. Methods Ecol. Evol. 12, 1620–1633 (2021).

    Article 

    Google Scholar 

  • Pearl, J. & Bareinboim, E. Transportability of causal and statistical relations: a formal approach. In 2011 IEEE 11th International Conference on Data Mining Workshops https://doi.org/10.1109/ICDMW.2011.169 (IEEE, 2011).

  • Munthe-Kaas, H., Nøkleby, H. & Nguyen, L. Systematic mapping of checklists for assessing transferability. Syst. Rev. 8, 22 (2019).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Dekkers, O. M., von Elm, E., Algra, A., Romijn, J. A. & Vandenbroucke, J. P. How to assess the external validity of therapeutic trials: a conceptual approach. Int. J. Epidemiol. 39, 89–94 (2010).

    Article 
    PubMed 
    CAS 

    Google Scholar 

  • Schloemer, T. & Schröder-Bäck, P. Criteria for evaluating transferability of health interventions: a systematic review and thematic synthesis. Implement. Sci. 13, 88 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Fernandez-Hermida, J. R., Calafat, A., Becoña, E., Tsertsvadze, A. & Foxcroft, D. R. Assessment of generalizability, applicability and predictability (GAP) for evaluating external validity in studies of universal family-based prevention of alcohol misuse in young people: systematic methodological review of randomized controlled trials. Addiction 107, 1570–1579 (2012).

    Article 
    PubMed 

    Google Scholar 

  • Avellar, S. A. et al. External validity: the next step for systematic reviews? Eval. Rev. 41, 283–325 (2017).

    Article 
    PubMed 

    Google Scholar 

  • Bareinboim, E. & Pearl, J. A general algorithm for deciding transportability of experimental results. J. Causal Inference 1, 107–134 (2013).

    Article 

    Google Scholar 

  • Degtiar, I. & Rose, S. A review of generalizability and transportability. Preprint at https://doi.org/10.48550/arXiv.2102.11904 (2021).

  • Bareinboim, E. & Pearl, J. Meta-transportability of causal effects: a formal approach. J. Mach. Learn. Res. 31, 135–143 (2013).

    Google Scholar 

  • Jamieson, D. Scientific uncertainty: how do we know when to communicate research findings to the public? Sci. Total Environ. 184, 103–107 (1996).

    Article 
    CAS 

    Google Scholar 

  • Burchett, H. E. D., Mayhew, S. H., Lavis, J. N. & Dobrow, M. J. When can research from one setting be useful in another? Understanding perceptions of the applicability and transferability of research. Health Promot. Int. 28, 418–430 (2013).

    Article 
    PubMed 

    Google Scholar 

  • Forscher, P. et al. Build up big-team science. Nature 601, 505–507 (2022).

    Article 

    Google Scholar 

  • Whalen, M. A. et al. Climate drives the geography of marine consumption by changing predator communities. Proc. Natl Acad. Sci. USA 117, 28160–28166 (2020).

    Article 
    PubMed 
    PubMed Central 
    CAS 

    Google Scholar 

  • Moshontz, H. et al. The Psychological Science Accelerator: advancing psychology through a distributed collaborative network. Adv. Methods Pract. Psychol. Sci. 1, 501–515 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Marschner, I. C. A general framework for the analysis of adaptive experiments. Stat. Sci. 36, 465–492 (2021).

    Article 

    Google Scholar 

  • Clark, M. Shrinkage in Mixed Effects Models https://m-clark.github.io/posts/2019-05-14-shrinkage-in-mixed-models/ (2019).

  • Gurevitch, J. & Hedges, L. V. Statistical issues in ecological meta-analyses. Ecology 80, 1142–1149 (1999).

    Article 

    Google Scholar 

  • Mengersen, K., Gurevitch, J. & Schmid, C. H. in Handbook of Meta-analysis in Ecology and Evolution (eds Koricheva, U. et al.) 300–312 (Princeton Univ. Press, 2013).

  • Hudson, L. N. et al. The database of the PREDICTS (Projecting Responses of Ecological Diversity In Changing Terrestrial Systems) project. Ecol. Evol. 7, 145–188 (2017).

    Article 
    PubMed 

    Google Scholar 

  • Dornelas, M. et al. BioTIME: a database of biodiversity time series for the Anthropocene. Glob. Ecol. Biogeogr. 27, 760–786 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Salguero-Gómez, R. et al. The COMPADRE Plant Matrix Database: an open online repository for plant demography. J. Ecol. 103, 202–218 (2015).

    Article 

    Google Scholar 

  • Salguero-Gómez, R. et al. COMADRE: a global data base of animal demography. J. Anim. Ecol. 85, 371–384 (2016).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar 

  • Pastor, D. A. & Lazowski, R. A. On the multilevel nature of meta-analysis: a tutorial, comparison of software programs, and discussion of analytic choices. Multivar. Behav. Res. 53, 74–89 (2018).

    Article 

    Google Scholar 


  • Source: Ecology - nature.com

    Methane research takes on new urgency at MIT

    Ocean microbes get their diet through a surprising mix of sources, study finds