Sustainable soil management practices are associated with increases in crop defense through soil microbiome changes

Microbial communities are essential for the health of ecosystems and promote numerous beneficial interactions between hosts and the environment¹. Thus, the global loss of microbial biodiversity jeopardizes beneficial interactions and the ecosystem functions they support^2,3. More diverse microbiomes are generally considered to be more stable and to provide more benefits for hosts and the ecosystem⁴, however, a deeper understanding of links between microbiome diversity and ecosystem services is still required to harness soil microbiome benefits and promote plant health. For instance, while greater soil microbiome diversity is associated with increased plant defenses and pest suppression⁵ and functional redundancy in the microbiome mediates the stability of ecosystem services generally⁶, the specific dimensions of microbiome diversity driving these benefits remain largely unknown and may depend on agroecosystem conditions. This lack of knowledge linking the microbiome with function limits our ability to predict how biodiversity loss will affect ecosystem services and constrains management efforts to steer microbiomes in agroecosystems^1,2,3.

The function of microbiome biodiversity can be further informed by socio-ecological models. Socio-ecological models recognize that beliefs along with economic, demographic, and farming system characteristics influence ecosystem services⁷. Organic agriculture is a key example of a socio-ecological system where the dimensions of microbiome biodiversity and associated ecosystem services are influenced by social, political, and economic variables⁸. For example, organic farmers may adopt different practices to meet the requirements of federal regulations, based on the cost of practice implementation, and the abiotic and economic contexts of their farms each year⁹. The theory of planned behavior explains that variation in practice adoption is also driven by beliefs, which include the benefits farmers assume to be true about a practice^10,11. This suggests the adoption of microbiome supportive management may vary by farmers beliefs in the economic benefits of the microbiome, and beliefs in which practices are microbiome-friendly.

Accumulating evidence indicates that a range of practices used within organic farming can increase soil microbial diversity and enhance plant associations with beneficial microbes in the rhizosphere, promoting ecosystem services and plant health when compared to conventional farms^12,13. Increased soil microbiome diversity on organic farms has been shown to mediate increased crop plant resilience to insect herbivores via changes in plant chemistry^5,14. While general practices that support microbiome diversity are increasingly well understood¹⁵, there remains a need to move beyond canonical organic and conventional comparisons towards a holistic understanding of agroecosystems that steer soil microbiomes towards pest suppression. Organic farms present such an opportunity because of variation in their management⁹ creating a natural experiment to identify farmers who are adopting practices that support soil microbiome biodiversity and functions on-farm, which can be shared with the farming community to enhance sustainable regulation of pest populations across farms.

Previously, we surveyed 85 organic farmers across New York (NY) state on their beliefs in the microbiome and farm characteristics⁹. In this published work, we clustered farmers by their beliefs in the factors mediating the microbiome population on their farm⁹, a process that classified participants into seven farmer groups (Fig. 1, Supplementary Figs. 1, 2). In this study, we used the same 85 organic farms to test the impact of differences in agronomic practice adoption across farms on soil microbiome diversity and functions using farmer-collected samples. To begin, we conducted soil microbiome metabarcoding with samples from across the 85 farms, and performed lab bioassays with field soil. Next, we used machine learning, which identified five farming practices (no tillage, cover cropping with grasses, pesticide use, composting, and targeted irrigation) that were linked to different aspects of microbiome diversity and plant defense responses. Specifically, we found loss of bacterial groups, replacement of fungal taxa, reduced abundance of Fusarium chlamydosporum, and increased abundance of Microtrichales spp., were all associated with increases in the plant defense compound jasmonic acid (JA), while increased abundance of Paenibacillus senegalensis was linked with decreases in two plant defense compounds, JA and salicylic acid (SA), across organic farms. We then used structural equation models (SEMs) to understand how farmer beliefs and other farm characteristics influence the adoption of these practices and their ecological outcomes. These SEMs revealed practices where adoption was driven by farmer beliefs, yielding insights for future extension approaches to support sustainable, microbiome-based pest management.

Soil microbial communities vary across organic farms in New York state

Farmers were asked to submit up to two soil samples with distinct suites of practices to ensure broad representations of different practices across NY state ⁹. We received 136 soil samples with practice use characteristics and 85 completed surveys on microbiome beliefs and general farmer/farm characteristics from across NY state (Fig. 1a). Across the soil samples submitted, 9343 and 6740 unique organizational taxonomic units (OTUs) were identified for bacteria and fungi, respectively. Seven microbiome farmer belief clusters were identified using participants perceptions of 13 statements regarding the factors that mediating the microbiome on their farm in a previous study (Fig. 1b) (see Supplementary Table 1 for cluster details)⁹. Points oriented on the first and third rotated components (RC1, RC3) (Fig. 1b) indicate farmer perceptions of on- and off-farm factors for influencing their soil microbiome (Supplementary Table 1)⁹. Bacterial and fungal OTUs varied taxonomically and statistically across belief clusters, and were composed of approximately 260 and 507 unique families, respectively (Fig. 1c, d). Approximately half of bacteria were distributed across three phyla, with 24.79%, 12.85%, and 12.67% of OTUs belonging to Planctomycetota, Proteobacteria, and Chloroflexota, respectively. Whereas, fungi were dominated by two phyla, where 56.86% and 21.25% of OTUs belonged to Ascomycota and Basidiomycota. Next, we conducted Monte Carlo reference-based consensus clustering, principal coordinates analysis (PCoA), and permutational multivariate analysis of variance (PERMANOVA) to understand microbiome structure relationships to belief clusters and other variables (Supplementary Fig. 3). Both farmer belief clusters and Monte Carlos consensus clusters explained a modest amount of variation in the microbiome in the unconstrained ordinations (Supplementary Fig. 3). When fungal and bacterial groups identified through consensus clustering were visualized on a map, they revealed geographic structuring, indicating abiotic factors contributed to regional variation (Supplementary Fig. 3).

Microbiome-mediated plant defense induction varies across organic farms

To evaluate the impact of different soil microbiomes on plant defenses and pest suppression, we grew peas (Pisum sativum) in soil microbiomes extracted from the exemplar farmers’ soil samples for six of the seven microbiome belief clusters (cluster 6 was excluded due to low membership; magenta points; Fig. 1a, b) (Supplementary Table 1)⁹. Based on the unsupervised clustering approach, exemplar microbiomes used in the bioassays represented diverse fungal and bacteria consensus clusters (Supplementary Fig. 3). Five weeks after planting, pea aphid (Acyrthosiphon pisum) reproduction (progeny/adult) and systemic induction of the plant defense hormones JA and SA were quantified from leaf tissues (Fig. 1e-g). Statistics were calculated with log transformed phytohormone concentrations. Both phytohormone concentrations and reproduction were analyzed using generalized linear models (GLMs) (package = glmmTMB; family = gaussian)¹⁶. As expected, JA and SA concentrations in leaf tissues inversely correlated with aphid populations (GLMs JA: Estimate (Est) = −0.507, Standard Error (SE) = 0.236; SA: Est = −0.348; SE = 0.174). The highest JA and SA levels were observed in plants inoculated with the soil microbiome from cluster four, which is the exemplar for farmers who believed external factors (e.g., changes in weather patterns, increases in extreme weather, conventional pesticides applied in bordering lands, amount of natural area in bordering lands) impacted their microbiome (Fig. 1e, f; Supplementary Table 1; see blue points; log values for phytohormone data available in Supplementary Fig. 4). Soil from this farm also produced plants with a lower standard deviation for aphid populations compared to their peers (Fig. 1g; cluster 4).

When compared with cluster four, less variable SA levels were observed in plants inoculated with soil extracts from the farms in cluster three or five, which included farmers that believed farm characteristics and management (e.g., compost application, time in organic farming) were the most important factors for microbiome-mediated pest suppression (Fig. 1f; Supplementary Table 1; see green and teal points). Plants inoculated with the soil microbiome from cluster seven, where farmers believed farm characteristics and management were not important for their microbiome, had the lowest JA levels (Fig. 1e; Supplementary Table 1). Soils from these farmers also produced plants with aphid reproduction that was 50% more variable when compared to the farmer cluster with the highest JA levels (Fig. 1e, g; see blue points).

Three farming practices reduce microbiome alpha diversity across sites

Next we used machine learning to determine the most important bivariate relationships between nine different aspects of microbiome diversity for both bacterial and fungal sequencing and the 16 farming practices that were widely adopted but also varied considerably across the submitted soil samples (Supplementary Fig. 1; Supplementary Table 2)⁹. Machine learning suggested the adoption of three practices correlated with decreases in alpha diversity across sites (Fig. 2a). Specifically, preplanting practices (e.g., tarping and solarization), which are used to manage soil pathogens¹⁷, were associated with a ≈ 9% decrease in fungal richness (118.26 fewer OTUs) (Fig. 2a). The adoption of mineral fertilizers was associated with a ≈ 24% decrease in Shannon’s diversity for fungi (21.6 fewer OTUs), indicating reductions in common taxa (Fig. 2a). Finally, adoption of no tillage was associated with a ≈ 12% decrease in Simpson’s diversity for bacteria (22.72 fewer OTUs), suggesting fewer dominant taxa, a pattern shown previously by¹⁸ (Fig. 2a).

Irrigation was linked to bacterial turnover, while diverse practices were linked to fungal turnover

Our machine learning analysis indicated the gain or loss of five practices correlated with changes in different microbiome beta diversity measures across farms (Fig. 2a; diversified cropping, cover cropping with grasses, cover cropping with legumes, compost application, and targeted irrigation). Bacterial beta diversity was primarily regulated by adopting or losing practices related to water management (Fig. 2a). Targeted forms of irrigation (e.g., drip irrigation) were associated with increases in bacterial loss (OTU and phylogenetic), decreases in bacterial replacement (OTU and phylogenetic), and decreases in OTU bacterial balance (Fig. 2a). Loss of only one practice was associated with microbiome turnover measures (Fig. 2a). Here, the abandonment of drip irrigation negatively correlated with phylogenetic bacterial loss. Compost application was the only non-irrigation practice associated with bacterial beta diversity. Here, the adoption of compost resulted in less phylogenetic bacterial replacement (Fig. 2a). In contrast to bacteria, fungal beta diversity measures were associated with more diverse practices (diversified cropping, different cover crops, or compost application) (Fig. 2a). The adoption of legume or grass cover crops was associated with increases in fungal balance and fungal replacement (OTU and phylogenetic) suggesting these practices reassort the numerical distribution of fungal OTUs and shift taxa identity, respectively (Fig. 2a). Conversely, the adoption of compost or crop diversification was associated with less fungal replacement and less fungal loss (OTU and phylogenetic), respectively (Fig. 2a).

Organic practices regulated the assemblage of 109 microbial taxa across soil samples

Finally, to determine which taxa may be underlying changes in microbiome diversity, we evaluated bivariate links between farming practice adoption and the abundance of specific microbial OTUs. Overall, twelve practices were identified that correlated with the differential abundance of 109 specific taxa across sites (27 fungal and 82 bacterial OTUs; Fig. 2b; Supplementary Fig. 5; Supplementary Table 3). Adoption of no tillage was associated with 54% of all differentially abundant OTUs (Fig. 2b; Supplementary Table 3; 18 fungal + 41 bacterial = 59/109 total OTUs) and primarily had negative impacts on fungal OTUs (Fig. 2b; Supplementary Table 3; 17/18 negative), and positive impacts on bacterial OTUs (Fig. 2b; Supplementary Table 3). The adoption of plant-based fertilizers and biological mulch was associated with the greatest number of increases in bacterial taxa. In contrast, targeted irrigation and compost applications were associated with the greatest number of decreases in bacterial taxa (Fig. 2b, Supplementary Table 3). Half the number of farming practices were associated with fungal compared to bacterial OTU differential abundance (6 practices fungi; 12 practices bacteria).

Cover crops and irrigation were associated with increases in plant defenses

To determine potential functions, we next leveraged only the microbiome changes identified above (Fig. 2), and examined bivariate associations between these nine biodiversity measures and 109 taxa with changes in plant defense compounds (Fig. 1; JA and SA). We used model-based imputation to extend the defense and pest suppression capacity correlations observed for the exemplar farms (Fig. 1e–g) to all farms using cluster classifications and microbiome measures as predictor variables. Because beta diversity was examined across farm pairs, we chose two microbiome turnover measures associated with loss of defense compounds across pairs (Fig. 3a). None of the three alpha diversity measures were associated with concentrations of either defense-related phytohormone, however two beta diversity measures were associated with loss in phytohormones across farm pairs (Fig. 3a). Specifically, increased fungal replacement (OTU and phylogenetic) was associated with decreases in JA and SA loss (i.e., higher levels of JA and SA). Similarly, increased bacterial OTU loss correlated with decreases in JA loss across paired sites (i.e., higher levels of JA; Fig. 3a). Collectively, these results suggest compost applications may reduce plant defense compounds through decreases in fungal replacement (Figs. 2a, 3a), while the adoption of grass cover cropping and targeted irrigation could increase plant defense compounds, via fungal replacement (OTU and phylogenetic) and bacterial loss, respectively (Figs. 2a, 3a).

No tillage adoption and reduced pesticide use is associated with increases in plant defense

Among the 109 OTUs evaluated, bivariate correlations were established between three taxa and changes in total plant defense concentrations. Of these taxa, Fusarium chlamydosporum (OTU 5), a known fungal plant pathogen, had a negative association with JA levels, while a bacterial species, Microtrichales spp. (OTU 1139), was positively associated with JA levels (Fig. 3b). In our model no tillage was associated primarily with decreases in fungal taxa, and increases in bacterial taxa, including F. chlamydosporum and Microtrichales spp. (Fig. 2b, Supplementary Table 3). These results suggest that the adoption of no tillage could be used to suppress F. chlamydosporum and promote Microtrichales spp., which should increase plant defenses. The third taxa identified, Paenibacillus senegalensis (OTU 4736), had a negative association with both SA and JA (Fig. 3b). Combined with our model results for practices, this finding suggests decreasing P. senegalensis populations through reduced soil pesticide use should increase plant defense compounds (Fig. 2b; Supplementary Table 3; Supplementary Fig. 5). Overall, our results indicate that two organic farming practices (no tillage and pesticides) alter specific soil microbes that are associated with changes in plant defenses.

Farmer microbiome beliefs indirectly promote pest susceptibility and suppression within farms

Using only the statistically important bivariate correlations leading to changes in plant defenses, we finally constructed SEMs to test the indirect impacts of farmer microbiome beliefs on microbiome-mediated pest suppression, and conditioned these models based on economic (farming as the main income source) and abiotic (time in organic and soil properties) characteristics^9,10. Two separate SEMs were constructed for practices associated with beta diversity (targeted irrigation, grass cover crops, and compost) and specific taxa (no tillage and pesticides). For the beta diversity SEM, we examined loss of microbiome beliefs, time in organic, JA, SA, and progeny across farm pairs, while totals within farms were used for the specific taxa SEM. Both SEMs suggested microbiome-mediated pest suppression is indirectly correlated with farmer beliefs in the microbiome (Fig. 4a, b; Supplementary Tables 4, 5).

In the beta diversity SEM, loss of beliefs in the microbiome mediated loss of pest progeny indirectly, and this was primarily mediated through decreased adoption of compost by farmers in clusters one, two, and seven compared to their counterparts, which increased fungal replacement, increased JA, and decreased pest population (Figs. 1b, 4a; Supplementary Table 4). However, the adoption of compost applications was increased on farms serving as the main source of income, which may decrease JA and increase pest populations (Fig. 4a, Supplementary Table 4). To a lesser extent, loss of beliefs in the microbiome drove increased progeny loss indirectly through increased adoption of targeted irrigation by farmers in clusters one, two, and seven (Fig. 4a; Supplementary Table 4). The primary direct driver of the adoption of targeted irrigation in the SEM was sandiness of the soil, however, and not beliefs in the microbiome (Fig. 4a). Similarly, the adoption of grass cover crops was primarily mediated through negative associations with farming as the main source of household income and decreases in the amount of time in organic management across farm pairs, and not farmer microbiome beliefs (Fig. 4a; Supplementary Table 4). Therefore, the less time farms were in organic production or the greater dependence on farming for income, the less likely farmers were to adopt grass cover crops, which increased fungal replacement (OTU and phylogenetic), promoted SA and JA, and reduced pests (Fig. 4a; Supplementary Table 4). Taken together, this suggests adoption of some microbiome-friendly practices can promote pests and are influenced more by farmer beliefs (composting), while others are influenced more by economic and abiotic factors (e.g., targeted irrigation).

In the specific taxa SEM, indirect impacts of microbiome beliefs were mediated through increased adoption of no tillage by farmers in clusters three, four and five, which promoted Microtrichales spp. and suppressed F. chlamydosporum abundances, increasing JA, and decreasing aphid populations (Fig. 4b; Supplementary Table 5). Overall, beliefs in the microbiome had the strongest direct influence on adoption of no tillage, however the longer time the farm was in organic production and the greater the amount of primary income arising from farming, the less likely farmers were to adopt no tillage, despite indirect benefits for the soil microbiome (Fig. 4b; Supplementary Table 5). Farmers who believed more factors impacted their microbiome were also less likely to use organic pesticides, which should decrease Paenibacillus senegalensis abundance (Fig. 4b; Supplementary Table 5). As Paenibacillus senegalensis had a negative association with JA concentrations, reductions in the bacteria should increase JA, and decrease pest populations (Fig. 4b; Supplementary Table 5). However, this indirect relationship was not a major driver of pest populations. In summary, the adoption of microbiome-supportive practices (no tillage) could be limited by farm characteristics and economics, while beliefs may partially counteract these patterns (Fig. 4a, b; Supplementary Tables 4, 5).

Viewing farms as complex socio-ecological systems presents an underexplored opportunity to promote the adoption of practices that enhance sustainable pest management. As the system of relationships between farming practice, soil microbiomes, and pest management becomes increasingly well understood¹⁵, farmer beliefs provide a mechanism to shift adoption patterns and enhance agroecosystem sustainability^9,10. Our research identified organic farming practices that shift soil microbiomes across farming systems and linked these microbiome changes with plant defenses and pest suppression. While previous research identified the benefit of organic agriculture for microbiome conservation and the regulation of pests (See refs. ^5,14 among others), our study is the first linking variation in practices across organic farms to functional soil microbiome shifts. Taken more broadly, we show that agroecosystem sustainability hinges not only on production practices per se, but on farmer beliefs and the contextualizing abiotic and economic characteristics which drive farmer decision making.

In our study, > 90% of the practices we evaluated mediated changes in the microbiome (alpha diversity, beta diversity, and species identity). Nevertheless, only the adoption of no tillage, targeted irrigation, and grass cover crops was associated with changes in plant defenses and pest populations through changes in the microbiome. Our results add enhanced plant defense and reduced pest populations to the known benefits of no tillage¹⁹, however, farmers were less likely to use no tillage the longer the farm was in organic management or if the farm was the main source of income (Fig. 4b; Supplementary Table 5). Bloom et al.⁹ also found that adoption of no-tillage decreased with farm size, which may indicate persistent barriers for scaling up no-tillage in organic systems, including difficulties with weed suppression and the economics of new equipment (e.g., interrow mowers) needed by farmers at larger scales²⁰. Simultaneously, our findings indicated that no tillage reduced the dominance of bacterial alpha diversity and the abundance of a plant pathogen, F. chlamydosporum²¹. Previous evidence suggests no tillage reduces soil pathogens when combined with crop rotation²² and reduces bacterial diversity ostensibly via a lack of soil disturbance and breakdown of crop residues on soil surfaces²³. Outreach efforts to better inform farmers of the full suite of biological no tillage benefits, such as microbiome-mediated pest suppression, could be an effective strategy to increase adoption, as beliefs in the microbiome were a primary driver of the adoption of no tillage in our SEM (Fig. 4b, Supplementary Table 5).

Less documented are the benefits of targeted forms of irrigation for beneficial soil microbiome management. Overall, our findings suggest that the adoption of targeted forms of irrigation (e.g., drip systems) are key regulators of bacterial beta diversity. Evidence suggests drip irrigation can promote plant growth^24,25 and reduce the spread of plant pathogens through the regulation of soil moisture and temperature²⁶. Taken with our results, targeted irrigation may have other benefits for crops, such as enhanced plant defenses that are mediated through changes in bacterial populations (Figs. 2a, 3a). We suggest the adoption of targeted irrigation benefits plant defense through the loss of antagonistic and plant pathogenic bacteria from the rhizosphere, though these mechanisms need further validation. While the literature regarding the benefits of compost applications is extensive (See ref. ²⁷ among others), the adoption of compost in our study was linked to decreases in plant defenses through the stabilization of fungal microbiomes (Figs. 2a, 3a), suggesting relationships between compost applications, plant health, and ecosystem services are nuanced. Indeed, attention to source inputs, temperature, and water are critical to destroy pathogens in compost during production²⁷. Working with farmers to identify approaches that minimize fungal replacement after compost use will be a critical next steps for optimizing microbiome-mediated ecosystem services across farms.

Our differential abundance analysis suggests some practices have contrasting impacts on bacterial and fungal OTUs (e.g., no tillage), while other practices primarily impact bacterial OTUs in both positive (vegetable fertilizer, mineral fertilizer, and biological mulch) and negative (compost and animal manure) ways. Several of these microbiome taxa were also identified as potential plant defense modulators. Specifically, our analysis suggests an abundance of F. chlamydosporum, a wilt-causing root pathogen²¹, correlates with reduced JA levels. Although additional studies are needed, Fusarium sp. may be preventing beneficial defense-promoting root-microbe interactions²⁸ or this may represent a plant trade-off, where JA is being reallocated to belowground defenses during pathogen infection²⁹, leaving above-ground structures vulnerable to pests. While F. chlamydosporum was associated with decreased plant defenses, Microtrichales spp. correlated with the upregulation of JA. Members of the Microtrichales are linked to phosphate solubilization^30,31, and phosphate is key for JA regulation³². Microtrichales may indirectly regulate JA by promoting changes in soil nutrient cycling and phosphate availability for JA biosynthesis; however, additional experiments are required to dissect this. More broadly, only ≈ 0.4% and ≈ 0.9% of OTUs were associated with practice adoption for fungi and bacteria, respectively, suggesting processes mediating ≈ 99% of the OTUs in our study remain unexplained.

Our findings indicate that not all measures of microbiome diversity are indicators of ecosystem services. Here, we show alpha diversity was not robustly correlated with farming practices or a primary indicator for microbe-mediated pest suppression, but instead, beta diversity offered better insights into microbiome-mediated ecosystem services in our system. While evidence suggests microbiome beta diversity is common, associated changes in function are rarer³³, potentially due to functional redundancy in microbe turnover⁶. Indeed, our results indicate that fungal taxa may be replaced with functionally redundant or phylogenetically related taxa, because less fungal replacement were related to decreases in JA concentrations and increased pest populations (Figs. 3a, 4a, Supplementary Fig. 4). Future research and development may focus on taxa identity, cultivating beneficial microbes, and avoiding pathogen accumulation, over general increases in microbiome biodiversity which is typically a weak predictor of ecosystem function³⁴. In more practical terms, our findings suggest key taxa could serve as indicators of plant defenses and pest suppression for rapid microbiome diagnostics within farms, and the practices we identified could be used in decision support tools paired with sequencing, allowing farmers to rapidly modify their microbiome.

Additional studies are still needed to decrypt the role of microbiomes in pest suppression and ecosystem services. Namely, higher throughput approaches are needed to phenotype the defense-including capacities of diverse agricultural soils while reducing methodological bias. Our study, among others^5,35,36 leverages inoculations of sterilized potting soil as the plant growth media. While this approach is standard for predicting microbiome function under field conditions³⁵, the established microbiomes in potting soil are not completely equal to the field microbiomes they originate from³⁶. Further, these methods take a large amount of work, limiting the number of soil microbiomes that could be evaluated. In the current study, we addressed throughput limitations by selecting one exemplar soil per belief cluster for microbiome functional analysis in the lab, and then used this data to predict the plant defense-inducing capacity of other soil microbiomes using multivariate imputation by chained equations³⁷. Therefore, more robust validation of the links between belief clusters, microbiome diversity and functions in plant defense is still needed using additional clusters or field trials³⁸. While advances in sequencing have made microbiome characterization rapidly accessible³⁹, linking these microbiomes to plant function across farming systems without introducing bias presents an ongoing challenge for researchers and decision makers.

To our knowledge, our research is the first to view microbiome-mediated insect pest suppression through a socio-ecological lens by linking it with farmer beliefs and adoption. We show that farmer microbiome beliefs have cascading indirect ecological consequences for pest suppression (Fig. 4a, b; Supplementary Tables 4, 5). Our model suggests that customized messaging to farmers with different characteristics and beliefs will be useful in promoting the adoption of practices that support the microbiome⁴⁰. For example, recent work by Bloom et al.⁹ indicated only 42% of farmers in the present study have beliefs consistent with the microbiome literature, suggesting most organic farmers would benefit from extension efforts, on-farm experimentation, and farmer-focused science that reconciles discrepancies between beliefs and ecosystem services. Our findings also indicate that time in organic management and farmer income mediate practice adoption, presenting targets for farm policy instruments (e.g., the Environmental Quality Incentives Program). Policymakers can use this knowledge to incentivize the adoption of microbiome-friendly practices through subsidies that support initial investments associated with new practices, when adoption is limited by income. More broadly, our results suggest there are misalignments between farmer beliefs and practices promoting microbiome function. Therefore, enhancing farmer knowledge via microbiome extension activities may improve pest suppression through the adoption of practices that are beneficial and the optimization of the practices that are detrimental to the microbiome but are considered desirable for other properties.

Study system, soil samples, and questionnaire

To generate soil samples for metabarcoding and pest suppression assays, instructions for soil sampling were sent to 279 organic farmers in New York, USA, along a with paper survey on microbiome beliefs, practices associated with the soil sample, farm characteristics, and farmer demographics as explained in detail by Bloom et al.⁹. Participants were able to submit up to two soil samples with distinct management practices from their farm (n = 85 farmers). The soil microbiome and associated practices for each sample were treated as separate observations. Because not all farms contributed two samples, farmer and general farm characteristics were sometimes linked to multiple samples with different practices and associated microbiomes, while others were not. This approach generated a richer data set and increased representation of practices; however, we acknowledge that samples coming from the same farm are not fully independent. Farmers were instructed to collect 10 soil subsamples consisting of a 6” deep × 2” thick core using a spade or shovel in a transect across the field as previously described^41,42. We recommended avoiding points that fell in unusual areas and spanned different soil types. Following sample collection, participants were instructed to thoroughly mix all sub-samples and transfer the homogenized mixture to a 1-quart sample bag (Ziploc; Part No. 682256). For shipping, participants were given a US Postal Service prepaid polyethylene expansion mailer (Quality Park Products; Part No. QUA46390). Immediately upon receiving the soil samples, a V-shaped sterile spatula was used to sample and store ≈ 50 ml of soil per sample at -20 °C for metabarcoding. The remainder of each sample was refrigerated at 1.6–3.3 °C until use in laboratory assays.

One hundred and thirty-six samples were received over a 3-month period (Fig. 1a). Sample, farmer, and farming system characteristics were quantified by the survey instrument previously described in ref. ⁹. Characteristics used in this study include: (1) farming practices used in the field where soil sample(s) were collected (Supplementary Table 2); (2) time in organic production, (3) percent of income coming from farming; and (4) beliefs clusters found by Bloom et al.⁹. We focus on time in organic production and percent income that comes from farming because they consistently mediate the adoption of farming practices known to influence the microbiome⁹. Moreover, time in organic management is known to promote farming system biodiversity⁴³. As described in ref. ⁹ the 85 participants who completed the survey were clustered by their beliefs using affinity propagation, and exemplar farms identified for each cluster. The soil samples from exemplars were used in bioassays (see Herbivore and plant defense assays), and cluster classifications served as predictors for farming practice adoption in our socio-ecological models. For these models, farmer beliefs were modified to a continuous variable by summing the values for each farm within the rotated component coordinate plane (Fig. 1; RC1 + RC3) (see Structural equation modeling). Due to varimax rotation, successive components no longer capture as much variance as possible; therefore (RC2) was found to explain the least variation after rotation and was excluded from our analysis⁹.

Soil DNA extractions

Prior to DNA extractions, soil samples were homogenized using the quartering method⁴⁴. In brief, approximately 25 ml of bulk soil was placed in a sterile autoclaved glass petri dish and divided into quarters. Each quarter was mixed individually with a sterilized spatula, the two quarters from each half were mixed, and the two halves were mixed to form a homogenous matrix⁴⁴. This procedure was repeated several times. After homogenization, soil samples were dried for 24 h in a biosafety cabinet (Labconco; Delta Series; Purifier Class II). Dried samples were further homogenized with a sterile autoclaved mortar and pestle until all soil aggregates were equal in size, then stored at -20 °C.

To extract soil sample DNA, we used the DNeasy PowerSoil Pro Kit (Qiagen; Cat. No. 47016) followed by ethanol precipitation⁴⁵ and DNA cleanup. We conducted DNA extractions using manufacturer protocols, with the following exceptions: cell lysis was conducted with a modified high speed paint shaker (Harbil; Part No. 24018) and a reduced homogenization time of 1 min. This approach promoted DNA yields and reduced shearing compared to other cell lysis methods (e.g., vortex), which we confirmed using DNA gel electrophoresis (data not shown). Further optimization was conducted during inhibitor removal and prior to DNA column binding, where samples were placed on ice for 5 min after briefly vortexing with aluminum chloride hexahydrate (Qiagen; Mat. no. 1108824) and guanidinium thiocyanate (Qiagen; Mat. No.1108825). DNA yield and purity were assessed using spectrophotometry (Thermo Scientific; NanoDrop^TM One^C; Cat. No. ND-ONE-W). All samples not meeting sequencing standards of the Dalhousie University Integrated Microbiome Resource (IMR) (Halifax, Nova Scotia, CA) underwent ethanol precipitation as in ref. ⁴⁶. For samples not reaching purity standards after ethanol precipitation, we used a DNA cleanup kit (New England Biolabs; Cat. no. T1030S) and followed the manufacturer protocol. Prior to sequencing, DNA underwent PCR to confirm purity (e.g., absence of inhibitors), and was standardized to ≈ 10 ng/µl. At least 100 ng of soil DNA per sample meeting purity standards (260/280 > 1.80; 260/230 > 2.0) was sent for library preparation and sequencing using MiSeq at the IMR facility. The V4–V5 region of the 16S ribosomal RNA region was sequenced to characterize bacterial communities (Forward primer: 515FB = GTGYCAGCMGCCGCGGTAA; Reverse primer: 926R = CCGYCAATTYMTTTRAGTT), and the internal transcribed spacer (ITS2) region of the rRNA gene was sequenced to characterize fungal communities (Forward primer: ITS86(F) = GTGAATCATCGAATCTTTGAA; Reverse primer: ITS4(R) = TCCTCCGCTTATTGATATGC).

Bioinformatics pipeline

Read preprocessing, data clustering, and post-clustering were conducted in AMPtk⁴⁷. Read preprocessing merged paired-end reads using usearch, removed forward and reverse primers, and concatenated samples, yielding 12,556,488 and 16,541,471 valid output reads for bacterial and fungal communities, respectively. Clustering of reads into OTUs was conducted using the unoise3 pipeline. In brief, the pipeline included filtering (maximum expected error < 1.0), dereplication, denoising (minimum size = 8), de novo chimera removal, and validation of ASV orientation. Reads were mapped to denoised ASVs (identity = 97%) and denoised sequences were clustered into biological OTUs (global identity threshold = 97%), yielding 11,669 and 7232 OTUs for bacteria and fungi, respectively. Post clustering was performed with the LULU algorithm (version = 0.1.0). To begin clustering, pairwise sequence similarity for match detection between OTUs was calculated using VSEARCH (version = 2.15.1; minimum identity threshold = 0.84; minimum query coverage = 0.90). Then, LULU merging was applied using a co-occurrence minimum ratio of 95% and minimum relative abundance of 1, yielding the final set of 9343 (2326 merged) and 6740 (492 merged) OTUs for bacteria and fungi, respectively.

Taxonomy was assigned in R (function: assignTaxonomy; package: dada2) using SILVA (version: 138.1) for bacteria⁴⁸ and UNITE (general dynamic release: 29.11.2022) for fungi⁴⁹, with species-level assignments made for bacteria using the “addSpecies” function⁵⁰. This approach implements the RDP classifier algorithm from ref. ⁵¹ with kmer size 8 and 100 bootstrap replicates. Phylogenetic tree construction was conducted starting with “muscle” for mass sequence alignment, with the “diags” argument used to enhance algorithm speed, and the current alignment returned after 24 hours⁵². Mass sequence alignments were trimmed using “trimal”, removing columns with gaps in more than 20% of sequences or a similarity score lower than 0.001, unless this removed more than 40% of the columns in the alignment, thus the minimum coverage was set to 60%⁵³. Trimmed mass sequence alignments were passed to “RAxML” for phylogenetic tree construction using the “GTRGAMMA” substitution model with 100 rapid bootstrap inferences and thereafter a thorough ML search⁵⁴. Importantly, due to limitations with the ITS region (e.g., sequence length variation; alignment ambiguity), we caution evolutionary conclusions from our analysis. Rather, phylogenetic tree construction was used to approximate relatedness among OTUs for our beta diversity measurements. The OTU and taxonomy tables, phylogenetic tree, and sample data from these analyses were then used for downstream analyses.

Exploratory microbiome ordinations

Variance-stabilized microbiomes were analyzed using Monte Carlo reference-based consensus clustering (package = M3C) to visualize unconstrained patterns of composition⁵⁵. Principal coordinates analysis (PCoA) with Bray-Curtis dissimilarities was also conducted to visualize unconstrained variation in the microbiome with farmer belief and M3C consensus clusters⁵⁶. Permutational multivariate analysis of variance (PERMANOVA) was then used to test whether farmer belief clusters and M3C-derived clustering explained a significant portion of the observed variation (function = adonis2)⁹ (Supplementary Fig. 3).

Microbiome biodiversity measures

Alpha and beta diversity for fungi and bacteria were measured using the iNext and Betapart packages, respectively^57,58. For alpha diversity, we used Hill numbers parameterized by diversity orders, including species richness, Shannon diversity, and Simpson’s diversity. Post clustering OTU tables for fungi and bacteria were used to calculate the asymptotic alpha diversity metrics using the iNext function for (read) abundance data with a 95% confidence interval⁵⁸ (Supplementary Fig. 6). Unlike interpolated and extrapolated values, asymptotic metrics estimate the true diversity expected under infinite and standardized sampling effort for each soil sample. Beta diversity measures for OTU tables were collected pairwise using the beta.pair, beta.pair.abund, and phylo.beta.pair functions⁵⁷. For incidence-based pair-wise dissimilarities, we used the Sørensen indices that accounted for spatial replacement and nestedness of OTUs across sites. OTU replacement accounts for substitution, whereas nestedness measures OTU loss across sites. Measures of nestedness were used to evaluate microbiome patterns across and within farms, along with Wilcoxon tests (function = wilcox.test) for fungi and bacteria. Thereafter, to promote independence, our analysis included only pairwise measures across rather than within farms. For clarity, we also refer to nestedness as loss above. Incidence-based measures were complemented by their phylogenetic equivalents, which account for relatedness across samples^59,60. For the loss and replacement terms, we found that microbiome beta diversity was phylogenetically structured, indicating assembly was non-neutral (Supplementary Fig. 7). In other terms, microbial beta diversity was likely driven by environmental gradients⁶⁰. Abundance-based beta diversity measures were found using the Bray indices, accounting for OTU balance and gradients⁶¹. The balance and gradient terms evaluate the numerical substitution of reads and the loss of reads from the OTU tables across sites, respectively.

Differential abundance analysis

To address the role of identity effects in mediating plant defenses and pest suppression, we used the practice adoption predictor variables (Supplementary Table 2) to parameterize a differential abundance analysis (package = ANCOMBC; function = ancombc2), where taxa were considered differentially abundant based on a false discovery rate-adjusted p-value < 0.05, using the default log fold change threshold of zero⁶² (See results in Fig. 2b). Differential abundance analysis began by merging our OTU table, farming practices (predictor variables), and taxonomy table into a SummarizedExperiment object⁶³. We then subset our data using prevalence filtering (prevalence = 0.1; function = subsetbyprevalenttaxa; package = mia)⁶⁴, and used the prevalence filtered data in our differential abundance analysis (function = ancombc2)⁶². We conducted the differential abundance analysis at the OTU level, with 100 bootstrap iterations⁶². Because ANCOMBC emphasizes statistical significance after multiple-test correction, all OTUs with significant adjusted p-values were retained for further analysis. The log corrected abundances for these taxa identified via differential abundance analysis then underwent selection as predictive variables for plant defenses (see Machine learning) and inclusion in SEMs. Log corrected abundance values are bias-adjusted log-transformed abundances, which are calculated for each OTU per sample after adjusting for sample-specific bias factors (e.g., library size, compositional biases)⁶². Because bias-corrected log-abundance values account for these biases, they no longer reflect absolute abundance (i.e., OTU counts), but instead offer an unbiased measure for our correlational analyses⁶².

Herbivore and plant defense assays

We replicated our herbivore and plant defense bioassay three times, with each assay conducted over a five-week period. Bioassays began by extracting the soil microbiome of the six exemplar farms for each farmer cluster (Fig. 1b) in a ¼ strength Hoagland’s solution as done previously by ref. ⁵. For microbiome extractions, 16 g of soil and 240 mL of ¼ strength Hoagland’s solution were shaken in duran flasks at 275 rpm for 1 hour. After shaking, flasks were allowed to rest at room temperature for 1 hour and then centrifuged for five minutes at 500 rpm and 4 °C (Sorvall RC 5 C Plus). Supernatants for each sample were returned to a Duran flask and kept at 4 °C until use. Peas (Pisum sativum L., variety ‘Perfection Dark Seeded’) were grown in sterilized 9 cm square plastic pots containing triple autoclaved potting soil. Exemplar farm microbiome extracts were applied at a rate of 15 ml twice per week for three weeks. At four weeks post seedling emergence, five adult pea aphids (Acyrthosiphon pisum) were caged on a single leaf, with one cage per plant (6 replications per exemplar farm per assay). After 24 hours, adult aphids were removed and F1 progeny were culled to five nymphs per cage. After 9 days, the number of F2 progeny was counted for each exemplar farm. After counting, the next undamaged developmentally matched leaf directly above aphid cages was harvested into LN₂ from all plants and stored at −80 °C until systemic phytohormone extraction and quantification following⁵. Correlations between exemplar microbiomes, plant defenses, and progeny were assessed using GLMs in R (package = glmmTMB) (Fig. 1e-g)¹⁶.

Phytohormone extraction and quantification

Prior to phytohormone extraction, leaf tissue was lyophilized (Labconco Freeze Dry System; Catalog no. 77520-00 L), weighed, and ≈ 25 mg of dried leaf tissue was homogenized with a modified paint shaker (Harbil; Part No. 24018). Phytohormone extractions were performed as in Blundel et al. 2020 using D4-SA (salicylic acid) and D5-JA (jasmonic acid) as internal standards. Dried sample extracts were resuspended in 200 μL of HPLC-grade methanol and 10 μL was injected onto a Dionex UHPLC (Thermo Fisher Scientific, Waltham, MA, USA) through a C18 reversed-phase HPLC column (Phenomenex Gemini) and an Orbitrap-Q Exactive mass spectrometer (Thermo Scientific) run on negative polarity. A gradient of 0.1% (v/v) formic acid in water (Solution A) and 0.1% (v/v) formic acid in acetonitrile (Solution B) was established at a flow rate of 600 uL per minute. A 10.5-minute gradient was established as follows: for the first minute, the composition of the liquid phase was 99% Solution A and 1% Solution B. From minute 1 to minute 8, the composition of the liquid phase started at 80% Solution A and 20% Solution B, and gradually shifted to 25% Solution A and 75% Solution B. From minute 8 to minute 9.5, the composition of the liquid phase was 0% Solution A and 100% Solution B. From minute 9.5 to minute 10.5, the composition of the liquid phase was 99% Solution A and 1% Solution B. Data acquisition and interpretation was conducted in Xcalibur (Thermo Fisher Scientific). Peak areas were recorded for internal standards and endogenous phytohormones. The endogenous peak areas were divided by the internal standard peak area and reported relative to the sample dry weights. No contamination was detected in methanol blanks with internal standards.

Covariate preparation and machine learning

Covariates used in the assessment of alpha diversity, beta diversity, and differential abundance included the farming practices used in the field where soil sample(s) were collected (Supplementary Table 2). For alpha diversity and OTU differential abundance practice presence or absence was recorded. For matching with fungi and bacteria beta diversity, farming practices were transformed into beta diversity terms, practice nestedness, and replacement⁵⁷. Here, we interpret replacement and nestedness as the gain (adoption) or loss (abandonment) of the practice across sites in the comparison. To find these terms, we evaluated the partial contribution of each farming practice to the overall turnover terms calculated pairwise across sites^65,66. This approach consists of removing one farming practice at a time, recomputing the partial turnover value for each term, and finding the percent contribution of the partial to the overall term⁵⁷. Here, negative and positive values for each term indicate if the practice is contributing to similarity or dissimilarity (turnover) in the practice across sites, respectively. For example, a value of 100% indicates the practice is either being lost or gained depending on the beta diversity term, while all other practices remained static.

Because using all sites for assays was untenable given space limitations, plant defense and pest population values were found for all sites using model-based estimation with the mice package in R³⁷. Here, the fungal and bacterial asymptotic alpha diversity values at each q value (range 0 – 2), log corrected abundance of differentially abundant taxa, microbiome beta diversity variables, and summed exemplar location within the rotated component coordinate plane (RC1 + RC3) were used to estimate the plant defense and pest population values for all samples in the study using multivariate imputation by chained equations³⁷. We then used machine learning (ML), stability selection, and stepwise AIC for model selection⁶⁷, allowing us to identify important bivariate relationships between farming practices, microbiome measures, plant defenses, and pest suppression (Supplementary Fig. 1). Our ML approach first used the cv.glmnet function (nfolds = 10) from the glmnet package⁶⁸ to find suitable lambda values (minimum and 1 SE), which were passed to the model fitting glmnet function. Results were visually inspected to assess top models, which were then verified using stability selection. Here, we performed a resampling procedure using the stabsel function (fit function = glmnet lasso; cutoff = 0.6; PFER = 1; sampling type = MB) from the stabs package⁶⁹ to identify the most influential variables. We then used stepwise AIC (package = MASS) on the stable variables^67,70 yielding the most influential predictor and response variables for use in SEMs (Supplementary Fig. 1). This process was repeated, working from farming practices to the microbiome, plant defenses, and pest suppression. Only the most influential response variables were included as predictor variables in the following model.

Structural equation modeling

We hypothesize that farmer beliefs would have cascading indirect effects on herbivore populations via farming practices, the soil microbiome, and plant defenses (Supplementary Fig. 1). To test this prediction, structural equation models were constructed using the SEM function in the Lavaan package⁷¹ following⁷² using a single-indicator latent variable approach to represent our response and predictor variables. This approach allows for the inclusion of latent constructs in the path diagram while avoiding overparameterization by using single observed variables to represent each latent factor. Each latent variable (e.g., farmer microbiome beliefs) was measured using an empirically selected highly indicative indicator variable (e.g., the summed RCs for beliefs). Relationships between model parameters for practices, microbiome measures, plant hormones, and pest suppression were informed by prior machine learning and stability selection in a stepwise manner (Supplementary Fig. 1). To avoid fitting problems, variables were scaled ad libitum using a generic scale function⁷³. Residual correlations between latent variables were included to account for unexplained covariance using modification indices (function = modificationIndices) until models reached acceptable fits (RMSEA > 0.05 and CFI ≈ 1). To quantify indirect effects, mediation pathways were specified algebraically in the model syntax. This approach estimates compound path coefficients for each mediation sequence. Model fits (chi-square) and parameters were estimated with maximum likelihood (estimator = MLM) and significance was accepted at p ≥ 0.05. Soil properties characterizing abiotic conditions were found using the gridded soil survey⁷⁴. To match with microbiome beta diversity, continuous predictors (e.g., belief cluster) and response variables (e.g., plant defenses) were differenced and binomial predictors (e.g., farming as the main income source [no or yes coded as 0 or 1]) were transformed into categorical predictors (1 = 0 to 0; 2 = 1 to 0; 3 = 0 to 1; 4 = 1 to 1) for comparisons across sites.

Soil properties characterizations

The GPS locations of sites were intersected with the gridded soil survey geographic (gSSURGO) database rasters deployed at the county-level for NYS from the National Cooperative Soil Survey⁷⁴. Via this intersection, we found the map unit key for each site, which we then related to the gSSURGO component table, which gives the soil properties by site per map unit. We then related the component table to the horizon table using the component key to derive data by horizon for each site. We then retrieved representative values for the following soil properties: pH; available water content; organic matter; percent sand, silt, and clay; and erodibility. Because there are several records for each map unit key, we then averaged the representative values for each site per soil property. Soil properties were highly correlated. Here, we collected the variance inflation factor (VIF) for our pool of seven soil properties and selected those with values below two. This VIF approach indicated that only the percent sand in the soil sample should be retained. Therefore, for our SEMs, we selected the percentage of sand estimated per farm as a proxy for soil properties in general.