Reliability of environmental DNA surveys to detect pond occupancy by newts at a national scale
Various estimates for great crested newt pond occupancy rates have been published with most relating to site or regional scale assessments. A naïve occupancy rate of 0.13 has been identified for a data set from the northwest of England29, while estimates based on conventional occupancy16 modelling of 0.31 for southeast England and between 0.32 and 0.33 for mid Wales were presented by Sewell et al.13. The only other national data which the authors are aware of are within the Freshwater Habitat National PondNet Study, which estimates a naïve pond occupancy of between 13 and 18%30, and the Amphibian and Reptile Conservation Trust National Amphibian and Reptile Recording Scheme, which suggests a 12% occupancy rate for the UK31. Using data from nearly 5000 ponds sampled across England, here we provide a more extensive national-level analysis while accounting for imperfect detection in the eDNA sampling protocol. Assuming a threshold of just one positive qPCR replicate in a sample, the naïve occupancy estimate of 0.30 is similar to the localised regional estimates made by Sewell et al.13 using direct observation methods. The posterior mean estimates of 0.198 for occupancy are comparable to most other estimates for great crested newt pond occupancy in the UK, but lower than the naïve estimate. The lower modelled estimates of occupancy than the naïve estimate suggest that false positives should not be ignored and need to be accounted for statistically using methodologies such as the eDNAShinyApp package used here23,24,27.The goodness of fit analysis was based on the MCMC output for each site and observed covariate levels in the data set. Some lack of fit was observed, with a predicted peak in amplification at 10 qPCR replicates but an observed peak at 12 qPCR replicates. There are several potential causes for this. For example, variation between laboratories could not be accounted for as these metadata were not made available. The assumption that error rates are the same across all laboratories may therefore not apply and could contribute to poorer model fit. Secondly, we did not consider error rates as functions of covariates, and this may also have contributed to a poorer fit.Stage 1 error was found to be smaller than Stage 2 error for both false positive and false negative error. However, Stage 2 error operates on individual qPCR replicates and not at the site level. If there was no error at Stage 2, we would observe either zero qPCR replicates amplifying or all qPCR replicates amplifying (i.e. 12 in the case of the data presented here). The majority of samples showed zero qPCR amplification (3429 samples), and this was strongly linked to absence of newts. For sites with amplification, we observed a greater number of samples amplifying between 1 and 11 qPCR replicates (1074 samples) than we did amplifying with all 12 qPCR replicates (422 samples). The qPCR replicates that do not amplify in samples containing target DNA are erroneous, even if other replicates within that sample do amplify and contribute to this high Stage 2 false negative error in the model output. Data simulated from the fitted model show that the frequency of samples that contain DNA at Stage 2 amplifying in less than five of the 12 qPCR replicates is very low (Fig. 2b). Given that all replicates need to be erroneous to alter the naïve assignment of a sample containing DNA to negative, Stage 2 false negatives at this sampling level are unlikely. However, this does not rule out Stage 1 false negative error which we estimate to be 5.2% (with wide credible intervals between 0.1% and 25.1%).Higher levels of Stage 2 replication remove lab-based false negative error. If eDNA is present within a sample and a high number of replicates are used, it is highly unlikely that all qPCR replicates will be erroneously negative, even when the false negative rate at the replicate level is high. Conversely, high levels of Stage 2 replication increase the likelihood of false positive error occurring32. Stage 2 false positive results are of greater consequence than the 2% the model output would suggest. Unlike false negative error where all Stage 2 replicates need to be erroneous to change the naïve assignment of occupancy of a sample, when a threshold of one amplifying replicate is applied, only a single replicate needs to be an erroneous to generate a false positive. With 12 qPCR replicates at Stage 2 and a 2% false positive error per replicate, a sample with no DNA present has a 24% chance of producing at least one amplification. Assuming this error is randomly distributed through samples with no DNA present and qPCR replicates, it is more likely that samples with small numbers of replicates amplifying would be erroneous than where large numbers of replicates amplify. This was confirmed in the goodness of fit analysis with the distribution of Stage 2 false positive replicates making up all samples amplifying with one or two positive qPCR replicates, while negligible false positive amplification was seen with four amplifying replicates or above (Fig. 2a). With only a single sample at Stage 1, false positive error is limited to the 1.5% per sample, as per the ({theta }_{10}) value in the occupancy model output.We would recommend that, where possible, results from individual sites are interpreted as a probability of site occupancy, based on modelled outputs such as those produced by the eDNAShinyApp R package23,27. The precision of these models is dependent on sample size. Where sample size is large, a reduced bias and narrower credible interval range is observed24. However, using occupancy modelling, Buxton et al.24 demonstrated that studies that contain only a small number of sites are unlikely to produce accurate and precise estimates. As a result, such assessments will need to continue to rely on a threshold value of amplifying qPCR replicates to define site occupancy. A naïve amplification threshold for assigning occupancy of one positive qPCR replicate is unwise and should be increased to reduce Stage 2 false positive error. Indeed, a threshold of three positive qPCR replicates would reduce false positive error, without increasing false negative error. Alternatively, redistributing the replication between Stage 1 and Stage 224, would also reduce the credible interval width and generate a more precise posterior mean estimate at Stage 1, in turn reducing the uncertainty around the occupancy estimate. A redistribution of replication leading to two samples collected from each site, both analysed using up to six qPCR replicates, as opposed to one sample analysed using twelve qPCR replicates, has been suggested24.Equal weighting of the ten covariates used in the traditional great crested newt HSI assessment25 may be ecologically unrealistic29. This is supported by the observations here, with only some of the HSI covariates identified as important for occupancy. The model applied by the eDNAShinyApp package23,27 successfully identified several covariates known to influence great crested newt occupancy, that are included within the HSI assessment25. These included occurrence of fish, water quality, shade, pond density, macrophyte cover, frequency of drying and geographic area; although our analysis was based on Easting and Northing, rather than the broad-scale suitability map used in deriving the original HSI25. However, several traditionally used HSI variables emerged as unimportant, i.e., waterfowl, terrestrial habitat quality, and area of pond; while ground frost, rainfall, surface wind and land cover type, are not included within the HSI assessment but were important.The importance and influence of the HSI suitability indices of fish, shade, pond density, water quality, macrophyte cover, and frequency of drying on pond occupancy were all as expected with wide literature support25,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48. The negative pond occupancy response to climatic covariates of ground frost and high precipitation are supported in relation to annual survival47. ‘It is worth noting that the PIP value for wind speed was only just over the threshold for inclusion as important. Although ponds are shallow with limited stratification possible, wind speed has been shown to influence the distribution of eDNA in deeper waterbodies49,50. Estimating the presence of fish in a pond by direct observation for the traditional HSI may be problematical, and metabarcoding approaches to eDNA surveys which offer information on presence of other species would improve the accuracy of covariates, such as fish presence40. Indeed, assigning “Possible” fish presence within the HSI when scoring a pond accounted for the same percentage (33.1%) of both positive and negative eDNA samples. This suggests that when surveyors are not confident of fish presence, they are using this category in equal proportions for both occupied and unoccupied ponds. Landcover and bedrock were also important for pond level occupancy. This is expected given the importance of terrestrial habitat, and water retention to the species (Figs. S1, S3)35,51. However, with very unbalanced sample sizes between the categories (Figs. S2, S4), and influence of nearby land cover types uncaptured by the data, this variable is difficult to interpret, and we suggest further examination. Nevertheless, the positive associations with woodland and grassland reflect established knowledge of habitat preferences36. Equally, as freshwater predominantly relates to rivers and lakes rather than ponds in the landcover dataset used, negative relationships reflect the lower suitability of these habitats36.Several covariates, however, did not exhibit the expected response for pond occupancy. Terrestrial habitat was not found to be important despite the species being only semi-aquatic, and previous studies emphasising the importance of this variable36. This may be a result of the original Oldham et al.25 terrestrial habitat assessment being simplified into four subjective categories in the ARG UK26 protocol: this may not be nuanced enough to differentiate terrestrial habitat usage using statistical modelling. Waterfowl were not identified by the model as important predictors of great crested newt pond occupancy, where they have been elsewhere29,41, with one study suggesting a positive relationship between waterfowl species richness and great crested newt occupancy40. The lack of importance demonstrated in this data set may indicate that other covariates outweigh waterfowl in terms of occupancy importance, or they may only become important predictors of occupancy at very high waterfowl densities rarely observed in this data set. Similarly, pond area was not found to be an important predictor of pond occupancy. There was no difference in the mean area for occupied or unoccupied ponds; however, no occupied ponds were found above 10,000 m2. We would anticipate that both very small and very large ponds to be unsuitable for great crested newts25,52.Northing but not Easting was found to be an important predictor of pond occupancy. A distribution gradient with latitude is a common feature of biodiversity generally, and in the UK great crested newts are much more patchily distributed in Scotland than in England53,54. Pond occupancy estimates varied by year, with a greater occupancy in 2018 than the other years considered. This is likely linked to climatic conditions and may relate to the timings of ponds drying in relation to eDNA sample collection. This may therefore be an artefact of unoccupied ponds being more likely to dry early in the season and therefore being excluded from occupancy estimates for dry years, or local migration to less suitable habitat if core ponds start to dry, however long term analysis of individuals within a metapopulation shows little support for this47. As a result, in very dry years, we would expect an increase in pond occupancy to be observed in the data. Although average early spring rainfall for England in 2018 was higher than in either 2017 or 2019, rainfall during the main eDNA survey window of May and June was considerably less in 2018 than in the other two years (Fig. S5). Similar variation in year on year occupancy rate has been observed elsewhere30.As with all sampling methods, imperfect detection is a general feature of eDNA surveys. When high levels of qPCR replicates are used, false negative error is predominantly due to failure to collect DNA in a sample rather than failure to detect DNA within the lab. False positive error can occur at both stages and is exaggerated at Stage 2 by high levels of replication; Stage 2 false positive error is most likely in samples with a low proportion of replicates amplifying. We recommend using statistical models to estimate the occupancy of individual sites, taking into consideration sampling error. Failing that, a naïve occupancy threshold of two or three amplifying qPCR replicates, adjusting for total levels of replication, should be applied before assigning a site as occupied or not.With specific reference to great crested newts, we estimate approximately 20% of ponds through their natural range within England are occupied. We estimate that eDNA sampling failed to collect DNA from approximately 5% of sites where it was present. However, if eDNA is collected it is highly unlikely to be missed during the laboratory phase using the present protocol. We estimate that eDNA is erroneously collected in approximately 1.5% of water samples causing Stage 1 false positive results. However, false positives at the laboratory phase were found to be 2% per qPCR replicate; it is likely that this error would account for the majority of samples amplifying with one or two qPCR replicates, as a result these need to be treated with caution. To maximise accuracy, we recommend redistributing replication between the two stages, as is recommended elsewhere, and that thresholds to define a replicate as positive are further examined24,55. It is important to recognise that visual surveys also experience imperfect detection13, with observation errors likely to be similar to or greater than the error experienced using eDNA methods, particularly if the recommendations presented here are put in place to minimise laboratory stage false positive error. The benefits associated with eDNA over traditional methods allowing rapid collection of large scale distribution data are invaluable and should not be devalued in relation to traditional methods15. Although not identified within the models as important predictors, waterfowl, terrestrial habitat, and pond area may remain important habitat features for great crested newts. These covariates may be less important than the other HSI covariates, may not be measured in a sufficiently nuanced way to enable their importance to be identified, or may have influence on a local but not national scale29,40. However, equal weighing of the ten HSI variables is an oversimplification with the effect of some variables, for example pond area, overinflated within the HSI analysis, whereas others are undervalued, for example fish intensity. It is important to measure HSI covariates accurately and consistently to allow them to be utilised in statistical analysis such as this, and a review of the covariates and weighting is warranted now large occupancy data sets are becoming available. More
