A total of 779 persons opened the survey, 486 persons started responding to it, and 308 persons from 40 countries submitted their responses. Seven of these 308 respondents had never heard about biases, and the remaining 301 persons (i.e. 38.6% of those who opened the survey) answered our questions about their attitude to biases.
Nearly all (98%) scientists who responded to our survey were aware of the importance of biases in science. Among these, 33% reported ‘very well’ for their awareness on this topic, 52% classified their awareness as ‘well’ and 13% as ‘poor’. Most of respondents learned about biases from their university courses (36%), from contacts with colleagues (22%) or from scientific literature (20%). Among the different kinds of biases, the best known was observer/observation bias (82%), followed by publication bias (71%) and selection bias (70%); confirmation, reporting/presentation, researcher, measurement, geographic and funding biases were known to 50–60% of respondents (see Appendix S1). Among the seven suggested definitions of biases (see Appendix S1), ‘the tendency to search for, interpret, and publish information in a way that confirms one’s pre-existing beliefs or hypotheses’ corresponded to the understanding of biases by 80% of the respondents, and ‘preferential publication of statistically significant results’ was ranked the second, being selected by 61% of respondents.
In the opinion of our respondents, the stages of scientific research differed in their susceptibility to biases (χ220 = 90.0, P < 0.0001) and were ranked as follows, from greatest to least bias susceptibility: interpreting the results > planning/designing the study > publishing the outcomes > reporting the outcomes > analysing the results > implementing the study. Similarly, the types of publications were considered prone to biases to different extents (χ216 = 86.8, P < 0.0001) and were ranked as follows: narrative reviews > studies based on observational data > studies based on modelling > studies based on experiments > meta-analyses.
Most scientists thought that the severity of the impact of biases on science in general and on their particular research field was medium or high, and a few considered that it was negligible. At the same time, our respondents estimated the impact of biases on their own studies as high almost three times less frequently and as negligible seven times more frequently when compared with their estimates of the impact of biases on other studies within their own research field (Fig. 1).
Impact of biases on science in general, on one’s own research field and on one’s own studies, as estimated by 301 respondents. Bars marked with different letters differ from each other at P < 0.05 within each group (χ2 test).
The respondents considered the most important ways to avoid biases (see Appendix S1) to be reporting all results, not only statistically significant ones (89% of respondents), checking for repeatability of all measurements (78%), performing a random choice of experimental units (78%) and using blinding (70%). At the same time, 15% of respondents believed that a haphazard (i.e., neither systematic nor random in a strict sense, and therefore subjective and prone to biases) choice of experimental units could also help to avoid biases. Most researchers reported that they were thinking about biases that could affect the outcomes of their own studies (81%) and that they planned and implemented particular measures to avoid biases in their research (75%), but only 61% of the respondents reported these measures in their publications.
As expected, the scientific productivity and teaching activity depended on the stage of the career. Senior scientists published more scientific papers during the three years preceding their responses to our questionnaire than did the mid-career and early career scientists (12.2, 8.6 and 3.8 papers, respectively). Senior and mid-career scientists reviewed more manuscripts during the same period (12.3, 12.7 and 4.7 manuscripts, respectively) and were more frequently involved in teaching than were the early career scientists (80, 82 and 55%, respectively).
The understanding of biases did not change with the career stage, as indicated by the similar selection of different definitions of biases by early, mid-career and senior scientists (χ212 = 6.71, P = 0.88). A higher proportion of undergraduate students and early career scientists had learned about biases from university courses when compared with mid-career and senior scientists, whereas the more advanced career group had learned about biases mostly from the scientific literature (Fig. 2). Early career scientists were aware of a larger number of bias types (Appendix S1) when compared with either the mid-career (S = 57, P = 0.005) or the senior scientists (S = 72.5, P = 0.0001). In particular, a larger fraction of early career scientists, compared with senior scientists, were aware of such important biases as confirmation bias (Fig. 3), observer bias (85 and 75%, respectively; χ21 = 4.34, P = 0.04), selection bias (77 and 64%, respectively; χ21 = 4.33, P = 0.04) and cognitive bias (34 and 21%, respectively; χ21 = 4.74, P = 0.03).
Sources of the first information about biases in relation to the stage of respondent’s research career, as reported by 35 undergraduate students, 122 early career scientists, 49 mid-career scientists and 95 senior scientists. Bars marked with different letters differ from each other at P < 0.05 within each group (χ2 test).
Selected characteristics of the respondents’ attitudes to biases in relation to the stages of their research careers, as reported by 124 early career, 50 mid-career and 97 senior scientists. Bars marked with different letters differ from each other at P < 0.05 within each group (χ2 test).
Early career scientists generally gave for the impact of biases on all stages of the research process a higher rating than the senior scientists did (S = 10.5, P = 0.03), with greatest differences seen in the percentage of respondents who admitted high impacts of interpretation bias (Fig. 3) and of reporting bias (34 and 22%, respectively; χ21 = 4.30, P = 0.04) in ecological research. Among methods that would allow the avoidance of biases, early career scientists mentioned blinding more frequently than senior scientists did (Fig. 3), while the importance of other methods was similarly appreciated by both groups of researchers. At the same time, senior scientists declared more frequently than early career scientists that they were ‘very well’ aware of biases, and the senior scientists estimated the impact of biases on their own studies as ‘negligible’ twice as frequently than the early and mid-career scientists did (Fig. 3).
The proportion of women among our respondents declined with the duration of their professional activity, from 57% of early career scientists to 38% of mid-career scientists, and finally dropped to 27% of senior scientists (χ22 = 19.6, P < 0.0001). This proportion was greater in high GDP countries than in low GDP countries (52 and 28%, respectively; χ21 = 14.8, P = 0.0001). Female respondents had published fewer papers than male respondents during the past three years (6.0 and 9.1, respectively; χ21 = 11.1, P = 0.026) and females were less involved in teaching than male respondents across all stages of their careers (61 and 78%, respectively; χ21 = 11.9, P = 0.0006).
Male and female respondents significantly differed in their attitudes towards biases. More men than women claimed that they were ‘very well’ aware of biases (39% and 27%, respectively; χ21 = 5.43, P = 0.02). At the same time, female scientists gave higher ratings for the impacts of biases on the different stages of research (S = 10.5, P = 0.03) and on the different fields of science (S = 18, P = 0.008) than male scientists did.
Female and male respondents had similar assessments of the severity of the impact of biases on science in general and on their particular research field (χ21 = 7.54, P = 0.06 and χ21 = 5.85, P = 0.12, respectively), but female respondents gave much more critical evaluations of the impact of biases on their own studies: 16% of women and 8% of men assessed this impact as ‘high’ (χ21 = 3.98, P = 0.046), whereas 13% of women and 27% of men considered it ‘negligible’ (χ21 = 8.59, P = 0.003) (Fig. 4). The latter result mostly reflects gender differences within the group of early career and mid-career scientists, while the attitude of senior scientists to biases in their own studies did not differ between women and men (Fig. 4).
Self-estimates of the awareness about biases and of their impact on one’s own studies and on other studies in one’s own research field in female and male scientists in relation to the stages of their research careers, as reported by (women/men) 70/50 early career, 19/27 mid-career and 26/65 senior scientists. Bars marked with different letters differ from each other at P < 0.05 within each group (χ2 test).
Respondents from high GDP countries were aware of a greater number of biases than were respondents from low GDP countries (8.5 and 5.9 biases per person, respectively; S = 75.5, P < 0.0001; Fig. 5). Within the high GDP countries, respondents from the USA declared themselves ‘very well’ aware of the importance of biases more frequently than European scientists did (42 and 24%, respectively; χ21 = 8.30, P = 0.004), and the American respondents knew about a greater number of bias types when compared with the respondents from Europe (9.1 and 7.5 biases per person, respectively; S = 63.0, P = 0.001).
Percent of respondents from high and low GDP countries (219 and 82 scientists, respectively) who were aware of the most commonly known biases. Asterisks indicate significant (P < 0.05) differences between high and low GDP countries (χ2 test).
Respondents from high GDP and low GDP countries estimated the impact of biases as high equally frequently with respect to science in general (32 and 40%, respectively; χ21 = 1.57, P = 0.21), their particular research field (27 and 38%, respectively; χ21 = 3.67, P = 0.06) and their own studies (11 and 17%, respectively; χ21 = 2.39, P = 0.12). Similar proportions of respondents from both groups evaluated the impact of biases on their own studies as negligible (20 and 21%, respectively; χ21 = 0.02, P = 0.90).
Respondents from high GDP countries were more aware of four (of the five suggested) methods for avoiding biases when compared with the respondents from low GDP countries (Fig. 6). At the same time, haphazard selection of experimental units was considered as a measure to avoid biases twice more frequently by respondents from low GDP countries than by respondents from high GDP countries (Fig. 6).
Percent of respondents from high and low GDP countries (219 and 82 scientists, respectively) who knew different methods for avoiding biases and who erroneously thought that haphazard selection also helps to avoid biases. Asterisks indicate significant (P < 0.05) differences between high and low GDP countries (χ2 test).
Similar proportions of scientists from high and low GDP countries (37 and 32%, respectively; χ21 = 0.73, P = 0.39) learned about biases from their university courses, but a significantly lower proportion (15%) of scientists from Eastern Europe and Russia obtained information about biases in this way when compared with this proportion (40%) of scientists from the rest of the world (χ21 = 4.94, P = 0.02).
Source: Ecology - nature.com
