The data come from a cross-sectional international survey of researchers, administered in 2021 as part of the European Commission-funded ‘Standard Operating Procedures for Research Integrity’ (SOPs4RI) project. The online survey utilised a systematic, stratified probability sample of over 60,000 researchers from across scholarly fields. The sample comprised authors of research articles indexed in Clarivate’s Web of Science citation database. The survey protocol was pre-registered with the Center for Open Science in February 2019 (available, along with reproducibility materials at Reference [43]). Further details about IRIS are contained in Allum et al. [44] Our analytical sample is comprised of 39,699 researchers. Of these, 57.6% were male. 40.8% primarily research in the natural sciences, 14.4% in the medical sciences, 30.8% in the social sciences and 14% in the humanities. 4.5% are employed in the United States of America, 86.7% in Europe and 8.7% elsewhere.
Variables
Questionable research practices
A total of eight behaviours pertaining to various aspects of the research process (e.g., publication, PhD supervision) that are widely considered to be questionable and undermining of the trustworthiness of scientific findings were generated for the purposes of the survey. Descriptions of these questionable research practices (QRPs) can be found in Table 1.
Each QRP was accompanied with a question asking: ‘thinking about your research carried out for your publications over the last three years, how often has the following occurred?’, with response categories ‘often’, ‘sometimes’, ‘rarely’, ‘never’, ‘does not apply’. The three-year timeframe was used to limit recall bias and to be able to state prevalence in more precise terms. The eight individual QRP item scores were aggregated for each respondent, resulting in a total score ranging from 0 to 32. This sum was then divided by the count of items that were both responded to and deemed relevant by respondent (i.e., where ‘does not apply’ was not selected). This scale was standardised to z-scores, ensuring each score is mean-centered and scaled by the standard deviation, with higher scores indicating greater average engagement. Operationalising ‘QRP engagement’ in this way is consistent with previous research that has attempted to model the relationship between various explanatory factors and QRPs (e.g., Gopalakrishna et al. [29]). Figure 1 displays the weighted distribution of our standardised mean QRP engagement (mean = -0.04, median = -0.18, standard deviation = 0.92).
Weighted distribution of standardised mean QRP engagement for 39,699 respondents
Respondent-level explanatory factors
A composite variable indicating scientists’ commitments to the normative ideals of science was computed based on five items assessing researchers’ values, each based on Merton’s delineation of the normative ideals of science (i.e., universalism, disinterestedness, organised scepticism, and communalism). These items were adapted from Anderson [45] and Anderson et al. [25]. They described a set of behaviours, and respondents indicated whether they personally feel that these behaviours reflect how researchers should behave, with response categories anchored at ‘yes, usually should’ (coded 1) to ‘no, never should’ (coded 5). Descriptions of these normative ideals can be found in Table 2. Items 3 and 4 were reverse-coded. For each respondent, an average score was computed based on the five items. This composite variable was standardised to mean zero and standard deviation one. Higher scores indicate a greater commitment to the normative ideals of science. Figure 2 displays the weighted distribution of our standardised composite variable, which represents the average adherence to the normative ideals of science (mean = 0.06, median = 0.12, standard deviation = 0.88).
Weighted distribution of our standardised mean commitment to normative ideals of science for 39,699 respondents
Each respondent reported their main disciplinary field and was assigned to one of four broad categories: social sciences, humanities, medical sciences (including biomedicine), and natural sciences (including technical sciences). This variable was recoded into three binary variables representing each scientific field, with ‘natural sciences’ omitted, as the reference category. Respondents additionally provided the type of employment contract that they are on from a pre-specified list: permanent contract, temporary contract, or no contract. We recoded this variable into two binary variables with permanent contract omitted as reference category. Respondents indicated the stage of their career, selecting from either early-career, mid-career, late-career or retired. This variable was also recoded into three dummy variables, with the reference category early-career. Each respondent reported their sex, with response categories male, female and prefer not to say. Those stating prefer not to say (N = 1109) were removed from the analysis. This was recoded as a dummy variable with male omitted as the reference category.
Organisation level explanatory factors
Each participant was asked to identify their workplace type from six sectors: academia, industry, not-for-profit research institute, government research centre, healthcare setting, or other. We recoded this variable into five indicator variables, using academia as the reference category. Respondents were also presented with a series of descriptions characterising what could be regarded as a functionally optimal working environment and were asked the extent to which the descriptions resemble their own working environment. The items selected for use in our analysis were the presence of adequate integrity training (i.e., ‘training in research integrity is provided to all researchers, at all career stages, by qualified trainers’), provision for handling integrity breaches (i.e., ‘researchers can consult a qualified person in confidence with any research integrity concerns. Breaches are detected and sanctioned in a fair and standardized way, protecting both whistleblowers and those accused of misconduct’) and a positive working culture (i.e., ‘collegial, and without harmful publication pressure, detrimental power imbalances or conflict’). These were measured on a five-point scale, anchored at ‘resembles my environment very closely’ (coded 1) and ‘resembles my environment not at all closely’ (coded 5). We reverse coded this variable so that higher scores indicated that the description is more reflective of their working environment. The variables were mean-centred and standardised to z scores. Additionally, respondents were asked to state whether their institution has a written statement on research integrity, with three answer categories: ‘yes’, ‘no’ and ‘I don’t know’. We recoded this variable into two dummy variables, with ‘yes’ as the reference category.
Contextual variables
Integral to the present research is the degree to which variability in QRP engagement is associated with differences between organisations and countries. We use the email addresses of survey respondents to create a variable indexing organisations, based on their domain name (e.g. @harvard.edu). Generic email addresses were identified using a non-exhaustive list of the most used general email addresses (e.g. Gmail, Hotmail). We deleted cases where the domain name was not associated with a research-producing organisation. In total, 9,304 non-institutional email addresses were removed, leaving our analytical sample containing a total of 7,666 unique organisations. Figure S1 in the supplementary material presents the frequency distribution and descriptive statistics for organisation. The second contextual variable, country, was based on the self-reported country of employment. We focused on the ‘country of employment’ as it reflects the researcher’s location and, consequently, their organisational setting. The final analytical sample consisted of respondents employed across 34 countries. These were predominantly in Europe.
Interaction terms
To assess the extent to which a researcher’s scientific values may act as a safeguard against workplace pressures and non-collegial working environments, thus reducing engagement in QRPs, we specify a cross-level interaction, interacting commitment to the normative ideals of science with working environment. We additionally test whether a researcher’s working environment has more of an influence on research conduct for those early in their career and those on non-permanent contracts, who potentially have more to gain from engaging in suboptimal practices. On this basis, we create several interaction terms, interacting the working environment on our career stage and employment contract indicator variables. Moreover, we expect the influence of a less collegial and more competitive working environment on engagement in QRPs to be reduced where integrity training is sufficient, there are procedures (incl. firmer penalties) for handling contraventions of good research practice, and where the researcher is aware of the institutions commitment to ensuring research integrity (i.e., by way of awareness of a written statement on research integrity). Therefore, we specify several interaction terms, interacting working environment with a) existence of integrity training, b) procedures for handling integrity breaches, c) there being no written statement on research integrity and d) lack of awareness of whether there is a written statement on research integrity.
The presence of integrity training, along with suitable procedures and personnel for addressing integrity violations, could potentially support researchers in adhering to Mertonian scientific values. As a result, we define interaction terms for the commitment to these ideals of science in relation to a) integrity training, and b) handling of integrity breaches. Finally, the relationship between having an awareness of a written statement on research integrity and QRP engagement will plausibly be stronger for those who are more strongly committed to the normative ideals of science. Conversely, we expect the presence of a research integrity statement to matter less, and be less influential on research conduct, for those who do not adhere to the normative ideals of science. On this basis, we generate two interaction terms, interacting commitment to scientific ideals of science on a) lack of awareness as to whether their institution has written statement on research integrity, and b) an awareness that their institution does not have a written statement on research integrity.
Analysis strategy
To account for the non-independence of responses to the QRP items within organisations and countries, and in recognising the hierarchical nature of the data (researchers nested within organisations and countries), we use a multilevel regression approach, implemented using the lme4 package in R (version 2022.12.0 + 353). Adopting a multilevel modelling approach means that we can partition the variability of QRP engagement across individuals, organisations, and countries, allowing us to examine the relative importance of these three components.
Our modelling strategy is as follows. In Model 1 we specify a random intercept model without any predictors, allowing us to assess the variability in QRP reporting at each of our three levels. In Model 2 we again specify a random intercept model, regressing standardised mean QRP engagement on individual-level predictors (incl. contract type, career stage, disciplinary field, and sex). In Model 3 we include our individual-level variable of primary interest, commitment to the normative ideals of science. In Model 4 we include organisation-level predictors (incl. workplace type, integrity breaches, integrity training and RI statement awareness), allowing us to see whether variability in QRP engagement between organisations is explained by these specific organisation-level attributes.
In Model 5 we add our final variable of interest, working environment, into a random-slope model. Including random slopes permits us to see how the relationship between QRPs and a) normative ideals of science, and b) working environment, varies across organisations. Finally, in Model 6 we include several single-level and cross-level interaction terms, allowing us to see whether the relationship between a) commitment to the normative ideals of science, and b) working environment, is moderated by career stage, contract type, RI statement awareness, integrity training and integrity breaches. All models are multiple-linear regression models using restricted maximum likelihood (REML). We have made the full R code available on the Open Science Framework (OSF) [46].
Add Comment