Scientific Papers

Test characteristics of shorter versions of the Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) for brief screening for problematic substance use in a population sample from Israel | Substance Abuse Treatment, Prevention, and Policy


Sample

Using similar methodology to an epidemiological survey from 2018 [12, 16, 17], cross-sectional data were collected in April 2022 from a quasi-representative general population sample of adults living in Israel. Respondents were recruited from a demographically diverse Web panel of individuals who choose to participate in surveys, maintained by the national digital collection agency iPanel [18]. The main sample included respondents aged 18–70, as those above 70 are less likely to participate in surveys; and Hebrew speaking and Jewish, since cultural differences would require substantial methodological adaptations [19]. To construct a quasi-representative sample of the adult, Hebrew-speaking, Jewish population in Israel, a stratified sample was drawn utilizing specified quotas [20] based on age, gender, geographic area, religiosity, and education, using estimates from the 2018 survey (to allow comparisons across surveys), which were based on Central Bureau of Statistics census data. Deviations up to 2.0% from the quotas were allowed. From those eligible within strata, potential participants were selected in two ways. First, all those surveyed in 2018 who were still registered with iPanel were invited to participate. Second, of those who had not participated previously, potential participants were selected at random. When individuals agreed to participate, they were screened against the quotas, until the target number of responses was met. Strict confidentiality was maintained, as iPanel did not have access to responses, and identification information was not available to the researchers. Survey methodology is consistent with the ICC/ESOMAR International Code on Market and Social Research [18]. The Institutional Review Board of the Reichman University approved this study.

Respondents provided electronic informed consent before beginning the online survey conducted via the Qualtrics platform [21]. The survey included items related to sociodemographics, substance use, addictive behaviors, and physical and mental health. Internet surveys may be better for collecting sensitive information such as illicit substance use or other addictive behaviors [22]. Upon survey completion, participants received online gift cards worth 22 New Israeli Shekel, which was ~ 6.50 US dollars. Quality assurance was maintained by: survey by invitation to registered respondents; 3 attention checks; and identifying respondents with unexpected response patterns. Of those invited to participate (11,750), 4,948 agreed, 1,944 were excluded due to quotas, 505 did not complete the survey (135 failed attention checks, 370 dropped out), and 25 were excluded based on response patterns, for an analytical sample of 2,474.

Measures

To assess substance use behaviors, ASSIST 3.1 was administered and scored [23]. Respondents selected substances they ever used non-medically: tobacco, alcohol, cannabis, cocaine, amphetamines, inhalants, sedatives, hallucinogens, opioids, and other substances. Culturally appropriate examples were given for each category. For each substance used, respondents reported on (1) frequency of use in the past 3 months (current use). Those with current use reported on frequency of (2) craving (strong urge to use); (3) health, social, legal, or economic problems due to use; and (4) failure to fulfill expectations due to use. Frequency response options were: never, once or twice, 1–3 times per month, 1–4 times per week, 5–7 times per week. Respondents with lifetime use were then asked (5) if anyone ever expressed concern about their use and (6) if they had trouble controlling use; responses options were no; yes, prior to the past 3 months (past); and yes, within the past 3 months (current). A separate module assessed the ASSIST 3.1 for non-medical use of prescription stimulants and prescription painkillers, rather than assuming these would be covered by the standard categories, as suggested by the US National Institute for Drug Abuse (NIDA) [24]. This study included analysis of measures for substances with > 2% prevalence of past three month use: alcohol, tobacco, cannabis, sedatives, prescription stimulants, and prescription painkillers.

For each substance, responses were weighted and a substance involvement score was calculated by summing response weights to the 6 questions, with the exception of tobacco, where failure to fulfill expectations was excluded. As done previously [12], except for alcohol, a binary variable (problematic use) was defined as a score of 4 or more (4+), combining moderate risk (4–26) and high risk (27+) levels, which correlated with substance abuse and dependence [6] that are now considered to indicate a combined substance use disorder [25]. For alcohol, 10 + was used as the cutoff for problematic use. In sensitivity analysis, 8 + was used to indicate problematic use for cannabis, which may have better sensitivity and specificity [26], probably due to changing use practices.

The ASSIST-Lite [9] contains items that have binary yes/no responses, adapted from the ASSIST 3.1 questions (listed above). These items were scored based on the ASSIST 3.1 responses. Item 1, any current use, was positive for those who responded at least “once or twice” for frequency of current use. Item 2, using at least weekly, was positive for those who responded at least “1–4 times per week” for frequency of current use. Item 3, craving at least weekly, was positive for those who responded at least “1–4 times per week” for frequency of craving. Items 4 and 5, current concern about use and current difficulty controlling use, were positive for those who responded “yes, within the past three months” to those questions. For different substances, different items were included: for cannabis and sedatives, items 1, 3, and 4; for prescription stimulants, items 1, 2, and 4; for prescription painkillers, items 1, 4, and 5; and for alcohol, items 1, 4, 5, and any binge use (4 or more drinks per occasion), as assessed in the Alcohol Use Disorders Identification Test. For tobacco, the ASSIST-Lite included items from the Fagerstrom Test for Nicotine Dependence which was not assessed in this survey. Instead, only item 1 was used, as suggested in the modified ASSIST-Lite developed by the UK National Health Services [27]. For each substance, the relevant items were summed to create ASSIST-Lite scores.

The ASSIST-FC [10] contains two ASSIST 3.1 questions, frequency of use and other’s concern about use. For each substance, weighted responses for those two ASSIST 3.1 items were summed to create ASSIST-FC scores. As planned, since ASSIST-FC scores performed better than ASSIST-Lite scores (see Test performance, below), binary ASSIST-FC versions were constructed with thresholds starting from 2+ (corresponding to moderate and high risk levels; 6 + for alcohol), and including 3 + and 4+ (alcohol, 5 + and 7+).

To explore the inclusion of craving, for each substance, the ASSIST-FCr score was calculated by summing the weighted responses for frequency of use and craving, and binary versions were constructed with thresholds from 2 to 4 for all except alcohol (7–9), with additional thresholds for cannabis (5,6) to match ASSIST 3.1 with the 8 + threshold. The ASSIST-FCCr score was calculated by summing the weighted responses for frequency of use and craving, and other’s concern, and binary versions were constructed with thresholds from 3 to 5 for all except alcohol (8–10), and additional thresholds for cannabis (6,7).

Sociodemographic moderators included gender (men; women) and age (18–25; 26–34; 35–49; 50–70).

Analysis

Sample descriptives were calculated for sociodemographic variables, substance use, and ASSIST 3.1 problematic use (moderate or high risk levels). Analysis was planned such that results from one step would inform the next step, e.g., the most informative of the two shorter versions would be kept for additional analyses. For each substance, receiver operator characteristic (ROC) curve analysis was used to assess the area under the curve (AUC), which evaluates the ability of the ASSIST-Lite and ASSIST-FC scores to identify those with ASSIST 3.1 problematic use. AUC scores ≥ 0.9 are considered excellent, and between 0.8 and 0.89 are considered good [28]. To determine which version performed better, AUC values for ASSIST-Lite and ASSIST-FC were considered to differ if the 95% confidence intervals (CI) were non-overlapping. For the better scores (ASSIST-FC), to see if scores showed differential performance by gender or age, AUC values were compared, by taking the difference between values for men and women, and for each age group and the control group (18–25). AUC values were considered different if the 95% CI for the difference did not overlap with 0.

Since the goal was to determine how well the short forms could identify those with ASSIST 3.1 problematic use (“gold standard”), for each substance, test characteristics for the ASSIST-FC binary variables (tests) were calculated from the true positives (TP; yes for ASSIST-FC and ASSIST 3.1), false negatives (FN; no for ASSIST-FC and yes for ASSIST 3.1), false positives (FP; yes for ASSIST-FC and no for ASSIST 3.1) and true negatives (TN; no for ASSIST-FC and ASSIST 3.1). Sensitivity, ability to correctly classify an individual as having the outcome of interest, measures the proportion of those with problematic use correctly identified by the test [TP/TP + FN]. Specificity, ability to correctly classify an individual as not having the outcome of interest, measures the proportion of those without problematic use correctly identified [TN/FP + TN]. Positive predictive value (PPV) is the proportion of those with a positive test who have problematic use [TP/TP + FP], and negative predictive value (NPV) is the proportion of those with a negative test who don’t have problematic use [TN/TN + FN]. Since the study goal is to identify tests to efficiently screen for patients who would benefit from brief intervention or referral to specialist treatment, where they would score high on further tests, e.g., ASSIST 3.1., tests with high specificity would be favored, where a positive score rules in problematic use, because without problematic use those would have tested negative; similarly, high PPV, indicating fewer false positives, would be favored.

Chance corrected agreement (kappa) was evaluated for ASSIST-FC binary versions and ASSIST 3.1, with kappa > = 0.61 considered good [9, 29].

To provide information about intervention needs, among those with ASSIST-FC binary versions, the percent with current use was calculated.

Exploratory analysis: including craving

ROC curve analysis compared ASSIST-FC, ASSIST-FCr, and ASSIST-FCCr scores. Test characteristics and agreement were assessed for binary versions of ASSIST-FCr and ASSIST-FCCr with ASSIST 3.1 problematic use, and were compared to estimates for ASSIST-FC; estimates with non-overlapping CI were considered to differ. Among those with binary ASSIST-FCCr, the percent with current use was calculated; all those with non-zero scores on ASSIST-FCr have current use.

Analysis was conducted using SPSS software version 28 [30].



Source link