Study setting
This study used retrospective cohort design, and administrative data were taken from the electronic medical records (EMRs) at Ontario Shores Centre for Mental Health Sciences in Ontario, Canada. Ontario Shores is a specialized mental health hospital in Ontario, Canada that serves a regional population of 2.8 million people. The hospital has 18 care units, 346 inpatient beds, and has over 94,000 outpatient visits annually. The proposed research will support the evaluation and continuous improvement of a multi-modal CBT program at Ontario Shores. Interventions in this program are provided by more than 30 clinicians (social workers, nurses, occupational therapists, and psychologists) across 30 sites.
Ontario Shores Anxiety and Mood Program (AMD) offers individual, group, eCBT (or internet-based CBT with coaching) and mixed modality that is a combination of two or three of these modalities offered to the patient. Individual therapy comprises 60-minute face-to-face structured psychotherapy sessions with a dedicated therapist. Group therapy involves in-person structured psychotherapy led by a single therapist and attended by two or more clients. To address mild to moderate depression and anxiety, Ontario Shores introduced eCBT on May 1, 2019. This modality incorporates coaching sessions, self-help learning modules, informational resources, and assigned homework to reinforce key concepts and impart mental health skills. Clients participating in eCBT, akin to other modalities, have the option to complete assessments throughout the program to monitor progress and enhancements. Aligned with the BC-CBT framework, this therapeutic approach seamlessly integrates in-person sessions with digital and online components for a comprehensive and flexible treatment experience.
Modality selection is determined by a client’s diagnosis, symptoms, and personal preferences. Based on the shared decision-making principals, mental healthcare providers will discuss the different potential treatment options with the client and come to an agreement on the best modality.
Data
Clinical staff members gathered and entered outpatient data inputted into the EMRs (a system called Meditech) at each client interaction. The outpatient records were collected for clients with an initial pre-session evaluation and at least one session of individual, group, or eCBT offered by Ontario Shores. The study utilised data acquired between May 1, 2019, and March 31, 2021, with a one-year follow-up through March 31, 2022, to assess the treatment completion of all clients. We looked back two years to ensure that the first client encounter had been included. For all clients, the first encounter date is the date of their initial pre-session evaluation prior to starting CBT treatment. The EMR data included demographic information from individuals who received eCBT, group CBT, individual CBT, or a combination of these treatments, as well as information from Integrated Community Access Program (ICAP) clinics that provide psychotherapy and medication for individuals with anxiety and mood disorders. At the time of data collection, our exclusion criteria were limited to 31 records pertaining to 5 distinct individuals for whom a confidential flag had been applied. Furthermore, we did not have access to data on services provided by the Ontario Structured Psychotherapy Program, which is a parallel dissemination of CBT occurring in the province during this period with unique treatment and data acquisition protocols.
Outcome measures
Outcome measures included CBT modality selection (i.e., individual, group, eCBT, mixed) and reliable and clinically significant improvement (RCSI) in symptoms of anxiety and depression from baseline to end of follow-up. In practice, healthcare providers will choose the best available modality and advise patients, accordingly, taking the client’s diagnosis and symptoms into consideration. Symptoms of anxiety were measured using the 7-item Generalized Anxiety Disorder (GAD-7) scale [14]. Depressive symptoms were measured using the 9-Question Patient Health Questionnaire (PHQ-9) [15]. Both measures have been validated previously and are commonly employed in clinical settings [16, 17]. The RCSI has been calculated by data and analytics professionals to support treatment planning and routine outcome monitoring inside the Ontario Shores Hospital. In the context of the CBT program, RCSI was defined as a shift in PHQ-9 of > = 6 points from baseline to final scores at the treatment’s conclusion, with an initial PHQ-9 score of > = 10. Similarly, GAD-7 changes > = 5 points from baseline to final scores at the end of treatment, where the first GAD-7 score was > = 8, signifying successful treatment [18, 19].
Data preparation
Our primary sample included 16,510 sessions for 1,098 unique client records who received CBT sessions in the Ontario Shores Mood and Anxiety clinics during the study period. In total, 6,864 electronic CBT, 3,996 group CBT, and 8,808 individual CBT sessions have been provided. All our sample started treatment and 97% completed two or more sessions.
Data management steps included excluding part of independent variables with more than 20% missing information. The decision to exclude variables with high percentages of missing information was primarily driven by the acknowledgment of poor data quality in out-patient databases. Imputing or attempting to fill in missing values for these variables might lead to biased or unreliable results, and therefore, it was deemed more appropriate to remove them from the list of variables considered for analysis. The variables retained in the analysis have undergone this careful data management process to enhance the overall validity of our research findings. We excluded sociodemographic characteristics with high percentage of missing information, such as education (98% missing) and racial identity (29% missing) as well as clinical variables such as primary diagnoses (28% missing) and secondary diagnoses (40% missing). For variables with low levels of missing data, including living arrangement (1%), employment status (6%), Canadian Index of Multiple Deprivation (CIMD) (5%), baseline GAD-7 (6%), and baseline PHQ-9 (less than 1%), we opted for Multiple Imputation (MI). This approach addresses missing values statistically, enhancing the completeness of the dataset and ensuring robustness in subsequent analyses. The MI process involved generating substitute values tailored to the type of each independent variable. In the end, no observations were dropped due to missing data.
Specifically, Bayesian linear regression was applied for continuous variables such as baseline client outcomes, logistic regression for dichotomous variables like living and employment status, and Polytomous logistic regression for categorical variables such as neighborhood deprivation. This approach ensured a comprehensive imputation strategy considering the diverse nature of the dataset’s variables. We utilized RStudio’s “mice” function within the “mice” package to implement MI.
Statistical analysis
Descriptive statistics were performed to compile client variables relevant to each CBT modality that were obtained from the client’s screening (i.e., age, sex, employment, and living status).
We estimated a multinomial logit regression for n = 1,098 clients to determine what client characteristics are related to entering each therapy modalities (Individual, group, eCBT, and mixed). We chose eCBT as the basis category due to its higher frequency. Using “multinom” function from “nnet” RStudio package, we modelled modality choice as a function of client age (adults between the ages of 17–70), sex at birth (male or female), living situation (alone or in a household), employment situation (employed or unemployed), Canadian Index of Multiple Deprivation (in five quantiles), baseline GAD-7, and baseline PHQ-9. The CIMD is a composite index that assesses various dimensions of deprivation and socioeconomic well-being across different geographic areas in Canada. Developed by Statistics Canada, the CIMD integrates multiple indicators related to income, education, employment, housing, and living conditions [20] to provide a comprehensive measure of deprivation. The index assigns quantitative scores to geographic areas based on the levels of deprivation observed. Higher scores indicate higher levels of deprivation, while lower scores suggest better socioeconomic conditions.
To answer our second research question on the predictors of having RCSI, we estimated a logistic regression model using “glm” function for post-treatment RSCI outcome in the PHQ-9 and GAD-7 measures. The outcomes were modelled as a function of CBT modality (Individual, group, eCBT, and mixed), age (adults between the ages of 17 and 70), sex at birth (male or female), living situation (alone or with others), employment situation (employed or unemployed), CIMD deprivation Index (in five quantiles), and baseline GAD-7 and PHQ-9 scores.
Add Comment