Study design
A retrospective population-based cohort design [26, 27] was used to compare QI rates generated from interRAI PC assessment data for home care clients designated to receiving PEoLC in Ontario, Canada prior to and during the COVID-19 pandemic. Retrospective analysis is an efficient and time-effective approach for monitoring and evaluating PEoLC, particularly over time periods and to compare across jurisdictions and demographic groups [27].
Data sources
The interRAI PC is a standardized clinical assessment instrument used to inform care planning and is designed for adults aged 18 and older with end-of-life (EOL) needs [28]. It was developed by a multinational research consortium as part of a suite of interRAI instruments and while it is not mandated for use in Ontario, it is frequently used for those receiving PEoLC [29]. To the best of our knowledge, all 14 regions were using the interRAI PC on intake into their palliative home care program. Over time, some regions have opted not to use it as a re-assessment tool. The interRAI PC assessment is typically done on admission to the home care program, for routine re-assessment, and at a time of the questions in the significant change, such as after hospitalization. The interRAI PC instrument has established validity and reliability for its measures [30], where the eight domains (symptoms/conditions, cognitive competency and communication, mood, functional status, preferences, social relations, spirituality, services and treatments) have an average kappa ranging from 0.76 to 0.95. A semi-structured interview process is used to obtain information on clients’ strengths, needs, and preferences to primarily guide care planning and service delivery. interRAI PC assessments are completed by trained professionals (usually a registered nurse) using a software application and these electronic assessments are shared with the relevant agency providers as part of the home care health record [31]. Data are self-reported by home care clients and/or their caregiver along with trained professionals who conduct the assessment and verify the data. The data are anonymized by Ontario Health Shared Services, and then stored at the University of Waterloo on a secure server for use by interRAI Canada Fellows and their students for research purposes.
Cohort periods
The COVID-19 pandemic was announced as a state of emergency by the Ontario provincial government on March 17th, 2020 [32]. The time periods for each cohort were selected based on when COVID was initially declared a pandemic in Ontario and for the duration of time data were available for this study. interRAI PC data were available until May 18th, 2021 when analysis began and so a timeframe of 60 weeks was available and applied to the COVID cohort. The same timeframe was applied to the pre-COVID cohort.
Study cohort
This study involved the comparison of two groups of home care clients aged 18 years and older receiving PEoLC in Ontario, Canada. The first group (pre-COVID) was home care clients with an interRAI PC assessment between January 1st 2019 to March 16th 2020, and the second group (COVID) were similarly assessed between March 17th 2020 to May 18th 2021. The two cohorts were further restricted to follow-up assessments (> 30 days) as admission assessments likely do not reflect the quality of care during the time of the assessment [33]. Where duplicates (individuals in both cohorts) existed, clients were assigned to the COVID cohort (and eliminated from the pre-COVID cohort) because there were fewer assessments in the COVID cohort. In cases where multiple non-admission assessments were available for an individual, the most recent assessment was kept.
Quality indicators
The following 16 prevalence-based QIs were used: hospitalizations in the last 90 days of life, emergency department visits in the last 90 days of life, falls, disruptive or intense daily pain, severe or excruciating pain that is inadequately controlled by medication, constipation, shortness of breath at rest, shortness of breath upon exertion, caregiver distress, negative mood, no advance directive, stasis/pressure ulcers, delirium-like syndrome, nausea or vomiting, and sleep problems. The QIs reflect a number of domains of PEoLC including structure and processes, physical, psychological and psychiatric, ethical and legal aspects of care [34]. These were selected based on an extensive literature review including a recent published study of validated QIs using the interRAI PC [33], and the limitations of available interRAI PC data elements of which several are endorsed by Health Quality Ontario as indicators of quality PEoLC [10, 11]. These QIs formed the basis for evaluating the quality of PEoLC for home care clients and met the criteria of being measurable, reflecting broadly the domains of the NCP framework [34], and in use to some extent in quality improvement and research initiatives. The QIs were defined using available interRAI PC data elements (Suppl 1) and based on definitions reported by Guthrie et al. [33].
Statistical analyses
All statistical analyses were completed on a remote secure server. Statistical analyses included descriptive statistics, where means and standard deviation (SD) values for continuous variables and percentages for categories were reported for covariates. Prior to examining differences in QI rates, propensity score matching was used to create matched cohorts based on demographic variables and other key covariates (Suppl 2) [35]. Following propensity score matching, the QIs were calculated and the differences in QI rates between cohorts were analyzed using a chi-squared test (= z test of 2 proportions) and effect size. Odds ratios (ORs) and effect sizes were generated to help in judging the clinical significance of group differences, as these measures are not driven by sample size [36]. All statistical analyses were completed using R Version 4.1.2. A two-tailed alpha level of 0.05 was used to identify statistically significant differences.
The two cohorts were matched on the following 21 covariates (Suppl 2): sex, age, Changes in Health, End-Stage Disease and Signs and Symptoms (CHESS) score [37], marital status, Local Health Integrated Network (LHIN) identifier, living arrangement, time since last hospital stay, number of days and total minutes in last week of formal care (e.g., home health aides, home nurse), individual instrumental activities of daily living (e.g., meal preparation, ordinary housework, managing medications) and individual activities of daily living (e.g., bathing, personal hygiene, walking, locomotion, transfer toilet, toilet use, and eating). The CHESS detects frailty and health instability, with higher scores indicating greater instability. The LHIN identifier reflects the geographic region where home care is coordinated and funded. These covariates were identified based on current literature and the availability of interRAI data elements.
As part of the statistical analysis, propensity score matching was used to reduce the effect of confounding variables between the two cohorts, thereby creating a more equivalent comparison cohort [35]. Acceptability of the matching procedure was based on falling below the recommended threshold of 0.1 of the standardized mean difference (SMD) for each covariate [38]. Nearest neighbour matching (NNM) was chosen as the primary propensity score method and a sensitivity analysis was used to explore weighted and overlap weighted propensity score methods. NNM is the most common form of matching used in propensity score analysis [39]. It is a matching method based on a distance measure and can employ a caliper. The method involves random selection of a treatment unit (COVID assessment) which is then matched to a control unit (pre-COVID assessment) that falls within the caliper and the process stops when all treatment units are matched. If a treatment unit is not matched, it is dropped.
The distance (caliper) was set to 0.2 for this study, which has been shown in prior research to be suitable for a variety of settings and regarded as optimal, thus it was used for this study [40]. The effect of using a larger caliper, or not using a caliper at all, would be to increase the sample size, but this typically results in poorer covariate balance. Matching without replacement (also called 1:1 matching) was another methodological decision made where each control unit is only matched to one treatment unit [41].
In terms of the additional propensity score methods, weights are employed to reflect the importance assigned to propensity scores and covariate balance during matching. The aim is to use more of the sample using multiple controls per treatment unit and weighted composites of controls, unlike NNM which uses paired-matching. Matching weight and overlapping weight were the two weighted methods used. The matching weight method was proposed by Li and Green [42] and is comparable to one-to-one pair matching without replacing; however, instead of discarding unmatched treatment units, no unit is ever rejected entirely but instead is down-weighted so that multiple controls can be matched with the weight distributed among these units where a fraction of the unit is contributing. As units are weighted so that they contribute less to the sample than with unweighted units, the effective sample size may be lower than with paired matched [43]. The overlapping weight method proposed by Li et al. [44] is another weighting method that matches on propensity score as well as the covariates. The method works similarly to the matching weight method, but the weights are based on both the propensity score and covariates which results in the exact balance on the means of all the covariates included. As a result, units are automatically down weighted with extreme propensity scores. The best practice for selecting propensity score methods is to try multiple methods and explore all those that meet the pre-established criteria for acceptability, as there is no universally superior method [43, 45]. In this study, the pre-established criteria for acceptability were adequate covariate balance. If the methods met these two criteria, they were considered equally valid for use in the analysis [43].
Add Comment