This study aimed to examine older adults’ compliance with EMA protocols, focusing on the impact of both inter- and intrapersonal variables. The findings provide valuable insights into the patterns associated with EMA compliance in this population and highlight some critical issues and challenges that should be considered when designing future EMA studies among older adults.
The average compliance rate of 77% across the entire sample is consistent with previous research in older adults [11]. However, in study 2, compliance was notably lower, possibly due to the lower age limit of 60 years, differing from the other three studies where the lower limit was set at 65 years. This variation in age limit may have led to a sample including individuals who are not yet retired. While our results suggested that the odds of compliance decrease with age, this trend may only apply to retired older adults. Another factor to consider is the focus of the research. Study 2 was the only study concentrating on dietary behavior. It might be that older adults are less inclined to respond to EMA questions related to dietary habits compared to physical activity and sedentary behavior. As the current study was not designed to examine the influence of study topic on compliance, future research should explore this possibility further. The observed increase in compliance over the first study days, except in study 3, suggests improved engagement as participants become more familiar with the assessment process. Future researchers might consider removing data from the first, or first two measurement days, if sufficient data is available, to mitigate the influence of initial adjustment. The notable variability in peak compliance between studies underscores the importance of considering study-specific factors that may influence participant compliance with EMA. Previous reviews have already summarized the study design characteristics that might contribute to compliance with EMA, such as the number of prompts per day and the number of items per prompt [1, 8,9,10,11], however, none of the reviews examined peak compliance over time. Although the four intensive longitudinal studies included in the current study shared similar study characteristics, differences in sampling strategy, number of prompts, number of items included in the EMA questionnaire, and target behavior might have affected (peak) compliance. However, examining these differences was outside the scope of the current study.
Based on our pooled results, the highest variance in older adults’ compliance with EMA is attributable to interpersonal factors. Age emerged as a significant predictor of compliance with older participants exhibiting lower odds of compliance. This finding is in line with our expectation based on digital literacy assumptions. The older the elderly get the less likely they have well-developed digital skills, which are crucial to successfully comply with device-based EMA [22, 23]. Furthermore, marital/cohabiting status also emerged as a significant predictor of older adults’ compliance with EMA. The positive association between marital/cohabiting status and compliance suggests a potential role of social support in fostering engagement. A recent systematic review stressed the importance of social support to overcome technical issues and increase motivation in mobile health interventions [24]. Finally, smartphone ownership also emerged as a strong predictor of older adults’ compliance with EMA. This underscores the importance of access to and familiarity with technology in facilitating engagement and adherence to EMA procedures. Although a small introductory training was provided for those without a smartphone in the current studies, our findings suggest that this may not adequately prepare participants to consistently comply with the EMA protocol. Hence, future research should carefully consider the inclusion of older adults without smartphones in EMA studies and should provide sufficient familiarization or training before the start of the study.
Furthermore, our study shed light on the impact of intrapersonal factors on older adults’ compliance with EMA. Although their role appeared relatively small compared to interpersonal factors, we observed that participants exhibited markedly lower levels of compliance in the evening compared to the morning or afternoon. This was not in line with previous research, conducted in children, adolescents and adults, showing higher levels of compliance in the evening [10, 25], and suggests that temporal fluctuations in motivation, activities, contexts or other intrapersonal variables might have influenced the willingness or opportunity of older adults to comply with EMA requirements [13, 26]. Understanding the temporal fluctuations in compliance provides an opportunity to optimize EMA data collection strategies.
Based on the current results, it is evident that non-compliance in our pooled dataset is not completely at random. As outlined in the introduction, non-compliance and missing data can be categorized into three distinct types: MCAR, MAR, and MNAR. Given that specific population subgroups, such as the oldest older adults, those living alone, and those without a smartphone, exhibit a higher likelihood for non-compliance, and that non-compliance tends to be more prevalent in the evening, it appears that the tendency for missingness is at best associated with observed data, aligning with the categorization of MAR. The presence of MAR data could introduce bias, potentially compromising both the internal and external validity of study outcomes, if not properly addressed [13, 27]. For instance, consider a scenario where one aims to describe the sedentary activities of older adults; activities typically occurring in the evening might be underrepresented. Similarly, when examining the social contexts of physical activities in older adults, activities usually performed by participants living alone may be inadequately represented. Therefore, overlooking or inadequately addressing non-compliance with EMA could indeed present a significant threat.
To improve the validity of future EMA studies in older adults, several mitigation strategies can be implemented, contingent upon the nature of the research question being addressed. For descriptive or predictive research questions, baseline oversampling of respondents with specific characteristics, or providing tailored protocols to improve their compliance may suffice [28]. These tailored protocols may include strategically scheduled assessments during periods when older adults are more likely to be receptive and attentive, or strategies that directly address the challenges or motivations associated with evening assessments. For causal research questions, we recommend drawing directed acyclic graphs (DAGs) during the study design phase [29,30,31]. DAGs serve as powerful visual tools, developed based on expert knowledge, about the hypothesized causal relationships between the various factors influencing compliance and can be used to identify selection/collider bias (i.e. the change in the association between an exposure and an outcome under study when conditioning on [via restriction, stratification or regression adjustment] a collider [i.e. a third factor influenced by both exposure and outcome]) [29, 31, 32]. Consider a study investigating the causal effect of emotions on snacking behavior. Within this study, compliance could act as a collider because it is likely to be affected by emotions as well as snacking behavior (i.e., participants not reporting unhealthy snacking because of social desirability). Selecting only those who comply with the EMA questionnaire, is the same as conditioning on the collider, potentially yielding biased estimates. By proactively identifying sources of bias through DAGs and applying mitigation strategies in the study design, researchers can enhance the validity of their EMA findings. Finally, the integration of statistical techniques like multiple imputation or inverse probability weighting should be applied in the case of MAR data patterns [33, 34]. Multiple imputation is a statistical approach employed to deal with missing data by imputing several sets of plausible values for each missing observation. These imputed datasets are then analyzed separately, and the results are combined using specialized methods that account for the variability introduced by the imputation process, providing more robust estimates. Inverse probability weighting is a statistical technique used to adjust for selection bias in observational studies. It involves assigning weights to each observation based on the inverse of the probability of being sampled or included in the study, thereby giving more weight to observations that are less likely to be selected. Both statistical methods offer robust approaches to handle missing data, thereby enhancing the reliability of study findings.
Strengths and limitations
This study is the first to combine EMA datasets from a diverse range of health behavior studies to gain in-depth insight into older adults’ compliance with EMA. By pooling the data, we were able to conduct the analysis on a large dataset, thereby assuring sufficient statistical power to examine intra- and interpersonal variability in older adults’ compliance with EMA. The recommendations formulated in this study will provide valuable guidance for researchers in developing more effective EMA protocols tailored to the unique needs and characteristics of older adult populations.
Important limitations that should be considered when interpreting study findings include firstly, the fact that all four included studies were conducted in Flemish community-dwelling older adults, which may limit the generalizability of the results to other populations of older adults, especially those from different geographical or cultural backgrounds. Replicating similar studies in diverse settings and populations would be crucial to ascertain the external validity of the identified factors influencing compliance. Secondly, the study exclusively focused on mobile EMA studies. While the use of mobile EMA has increased significantly in recent years, the results cannot be transferred to alternative methods, including paper diaries or phone call check-ins, which are still widely used to collect EMA data. Future studies could benefit from a more inclusive approach that combines mobile EMA with alternative methods. However, measuring compliance when using alternative methods is much more challenging. Lastly, the study lacks detailed insights into the reasons behind non-compliance – making it impossible to rule out MNAR. Future studies should collect and analyze qualitative information by means of interviews or focus group discussions, such as in the study of Ziesemer et al. [35], or use short recording of people’s surroundings when they did not comply with an EMA, such as in the study of the study of Sun et al. [36] to gain a deeper understanding of the factors contributing to lower compliance in certain groups or during specific times of the day. This supplementary data would provide actionable insights for refining study designs and tailoring interventions to address the unique needs and challenges of specific subgroups to increase compliance.
Add Comment