Selection of sources of evidence
The search and screening results at each stage are shown as a PRISMA flow chart in Fig. 1. We screened unique 1124 titles and abstracts and 113 full-text documents. Of those, 59 original research papers met the eligibility criteria. The full list of included studies can be found in Additional file 2, and the record of excluded full-text studies can be found in Additional file 3.
Characteristics of sources of evidence
Eight out of 59 included studies used quantitative methodologies, 33 qualitative, and 18 mixed methods. Twenty-four were conducted in the USA, 7 in the Netherlands, 6 in Sweden, and 4 in Canada, Australia, and England. The remaining 10 studies were conducted in other countries. The year of publication ranged from 1998 to 2022 (median = 2017). The study aims reflected high variability in the scope of investigations, ranging from those interested in implementation processes and evaluation, through stakeholder perspectives and experiences, to comparative and regional analysis. Descriptive data about the included studies are presented in Table 1.
Table 2 displays the characteristics of evidence sources, presenting the frequency of observations and the sources by thematic categories. Table 2 also shows the frequencies of observations for each determinant cross-tabulated by each implementation outcome. Implementation stood out as the dominant outcome, covered in 46 articles, while adoption and penetration/reach were discussed in 29 and 26 of the articles, respectively. Sustainment found the least attention (n = 16). The data prominently showcased local determinants, including work infrastructure (n = 34), mission alignment (n = 33), and culture (n = 29), emphasizing their significance in the adoption and implementation phases. Of these, mission alignment was highly prevalent in sustainment articles. The concept of agency leaders was discussed in 38 articles, and their role was highly present in adoption, implementation, and sustainment articles. Researchers (n = 11) and political/administrative decision-makers (n = 25) were most frequently cited in articles concerning sustainment and penetration/reach. They had in common their frequent association with national strategies (n = 25), while the former was more often associated with evaluation, monitoring, and feedback (n = 25) and the latter with financing (n = 41). Discussion on sustainment and penetration/reach also frequently associated with national strategies, legislative context (n = 31), and financing. External support professionals (n = 20) were relatively highly represented in the articles on sustainment. Figure 2 is a diagram that depicts the relative positions of categories and directions of influence between them in a conceptual model.
Associations between the systems of evidence-to-practice, external contextual determinants, and implementation outcomes: a conceptual model
External contextual determinants
Policies and laws: national strategies and systemic integration
National or regional strategies were described as promoting the uptake and implementation of IPS [38] and appeared to be backed by administrative decisions about responsibility-sharing or funding. These policies included national mental health strategies [39], guidelines [40], and agreements on implementation support issues [18, 19]. The IPS model was perceived as a contributor to the strategic goal of implementing the recovery approach and serving as a vehicle for producing system reform at national and regional levels [18]. Congruence with other national policy goals and frameworks, such as social inclusion [39] and participation [41], was found to facilitate the incorporation of IPS principles into national mental health care policies. Systematic approaches in providing implementation support could support national strategies [42] whereas a mismatch between overarching national strategies and a lack of programs to implement IPS to achieve the goals of these strategies was reported to lead to lower penetration or adaptation of the IPS model [39, 43, 44].
One feature of the national strategies was the aim of expanding the clientele from persons with severe mental disorders such as psychotic disorders to those with any mental disorder, leading to implementing IPS in various care settings, e.g., forensic or psychiatric housing programs [45, 46]. The implications of different work infrastructures on implementation are discussed in a separate section below (work infrastructure).
Policies and laws: legislative context
Legislative contexts concerning mental health and employment were reported to impact the implementation of the IPS model. Laws that mandate employment services for individuals with severe mental illness [47] or policies redirecting services from activities not following the IPS model [48] increased the adoption of IPS programs. On the other hand, the availability of competing practices [49,50,51], procedures mandated by policies but not supported by research, such as work capacity assessments [52,53,54] or mandated lengthy referral processes [54], were reportedly at odds with the implementation of IPS with adherence to model guidelines. Social insurance criteria that excluded clients based on expected employment outcomes [55] or received benefit types [56] were also reported as barriers. The policy of allocating decision-making and management of services to local authorities was reported to hinder adoption due to low prioritization at the local level [41]. Laws and regulations related to sharing client information and access to data and mandated use of multiple information systems were reported to complicate the implementation of IPS [44, 57, 58]. Legally mandated limitations on using data could be circumvented by strategic actions by the administrative authorities or local leaders [18, 44].
Financing
The availability of funding was critical for adopting and implementing IPS across the settings. National or regional development projects were often used in the adoption phase [59, 60]. Sustained direct funding schemes through health ministries or other governmental organizations were used to increase the use or adoption within the service system or provide the necessary flexibility to implement the model as intended at the local level [38, 48, 60]. A state-level funding mechanism was associated with statewide uptake of the model [49]. Payments based on achieved results were reported to facilitate sustained implementation [49, 61]. Many studies reported that a well-managed transition from projects to sustained programs was a critical period.
Specific funding mechanisms were reported as barriers to the successful implementation of IPS. Payment models that were based on specific medical diagnoses rather than outcomes [62, 63] and separate or divided sources of funding [18, 41, 42, 51] hindered the implementation. Set or predefined funding duration to funding [41, 46, 57, 59, 64], restrictions on financing employment services as health services [51, 65], and rules that penalize short employment contracts [51] were also perceived to impact the quality of implementation negatively. Funding contracts covering a broader set of programs could include criteria conflicting with the IPS fidelity criteria [50, 63].
Training and technical assistance
Training and technical assistance were reported to facilitate the implementation of IPS. Sources for training and assistance included support from national, state, and regional organizations [66, 67] and IPS/EBP development projects [42, 68], as well as openly available guidelines and training material provided by the program’s developers [55]. These supports reportedly helped those putting the model into practice with goal setting and providing a sense of purpose [44, 50], helped providers to work systematically according to protocols and improved their knowledge of evidence-based practices [44, 68], and provided opportunities to share knowledge and experiences with other sites [55]. Agency leaders [44] and staff [62, 69] were reported to benefit from initial training and assistance [65].
Evaluation, monitoring, and feedback
National, state, and regional organizations [19, 42, 67] and outside experts were used to conduct evaluations and monitoring of the implementation of IPS that were often reported in conjunction with training and technical assistance. Routinely assessing implementation was perceived to help ensure that the model is implemented as intended over time [62], and imposing continuous evaluation by agency leaders may increase the probability of the sustainment of the program [70]. In some cases, fidelity above a certain threshold was used as a prerequisite for funding by national or regional decision-making organizations [71]. Disseminating the results on the effectiveness of IPS reportedly increased the model’s adoption [18], and evaluations and monitoring were used to motivate leaders to maintain or reinstate high-fidelity services [40]. In contrast, the lack of results from monitoring or evaluations could discourage agency leaders from following national guidelines that promoted the use of IPS [46].
Local factors affected by external context
Mission alignment
Both the recovery approach [41, 59, 63, 72] and evidence-based policy commitment [69, 73] were reported to facilitate the reorienting of organizational goals to be consistent with IPS implementation and sustainment. The shift in organizational goals was associated with the de-implementation of vocational services that lacked evidence-based support and were supported by structural changes and financial arrangements through administrative decisions [74].
The non-alignment with organizational goals was reported to hinder the model’s implementation. The model could be at odds with existing organizational goals based on traditional medical or vocational services [75, 76]. These goals could be mandated by existing rules and regulations [53]. Challenges were reported when collaborating partners from different organizations had different goals in their respective organizations [52, 53, 60, 77], which could lead to giving lower priority to collaborating with the IPS team [52, 70].
Culture
Acceptance of the model by the professionals, professional norms, and local attitudes was reported as important for the uptake and implementation of the model. Understanding the program logic [60, 73] and recognizing unmet user needs [72, 78] were associated with the changes in acceptance of the model and professional norms. Several studies found that influencing practitioners’ professional norms and attitudes was an important goal during the adoption period. During this time, practitioners could learn about the rights and needs of users, the benefits of IPS, and community resources; changes in these attitudes would lead to better implementation results [55, 58, 72, 79]. Receiving training and support from site managers and national organizations [65], experiencing bringing together service functions as intended, and sharing success stories [58] were perceived to facilitate the implementation and sustainment of the model.
In several studies, the practitioners were reported to view IPS as conflicting with the core beliefs or principles of care. The practitioners may see employment or financial self-sufficiency as a not crucial outcome for health services [49, 70, 77], or they may see IPS as an inferior or unnecessary service [46, 63, 77]. Negative attitudes about the capabilities of the target group could lead to lower referrals to IPS [41, 44, 57, 65, 80], referrals to employment services not supported by research [48, 55, 57], exclusion from the service [55, 80], inadequately bringing together service functions [48, 65], and poor collaboration with external partners [53, 63].
Work infrastructure
IPS was implemented within mental health services, outside of mental health services, or as a collaboration between different organizations. Programs in community mental health settings rated higher fidelity than those in rehabilitation centers, housing units, or independent programs [45, 81, 82]. Providing the service in a mental health care setting was reported to lead to higher and shorter referral processes. Transforming a work setting to a high-fidelity IPS service was reported to require creating or protecting designated or reserved staff roles, adjusting the number of clients assigned to a single professional, or renegotiating the existing job descriptions [62, 69, 74]. The infrastructure related to continuous support was reported to promote the model’s sustainment [70].
In the situations where multiple organizations implemented the model together, strategies and agreements on financial matters [54, 83], identifying shared clients [83], and practical arrangements such as office space [83] and designated contact persons [64] were reported to facilitate implementation. The willingness to share expertise and the complementary experiences of different stakeholders [46] can also help with implementation. On the other hand, organizations that are expected to collaborate may resort to conflicting service processes [52, 53, 56, 60]. In situations where multiple organizations worked together, the absence of formal agreements led to poor referrals [54] and hindered effective implementation [41].
Systems of evidence-to-practice
Researchers
Researchers’ active involvement in developing and implementing strategies for disseminating the IPS model included collaborating directly with political and administrative decision-making, national and regional support organizations, and the implementing agencies. In the USA, the promotion of the decision-makers’ participation in the learning community was found to encourage interagency collaboration at the state level [47], including arrangements for state-level funding [66, 84] and evaluation and training support [47, 84], resulting in a higher number of IPS programs per state population and faster growth in penetration [84]. In Australia, national-level advocacy included a group of researchers promoting the IPS model to state and federal politicians and government department administrators, leading to decisions related to funding and development projects [40].
The US Learning Collaborative, a researcher-led initiative for disseminating the IPS model, has also produced numerous research collaborations supporting the model’s spread across the settings [19]. These collaborations were found in the form of partnering with the developers of the model to produce new evidence or support implementation [18, 40, 51, 59], training experts at the national level [40], or collaborating with those putting the model into practice directly [48, 51].
Political/administrative decision-makers
The model’s penetration was facilitated by decisions by the state politicians and administration [18, 40, 48, 55] or local political decision-makers [77, 85]. The dedication and enthusiasm of actors at the administration level were reported to facilitate the necessary collaborations [41, 48, 49, 62]. Enthusiastic state IPS coordinators and administrative authorities were reported to foster a culture shift in agencies, leading to high-fidelity implementation and sustained model use [49, 71].
Political/administrative decision-making was reported to induce changes in national policies. Recurrent funding decisions [48] and funding designated for IPS were reported as a facilitator of local implementation and service system penetration [49]. Also, decisions to change policy regulations and protocols, rules for referring to services, and providing support resources for implementing the model were reported to support the system-level adoption of the model [18, 49]. Enforcing national strategies and guidelines was reported to stipulate political or administrative decision-making at the local level [38]. Administrative collaboration, including coordination, consensus-building, or formal agreements between responsible agencies [18, 51, 54] or professional networks [83], reportedly facilitated coordinated referral processes and joint data collection efforts, resulting in higher penetration [49, 50] and quality of services at the aggregate level [18].
Administrators’ commitment to models not supported by research [49] and the lack of state-level collaboration between administrators in different agencies were associated with non-aligned strategies for employment services for the target group [47]. Studies also reported the ambiguity that decision-makers face when facing different potential service models [41, 70] and when considering increasing the penetration outside the specialized mental health care system [41]. One study reported ambiguity in that the strategies might recognize the significance of enhancing employment rates for individuals with mental disorders but consistent implementation plans were lacking [39]. In addition, administrative hesitancy was linked to the lack of power in decision-making [70, 83].
External support professionals
National or regional supporting organizations were reported to promote collaboration, funding, training, and evaluation. Their form varied from organizations created to support individual IPS projects [18, 19, 46, 67] to quality improvement collaborations involving several EBPs [42, 68] and contracting support services from other sites that implement the service [18]. These collaborations often included partnerships with and resources from university researchers [18, 66]. Implementing these supports could be a feature of a dissemination plan [49], and the number of active IPS programs was associated with the number of national trainers [19].
Support organizations could help the implementing sites to create implementation strategies [65, 83], budget plans involving one or more agencies [62], and encourage agency leaders to proceed with the implementation in problematic situations [83]. Training, technical assistance, fidelity, and outcome monitoring were often reported as critical aspects of implementation support [18, 46, 49, 83, 86]. Evaluation and monitoring data were reported to have been used to increase accountability and motivate decision-makers to increase funding [18, 49]. Centralized enforcement of adherence to model guidelines and outcome monitoring was found to improve the quality of implementation over time across sites [49]. Fidelity and outcome monitoring also reportedly facilitated both national consensus-building and supervision based on achieved results at the local level [18].
Poor or lack of national implementation support was reported to lead to fewer links and communications between academics and implementing agencies and low leadership involvement [44]. Removal of regional leadership and a decline in national/regional training and evaluation supports were found to lead to lower quality implementation of once-sustained programs [40]. Short timeframes for national development projects that provided external support for local sites were associated with challenges in achieving organizational structural changes in the service-producing organizations [46, 68].
Agency leaders
Senior leaders, often motivated by the recovery approach and the evidence base [48, 59, 63], were the actors who promoted ‘systemic transformation’ [51] and placed the IPS within a broader area of strategy for psychosocial services provided by the care organization [72]. Committed senior leaders communicated the importance of the recovery approach and services tailored to each person’s specific needs, which was reported to lead to higher quality services [62, 71, 87]. Prioritizing and enforcing strategies and actions, often using the steering group, was decisive as it affected several aspects of the effort, including the affecting organizational policy, promoting the program’s credibility among the professionals, the methods for cooperation, and the financing decisions [51, 55, 62, 63, 70, 71, 83]. Senior leaders’ commitment to the guidelines to ensure the IPS program is being implemented as intended [40, 54] and enforcing fidelity monitoring [40, 70] were reported to facilitate sustained implementation. In the situations where multiple organizations worked together, combining leadership outside the provider organization was perceived beneficial for implementation [59].
Agency senior leaders’ failure to align the IPS model principles with organizational goals and inadequate agency prioritization [44, 65, 71, 77] led to poorer implementation or non-sustainment. Lack of enthusiasm and promotion of the model [70, 71], not being able to channel funding [65, 77], and not using performance-related indicators [46] can also hinder its implementation or sustainment. Failure of steering groups to commit or their dissolution after the project period was reported to cause a cessation of funding or poor coordination with external partners [56, 77].
Add Comment