Scientific Papers

The INSIGHT project: reflections on the co-production of a quality recognition programme to showcase excellence in public involvement in health and care research | Research Involvement and Engagement


We believe that the Insight | Public Involvement Quality Recognition and Awards Programme represents the first co-produced programme that recognises, celebrates and shares how organisations and individuals involve the public in design and delivery of health and care research, using a strengths-based, appreciative inquiry approach. It is based on a collaboration between NHS, academia and community sectors, as well as strong public contributor representation. It also acted as an exemplar for the type of working together that the programme aimed to inspire in its participants.

Other public involvement recognition schemes and quality frameworks

There are a large number of frameworks aimed at supporting, evaluating and improving the quality of public involvement though, in their review of these frameworks, Greenhalgh et al. [4] concluded that ‘most published frameworks have been little used beyond the groups that developed them’. They offer suggestions on how to develop a suitable framework and recommend working with ‘patient collaborators’ to ‘plan and deliver a series of co‐design workshops to generate a locally relevant and locally owned framework’. We broadly adopted this co-development approach with the intention of creating a programme that would have wider applicability and provide a mechanism for facilitating greater interaction between involvement practitioners.

We acknowledge that our programme is not the only initiative that offers awards for PPI activity. Over the last decade, regional NIHR Clinical Research Networks (CRN), including the West Midlands CRN, have had annual awards that recognises high quality PPI activity [17, 18] while the NIHR School for Primary Care have also offered prizes for public involvement [19]. These generally focus on specific PPI initiatives by small teams, rather than at organisational level. Our programme has a much wider reach and more systematic approach, with a drive to improve best practice, and facilitate its spread across healthcare and social research-active organisations.

The Engage Watermark is an award granted by the National Co-ordinating Centre for Public Engagement to higher education institutions to recognise their strategic support for, and commitment to improve, public engagement [20]. It is aimed at institution/faculty level, also has 4 levels (bronze, silver, gold, platinum) and provides a set of recommendations for improvement. Its reach extends beyond healthcare research but is limited to the higher education sector. It appears to use more of an audit and benchmarking approach and therefore feels more of an accreditation programme than the quality recognition approach that our use of appreciative inquiry brings. Importantly, its focus is on public engagement (attracting public interest in the research an organisation is doing) rather than involvement (including public contributors in the design and delivery of healthcare research). The Watermark programme therefore has some similarities to our programme and there may be mileage in drawing from this expertise.

Some universities have their own awards that recognise excellence in PPI activity [21, 22], though these are only open to the organisation’s staff members. At a project level, the European Research Council provide a Public Engagement with Research Award [23], designed to recognize and celebrate European Research Council grantees who have demonstrated excellence in public engagement and outreach. They offer monetary prizes (€10,000) for each of three awards—Involve (citizen science), Inspire (public outreach), Influence (media and policy). Winners are invited to a European Research Council-affiliated conference and are given increased media coverage. While this programme also provides some useful concepts that may be valuable in the future development of our programme, its project focus and restriction to grant awardees limits its current usefulness.

As well as quality awards schemes, several other PPI quality improvement tools exist. For example, the Guidance for Reporting Involvement of Patients and the Public (GRIPP) provides a framework for reporting study-level capture of PPI activities [36]. This is used extensively and extremely valuable for outlining PPI activities for projects and its embedding in the process for publication for journal such as BMC Research Involvement and Engagement encourages improvement in PPI quality.

The CUBE framework, was developed as a tool to help with planning and evaluating the process of PPI in research [40, 41]. It examines the quality of PPI across four domains; the strength of the public voice, diversity in ways for public contributors to be involved, the degree of attention to public concerns and organisational attitude to change. Hence, there is overlap across some of the UKSPI standards and the workshop approach suggested CUBE would be a useful way to review and evidence the quality of PPI activity within an organisation as part of an INSIGHT self-assessment. In terms of the UKPSI ‘impact’ standard, the Public Involvement Impact Assessment Framework (PiiAF) represents an excellent tool to record and is ‘primarily aimed at researchers who wish to design an assessment of the impact of public involvement in their research’ [41,42,43]. We propose that this would be a valuable tool to evaluate the effectiveness of INSIGHT as well as providing a means for collecting impact evidence for organisations who wish to participate in the INSIGHT programme. More recently, the Public Involvement in Research Impact Toolkit (PIRIT) has been co-developed by public contributors and staff members for use at the Marie Curie Research Centre and the Wales Cancer Research Centre [44]. It comprises tools to support researchers, particularly in public involvement planning and impact reporting at a project level. It aligns to the UKSPI and provides an excellent checklist to ensure public contributors are involved at every stage of a project, whilst also capturing their contribution and impact on how the project was developed and delivered.

While these schemes provide significant complementarity to INSIGHT and would represent means to collect organisational evidence of high-quality PPI for submission to INSIGHT, most are not aligned directly to the UKSPI and do not cover the breadth of PPI activity encompassed by the UKSPI. Furthermore, they do not celebrate or facilitate spread of PPI excellence, and are more focused on the benefits to an individual organisation or research team.

Programme structure

As mentioned in the Introduction, the UKSPI shied away from the introduction of a ‘rating’ system alongside their six standards. We too wanted to avoid anything that hinted at an accreditation scheme or audit process as this was felt to be a disincentive for participants. However, the use of quality levels in the context of the Expert Citizens Insight Evaluation programme [25], underpinned with an appreciative inquiry, strengths-based approach, was felt to be a workable model as a positive means of identifying existing good practice and support continuous improvement. Our public contributors embraced this approach, and the pilot sites appreciated the support in facilitating ongoing improvement that this approach gave them. The ability to quote the achieved level in grant applications and on organisational marketing material was felt to be a positive incentivisation for participation by the public contributors and pilot sites.

Benefits afforded by the INSIGHT programme

As summarised above, we believe that our programme brings together key elements of existing initiatives to create a programme that has several important, often unique, benefits:

Flexibility

Given the generic approach we have used, it was felt that INSIGHT could be suitable for national or even international application. By incorporating within the assessment process a mechanism to account for organisation size and resources it is flexible enough to be applicable to any type of organisation (i.e. any involved in health and social care research; including commercial research organisations such as pharmaceutical companies, contract research organisations and medical equipment manufacturers, as well as those small third sector groups or charities that are frequently overlooked). Furthermore, by combining both the INSIGHT assessment scheme for organisations and departments alongside the INSIGHT quality awards event for individuals and small teams, it accommodates any PPI-active groupings from individuals to whole organisations. This would address some of the limitations of existing programmes as described above [17,18,19,20,21,22,23].

Independent, lived experience-led assessment

It provides an independent assessment of public involvement activity, co-led by public contributors, which would assist in validating statements made on grant applications, websites, etc., regarding public involvement activity within an organisation. While healthcare research funders and governance bodies such as research ethics and the UK Health Research Authority increasingly require evidence of public involvement at every stage of the grant development and delivery [10,11,12,13], independent assessment of this is sometimes challenging [9].

Profile

The celebratory nature of the national annual Awards Event provides the opportunity to raise the profile of public involvement activity. It also provides opportunity to demonstrate the importance and celebrate the impact of PPI activities. While some current public involvement activities are publicised, their reach is often more locality-/region-focused [17, 18, 21, 22], or project-based [23].

Spread and Improvement

The Awards Event (particularly the use of a special award for spreading improvement), along with our plans for a repository of involvement ideas, training and toolkits, facilitate the spread of public involvement activities across health and social care research organisations. Such a repository could supplement and align with the NIHR CED’s Learning for Involvement website [45]. We are not aware of such a central, national resource of approaches and ideas for improving PPI, though it is believed that there are plans for the NIHR to bring all their PPI information into a single resource centre (CED—personal communication). Furthermore, the programme aims to expand the number of public contributors; directly by encouraging new public assessors to be part of the assessment process, and indirectly by encouraging participating commercial and non-commercial organisations to improve their PPI activity. It also supports the drive for improvement in the level and quality of public involvement at individual to organisational level and creates an incentive to develop strategic approaches to this activity. While other organisations, including the NIHR CED and CRN PPI networks [14, 17, 18], exist to facilitate spread and improvement in public involvement activity, our aim is to work with these groups to expand this further using a systematic approach based on a common set of quality indicators and levels.

Incentivisation

We feel that our focus on a ‘carrot’ (pull) rather than ‘stick’ (push) approach to improving public involvement, by use of an appreciative inquiry approach with celebration of best practice rather than an audit-based programme, is more likely to incentivise participation in the scheme. While programmes such as the Engage Watermark [20] has significant potential for expansion from engagement into involvement, and across to healthcare organisations, we felt that its audit style approach may disincentivise participation and may go against the principles behind why the UKSPI were developed [15].

Innovation

Our programme encourages innovation in public involvement activities and this is further recognised by the special award for innovation as part of our Quality Awards Event. In the context of the ever-changing technological landscape and use of social media, not to forget the innovations brought about by the COVID-19 pandemic, our aim was to encourage new ways of expanding public involvement.

Equality, diversity and inclusion

Equality, diversity and inclusion represents a key priority for the NIHR [46], including in public involvement activity [47]. Our programme aims to address the general lack of diversity in public contributors, including by encouraging equality, diversity and inclusion in membership of assessment panels and by the use of a special Encouraging Diversity Award.

Impact

Our overall aim is to improve the quality and relevance of health and social care research and we firmly believe that greater public involvement is a key component of this. Numerous studies have indicated the potential ways in which public involvement may improve clinical research, though how these can be accurately evaluated remains a topic of debate [12, 48]. Public involvement is not only about impact on the research itself, but may also encompass benefits such as empowerment of public contributors and improving the wider relationship between the public and researchers [4], as well as making health and social care research more accessible to the public [6]. Some have suggested that the potential negative impact of public involvement has been little studied [48], though while such balancing measures should indeed be included in evaluation of impact, it is difficult to imagine that harms would outweigh the benefits. For example, our public contributors have described how their health literacy improved through involvement in PPI activities.

Co-production

As noted above, Greenhalgh et al. [4] identified 65 frameworks related to co-production in research. These frameworks generally focused on their use within individual projects and targeted specific aspects such as power-sharing, participant recruitment, research priority-setting, report writing or partnership development.

While we did not adopt a specific framework, we aimed to incorporate the key principles of co-production covered by these frameworks in the development of our programme, together with an equal power partnership in its delivery, in order to ensure that the public voice is genuinely represented in a scheme focusing on such a topic. It became clear during the task and finish groups that public contributors ask questions and generate ideas that would not readily be apparent to academics. Wicks et al. [11] stated that ‘One of the main stumbling blocks to “coproduction” of research with patients and the public is that professionals lack knowledge, skills, and experience on how best to do it.’ While the NIHR have published guidelines on co-production [35], our experience from this study illustrated that clinical and academic interpretations of this term vary considerably and that true co-production is rare. Indeed, one benefit from involving Expert Citizens as a core partner was that they focus heavily on co-production and have delivered workshops on the subject from the point of view of people with lived experience [49], thereby reversing the usual balance of ‘professional teaching the public’ in this regard.

In terms of the project itself, we brought together ‘professionals’ with public contributors from very different backgrounds in the TFGs. Given that co-production, with its bringing together of people with different and frequently strongly held views, can itself be challenging at times [24, 35], we expected some challenging TFG sessions I this regard. However, while the breadth of expertise and experience inevitably generated a diverse range of views, consensus was overall very easy to reach and we did not encounter any instances on overt or intractable disagreement.

Reflection and learning

In addition to gaining feedback on the materials we developed throughout the project, and making revisions accordingly, it was important to capture the learning and reflections from public and professional participants. On the whole, comments from public contributors suggested a very positive experience and enthusiasm for the project. There was an overall sense of genuine co-production with the freedom to express views and see those views initiate change. However, we learned that it was important to clarify roles and take time to explain the background and aims of the project at the beginning, particularly as it was recognised that these aims were ambitious. Similar to previous observations [50], adapting to the needs of individual public contributors was important and we identified that smaller group sessions outside the main TFGs was extremely valuable for some participants. In a systematic review of patient engagement in research, Domecq et al. [1] also identified that spending adequate time to build reciprocal relationships between public contributors and researchers, fostering mutual respect and being clear on what is expected of public contributors were seen as important.

Our initial plan was for the TFGs to operate as face-to-face meetings. However, the COVID-19 pandemic-associated restrictions prevented that and all groups were conducted using an online platform. This was disappointing in respect to the reduced ability to generate the inter-personal interactions and bonding between group members. However, notwithstanding a few technical glitches at times, all participants adapted to the online format, and we were able to hold a larger number of sessions due to reduced costs. As also experienced by the Blueprint Writing Collective [50], a hybrid approach has become a more common feature in the Impact Accelerator Unit at Keele University (the Unit which leads on PPI within the University’s Medical School), though we are mindful of the risks of digital exclusion.

We also identified that working alongside a third sector organisation comprising people with lived experience (Expert Citizens) and who had created the original Insight framework, provided a bridge between the health-associated professionals/academics, and those public contributors who were unfamiliar with the Insight concept. This use of third sector organisations to facilitate meaningful public involvement is not new [50] and can facilitate building bridges that otherwise might be challenging to academics working independently.

The professional members of the team also recognised their own learning from interacting with an array of public contributors with diverse experiences and expertise. In our project, the academic team aimed to use a co-production approach from the start. However, it can be challenging to adhere to ‘… principles of respect, trust, reciprocity, and co-learning…’ (as described by Kirwan et al.as one of their core guidelines for patient engagement [51]) as these take significant time and commitment to adopt by researchers who are used to driving the research agenda themselves. During the project, the immense value of the experience and expertise provided by public contributors became very clear, making these guidelines much easier to follow. The power of the public voice may have been assisted by the fact that we elected to involve a large number of public contributors (almost half of TFG members).

Challenges and next steps

We recognise that this programme is in its infancy and requires significant further evaluation and refinement, as well as continuing support from key stakeholders such as the NIHR, before it becomes a fully-fledged programme. To this end, we have been encouraged by the CRN West Midlands’ support for the project to date and by their continued commitment to support a regional roll-out and evaluation of the Insight | Public Involvement programme across NHS partners within the region. The evaluation will present challenges, as illustrated by Russell et al. [48] and Boivin et al. [12], and will need to include an appropriate a mix of outcome, process and balancing measures using a mixed methods approach. This, along with a realist evaluation INSIGHT to determine how it could be adapted for different health and social care settings, forms part of the next steps of the programme’s development.

We also acknowledge that a sustainable business model needed to be developed, along with the associated marketing strategy and allied commercial considerations. These aspects formed part of phase 2 of the programme’s development, which included independent market research and collaboration with the Chamber of Commerce to develop the business model. This indicates that the programme is sustainably over the long term.

While the programme was developed so that it can be applied across most settings where health and social care research is carried out, including the private sector, the pilot work has focused on the UK NHS and academia. It may require further refinement to be applicable more widely (e.g. commercial sector, social care, third sector organisations) and beyond the UK. However, we believe that the core structure is sufficiently flexible and resilient so as to not require wholesale changes.



Source link