Article Text
PDF
Abstract
Background The National Health Institute constitution enshrines the central role of patient and public involvement (PPI) in order to place patients at the heart of the NHS. The sexual health field presents unique challenges for PPI in the tension between current PPI practice versus the need for confidentiality/feelings of shame/stigma. However, there is little evidence around the goals, evaluation measures or theoretical underpinnings of PPI.
Objectives In order to improve current PPI practice in the sexual health field, audits were carried out on PPI plans in both service and research sectors.
Methods 18 local sexual health service contacts completed the audit through snowballing. The tool was refined and five research projects completed the audit from the Health Protection Research Unit in BloodBorne and Sexually Transmitted Infections. Responses were collated and a thematic analysis by two independent researchers carried out. Common areas for improvement were identified.
Results Audit tool responses evidenced wide variability in practice. Issues included conflation of PPI work and qualitative research; limited ‘patient satisfaction’ approaches; lack of PPI goals; methodological reliance on ‘visible’ methods such as focus groups; lack of responsiveness around patient needs and poor resourcing of PPI work. Research specific issues included ‘late’ PPI after key decisions had been made and poor lay summary validity.
Discussion Two audits evidenced a range of areas for improvement of PPI practice in sexual health. Clear definition of ‘what PPI is for’ aligned with evaluation measures would begin to build an evidence base for the contribution of patient voice.
- service delivery
- research
- audit
- ppi
Statistics from Altmetric.com
Key messages
Patient and public involvement (PPI) practice in sexual health was variable and may not cater for the sensitive/stigmatised nature of the patients it seeks to give voice to.
PPI practice struggled to define goals and methods of evaluating success in achieving those.
PPI in sexual health needs to offer a range of involvement options (being mindful of embarrassment/anonymity) and both define and assess goals and outcomes.
Introduction
Patient and public inclusion is increasingly recognised as vital to both healthcare services and health research, but means many different things to different groups. It can be used to define a range of activity under the banner patient and public involvement (PPI), from patient feedback to co-production in which patients play an active role in planning and delivering work. This heterogeneity poses significant challenges for communication, implementation and the quality assurance of PPI initiatives. This article discusses PPI as it applies to sexual health work in the UK, framing the need for improved evidence-based practice against the inherent challenges of patients’ need for privacy in this context, which mitigates against a willingness to share experiences.
Extensive guidance around how to carry out PPI in healthcare is available from INVOLVE,the government funded, national advisory group on public involvement in the NHS. The service standards within the sexual health field repeatedly reference the need for PPI, for example, NHS England,1 BASHH,2 DoH3 and FRSH.4 However, it is also the case that there is little practical guidance on how to translate these national PPI policies and guidelines into methods and processes that are appropriate to sexual and reproductive health (SRH) services ‘on the ground’. Moreover, there is little published in this area, and therefore little evidence of what PPI is taking place or its likely impact/effectiveness. The most useful guidance developed specifically for sexual health, and endorsed by the DoH,3 was the London Sexual Health Programme (SHAPE: Sexual Health5); however this website has been discontinued. Therefore, what good PPI looks like for sexual health services remains unclear.
PPI in research is a UK health research funding requirement.6 It is also embedded in the NHS Constitution7 and is a statutory duty of health services (Health and Social Care Reform Act, 2009), which aims to place the patient at the heart of the National Health Service (NHS). Systematic reviews of the literature have evidenced a lack of conceptual and theoretical underpinning to PPI, and the evidence base for the use and impact of PPI work is limited. Theoretical work around PPI in research is beginning to develop,8 including calls for the use of robust measurement of impact.9 10 However, studies have also found that there is little agreement around what PPI in research is for11 or what theory it uses.12 We evidenced this via our own ad hoc survey of those involved in PPI work. A short survey at an INVOLVE (national advisory group on PPI in the NHS) annual conference8 showed that within (n=33) workshop participants, definitions of the goals of PPI varied, including, listening to patient voice, improving the quality of services, tailoring services, improving patient experience and making work more relevant to the recipients. PPI may be trying to achieve all or none of these outcomes which themselves overlap but assessing PPI ‘success’ depends at least on a common definition of what PPI is for and then measuring this.
This lack of clarity around what PPI is for may make it vulnerable to tokenism or a ‘do it because we have to’ approach. The lack of evaluation of PPI also prevents evidence-based improvement. More worryingly, it could prevent us identifying where it may cause harm. Amplifying patient voiceshould improve services, improve patient experiences and ultimately improve health outcomes.12 However, this requires an evidence base.
In order to address this gap and the unique challenges of securing patient voice within a service where embarrassment/shame mitigate against visible involvement, the authors developed a short audit tool for PPI, structured in such a way as to ensure PPI work is defined in terms of goals and clearly measurable outcomes. The tool was designed to capture current practice (e.g., by identifying current plans or policies), to identify key areas for improvement (e.g., where there are gaps in goal setting, outcome measurement or understanding) and to induce better practice, as respondents were required to articulate their concept of what PPI is for and how success could be measured. Two audits were conducted: one examined service PPI in a local sexual health specialist service, the second audited the research plans across the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Blood Borne and Sexually Transmitted Infections programme of research. The uniquely challenging context of sexual health patient engagement13 within which the tool was developed is the focus of a wider portfolio of research.
This paper both outlines the tool itself and presents key areas requiring improvement from its use within both sexual health service delivery and research.
Methods
Two audits were carried out: one within delivery of sexual health services and one across a large programme of research projects around sexual health. Detailed information on the audit tool and its development is presented later. The audit tool was initially developed from the key concepts of audit, good PPI and data from patients13 (see below), and was refined through the development process to produce the service audit tool presented here.
The service audit tool was electronically circulated among a body representing all the sexual health services within the area (the Sexual Health Improvement for Population and Patients, Health Improvement Team (SHIPP))i. Recruitment took place through snowballing of the SHIPP contacts, and 18 participants filled in the anonymous online audit tool. Reach of the audit was checked against both organisations and levels to ensure a good range of data.12 Returns were collated and a thematic analysis carried out independently by two researchers. Key common strengths and weaknesses were identified across PPI activity. Comparison against best practice was not possible as no such standard exists; however, both researchers were familiar with the range of current PPI and PPI in sexual health guidelines. Key findings were shared with the SHIPP/Health Integration Team (HIT) including sexual health services commissioners.
The research audit was commissioned by the lead of a large and varied research programme (NIHR HPRU) around sexual health, to drive improved PPI. Five of nine individual research projects filled in the audit tool and submitted PPI plans electronically. Audit tool returns were again collated and thematically analysed independently by two researchers for common strengths and weaknesses. Each project received individual feedback around PPI improvements but were also informed of improvement recommendations identified across the programme as a whole (reported here).
PPI in developing the audit tool
PPI informed the development of the audit tool itself through survey work with young people around their experience of PPI across sexual health services. Theresults and method of this study are published elsewhere.13 The work was also shaped by the HIT which represented recipients of the audit (all organisations working in sexual health across Bristol). Furthermore, the authors are working on a ‘story-based PPI’ approach to capture the definitions of how a variety of vulnerable patients across sexual health services (Sexual Assault Referral Centre, abortion services, etc.) want to contribute to PPI and how they define success criteria.
The PPI Audit Tool*: questions that improve practice
A series of 12 questions, examining three areas, was developed to build users' understanding of what they were trying to achieve with their PPI work, as well as some assessment of the organisational resourcing of the PPI work itself. The questions were developed in the first instance to capture the range of PPI work taking place within different sexual health organisations within the SHIPP area and drew from existing guidelines on what PPI should look like from INVOLVE,10 as well as previous research work carried out to explore the particular needs of users accessing sexual health services.12
The structure of the tool aimed to ground respondent’s understanding of their PPI work in the goal they were trying to reach (Question 1) and use this to encourage them to not only define but measure that success (Question 8). At the same time the questions needed to capture the full range of activities that might be seen as PPI and focus on the building blocks of good PPI planning10 such as resourcing, policies, named roles, etc.
Identifying PPI work
What is PPI for?
How could the voice of patients or the public change the work you are undertaking?
What is your current activity to involve service users and potential service users?
What plans or policies do you have for PPI work, can you submit with this audit?
Effectiveness of PPI work
What is the goal of your PPI?
Who is involved (how diverse) in PPI work at what stage, in order to achieve what?
How will PPI shape your service/research?
From the goal already identified can you select outcomes or measures that demonstrate the impact of the PPI work?
Resourcing PPI work
Who looks after PPI in your organisation?
What is their role?
What resources do you have for PPI work (finance, time, patient groups/collaborators)
Do you have the resources you need? What more do you need?
*The authors request citation of this paper if the tool is used or adapted.
The common issues and areas for improvement identified across services and research projects are reported below.
Results
Service delivery audit
Findings from the use of the audit tool within a regional delivery of specialist sexual health services are reported elsewhere.13 However, the key themes that emerged from the input of n=18 individuals, representing all levels of service (Clinical Commissioning Group (CCG) commissioners sexual health services themselves including primary care, clinics and charitable providers), fall into six areas (in bold):
A varied programme of some interesting and innovative PPI work was evidenced. The audit tool identified areas for improvement including wide variability around PPI practice, with many reporting no PPI policy and others submitting advanced patient centred work. In terms of the key aim of the tool itself, encouraging clear identification of PPI goals, many had not conceptualised what PPI was for, often conflating PPI work with qualitative research involving patients. This meant there was no way of measuring how successful work had been in reaching any goals. Change in the logistics of service delivery (appointment times, availability of information, etc.) was often cited as a key outcome, which reflects a minimal ‘patient satisfaction approach’ to PPI. Methods also demonstrated a lack of responsiveness to patient groups/confidentiality concerns, for example, by requiring that all feedback from patients happen in group-based, face-to-face settings that are difficult to recruit to. And, there was some reporting of innovation being undermined by standardised ‘NHS’ PPI systems. Finally, the audit tool also highlighted a lack of training as a key barrier to PPI, although 8 out of 10 services reported offering PPI training to staff.
The audit tool successfully identified areas that required improvement and, in reporting back results across the SHIPP HIT group, enabled areas of the service to benchmark their PPI work against others and begin to improve this aspect of their work.
Research programme audit
The use of the tool to audit the NIHR HPRU in Blood Borne and Sexually Transmitted Infections large programme (three themes, nine projects) of research work within the sexual health field enabled us to identify some key overlapping issues around PPI but also some key themes that were specific to research, as opposed to service delivery. Five of the nine projects returned audit data.
Overall, evidence was submitted of valuable PPI work being done, but again there was some variability in approaches to conceptualising PPI captured by the audit tool. They included ideas such as quality of the research, democracy, ethics, understanding the context of the research, engaging people, relevance and transparency. So, again, there was wide variability and, in some instances, PPI was conflated with qualitative research (double badging original qualitative research undertaken for the project as patient involvement work). Projects had difficulty identifying their PPI goals, and there was little evidence of any measurement of whether goals or outcomes had been reached. Resourcing of PPI was seen to exist only at a project level which, within a larger programme of work, suggested that wider organisational roles, training and policies could be put in place to support front-line PPI work. Information around specific funding for PPI was often vague and suggests that clear financial support for PPI work could be better defined. Methodological approaches relied heavily on the use of face-to-face groups, which within the specific context of sexual health makes recruitment very difficult, as stigmatised patients often request confidentially and anonymity.11 13 The proposed timing of PPI planning was often after key decisions of focus and methods had been takenby researchers, and therefore, its scope was limited to elements such as the tailoring of materials. PPI work should be planned from the outset of a project and allow patient input into the research question and methodology. Finally, the quality or true accessibility of lay summaries was clearly identified as an issue.
Findings were fed back to the organisations concerned.
Discussion
The audit tool led to the identification of key areas of improvement that in many areas reflected findings from an existing systematic review,9 while at the same time signalling areas of good practice within PPI. The audit work demonstrated a need to push PPI initiatives to be clear about where on the continuum of involvement their work sits: patient and public voice at the heart of both service and research design (ideal), to projects that simply require tailoring of communications or logistic service delivery (often the reality). The full scope is required, and the goals of each would be very different, so defining these at the outset of work is both logical and desirable.
The need for greater sensitivity and confidentiality required by patients in the sexual health context13 seems largely unmet, primarily because of the reliance on group-based, face-to-face work across both service delivery and research contexts. This suggests that the SRH community should embrace a wider range of methods around patient engagement as a way to cater for the sensitivity and anonymity required by the field. Intrinsic to this is the need for greater diversity in the types of voices heard by research and services in sexual health. Indeed, good PPI could be fundamental to any attempts to build trust with seldom heard groups, that is, by asking patients and the public how they would like to contribute and showing them clearly how their contribution has been used. However, this is not only the case for sexual health: the principle of greater variability in method, tailored to patient expectations and experiences, can be seen as key to successful engagement with PPI across a range of different health fields.
There was little evidence of any measurement or quality control in any of the PPI work that was audited here. Our recommendation that this area is developed not only to better evidence the effect on patients giving their time, but also to scope and maximise this benefit while minimising potentially harmful, tokenistic approaches. It is our ethical duty to report back to those giving their time and above all show them what their contribution has truly achieved. Measurement should be theory-based, using a concept of theory of change as used in evaluation. Theory of change approaches, explained briefly, define the pathway or causal linkages from activity to desired change.12 Using such approaches will challenge projects to articulate their PPI assumptions, but also to ensure that PPI goals are clear and make sense within a given context and problem. It will also allow researchers and services to make clear links between activities and outcomes, and to know if, when and how PPI has benefited those involved.
Footnotes
↵i http://www.bristolhealthpartners.org.uk/health-integration-teams/sexual-health-improvement-hit/
Contributors The work was carried out by JM and DG on behalf of the SHIPP/HIT (http://www.bristolhealthpartners.org.uk/health-integration-teams/sexual-health-improvement-hit/) and the NIHR HPRU.
Funding The work was funded by the SHIPP/HIT and the NIHR/HPRU.
Competing interests None declared.
Patient consent Not required.
Ethics approval University of the West of England, Faculty Research and Ethics Committee.
Provenance and peer review Not commissioned; externally peer reviewed.
Collaborators The National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Blood Borne and Sexually Transmitted Infections at UCL in partnership with Public Health England (PHE) and in collaboration with the London School of Hygiene & Tropical Medicine, London, UK.
Linked Articles
- Editorial
- Highlights from this issue