PPI Monitoring to Learn

PPI Monitoring to Learn process and guidance
Please note:
- This guidance currently uses the term “Theme PPI Lead” to refer to public contributors linked to Themes.
- The document is also written based on the NIHR ARC TV structure.
- Terminology will need to be amended for other PPI infrastructure.
- This document is primarily written for ARC TV and therefore suggests a way of doing PPI Monitoring to Learn. It is not intended to be directive or prescriptive and should be adapted for the context in which it is being implemented.
Aims
The PPI Monitoring to Learn approach is intended to:
- Encourage planned and meaningful PPI involvement across all ARC-funded (or closely ARC-connected) research.
- Develop a clearer understanding of the PPI activity taking place within ARC research, including challenges, adaptations, and emerging learning.
- Offer constructive advice and guidance that may help strengthen PPI practice.
- Facilitate the sharing of learning and good practice across the ARC.
- Highlight learning needs and opportunities that could support ongoing PPI development.
Roles
PPI Team
It is suggested that:
The PPI Team take responsibility for:
- Helping ensure Theme PPI Leads have access to current activity lists for their Themes.
- Providing support to Theme PPI Leads as they engage with research teams in planning PPI and considering monitoring activities.
- Creating spaces for regular conversations among Theme PPI Leads about PPI work within each Theme.
- Summarising PPI activity within Themes and across the ARC.
- Collating feedback for each Theme or project and supporting Theme PPI Leads in sharing this feedback.
- Enabling the exchange of learning and examples of good practice.
- Identifying potential PPI learning or development opportunities and exploring how these might be taken forward.
Theme PPI Leads
It is suggested that:
Theme PPI Leads can choose to contribute to this work. While participation is voluntary, it would be helpful for at least one Theme PPI Lead within each Theme to take on this role.
Theme PPI Leads who opt in will be supported by the PPI Team to:
- Work alongside Theme leads and researchers in shaping PPI plans for ARC-funded or ARC-linked research.
- Contribute to the development of broader PPI approaches within their Theme (e.g., involvement in decision-making, reporting processes, and learning opportunities).
- Help ensure PPI plans are recorded for all projects and for the Theme as a whole.
- Explore PPI monitoring plans and timelines with researchers as part of PPI plan development.
- Support monitoring activity within the Theme’s projects.
- Share reflections on PPI activity and what is happening in their Themes at Theme PPI Leads meetings.
- Collaborate with the PPI Team to consider feedback for individual projects or Themes.
Process
It is suggested that:
- Theme PPI Leads can liaise with Theme leads and the PPI Team to clarify which projects sit within their Theme.
- Theme PPI Leads can work with their Themes to shape Theme-level PPI plans, recognising that approaches may vary and should be agreed early in the process.
- Research teams will be encouraged to create their own PPI plans, with the option of seeking support from the Theme PPI Leads.
- Theme PPI Leads will share updates on Theme-level and project-level PPI planning with the Theme PPI Leads group and the PPI team.
- PPI plans will be reviewed and ‘signed off’ by the PPI Team prior to a project beginning.
- Theme PPI Leads should keep an overview of monitoring schedules and share updates with the Theme PPI Leads group and PPI Team.
Support
It is suggested that:
Theme PPI Leads taking on this role will have access to support, which could include:
- Payment for time (to be agreed individually, with flexible use for tasks as needed).
- One-to-one discussions to help plan their approach.
- Group training or reflective discussion sessions.
- Exploration of different models of PPI.
- Guidance aligned with the UK Standards for Public Involvement.
Tasks
Possibilities include:
Collecting Data
- For each Theme, information about Theme-level and project-level PPI activity will be documented.
Discussing Data
- Theme PPI Leads will have opportunities to present and reflect on PPI within their Theme at Theme PPI Leads meetings.
- As projects and Themes engage in formal review and monitoring, Theme PPI Leads may report back any insights to the Theme PPI Leads group and PPI Team.
Analysing Data
Presentations and discussions aim to deepen understanding of PPI activity by exploring:
- Appropriateness, quality, and possible enhancements.
- Examples of good practice and learning that could be shared.
- Challenges, adaptations, and lessons emerging from monitoring.
- Broader insights about PPI across the ARC.
Learning From and Using Data
After discussions:
- The PPI Team and Theme PPI Leads will shape feedback for each project or Theme and consider effective ways to share it.
- With support from Theme PPI Leads, the PPI Team may summarise PPI across Themes and the wider ARC for reporting and knowledge-sharing.
- Particularly valuable examples of practice or learning—ranging from early steps in PPI to more innovative approaches—may be highlighted for wider dissemination.
- Learning gathered collectively may inform potential training or development opportunities.
Potential Outputs
- ARC annual PPI report
- Annual ARC report to NIHR
- PPI case studies or stories
- Blogs
PPI Monitoring to Learn planning and monitoring form
PPI in projects: This form is to help plan PPI in ARC projects/studies. You will then review this plan at regular intervals during the life of the project/study.
|
Please tell us about where you and this project fit in the ARC family: | ||||||||
|
Theme |
|
Project Name
|
| |||||
|
Brief description of project |
| |||||||
|
Study PI |
|
PPI Contact (if not PI) |
| |||||
|
|
|
Email (if not PI) |
| |||||
|
Project/Researcher Overview – include any existing PPI experience and any notable impressions.
Links for useful resources for planning PPI: UK Standards for Public Involvement, Public Involvement in Research Impact Toolkit (PIRIT) – Marie Curie Research Centre – Cardiff University, | ||||||||
|
Topic |
Questions & Examples |
PPI Plan/Ideas – Date developed |
How we did – Date reviewed | |||||
|
STAKEHOLDERS
|
· What do you think PPI is/can be? · Who are your key stakeholders? · Why are they important?
Examples: patients, carers, parents, particular community, clinical staff, commissioners etc?
|
|
Prompts for review: Did you identify the right stakeholders? Have you identified new stakeholders? What impact have any changes to your plans had? What have you learnt? What would you do differently? What are your PPI plans moving forward? | |||||
|
Changes
|
Challenges
|
Learnt
|
What will you do differently?
| |||||
|
Summary of activity:
| ||||||||
|
INCLUSION
|
· What can you do to make sure your PPI is inclusive? · How will you support the people you involve? Examples: inclusive recruitment practices, fees, expenses, where will people be recruited from (e.g. communities, geography, conditions), training/learning opportunities, support, accessible language, feedback to PPI members, venues & communication, link to (ARC) PPI networks etc |
|
Prompts for review: Were you able to recruit the appropriate people in the timeframe you expected? Has the support you planned been appropriate? What impact have any changes to your plans had? What have you learnt? What would you do differently? What are your PPI plans moving forward? | |||||
|
Changes
|
Challenges
|
Learnt
|
What will you do differently?
| |||||
|
Summary of activity:
| ||||||||
|
INFLUENCE
|
· What is the aim of your PPI? · How will you ensure ‘a strong stakeholder and/or public voice’ in decision making? · Are there limitations to how PPI can change or influence your study? · What can (have) they influence(d) or shape(d) in the study? Examples: question(s); agreeing outcomes measures; study design; dissemination etc. |
|
Prompts for review: Has your PPI influenced parts of the research? Have there been any surprises? What impact have any changes to your plans had? What have you learnt? What would you do differently? What are your PPI plans moving forward? | |||||
|
Changes
|
Challenges
|
Learnt
|
What will you do differently?
| |||||
|
Summary of activity:
| ||||||||
|
APPROACH
|
· How will people be able to get involved in the study? · What methods or approaches will you use? · Will there be opportunities for involvement at different levels (engagement through to co-production)?
Examples: set up a group, independent activities, peer researchers, multi stakeholder group, in person/online, working with an existing group, crowdsourcing via social media or public events etc |
|
Prompts for review: Have the approaches you planned to use been effective? Have these activities changed your research? What impact have any changes to your plans had? What have you learnt? What would you do differently? What are your PPI plans moving forward? | |||||
|
Changes
|
Challenges
|
Learnt
|
What will you do differently?
| |||||
|
Summary of activity:
| ||||||||
|
ACTIVITY
|
What activities will your PPI take part in?
Examples: reviewing patient information; considering impact or burden on participants; interpreting data; dissemination activities and outputs etc.
|
|
Prompts for review: Have you been able to involve people in activities as planned? Have there been multiple ways to get involved? What impact have any changes had? What have you learnt? What would you do differently? What are your PPI plans moving forward? | |||||
|
Changes
|
Challenges
|
Learnt
|
What will you do differently?
| |||||
|
Summary of activity:
| ||||||||
|
RESOURCES
|
· How many public contributors do you plan to involve? · How often will they be involved? · How much PPI Budget will you need to do what is planned? · How much staff hours and resources do you need to facilitate the PPI plan?
|
|
Prompts for review: Have resources been sufficient (people’s time, money, frequency of activities etc)? Have your planned activities taken longer than expected? What impact have any changes had? What have you learnt? What would you do differently? What are your PPI plans moving forward? | |||||
|
Changes
|
Challenges
|
Learnt
|
What will you do differently?
| |||||
|
Summary of activity:
| ||||||||
|
SIGNIFICANT CHANGES
|
What difference do you think the PPI will make and to whom/what?
Examples: change in knowledge, skills, or confidence; affected relevance
|
|
Prompts for review: Please tell us about: | |||||
|
One significant change to the study.
| ||||||||
|
One significant change to the PPI members you have involved.
| ||||||||
|
One significant change to the research team.
| ||||||||
|
If there has (have) been no change(s) please explain why you think that is.
| ||||||||
|
PPI IMPACT & EVALUATION |
· How do you plan to capture the impact or evaluate the PPI? · When will you review this PPI plan?
Examples: meeting minutes; PPI numbers and outputs; before and after patient information developed; lay summary; recruitment and retention rates; PPI impact and evaluation tools such as The Cube, Impact log/’You said we did’ tools/feedback, Piiaf, PIRIT or GRIPP2 We recommend: · PPI plans should be formally reviewed at least twice during a project unless it is two years or less in length. · that projects build in regular ‘internal’ reviews to make sure PPI is working well for the project and the public contributors. |
|
Prompts for review: | |||||
|
Please tell us about the PPI impact/evaluation tools you have been using.
| ||||||||
|
Please tell us how your PPI members (will) contribute to the PPI impact/evaluation you are undertaking.
| ||||||||
|
Please tell us about how your PPI members have contributed to this monitoring.
| ||||||||
|
If you haven’t already done so elsewhere, please tell us about how you feedback to your PPI members?
| ||||||||
|
SUPPORT TO DEVELOP THE PPI PLAN/REVIEW
|
· Please record any help or support you have received while developing your PPI plan or doing the review? · How helpful was the support? · Could you have done it without the support? · What was the most important aspect of the support you received? · Would you need that support in the future? · How will it affect future PPI you do?
Examples: PPI funding, guidance about how to plan PPI, another team member, PPI Champion, Public contributor with/without lived experience etc
|
|
Prompts for review: Please tell us about any additional PPI support you have received since developing your PPI plan.
| |||||
PPI Monitoring to Learn PPI impact and evaluation guidance
Why is it important to evaluate PPI and understand the impact it has?
Better for PPI
The more we understand the impact of PPI and evaluate it the better we will understand what we are doing, what works for who and when.
The same process or method will not always work for all settings. For example, the method you use for involving adults will not work as well when you want to involve young children or those living with dementia or learning difficulties. You might need to tweak what you do or completely redesign what you do.
Evaluating your PPI process and understanding the impact and how things worked or didn’t will hopefully help you to plan future involvement. Sharing your evaluation also helps others embarking on PPI in research, consider it a ‘library of starting points’.
Ethical Aspects
We are viewing ethics in a broad sense, including the moral and cultural issues of working with different people and communities, and the need to avoid waste in PPI and research more generally.
Things may appear to have gone well but unless you ask people about how it felt for them you might not know why and what the issues were. For example, the public members might have really enjoyed the experience but the team facilitating the PPI might have had a dreadful experience due to the extra time and effort to make the PPI so good. This knowledge helps to either be proportionate or to ensure additional resources are built in to facilitate the PPI in future projects.
Alternatively, through the evaluation process and looking at the impact PPI has had it might seem that nothing in the research changed. The review process should also help identify why this is: Have suggestions been ignored? Were people involved too late to influence anything? Were they involved in the wrong way/time?
Recording and evaluating what was done will help to avoid ‘re-inventing the wheel’ and even more importantly help to avoid using approaches that don’t work, or improve approaches by ‘tweaking’ them, so they work better.
Return on investment
Funders and funding panels are becoming increasingly astute about assessing PPI plans in applications. They are looking for evidence that the planned PPI is appropriate, meaningful, and effective. Building this evidence into applications will hopefully assist research plans in securing future investment.
Overall, PPI impact and evaluation work should help you to develop PPI plans for future research that are evidenced by what you have learnt from previous activities. Alongside this you should also be referencing good practice standards and guidance. Thus, showing that you are aware of national guidance and standards for PPI and that you understand the wider PPI context and PPI ‘direction of travel’. The evaluation of your PPI can help you to show how you are reflecting on, developing, and improving your own PPI practice.
What do we mean by ‘PPI Impact’ and ‘PPI Evaluation’?
In 2019 the NIHR defined the impact of PPI in NIHR health and social care research as: ‘The changes, benefits and learning gained from the insights and experiences of patients, carers and the public when working in partnership with researchers and others involved in NIHR initiatives.’
PPI impact is all about what impact the PPI you have done has had on the ‘who’ and the ‘what’.
The ‘who’ and the ‘what’ might be:
- your public contributors
- the research team
- individual researchers
- support staff
- the research or specific parts of the research
- the methods or design of the research
- recruitment and retention
- the organisation
- how you disseminate the work etc.
It is important to remember that it is difficult to separate out the specific impact of PPI on all of the research or even individual components as the PPI (if it is not tokenistic) will be an integral part of the research in the same way other parts of the research are (such as methodology, analysis, statistical results etc) as they are all affected by the context and what came before and what happened after.
What is important is to investigate what happened, how people felt and can you connect changes to inputs (whether they are specific or general). This is where making use of the resources identified in the tables that follow is important. Gather your evidence as you go and try and track it where you can to see what impact it has.
PPI evaluation is all about evaluating your PPI practice to make sure you continually improve your practice in terms of impact and experience.
PPI evaluation can be ongoing (e.g. continually ‘doing, reflecting and reviewing’ what you do) or it can be time specific (e.g. the beginning, middle and end of a research project).
Both involve reflecting on what went well and what went less well and thinking about how this understanding can help you adapt your PPI practice either as you move forward in the research you are doing or as you move towards a new project. It is probably best if you can do both.
So how do we do all of this?
What should we be looking at? Many of the resources listed below explore some of the things to consider when evaluating PPI and its impact. One way to think about it is to consider the different types of potential impact, the following examples are taken from Lammons et al, 2025.
- Impacts on People
- Impacts on Researcher
- Impacts on Services and Systems
- Impacts on PPIE Processes
Resources already exist in abundance. You do not need to ‘re-invent the wheel’ but you might need to adapt something to work for you, the research, the community you are working with and the PPI you are planning.
This is a webinar from NIHR that explores some of the resources detailed below: Evaluating the Impact of Patient and Public Involvement in Research.
The following tables provide a list of some of the examples that are already out there. There will be others that are not listed here and it is unlikely that something will fit your research exactly, but that’s OK, you can adapt or search for other examples once you have an idea of the kind of resources that others have found helpful and what might be helpful for your project.
Table 1 PPI Planning resources
|
Resource name |
Description |
Potential use |
Useful links |
|
PPI Monitoring to Learn |
NIHR ARC OxTV developed a planning and monitoring resource to help ARC researchers develop and monitor their PPI plans. |
All ARC research can be supported to use the Monitoring to Learn resources to develop an appropriate PPI plan and to monitor it throughout the course of the research. |
TO BE ADDED
This resource might also be useful: Monitoring to Learn Tools – PEM Suite
|
|
UK Standards for Public Involvement |
These are a set of PPI Standards co-designed with the PPI community in the UK. They explore what to aim for in PPI to ‘do it well’. |
A useful resource that uses questions to help you explore each of the six standards. The standards can be used as a framework to present your PPI plan. A number of funders now expect PPI plans to at least reference the standards. | |
|
The Four Pi National Involvement Standards |
Developed and co-created by the National Survivor User Network. The ‘P’s reference: · Principles: the underlying values of involvement · Purpose: defining why people are being involved and specific goals · Presence: ensuring the right people are involved · Process: defining how involvement happens ‘I’ is for Impact of PPI |
These standards were developed to support work in PPI in healthcare services and support, especially mental health. They can also be used in healthcare research. There are resources available to support using the 4Pi standards. | |
|
Charter for Public Involvement and Engagement |
The Charter sets out best practice for fully embedding public involvement and engagement (PPIE) specifically in public mental health research. It covers definitions, principles, and values as well as information about resources and support. |
The charter provides 12 guiding principles for effective public involvement and engagement. They provide a framework to explore PPIE. |
https://sphr.nihr.ac.uk/public-involvement/charter-for-public-involvement-and-engagement/
|
|
PIRIT (Public Involvement in Research Impact Toolkit) |
PIRIT is designed to support researchers in planning and tracking public involvement in research. It includes two main tools: · Planning Tool: A checklist of potential public involvement activities and relevant UK Standards, guiding researchers through the research pathway. · Tracking Tool: A simple spreadsheet to record when and how public contributions occurred, along with what changed and why it matters. |
This resource can help you plan your PPI and then track it. It is helpful as it is closely linked to the UK Standards for Public Involvement so can help you to use the standards as a framework for your PPI plan. |
PIRIT download form – Marie Curie Research Centre – Cardiff University
|
|
Equality Impact Assessment (EqIA) |
An Equality Impact Assessment (EqIA) is a systematic approach designed to evaluate how proposed policies, practices, or research activities may affect different groups of people, particularly those with protected characteristics under the Equality Act 2010. The primary goal is to ensure that all individuals have equal access to opportunities and that any potential negative impacts are identified and mitigated. |
Doing an EqIA can help with the thinking around developing inclusive PPI. It can help you think about who you should be involving and how you can make the process of involvement inclusive. |
Table 2 PPI Evaluation resources
|
Resource name |
Description |
Potential use |
Useful links |
|
The Cube |
‘The Cube’ is an evaluation framework that uses a digital tool to support PPI. The framework consists of four questions which allow for sliding scale responses with the addition of contextual text-based comments.
‘The Cube’ evaluation framework can be used with or without the digital tool. |
This resource is useful for evaluating PPI across the course of a research project. It can help to improve PPI during a project.
It can be delivered digitally using the digital tool or in person using a flipchart and post it notes. |
The Cube Evaluation Framework – for researchers.docx – Google Docs
|
|
Impact Log |
An impact log is a simple method of recording PPI impacts and outcomes (in real time).
There are many examples of this. The example we have provided is from People in Health West of England. |
This is a useful resource for basic evaluation and identifying impact of PPI. | |
|
PPI Monitoring to Learn |
See above |
The monitoring element of the tool allows you to track progress against plans over time. You can also adjust plans as appropriate. |
TO BE ADDED |
|
PIRIT (Public Involvement in Research Impact Toolkit) |
See above |
The tracking tool allows you to track progress against plans over time. |
PIRIT download form – Marie Curie Research Centre – Cardiff University
|
Table 3 Reporting PPI resources
|
Resource name |
Description |
Potential use |
Useful links |
|
GRIPP2 (Guidance for reporting involvement of patients and the public) |
GRIPP2 is guidance for reporting PPI. The aim of GRIPP2 is to improve the quality, transparency of the PPI evidence base to ensure practice is based on best evidence. |
Gripp2 is specifically designed to support reporting of PPI by using a co-designed list of items to report on. | |
|
PPI recording tool |
The purpose of this tool is to provide researchers with a comprehensive and usable recording checklist to document patient and public involvement in research. The tool was designed by researchers to map on to the GRIPP2 reporting checklist. |
This resource is useful to help gather the information needed to use the GRIPP2 framework to report on PPI. |
Reporting tools — Nuffield Department of Primary Care Health Sciences, University of Oxford then click on the relevant link. |
|
REPRISE Reporting Tool |
This reporting guidance helps researchers reflect on PPI when reporting their Research Priority Setting exercises. The tool has 10 sections for priority setting (context, governance, framework, stakeholders, identification, prioritization, output, evaluation, translation, funding/conflict). |
Helps structure writing about PPI in Research Priority Exercises as well as detailing who was involved (patients, carers, professionals), how they were engaged (methods), what priorities emerged, and how these were used. | |
|
Monitoring to Learn tools |
A co-created agile tool for identifying the right measurements for patient engagement practices.
· Learning based on meaningful and measurable metrics · Increased impact of patient engagement initiatives through monitoring and evaluation · Exploration of new ways of improving patient engagement processes · Dialogue facilitation between stakeholders · Better decision-making in medicines development, healthcare management and beyond |
This resource can help assess the process, outcome, impact and results of a PPI initiative, activity, or strategy, and then set new goals and objectives for PPI work. |
Table 4 – Papers and other resources
|
Resource name |
Description |
Potential use |
Useful links |
|
Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot, Greenhalgh et al 2019 |
This is a review of different ways of approaching PPI in research – 65 separate examples were reviewed and evaluated using a tool called Canadian Centre for Excellence on Partnerships with Patients and Public (CEPPP). Five types of frameworks were identified; Power Focussed, Priority Setting, Study Focussed, Report Focussed and Partnership Focussed. They are explained in Table 1. |
There was limited evidence of transferability of the frameworks, with authors concluding that ‘off the shelf’ generic frameworks often needed adapting. This paper offers an evidence base of approaches to start from.
|
https://onlinelibrary.wiley.com/doi/10.1111/hex.12888
|
|
Guidance document: Evaluating public involvement in research, Kok et al 2018 |
This guide assembles four different approaches to evaluating PPI providing summaries and pros and cons for each approach. Two approaches are already described in Table 1, Impact Log and Cube. Public Involvement Impact Assessment Framework (PiiAF) and Realist Evaluation are not. |
Each example benefits from a context section, what the approach comprises (methods and tools) and how it can be used. Authors helpfully describe Key Points relating to each approach helping users to decide which one to use, or not. | |
|
Measuring impacts of patient and public involvement and engagement (PPIE): a narrative review synthesis of review evidence, Lammons etc al, 2025 |
This review collected and reviewed 27 published accounts of impact and PPI, covering UK, US, Canada, and Australia. Authors found that reporting on impact in PPI was variable. The considered four types of impacts: on people (patients, researchers etc); different phases of the research; services and systems in research; and the PPI itself. They report positive and negative impacts. |
Before deciding how to approach measuring impact in PPI this review provides a helpful overview that is evidence based. Table 4 provides summary information of all the included studies. Authors invite readers to use their four categories to consider impact in PPI. |
https://link.springer.com/article/10.1186/s40900-025-00748-6 |
