Starting out

We would've missed so much impact if we hadn't asked them [community partner] how they felt it went!

Practitioner, Science Oxford

Evaluating our work can be seen as a bureaucratic tick box exercise, required to ensure we are accountable for the outcomes of a project when writing reports. Whilst honest accountability is an important goal of evaluation, good evaluation should be embedded at all stages of the programme and should help us do our jobs better, more effectively and impactfully. Good evaluation can also ensure we are being equitable, that wellbeing is accounted for, and that expectations of participants are explored and met.

Enabling all stakeholders to take part in the evaluation process in an open and transparent way is the key to embedding the learning process for all participants. It reduces the risk that participants become subjects of someone else’s observation and research or are ‘left hanging’ at the end of a project without suitable feedback or understanding of their participation.

The first step is to think about who should be involved in the evaluation and learning process, as discussed in the previous partnerships sections. The next step is to find out how we make a start.

We often put together evaluation frameworks as part of reporting to a funder, and these frameworks are often based on a funder’s objectives. Evaluations can also be piecemeal – quite different from one project to the next – but approaching evaluation in a case-by-case basis doesn’t necessarily create learning for the whole organisation, beyond the individual project. Be bold! View each project you do as an opportunity to build up a coherent picture over a longer period of time.

For your current project, make sure you have clear aims and objectives. Identify these – perhaps in a simple table for clarity – and take them to discuss with your community partner or participants. Discussing these at the beginning ensures that the project concept and planning meet your original aims, but equally meet the aims of your partner.

If you are able to start this discussion early (even ahead of writing the funding proposal) you can bring shared aims for what you want to achieve together as the focus for your evaluation. Be mindful of time and resource your partner/participants can give at this early stage, but a collaborative approach ensures far greater buy-in from all stakeholders when it comes to capturing the impact and embedding the learning.

It can be a very useful process to lay out your plan as a Logic Model or a Theory of Change. This can be a very simple document that sets out a list of your aims and objectives, inputs, processes, outputs and outcomes that you can refer back to. It is also worth considering the various influences, assumptions and risks that you should manage.

Be prepared for things to change, evolve and develop, especially where the relationship with the community partner is new and/or exploratory. It is worth ensuring you capture and communicate these changes for your evaluation. It is often the unexpected outcomes that provide greatest learning for project holders.

Evaluation is not just about accounting for your work, but also helping you do your job better and giving the time and space to reflect on your work and practice during your project. Keep processes iterative and stay honest and open about changes.

Evaluation is the way in which you can both prove and improve your practice.

It can be helpful to think about evaluation as a learning journey – making sense of your work – rather than just a requirement for a deliverable. Evaluation requires a lot of thought and planning at the start of the project and is best done ‘little and often’ rather than just at the end of a project.

Part of this is making sure practitioners have the time and space to reflect on their work both independently and, where possible, with partners. This reflective practice helps support the expertise of the practitioners and also makes reporting the changes and learning across the project much easier. Capturing change within your partnership, such as perceptions of each other’s motivations and objectives, can be a very useful thing to reflect upon.

Action Research

An increasingly popular model of science engagement includes incorporating models of Participatory Action Research and/or Peer Research. This involves participants (including lived experience), practitioners and researchers being actively involved in exploring a particular issue and making positive changes.

This framework of enquiry involves iterative cycles of research, action and reflection and helps challenge more extractive models of research and engagement, instead recognising that the communities themselves are best placed to explore, research and enact change in their own contexts.