How to capture the magic

There are many creative and playful methods that can ‘gamify’ certain questions into activities (e.g. participants returning lab-coats onto coloured hooks based on their answer to how enjoyable they found the session).

Different approaches to acquiring the answers you seek can help reduce the need for long questionnaires.

Reflection.jpg

Top Tip

You don’t have to rely on a post-event survey form to gather feedback. Questions could be done during ticket sign-ups, as interviews or observations, or embedded into the experience itself.

Case Study from Explore Your Universe

Used to strict requirements from project funders, during the early stages of Phase A, staff from one science centre spent a large proportion of a first session completing feedback forms with the families. The forms needed facilitation and participants struggled with the level of literacy required by the multiple questions, particularly as the forms had not been translated into the participants’ first language.

Learning from this was rapid and taken forward to produce far simpler and more creative ways (single questions, stickers, emojis etc.) to capture experience. A flexible approach to evaluation methodology avoids diminishing the activity, experience or relationship when capturing impact.

Be Creative

Co-creating evaluation tools with community partners

Community partners and, where possible, participants should be involved in the design and co-creation of evaluation tools. They will know, for example, whether the language needs to be modified or translated, if the content needs more detail or simplification, or if images are misleading. They can also help their families or young people to fully understand voluntary informed consent.

Before you decide on a particular tool be sure to:

  1. Sit down with your community partner and discuss the tools you are using. Explain what they are measuring and why you wish to get these answers.
  2. Find out what they wish to measure and settle on commonalities. Avoid adding to the list, instead agree what you really will use to avoid overloading your participants with questions.
  3. Think about the methods that are practical and how you can gather the data and inform participants on consent.
  4. A balance needs to be struck between asking for the community partners’ support and demanding too much of them.

Many of the most meaningful and long-lasting impacts of partnership work are too subtle or awkward to translate into quantitative data (for example, the trust developing over the course of a project between partners). This is where qualitative approaches and case studies can help capture the full story.

Top Tip

Keeping a reflective journal is a great tool for qualitative and quantitative data capture. It should be completed after every session and can help capture the ‘magic’ key moments, the stories of engagements and the unexpected outcomes.

In addition to metrics data (e.g. number of attendees) reflection journals can also capture formative evaluation of all the changes you made along the journey following feedback.

Both your own experience and that of your partner should be a valued part of the evaluation picture. It is often your partner who can spot early signs of challenge or draw attention to ‘lightbulb’ or ‘meerkat’ moments, or other indicators of impact that would otherwise be missed. Including this voice in your evaluation uncovers a wealth of impact and understanding from a different but critically important perspective.

This can be effectively captured using interviews or completing reflection journals together.

Case Study from Explore Your Universe

At the start of the programme, designed partnership cards were used to share how new relationships were settling in. The cards contained single words such as ‘tourist’, ‘guide’, ‘pilot’, ‘passenger’ and even ‘baggage’ as options to choose from! The cards were conversation starters, but pairs did show some interesting dynamics. An exceptional, long-lasting partnership shared ‘teammate’ from the perspective of both the science centre and community partner practitioners. Whereas a partnership that struggled to meet an equitable balance in the first phase of delivery shared ‘passenger’ and ‘performer’ from the community partner’s perspective.

The reflection journals were a useful tool and the principles of reflective practice that were stimulated through conversation about the journals will be integrated into team planning in W5 going forwards.

Practitioner, W5
Capturing the magic

Pre and post data

If you are looking at multiple interactions or changes over a sustained engagement, another consideration is whether you want to connect results to an individual (for tracking impact or change). This raises questions about anonymity, but there are creative ways you can anonymously track individuals from pre-to-post, without asking for names or date of birth outright.

Interrogating practice with the reflective diaries gave us a new dimension to how we were doing things... Reflective diaries are now central in everything we do at Aberdeen Science Centre. All team members are encouraged to use reflection on a daily basis.

Practitioner, Aberdeen Science Centre
Allow-time-l.blue.jpg

Allow enough time

Build in plenty of time to do this properly, not at the last minute when everyone is jumping on the minibus home!

Gathering quality reflections doesn’t need to be a formal focus group. Depending on the age and nature of the group, you could gather reflections by having a short discussion, or by asking everyone to think for 5 minutes before writing anything down on their form. Either way, it is worth encouraging them to take their time.

Top Tip

Find downloadable examples of pre-and-post tools, reflection journals and other creative evaluation tools at inclusion.sciencecentres.org.uk

Links and downloads