Teacher Feedback Survey Case Study
Schools and Teachers Programmer, The National Gallery
About the project and our evaluation challenge
The National Gallery (NG) welcomes about 25,000 pupils and their teachers every year. We had been using automated surveys for about eighteen months but were keen to review the questions in light of some recent changes to programme design and GDPR legislation. We also run Continuing Professional Development (CPD) courses for teachers and this project also gave us the opportunity to improve the way in which evaluated these sessions. In the past, we have used paper evaluation forms handed out at the end of the day. We started by looking at how we could evaluate the CPD offer around the NG’s flagship Take One Picture (TOP) Project which has been running for over 20 years.
This case study focuses on the evaluation of gallery educator-led sessions for schools and teacher CPD courses.
Setting the scene
We offer hour-long gallery educator-led sessions on a variety of different themes and curriculum areas. The sessions take inspiration from paintings and museum objects, buildings and spaces and are designed to support the development of curriculum related knowledge, creative thinking and confidence. The first part of our evaluation challenge was to develop and pilot automated teacher feedback surveys linked to evaluate the effectiveness of these programmes.
The National Gallery TOP CPD days are attended by over 2000 teachers from across the UK annually. Every year a focus painting from The NG’s collection is selected and the CPD day offers ideas and inspiration for teachers to respond creatively to its themes, subject matter, historical context or composition. The TOP methodology is non-prescriptive, encouraging teachers to work with pupils to develop their own lines of enquiry according to what works best for each school’s context. This allows teachers to make their own decisions about the project’s length, curriculum areas they wish to cover, the nature of creative outputs, and year groups involved. A selection of work by schools is shown in an annual exhibition at NG (highlighting the learning processes), with additional digital content, and exhibitions and celebrations in local communities. We were interested to see if the format of the day was effective in supporting the TOP pedagogy and to ask teachers to reflect on how the CPD might impact on their planning and teaching back at school.
The Practical Evaluation project enabled us to work together with colleagues from The Fitzwilliam Museum to develop logic models for our schools and teachers programmes to clarify what we were aiming to achieve and how. We discovered many similarities in terms of our approach to working with museum and gallery collections. We also both use Artifax, an event booking system, to manage our schools bookings.
At The National Gallery, we had already worked with Eric Jensen to design a survey for teachers and we were able to integrate Artifax with an automated online survey system. We ask teachers at the point of booking for consent to send them an evaluation request. Then Artifax automatically links to the automated survey system, handing off the teacher’s email and date of their booking. The automated survey system is set up in advance to send the teacher an email invitation to fill in a feedback survey on the day of their gallery and up to three reminders in the weeks following their visit (only sent if the survey is not completed).
Reflections on using automated surveys with schools and teachers
The National Gallery has had 625 teachers enrol for the automated feedback survey in the last 12 months, of which 324 questionnaires have been completed. This impressive response rate of just over 50% indicates that being sent the links via email means that teachers are more likely to respond. The survey system dashboard provides a very useful overview of how many emails have been sent out, how many surveys have been completed and what the average survey completion time is. You can also add charts to the display to help you monitor particular aspects of your programme. When reviewing the results of the survey, the system displays the quantitative data (e.g. age of pupils, marketing channels and level of satisfaction scales) in colour coded bar charts with green indicating a positive response and red a negative response. Qualitative feedback (e.g. comments) are displayed in colourful word clouds. This makes the data very quick and easy to access giving us headline information on how the programme is performing. (Further details is available by clicking ‘see more’ within the dashboard)
The data we have collected has been very useful to help us to make evidence-based arguments to others within the organisation. For example, the information we collect on facilities has been useful as part of ongoing discussions about toilets for school groups as has the feedback about the greeting and welcome groups receive when they arrive at the Education Centre. This feedback will help us to improve the service we offer to schools. From a learning perspective, the feedback on student experience is the most valuable, as this is at the very heart of our schools programme. Level of agreement statements such as ‘The content of the session sufficiently addressed my pupils’ curriculum needs’, ‘The educator encouraged my pupils to look carefully and paintings’ and ‘The educator encouraged my pupils to form their own interpretations’ help us check that our delivery methods match our programme-level goals. For example, the goals that relate to these outcomes are, ‘Pupils develop new curriculum-related knowledge on the session theme’, ‘Pupils gain skills in visual analysis’ and ‘Pupils feel capable of making their own personal interpretations and responses to paintings.’
The feedback we receive from teachers after our day-long CPD days is also very useful. We have an even higher response rate with these, with 85 completed surveys out of an enrolment of 141. This higher response rate of 60% might be explained by the fact that the CPD session feedback is about supporting reflective practice as well as evaluating the effectiveness of the day. Teachers have the benefit of anonymity filling it in online after the event and so they are able to be more thoughtful and honest. The more open ended comments that they leave are very useful as they provide detail of how the CPD day links to work they are planning back at school and contains suggestions of what we could do to improve the session and resources we offer.
Working with Eric as part of this project has ensured that the questions we ask and the data we collect is both useful and linked to our programme level goals. This will help us to both communicate better with schools, teachers, funders and other stakeholders and ensure that our programmes continue to be relevant to the audiences that we serve.