Measuring service learning outcomes - developing a tool for reflection and learning

by Pippa Chenevix Trench & Bart Dankaerts

Service learning has been shown to increase student awareness of local and global issues facing society and the environment. When service learning is well implemented the impact on students can be many and deep –improving critical thinking and problem-solving skills, self-awareness and awareness of others, communication skills, empathy, perseverance, self-reliance, and the confidence to take action independently (e.g. Celio, Durlak & Dymnicki, 2011). Taken together these outcomes lead to students becoming competent and knowledgeable global citizens with the confidence and motivation to actively engage in their world. In our model, we refer to this as the three A’s: Aware, Able and Active Global Citizens.

But how do we know whether students are achieving these desired outcomes? And how do we know the degree to which school supported service learning opportunities are effective in helping students reach these goals?

In early spring 2017, the two of us met to talk about developing a tool that would help us answer some of these questions for our students at the American School of the Hague (ASH). At that time we assumed that we could find such a tool already existing and “tweak” it for our own use. We found many survey tools that assessed “Global Citizenship”- mainly designed for university level students. But none that expressly related to the service learning outcomes we were interested in and none explicitly designed for middle and high school students.

So we went back to first principles and decided to start from scratch. We are still on that journey. The figure below outlines the process that we have followed so far.

Process

We developed a questionnaire and asked students for their perspectives: from the importance they attribute to gaining a deep understanding of the context in a service issue, to their own rating of specific skills that are important for successful service learning and can be strengthened through the experience. These questions require students to reflect on their own learning as well as on service learning processes. Student and teacher feedback has been crucial in developing the tool. The process has been highly iterative; for example feedback from teachers and students have sent us back to reflect on the very purpose of the tool as well as considering re-phrasing of questions and the way in which the questions are presented.  

Some of the issues we are grappling now with include:

  • Vertical alignment: Starting with our desired student outcomes at high school level, we must now look at stages in development among younger grades and adapt the language and expectations of service learning outcomes among younger students.
  • Student feedback: Any tool is only as good as the data and information it provides. In addition to piloting the questions we are currently focusing on what type of feedback is most helpful to individual students. This is not intended to be a summative assessment tool, and so any feedback needs to be of a constructive and formative nature.
  • School feedback: How adaptable should the tool be to reflect the needs and current status of a program in different schools?
  • The data is based on student perspectives; in aggregated form this data may open conversations among faculty and support staff to consider whether their programs are achieving their intended outcomes on student learning or not, but they cannot be considered as evaluative. What action (workshops, coaching, etc.) is best suited to support this reflection and guide schools to develop stronger programs and build on their strengths? 
  • There are many important aspects of service learning  that our tool does not try and address, such as the effectiveness of any service activities on the target communities/environment or the degree to which service learning experiences meet recognized standards. What instruments are best suited to exploring these equally important aspects of a service learning  program?

This is work in progress. Without data on individual student outcomes, how can students understand their own learning and how can teachers and schools differentiate their support to students to ensure a student has a genuine learning experience? And, in addition to the survey itself, we must develop materials and processes to support the schools and students develop their service learning initiatives based on best practice and their own contexts and priorities.   

What started as a simple decision to come up with a quick questionnaire has developed into something more complex as we started to see its potential as a tool to strengthen global citizenship rather than just measure it.  

We will continue to share our progress towards this goal as our work continues. We would value feedback and ideas from others interested in this area of work.

  • Bart Dankaerts currently serves as the CAS & Service Learning Coordinator at the American School of The Hague.
  • Pippa Chenevix Trench is an expert in action-research on anthropology and education, with a background in international development and co-curricular education design and administration.