1. Home
  2. Tools & Resources
  3. Evaluation case studies, and what was learned from them

Evaluation case studies, and what was learned from them

This suite of publications comprises two evaluation case studies and one report drawing important learnings from these two evaluations. Superu asked Robin Peace, Associate Professor Evaluation and Research at Massey University, to peer review the reports, and she said that:

“… this suite of reports tells an innovative story about government willingness to look for an understanding, not just of how well two NGOs are doing delivering their services - although it also does that - but more particularly how organisations can learn from evaluation capability building and become more self-determining in their ability to understand and effectively report to funders on the nature and quality of their business.”

What helped NGOs to do evaluation well What helped the evaluators
  • Commitment of management and staff time and energy
  • Participants who were keen to learn from the process
  • Using external facilitator(s)
  • Participants with in-depth knowledge of the programme being evaluated
  • Good organisational infrastructure
  • Contributions being welcomed  from all stakeholders
  • Collaboration over selection and funding between NGO and funders
  • Funders with a focus on both evaluation and capacity building
  • Using appropriate tools, resources, processes (especially workshops and coaching sessions)
  • Knowledgeable and professional team
  • Ability to integrate responsiveness to Maori
What did not help NGOs What did not help the evaluation team
  • The need to source and pay for external expertise
  • Staff having to take time out from their core roles to contribute to and manage the evaluation
  • Evaluation outcomes were not sufficiently understood or prepared for
  • NGO capacity issues (financial and staff time)
  • Having to focus on both evaluating and capacity building
  • Lack of clarity about the reporting that funders expected or required
  • The challenge of incorporating Maori responsiveness into mainstream organisations
  • Inability of small-scale evaluations to prove effectiveness

 

 

                           

The first case study – An evaluation of the Barnardos Whangarei In-home parent Mentor Programme – details an outcomes evaluation undertaken with Barnardos. The evaluation focused on how well the programme met its objectives in changing clients’ lives for the better (ie effectiveness).
 

The second case study – A process evaluation of the Children's Mentoring Programme at Pillars, Christchurch – explains the process evaluation undertaken with Pillars on their mentoring programme for children with a caregiver in prison. The purpose was to assess how well the programme worked for its clients (ie efficiency).

 

 

 

 

 

 

 

 
  The third report – Final lessons report on evaluative capability and preconditions for undertaking an evaluation – identifies the critical factors that helped and hindered the evaluations, as well as the systems and skills organisations should ideally have in place before undertaking a process or outcomes evaluation. A summary of these findings is found in the table below.

 

Last update: 9 Oct 2017