Collecting evidence

Expectations for evidence-informed policy and practice are rising.  To demonstrate effectiveness, you will need to think about the purpose of the evidence you are gathering. 

Large-scale evidence gathering is not needed for every initiative. Things to think about include the amount and trustworthiness of existing evidence, stakeholder requirements, the scale of your service or programme, and the value of current (and/or) future investment.

Good evidence should be obtained using reliable sources and a robust methodology. 

 

What is good evidence?

Section 4.5 of the Evaluation standards (developed in partnership with the Aotearoa New Zealand Evaluation Association (ANZEA) discusses methods for collecting evidence and trustworthy results.  

Finding and appraising evidence for what works provides guidance on appraising different types of evidence, as well as a list of sources of information on evidence-based programmes.

Module 3 of the Evaluation handbook discusses how change can be measured.

The Evidence rating scale looks at the strength of evidence for an initiative.   This sits alongside an accompanying document that presents international case studies for what works.

 

How to link and use existing data

There are many types of evidence, but the use of administrative data is becoming more popular. The Guide to using administrative data is a practical introduction to the subject.

The Government outcomes catalogue is a searchable catalogue containing social sector sources of data for the NZ government's priority programmes.  It is designed to help inform policy and programme development, and to develop outcomes and evaluation frameworks.

                                        

 

A body of facts or information that can be presented to inform judgement. Evidence can be quantitative or qualitative, and may come from various sources including performance monitoring, research, evaluation, statistics and expert opinion.

Evaluation planning for funding applicants - this step-by-step guide includes information on engaging stakeholders, establishing a purpose and describing your initiative.

Evaluation guide for funders – this guide promotes a collaborative approach to working with providers.

Evidence checklist – this list describes the scope of evidence, including evaluation planning.

Superu commissioned two evaluation case studies. The lessons learned report outlines what is needed to get an organisation ready to do an evaluation of their work. The evaluation findings for the mentoring programmes (one a process evaluation, the other an outcome evaluation) are also available.

Superu has also  commissioned a range of other evaluations (e.g. summative, cost-benefit, place-based) related to the Prime Minister's Youth Mental Health Project.

We know words matter. To help you out we have put together a list of terms that can be used to mean the same (or nearly the same!) thing, together with a brief explanation.

ANZEA - Aotearoa New Zealand's professional association for evaluators.

Information from MSD's community investment strategy about building your evidence base

Discover and use data at data.govt.nz

Good practice guidelines from Stats NZ

Information from Te Puni Kokiri on measuring performance

Types of data and sources are discussed at What Works NZ.

Evaluation Scotland has produced a resource about gathering and using other people's evidence

Last update: 4 Jul 2017