Evaluation Roundup - February 2020
New and noteworthy — Reads
Foundations’ evaluation and learning practices
The Center for Evaluation Innovation recently released its most recent review of foundations’ evaluation and learning practices. This report offers benchmarking data on foundation evaluation practice collected in 2019 from 161 foundations. What were the results from this year’s survey?
Here are some key takeaways:
1. Language is shifting - More job titles for foundation evaluation leaders include the word “learning”; while fewer contain the word “ evaluation”. Results also showed that nearly half of the foundations contained the name “learning” in the unit or department’s name.
I have definitely felt this shift, but is it a good one? I’m all about learning, but evaluation is a key component to strategic learning.
2. Evaluators are asked to do more with less – Despite larger foundations having more program staff the number of evaluators did not increase. In fact, the ratio of full-time staff to evaluation staff is widening (i.e. less evaluation staff are dedicated to more full-time staff). In addition, most foundation evaluators had responsibilities beyond evaluation as part of their work – an increase from 2015.
Evaluators have unique skills that can be used across organizations in a variety of capacities. Because of this, I know I am often pulled into tasks that may not be specific to my job as an evaluator but something my clients feel I should support.
3. Evaluation use is still an issue - Foundation staff are the primary intended users of evaluation efforts, over grantees and others in the field. Yet the biggest evaluation challenge faced is having evaluations result in meaningful insights for the foundation.
So, if the primary users aren’t using the results in a meaningful way and foundations aren’t engaging external stakeholders (either throughout the evaluation or even when it comes to sharing findings) then what’s the point? Is this what is driving the shift to focus on “learning”?
Learnings from early experiences of country-led SDG evaluations
The 2030 Agenda for Sustainable Development was adopted by UN member states in 2015. Since 2015, countries have been developing national Sustainable Development Goal strategies and action plans targeted at achieving the 17 Sustainable Development Goals (SDGs), but grappling with how to evaluate progress of such an enormous and complex agenda.
The International Institute for Environment and Development (IIED) recently published a guide to support country-led Sustainable Development Goal (SDG) evaluation. This guide offers learnings from early experiences of country-led SDG evaluations in Finland and Nigeria. The guide can be used to support evaluation commissioners and managers designing a national SDG evaluation.
New and Noteworthy — Resources
Creative ways to solicit stakeholder feedback
Public Profit recently released a guide evaluators can refer to when looking for alternate ways to capture feedback. Survey fatigue is a real thing and sometimes us evaluators need more creative, engaging ways to engage with our stakeholders. This guide provides step-by-step guidance on 15 different approaches, organized into three sections: visual, kinesthetic and verbal. It’s a very accessible guide that clearly outlines the activity’s purpose, gives an overview of how it works, and provides step-by-step instructions. They have also published similar guides for soliciting feedback from youth and virtually.
Strategizing and planning – Save the Children’s strategic foresight toolkit
Speaking of clear and accessible tools, you need to check out Save the Children’s recent strategic foresight toolkit – especially if you are doing any strategic planning. In this toolkit Save the Children compiles participatory techniques that combine evidence and creativity to help teams or organizations create collective wisdom and embed learning in decision-making. There are numerous templates and resources included in the toolkits. While they are not necessarily specific to evaluation, there are many that can be adapted. Check out the “Futures Wheel” - I can foresee myself using this to show outcome chains as opposed to traditional designs we are used to.
Your guide to the best Theory of Change software
Inspiring Impact did the work and outlined the different options out there to create the maps for theories of change. Their top four: 1) Google Drawings, 2) MindMup, 3) Coggle, and 4) Changeroo. The first three are not specific theory of change software, so you’ll need to know what you are doing; however, Changeroo was created to develop theories of change and has elements specific to theories of change. For example, when you click on a block in a theory of change you are prompted to identify the type of outcome and stakeholder it concerns. The drawback – it is most expensive of all the options.