How do we future proof an evaluation approach?

With the world soon entering the third COVID year, more unpredictable, uncertain futures lay ahead of us. Evaluation can be a critical change agent, towards a more future-focused and resilient development path and society. This in mind we have been developing the new Sitra (the Finnish Innovation Fund) evaluation framework and approach in the past months.

In the same spirit, Sitra invited a group of like-minded evaluation enthusiasts to discuss and exchange ideas about where evaluation is heading to, and what is needed for making this path more transformational. In short, the discussions helped to ‘future-proof’ the Sitra evaluation approach, making sure the new framework embraced adequately the main ingredients of transformative evaluation function.

By “transformational evaluation” we mean the potential of evaluation to understand, analyze and accelerate equitable change. Evaluation can be powerful and play an important role as a change agent because evaluations are widely conducted and findings used by businesses, decision-makers in the public sector, research communities and the civil society, amongst others. Evaluation can support in the sense-making of complex issues as well as decision-makers’ timely and relevant action.

So, what is it that is making us rethink evaluation?

Even before COVID, we were looking at an ever-increasing unpredictable, uncertain, and complex development trend that partially exceeded any trend anticipation and partially lagged behind because of a myriad of reasons, including polarization, inequity and disruption. With COVID, the role of backward-looking evaluation further depreciated, and the focus has shifted to timely, developmental, transformational and learning-focused evaluation, that would make us more resilient for the unknown.

Meanwhile, future-focused organizations and evaluators have already started the dialogue on infusing foresight into evaluation, which is also one of the determining ideas in the new Sitra evaluation approach and framework.

And what is it that we need for a more transformational evaluation?

As a starting point, we see transformational evaluation as learning-focused and participatory. These below-listed ingredients are not meant as an exhaustive list and are presented as perspectives for consideration.

1. We need to focus on evaluating systems change. Transformational evaluation builds on contextuality – meaning evaluators connecting with and analyzing better the complex, interconnected system in which the intervention is being implemented. From the perspective of evaluating systems change, it is critical to find the right methods and approaches to capture the unpredictability, nonlinearity and dynamism of systems change.
2. Marrying evaluation with foresight. As can be expected, evaluators cannot imagine evaluation being powerful and transformative without an infusion of foresight into evaluation work. This refers to the potential of a more future-proofed evaluation, where methods, evaluation questions, and recommendations do not lean on a predetermined singular future, but are adaptable to more unstable, multi-scenario and fast-changing (eco)systems.
3. Eco-efficiency! Transformational evaluation needs to support the sustainability agenda by addressing eco-efficiency – there is hardly an intervention or phenomenon out there which does not intersect or interface with the climate crisis and biodiversity loss. This relates also to the use of broader (revised OECD) definition of impact and moving towards cross-impact analysis where possible.
4. Diversity, equity, and inclusion should have an increasingly significant role in transformational evaluative work. For evaluation to be powerful, it needs to be power challenging, and examine systemic and structural barriers hindering positive, inclusive change. Through evaluation we can strengthen policies and programs that promote the representation of different groups.

+1

We also need young, diverse evaluators with evaluative mindsets and appetite for innovative methods from across the globe to embrace transformational evaluation!

Ps. In our next blog, we will write about how Sitra has processed these ingredients into a future-proofed evaluation framework.

Sitra would like to thank the thought contributors to this blog: Thomas Archibald, Henrik Brinkmann, Stefano D’Errico, Annette L Gardner, Hallie Preskill, Petri Uusikylä, Katri Vataja (Sitra).

Further reading:

Van den Berg, Rob D., Cristina Magro and Marie-Hélène Adrien (eds.). 2021. Transformational Evaluation for the Global Crises of Our Times. Exeter, UK: IDEAS.

Patton, Michael Quinn (2020): “Evaluation Criteria for Evaluating Transformation: Implications for the Coronavirus Pandemic and the Global Climate Emergency”. Journal of American Evaluation.

Hallie Preskill, Srik Gopal, Katelyn Mack & Joelle Cook (2014), Evaluating Complexity: Propositions for Improving Practice, FSG.

Patton, Michael Quinn (2010): Developmental Evaluation. Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press, New York.

Vataja, Katri, Dufva, Mikko & Parkkonen, Pinja 2019. ”Evaluating the Impact of a Futures-Oriented Organization”. World Futures Review, 11 (4), 320–339.