Evaluation as a reinforcer of democratic ideals

This is the first in a series of blogs focusing on the future of evaluation. We start by looking at how post-normality challenges the traditions of evaluation and how evaluation reinforces the realisation of democratic ideals in our era.

The aim of evaluation is to build a better world. Evaluation has evolved as a result of the modernisation of society, and its purpose is to promote openness and transparency in governance and policymaking as well as the development of operating methods. Evaluation produces knowledge to support decision-making based on the collection and analysis of information and, as such, it is part of the democratic system. In an ideal world, the knowledge produced by evaluation would also lead to informed and carefully considered decisions and policies.

The social structures on which evaluation has been built are undergoing a transformation. Veteran evaluation theorist Thomas A. Schwandt has described our shift from normal evaluation to post-normal evaluation. “Normal” evaluation has been guided by the notion of scientific rationality. It has been focused on promoting the efficiency and effectiveness of interventions and the strengthening of social progress and modernisation. Evaluation has served policymakers by producing knowledge concerning the success of policies aimed at tackling social challenges and assessing their impacts.

However, today’s political environment is not as predictable as it used to be. Relying on historical knowledge and evaluating past interventions and policies cannot be the primary strategy for achieving the knowledge for future interventions. This raises the question of how evaluation can promote and build the fair and sustainable society of the future.

How does the post-normal era challenge evaluation to evolve?

We highlight three perspectives for assessing the role of evaluation in supporting social development.

1. The ethical duty of evaluation is to build a comprehensive understanding of the evaluand

Over the past few decades, a multidisciplinary community of experts has grown up around evaluation, along with a wide range of approaches and methods. In the post-normal world, however, expertise and methods no longer have the same intrinsic value as before.

The value of evaluation methods is determined by the reliability and usability of the information they produce regarding the evaluand. You need the ability to choose and apply methods that give justice to the phenomena being evaluated and get to the heart of their nature and special features. The practice of evaluation is increasingly characterised by the defining meta-trends  of post-normality, such as the interconnected, systemic and complex nature of phenomena.

In practice, this means that the methods intended for the evaluation of linear phenomena do not sufficiently get to the heart of complicated cause-and-effect relationships. Giving due consideration to complexity and uncertainties is not only a technical issue, but also an ethical issue regarding the extent to which evaluation can shed light on reality and that which is significant.

The use of multi-criteria and mixed-method approaches is part of credible and reliable evaluation. Nevertheless, it is not possible to analyse any phenomenon exhaustively and from every perspective. To that end, it is necessary to draw boundaries, and the transparency and justification of these boundaries will become increasingly important in the post-normal era.

Global megatrends ranging from accelerating climate change to growing inequality challenge evaluation frameworks and competences to evolve. Blue Marble Evaluation is a good example. Its focus is on the evaluation of sustainability, fairness and systemic change. Evaluations of the UN’s Sustainable Development Goals (Agenda 2030 and SDG) are aimed at tackling the same challenges.

2. The significance of evaluation in decision-making

Wicked problems and the transformation of the knowledge environment compel us to rethink the use of knowledge in public decision-making. This broader phenomenon also presents a challenge to evaluation. The value of evaluation in decision-making has traditionally emerged largely from the independent role of evaluation, which has been perceived as ensuring the neutrality and objectivity of knowledge. However, for the knowledge produced by evaluation to be effective and support the realisation of democratic ideals, it cannot be separate from the rest of the world.

According to Thomas A. Schwandt, evaluation must be perceived as a systemic intervention that produces knowledge that serves practical reasoning. For this to be achieved, interaction will become increasingly important in evaluation. Schwandt emphasises that the purpose of evaluation is to increase our understanding of the phenomena being evaluated. Evaluation may help us determine how and why we need to act to make things better.

And what about future orientation and the time span of evaluation? How could evaluation better take into account potential future developments to ensure that it will not be outdated in terms of usability as soon as it has been completed? In our view, this is an issue that concerns not only the timing of evaluation but also new future-oriented approaches and the interface between evaluation and foresight.

3. Questions of values and power are part of evaluation

One feature of liberal democracy is the freedom to assess the premises, objectives and results of public action. Evaluation must take into consideration the power structures and various interests related to decision-making and, for example, determine whose interests and values drive policy decisions. In the international dialogue on evaluation, this is reflected in approaches that emphasise sensitivity to various socio-cultural power structures.

Liberal democracy requires free media that has access to the kind of neutral and independent information produced by evaluation. At the same time, it is important to note that the media also plays a key role in the transformation of democracy. Anders Hanberger (2018) has analysed the relationship between evaluation and democracy. As part of the analysis, he addressed the mediatisation of evaluation or, in other words, how the logic of media influences the process and results of evaluation, either consciously or inadvertently. At present, democratic evaluation is poorly prepared for managing current threats to democracy, increasing polarisation and the mediatisation of politics and governance.

We believe that in the future, evaluation will continue to play an important role in the development of liberal democracy. Nevertheless, strengthening the role of evaluation in society calls for a clarification of the core competence of evaluation, which concerns values and valuing.

In these post-normal times, it is increasingly important to bring visibility to the values and criteria that guide the objectives of interventions and against which their success is assessed.

Further reading:

Hanberger, Anders (2018), “Rethinking democratic evaluation for a polarised and

mediatised society”, Evaluation, 24 (4), 382-399.

Schwandt, Thomas A. (2019), “Post-normal Evaluation?”, Evaluation, 25(3), 317-329.

Link: https://journals.sagepub.com/doi/abs/10.1177/1356389019855501

Patton, Michael Quinn (2019), “Expanding Futuring Foresight through Evaluative Thinking”, World Futures Review, 11 (4), 296-307.

Link: https://journals.sagepub.com/doi/full/10.1177/1946756719862116

Patton, Michael Quinn (2019): Blue Marble Evaluation: Premises and Principles. Guilford Press, New York. (Published 1 November 2019)