EVALSDGs Blog: Transformative evaluation: tackling the complexity of the SDGs, from a new institutional economics perspective
By Denis Jobin, Eval SDG Co-Chair and Senior Evaluation Specialist, UNICEF Evaluation Office (2024)
In an era defined by ambitious global agendas, the Sustainable Development Goals (SDGs) stand out as a comprehensive blueprint to guide nations and development partners towards a more sustainable, equitable, and prosperous future for all. Comprising 17 interconnected objectives, with more than 231 indicators (1) the SDGs tackle the grand challenges of our time—from eradicating extreme poverty and hunger to combating climate change and working towards gender equality.
As we delve deeper into the journey towards 2030, the target year for achieving these goals, it becomes increasingly clear that the complexity of these goals demands more than just commitment; it also requires a revolution in how we approach our evaluation.
The SDGs are not only ambitious in their scope but intricate in their interdependencies. Actions taken to advance one goal can have profound impacts—both positive and negative—on others. This complexity underscores the necessity for enhanced evaluation approaches and methodologies that can navigate these nuances, providing insights that are not only comprehensive but also context sensitive. Traditional evaluation methodologies, while foundational, often fall short in capturing the dynamic interactions and the multifaceted nature of the sustainable development agenda. As we strive to make strides in this global endeavor, the call for innovative, adaptive, and integrative evaluation strategies has never been more critical. Hence, over the recent years, the pressing call for more transformative evaluation for transformative changes.
What is transformative change?
There is no clear definition of what transformative changes is, and thus transformative evaluation. For Patton and Felcis (2), there cannot be a universal definition of what transformation means, because it is ultimately contextual and emergent. Indeed “Transformation means deep, meaningful, and substantial systems change. A transformative trajectory conceptualizes the pathway to large-scale impact. But there can be no universal definition or specification of what constitutes transformation. Transformation is ultimately contextual and emergent.”
Underlying principles of transformative evaluation
While there isn’t any agreed definition, we can identify several principles underlying transformative evaluation:
- Inclusivity and participation: Emphasizing the importance of including diverse perspectives and value frame from most deprive communities and individuals, such as First nations communities, that are affected by collection action decisions and the change process in evaluative activities. The aim is to ensure that all voices, value frame of reference are heard and considered for determining what is working and for whom, and ultimately in contributing to social cohesion and what is our shared understanding of what is a just society.
- Empowerment: Seeking to empower individuals and communities by developing their capacities (skills, knowledge), and capabilities (abilities to put these capacities in motion and action), thereby enabling them to steer the changes what they need, based on their own needs, values and goals.
- Social justice: Focusing on reducing inequalities by changing the power dynamic within and outside communities, and how communities relate to the collective actions affecting them. This involves critically examining and challenging existing power dynamics with respect for individuals’ rights, dignity, and values. and structures embedded into institutions that shape human interactions.
- Learning and adaptation: transformative evaluations are one that are encouraging continuous learning and adaptation based on contextual insight and local context.
- Systems and holistic approaches: Recognizing the complexity of SDGs, and how they interwind, it is essential systems and the interconnections between different elements. Considering the whole system and the interdependencies within it, rather than isolated components, to foster comprehensive and lasting change.
- Long-term perspective: Adopting a long-term view that focuses on sustainable outcomes and considers the future implications of current actions.
In 2021, Van Der Berg, Magro and Adrian (3), noted the need for transform the way we approach evaluation. Indeed, adapting our evaluation methodologies is imperative to address the complexity inherent in transformational changes effectively. Transformations represent significant, systemic shifts that extend across multiple interconnected systems, characterized by complex dynamics that traditional linear and isolated evaluation methods fail to capture. These systemic changes, exemplified by historical shifts such as the agricultural revolution and the rise of industrial and information technologies, underline the intricate interplay between human societies, economic systems, and the natural environment. A systems perspective, acknowledging non-linear feedback loops and the cascading effects of actions across different domains, is crucial for understanding and guiding these transformations towards sustainable and equitable outcomes.
The unpredictable nature of transformational changes necessitates evaluation approaches that can navigate uncertainty and identify leverage points within and across systems. Historical instances of transformation, often unfolding without conscious direction, highlight the need for evaluative methods that inform strategic, proactive efforts to shape desired futures. By adopting comprehensive, systems-oriented evaluation strategies, we can better grasp the multifaceted impacts of transformations and contribute to guiding human societies through the challenges and opportunities of systemic change toward more sustainable, just, and resilient futures.
New institutional Economics offer opportunities informing evaluation approaches
In response to the complexities and interdependencies inherent in transformative changes, New Institutional Economics (NIE) offers a valuable framework for evaluating such shifts in a more holistic and comprehensive manner. By focusing on the roles and impacts of both formal and informal institutions on economic behavior and performance, NIE provides insights into how laws, regulations, policies, norms, traditions, and social practices collectively influence the outcomes of various institutional arrangements, including programs, policies, networks, and communities. This interdisciplinary approach, which integrates economics with law, organization theory, political science, sociology, and anthropology, aims to understand the multifaceted nature of social, political, and commercial institutions, offering a basis for examining institutional formation, purpose, evolution, and potential reform.
At the heart of NIE lies the analysis of how institutions, viewed as the primary units of analysis, come to be and evolve, shaped significantly by incentives that influence human behavior. The discipline encompasses several key theories, such as agency theory, transactions cost theory, and contract theory, which collectively contribute to a deeper understanding of institutional change. Through its comprehensive approach, NIE provides a robust analytical framework for examining the complexities of transformation, emphasizing the crucial role of institutions and incentives in navigating the challenges and opportunities presented by systemic changes. This perspective not only enhances our understanding of institutional dynamics but also underscores the importance of considering the broader institutional context in efforts to guide and sustain transformative change. There are limited applications in the evaluation realm, however some practical applications can be found here: Aid, Incentives and Sustainability – An Institutional Analysis of Development Cooperation – Main Report (7); and here: Application of game theory and new institutional economics in establishing a National Voluntary Organisation for Professional Evaluation in Nigeria (8) ; A Transaction Cost-Based Approach to Partnership Performance Evaluation (9); and Evaluation and Development The Institutional Dimension (10).
References
(1) SDG Indicators Global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development, extracted on March 31, 2024: SDG Indicators — SDG Indicators (un.org)
(2) Transformational Eval Week: Principles for Evaluating Transformation by Michael Quinn Patton and Weronika Felcis extracted on March 31, 2024 : https://aea365.org/blog/transformational-eval-week-principles-for-evaluating-transformation-by-michael-quinn-patton-and-weronika-felcis/
(3) Van den Berg, Rob D., Cristina Magro and Marie-Hélène Adrien (eds.). 2021. Transformational Evaluation for the Global Crises of Our Times. Exeter, UK: IDEAS. Extracted Marc 31 2024: https://ideas-global.org/wp-content/uploads/2021/07/2021-IDEAS-book-Transformational-Evaluation.pdf
(4) Mertens, D. M. 2009. Transformative research and evaluation. New York: Guilford Press. & Mertens, D. M., F. Cram and B. Chilisa (eds.) 2013. Indigenous pathways into social research. Walnut Hills, CA: Left Coast Press.
(5) Picciotto, Robert 2015. Democratic evaluation for the 21st century. Evaluation 21 (2): 150–166.
(6) Mazzucato, Mariana 2018. The value of everything. Making and taking in the global economy. Penguin.
(7) Elinor Ostrom (team leader), Clark Gibson, Sujai Shivakumar, Krister Andersson. Aid, Incentives, and Sustainability An Institutional Analysis of Development Cooperation Extracted mar h 31, 2024 : https://cdn.sida.se/publications/files/sida1598en-aid-incentives-and-sustainability—an-institutional-analysis-of-development-cooperation—main-report.pdf
(8) Jobin, D., & Lawal, Z. (2017). Application of game theory and new institutional economics in establishing a National Voluntary Organisation for Professional Evaluation in Nigeria. African Evaluation Journal, 5(1), 10 pages. doi: https://doi.org/10.4102/aej.v5i1.197
(9) Jobin, D. (2008). A Transaction Cost-Based Approach to Partnership Performance Evaluation. Evaluation, 14(4), 437-465. https://doi.org/10.1177/1356389008095487
(10) Evaluation and Development The Institutional Dimension By Eduardo Wiesner https://www.routledge.com/Evaluation-and-Development-The-Institutional-Dimension/Wiesner/p/book/9780765804235