Written by Denis Jobin, EVALSDGs Co-Chair, Senior Evaluation Specialist, UNICEF Evaluation Office, and Serena Rossignoli, PhD, Head (Ag.), Office of Independent Evaluation, Caribbean Development Bank. 6 January 2025
Achieving the Sustainable Development Goals (SDGs) requires evaluations that are both independent and collaborative. Independence ensures objectivity, allowing evaluators to conduct their work without undue influence while engaging with management enhances relevance and credibility. This balance fosters trust and actionable insights, driving effective governance and sustainable development. Balancing independence with meaningful engagement is essential for producing evaluations that are both credible and impactful in achieving SDGs.
Evaluations are not just tools for accountability; they are powerful mechanisms for learning and improvement and, as such, instruments of public governance capable of supporting the achievement of SDGs. Yet, their impact depends on how well evaluators and management work together to translate findings into actionable improvements and advance the organization’s missions through its programming. It’s not about evaluation only, but first, it’s about the program and organizational mandate.
In our experience, the balance between independence and engagement is critical, and fostering trust and credibility between these functions is essential to creating a culture focused on achieving results, enhancing organizational learning, and achieving mission objectives.
This interplay between independence and collaboration, which we experienced in multilateral IFIs (international financial institutions) or government and consulting settings, mirrors the insights shared by the International Organization of Supreme Audit Institutions (INTOSAI) an organization that deals with independence for far longer than professional Evaluators, on the same topics: independence is not an end, but a mean to add value to management and programing, in a credible manner. We are referring mainly to principles from supreme audit institutions (SAI) work such as the Lima Declaration P-1[i] (Importance of open communication between evaluators (auditors) and management (auditees entities)), Mexico Declaration P-10[ii] (while independence helps maintain objectivity and credibility, its importance engaging with management during the evaluation process to maximize relevance and impact) and INTOSAI GOV 9020[iii] (importance of engaging with management during the evaluation process). Drawing from these principles and bridging experience from supreme audit institutions, this blog explores how effective evaluation partnerships, built on trust, independence, and credibility, can elevate organizational impact by sharing insights derived from our personal experience.
Striking the Balance: Independence and engagement
One of the key challenges in evaluation is maintaining the delicate balance between independence and engagement. Evaluators must remain objective to provide credible evidence, yet they cannot afford to operate in silos. A lack of engagement with management risks producing findings that need to be addressed or understood. So, we believe in having open communication, listening to each other, and agreeing to disagree with clear rationale with substantiated arguments.
INTOSAI emphasizes that independence is not isolation. Similarly, in evaluation, independence ensures the credibility of findings, while constructive engagement ensures their relevance, applicability, and impact.
Building Trust Through Transparent and Credible Engagement
Trust is the cornerstone of an effective evaluator-management relationship. For Kenneth J. Arrow, trust is the necessary lubricant for societal and government transactions. Trust is also necessary for even the most rigorous evaluations to gain traction, as it is needed not only in the evaluation product but also in the evaluation process leading to it. In return, transparency in methods, generating findings, and co-creation of recommendations fosters this trust. Indeed, Evaluators, like auditors, must clearly communicate their methodologies, assumptions, and limitations. This transparency not only enhances the credibility of the findings but also encourages management to view evaluations as tools for improvement rather than criticism.
In practice, we have found that trust is built through consistent, open communication. When management understands the evaluation process and sees evaluators as partners in achieving organizational goals, they are more likely to act on the findings.
To ensure building trust, it’s important in our experience to engage very early in the planning and design of evaluation activities both at the plans level, informing priorities and evaluation investments independent decision, and activities level informing purpose, scope, and foci.
Collaboration as a Catalyst for Learning: early use of reference group and co-generation recommendations workshop
Organizational learning is a shared responsibility. Evaluators bring evidence and analysis, while management brings operational insights and implementation capacity. Together, they can co-create actionable solutions.
Our experiences support the fact that collaboration fosters a culture of accountability and learning. When evaluators and management engage early in the evaluation process, they can align expectations, ensure the relevance of evaluations, facilitate access to relevant data, and co-develop recommendations that are practical and aligned with organizational priorities. The Office of Independent Evaluation (OIE) at the Caribbean Development Bank is implementing a new evaluation process that emphasizes early and continuous collaboration with management and operations. By adopting consultative and collaborative approaches, management and operations are actively engaged from the design phase onward. To support the development of inclusive evaluations, OIE is establishing Focal Points (FPs) and Evaluation Reference Groups (ERGs).
Evaluation Reference Groups (ERGs) are composed of internal stakeholders, such as representatives from management and operations, and, where relevant, external stakeholders, including thematic and regional experts. ERGs serve as the primary counterparts to OIE throughout the evaluation process, providing inputs from the design stage to the finalization of evaluation reports. Their involvement enhances the relevance of the evaluation process and fosters sustained collaboration across all phases. ERGs play a key role in shaping evaluation questions to address areas of interest, including challenges and positive insights, and in co-developing actionable recommendations.
The ‘Engagement Lab’ provides a dedicated platform for collaboration among evaluation consultants, OIE staff, and ERGs to jointly determine the most relevant and actionable recommendations based on evaluation findings. OIE leads this collaborative effort, facilitating dialogue, guiding exchanges, and promoting open discussions to ensure a balanced approach that upholds the independence of evaluations while fostering inclusivity.
The appointment of Focal Points (FPs) further supports collaboration with operations. FPs have a dual role: facilitating access to key documents and stakeholders to support the evaluation process and contributing to the accuracy of evaluation findings.
Both ERGs and FPs are instrumental in the evaluation review phase, offering input on the accuracy of evaluation reports and sharing their perspectives on findings and recommendations. During this final step, OIE provides additional information and clarifications as needed, determines whether to accept comments and explains decisions when certain suggestions are not incorporated into the final report. This iterative process ensures transparency and reinforces the credibility and utility of evaluations.
In our experience, structured collaboration mechanisms—such as early engagement in evaluation planning, data collection and analysis, and post-evaluation debriefs—serve as critical touchpoints for alignment. For example, in UNICEF, we insist on organizing a co-generation recommendation workshop, where the findings of facts are reviewed against the analytical framework and criteria using evaluative reasoning to formulate robust yet sensible conclusions. The following principles guide us, adapted from the Office of Auditor General of Canada Value for Money audit Manual[iv]:
- Evaluators should make recommendations that flow logically from the observations, the findings, and conclusions that are directed at resolving the cause of identified deficiencies and clearly state the actions recommended.
- Effective recommendations encourage improvements in management and operations. Recommendations are not always required or mandatory.
- Recommendations are effective:
-
- thought out during the examination phase;
- entity-specific, even in government-wide and sectoral audits;
- focused on areas of significant risk;
- fully supported by the audit findings and conclusions;
- consistent within the audit report and mindful of recommendations made in previous audit reports, where applicable;
- positive in tone and content;
- clear on the desired final outcome;
- succinct but detailed enough to stand alone;
- aimed at correcting the underlying causes of the weakness;
- results-oriented, giving an indication of the intended outcome;
- practical, such that the entity, taking into account legal and cost constraints, can implement them in a reasonable time frame; and
- specific enough to allow for monitoring and assessing progress made in implementing them—but not overly prescriptive.
These principles help bridge the gap between evaluators’ findings and management’s operational realities.
Lessons for Enhancing Organizational Impact
Drawing from our experiences, the following strategies can strengthen the evaluator-management relationship, and we share some concrete experiences.
- Early Engagement: Involve management early in the evaluation process to align on objectives, scope, and priorities. By engaging management early in the design of the TORs/Conceptual notes, management can inform the scope of the evaluation to enhance use and utility. Reference groups in CBD and UNICEF are good places to do so.
- Structured Dialogue: Establish regular touchpoints, such as debriefs and feedback loops, to ensure continuous alignment. We make a point of clarifying data and observations from findings of fact (data analyzed against criteria, analytical framework, and triangulated) and from the conclusions (the most direct answer to the evaluation questions), which uses evaluative reasoning. By keeping these elements clear, we ensure a structured dialogue with management and build trust and credibility along the way.
- Focus on Learning: Emphasize the role of evaluations as tools for learning and improvement rather than mere accountability exercises.
- Foster a Culture of Trust: Build relationships based on mutual respect, shared goals, and open communication.
Practical evaluation is as much about relationships as it is about evidence. By fostering trust, maintaining independence, and engaging constructively, evaluators and management can collaborate to drive meaningful improvements. The lessons shared above provide a valuable blueprint for balancing these dynamics and enhancing organizational learning and mission achievement.
As we continue our journey to refine our evaluation practices, one truth remains clear: the most impactful evaluations are not only those in which evidence meets some quality standards but those in which true collaboration transforms credible findings into progress.
[i] https://www.issai.org/pronouncements/intosai-p-1-the-lima-declaration
[ii] https://www.intosai.org/fileadmin/downloads/documents/open_access/INT_P_1_u_P_10/INTOSAI_P_10_en_2019.pdf
[iii] https://www.issai.org/pronouncements/guid-9020-evaluation-of-public-policies/
[iv] https://www.oag-bvg.gc.ca/internet/methodology/performance-audit/manual/index.shtm