Danish Management was in 2015 awarded the contract for the OECD-DAC Review of Evaluation Systems for overseas development assistance. The result of this review, an official OECD publication is now available online and in the OECD bookshop in Paris.
The purpose of this review was to provide information to assist development organisations in building effective evaluation systems and processes to deliver high quality, credible and useful evaluations. It provides information about evaluation systems in development agencies and multilateral organisations, and analyses experiences from managing evaluation systems across organisations.
The data that informs the review was collected through a combination of a desk review of member evaluation systems as described in their guideline documentation, interviews with agencies and organisations that have recently implemented significant changes, and an online survey of all members of the Evalnet Network. 40 evaluation units participated in the review and provided very valuable input that is incorporated in the Evalnet Member profiles for each agency/ organisation.
The review identified recommendations to improve the effectiveness of the evaluation of development assistance, including:
- Ensure the relevance of evaluation by aligning it to needs of other units: the relevance of evaluation findings is linked closely to the timing of evaluations. When the work programme is defined to fit in with the needs of operations units at the right time it is likely to ensure that evaluations remain relevant.
- Enable programme designers to ‘think’ evaluation: although there is some discussion about the role of evaluators in project formulation and the implications this has on independence, it is clear that by developing the capacity of operations units in evaluation methods and principles, the evaluability of programmes, the quality of the evaluations themselves will improve.
- Increase partners’ participation to get long-term benefits: the participation of development partners is acknowledged as being a fundamental principle for evaluation, despite the challenges of doing so. Engaging development partners from the inception of a project/ programme is considered to be valuable to encourage sustainability and ownership, and their engagement in the evaluation process furthermore improves the quality of the evaluations performed for example through better analysis of the findings and uptake of the recommendations.
- Increase public disclosure: publication of evaluation findings serves the principle of transparency but also promotes recognition for achievements and increases the influence that evaluation has in an organisation. A number of members reported that obligatory public disclosure resulted in the improvement of the quality of the reports, their usability and the feasibility of recommendations, as well as the improved selection of evaluation topics. This also provided more credibility and legitimacy to the evaluation systems and thus enhancing their effectiveness.
- Target the communication of evaluation results to the audience: dissemination and knowledge sharing requires an understanding of the target audiences, their needs and the most effective media for that audience. Establishing an organisation-wide communications and knowledge management strategy that specifies the type of materials that will be produced and shared and in what forms may improve the use of evaluations.
- Develop an evaluative culture: several members are working to develop an evaluative culture. This poses a significant challenge, but there are elements of the evaluative culture that may be beneficial to introduce while developing a broader strategy. The use of self-evaluation is suggested to encourage internal reflection and learning, and the incorporation of an organisational approach to knowledge management would begin to encourage an evaluative culture.
To download or order a copy of the review, please visit the OECD library here.