[Front page] [Contents] [Previous] [Next] |
Project Cycle Management Manual
5. Evaluation
5. Evaluation
5.1 Framework for Evaluation
5.2 Evaluations Carried
Out by External Consultants
5.2.1 Terms of Reference
5.2.2 Contracting
5.2.3 Implementing the Evaluation
5.2.4 The Evaluation Report
5.3 Lessons Learned
5.4 Inspection and Examination
of Projects
5. Evaluation
Evaluation
Evaluation is a systematic and objective assessment of the design, implementation and
results of an on-going or completed project or programme. The aim of the evaluation is to
determine the efficiency, effectiveness, impact, sustainability and relevance of the
project. Evaluations are undertaken at the initiative of DANCEE.
The evaluations of DANCEE supported projects serve two purposes:
 | To improve the quality of DANCEE support through better knowledge of the content and consequences of projects; |
| To document the results to the political authorities and the general public. |
Reviews
Reviews are midway between monitoring and evaluation, as they involve a fresh look at
the objectives, design and performance of a project. Compared to evaluations, reviews are
more limited in scope, time and focus. Reviews are used to verify the relevance and
efficiency of the project. The focus of the review is in particular on any needs for
adjustments of the project. Reviews are also carried out at the initiative of DANCEE.
The term "evaluation" is used below, when speaking of reviews and
evaluations.
5.1 Framework for Evaluation
Evaluation procedures
DANCEE applies the same evaluation procedures for tender and application projects.
DANCEE operates with the three types of evaluations shown in Table 5.1 below.
Table 5.1: Structure for Project Evaluation
| Type of Evaluation | Timing | Focus |
| Mid-term Review[4] | Mid-term | The project’s relevance Efficiency of activities The project’s effect The project’s impact |
| End-of-Project Evaluation | Last project year | The project’s effect The project’s impact Sustainability of project interventions |
| Ex-post Evaluation | 2 - 5 years after completion | Sustainability of project interventions |
Mid-term reviews
Mid-term reviews are carried out on projects with a duration of more than three
years, on complicated and complex projects and on projects where problems have been
identified. Mid-term reviews are carried out by a team of external consultants appointed
by DANCEE. The mid-term review will verify the relevance, sustainability and efficiency of
the project and will recommend relevant adjustments.
End-of-project evaluations
End-of-project evaluations will as a minimum be carried out as a desk evaluation by the
MoEE representative. End-of-project evaluations may also be carried out by a team of
external consultants appointed by DANCEE. The aim of the end-of-project evaluation is to
assess what lasting impact the project is likely to have. The focus of the end-of-project
evaluation is on the effectiveness of the project and the impact of the project.
Furthermore, the end-of-project evaluation will assess the sustainability of the project.
Ex-post evaluation
DANCEE will normally carry out ex-post evaluations of groups of projects. The ex-post
evaluation will focus on the sustainability of the projects' impacts. Ex-post evaluations
will be carried out by a team of external consultants appointed by DANCEE, 2-5 years after
project completion.
Furthermore, DANCEE and its representatives are entitled to carry out inspections and examinations of projects, as described in Section 5.4.
5.2 Evaluations Carried Out
by External Consultants
5.2.1 Terms of Reference
ToR
ToR for reviews and evaluations carried out by external consultants are developed by
DANCEE. The ToR defines the objectives of the evaluation and indicates the issues which
DANCEE finds of particular importance. In addition, information is provided on the
background for the evaluation, the composition of the evaluation team and the timing of
the evaluation. A standard outline of the DANCEE ToR is shown in Table 5.2.
5.2.2 Contracting
The evaluation team
An external review or evaluation team will consist of 1-4 persons. The team can
include both international consultants and consultants from the partner country, all the
members being selected for their technical expertise. The team will present their
independent views on the project.
The beneficiary
A representative from the beneficiary will act as a resource person for the
evaluation team and will ensure that lessons learned are discussed with the beneficiary.
The team leader
DANCEE will sign a contract with the evaluation team. The ToR forms part of the
contract. The contract will specify the team leader of the evaluation. The team leader
will be responsible for the implementation of the evaluation and for the final reporting.
DANCEE will hold a briefing meeting with the team. The purpose of the meeting is to ensure that the team has the broad understanding of the project to be evaluated, to identify needs for further documentation and to discuss the ToR.
5.2.3 Implementing the Evaluation
The evaluation team
The evaluation team is responsible for planning and undertaking the evaluation. The
team leader will be responsible to DANCEE but will delegate assignments to the team
members. The local project coordinator will upon request assist with the planning of a
programme, visits and meetings for the evaluation team.
Local project coordinator
Upon arrival in the partner country the team will consult with the local project
coordinator and will brief MoE on the programme for the evaluation.
Table 5.2: Standard Outline of DANCEE's ToR for evaluations
| Main Elements in the ToR | |
| Project background | Project context and rationale Brief description of the project logic (the objectives and outputs of the project LFA) |
| Objective of the evaluation | Reasons related to formal decisions Reasons related to project performance The intended use of results |
| Scope and methods | The scope of the evaluation The type of analysis and method to be used and the degree of detail |
| Issues to be covered | Extent to which all or only selected areas of analysis and
issues are to be covered (efficiency, effectiveness, impact, relevance and sustainability) Extent of specialisation |
| Evaluation team | Number of team members and their roles Required professional background and experience |
| Timetable | Approximate dates and allocation of time prior to, during and after the field work |
| Consultations in the field | Authorities, institutions and groups to be consulted during and at the end of the field work |
| Reporting | Deadlines for draft and final reports Specification of technical standards for final report |
| Documentation | Project document Project reports Relevant background reports |
Debriefing note
Before departure from the partner country, the team will debrief MoE. The head of the
evaluation team will present a debriefing note to MoE. This note will form the basis for a
discussion with MoE, the beneficiary and other stakeholders about the results of the
evaluation.
The debriefing note should not exceed ten pages and will specify the findings and recommendations of the evaluation team. An outline for a debriefing note is shown in Figure 5.1 below.
Figure 5.1: Outline of debriefing note
| Date and place of meeting Team composition Introduction:
Activities undertaken Key findings:
Recommendations Assumptions |
5.2.4 The Evaluation Report
The evaluation team will write its evaluation report based on the findings and recommendations outlined in the debriefing note. The team will prepare a draft final report, which will be scrutinised by DANCEE before the final report is produced. The final report should not exceed 50 pages.
The format of the evaluation report is outlined in Table 5.3. The importance accorded to each of the points should reflect the ToR. The report will focus on the achievements of the project.
The evaluation team will comment on positive and negative aspects of the project, conclusions being operational and action-oriented.
DANCEE shall approve the draft evaluation report in writing to the team leader within 20 working days after reciving the draft.
5.3 Lessons Learned
The application of the evaluation results
One of the main purposes of carrying out an evaluation is to applicate the evaluation
results and lessons learned to new projects.
In the case of a mid-term review, the final evaluation report is given to the steering committee. The steering committee will review, approve or disapprove, and implement recommendations. DANCEE is responsible for submitting the final evaluation report to the project holder and the steering committee.
Final or ex-post evaluation
In an end-of-project or ex-post evaluation, the emphasis is on lessons learned for
new projects and for future strategies. End-of-project evaluations will be discussed at
the final steering committee meeting. The steering committee will discuss the lessons
learned and will suggest adjustments to the beneficiary to ensure sustainability of
project results. DANCEE and MoE will discuss the outcomes of ex-post evaluations with
relevant stakeholders.
5.4 Inspection and Examination of Projects
DANCEE and its representatives shall be entitled to inspect and otherwise examine projects, including (but not limited to) all and any accounts, receipts, contracts, and to make on-site inspections, during and after completion of the project. The project holder and the beneficiary are required to grant access and render the assistance necessary for such inspection and examination.
These provisions also apply to investigations conducted by the Office of the Auditor General of Denmark.
Table 5.3: Evaluation Report Format[5]
| Table of contents | ||||||||
| I. Cover sheet II. Map of location III. List of abbreviations IV. Executive summary |
||||||||
| 1. Introduction 1.1 Background for the evaluation 1.2 Brief description of the project 1.3 Evaluation methodology including general approach used, main sources of data, main instruments, professional profile of evaluation team and limitations associated with methodology and approach 1.4 Structure of report |
||||||||
| 2. Project relevance 2.1. Rational and context of the project at its inception 2.2. Changes in project context during implementation 2.3. Relevance of project in relation to partner country priorities 2.4 Relevance of project to country programme and complementarity to support from other donors 2.5 Relevance to DANCEE priorities |
||||||||
| 3. Efficiency 3.1. Project progress compared to plans 3.2. Cost and utilisation of resources compared to budget and plans 3.3. Achievements of outputs 3.4. Outputs in relation to resource utilisation |
||||||||
| 4. Effectiveness 4.1. Expected achievements at project start 4.2 Actual or expected achievement at time of evaluation 4.3 Factors and processes affecting achievements |
||||||||
| 5. Impact 5.1. Priorities, needs and demands of beneficiary 5.2. Foreseen and unforeseen impacts on the environment 5.3. Foreseen and unforeseen impacts on the beneficiary 5.4. Other major impacts of the project 5.5. Factors and processes which explain project impacts 5.6. Environmental impacts in a short term, medium term and long term perspective |
||||||||
| 6. Sustainability 6.1. The extend to which the project impact is/will be sustainable 6.2. The extend to which the beneficiary can maintain project results 6.3. Factors affecting sustainability (political, institutional, economic, technological, socio-cultural and environmental) |
||||||||
| 7. Lessons learned 7.1. Operational lessons related to the project itself 7.2. Developmental lessons related to the societal and environmental consequences of the project 7.3. The most outstanding result or issue of the project 7.4. The most problematic issue of the project |
||||||||
| 8. Conclusions and recommendations 8.1. Conclusions 8.2. Recommendations |
||||||||
Annexes
|
||||||||
Notes:
- Only for projects with a duration of more than three
years or in the case of projects where problems have been identified.
- The format is meant as a checklist. The evaluation team shall focus on issues of special importance to the project being evaluated.
[Front page] [Contents] [Previous] [Next] [Top] |