How do we Evaluate Programmes?
Planning the M&E action
In every project/programme, there must be a specific activity of Monitoring and Evaluation which has its outputs and indicators.
Monitoring tracks and measures the progress towards achieving an output, outcome or impact. However, the goal cannot be measured directly. It must first be translated into a set of indicators that, when regularly measured, will provide information whether or not the output, outcome or impact are being achieved. This is especially important when the outcomes of an action are interlinked with outcomes of other actions and together these feed into the outcomes and impacts of a wider programme.
Monitoring is conducted by the concerned staff for outputs and activities falling under her/his specific responsibility within the purview of the logical framework. For each activity, the concerned staff must enter information into specifically developed Monitoring Formats that are aligned with her/his annual workplans.
Each month (or every week if it is a rapid emergency response intervention through sitreps or "situation reports"), the project manager compiles together the monitoring information of various staff and sends it to the Programme Manager. This is the monthly report of the Project Manager. Each monthly report, besides sending the monitoring information, must also explain the major issues emerging during project implementation. This monitoring information feeds into in-itinere evaluations and finally into ex-post evaluations.
Evaluation can be conducted either:
There are three main kinds of evaluation:
Although an in-itinere evaluation can be facilitated by an external expert, it is usually a process of self-evaluation (also called mid-term review) conducted by staff within the organisation during the implementation phase. The main objective of such an evaluation is to empower the implementing team in assessing its project/programme and taking corrective measures in time to improve performance. In order to improve the performance of a project/programme, the distinction between objectives, outputs and activities must be elaborated and demonstrated.
An in-itinere (formative) evaluation, therefore, focuses on studying and correcting the logical coherence of the relationship between project objectives, outputs, activities and inputs such that the project's aim of bringing in the lives of the beneficiaries becomes achievable. In order that an in-itinere evaluation be comprehensive and effective in improving project performance, following are the steps to be taken by the evaluator:
The main result of the activity of Monitorign and Evaluation is the empowerment of the implementing team in conducting effective self evalaution.
This approach to evaluation is based on the Results-Based management framework that defines monitoring as a continuous process of collecting and analyzing information, and comparing actual results to expected results in order to measure how well a project, program or policy is being implemented. It defines evaluation as an assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact, and sustainability. The intention of such an evaluation is to provide information that is credible and useful, enabling incorporation of lessons learned into the decision making process of recipients.
The outputs of the M&E activity will be:
Results based Monitoring and Evaluation
A results-based M&E system provides crucial information about the performance of an organization. It can help policy makers, decision makers and other stakeholders answer the fundamental questions of whether goals were achieved and can help demonstrate improvements. By reporting the results of various interventions, organizations can promote credibility and public confidence in their work. Such practices also support the development agenda, among major donors like EC, CIDA, DFID, World Bank, that is shifting towards greater accountability for aid lending. A good results-based M&E system can be extremely useful as a management and motivational tool. It can help focus people’s attention on achieving outcomes that are important to the organization and its stakeholders and provide an impetus for establishing key goals and objectives that address these outcomes.
Results based M&E also provides managers with crucial information on whether the theory of change guiding the intervention is appropriate, correct, and adequate to the changes being sought through this intervention. Once targets are established and the organization is striving to achieve them, the M&E system can provide timely information to staff about progress and can help with the early identification of any weaknesses that require corrective action.
A results based M&E system is an essential source of information for streamlining and improving interventions to maximize the likelihood of success. Such systems can also provide a view over time of the status of a project, program, or policy. Because governments and other organizations often have multiple projects, programs, and policies implemented at any one time, it is essential to have some means of tracking how well they are working. A results based M&E system helps identify promising interventions early so that they can potentially be implemented elsewhere. Having data available about how well a particular project, practice, program, or policy works provides useful information for formulating and justifying budget requests. It also allows judicious allocation of scarce resources to the interventions that will provide the greatest benefit.
Guideline: Focusing the evaluation design
Guidelines: How to conduct a useful M&E action
setting the M&E action objectives; Determine what information the evaluation must provide.
Deciding the concrete outputs of the action (reports, presentations, etc.)
Deciding how the outputs of the evaluation activity will be used by evaluation stakeholders
Templates