Developing Evaluation Guidelines

 

How do we Evaluate Programmes?

 

 

The present Indian context: the necessity to articulate plans based on the principles illustrated in the NCMP

 

The UPA Government has asked various public bodies to prepare policy plans based on the NCMP.  However public administration bodies are not always best equipped with the competence of formulating plans that are:

-         based on clearly stated and realistically reachable objectives,

-         designed on the assessment of the needs of the intended beneficiaries,

-         articulated in such a way that is possible to objectively evaluate the results obtained. 

 

Amongst the objectives of the NCMP is the modernization of public administration and the empowerment of local governments. However there is need to provide assistance to public administration and local governments in developing their capacity to improve their own functioning.

 

This need of finding workable solutions for such a lack of specific expertise is all the more urgent in the purview of a direct relationship between the Central Government and the grass root bodies (panchayats, etc.) that UPA Govt. intends to empower. 

 

 

The policy tools  developed by the European Union. What they are and why they cold help.

 

A peculiarity of the development of governance of European Union has been its “non imposing” character upon the various national governments that are EU members.  Therefore developing a policy structure for the EU has mainly been an activity of “promotion” of  improved governance.  The relative freedom from immediate political considerations that policy planners at EU have enjoyed has given them the possibility of aiming at long lasting overall improvement of EU governance; this has mainly been considered as an improvement of the competence, the honesty and the transparency of public administrations at all levels.

 

For achieving this objectives in a context where EU could not (and would not) issue “directives”,  a series of policy designing tools and policy evaluation patterns have been developed. EU is promoting the adoption of such methodologies and tools at all levels of governance. However the enforcement of such tools is statuary only within the purview of plans that are directly financed with funds provided by EU Commission.

 

The need to implement such tools into very different context and to promote their adoption by administrative bodies that are not subject to EU jurisdiction, has resulted into a collective effort to design user friendly tools, and in developing PA training expertise and training resources.

 

The tools, resources and expertise therefore developed could be, after adequate reengineering,  adopted to suite the Indian context.   What facilitates the transfer is also the fact that such tools have been developed in order to implement EU policy principles in their effort to pursue a “human centred” approach to economic development, such as leading to a balance between the growth of aggregate wealth and the improvement of distribution of resulting benefits and opportunities to all the citizens.

This concern is similar to the spirit of NCMP to redistribute opportunities and wealth amongst all Indian citizens and struggle to give equally to all the same social opportunities, protective security and substantive freedom.

 

The main tools

 

Based on the methodology of the Project Cycle Management (PCM), EU has developed a grid named “the logical framework” that empowers the policy planner in identifying:

-         specific objectives that are coherent with the overall policy objectives;

-         specific outputs (expected results) that are conducive to intended objectives;

-         specific activities that are conducive to the expected results;

-         the required resources (human, technological, financial) required to carry out the intended activities;

-         the objective indicators and the sources that allow transparent evaluation of the efficacy, the efficiency and the impact of the programme.

(see the table 1 enclosed)

 

Programmes articulated on the basis of  Logical Framework can be more easily evaluated (before, during and after the implementation) by the financing agency and by the community of intended beneficiaries and other stakeholders.   (see the table 2 enclosed).

 

Solid literature has been produced to standardize the usage of the logical framework and support its adoption.

 

The author of these note would like to suggest a general adoption (after reengineering) of the logical framework in the preparation of programmes and projects that are intended to implement the NCMP. The use of a common standard could be very helpful for promoting transparency, for duplication of best practices and dissemination of project results.  This would facilitate Government-citizens communication and built support for UPA Government policy initiatives.  It would also be a practical tool for implementing the capacity and skills of public administration in India.

 

It is possible to obtain EU and EU member countries support in the process of reengineering and disseminating the logical framework could be achieved as this would be consistent with bilateral agreements and cooperation plans.

 

In case you find the idea deserving your attention, we remain at your disposal for any further clarification and development.

 

 

 

 

 

 

Project cycle management

Logical Framework

 

	Intervention logic	Objectively verifiable indicators	Sources and means of verification	Assumptions
Overall objectives	What is the overall broader objective, to which the project  will contribute ?	What are the key indicators related to the overall objective?	What are the sources of information for these indicators?
Project purpose	What are the specific objectives, which the project shall achieve?	What are the quantitative or qualitative indicators showing whether and to what extent the project’s specific objectives are achieved?	What are the sources of information that exist or can be collected? What are the methods required to get this information?	What are the factors and conditions not under the direct control of the project which are necessary to achieve these objectives? What risks have to be considered?
Expected results	What are the concrete outputs envisaged to achieve the specific objectives? What are the envisaged effects and benefits of the project? What improvements and changes will be produced by the project?	What are the indicators to measure whether and to what extent the project achieves the envisaged results and effects?	What are the sources of information for these indicators?	What external factors and conditions must be realised to obtain the expected outputs and results on schedule?
Outputs
Effects and changes
Activities	What are the key activities to be carried out and in what sequence in order to produce the expected results?	Means What are the means required to implement these activities, e. g. personnel, equipment, training, studies, supplies, operational facilities, etc.	What are the sources of information about project progress?	What pre-conditions are required before the project starts? What conditions outside the project’s direct control have to be present for the implementation of the planned activities?