Cell for Evaluation Studies

Activities

Evaluation Methodology made easy by the Cell for Evaluation Studies at MGSIPAP(October 2011)

Introduction:

The Cell for Evaluation Studies (CES) at Mahatma Gandhi State Institute of Public Administration, Punjab (MGSIPAP) is an impartial evaluation agency which specializes in evaluation of government programmes.. Evaluation studies provide valuable inputs for taking corrective action in respect of ongoing programmes and designing new programmes. A programme often fails due to lack of good strategies and enabling management support. A special feature of evaluation studies undertaken by this cell is that it examines these aspects critically and goes a step further to suggest suitable changes in the objectives of the programmes, strategies, and management systems.

Meaning of Evaluation:

An evaluation is an assessment, as systematic and impartial as possible, of a project, programme or policy. It aims at determining the relevance, impact, effectiveness, efficiency and sustainability of the interventions / programmes / projects. Evaluation may be done either concurrently or on termination of the project.

Audit and evaluation are different:

The audit tradition has an investigative and accounting orientation. Audit is more concerned with compliance with the prevailing rules and proper sanctions while incurring expenditure. Performance audit seeks to measure performance in terms of outputs with reference to the baseline targets. Evaluation is more concerned with outcomes and impact. The measurement of outcome and impact of a programme is generally not within the purview of the auditors.

Monitoring and evaluation are different:

Monitoring is a continuous function that uses systematic collection of data on specified indicators to provide management of an ongoing development intervention with indications of extent of progress and achievement of objectives and progress in use of allocated funds. Monitoring of physical and financial targets is usually done periodically through PERT and CPM charts. Evaluation is usually undertaken on the termination of a projects although projects having large allocations or longer terms also concurrently evaluated.

Cost benefit analysis and evaluation are different:

Cost benefit analysis is undertaken to assess viability of a development intervention before its implementation as it attempts to identify and measure all items of costs and benefits due to a project or a programme and examines if the benefits exceed the costs through a single numerical measure. Cost benefit analysis makes a number of assumptions. Cost benefit analysis is undertaken before the implementation of a project. Evaluation is undertaken after the project is implemented and may be undertaken either concurrently when the project is under implementation or on the termination of the project.

Evaluation Process:

Planning

1. Study of scheme with objectives, other papers and objectives of evaluation.

2. Setting up expert committee for each scheme under evaluation to guide, supervise and finalise evaluation process and report.

3. Selecting evaluation design: choice of evaluation designs is related to formulation of evaluation questions (descriptive, normative and impact), selecting methodology of evaluation i.e. quick or full evaluation, from primary source or secondary sources, multi site or cluster based or participatory in nature, and the statistical design, consisting of sample size, sampling technique, estimation procedure etc.

4. Designing questionnaires (simple, direct and facilitating information).

5. Separate questionnaire for each stakeholder.

Doing

1. First stage consultation with stakeholders to finalize objectives of evaluation vis-à-vis objectives of project and questionnaires.

2. Collection of quantitative and qualitative information.

3. Second stage consultation with stakeholders in the mid way of field enquiry to review progress.

4. Tabulation of data, preparation of analytical draft report and formulation of findings.

5. Variations in tabulated data vis-à-vis ground realities should be looked into by revisiting the respondents.

Reporting

1. Third stage consultation with stakeholders to review main findings, recommendations and action points.

2. Finalization of evaluation report with workable suggestions/alternatives and its submission to concerned department

Feedback

Action Points and follow up with the Department

Important Features of the Evaluation Methodology at MGSIPAP

1. Important feature of our evaluation studies lies in the consultation workshops jointly conducted by this Cell and Administrative Reforms Division of the Institute to get feedback of the stakeholders and officers associated with the programme implementation at the entry, mid and exit stages of the evaluation study.

2. The Administrative Reforms Division of the Institute has prescribed certain protocols to dovetail diagnostic management studies in evaluation plan through an appropriate oversight mechanism in order to get inputs for management reforms.

Contact: Shri A.S. Ahluwalia, Faculty I/C, CES, asahluwalia@mgsipap.org