Cell for Evaluation Studies

Mission

The Centre for Evaluation Studies acts as an independent and impartial agency for evaluation of various programmes/ schemes of the state government to assess their efficacy in meeting the objectives of the scheme and the extent to which the benefits have reached the targeted population. The purpose of the Centre is to improve effectiveness of programmes/schemes by way of providing inputs/suggestions for taking corrective action in respect of ongoing schemes as well as designing new schemes.

Objectives

1. To evaluate schemes/programme of different departments of Punjab Government on request and to come out with analytical evaluation reports.
2. To undertake quick evaluation of schemes /programmes of public importance.
3. To undertake orientation and capacity building programmes, both at the state headquarters and in the field for government departments, individuals and NGOs involved in evaluation of development programmes or schemes.
4. To seek technical collaboration with other institutes of national and international importance engaged in similar work.
5. To seek empanelment with central government departments and institutes for evaluation and impact assessment of welfare projects.
6. To create a data base on development evaluation containing results of evaluations, lessons learnt, best practices etc. in a user friendly format.
7. To conduct workshop/seminars on methodological issues of evaluation.
8. To produce documentary films on important evaluation studies.
9. Developing, grading, monitoring and Certification of Field Investigators and Supervisors for evaluation work.

Activities carried out so far

1. Sentisation training programme on Programme Evaluation and Performance Review for senior officers of Punjab Government.
2. Evaluation of seven schemes under SCA.
3. Training Programme on "Programme Evaluation" or middle level officers of Punjab Government with a follow up workshop to identify schemes/ projects/ programmes for evaluation.
4. Training Programme for empanelled Supervisors and Field Investigators at Bathinda and Jalandhar.
5. Development of a Methodological Note on Programme Evaluation.
6. A comprehensive study of methodology adopted by World Bank for evaluation of its projects.

Future Activities Planned

1. Field level Training Programmes on evaluation on regional/district basis.
2. Training to field staff with reference to scheme/programme to be taken up for evaluation.
3. Evaluation of ITIs and schemes of Dairy Department.
4. Preparation of detailed methodology for Evaluation Studies.
5. Developing, Grading, Monitoring and Certification of List of Field Investigators and Supervisors on continuing basis.

Recent Activities

Evaluation Methodology made easy by the Cell for Evaluation Studies at MGSIPAP(October 2011)

Introduction:

The Cell for Evaluation Studies (CES) at Mahatma Gandhi State Institute of Public Administration, Punjab (MGSIPAP) is an impartial evaluation agency which specializes in evaluation of government programmes.. Evaluation studies provide valuable inputs for taking corrective action in respect of ongoing programmes and designing new programmes. A programme often fails due to lack of good strategies and enabling management support. A special feature of evaluation studies undertaken by this cell is that it examines these aspects critically and goes a step further to suggest suitable changes in the objectives of the programmes, strategies, and management systems.

Meaning of Evaluation:

An evaluation is an assessment, as systematic and impartial as possible, of a project, programme or policy. It aims at determining the relevance, impact, effectiveness, efficiency and sustainability of the interventions / programmes / projects. Evaluation may be done either concurrently or on termination of the project.

Audit and evaluation are different:

The audit tradition has an investigative and accounting orientation. Audit is more concerned with compliance with the prevailing rules and proper sanctions while incurring expenditure. Performance audit seeks to measure performance in terms of outputs with reference to the baseline targets. Evaluation is more concerned with outcomes and impact. The measurement of outcome and impact of a programme is generally not within the purview of the auditors.

Monitoring and evaluation are different:

Monitoring is a continuous function that uses systematic collection of data on specified indicators to provide management of an ongoing development intervention with indications of extent of progress and achievement of objectives and progress in use of allocated funds. Monitoring of physical and financial targets is usually done periodically through PERT and CPM charts. Evaluation is usually undertaken on the termination of a projects although projects having large allocations or longer terms also concurrently evaluated.

Cost benefit analysis and evaluation are different:

Cost benefit analysis is undertaken to assess viability of a development intervention before its implementation as it attempts to identify and measure all items of costs and benefits due to a project or a programme and examines if the benefits exceed the costs through a single numerical measure. Cost benefit analysis makes a number of assumptions. Cost benefit analysis is undertaken before the implementation of a project. Evaluation is undertaken after the project is implemented and may be undertaken either concurrently when the project is under implementation or on the termination of the project.

Evaluation Process:

Planning

1. Study of scheme with objectives, other papers and objectives of evaluation.

2. Setting up expert committee for each scheme under evaluation to guide, supervise and finalise evaluation process and report.

3. Selecting evaluation design: choice of evaluation designs is related to formulation of evaluation questions (descriptive, normative and impact), selecting methodology of evaluation i.e. quick or full evaluation, from primary source or secondary sources, multi site or cluster based or participatory in nature, and the statistical design, consisting of sample size, sampling technique, estimation procedure etc.

4. Designing questionnaires (simple, direct and facilitating information).

5. Separate questionnaire for each stakeholder.

Doing

1. First stage consultation with stakeholders to finalize objectives of evaluation vis-a-vis objectives of project and questionnaires.

2. Collection of quantitative and qualitative information.

3. Second stage consultation with stakeholders in the mid way of field enquiry to review progress.

4. Tabulation of data, preparation of analytical draft report and formulation of findings.

5. Variations in tabulated data vis-a-vis ground realities should be looked into by revisiting the respondents.

Reporting

1. Third stage consultation with stakeholders to review main findings, recommendations and action points.

2. Finalization of evaluation report with workable suggestions/alternatives and its submission to concerned department

Feedback

Action Points and follow up with the Department

Important Features of the Evaluation Methodology at MGSIPAP

1. Important feature of our evaluation studies lies in the consultation workshops jointly conducted by this Cell and Administrative Reforms Division of the Institute to get feedback of the stakeholders and officers associated with the programme implementation at the entry, mid and exit stages of the evaluation study.

2. The Administrative Reforms Division of the Institute has prescribed certain protocols to dovetail diagnostic management studies in evaluation plan through an appropriate oversight mechanism in order to get inputs for management reforms.

Contact: Shri A.S. Ahluwalia, Faculty I/C, CES, csdeg@mgsipap.org