Experiences from conducting Third-Party Project Evaluation
NA-Online
About the Event
The format is more like a dissemination workshop keeping in mind our participants, local JEEViKA, Pradan, AKRSP(I), and other nonprofit organizations as participants.
The theme that we choose for this workshop is “build forward better”, particularly, “How can evaluation help decision-makers and other stakeholders to make better-informed decisions to ‘build forward better’?
The workshop intends to highlight the importance of monitoring and evaluation of projects to “build forward better”. The evaluation of the intended project outcomes requires extensive information on pre and post-project interventions and projects that have proper monitoring efforts put in place tend to perform better on the outcomes. Given that most project evaluations are done by third party agencies who are not related to the key stakeholders and the project intervention, the absence of information on pre and post scenarios or absence of proper monitoring mechanism, or lack of appropriate indicators all seriously limit the efforts of agencies that undertake the project evaluation.
Project evaluation alone as a contractual task then stretches the imagination of the evaluation team to circumvent the shortcomings to assess the promised deliverables by the implementing agencies as outcomes of the project interventions. We present three such case studies where the evaluation of projects or government programs were carried with a pragmatic approach more than an innovative approach so that the project implementing, and monitoring agencies can benefit in “building forward better”.
As part of this workshop, we plan to have a general overview of M&E in the context of the topic described above followed by three case presentations:
The first case study is on “Third Party Evaluation of 29 Centralized Kitchens and 15 % related Schools Serving Mid-Day Meals to Bihar Schools”. The study is limited to certain geography of Bihar and it required evaluating the hygiene factors, practices followed in the kitchens and the fulfillment of objectives of providing nutritious food to the school children. The study uses a mixed-methods approach with extensive triangulation of data to develop a set of indicators that help rate the schools on green for good, yellow for average, and red for poor. The model can be adopted by all the kitchens across the nation or the country to continuously monitor their performance in their pursuit of providing nutritious and quality food.
The second case study is about an end-term evaluation of a CSR-funded project to empower women and their health, enhance the incomes of beneficiaries, particularly farmers, and promote education and technology in a sustainable manner. Neither the baseline data nor the internal data on evaluation were available with the implementing agency and it made an evaluation of the project extremely difficult. Time was another major constraint. How the difficulties were overcome to the possible extent and what care has been taken to report the findings constituting a learning experience.
The third case study work is in progress and will be included in the workshop post its completion in May 2021.
None of the above projects mentioned above have a monitoring component and all of them are evaluation projects and little data was made available for the evaluation teams. It is in this context the workshop addresses the following questions:
1. What data should we request from the clients before accepting the project?
2. How difficult it is to understand what needs to be evaluated without information or access to monitoring parameters of the project?
3. What if the data is not available and yet the evaluation needs to be done?
4. How to devise a methodology to evaluate such programs particularly when the evaluating party is not directly involved in monitoring the project and limited data is available about the project?
5. What care must be taken while writing and interpreting results in the report?
The theme that we choose for this workshop is “build forward better”, particularly, “How can evaluation help decision-makers and other stakeholders to make better-informed decisions to ‘build forward better’?
The workshop intends to highlight the importance of monitoring and evaluation of projects to “build forward better”. The evaluation of the intended project outcomes requires extensive information on pre and post-project interventions and projects that have proper monitoring efforts put in place tend to perform better on the outcomes. Given that most project evaluations are done by third party agencies who are not related to the key stakeholders and the project intervention, the absence of information on pre and post scenarios or absence of proper monitoring mechanism, or lack of appropriate indicators all seriously limit the efforts of agencies that undertake the project evaluation.
Project evaluation alone as a contractual task then stretches the imagination of the evaluation team to circumvent the shortcomings to assess the promised deliverables by the implementing agencies as outcomes of the project interventions. We present three such case studies where the evaluation of projects or government programs were carried with a pragmatic approach more than an innovative approach so that the project implementing, and monitoring agencies can benefit in “building forward better”.
As part of this workshop, we plan to have a general overview of M&E in the context of the topic described above followed by three case presentations:
The first case study is on “Third Party Evaluation of 29 Centralized Kitchens and 15 % related Schools Serving Mid-Day Meals to Bihar Schools”. The study is limited to certain geography of Bihar and it required evaluating the hygiene factors, practices followed in the kitchens and the fulfillment of objectives of providing nutritious food to the school children. The study uses a mixed-methods approach with extensive triangulation of data to develop a set of indicators that help rate the schools on green for good, yellow for average, and red for poor. The model can be adopted by all the kitchens across the nation or the country to continuously monitor their performance in their pursuit of providing nutritious and quality food.
The second case study is about an end-term evaluation of a CSR-funded project to empower women and their health, enhance the incomes of beneficiaries, particularly farmers, and promote education and technology in a sustainable manner. Neither the baseline data nor the internal data on evaluation were available with the implementing agency and it made an evaluation of the project extremely difficult. Time was another major constraint. How the difficulties were overcome to the possible extent and what care has been taken to report the findings constituting a learning experience.
The third case study work is in progress and will be included in the workshop post its completion in May 2021.
None of the above projects mentioned above have a monitoring component and all of them are evaluation projects and little data was made available for the evaluation teams. It is in this context the workshop addresses the following questions:
1. What data should we request from the clients before accepting the project?
2. How difficult it is to understand what needs to be evaluated without information or access to monitoring parameters of the project?
3. What if the data is not available and yet the evaluation needs to be done?
4. How to devise a methodology to evaluate such programs particularly when the evaluating party is not directly involved in monitoring the project and limited data is available about the project?
5. What care must be taken while writing and interpreting results in the report?
Sessions
NA-Online
June 3, 2021
11:45 AM - 11:55 AM
The coordinator for the workshop will brief the participants about the importance of the event and introduces the speakers during the day's workshop
Speakers
| Name | Title | Biography |
|---|---|---|
| Sridhar Telidevara | Dr. | Dr. Sridhar Telidevara holds a Ph.D from State University of New York, University at Buffalo. He has several years of experience in the steel industry, developmental research and teaching experience prior to joining DMI. His primary research interests are livelihoods monitoring and evaluation, discrete choice modelling and financial inclusion. His teaching interests include Production and Operations Management, Research Methods, and Developmental Economics. He has published three papers in international journals and contributed three chapters to a book on Microfinance: Challenges and Opportunities. He is interested in Livelihood Promotion through Skill Development and Social Entrepreneurship, and Health Care. |
NA-Online
June 3, 2021
14:00 PM - 15:00 PM
an end-term evaluation of a CSR funded project to empower women and their health, enhance incomes of beneficiaries, particularly farmers, and promote education and technology in a sustainable manner. Neither the baseline data nor the internal data on evaluation were available with the implementing agency and it made evaluation of the project extremely difficult. Time was another major constraint. How the difficulties were overcome to the possible extent and what care has been taken to report the findings all constitute a learning experience.
Speakers
| Name | Title | Biography |
|---|---|---|
| Dr. Sridhar Telidevara | Dr. | Sridhar Telidevara holds a Ph.D from State University of New York, University at Buffalo. He has several years of experience in the steel industry, developmental research and teaching experience prior to joining DMI. His primary research interests are livelihoods monitoring and evaluation, discrete choice modelling and financial inclusion. His teaching interests include Production and Operations Management, Research Methods, and Developmental Economics. He has published three papers in international journals and contributed three chapters to a book on Micro-finance: Challenges and Opportunities. He is interested in Livelihood Promotion through Skill Development and Social Entrepreneurship, and Health Care. |
NA-Online
June 3, 2021
15:30 PM - 16:30 PM
The third study focuses on improper and inefficient handling of cattle dung resulting in the high release of methane and global warming in rural communities notwithstanding the cash-outflow from the villages for energy. Production of biogas and slurry and organic manure is a time-tested technology to overcome these ills. Its rapid adoption across a large number of households and minimization of net cash out-flow from household and village and creation of a circular economy has become important. Towards this objective, an organization that works with women dairy farmers decided to carry out a study on the adoption and usage practices before and after setting up of these units as well as study the spillover effect if any. The baseline and end-line surveys entails carrying out a structured assessment of resources committed for a specific goal and gathers information at micro level and tries to understand the process and measures the outcome. The study is vide a panel data consisting of 50 beneficiary households in two villages along with a control group. The intent behind this monitoring and evaluation exercise is to generate evidence for informed decision making by a larger group of farmers.
Speakers
| Name | Title | Biography |
|---|---|---|
| Prof. Amrita Dhiman | Ms. | Prof. Amrita Dhiman is an academician with considerable industry experience and understanding of real-life situations at farm level having worked with corporates and government institutions for more than seven years. Currently, actively involved in teaching, research, consultancy, and capacity building activities at the DMI. She is a Post-Graduate in Agri-Business Management from VAMNICOM, Pune, and a Graduate in Agriculture from G. B. Pant University of Agriculture and Technology, Pantnagar. She is a recipient of Chancellor's Gold Medal as the best all-round graduating student of GBPAUT. She is also the Gold Medalist for the outstanding level of study at VAMNICOM. She has the experience of working with State Bank of India in Agri finance and as a research analyst for Agri commodities with Karvy Comtrade Ltd at Hyderabad. While working with Mahindra & Mahindra, she was involved in strategic planning, implementation, corporate communication, marketing, and branding activities. She was involved in many government programmes and schemes like MGNREGA, IAY, SBM, PMAGY, NSAP, etc. as a Prime Minister's Rural Development Fellow under the Ministry of Rural Development in Bihar. She played a lead role in documenting the initiatives taken up for improved governance and outreach. Articles and reports written by her about these initiatives have been published in government newsletters/websites. A number of her analytical reports have been published in Invest & Harvest & The Finapolis, The Hindu Business Line, and EPW. As a faculty at DMI, she now handles courses on agri-business management, public policy, local governance, etc. She is also in-charge of admissions at DMI and is actively involved in the action research "Empowering Panchayats towards Participatory Governance". She is leading many ongoing projects in the field of agriculture and related issues. |
NA-Online
June 3, 2021
16:35 PM - 16:45 PM
Feedback received for each session from the participants, a summary of the workshop, and a vote of thanks by the Coordinator
Speakers
| Name | Title | Biography |
|---|---|---|
| Dr. Sridhar Telidevara | Dr. | Sridhar Telidevara holds a Ph.D from State University of New York, University at Buffalo. He has several years of experience in the steel industry, developmental research and teaching experience prior to joining DMI. His primary research interests are livelihoods monitoring and evaluation, discrete choice modelling and financial inclusion. His teaching interests include Production and Operations Management, Research Methods, and Developmental Economics. He has published three papers in international journals and contributed three chapters to a book on Micro-finance: Challenges and Opportunities. He is interested in Livelihood Promotion through Skill Development and Social Entrepreneurship, and Health Care. |
NA-Online
June 3, 2021
12:00 PM - 13:00 PM
The first case study is on “Third Party Evaluation of 29 Centralized Kitchens and 15 % related
Schools Serving Mid-Day Meals to Bihar Schools”. The study is limited to a certain geography
of Bihar and it required evaluating the hygiene factors, practices followed in the kitchens, and
the fulfilment of objectives of providing nutritious food to the school children. The study uses
a mixed methods approach with extensive triangulation of data to develop a set of indicators that help rate the schools on green for good, yellow for average, and red for poor. The model
can be adopted by all the kitchens across the nation or the country to continuously monitor their
performance in their pursuit of providing nutritious and quality food.
Speakers
| Name | Title | Biography |
|---|---|---|
| Dr. Shankar Purbey | Dr. | Dr Shankar Purbey, holds PhD in Industrial Engineering & Management from IIT Dhanbad. Prior to joining DMI, he was associated with IIM Shillong for more than ten years. He has more than 15 years of industry, teaching and research experience. He was awarded the National Doctoral Fellowship by AICTE while pursuing his PhD. He has international publications to his credit and has presented papers in various National and International conferences. He was also extensively involved in consultancies assignment including executive training programs for both government and non -governmental organization. He has coordinated more than fifteen executive training programs and three certificate courses. Certificate courses include course on Tourism & Hospitality Management and Quality Assurance in Healthcare delivery. In the area of Project Management alone he has conducted more than ten executive training programs for clients like Oil India, AEDC Ltd., NERCORMP, MCS officers, Ministry of DoNER, Indian Oil, Ministry of Heavy Industry -CPSEs and SLPEs, and Animal Husbandry & Veterinary department Meghalaya . |