An evaluation which looks at the impact of an intervention on final welfare outcomes, rather than only at project outputs, or a process evaluation which focuses on implementation. The proper analysis of impact requires a counterfactual of what those outcomes would have been in the absence of the intervention. An impact evaluation provides information about the impacts produced by an intervention. This is the fourth and final chapter of the fourth working paper of the international initiative for impact evaluation 3ie.
The international initiative for impact evaluation 3ie is a new organisation devoted to enhancing development effectiveness through supporting the production and use of evidence from rigorous. Impact evaluation is the main means for empirically testing what actually happens when interventions are implemented. Rmit university and the international initiative for impact evaluation 3ie. The most basic ethical principle in an evaluation is that the delivery of. Goldilocks assembled this set of resources for use in designing and running an impact evaluation. Guskeys evaluation model represent the dependent variables in. The international initiative for impact evaluation 3ie seeks to improve the lives of poor. Despite wide agreement that this approach will address the why question, it has not often been. In previous chapters, we provide information about how to develop evaluation metrics for specific aspects of. Quantitative and qualitative methods in impact evaluation and measuring results. Impact evaluation methodologies impact evaluation workshop, lilongwe, dec. This means that an impact evaluation must establish what has been the cause of observed changes in this case impacts referred to as causal attribution also referred to as causal inference.
For more information on high quality impact evaluation, see 3ie s principles for impact evaluation and the following link for an overview of impact evaluation methods. Quantitative and qualitative methods in impact evaluation and. Monitoring and evaluation methods 1 needs assessment 2 process evaluation 3 impact evaluation analysis of beneficiary needs analysis of program implementation measures how much the program impacts beneficiaries 1. Matching impact evaluation design to the nature of the. Beginning with the need for a theorydriven evaluation, and ending with a set of concrete tools to use in running an evaluation, they cover a range of practical materials useful. However, 3ie also recognizes the considerable skills in conducting quality impact evaluations in institutions located in. It is widely accepted that a theorybased approach to impact evaluation, one that maps out the causal chain from inputs to outcomes and impact and tests the underlying assumptions, will shed light on the why question. Mouza, 2009 and this is something that we have addressed through our three years of research. Principles and practice, international initiative for impact evaluation working paper no. In this chapter, rogers argues that appropriate impact evaluation design requires situational responsiveness. For an example in process evaluation, please see the norwegian case electricity savings in households n 3 in chapter 8. Pdf the international initiative for impact evaluation 3ie is a new organisation. This chapter provides an overview of basic evaluation principles, including. In theorybased approaches, the specific steps in a causal chain, the specific causal mechanisms.
To learn about our use of cookies and how you can manage your cookie settings, please see our cookie policy. Impact evaluations and development nonie guidance on impact evaluation 41 6 build on existing knowledge relevant to the impact of interventions 43 key message 45 part ii managing impact evaluations 47 7 determine if an impact evaluation is feasible and worth the cost 48 key message 49 8 start collecting data early 49 8. Pdf the international initiative for impact evaluation 3ie. International initiative for impact evaluation 3ie home 3ie. The ifad impact evaluation guidelines accordingly define impact as the the attainment of development goals of the project or program, or rather the contributions. How can impact evaluation identify not just what does or does not work, but why. Evaluating the impact of professional development on teaching practice. The impact of professional development on teaching practice 705 implementing epd learning in their teaching practice. Searching this database will bring up published impact evaluations from our impact evaluation repository as well as completed 3ie funded impact evaluations. In terms of methodological standards, evaluation designs must be capable of addressing. The international initiative for impact evaluation 3ie defines rigorous impact evaluations as. By closing this message, you are consenting to our use of cookies.
Theorybased approaches to evaluation attempt to understand an interventions contribution to observed results through a mechanistic or process interpretation of causation, rather than determining causation through comparison to a counterfactual. The international initiative for impact evaluation 3ie works to. Global developments evaluation gap working group in 2004, international initiative on impact evaluation 3ie in 2008, masters of science in impact evaluation for international development at the university of east anglia, and the centre for development impact at the. Impact evaluation 3ie estimates that since the study some 800 impact evaluations of social interventions have been completed or are in progress in low and middleincome countries. Calls for rigorous impact evaluation have been accompanied by the quest not just to find out what works but why. Read, download and publish impact evaluation magazines. Outline of principles of impact evalation discussion dra. Read, download and publish impact evaluation magazines, ebooks for free at. Credibility principles and monitoring and evaluation 8 5. Goldilocks resource resources and tools for impact evaluation. There are five key principles relating to internal validity study design and external validity generalizability which. Howard white identifies six principles for the successful application of the approach mapping out the causal chain programme theory, understanding the context, anticipating. Global developments evaluation gap working group in 2004, international initiative on impact evaluation 3ie in 2008, masters of science in impact evaluation for international development at the university of east anglia, and the centre for development impact at the institute of development studies, among others.
Calls for rigorous impact evaluation has been accompanied by the quest for what works and why. What methods may be used in impact evaluations of humanitarian assistance. In response to this growing consensus there are a number of international initiatives on impact evaluation. International impact evaluation initiative 3ie database. Other interpretations of impact evaluation include. As more resources are being allocated to impact evaluation of development programs,the need to map out the utilization and influence of evaluations has been increasingly highlighted. Process evaluation 3 impact evaluation analysis of beneficiary needs analysis of program implementation measures how.
The international initiative for impact evaluation 3ie is a new organization devoted to enhancing development effectiveness through supporting the production and use of evidence from rigorous impact studies. Background the international initiative for impact evaluation 3ie is an international grantmaking organization that promotes evidenceinformed equitable, inclusive and sustainable development. An impact evaluation provides information about the impacts produced by an intervention positive and negative, intended and unintended, direct and indirect. Impact evaluation assesses the changes that can be attributed to a particular intervention, such. International initiative for impact evaluation 3ie two main types of study design are currently used within, and bilateral donors and impact evaluations. The international initiative for impact evaluation 3ie has been created to enhance development effectiveness through the promotion of evidencebased policymaking. The formal literature on impact evaluation methods and practices is large, with a few useful overviews. Principles and practice howard white executive director international initiative on impact evaluation, 3ie email. This paper does not intend to provide a detailed account of different approaches to impact evaluation nor an overview of proposed solutions to specific impact evaluation challenges2. December 2011, 3ie working paper 14, available as pdf abstract.
Stockholm international peace research institute sipri vegard iversen. A theorybased approach to impact evaluation maps out the causal chain from inputs to outcomes and impact, and tests the underlying assumptions. International initiative for impact evaluation 3ie, new delhi. We support the generation and effective use of highquality evidence to inform.
Different impact evaluation methodologies differ in how they construct the counterfactual counterfactual. Although the commitment to theory based impact evaluation is there in principle, few. Beginning with the need for a theorydriven evaluation, and ending with a set of concrete tools to use in running an evaluation, they cover a. Wss together with 3ie organized a learning session on the. Impact assessment and internal audit evaluation and risk. Overview on impact evaluation paul gertler cega, ucberkeley. It also quotes the ifad impact evaluation guidelines, which define impact as the the attainment of development goals of the. International initiative for impact evaluation 3ie home.
Overview of impact evaluation page 2 an impact evaluation should only be undertaken when its intended use can be clearly identified and when it is likely to be able to produce useful findings, taking into account the availability of resources and the timing of decisions about the programme or policy under investigation. University of manchester yashodhan ghorpade institute of development studies tilman bruck. This paper explores the factors affecting the success of impact evaluations, drawing from the experience of two major funders of evaluations. In parallel to these trends, behavioral economics has increasingly attracted. Conceptualizing capacity building capacity building can be defined straightforwardly as a process for strengthening the management and governance of an organization so that it can effectively achieve its objectives and fulfill its mission. However, care must be taken about generalizing from a specific context. Quantitative and qualitative methods in impact evaluation. Pdf the international initiative for impact evaluation.
But, at the risk of sounding facetious, this is an outputbased assessment with no beforeandafter comparison. These evaluations provide crucial evidence on what works, for whom, why and at what cost. On our website, we host the largestofitskind impact evaluation repository that includes summaries of both 3ie funded and other studies. The oecd defines impact evaluation as an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended or unintended oecd 2006. How and when to implement impact evaluation and assessment in the policy process. Matching impact evaluation design to the nature of. International initiative for impact evaluation working paper 3 theorybased impact evaluation. Impact evaluation has been defined differently over the past few decades. Outline of principles of impact evaluation part i key concepts definition impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended or unintended.
We first define successful evaluations using three key dimensions. How to differentiate between impact evaluation and impact assessment, and the principles and practices of each what to assess in terms of the social, economic and environmental impacts of a policy, programme or project how and when to implement impact evaluation and assessment in the policy process. International initiative for impact evaluation 3ie publication date. The oecddac criteria reflect the core principles for evaluating development. The international initiative for impact evaluation 3ie. International initiative for impact evaluation 3ie. Goldilocks resource resources and tools for impact. World bank strategic impact evaluation fund, 3ie, t he campbell collaboration, the cochrane collaboration, the alliance for health policy and systems research and the eppi centre. This ids bulletin is the second of two that follow an institute of development studies event seeking to define an agenda for research and practice of development impact evaluation.
59 1215 660 1044 438 917 530 385 430 728 115 251 1051 1352 1251 1587 1359 1182 433 696 293 749 1604 1093 869 1317 1605 766 1607 427 909 441 1316 1150 144 404 38 771 63 1