Millions Fed: Proven Successes in Agricultural Development

Methodology

David J. Spielman, Rajul Pandya-Lorch, Kenda Cunningham, and Sivan Yosef

A study such as this one cannot be an exhaustive review of all policies, investments, and programs in agricultural development—five decades of rich and diverse experiences simply cannot be summed up in a single volume. So instead, this book focuses on relatively large-scale and long-term successes that were backed by strong evidence of positive impact.

The methodology used to identify and analyze these successes is detailed below. This methodology draws on several previous efforts to document successes in development, including studies by Gabre-Madhin and Haggblade (2004) and Levine (2004).1 Additional insights were extracted from studies by the World Bank (2008, 2006), Uphoff, Esman, and Krishna (1998), Krishna, Uphoff, and Esman (1997), and the Asia-Pacific Association of Agricultural Research Institutions (2009).2

The method for arriving at the 20 case studies of success found in this book involved four steps: (1) throwing the net out, (2) sorting the catch, (3) selecting the most appropriate cases, and (4) synthesizing the evidence.

  1. Throwing the net out. A first step in identifying these successes was to seek input from those who participate in or study agricultural policies, investments, and programs that aim to improve food security and reduce hunger—practitioners, scholars, policymakers, and many others. To this end, the project team circulated a global “Call for Nominations” on successes in agricultural development in late 2008 while it simultaneously compiled suggestions from experts in the field and from information garnered from scholarly literature, project documents, websites, and other sources. By early 2009, about 250 potential successes were identified.

  2. Sorting the catch. The project team sorted through these potential successes using two qualifying criteria that had to be met in order for a case to be considered further, and five evaluative criteria that focused on the specific attributes and impacts of the intervention.

    The first qualifying criterion was that the intervention must have been operational in at least one developing country. For the purposes of this project, developing countries are those classified as low-income, lower-middle-income, or higher-middle-income countries according to the World Bank-defined income groups or the equivalent classification that was current when the intervention was being implemented.3 Note that this criterion does not imply that interventions were chosen because they only benefited developing countries—if an intervention generates benefits that also accrue to high-income industrialized countries, it was still considered.

    The second qualifying criterion was that the intervention must have engaged agriculture directly—that is, it must have operated on constraints that are specific to agriculture. This criterion excludes certain types of interventions that operate on agriculture indirectly. For example, although there is strong evidence indicating that basic education, health, and sanitation programs targeting the rural poor contribute to increasing their labor productivity, and thus their incomes and nutritional status, these interventions were not considered here because their point of entry is not directly related to the production, distribution, or marketing of agricultural goods and services.

    Similarly, while rural school feeding programs, rural conditional cash transfer programs, rural safety net programs, and food aid are often viewed as important to increasing rural incomes, building rural assets, and improving nutrition, their indirect impact on agriculture means that they were ruled out from consideration here.4

    Once these criteria were met, the potential success had to meet five evaluative criteria:

    Importance--the intervention should have tackled an important food-security problem by addressing the needs of a vulnerable group;
    Scale--the intervention should have operated at scale, measured in terms of whether the number of beneficiaries exceeded several hundred thousand individuals or whether the intervention was, at a minimum, national in coverage;
    Time and Duration--the intervention should have been (1) fully operational at scale long enough to generate significant reductions in hunger or improvements in food security and (2) implemented in the past 50 years;
    Proven Impact--the intervention should have been supported by documented and rigorous evidence of a clear and measurable impact on individual or household hunger or nutritional status; and
    Sustainability--the intervention should have been sustainable, whether in financial terms (cost-effectiveness) or in broader social, political or environmental terms.

  3. Selecting the most appropriate cases. While efforts were initially made to apply each criterion as a necessary (but not sufficient) condition for selection, it was recognized that no case study could meet all criteria. As such, the criteria were relaxed somewhat, although the qualifying criteria were maintained as necessary conditions, along with the following evaluative criteria: importance, scale, and time/duration.

    With respect to the proven impact and sustainability criteria, very few case studies were supported by what may be termed “state-of-the-art” impact assessments that documented the effects of an intervention through randomized experiments that establish attribution by combining before-and-after comparisons with treatment-and-control comparisons. In many cases, such experiments did not exist when the intervention was in operation; in other cases, neither the resources nor the expertise were available to undertake such data-intensive assessments. Relaxing these criteria meant that alternative forms of evidence were accepted. These alternative forms of evidence include: geospatial imagery documenting changes in agroecological landscapes over time; quantitative evidence using estimation techniques that were not necessarily the most up-to-date methods; and qualitative evidence gleaned from policy analyses and from surveys conducted among direct beneficiaries.

    As a result of these criteria, this book does not feature several types of successes. First, the book does not cover nonagricultural interventions, such as rural health, rural education, or rural social protection programs, for example. These programs undoubtedly comprise an important class of intervention, but they do not directly address agriculture in its strictest sense. However, one success case that was included--improving micronutrient consumption in Bangladesh--does feature a rural nutrition program that promotes home-based production of fruits, vegetables, and livestock to improve nutrition and health among the poor.

    Second, this book does not examine programs that integrate agriculture with health, education, microfinance, microenterprise, governance, and other development priorities. These too represent an important class of intervention, but because of the complex synergies between these many activities, it is hard to disentangle the evidence. However, several successes featured in this book, while primarily defined as agricultural development programs, do examine the importance of integrated approaches. The study of community forestry in Nepal, which highlights the importance of integrating agricultural development with local governance, is one such success.

    Third, this book does not cover cases of failure. Often, learning lessons is as much about observing the failures as it is about observing the successes. But while there are many failures in agricultural development from which to learn--and many studies that highlight the causes of their failure--this book chooses to focus on the successes only, primarily because it is the absence of successes in agricultural development that has marginalized its importance in discussions of how to improve food security and reduce hunger in developing countries.

  4. Synthesizing the evidence. Each success highlighted in this book is based on a synthesis of evidence from multiple sources that range from first-hand accounts by individual participants and beneficiaries to large-scale impact-evaluation studies that combine both quantitative and qualitative evidence at the highest levels of academic rigor.

    But few successes are evidenced by a common set of impact-assessment methodologies, indicators, or conclusions--there is no one indicator that can appropriately describe the numbers of millions fed, or the quantitative improvement in food security, resulting from an intervention.

    This may not be a disadvantage to the present analysis when considered more closely. A glance at the chapters in this book indicates that the interventions vary so greatly--in terms of what they aim to achieve, how they do so, and what they actually end up accomplishing--that a single indicator runs the risk of reducing an intervention’s impacts to something entirely impractical.

    Moreover, the casual reader should not conclude that only those interventions that are backed by rigorous impact-assessment materials and definitive indicators are successes. For example, a program that was not rigorously evaluated by teams of independent scholars conducting lengthy household surveys may nonetheless be a success. Or a program that was initially seen as a success may nonetheless fail in the long term.

    In fact, there are many successes in agricultural development that are not covered by this volume but that may have equal merit. Examples include smallholder cultivation of high-value export crops in Kenya and Guatemala; systems of rice intensification (SRI) that have become popular in several countries during the past two decades; New Rice for Africa (NERICA), which is being developed and disseminated for farmers in several Sub-Saharan African countries; or the Greenbelt Movement in Kenya that has encouraged community-based tree planting on a massive scale.

    But even with these caveats in mind, we know that there are clearly discernible pathways--interventions that seek to improve crops, livestock, forestry, and fisheries, conserve natural resources, and strengthen the markets, institutions, and policies that relate to these social and economic activities--that link agricultural development with improvements in food security. These pathways, and the stepping stones along them, are the main focus of impact assessments, and thus the main focus in proving success.

Footnotes

  1. Gabre-Madhin, E. Z. and S. Haggblade. 2004. Successes in African agriculture: Results of an expert survey. World Development 32 (5): 745-766; Levine, R. 2004. Millions saved: Proven successes in global health. Washington, D.C.: Center for Global Development.

  2. World Bank. 2008. World development report 2008: Agriculture for development. Washington, D.C.; World Bank. 2006. Agriculture investment sourcebook. Washington, D.C.; Uphoff, N., M. J. Esman, and A. Krishna. 1998. Reasons for success: Learning from instructive experiences in rural development. West Hartford: Kumarian Press; Krishna, A., N. Uphoff, and M. J. Esman. 1997. Reasons for hope: Instructive experiences in rural development. West Hartford: Kumarian Press; APAARI (Asia-Pacific Association of Agricultural Research Institutions). 2009. APAARI success stories. www.apaari.org/publications/apaari-success-stories/.

  3. World Bank. 2009. Data and statistics: Country groups. go.worldbank.org/K2CKM78CC0.

  4. Conditional cash transfer programs such as the Programa de Educación, Salud y Alimentación (Progresa) in Mexico were, in fact, treated as health (rather than agricultural) interventions by Levine (2004) in the predecessor to this project entitled Millions saved: Proven successes in global health.

Download this methodology document

Download list of submitted nominations