Enabling evaluation: Building capacity with our partners
March 10, 2016Since 2014, IFPRI’s Markets, Trade and Institutions Division has partnered with FONGS, a federation of farmers’ associations in Senegal, to test an intervention on the alleviation of informational and capital constraints faced by smallholder farmers. Building capacity and capitalizing on IFPRI’s experience in evaluation is a key project goal. To meet this goal, IFPRI is working with FONGS in two main areas: training a subset of project facilitators as enumerators for data collection and providing feedback and tools to improve FONGS’ standard agricultural assessment tool for future use. Eight farmers’ associations are involved in the project, each of which oversees 10-15 facilitators who work with project households by conducting monthly advisory visits and agricultural assessments and planning.
In April 2015, we trained eight facilitators (one from each association) for midline data collection. We are currently preparing for endline data collection training with 16 facilitators. FONGS plans to use these 16 individuals to train other facilitators and to lead their associations in improved monitoring and evaluation.
FONGS’ facilitators are typically farmers themselves, working and living in the zone where their association is based. They are hired for their dynamism, commitment, and facilitation skills, not because they have experience administrating a complex monitoring or evaluation tool. Our eight midline facilitators had an average education equivalent to 9th grade, and most of them had never administered a complex household questionnaire before. None of them had ever used a tablet for data collection. Our capacity-building strategy included hiring four external supervisors to mentor and supervise the local facilitators during training and data collection. These supervisors were all university graduates with extensive field research experience, who had participated in countless surveys using Open Data Kit (ODK), the program we chose to use for data collection. The facilitators worked closely with their supervisors and IFPRI trainers throughout the five-week training and data collection period. They mastered interview techniques and concrete skills related to the use of tablets for data collection, math (estimating the value of specific food items when households buy a mixed basket of vegetables at a fixed price, calculating the average value of household assets, etc.), and understanding the importance and meaning of obtaining informed consent.
This intensive training in quantitative data collection and experience has provided a new set of knowledge and skills to FONGS’ associations. Their facilitators are now more invested in the long-term development and success of their associations (this project’s facilitators have, to date, already worked for an average of 10 years with their respective associations), so investing in facilitators’ skill sets in turn strengthens the associations’ abilities to conduct monitoring and evaluation activities.
In addition to training facilitators, we have also provided feedback and tools to improve our partners’ basic agricultural assessment (BAA) tool. Administered on an annual basis, the BAA is a questionnaire that captures basic information on household composition, revenue, agricultural and livestock production, and agricultural and non-agricultural expenditures. The BAA is designed to facilitate discussion and identify opportunities for improvement by informing the household, the facilitator, and the farmers’ association of the household’s agricultural and financial situation. The BAA is being made part of FONGS’ basic package of services and is used to annually monitor several thousand households across Senegal. FONGS has also shared this tool with farmer organizations in Mali, Chad, Niger, Guinea, Guinea Bissau, Cameroon, Nigeria, and Benin; this knowledge-sharing could potentially enable comparisons across West Africa.
The BAA is used in our project to conduct annual assessments, serving as a source of data and the basis of content for facilitators’ monthly advisory visits. During cleaning and preliminary analysis of BAA 2014 data, we targeted concrete areas for improvement in order to develop a modified tool that would be easier for facilitators to administrate and that would produce cleaner, more comprehensive data for monitoring and evaluation use. We met with FONGS prior to the 2015 and 2016 BAA administration to share suggestions based on the previous year’s data, and create improved versions of the tool together. Changes implemented included: modifying written instructions to clarify concepts that were not being respected during administration; adding crops and livestock to their respective modules to better capture their contribution to livelihoods; inserting a small module on crop by-products, which are a source of agricultural income but were not previously measured in the BAA; reworking sections to ensure that categories were comprehensive and mutually exclusive; and modifying the formatting in several modules to address data inconsistencies which clearly originated from a misinterpretation of how to administer and fill in those modules.
The BAA is a paper-based tool that requires manual data entry to make the data useful for monitoring and evaluation. Another major contribution of the project to FONGS’ BAA process was the development of a data entry mask by IFPRI staff. This mask is an easy-to-use, Excel-based form set up to visually mimic the paper BAA. The mask prompts the data entry agent and imposes basic constraints to ensure cleaner data, produces a database of all households, and even calculates summary statistics on variables of interest to FONGS (such as total value of crop and livestock production, total expenditures, etc.). FONGS’s monitoring and evaluation team can now more easily and effectively monitor and track annual changes in the households with which they work. FONGS plans to continue using this mask for future BAA data collection beyond the scope of this project.
We believe the capacity-building activities of this project will enable strengthened monitoring and evaluation at many different levels: at the local level by households, facilitators, and associations; at the national and regional levels by FONGS, government stakeholders and policymakers; and among other West African agricultural organizations.