Evidence-based policy research depends on reliable data. But for many countries, publicly-available data do not exist at the scale or granularity needed. Uzbekistan is a striking example. The last nationwide census was conducted in 1989, before independence, and the country’s first post-independence population census, combined with an agriculture census, is planned for 2026. While official statistics and some survey microdata exist, recent, detailed unit-level data—especially on farming—remain limited and hard to access. This scarcity creates pressure on researchers—when we finally get into the field, we want to measure everything at once.
In this context, our team implemented a rare, large-scale farm data collection effort in Uzbekistan in 2024-2025, conducting two complementary surveys through in-person interviews across four provinces: one with 1,632 small farm households and one with 900 larger farmers. Both surveys confronted, in different ways, the challenges of collecting high-quality data in a data-scarce setting.
What follows are reflections on what we learned—both the challenges we expected and the ones we did not—which we hope will resonate with researchers working in similar environments and help them in their own data collection efforts.
Overambitious survey design
The most foundational tension came from the very problem we were trying to solve. Because Uzbekistan is data-scarce, every stakeholder we were working with had urgent information needs. Different research teams, funding streams, and project objectives were layered onto a single instrument, producing a questionnaire that was long, complex, and—in hindsight—too ambitious.
The household survey combined modules on production and assets, gendered decision-making, climate shocks and adaptation, water and energy access, plus several willingness-to-pay and choice experiments. The large farmer survey had its own set of complex farm modules and experiments.
Each module seemed reasonable on its own, but together they led to interviews that typically exceeded an hour and, in the large-farmer survey, often approached two hours, with intricate skip patterns and complex questions requiring significant concentration from both enumerators and respondents. For the survey company and field teams, the survey’s scale and complexity were demanding, contributing to skepticism, confusion, and fatigue on both sides.
Translations, interpretation, and implementation
The second tension appeared in how the instruments were translated and implemented. Even with careful phrasing at the design stage, meaning often shifted once translated and deployed: questions we thought were unambiguous were interpreted differently across enumerators and regions. Words that felt correct in English carried different connotations in Uzbek, especially for abstract ideas like “expectations” or “empowerment.” Subtle distinctions that mattered to us, such as deciding alone versus jointly, had no clear local equivalent. Even with extensive enumerator training, misunderstandings persisted.
Enumerators paraphrased or shortened long or technical questions, trimmed option lists, and improvised explanations when respondents struggled with experimental scenarios. These adaptations were often well-intentioned but not standardized, and small differences accumulated into meaningful variation in how some instruments were understood and answered. Without audio recordings and in-field observation, much of this would have been missed.
When local realities push back
In the smallholder household survey, refusals were unevenly distributed, with some districts recording rates several times higher than others. In some areas, residents told interviewers not to return, sometimes out of discomfort with sharing information. Enumerators were also reluctant to travel to “far” or difficult locations. The survey firm then asked if we could shift remaining interviews to closer, more cooperative districts—a seemingly modest request that would, in practice, distort the sampling plan by pushing the sample toward “easier” areas.
Enumerator availability added another constraint. Many interviewers were teachers who could only work in a narrow window during school holidays. For the large farmer survey, partners warned that fieldwork overlapped with the busiest agricultural period and that some sampled districts were simply hard to reach, pushing them to request substitutions or extra districts to spread the burden.
Enumerator at the center of risk and quality
These design choices and local constraints landed hardest on enumerators. They were the ones facing long surveys, uneven community trust, refusals, tight timelines, and high travel costs in remote areas. Paradata, such as timestamps, interview durations, GPS traces, and recorded enumerator actions, together with audio recordings and back-checks, made visible how these pressures shaped implementation and, ultimately, data quality.
Some enumerators produced longer, more complete interviews, even when it meant lower productivity. Others adapted in ways that severely affected data quality. Monitoring revealed recurring patterns: unusually short interviews, paraphrased or compressed questions, incomplete reading of options, and in some cases, modules that appeared not to have been administered as designed, or at all. These issues were not random but concentrated among a small group of enumerators.
This created a difficult choice for us: keeping data we did not fully trust or redoing all the interviews by specific enumerators.
Quality control as a negotiated process
Given these pressures, rigorous monitoring was essential. We took quality control seriously from the outset, relying on continuous monitoring and follow-up throughout fieldwork. Regular monitoring highlighted patterns that warranted closer scrutiny, including implausible values—such as an unexpected share of “non-farming” respondents in a sample of farming households—alongside unusually short interviews and missing sections.
In response, we reviewed every survey record—not just a random subset—using paradata and monitoring outputs, and relied on audio recordings (where available) and back-checks, discarding and redoing a non-trivial share of interviews, sometimes partially by phone. These corrections strained enumerator morale, stretched limited budgets, and required revisiting respondents in already difficult-to-reach areas. They also exposed the limits of our partners’ monitoring and data-management systems. Although technically strong and collaborative, their existing systems required adaptation to meet the requirements of this project. Together, we developed additional high-frequency checks and automated workflows to keep processing aligned with fieldwork.
In the end, quality control became as much negotiation as technical process, balancing rigor with financial, logistical, and human constraints. The dataset we now use reflects not just methodological principles, but practical tradeoffs made under those constraints.
Reflections: What this experience reveals about data collection in data-scarce settings
Looking back, this experience highlights how hard it is to align ambition with the practical realities of fieldwork, especially in settings where detailed data are limited and the pressure to collect everything at once is strongest.
One central lesson for us is that in contexts like this, it is better to aim for slightly narrower content and more robust delivery.
A second lesson is that a questionnaire is never truly fixed. Translation, interpretation, and day-to-day delivery inevitably reshape how questions are understood in the field. Audio recordings and in-person observation helped us detect this drift, but they did not eliminate it. We now treat this kind of drift as an inherent feature of complex surveys and design surveys accordingly.
Third, local realities quietly reshape sampling plans. Travel barriers, refusals, seasonal workloads, and unanticipated respondent circumstances pull fieldwork toward what is feasible rather than what is ideal. Data collection in data-scarce settings needs to assume that these issues will arise and build in mechanisms to document and manage them, rather than expecting the frame and the realized sample to match perfectly.
A fourth lesson is that enumerators should be treated as a core design constraint rather than just as implementers. When long instruments and uneven field conditions make the job untenable, quality suffers despite close monitoring. Designing for quality therefore requires continuously making the job tenable—through realistic workloads, simpler instruments, clear expectations, and incentives that reward care rather than speed.
Finally, quality control is not purely technical; it is negotiated. Even with strong monitoring tools, it involves choices under time, budget, and field constraints, and these decisions are inseparable from partner capacity, which should be treated as part of survey design rather than a downstream implementation detail. Future projects in similar settings should thus treat monitoring systems, rework protocols, and budget space for corrections as upfront design choices, rather than problems to be solved after data collection begins.
The challenges we encountered are not unique to Uzbekistan; they reflect patterns familiar across many data-scarce settings and worth anticipating elsewhere. Ultimately, the quality of data is shaped not by the elegance of the questionnaire, but by the capacity, incentives, and lived realities of the people who collect and provide it.
Sharanya Rajiv is a former Research Analyst with IFPRI’s Development Strategies and Governance Unit; Tushar Singh is a Senior Research Analyst with IFPRI’s Natural Resources and Resilience Unit based in New Delhi. Opinions are the authors’.
This work was carried out under the CGiAR Scaling for Impact and Gender Equality and Inclusion Science Programs, supported by CGIAR Trust fund contributors. We acknowledge the support of the Ministry of Agriculture Resources of the Republic of Uzbekistan and the International Strategic Centre for Agri-Food Development (ISCAD) in making this survey possible. Finally, we are grateful to the respondents for their time in participating in this survey.
