ICN2-Diet Matters: Approaches and Indicators to Assess Agriculture's Role in Nutrition


Published on

Diet Matters: Approaches and Indicators to Assess Agriculture's Role in Nutrition
By Diego Rose, Brian Luckett, and Adrienne Mundorf
School of Public Health & Tropical Medicine
Tulane University

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Thank you to all who helped organized this meeting, and thanks to Brian Thompson and all his colleagues at FAO that invited me.Title is a play on words, which may not translate well across all the languages. On the one hand, this paper addresses matters of diet related to its measurement at different levels of analysis.On the other hand, we start with the premise that diet is important for understanding how agriculture can improve nutrition.
  • Focusing on dietary measures is challenging because outcomes are multidimensional. There are about 40 nutrients, and many other components of food considered important for a healthful diet. This makes it difficult to evaluate food-based interventions, by looking at nutrients, since results may not always go in the same direction. It is also hard for policy and advocacy purposes. You might find politicians who would be concerned about THE nutrition problem, but certainly not 40 of them.So nutritionists have developed ways to aggregate data into a single index. For example, the mean probability of adequacy averages together the probabilities of adequacy for nutrient under study. There are also ways to do this for food groups.Measurement is also challenging – dietary intake involves a complex set of behaviors, and so-called "Gold standard" measurement methods (i.e. the best methods available), such as the 24-hour recall, are costly.One way of dealing with this is to develop proxy indicators that are cheaper to collect. Diet diversity indicators are simple assessments of diet quality, usually based on just a count of foods or food groups consumed.
  • There has been an explosion of research on validating these diet quality proxies. And yet we are missing universal approaches to this problem.The proxy indicators themselves vary. Researchers look at foods, food groups, and food grouping systems. Some have excluded "minimum quantities" of foods consumed because they might inflate the proxy food count, but not have nutritional relevance.Then there are many benchmarks against which the proxies are validated. Some researchers have used the Mean Adequacy Ratio. Some of the more recent work has moved to using Mean Probability of Adequacy. There is also the Mean Micronutrient Density Adequacy. And even within these benchmarks, there is no consistently-chosen threshold. Some have used 75% as a cut-off point for an acceptable probability of adequacy. Some have used 50%. Some make evaluations at many levels.The benchmark values against which proxies are validated come from different and costly data collection methods. And, perhaps surprisingly, many times the proxies themselves are calculated using data from these same methods.And there are lots of different criteria used to evaluate when proxies perform well. Continuous measures, like the area under the Receiver Operator Curve (a term borrowed from radio electronics to measure signal to noise) might make more sense since the links from proxy to benchmark are not influenced by the specific choice of thresholds or cutoffs, as are measures of sensitivity or specificity. But ultimately, we need thresholds to be able to count those that are affected.
  • Unfortunately these thresholds – i.e. whether it is 6 food groups, or 7 or 8, etc. – tend to be country-specific. The multi-country studies, using similar methods indicate that the best indicators and thresholds are not the same in each country. All of the proxies show significant correlations with the benchmarks indicating that food variety is linked to nutrient intake. But the predictive power of these proxies are only modest. It wouldn't make sense to use these for evaluating impact, but rather instead to be used for population monitoring of project process, or trends over time, or general planning.Eliminating minimum quantities improves predictive power of proxies, but has been rarely tried in a survey-based proxy data collection format.
  • Most agricultural interventions that improve diets operate through improvements to household food security, which is also a multi-dimensional phenomenon. Perhaps the best vehicle for understanding the food available to a household is the food expenditure module, which is typically part of a larger survey used in poverty monitoring. One indicator developed from this module is the food energy available to the household. Expenditure modules are used to assess food poverty, which compares the value of a household's food consumption to the cost of an energy-adjusted, typical food basket. If there is one thing economists want to get right in this type of survey, it would be expenditures on food, since it is such a large part of low-income households' budgets.So both of these could be used to assess interventions which seek to improve household food security.In the last couple of years, the module is getting more attention by nutritionists. Assessing household consumption of food groups has been used for planning fortification studies, and so could shed light on bio-fortification and other specific crop/livestock interventions. There are a number of suggestions on how to make the economist's tool, more friendly to nutritionists, beginning with the way foods are aggregated in the module.
  • Diversity proxies have also been used at the household level. Perhaps the most well known are FAO's Household Diet Diversity Score and WFP's Food Consumption Score. Interestingly, despite the different needs and approaches of the two agencies, they have harmonized how they collect data, so that both types of indicators can be generated from the same survey. None of the household measures, on either this slide or the previous, can address intra-household allocation, so dietary assessment is still needed to see whether improvements at this level are translated into improvements at the individual level.
  • National measures of food supply assist in understanding the availability dimension of food security. The FAO Food Balance Sheet methodology enables calculation of the aggregate amount of food available for human consumption for a list of commodities. The sum of energy available from these foods is divided by the population size to estimate the available calories per capita. FAO uses this as well as additional information about energy needs and measures of variation from other data in order to create its Prevalence of Undernourishment indicator. Others have used food supply data to assess availability of specific nutrients in the food supply, or to assess the quality of the overall food supply. These measures allow for national comparisons over time, but cannot provide insights on the within-country distribution of food insecurity.
  • A number of ongoing data collection systems contribute to our understanding of agriculture's role in diet. The Demographic and Health Surveys (DHS), funded by USAID, collect food group frequency data on children (and previously on women) that allowed for calculation of diet diversity scores, as well as anthropometry, infant feeding practice, and anemia. UNICEF's Multiple Indicator Cluster Survey (MICS) collects similar data and the two systems often coordinate to prevent duplication. The World Bank's Living Standard Measurement Study (LSMS) uses income and expenditure surveys that include a consumption module on food available to households. WFP's VAM unit collects household food security data in comprehensive national and emergency surveys and via an ongoing monitoring system. These surveys routinely collect information on the food consumption score as well as household coping strategies.FAOSTAT is a repository of information from agricultural sources that includes national food supplies.
  • As far as recommendations, I have grouped this part into 3 slides, starting with current best practices.Clarity is needed on where interventions are made within the causal framework and what is planned for change. Assessing outcomes close to the point of intervention allows for understanding if objectives were met. Distal impacts can also be measured, but would likely reflect on other factors not addressed by a programme. For example, using anthropometry to evaluate an agricultural intervention can be misleading since growth is influenced by health, sanitation, and care.Diverse indicators allow for triangulation and a better understanding of changes. So a short- and medium-term strategy is to focus on a number of indicators that have already been developed.The Women's Diet Diversity Score would get widespread use if it is returned to the DHS and MICS survey platforms. The Food Consumption Score provides a simple way to monitor diversity of household consumption, and should continue in WFP's VAM survey system, in addition to the LSMS. Where possible, the LSMS should begin experimenting with a 24-hour diet recall on a target individual in the household. This would allow for a greater understanding of how household food gets translated into individual consumption. Finally, use of the Food Balance Sheet data and calculation of the related Prevalence of Undernourishment indicator should be continued as a way of providing insights into the availability dimension of food security.
  • Often researchers focus their validation efforts on a newly-developed indicator. But there is wide variation in the 'gold standards' employed, the criteria for judging success, and the country-level data being used. This makes it difficult to draw conclusions about which indicators are most effective. Integrated research is needed that allows for better comparisons of different types of indicators. Much of the testing of proxy indicators has been justified on the grounds that the 'gold standard' for dietary intake (e.g. the 24-hour recall) is too costly. Yet costs of the proxy approach are rarely reported, which limits our ability to make useful decisions about which indicator to support. Technological changes allow for new possibilities in survey implementation. Distance learning can assist capacity-building to implement complex modules. Smart phones and tablets offer applications that can simplify the interview process, reduce data entry errors and costs and achieve rapid data transmission. Given these developments and the middling performance of many diet proxies, a long-range strategy should develop state-of-the-art measurement procedures more economically, rather than continue the emphasis on proxy approaches. Good field methods do not exist for assessing energy expenditure, which is central to assessing an individual's dietary adequacy. One approach tested in the U.S., uses a 24-hour time diary recall, and merges the information from this with reference values on the energy cost of activities. Adaptations and testing will be needed to use this or other methods in the context of low-income countries. 
  • Reliable data collection is costly, so we should look for synergies between agencies wherever possible. One approach would be for agencies to pay for a module to be included in an existing survey. For example, WFP has sponsored its food frequency modules on the World Bank's LSMS surveys. The cost is much less for WFP than a national survey, and the World Bank benefits by having data to carry out additional country-level analyses.  Specific variants within a class of indicators such as diet diversity use similar data elements. A harmonized platform of data collection would allow for different indicators of the same class to be calculated from the same data, advancing our knowledge of which of different indicators works best. FAO and WFP have harmonized their data collection instruments so that their diversity indicators can be calculated from the same data. Individuals and agencies develop attachments and even constituencies around specific indicators. But there are too many variations of specific indicators. To be more efficient in accumulating knowledge of dietary changes, we should develop standardization in the way indicator data are collected, analyzed and interpreted. Panels composed of a diverse set of experts from a broad set of disciplines and countries which are financed jointly by interested agencies can facilitate this approach. Results of these panels should be distributed widely, including to academic journal editors, so that researchers, agency officers and ministry officials use common approaches to collecting and analyzing data.  Many indicators have been validated using continuous measures, but thresholds are needed so we can count the affected and determine the magnitude of a problem. Thresholds require judgment, but too often cut-points have been determined in an ad-hoc way. Expert interdisciplinary panels are needed to make judgments about where cut-points should be drawn and to communicate the method and reasoning behind their approach in a transparent way. At a minimum, the panels should include professionals from the nutrition, economics, communications and policy fields.
  • ICN2-Diet Matters: Approaches and Indicators to Assess Agriculture's Role in Nutrition

    1. 1. Diet Matters: Approaches and Indicators to Assess Agriculture's Role in Nutrition Diego Rose, Brian Luckett, and Adrienne Mundorf School of Public Health & Tropical Medicine Tulane University PREPARATORY TECHNICAL MEETING FAO Headquarters, Rome, Italy 13-15 November 2013
    2. 2. Main objectives • Outline plausible mechanisms in which agriculture and food-based interventions can improve nutrition through the diet pathway • Identify approaches and indicators to measure progress in this area
    3. 3. Outline • Augmented Causal Framework • Indicators – Individual – Household – National • Data Collection Systems • Recommendations
    4. 4. A Standard Framework Outlining Causes of Undernutrition Short-term consequences Long-term consequences Maternal and child undernutrition Disease Inadequate dietary intake Unhealthy household environment and inadequate health services Household Food Insecurity Inadequate care and feeding practices Household access to adequate resources: land, education, employment, income, technology Inadequate financial, human, physical and social capital Outcomes Immediate causes Underlying causes Basic causes Sociocultural, economic, and political context
    5. 5. Zoom-in on determinants of household food security Household Food Security (Adapted from WFP, 2005) Food purchases Food receipts Gathering Fishing Hunting Cash income Own production (food & cash crops, livestock, fish farm) Sales Employment Non-agricultural production Trading Cash receipts Debts
    6. 6. A Standard Framework Outlining Causes of Undernutrition Short-term consequences Long-term consequences Maternal and child undernutrition Disease Inadequate dietary intake Unhealthy household environment and inadequate health services Household Food Insecurity Inadequate care and feeding practices Household access to adequate resources: land, education, employment, income, technology Inadequate financial, human, physical and social capital Outcomes Immediate causes Underlying causes Basic causes Sociocultural, economic, and political context
    7. 7. Individual-Level Measures • Dietary outcomes are challenging due to multi-dimensionality – many nutrients, all essential • So, aggregation of data into a single index – e.g. Mean Probability of Adequacy • Measurement is also challenging, costly – diet is a complex set of behaviors – 'Gold standard' methods are costly • Proxies for diet quality – simple indicators of diet diversity
    8. 8. Dietary Proxy Validation Research Lack of universal approaches • Different indicators – Foods, food groups, fd grp systems ± min quantity • Many different benchmarks – MAR, MPA, MMDA – Several different cutoff points – 50%, 75%, etc • Different collection methods – Weighed food record, 24-hour recall, food frequency • Different criteria – Correlations, sensitivity, specificity, area under ROC, prediction equations
    9. 9. Diet Proxy Validation Research Outcomes • Modest predictive power – good for population monitoring: project process, trends over time, or general planning – not good for evaluating impact • Best proxies tend to be country-specific • Elimination of 'minimum quantities' improves estimates – can it be implemented in a low-cost field method?
    10. 10. Household Level Measures Food Expenditure Modules • Energy availability, food poverty – household food security interventions • Food group availability – biofortification studies – other specific crop/livestock interventions
    11. 11. Household Level Measures Diversity Proxy Indicators • Household Diet Diversity Score – 16 groups in data collection collapsed to 12 – 24 hr recall (not quantitative) – 1 point for each group • Food Consumption Score – 8 groups – 1 week food frequency – weights used in calculation
    12. 12. Country Level Measures Using Food Supply Data • Prevalence of undernourishment – energy per capita + distributional measure + threshold • Micronutrient densities – micronutrients per 1000 kilocalories – compared to micronutrient density goals • Healthy eating index – score based on U.S. diet guidelines – food groups, negative components (fats, sugars)
    13. 13. Data Collection Systems • Demographic and Health Surveys – Measure USAID • Multiple Indicator Cluster System – UNICEF • Living Standard Measurement Study – World Bank • Vulnerability Analysis and Mapping Surveys – World Food Program • FAOSTAT – National food supply data, other info
    14. 14. Recommendations Best Practices • To evaluate agricultural programs & policies: assess outcomes proximal to interventions • Continue to foster a diverse set of indicators for population monitoring – Women's Diet Diversity Score on DHS, MICS – Food Consumption Score on VAM, LSMS – Pilot-test 24-Hr Recall on LSMS – Food Balance Sheet indicators
    15. 15. Recommendations Research & Development • New proxy validation research should – integrate several indicators – include information on costs • Focus on making 'gold standards' less costly, rather than on making more low-cost proxies • New research is needed on developing indicators of energy expenditure
    16. 16. Recommendations Nutritional Diplomacy • Seek inter-agency collaboration in – survey implementation – instrument, indicator development • Use representative expert panels to develop consensus on – specific indicators, overall measurement approach – thresholds to count the affected