Farmer field schools: from agricultural extension to adult education

Transcription

1 International Initiative for Impact Evaluation Farmer field schools: from agricultural extension to adult education Making Impact Evaluation Matter Manila, 1 st -5 th September 2014 Hugh Waddington, Birte Snilstveit, Jorge Hombrados, Martina Vojtkova, Daniel Phillips, Philip Davies, Howard White

2 Implementation history

3 FFS project objectives Most FFS projects focus on agricultural production and food security Increasing focus on empowerment

4 Polarised debate on FFS...the program did not have significant impacts on the performance of graduates and their neighbors Results demonstrated remarkable, widespread and lasting developmental impacts

5 What does a systematic look at the evidence say? Literature reviews are like sausages... I don t eat sausages as I don t know what goes into them Dean Karlan

6 A standard systematic review of effects 28,000 study titles and abstracts screened 460 relevant FFS studies screened at full text rigorous impact evaluations: 15 projects

7 3ie s integrated synthesis of systematic reviews 28,000 study titles and abstracts screened 460 relevant FFS studies screened at full text targeting: 92 projects rigorous impact evaluations: 15 projects

8 3ie s integrated synthesis of systematic reviews 28,000 study titles and abstracts screened 460 relevant FFS studies screened at full text targeting: 92 projects rigorous impact evaluations: 15 projects barriers and enablers: 20 projects

9 3ie s integrated synthesis of systematic reviews 28,000 study titles and abstracts screened 460 relevant FFS studies screened at full text global project portfolio: 327 projects targeting: 92 projects effects: rigorous IEs 15 projects barriers and enablers: 20 projects benefit/ cost estimates: 4 projects high risk of bias IEs (77 projects)

10 Farmer field school intervention Inception Targeting Farmer training Dissemination Training of FFS facilitators Curriculum development Finance & systems Effective targeting mechanisms Farmers are able to take part Farmer group formation Season-long training attended by farmers Facilitation through discovery-based group learning Field days, exchange visits Platform building Training of farmer trainers

11 Theory of change: intermediate outcomes Inception Targeting Farmer training Dissemination Capacity building Adoption Diffusion Legend: Intervention components Intermediate outcomes

12 Theory of change: impacts Inception Targeting Farmer training Dissemination Capacity building Adoption Diffusion Legend: Intervention components Intermediate outcomes Impacts Impact Sustainability

13 Policy: prices of inputs; regulations Relationship between input producers and extension/training system Market access for farmers Existing farmer practices Inception Targeting Farmer training Dissemination Capacity building Adoption Diffusion Legend: Policies & practices Intervention components Intermediate outcomes Impacts Impact Sustainability

14 Presentation 1: Systematic review of targeting

15 Presentation 2 Systematic review of impacts and barriers to and enablers of impacts Martina Vojtkova 3ie

16 3ie s integrated synthesis of systematic reviews 28,000 study titles and abstracts screened 460 relevant FFS studies screened at full text global project portfolio: 327 projects targeting: 92 projects effects: rigorous IEs 15 projects barriers and enablers: 20 projects benefit/ cost estimates: 4 projects high risk of bias IEs (77 projects)

17 Integrated synthesis along the causal chain Context Inception Targeting Farmer training Dissemination Capacity building Key Policy environment Intervention components Adoption Diffusion Intermediate outcomes Impacts Impact Sustainability

18 Improved outcomes for FFS participants

19 Risk of bias assessment: Included impact evaluations 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Overall risk of bias assessment ) Selection bias & confounding ) Spillovers & contamination ) Outcome reporting bias ) Analysis reporting bias ) Other sources of bias 97 3 Low risk of bias Medium risk of bias High risk of bias

20 Critical appraisal: Studies of barriers and facilitators Research aim 100% Context 100% Sampling 80% 20% Sampling characteristics Data collection 90% 95% 10% 5% Data recording 35% 40% 25% Analysis 35% 40% 25% Link to relevant lit/ theory 85% 5% 10% Appropriate methodology 50% 45% 5% Appropriate sampling 40% 50% 10% Appropriate methods of data collection 40% 60% Appropriate analysis 15% 20% 55% 10% Triangulation 85% 15% Clarity of analysis and conclusions 25% 10% 65% Conflict of interest 15% 80% 5% Ethical considerations mentioned 25% 70% 5% Yes No Partially Unclear

21 Summary meta-analysis findings FFS participants

22 Barriers and facilitators of knowledge Training, skills, experience an characteristics of facilitators Top down vs participatory approach to training delivery Relevance and appropriateness of curriculum Language and terminology Resources for FFS schools Farmers motivation

23 Weak evidence of improved empowerment Only one quantitative study reported increased perceived ability to solve problems, express their opinion and participate in the community Qualitative studies: Self-confidence Improved social capital Leadership roles Gender relations

24 Summary meta-analysis findings FFS participants

25 Barriers and facilitators to adoption Approach to training delivery Relevant and appropriate curriculum Conflicting agricultural policies, institutional legacy and incentives Social capital Observability of benefits Complexity Access to inputs / capital Active follow up

26 Summary meta-analysis findings FFS participants

27 Improved yields may not be achieved at scale Study ID ES (95% CI) Pilot project Peru: DE Foundation Coffee Project China: FAO/EU IPM Programme for Cotton in Asia Pakistan: Cotton IPM programme (FAO, EU, GoP) Ecuador: Plataformas Program (FAO) Tanzania: East African Sub-regional Pilot Project Phase II (FAO) Pakistan: FAO/EU IPM Programme for Cotton in Asia Kenya: East African Sub-regional Pilot Project Phase II (FAO) Ethiopia: Participatory Forest Management Project in the Belete-Gera Regional Forest Priority Area Subtotal 1.13 (1.05, 1.22) 0.86 (0.63, 1.18) 1.08 (1.03, 1.14) 1.09 (1.03, 1.15) 1.22 (0.97, 1.53) 1.23 (1.00, 1.51) 1.24 (1.01, 1.54) 1.81 (1.15, 2.84) 2.71 (1.11, 6.60) 1.13 (1.05, 1.22) Scaled-up programme Vietnam: FAO Programme for Community IPM in Asia Indonesia: National IPM Training Project Phase II (FAO, World Bank, GoI) Subtotal 0.98 (0.96, 1.01) 0.97 (0.72, 1.31) 0.98 (0.96, 1.01) 0.98 (0.96, 1.01) FFS reduces yields FFS improves yields

28 No evidence of diffusion to FFS neighbours

29 Why no diffusion? Experience-based nature Complexity of practices Observability of benefits Social cohesion Socioeconomic differences

30 Funnel of attrition explains successes & failures

31 BCRs>1 require strong assumptions on adoption

32 Implications for policy FFS is effective in pilot projects Average improvement in yields 10% Average improvement in net revenues 18% FFS does NOT appear effective in scaled-up programmes Problems in recruiting and training facilitators Problems of price distortions and other policy factors Problems in targeting marginalised farmers Even in pilot programmes, FFS may NOT be cost-effective Evidence suggests there is little or no diffusion Very little evidence available on empowerment

33 Implications for programmes Recruitment and training of FFS facilitators Complementary interventions: up-stream/down-stream Targeting: location (areas with clear needs), participants (time availability, complementary inputs, relevant curricula) Sustainability: formal support/back-stopping of FFS alumni Scale-up: gradual approach favouring depth over breadth Intervention design?

34 Implications for research and evaluation Rigorous impact evaluations measuring empowerment outcomes Factorial designs examining training intensity Long-term follow-up and scale-up Factual data from FFS facilitators and extension workers

35 Thank you Policy-friendly report: Full technical report: