Subnational Institutional Strengthening Project

Transcription

1 Subnational Institutional Strengthening Project COLOMBIA NATIONAL PLANNING DEPARTMENT iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015 : #iegovern

2 What is the problem? What is the problem? - Weak Subnational Governments (SNG) performance in core public management areas, which impact on the effectiveness of public sector service delivery. Intervention(s) - Providing management tools to SNG - Applying incentives for Territorial Management - Providing technical assistance - Strengthening control and territorial management - Promoting effective citizen participation on the public management cycle 1

3 Research Questions Research questions: What are the effects of the program on SNG s management capacities and on public services delivery? Theory of Change Program - Providing management tools to SNG - Applying incentives for Territorial Management - Providing technical assistance - Strengthening control and territorial management - Promoting effective citizen participation on the public management cycle - Efficiency of public management processes and resources allocation - Build capacity of public officials - Data availability (processes and results) - Transparency - Legitimacy of governments Effective and efficient public management and improvement in service delivery 2

4 Methods & Design Method RCT for incentives Randomized time of intervention Difference in differences Clusters / sample size About 1,200 municipalities in 32 departamentos 50 treatments, 50 controls Other How to define treatment? / Attrition (term limits) Heterogeneous treatment effects 3

5 Outcomes & Measurement Outcomes: Intermediate outcomes (Management Quality) - Increase local tax collection - Improve fulfillment of municipal development plan output goals - Modernized management processes and systems - Increase budget execution - Effective citizen participation Final Outcomes - Improvement on services delivery (health, education, water and sanitation, infrastructure) - Higher citizens satisfaction with public services Unit(s) of observation: - Municipalities (local governments), Citizens 4

6 Thank You! WEB facebook.com/ieknow #impacteval blogs.worldbank.org/impactevaluations microdata.worldbank.org/index.php/catalog/ impact_evaluation #iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015

7 National Crime Prevention Policy in Mexico PABLO VÁZQUEZ CAMACHO SECRETARÍA DE GOBEERNACIÓN MÉXICO iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015 : #iegovern

8 Index THE NATIONAL CRIME PREVENTION POLICY IN MEXICO: A QUICK OVERVIEW PROBLEMS WITH IMPLEMENTATION SOLUTION: TOP-DOWN OVERSIGHT / BOTTOM-UP ENGAGEMENT EVALUATION DESIGN EXPECTED OUTCOMES Q&A

9 National Crime Prevention Policy Main Goals: Reduce risk factors and enhance protection factors against violence and crime; promote collective efficacy, especially in high crime communities and enhance trust between citizens and authorities. Plan Nacional de Desarrollo : México en Paz The National Crime and Violence Prevention Program is Mexico s national crime prevention policy. Citizen Security Framework

10 National Crime Prevention Policy Procesos de transformación en los que participan de manera activa: 3 Órdenes de Gobierno 9 Secretarías de Estado Sociedad Civil Organizada Academia Iniciativa Privada Medios de Comunicación Ciudadanía (PRINCIPALMENTE)

11 National Crime Prevention Policy Se ha logrado incidir directamente en sectores específicos de la población con un enfoque territorial para la atención de las comunidades más vulnerables Presencia en 74 demarcaciones de las 32 entidades federativas 241 polígonos 94 municipios y 5 delegaciones del DF, con población mayor a 100 mil hab. 7 zonas metropolitanas Todas las capitales del país El programa toma como base diagnósticos e identifica factores de riesgo, territorios y poblaciones prioritarias

12 National Crime Prevention Policy Drug Abuse Youth Gangs Victimizeed Women Urban Design

13 Identified Problems Poor Project Delivery - Poor Timeliness 1: Before contract - Poor Timeliness 2: After project is awarded - High Cost: compared to other projects in country - Lack of Transparency in assigning contracts - Low Consumer satisafaction: % of people that feel pleased / % of people that know the project - Low Impact Correspondance: Projects are conducted where they are not required according to local assessments Lack of effective top down oversight Lack of effective botton up engagement Poor oversight capabilities Lack of effective enforcement mechanisms Lack of information about projects No enforcement mechanisms

14 Treatment Strategies Top Down Oversight Treatment Package Training Implementors will be trained on administrative processes, plataforms use and other topics that are key to quality implementation Auditing: Conduct preemptive audits on implementors (conducted by Secretaría de la Función Pública); Sanctioning: through publishing preemptive audit results in a specialized transparency website (nosmuevelapaz.org) Bottom Up Engagement Treatment Package Training: Providing direct and indirect beneficiaries with information about specific programs through the internet, town hall meetings and telephone applications Auditing: Providing direct and indirect beneficiaries with basic auditing tool through telephone application Sanctioning: Enabling moral sanctioning of authorities by publishing service delivery scores in a specialized transparency website (nosmuevelapaz.org)

15 Evaluation Design Component Description Questions What type of oversight is more effective in prompting quality of service in the public sector? Top Down? Bottom Up? Alternative: What is the impact of technology in enhancing the effectiveness of bottom up oversight? Unit of Analysis Public Space Recovering Proyects conducted by the National Crime Prevention Policy (PRONAPRED) or by the National Public Space Recovery Program (PREP) in Crime Prevention Polygons Methodology Randomized Control Design (RCT) on 250 Polygons (½ with app, ½ no app) Randomized Contro Design (RCT) on one of the halves in order to have ½ with audit and ½ with no audit Four Groups: Non-Treated, Treated with TD, Treated with Bottom Up, Treated with Both Sample Size At least 750+ public spaces will be recovered in 2015 within the 250 crime prevention polygons Spillover What is the most effective way of delivering construction programs (regional presence or third party implementors exogenous variation on who delivers)? Different Media attention on certain municipalites

16 Expected Outcomes Citizens Engagement Timeliness 1: Reduction in the time spent to complete de administrative process (1/1 match between time spent and timeframe provided by regulation); Timeliness 2: Reduction in time spent in delivering construction work (1/! between planned timeframe and plan spent in implementation; Effective Oversight Cost: Reduction in difference between project costs and benchmark costs at the state level and at the national level Quality Delivery of Services Satisfaction: Increase in the number of people that are aware of the project and increase in the number of people that feel pleased with the project after learning about it s existence Impact Correspondence (geographic): 1/1 correspondence between geographical needs for public space and public spaces built.

17 Treatment Strategies Live Info on Implementation

18 Treatment Strategies Specialized Website Public Implementation Calendar Public Scoring on Implementation Quality

19 Treatment Strategies Phone App Features Direct / Indirect Beneficiary identifier (in the case of public spaces whether they live in the neighborhood or not) Beneficiaries demographics survey On site verification through GPS and image uploading technologies Beneficiary Satisfaction Survey Feedback and citizen Interaction tool

20 Thank You! WEB facebook.com/ieknow #impacteval blogs.worldbank.org/impactevaluations microdata.worldbank.org/index.php/catalog/ impact_evaluation #iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015

21 Making Revenue Collection Easier : PMO RALG TANZANIA iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015 : #iegovern

22 What is the problem? What is the problem? - Poor collection of own source revenue (local tax collection). Intervention(s) - Introducing electronic revenue collection information system - Updating the list of tax payers. - Awareness creation - Sending Specialized reminders to tax payers 1

23 Research questions Research Questions What is the effect of having electronic system in improving revenue collection? What is the impact of reminders that emphasize different motivations on improving revenue collection? Social pressure: This percentage of your LGA is paying Reciprocity: These are the services your LGA uses the money for Information: Now it s easy to pay, and here s how (with the phone, etc.) Threat: If you don t pay on time, here s the penalty. 2

24 Methods & Design 1. Evaluation of electronic tax system: Method: Difference in differences Sample size: Use administrative data on revenue from all Tanzania LGAs, from several years before the system 2. Evaluation of messages to increase tax morale: Method: Randomized control trial Sample size: Implement in 2 LGAs, with >8,000 taxpayers in each group (4 treatment groups + 1 control group) Randomize at individual level o Sharing letters across people is unlikely Design to analyze different effects for business versus individuals, or rich and poor individuals 3

25 Outcomes & Measurement Outcomes: - Increase of own source revenue - Level of tax compliance - Willingness to improve revenue payment - Timeliness of payments Units of observation: - Evaluation of electronic tax system: LGA level Evaluation of tax morale message: Individual level 4

26 Thank You! WEB facebook.com/ieknow #impacteval blogs.worldbank.org/impactevaluations microdata.worldbank.org/index.php/catalog/ impact_evaluation #iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015

27 Tuungane DEMOCRATIC REPUBLIC OF CONGO iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015 : #iegovern

28 What is the problem? Local governance structures do not adequately address the needs of communities around service delivery 1

29 Intervention(s) Increasing service provider accountability to communities in eastern DR Congo. 1

30 Theory of Change 1

31 Research questions Research Questions How does including a community scorecard improve accountability and local service provision? (Bottom-up) How does including line ministry involvement improve accountability in local service provision? (Top-down) Interaction of the two approaches? 2

32 Methods & Design Top-down oversight NO YES Bottom-up community monitoring NO Default 92 VDCs Default + ministry involvement 91 VDCs YES Default + community scorecard 90 VDCs Default + community scorecard + ministry involvement 88 VDCs 3

33 Outcomes & Measurement Outcomes: Accountability Transparency Community Participation Measurement: Survey task for users to request information from service providers Presence in last village meeting and participation Quality of service provided 4

34 Thank You! WEB facebook.com/ieknow #impacteval blogs.worldbank.org/impactevaluations microdata.worldbank.org/index.php/catalog/ impact_evaluation #iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015

35 Community Driven Recovery & Safety SOMALIA DANISH REFUGEE COUNCIL iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015 : #iegovern

36 What is the problem? What is the problem? armed violence and conflict impede realization of the MDGs. (Geneva Declaration on Armed Violence, 2006, p. 1) Current Solutions and shortcomings: - Parallel structures - Only engaging gatekeepers. - Lack a safety and security component. Intervention(s) - Using existing structures - Innovative and tested methods - Adding a safety and security component 1

37 Research Questions Does sustained engagement with communities in the planning process enhance the effectiveness of community development interventions? What is the additional impact of a safety and security component of community development interventions? Does requiring gender quotas on development committees improve women s related outcomes? What is the impact of unconditional grant transfer on recovery, development and safety outcomes at the community level? 2

38 Methods & Design 160 communities 80 communities CDR/S 40 communities Unconditional grants 40 communities Control 40 communities Women quotas 40 Communities CDR

39 IE will focus on a range of development and safety outcomes Unit(s) of observation: Measured at community, household and individual level. 4

40 Thank You! WEB facebook.com/ieknow #impacteval blogs.worldbank.org/impactevaluations microdata.worldbank.org/index.php/catalog/ impact_evaluation #iegovern Impact Evaluation Workshop Istanbul, Turkey January 27-30, 2015

41 Theory of Change 1) If communities are empowered to decide on the design, implementation and evaluation of their own recovery and safety projects; 2) If they experience an environment with enhanced access to security, justice and basic services 3) If local governance structures are strengthened to become more responsive to recovery and safety needs of all members within communities Then CDRS targeted communities in Somalia will experience increased stability. 2