IMPAQ INTERNATIONAL LLC. PRESENTER: ANNE CHAMBERLAIN

Transcription

1 IMPAQ INTERNATIONAL LLC. PRESENTER: ANNE CHAMBERLAIN

2 AMERICAN EVALUATION ASSOCIATION DESIGN THINKING AS FRAMEWORK FOR EVALUATION + BEHAVIORAL SCIENCE COLLABORATION

3 OVERVIEW Design Thinking and Human-Centered Design Behavioral Science Where Program Evaluation and Behavioral Science Intersect Where They Don t How Should We Work Together?

4 DESIGN THINKING Learning from the design process. Plan The systematic (if not explicit) decision-making that yields a design. Act Do Study

5 HUMAN-CENTERED DESIGN A process that yields Products products or processes that are made in response to human behavior and decisionmaking. Human Behavior Programs Processes

6 DESIGN THINKING + HUMAN-CENTERED DESIGN Focus on how decisions are made Draw heavily from behavioral science

7 DESIGN THINKING + HUMAN-CENTERED DESIGN Focus on how decisions are made Behavioral Science Draw heavily from behavioral science

8 BEHAVIORAL SCIENCE Rigorous, iterative testing Quantitative evidence of patterns in behavior (decisions) Use of evidence-based changes to fix program snags

9 BEHAVIORAL SCIENCE Energy consumption report-cards 401-k opt-out default

10 NOTHING NEW, PER SE Graphic designers, instructional designers 60% 50% 40% 55.74% Anthropologists, 30% sociologists 20% 21.83% 12.18% 10% 0% Director or Administrator 6.7% Staff Pathologist 3.15%.41% Educator

11 WHY NOW Behavioral science is in the RFPs Behavioral scientists have the ear of policymakers

12 BEHAVIORAL SCIENCE Contract Purpose Evaluand Role of BS BS Requirements ACF: Exploratory Existing TANF offices Inform approach knowledge will be helpful TANF that have changed org to research culture & exemplars DOL: X- Improve program Small, BS-informed Intervention PI: 5 years applying social & BS Progrm performance innovations Investigative principles to social welfare approach programs. PhD in BS or another relevant field. DOL: Identify relevant BS TAPS program (QED) Program TWG VETS principles & findings for Small enhancements enhancements PI: 5 years applying social & BS TAP program (e.g., principles in empl/training Improve program intervention) progs. PhD in BS or another performance relevant field. Impact study ED: IES Impact of parent Evaluator will help Intervention TWG messaging strategies determine message Investigative frequency, approach sequencing, content

13 INTERSECTIONS: PURPOSE To improve social programs Responsibilities for General Public Welfare To produce positive social change Enhancement of the Public Good Service in the Public Interest Fostering Public Understanding

14 INTERSECTIONS: TRAINING & TOOLS Research Surveys Interviews Observations

15 DIFFERENCES: ROOTS AND PERSPECTIVE Anthropology Sociology Maths Business Public Policy Psychology Social Justice Hard Science, Empirical Research Cultural Efficiencies Responsiveness

16 DIFFERENCES: ENTRY POINT Program Evaluators can arrive at multiple Program Lifespan points in program life preferably early Behavioral Scientists typically brought in late Pre Early Mature Implementation

17 DIFFERENCES: GRAIN SIZE Program Behaviors Program Components Decision points

18 DESIGN THINKING Involve behavioral scientists at the start Program Lifespan Collaboration on formative evaluation Summative evaluation remains purview of program evaluators Pre Early Mature Implementation

19 CLOSING THOUGHTS When does a nudge become a program? When the nudge is a program, should we work from opposite ends?

20 CLOSING THOUGHTS How can we use reflective practice to learn from working together?

21 CONTACT INFORMATION Anne Chamberlain