Going Beyond the Numbers: Critical Methodologies for Measuring, Assessing and Evaluating Change

Transcription

1 Ging Beynd the Numbers: Critical Methdlgies fr Measuring, Assessing and Evaluating Change Dr. Heather Metcalf, AWIS Dr. Liliana Rdriguez-Camps, University f Suth Flrida Dr. Jerland Jacksn, University f Wiscnsin-Madisn

2 Disrupting the Pipeline: Critical Analyses f Student Pathways Thrugh Pstsecndary STEM Educatin Dr. Heather Metcalf Directr f Research and Analysis, AWIS metcalf@awis.rg

3 Originated in the 1920s Critical Thery Builds n Marxist critiques f capitalism t ffer brader scial and cultural critiques (Habermas, 1971) Histry f emancipatry and scial-transfrmative gals Reveals hidden pwer arrangements (Baez, 2007) Reading between the lines f idelgy (Metcalf, 2014) Attentive t language, knwledge, pwer, ppressin, and their reprductin Relatinship between cncepts and subjects Argues fr intersectinality, interlcking systems f ppressin, tapestry (Crenshaw, 1991; St, 2005)

4 Feminist Science Studies Applies critical thery t science as sci-cultural text What is the feminism f FST? Nt just abut the study f wmen and/r gender Brader study f entangled & inseparable systems f pwer, ppressin, & subject frmatin Cnsiders the histry f hierarchy, ppressin, exclusin within STEM (Hacker, 1989; Oldenziel, 2004; Tbias, 1993) Raises questins abut: (Haraway, 1991; Harding, 2006; Keller, 2001) Bias, knwledge cnstructin in STEM schlarship Ntin f bjectivity Gendered, classed, sexualized, racialized assumptins STEM s cnnectin t militarism, envirnmental destructin, glbal capitalism, & research used t prve inferirity f wmen & peple f clr

5 Critical Methdlgies Many critical therists use qualitative methdlgies, but imprtant t engage quantitative wrk critically Allw fr reflectin upn the ways in which we: Cllect Measure Interpret Analyze Has transfrmative ability Must be understd within cntext Ex: Survey design

6 Applying Critical Mixed Methdlgies Critical discurse analysis f STEM wrkfrce studies and surveys, paired with critical regressin analyses f 2006 SESTAT data (Leaky) Pipeline mdel f recruitment and retentin Pipeline histry 1970s, lingering Cld War fears paired with technlgical & glbal cmpetitin NSF mdel t quantify and predict number f scientists and engineers needed Q1 + fi - f = Q2 Where Q1 = the number f peple in stck at the beginning f perid, fi = the sum f flws int the stck, f = the sum f flws ut f the stck, and Q2 = the number f peple in stck at end f perid Basis f R&R effrts aimed at wmen & minrities in STEM fr 40+ years, but prblems f inequity, underrepresentatin & wrkfrce shrtage claims persist

7 Disrupting the Pipeline Pipeline mdel survived despite much criticism Flawed predictins Supply-side fcus Pr measurements Linearity and inability t accunt fr varied career paths Tendency t hmgenize peple, fields, sectrs, and stages Discursive view f peple as passive pipeline flw Lack f fcus n systemic change and pwer relatins, particularly thse related t gender, race, class, sexuality, ability, and citizenship

8 2006 SESTAT Surveys NSF Scientists and Engineers Statistical Data System (SESTAT) Natinal Survey f Cllege Graduates Natinal Survey f Recent Cllege Graduates Survey f Dctrate Recipients STEM Measures scientist r engineer S&E fields S&E degrees Demgraphic Measures gender (sex) race ethnicity Retentin Measures pipeline relatedness

9 Critical Regressin Analyses Binary Lgistic Regressin: Traditinal Pipeline Mdel Retentin: Degree in STEM, Jb in STEM Demgraphics: Aggregated, mutually exclusive categries Disaggregated, mutually exclusive categries Disaggregated, interacted categries

10 Critical Regressin Analyses Ordered Lgistic Regressin Retentin: Degree-ccupatin Relatedness Demgraphics: Aggregated, mutually exclusive categries Disaggregated, interacted categries

11 Using critical methdlgies can Cnclusin Deepen ur understanding f retentin and demgraphic issues Generate a richer pl f resurces achieving, rather than undermining, equity-related gals Help us see the subjective chices made in the cnstructin and analysis f large-scale data sets By calling int questin ur wn and thers underlying assumptins, methds, mdels, and measures, bradening ur thinking, and richly cntextualizing ur wrk, we step utside ur cmfrt znes t pen up the pssibility f seeing ur institutins and ur wrlds differently and changing them. New Directins fr Institutinal Research, Special Issue n New Schlarship in Critical Quantitative Research, 2014

12 1321 Duke Street Suite 210 Alexandria, VA (703) (phne) (703) (fax)