LIFE-M. Longitudinal, Intergenerational Family Electronic Microdata

Transcription

1 LIFE-M Longitudinal, Intergenerational Family Electronic Microdata Martha J. Bailey Professor of Economics and Research Professor, Population Studies Center University of Michigan

2 What is LIFE-M? A large scale public database that Spans late 19 th and entire 20 th century Uses vital records as a basis for linkage Combines vital records to census data (1880, 1900, 1940) Release planned: 2020 Main Contributions Birth to death coverage: birth family structure, marriage family structure, economic variables, mortality Intergenerational coverage for four generations Add women to historical linked data Geographic detail at vital events enables other links Limitation: not all states have vital records

3 LIFE-M s Generational Structure LIFE M cohorts include the veterans of two World Wars and the children of the Great Depression. This group largely defines aging Americans in the last half of the 20th century Core sample between UA and HRS cohorts; other generations overlap with these other samples

4 LIFE-M s Data Structure 1900 Census birth place, race, occupation, age, address 1880 Census G0 parents; Birth place, race, occupation, address G1 as children, G1 siblings Births Records infant full name (G2), day & place of birth, parents birth names (G1) Link girls to their marriage families Death Records decedent full name (G0-G2), parents names (G0-G1), day & place of death Marriage Records bride & groom full names (G1-G3), day & place, parents names (G0-G2) Census G0, G1, G2 birth place*, children born*, age marriage*, spouse name*, age*, occupation, education, employment, wages, address G3 as children: birth place*, siblings Key: G0 born <1860 (~UA cohorts); G1 born ; G2 born ; G3 born (~HRS cohorts)

5 LIFE-M s Contributions 1. Large scale dataset covering the early 20 th century to provide longitudinal and intergenerational information for health and economic outcomes; variables include

6 LIFE-M s Contributions 2. Unprecedented coverage of women and large samples of racial minorities and immigrants Birth records contain mother s birth name and married names (so do marriage records) Larger sample sizes for underrepresented groups and common names SES gradient in childbearing (less educated, foreign born, racial minorities tend to have more children) Lower SES are also less likely to be linked across censuses (illiteracy, common names, minor name changes are all problems) Birth records are effectively an oversample of this underlinked group, allowing us to correct errors, disambiguate names, and link with greater certainty

7 LIFE-M s Contributions 3. Geographic information facilitates linkages to other (policy!) datasets Place of BMD event (town/county) or census enumeration provides geography for linking LIFE M to administrative data, policy measures, or other datasets containing contextual, demographic, and economic variables for specific research purposes Facilitates analyses of long run effects

8 LIFE-M s Contributions 4. Potential enhancements to modern data: Background variables for many modern surveys (e.g., PSID, HRS) Potential links to IPUMS historical census Potential links to CLIP project and AoS

9 HOW WELL DO AUTOMATED LINKING METHODS PERFORM? EVIDENCE FROM THE LIFE M PROJECT Martha Bailey 12, Connor Cole 1, Morgen Henderson 1, and Catherine Massey 1 1 University of Michigan and 2 NBER

10 How Well Do Automated Linking Methods Perform? Automated linking forms the basis of many on going big data projects Hand linking is cost prohibitive Lack of ground truth limits knowledge regarding the performance of different automated linking methods in historical settings and samples

11 What this Paper Does Use 2 new ground truth samples LIFE M: Birth certificates for Ohio boys born linked to 1940 Census; double clerical review with discrepancy resolution Oldest Old Union Army vets: Dora Costa (2016) Evaluate the performance of 3 automated linking methods: Ferrie (1996), Abramitzky et al. (2012), Feigenbaum (2016) 2 phonetic name cleaning methods: NYSIIS and Soundex 1 method of record disambiguation: Nix and Qian (2015)

12 Ground Truth Samples This talk focuses only on LIFE M Ohio Boys 19,000 Ohio birth certificates random sample of boys born between 1909 and 1920 and their siblings linked to the 1940 census Vetted against genealogical method: Joe Price at BYU used family history students to hand link ~500 of our boys to the 1940 census: 96 percent of links agree (4% disagreement)

13 How LIFE M Data is Trained Blind, independent review process Two highly trained RAs choose from a set of computergenerated candidate links using name, date of birth (or age), and birth state In the three percent of cases where the two initial reviewers disagree, the records are re reviewed by an additional three individuals to resolve these discrepancies We also use weekly meetings to discuss difficult linking cases and random audit batches to monitor the quality of data links for each trainer

14 Performance Criteria 1. Match rates 2. Representativeness of data 3. Type I error rates (false positives/bad links) 4. Type II error rates (false negatives/links not found)

15 Match Rates: LIFE M Boys LIFE-M Ferrie 1996 (Name) Ferrie 1996 (NYSIIS) Ferrie 1996 (SDX) Ferrie 1996 (Name) - all potentials Ferrie 1996 (NYSIIS) - all potentials Ferrie 1996 (SDX) - all potentials Abramitzky et al. (Name) Abramitzky et al. (NYSIIS) Abramitzky et al. (SDX) Abramitzky et al. (Name) - Corrected Abramitzky et al. (NYSIIS) - Corrected Abramitzky et al. (SDX) - Corrected Nix and Qian (Name) Nix and Qian (NYSIIS) Nix and Qian (SDX) Nix and Qian (Name) - all potentials Nix and Qian (NYSIIS) - all potentials Nix and Qian (SDX) - all potentials Machine Learning

16 Performance Criteria 1. Match rates All methods perform well 2. Representativeness of data No method performs well 3. Type I error rates (false positives/bad links) 4. Type II error rates (false negatives/links not found)

17 False Positives/Incorrect Links Process for determining T1 errors in LIFE M: Generate links for other automated methods Treat agreements of automated method and LIFE M as true Clerical RE review of disagreements

18 Police Line Up Other nonlink Other LIFE-M Other link nonlinkother nonlink 2 Data Trainers + Review by 3 others if disagreement

19 False Links: LIFE M Boys LIFE-M Ferrie 1996 (Name) Ferrie 1996 (NYSIIS) Ferrie 1996 (SDX) Ferrie 1996 (Name) - all potentials Ferrie 1996 (NYSIIS) - all potentials Ferrie 1996 (SDX) - all potentials Abramitzky et al. (Name) Abramitzky et al. (NYSIIS) Abramitzky et al. (SDX) Abramitzky et al. (Name) - Corrected Abramitzky et al. (NYSIIS) - Corrected Abramitzky et al. (SDX) - Corrected Nix and Qian (Name) - all potentials Nix and Qian (NYSIIS) - all potentials Nix and Qian (SDX) - all potentials Machine Learning 0.02 Linking decisions reversed

20 Missed Links: LIFE M Boys LIFE-M Ferrie 1996 (Name) Ferrie 1996 (NYSIIS) Ferrie 1996 (SDX) Ferrie 1996 (Name) - all potentials Ferrie 1996 (NYSIIS) - all potentials Ferrie 1996 (SDX) - all potentials Abramitzky et al. (Name) Abramitzky et al. (NYSIIS) Abramitzky et al. (SDX) Abramitzky et al. (Name) - Corrected Abramitzky et al. (NYSIIS) - Corrected Abramitzky et al. (SDX) - Corrected Nix and Qian (Name) Nix and Qian (NYSIIS) Nix and Qian (SDX) Nix and Qian (Name) - all potentials Nix and Qian (NYSIIS) - all potentials Nix and Qian (SDX) - all potentials Machine Learning

21 Plot of T1 v. T2 Errors False Matches (T1 Errors) Nix and Qian (SDX) - all potentials Nix and Qian (NYSIIS) - all potentials Abramitzky et al. (SDX) Nix and Qian (Name) - all potentials Abramitzky et al. (NYSIIS) Abramitzky et al. (SDX) - Corrected Abramitzky et al. (NYSIIS) - Corrected Ferrie 1996 (SDX) Abramitzky et Ferrie 1996 (NYSIIS) Machine Learning Abramitzky et al. (Name) - Corrected LIFE-M Ferrie 1996 (Name) Missed Links (T2 Errors)

22 Summary of Findings Evaluate three dimensions of performance Match rates NYSIIS/Soundex may increase or decrease match rates Match disambiguation techniques dramatically increase match rates Representativeness None of the methods (including ground truth) are representative All tend to link more affluent individuals Propensity score reweighting seems to address this well Type I errors (false positives/erroneous links) Staggeringly high rates of false links Record disambiguation techniques raise these rates Erroneous matches are not representative and increase with match rates Type II errors (false negatives/missed links) Very high we are missing a lot of links!

23 Constructive Suggestions 1. Concede non representativeness of samples Make greater use of common record features such as name length or exact day of birth (when available) may provide important information about sample representativeness Use inverse propensity weights for linked samples to help balance both observed and potentially unobserved characteristics (DiNardo et al. 1996, Heckman et al. 1998) 2. Take steps to reduce T1 errors Maximizing samples sizes is different than increasing good links One (low cost) idea: Compare samples of automated links across several methods (Stata do files are available with this paper): use this to diagnose type I errors and their causes and discard problematic cases 3. Do not use NYSIIS and Soundex to improve match rates Errors arising from these name cleaning algorithms appear systematically related to a number of record characteristics, making it unclear how they should affect inferences