Comparing the Quality of 2010 Census Proxy Responses with Administrative Records Mary H. Mulry & Andrew Keller U.S. Census Bureau 2015 International Total Survey Error Conference September 22, 2015 Any views expressed are those of the authors and not necessarily those of the U.S. Census Bureau
2020 Census research Goals: reduce cost & improve data quality Designed to support fundamental changes in design, implementation, & management Includes exploring potential uses of third-party & federal administrative records (AR) Nonresponse Followup (NRFU) of addresses with no self-response via mail or Internet This paper focuses on AR use vs. proxy interviews. 2
Our research questions Are proxy responses more or less accurate than admin records? What variables correlate with variation in the quality of proxy responses & admin records? For individual records For records grouped by housing unit (HU) 3
Examine quality of 2 lists of population Population: people whose Census Day residence is in a HU enumerated in 2010 Census NRFU by proxy respondents Proxy after 6 attempts for HH member response List 1: census enumerations List 2: administrative records For context, will also examine lists for people in NRFU HUs enumerated by HH members 4
Data for our study Census proxy responses: 2010 NRFU data Admin records: IRS 1040s filed in all of 2010 Medicare records from all of 2010 To compare, construct gold standard using 2010 Census Coverage Measurement Program (CCM) residence codes assigned for sample of block clusters Block clusters are adjacent blocks grouped to have at least 30 HUs 5
CCM residence codes qualify for gold standard CCM sample data receives intensive evaluation for use in coverage error estimation Overlapping samples of enumerations (E) and independent list (P) in sample of blocks clusters Case-by-case electronic & clerical matching Household-based matching algorithm Nationwide electronic search of census to find duplicate enumerations Field followup to resolve ambiguities 6
Merge E & P sample records to create combined CCM Use CCM residence codes to classify combined CCM records into 3 categories: Residents of CCM blocks on Census Day Not residents of CCM blocks on Census Day Unresolved residence on Census Day E Sample (census) whole person imputations and records without a name & 2 characteristics did not have sufficient info to receive a CCM residence code 7
Match to CCM to assign residence codes to Ad Rec Ad Rec list Combined CCM list Matches Non matches CD Residents Unresolved residence Not CD residents CD Residents Admin records matched to CCM records at same address using Protected Identification Keys (PIKs) 8
AR status of HUs in both E & P samples & occupied in census by NRFU respondent (unweighted) Proxy HH member Has AR records 5,310 51% 16,876 61% No AR records 5,106 49% 10,647 39% total 10,416 27,523 9
Number of records in NRFU HUs in study by type of respondent (unweighted) Admin records NRFU proxy HU 12,880 20% 11,766 19% HH member 50,876 80% 51,485 81% total 63,756 63,251 10
CCM residence status of records (unweighted) Residence status of record Proxy HUs HH member HUs NRFU N=11,766 AR N=12,880 NRFU N=51,485 AR N=50,876 CD Resident 56% 48% 87% 71% Not CD resident 4% 4% 3% 2% Unresolved CD residence 16% 4% 6% 3% Insufficient info for CCM 2% 2% Whole person imputation 21% 1% At another census address 17% 10% Not linked to census record 27% 13% 100% 100% 100% 100% 11
CCM residence status of records (unweighted) Residence status of record Proxy HUs HH member HUs NRFU N=11,766 AR N=12,880 NRFU N=51,485 AR N=50,876 CD Resident 56% 48% 87% 71% Not CD resident 4% 4% 3% 2% Unresolved CD residence 16% 4% 6% 3% Insufficient info for CCM 2% 2% Whole person imputation 21% 1% At another census address 17% 10% Not linked to census record 27% 13% 100% 100% 100% 100% 12
CCM residence status of records (unweighted) Residence status of record Proxy HUs HH member HUs NRFU N=11,766 AR N=12,880 NRFU N=51,485 AR N=50,876 CD Resident 56% 48% 87% 71% Not CD resident 4% 4% 3% 2% Unresolved CD residence 16% 4% 6% 3% Insufficient info for CCM 2% 2% Whole person imputation 21% 1% At another census address 17% 10% Not linked to census record 27% 13% 100% 100% 100% 100% 13
CCM Correct CD Residence rate for NRFU records varies by PIK status (unweighted) 100% 90% 85.7% 92.7% 80% 73.7% 70% 60% 56.7% 50% 40% 30% 20% 10% 0% NRFU Proxy w/ PIK assigned NRFU Proxy w/o PIK assigned NRFU HH Member w/ PIK assigned NRFU HH Member w/o PIK assigned 14
CCM evaluation of admin records grouped by HU for proxy respondents (unweighted) HU status Count % ARs perfect 1,722 32% ARs have error or unresolved 3,180 60% ARs have 1 or more omissions 408 8% Total 5,310 100% 15
Summary: Are proxy responses more or less accurate than admin records? 51% of census-occupied proxy HUs have admin records for IRS 1040s & Medicare from all of 2010 49% do not have records in these files Proxy enumerations that can be assigned PIKs tend to be at correct CD residence Many proxies are good. Those that can give us name & enough info to assign a PIK tend to provide correct residence Admin records for 32% of proxy HUs have perfect composition, based on CCM Duplication may be a concern when using admin records to enumerate whole HHs 16
Contact mary.h.mulry@census.gov andrew.d.keller@census.gov