Data Integration Activities on the Way to the Dutch Virtual Census of 2011

Transcription

1 Data Integration Activities on the Way to the Dutch Virtual Census of 2011 Eric Schulte Nordholt Statistics Netherlands Division Social and Spatial Statistics Department Support and Development Section Research and Development MSP in Stockholm (November 2009)

2 Contents History of the Dutch Census Introduction Virtual Census of 2001 Data sources Combining sources: micro linkage Combining sources: micro integration Census tables Comparison with other years Comparison with other countries Data integration activities between the 2001 Census and the 2011 Census Conclusions 2

3 History of the Dutch Census TRADITIONAL CENSUS Ministry of Home Affairs: 1829, 1839, 1849, 1859, 1869, 1879 and 1889 Statistics Netherlands: 1899, 1909, 1920, 1930, 1947, 1960 and 1971 Unwillingness (nonresponse) and reduction expenses no more traditional CENSUSES ALTERNATIVE: VIRTUAL CENSUS 1981 and 1991: Population Register and surveys development 90 s: more registers 2001: integrated set of registers and surveys, SSD 3

4 Introduction Virtual Census of 2001 Why a Census? Statistical information for research and policy purposes What kind of information? Size of (sub)population(s) Demographic and socio-economic characteristics, at national and regional level Gentlemen s agreement Eurostat: coordinator of EU, accession and EFTA countries in the 2001 Census Round Census Table Programme, every 10 years 4

5 Data sources Registers: Population Register (PR), 16 million records demographic variables: sex, age, household status etc. Jobs file, employees, 6.5 million records, and self-employed persons, 790 thousand records dates of job, branch of economic activity Fiscal administration (FIBASE) jobs, 7.2 million records, and pensions and life insurance benefits, 2.7 million records Social Security administrations, 2 million records, auxiliary information integration process Surveys: Survey on Employment and Earnings (SEE), 3 million records, working hours, place of work Labour Force Survey (LFS), 2 years: records education, occupation, (economic) activity 5

6 Combining sources: micro linkage Linkage key: Registers Social security and Fiscal number (SoFi), unique since November 2007: Citizen Service Number Surveys Sex, date of birth, address (postal code and house number) Linkage key replaced by RIN-person Linkage strategy Optimizing number of matches Minimizing number of mismatches and missed matches 6

7 Combining sources: micro integration Collecting data from several sources more comprehensive and coherent information on aspects of a person s life Compare sources - coverage - conflicting information (reliability of sources) Integration rules - checks - adjustments - imputations Optimal use of information quality improves Example: job period vs. benefit period 7

8 Census tables (1) Preliminary work before tabulating Census Programme definitions: not always clear and unambiguous, e.g. economic activity Priority rules (characteristics of) main job (highest wage) employee or employer job or (partially) unemployed job or attending education job or retired engaged in family duties or retired age restrictions Tabulating register variables: Simply straightforward counting from SSD register data 8

9 Census tables (2) Tabulating survey (and register) variables Mass imputation? Pro s: reproducible results Con s: danger of oddities in estimates (e.g. highly educated baby) Traditional Weighting? Pro s: simple, reproducible results (if same microdata and weights) Con s: no overall numerical consistency between survey and register estimates Demand for overall numerical consistency one figure for one phenomenon idea all tables based on different sources (e.g. surveys) should be mutually consistent 9

10 Census tables (3) Ethnicity: register Education: survey 1 and survey 2 Employment status: survey 2 Estimate: T1: educ x ethnic and T2: educ x employ ethnic 1...k educ Lo... Hi employ 1...m Register Survey 1 educ x ethnic not- NL NL Total educ Lo Survey 2 educ Hi Total ethnic nonemployed not- NL NL employ x educ employed Total educ Lo Total educ Hi Total

11 Census tables (4) Repeated Weighting (RW) : tool to achieve numerical consistency (VRD-software) Basic principles of RW: estimate table on most reliable source (mostly source with most records, e.g. register) estimate tables by calibrating on common margins of the current table and tables already estimated (auxiliary information) repeatedly use of regression estimator: - initial weights (e.g. survey weights) calibrated as minimal as possible - lower variances - no excessive increase of (non-response) bias (as long as cell size>>0) each table has its own set of weights 11

12 Census tables (5) Calibrate on ethnic, then on educ x ethnic ethnic 1...k educ Lo... Hi employ 1... m Register Survey 1 2 educ x ethnic educ Lo educ Hi Total not- NL NL Total sampling units Survey ethnic nonemployed not- NL NL employ x educ educ Lo educ Hi employed Total Total Total

13 Number of inhabitants (x mln) Mean household size Comparison with other years Number of inhabitants and household size Census year 0 Number of inhabitants Mean household size 13

14 Comparison with other countries Traditional Census (complete or partial enumeration): Most countries (Estonia, Slovenia, Greece and the UK) Mixture of traditional Census and Registers: Some countries (Norway and Switzerland) Entirely or largely register-based Census: A few Nordic countries (Sweden and Finland) Virtual Census: The Netherlands 14

15 Data integration activities between the 2001 Census and the 2011 Census (1) Analyses ( GB/menu/themas/dossier s/volkstellingen/publicatie s/2005-virtual-dutchcensus-art.htm) Book and extra chapter ( GB/menu/themas/dossier s/volkstellingen/publicatie s/2001-b57-e-pub.htm) 15

16 Data integration activities between the 2001 Census and the 2011 Census (2) Integrated Public Use Microdata Series ( Lectures (Conferences, Universities, Research institutes, Statististical offices) ESTP-course Registers in Statistics (Oslo) International Statistical Seminar Eustat in Bilbao ( Digitalizing ( Recommendations and register-based statistics CENEX on ISAD ( European census regulation 16

17 Data integration activities between the 2001 Census and the 2011 Census (3) Sources (the PR as backbone of the census, changes in contents and quality of registers, remaining information from surveys) Estimation method (repeated weighting, new version of the software, fall-back option of weighting to PR, less imputations) Statistical Disclosure Control of the hypercubes (Task Force on the EU Methodology for Census Data Disclosure Control) Web service (Tabular data in SDMX format) 17

18 Conclusions (1) A Dutch Virtual Census: yes, we can! Micro integration remains important Repeated weighting was a success Advantages: Relatively cheap (small cost per inhabitant) Quick (short production time) Disadvantages: Dependent on register holders (statistics is not their priority), timeliness of registers, concepts and population of registers may differ from what is needed (keep good relations with the register holders!) Publication of small subpopulations sometimes difficult or even impossible because of limited information 18

19 Conclusions (2) Other aspects: Less attention for the results of a virtual census than for a traditional one Difficult to keep knowledge and software up-todate (Census is running every ten years) Enormous international interest in virtual censuses A lot of interesting census work in the coming years! 19

20 Time for questions and discussion 20