UPDATED INFORMATION ON DISSEMINATION CENSUS HUB AND OTHER DISSEMINATION ROUTES. Working Group "Population and Housing Censuses"

Transcription

1 EUROPEAN COMMISSION EUROSTAT Directorate F: Social statistics Unit F-2: Population Luxembourg, Doc. ESTAT/F2/CENS/2014/3 Working Group "Population and Housing Censuses" Luxembourg, September 2014 BECH building, room QUETELET UPDATED INFORMATION ON DISSEMINATION CENSUS HUB AND OTHER DISSEMINATION ROUTES Item 3 of the Agenda

2 1. Introduction This document provides a summary of the progress made by Eurostat on the topics presented under items 5 ("The Graphical User Interface of the Census Hub") and 6 ("Other dissemination routes for the 2011 Census data") of the previous meeting of this Working Group (4-5 June 2013). The referenced documents are available in the "Census" CIRCABC section on Eurostat website, available at this link. The following topics are covered: an update of the IT developments concerning the Census Hub software, meaning both: a) the "central node" of Census Hub, i.e. the software module with which users will be able to search and retrieve their data of interest from the system b) and the Census Hub as an overall IT system, the first example of such a "data sharing" system developed in social statistics in the ESS This update is provided in section 2 of this document, together with a preliminary and highlevel assessment of the performances of the system as measured so far; the "Census 2011" new domain, which will be nested into Eurostat's online database see section 3.1 of this document; an updated status of the "statistical dashboards" on the 2011 census, as introduced to the Working Group in June 2013, in the document "Other dissemination routes for the 2011 census data" quoted above see section 3.2 of this document; an updated status of the interactive thematic maps on the 2011 census, ibidem see section 3.3 of this document. 2. Update status on the Census Hub (as IT project) The Census Hub (available at represents the first successful example for the ESS as a whole of distributed database built on the concept of "data sharing". From the technological viewpoint the Census Hub is fully based on SDMX, and represents a solution which, at least theoretically, could easily be applied to other statistical domains. Developing and setting up the IT infrastructure to ensure a reliable link between the Census Hub "central application" and the "nodes" in the NSIs was a major collective achievement of the ESS as a whole. In the past few months Eurostat has monitored closely the degree of responsiveness of the system, i.e. the processing speed of the central application in dispatching the queries and elapsed time for retrieving/displaying the data. After an initial phase characterised by some connection problems with some countries, this monitoring activity has revealed that, currently, the system is performing 2

3 well both in terms of the behaviour of the central application and, most importantly, in terms of the connection speed with Member States. As of today, there are no severe pending issues related to the IT part of the Census Hub project, so that the Hub itself can now be seen not only as an innovative technology, but also as a reliable and robust implementation of SDMX. In the picture below: the Graphic User Interface of the Census Hub, as it appears to users. 3. Eurostat dissemination routes for the 2011 census Although the Census Hub is the main dissemination tool for the 2011 census data, the data will also be disseminated in other ways. This part of the document covers these other dissemination routes. 3.1 Dissemination via Eurostat online database As it was discussed at the last meeting of the Working Group, Eurostat will "supplement" the Census Hub main dissemination tool for the EU 2011 census data with a set of aggregated summary tables, which will be disseminated using Eurostat online database. Annex 1 present the list of tables that was implemented for this new "2011 census domain", following the discussion at Working Group meeting in June Before being published, these tables will be checked using the EDIT software, a tool available at Eurostat for the "quality screening" of incoming data ( This EDIT software (which will receive data directly from the Census Hub) allows further levels of quality checks, in addition to the procedure discussed in the previous item of the Agenda. In particular it allows the verification of all "additivity rules" implied by the hierarchies present in each dimension of the tables ("total" equalling the sum of "males" and "females", for instance). This 3

4 additional quality check can prove particularly useful for the "geographic dimension", for which several hundred nested constraints apply at EU level. The IT procedure that fetches data from the Hub, processes them and then transfers them to the Eurostat online database is, roughly speaking, quasi-automatic (in the sense that manual interventions are needed only in case of failures in the EDIT quality screening phase). This procedure would allow for a "near to real-time" update of Eurostat database, provided that both possible revisions as foreseen by the EU census legislation and other data updates are communicated to Eurostat by NSIs in due time. 3.2 Statistical dashboards Eurostat has already published several "statistical dashboards" (web-based widgets for the interactive display of data through graphs and tables) on some "dedicated sections" of its website. The most notable examples include the domains price statistics, waste and environmental statistics, and regional statistics. Eurostat is currently developing a set of dashboards for the 2011 census using the NComVA tool described below. These draft dashboards are currently stored in the so-called "test environment" of Eurostat's website (an environment which is unfortunately not accessible outside the European Commission). The dashboards have been produced using a software called NComVA ( based on a prototype developed at the University of Stockholm. This is the same software tool that the Italian NSI, ISTAT, used to develop the portal "Noi Italia" ( Despite the various limitations of this tool, Eurostat experience suggests that it is ideally suited to the visualisation of one of the key features of census data: their level of geographical detail. In particular, the NComVA tool fits perfectly for the display NUTS2 data for the EU. NComVA offers the following: a stylized map of the EU at NUTS2 level, which is dynamically linked to the statistical plot (or plots) illustrating the data a limited, but meaningful, set of statistical plots, including: o o o bivariate bubble charts (i.e. x-y scatter-plots where the symbol size depends on a third variable z) dot plots bar charts the possibility to "feed" the tool either with plain CSV files containing the data of interest, or with a dynamic link to Eurostat's database the possibility to crate "views" (i.e. particular configurations that would highlight data in a way that is meaningful to certain users, for example "focus on country XYZ") 4

5 However, NComVA is not among the cutting-edge technologies available nowadays to produce statistical dashboards. It is evident that, among other factors, the "Big Data" boom has boosted considerably the many recent technological development in this field. In addition, there are some limitations to the flexibility of NComVA. For instance, certain statistical plots that would be appropriate for census data (such as age pyramids for example) cannot be incorporated easily into the NComVA tool. As often happens in IT, choosing one software instead of another results in a trade-off. In this respect, at least according to Eurostat's experience, NComVA is neither a bad tool nor the optimal one. Clearly, the finalisation of the 2011 census statistical dashboards requires, as a prerequisite, the prior completion of the "quality control" stage, as discussed in the previous item of the Agenda. For illustrative purposes only, the following figure shows some snapshots of the statistical dashboards that will be made available to users of the 2011 census data. 5

6 3.3 Interactive thematic maps In addition to the statistical dashboards, Eurostat is developing a set of interactive thematic maps on the 2011 census. These maps are being developed by the Eurostat census team in close collaboration with the GISCO team of Eurostat Unit E4 (team responsible for GIS and cartography at Eurostat). The technology used is based on ESRI ArcGis. Eurostat has a long experience in producing interactive thematic maps. For example, all recent editions of the "Eurostat Regional Yearbook" have been accompanied by a dedicated set of online maps, see As discussed in the document "Other dissemination routes for the 2011 census data" presented at the previous meeting of this Working Group, some additional features of Eurostat thematic maps were developed specifically for the 2011 census. In particular: The "background map" was made visually more attractive by adding some additional elements, such as reliefs, main urban agglomerations, roads, rivers, lakes, etc. Choropleth maps were drawn with semi-transparent colours, rather than with fully opaque ones. This allows the features on the background map to remain visible, and makes the overall map look more attractive. In particular, the user-friendliness of the map improves greatly at higher levels of zoom (at NUTS3 level for instance) since the user can easily recognise familiar geographic areas and features. The geographic detail of data displayed on the map (national, NUTS1, NUTS2, etc.) depends on the zoom level selected by the user. The narrower the zoom level, the higher is the level of geographic detail displayed on the map (see the figure below). The labelling of the different regions/municipalities was considerably improved, again with the aim of achieving better user-friendliness. As with the statistical dashboards discussed in the previous section, the finalisation of the interactive thematic maps requires that the "quality control" stage is duly completed. Annex 2 presents, for information and discussion, the set of interactive maps that are currently under development (subject to data availability and sufficient data quality). The following figures shows some examples of the 2011 census interactive thematic maps. 6

7 An illustration of the interactive thematic maps currently being developed by Eurostat on the 2011 census. As the figure shows, the level of geographic detail of the map (NUTS1 in the top panel, NUTS2 in the middle one and NUTS3 in the bottom one) depends on the zoom level chosen by the user. In addition to this, the thematic map was made visually more appealing by adding a series of graphic elements (reliefs, rivers, roads, etc.) that make the map easier to use and interpret. 7

8 4. Questions to the Working Group The Working Group is invited: to comment on the ongoing work to disseminate the 2011 census data via statistical dashboards, thematic maps and the online database to share any relevant national experience and findings 8

9 ANNEX 1 Structure of the "2011 census" domain in Eurosat online database Table number 1 cens_11ag_r3 Population by single year of age and NUTS 3 region Dimensions: GEO.M. (1884), SEX. (3), AGE.H. (128) HC55 This HC does not have PMDs Table number 2 cens_11ms_r3 Population by marital status and NUTS 3 region Dimensions: GEO.M. (1884), SEX. (3), AGE.M. (28), LMS. (9) HC46 HC46_3 Table number 3 cens_11fs_r3 Population by family status and NUTS 3 region Dimensions: GEO.M. (1884), SEX. (3), AGE.M. (28), FST.H. (9) HC51 This HC does not have PMDs Table number 4 cens_11aed_r2 Population by current activity status, educational attainment and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.M. (28), CAS.L. (6), EDU. (10) HC13 HC13_1 9

10 Table number 5 cens_11ao_r2 Population by current activity status, occupation and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.M. (28), CAS.L. (6), OCC. (13) HC10 HC10_1 Table number 6 cens_11an_r2 Population by current activity status, economic activity (NACE Rev. 2) and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.M. (28), CAS.L. (6), IND.L. (13) HC10 HC10_3 Table number 7 cens_11empo_r2 Population by status in employment, occupation and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.M. (28), SIE. (7), OCC. (13) HC11 HC11_1 Table number 8 cens_11empn_r2 Population by status in employment, economic activity (NACE Rev. 2) and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.M. (28), SIE. (7), IND.L. (13) HC11 HC11_3 Table number 9 cens_11ctz_n Population by country of citizenship at national level Dimensions: GEO.N. (32), SEX. (3), AGE.M. (28), COC.H. (235) HC27 HC27_1 10

11 Table number 10 cens_11ctzo_r2 Population by group of citizenship, occupation and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.L (7) COC.L. (7), OCC. (13) HC16 Not available Table number 11 cens_11cob_n Population by country of birth at national level Dimensions: GEO.N. (32), SEX. (3), AGE.M. (28), POB.H. (234) HC26 HC26_1 Table number 12 cens_11cobe_r2 Population by group of country of birth, educational attainment and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.L (7), POB.L. (7), EDU. (10) HC23 HC23_2 Table number 13 cens_11coba_r2 Population by group of country of birth, current activity status and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.L (7), POB.L. (7), CAS.L. (6) HC15 HC15_1 Table number 14 cens_11cobo_r2 Population by group of country of birth, occupation and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.L (7), POB.L. (7), OCC. (13) HC15 Not available 11

12 Table number 15 cens_11arco_r2 Population by period of arrival in the country, country of birth and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), YAE.L. (11), POB.L. (7) HC25 HC25_1 Table number 16 cens_11fts_r3 Families by type, size and NUTS 3 region Families Dimensions: GEO.M. (1884), TFN.H. (19), SFN.H. (14) HC52 This HC does not have PMDs Table number 17 cens_11htts_r2 Private households by type, tenure status and NUTS 2 region Private households Dimensions: GEO.L. (489), TPH.H. (24), TSH. (5) HC05 This HC does not have PMDs Table number 18 cens_11hou_r2 Population by housing arrangement and NUTS 2 region Dimensions: GEO.L. (489), SEX. (3), AGE.L. (7), HAR.H. (8) HC40 HC40_1 Table number 19 cens_11dwob_r3 Conventional dwellings by occupancy status, type of building and NUTS 3 region Conventional dwellings Dimensions: GEO.M. (1884), OCS. (4), TOB. (7) HC53 This HC does not have PMDs 12

13 Annex 2 Draft proposal of interactive thematic maps to be developed on the 2011 census data Table title Geographic level Demographic charachteristics Share of persons aged less than 15 NUTS3 HC46_1 Share of persons aged 65 or more NUTS3 HC46_1 Share of persons aged 85 or more NUTS3 HC46_1 Young-age dependency ratio NUTS3 HC46_1 Old-age dependency ratio NUTS3 HC46_1 Families and households Median family size NUTS3 HC52 Share of families without resident children NUTS3 HC52 Share of of lone parent families (as% of all families with children) NUTS3 HC52 Share of one-person households NUTS3 HC49 Share of families with 6 or more persons NUTS3 HC52 Source (HC and PMD) Elderly people Share of persons aged 85 or more living alone NUTS3 HC48 Share of persons aged 85 or more living in an institutional household NUTS3 HC48 Share of women among persons aged 85 or more NUTS3 HC46_1 Share of persons aged not economically active NUTS2 HC01_4 Education Share of students in the population aged NUTS2 HC10_1 Share of persons aged with tertiary education NUTS2 HC02_1 Share of persons aged with tertiary education NUTS2 HC02_1 Employment Employment rate NUTS2 HC01_4 Female employment rate NUTS2 HC01_4 Gender gap in the employment rate NUTS2 HC01_4 Activity rate NUTS2 HC01_4 Gender gap in the activity rate NUTS2 HC01_4 Older persons employment rate, persons aged NUTS2 HC01_4 Employment by economic sector Share of persons employed in agriculture NUTS2 HC10_3 Share of persons employed in industry NUTS2 HC10_3 Share of persons employed in construction NUTS2 HC10_3 Share of persons employed in services NUTS2 HC10_3 Median age of persons employed in agriculture NUTS2 HC10_3 Number of persons working in the education sector (per 100 students) NUTS2 HC10_3, HC10_1 Employment concentration (share of persons working in the five NUTS2 HC10_3 13

14 largest economic sectors) Number of persons working in the public administration, defence and compulsory social security (per 1000 inhabitants) NUTS2 HC10_3 Employment by occupation Median age of managers NUTS2 HC43_2 Share of women among managers NUTS2 HC10_1 Share of employed persons aged working as professionals, technicians and associate professionals NUTS2 HC10_1 Unemployment Unemployment rate NUTS2 HC01_4 Youth unemployment rate Median age of persons employed as managers NUTS2 HC10_1 Share of unemployed aged with tertiary education NUTS2 HC13_1 Share of lone parents who are unemployed NUTS2 HC06_2 Foreign population Share of foreigners NUTS3 HC46_2 Median age of foreigners NUTS3 HC46_2 Share of foreigners who are non-eu citizens NUTS3 HC46_2 Share of foreigners who were born abroad NUTS2 HC25_5 Share of foreigners aged with tertiary education NUTS2 HC23_3 Gap in the unemployment rate between foreigners and nationals NUTS2 HC16_1 Foreign-born population Share of foreign-born NUTS3 HC46_1 Median age of foreign-born NUTS3 HC46_1 Share of foreign-born who were born outside the EU NUTS3 HC46_1 Share of foreign-born aged with tertiary education NUTS2 HC23_2 Share of foreign-born who entered the country after 2000 NUTS2 HC25_1 Share of foreign-born who are nationals NUTS2 HC25_5 Gap in the unemployment rate between foreign- and nativeborn NUTS2 HC15_1 Housing Share of unoccupied dwellings NUTS3 HC53 Share of dwellings built after 1980 NUTS3 HC53 Share of owner-occupied dwellings NUTS2 HC41_1 Share of persons not living in a conventional dwelling NUTS2 HC38_1 Share of persons having changed residence in the previous year NUTS3 HC46_6 Share of dwellings occupied by 6 or more persons NUTS3 HC54_2 14