Sierra Leone 2015 Population and Housing Census POST ENUMERATION SURVEY RESULTS AND METHODOLOGY

Transcription

1 Sierra Leone 2015 Population and Housing Census POST ENUMERATION SURVEY RESULTS AND METHODOLOGY STATISTICS SIERRA LEONE (SSL) JUNE 2017

2

3 POST ENUMERATION SURVEY RESULTS AND METHODOLOGY BY MOHAMED LAGHDAF CHEIKH MALAININE SIERRA LEONE 2015 POPULATION AND HOUSING CENSUS

4 We wish to thank the Government of Sierra Leone for the financial and oversight support to the project. Special thanks goes to our development partners DFID, Irish Aid, UNFPA and UNDP for providing the funds, technical support and guidance in the implementation of the Census project. DISCLAIMER Statistics Sierra Leone cannot be held responsible for errors, or any consequences arising from the use of information contained in this report. All rights reserved. This document may be freely quoted or reproduced, in part or in full, provided that the source is acknowledged. iv

5 NOTE TO THE READER NOTE TO THE READER The Sierra Leone 2015 Population and Housing Census (PHC) is the fifth modern census conducted in the country. It was implemented by Statistics Sierra Leone (SSL), supported by a UNFPA technical team, a Census Technical Committee (CTC), a United Nations Technical Committee on Census (UNTCC) and a Census Steering Committee. The Government of Sierra Leone, UNFPA, DFID, Irish Aid, and UNDP provided financial support to the 2015 PHC. The 2015 PHC started by the launching of the cartographic exercise in May The pilot census was conducted in May 2014 and two data users conferences were held in March 2014 and July 2015 respectively. The main census enumeration initially planned for December 2014 was first postponed to April then to December 2015, due to the unprecedented Ebola outbreak in the country. The main census enumeration was conducted successfully from 5 to 18 December The data processing was competed in October The census provisional results were published in March 2016 and the final results in December The census process was monitored by teams of independent and qualified UN monitors, national monitors and international monitors. The monitors concluded that the Sierra Leone 2015 Census was conducted according to international norms and standards, and rated it as to be generally good and satisfactory. v

6 TABLE OF CONTENTS n List of tables...vii List of abbreviations... viii EXECUTIVE SUMMARY...1 CHAPTER 1: INTRODUCTION Background PES Scope and Objectives Organization...2 CHAPTER 2: METHODOLOGY PES concept and procedures PES sample design Sampling frame Stratification Sample size Field operations Matching Reconciliation visits Data processing Main challenges...8 CHAPTER 3: COVERAGE EVALUATION Population estimates Coverage rates Coverage errors CHAPTER 4: CONTENT EVALUATION Content evaluation indicators Content analysis results Content Analysis for Relationship to Head of Household Content Analysis for Sex Content Analysis for Age Content Analysis for Marital Status Content Analysis for Literacy...18 CHAPTER 5: CONCLUSION AND RECOMMENDATIONS Conclusion Recommendations...20 BIBLIOGRAPHY...21 ANNEXES...22 Annex 1: PES Concepts and definitions...22 Annex 2: Dual System Estimation...23 Annex 3: Matching Guidelines...26 vi

7 LIST OF TABLES Table 2.1: Distribution of EAs by domain and by type of residence...4 Table 2.2: Distribution of households by domain and by type of residence...4 Table 2.3: Sample distribution (EAs)...5 Table 3.1: Population estimates by Sex,Age, Area of Residence and Region...9 Table 3.2: Coverage Error Rates by Sex, Age, Area of Residence and Region...11 Table 3.3: Coverage Errors by Sex, Age, Area of Residence and Region...12 Table 4.1: Interpretation of Indices for Content Errors...14 Table 4.2: Net Difference Rate and Indices of Inconsistency for Relationship to HHH...15 Table 4.3: Net Difference Rate and Indices of Inconsistency for Sex...16 Table 4.4: Net Difference Rate and Indices of Inconsistency for Age...17 Table 4.5: Net Difference Rate and Indices of Inconsistency for Marital Status...18 Table 4.6: Net Difference Rate and Indices of Inconsistency for Literacy...19 vii

8 LIST OF ABBREVIATIONS AII DSE CI CTC DFID EA GDR GIS GoSL HHH Aggregate Index of Inconsistency Dual System Estimation Confidence Interval Census Technical Committee Department for International Development - UK Enumeration Area Gross Difference Rate Geographic Information System Government of Sierra Leone Head of Household MoFED Ministry of Finance and Economic Development NDR PES PHC PPS PSU RA RV SSL UN Net Difference Rate Post Enumeration Survey Population and Housing Census Probability Proportional to Size Primary Sampling Unit Rate of Agreement Reconciliation Visits Statistics Sierra Leone United Nations UNDP United Nations Development Program UNFPA United Nations Population Fund UNTCC United Nations Technical Committee on Census viii

9 EXECUTIVE SUMMARY The Post Enumeration Survey (PES) for the Sierra Leone 2015 Census is the first fully implemented and analysed PES in the country. It was implemented by Statistics Sierra Leone (SSL) with UNFPA technical support, and the generous funding support from the Government of Sierra Leone, UNFPA, DFID and Irish Aid. PES was monitored by a team of international and independent monitors. The monitors concluded that the Sierra Leone 2015 PES was well organized, and rated it as a good exercise. The 2015 PES main objectives were to determine the level of census coverage and evaluate its data quality for key socio-demographic characteristics. The PES was conducted on a sample of 240 Enumeration Area (EA) out of a sampling frame of 12,856 EA, which is considered as a large sample size. The PES enumeration was conducted from 27 February to 4 March 2016, that is within the UN recommended period of less than three months from the main census enumeration. All PES planned activities were fully implemented and in line with international norms and standards, including trainings, initial matching, reconciliation visits, final matching, data processing and data analysis. The PES results reveal that the Sierra Leone 2015 Census has high coverage and is of a good data quality. The Census coverage rate is estimated at 97.9 percent, which is one of the highest coverage rates in the sub-saharan African region. The coverage rate is higher by almost two points in rural area (98.6%) than in urban area (96.5%). By region, the highest coverage rate was registered in Southern region (99.3%), followed by Northern region (98.1%), Eastern region (97.8%) and the lowest in the Western area (95.8%). By characteristics, the coverage rate is almost the same for males (97.74%) and females (97.98%). However, results suggest that the coverage is lower for youth population (97.8% for less than 15 years), than for elder population (98.4% for the age group 65 years and over). The content analysis was done for five selected socio-demographic characteristics, namely: relationship to head of household, sex, age, marital status and literacy. For the five selected demographic variables, findings show high level of agreement and low or moderate level of inconsistency. The agreement rate varies between 86.3 percent for the variable literacy, followed by age (91.4%), marital status (93.9%), relationship to head of household (94.9%), and is as high as 99.6% for sex. In addition, the results suggest low level of Aggregate Level of Inconsistency (AII < 20) for four out of the five selected variables; and moderate inconsistency for the fifth variable, -literacy (AII = 21.80). Finally, PES identified the key challenges encountered during the census implementation, and proposed adequate recommendations. It is recommended for SSL and UNFPA to take into consideration the PES findings and recommendations for better data quality of futures PES, populations censuses and large household surveys in Sierra Leone. 1

10 CHAPTER INTRODUCTION1 1.1 Background The Sierra Leone 2015 census Post Enumeration Survey (PES) is the first fully implemented and analyzed PES in the country. It was conducted by Statistics Sierra Leone, with the technical support from UNFPA. The UNFPA technical team role was to ensure that the Sierra Leone 2015 PES is independent, transparent and conducted according to international norms and standards. In addition, the PES process was monitored by a team of independent international monitors. The international monitors rated the PES as well organized and a good exercise. 1.2 PES Scope and Objectives 1.3 Organization In line with the UN operational guidelines for PES, the Sierra Leone 2015 PES was carried into eight main phases: 1. drafting of PES instruments (questionnaires, manuals ), 2. pilot PES, 3. enumeration, 4. initial matching, 5. reconciliation visits, 6. final matching, 7. data processing, and 8. data analysis and report writing. The PES for the Sierra Leone 2015 Census covered the entire country with a sample of 240 EAs selected from the four regions, each taken as a stratum for sampling purposes. PES concerned only household population. The 2015 PES was carried out to determine the level of census coverage rate and evaluate its data quality (content errors). The PES main objectives were: 1. to measure under-coverage and over-coverage of persons, 2. to measure levels of agreement for responses to questions on selected characteristics: relationship to head of household, sex, age, place of birth, marital status and literacy, 3. to evaluate the comprehensiveness and definition of area primary sampling units (EA) 4. to learn from procedural and conceptual limitations in the census which need improvement in future, and 5. to provide a statistical basis for adjustment of census results, if required. 2

11 CHAPTER METHODOLOGY2 This chapter highlights the key elements of the methodology applied to implement the Sierra Leone 2015 PES. After a first introductory section about the PES concept and procedures, the chapter highlights the PES sampling plan, field operations, matching and data processing. The last section presents the main challenges encountered during the PES implementation. 2.1 PES concept and procedures There are two types of errors in a population census: the coverage error and the content error. The coverage error refers to either an under-count or over-count of units owing to omissions of persons/ housing units or duplication/erroneous inclusion, respectively. Content error pertains to the error in the characteristics that are reported for the persons or housing units that are enumerated. Both types of error can affect the distribution of the population with respect to their characteristics. There are three types of coverage error: (i) omissions, (ii) duplications and (iii) erroneous inclusions (UNSD, 2010). The Sierra Leone 2015 PES used the Dual System Estimation (DSE) to evaluate the census coverage rates. The DSE methodology assumes that the PES is independent from the Census and the population remains unchanged during the period of the study. The DSE is explained in annex 2. PES Procedures There are three PES procedures to evaluate the census coverage. The procedures are defined by the UN manual on the PES operational guidelines as follows: Procedure A: This procedure reconstructs the households as they existed at the time of the census. Procedure B: It identifies all current residents living or staying in the sample household at the time of the PES. Procedure C: It is a combination of procedures A and B. In this procedure, the questionnaire is designed to obtain a listing of all persons currently living at the sample address or location and all possible locations (as in procedure B) of the members of household on a census day including a listing of persons who belonged to the sample address on census day, but were not resident at the time of the PES. In the case of Sierra Leone, as in most countries who conducted PES, the procedure C has been applied. The procedure C is relatively more expensive than the two other procedures. However, it has the advantage of reducing matching difficulties and improving the estimation of movers. 2.2 Sample design The aim of the PES sample design is to produce representative results for the country as a whole, for the urban and rural areas and for each of the country s four regions Sampling frame Administratively, Sierra Leone is divided into 4 regions. Each region is subdivided into districts, district into chiefdoms, and chiefdom into sections. In total, there are 14 districts, 149 chiefdoms, 12 census wards and 1,342 sections. In addition to these administrative units, during the 2015 Sierra Leone Census mapping exercise, each section was subdivided into convenient area units called Enumeration Area (EA). The list of EAs contains number of households and urban rural specification for every EA. The census EAs were used as primary sampling unit (PSU), also called cluster. The sample of the PES was selected from the frame of PSUs. The frame includes only household population. Table 2.1 below gives the distribution of EAs and their average size by province and by urban rural. 3

12 Table 2.1: Distribution of EAs by domain and by type of residence Domain Number of EAs Average EA Size (number of households) Urban Rural Total Urban Rural Total Eastern 917 1,880 2, Northern 1,144 3,386 4, Southern 464 2,226 2, Western 2, , Sierra Leone 5,299 7,557 12, Source: Mapping and GIS unit, SSL. In total, there are 12,856 EAs in Sierra Leone, 5,299 EAs in urban areas and 7,557 EAs in rural areas. On average, a census EA counts 91 households in the urban areas and 102 households in the rural areas, with an overall average of 97 households per EA. Table 2.2 below gives the distribution of households by domain and by urban rural residence. Table 2.2: Distribution of households by domain and by type of residence Domain Households Proportion Urban Rural Total Urban Rural Total Eastern 88, , , Northern 97, , , Southern 44, , , Western 249,228 6, , Sierra Leone 479, ,356 1,248, Stratification To improve the efficiency of the PES sample design, the sampling frame was categorized into strata. Stratification is to facilitate the calculation of estimates for respective domains. The first level of stratification therefore corresponds to the geographic domains of estimation, namely: region and urban rural residence. The advantages of stratification as it relates to PES are: (i) the entire population of EAs is divided internally homogeneous but externally heterogeneous subpopulations, for example, rural and urban, (ii) within each stratum, a separate sample is selected from all sampling units in the stratum, and (iii) from the sample obtained in each stratum estimates can be obtained. In total, 8 sampling strata have been constructed. 4

13 Samples were selected independently in each stratum, by using a probability proportional to size selection. An implicit stratification and proportional allocation has been achieved at each of the lower administrative levels, by sorting the EAs within each sampling stratum, according to lower administrative units Sample size The sample size was determined using the following formula, and considering operational and budgetary constraints: Where: n = required minimum sample size 4( p)( 1 p) n = 4 = factor corresponding to 95% level of confidence ( d ) 2 p = estimate prevalence of the outcome being measured d = minimum desired precision or maximum tolerable error Table 2.3: Sample distribution (EAs) Domain Number of EAs Urban Rural Total Eastern 917 1,880 2,797 Northern 1,144 3,386 4,530 Southern 464 2,226 2,690 The margin of error (d) and confidence level was fixed at 2.81 percent and 95 percent respectively. In total, 240 EAs were selected for the PES; with Probability Proportional to Size (PPS) selection. The measures of size used during the PPS selection were the number of households per EA as obtained in the mapping exercise. Western 2, ,839 Sierra Leone 5,299 7,557 12,856 Output Values Estimates Value Estimate, p 0.95 CI at (95%) Upper Lower Sample Size (EAs) 240 Ave. Standard Error Source: Computed. 5

14 2.2.4 Weighting The base weight of the sampling units (EA) is equal to the inverse of the sampling rate. It varies from stratum to stratum. As EAs are selected within each stratum, the basic weight Wh for the sample persons in stratum h is calculated as follows: W h = N h n h Where, Nh = total number of EAs in the frame for stratum h, and nh = number of sample EAs selected in stratum h. The weight for each person within an EA is equal to the EA sampling weight. 2.3 PES Instruments The PES instruments were drafted by the PES core team with technical support from UNFPA. The instruments included the PES questionnaire, technical sheets, matching guidelines and manuals of instructions for various PES staff (enumerators, supervisors, matchers and data entry operators). The PES instruments were drafted in line with UN recommendations and approved by the Census Technical Committee (CTC). Updated maps were provided by SSL mapping unit and used for the PES data collection. The PES instruments were tested during the pilot PES and the necessary improvements were made before the actual PES enumeration. 2.4 Field operations The field operations include pilot PES and enumeration. The pilot PES was conducted from 9 to 13 October 2015 in two EAs, one in urban area and one in rural area. Ten staff from the PES core team took part in the exercise, 8 as enumerators and 2 supervisors. Prior to the exercise, the PES field team benefited from a five-day training. The pilot PES aim was to test the proposed PES methodology. It also provided the PES team with an understanding of the questions, concepts and definitions to be used for the main PES exercise. The PES recommendations were used to improve and adapt the methodology to the Sierra Leone context. The main PES enumeration was conducted from 27 February to 4 March 2016, while the main census enumeration was conducted form 5 to 18 December Hence, the PES enumeration was conducted during of period of less than three months from the main census enumeration, as recommended by UN. The PES was conducted on a sample of 240 Enumeration Areas (EA). The PES filed staff were selected from the best candidates. Necessary arrangements were made to ensure independence between PES and Census. For instance, all PES staff who took part in the main census were deployed in a district different, and as far as possible, from the one where they conducted the main census. PES staff benefited from adequate trainings. The training of trainers was conducted from 9 to 10 February and training of field staff from 19 to 24 February Matching Matching is one of the crucial PES activities. It aims at comparing PES data with the one collected in the corresponding Census EAs. Matching was conducted into two phases: (i) initial matching, where information from the PES and census questionnaires was used to assign the initial match status and (ii) final matching, where the final match status was assigned based on reconciliation visits outcomes. 6

15 The matching exercise was conducted from 16 May to 21 September All the 240 PES EAs were covered. There were 16 matchers initially selected for the exercise, but later increased to 24 matchers. No major challenges were encountered during the matching exercise. Matching was first done for households and then for individuals. There were 2 training workshops conducted for the matching staff. The first training was on matching of households after which the staff matched PES and census households. Upon completion of household matching, a second training was conducted on matching individuals in the households. From the matching exercise, 141 EA were identified to be requiring reconciliation visits. The matching methodology used for the Sierra Leone 2015 PES is explained in the guidelines presented in annex Reconciliation visits The reconciliation visits (RV) consist of field follow-up visits to some households in the PES sampled EAs. They aim at collecting relevant information to determine the final match status of unresolved cases identified during initial matching. The RV specific objectives were: (i) to resolve the final match status for possible match cases, (ii) to determine whether households and/or persons enumerated in the census but not in the PES were correctly or erroneously enumerated in the census, (iii) to determine whether households and/or persons enumerated in the PES but not in the census were correctly or erroneously enumerated in the PES, (iv) to clarify doubtful cases or cases with insufficient or unclear information, and (v) to investigate EAs where boundary or enumeration quality problems are suspected. Twenty-four enumerators and 6 supervisors were trained on the RV from 27 to 29 September By 1 October 2016, teams were deployed in the field and started the RV. The RV were completed successfully by 10 October Data processing The PES data entry program was developed using CSPro software by SSL with the support of the UNFPA technical team. There were 24 data entry operators for the PES data capture, under supervision of 3 SSL staff. Initially, there were 20 data entry operators, but after one week, additional four were recruited to ensure completion of the exercise on time. The data entry staff was trained on the data entry application from 28 to 30 September PES Data entry lasted from 3 to 31 October After the competition of the data entry, the data files were cleaned and computed weights were applied. Then tables were generated from the weighted data as programmed in the PES tabulation plan. 7

16 2.8 Main Challenges The main challenges encountered during the PES implementation were: Cases of difficulties in map reading and large outgrown EAs: As it was the case in the main census, PES enumerators faced difficulties with reading some EA maps, and in some cases covering large EAs, particularly in urban areas. To overcome the mapping challenges, enough mappers were timely deployed to support the PES enumeration exercise. Delay in PES matching: The PES matching started on 16 May 2016 with an initial 16 matchers and 4 supervisors. The matchers faced some challenges related to mapping and locating some households. The challenges increased the workload by forcing matchers to extend the exercise to more adjacent EAs (each EA has 2 to 5 adjacent EAs). To overcome this challenge, an additional six matchers were recruited, trained to support the exercise. In addition, 10 labourers were recruited to help on tracing questionnaires movements during the matching exercise. The additional staff joined the team of matchers on 18 June The taken measures boosted the exercise considerably. The exercise was completed successfully in four months, instead three months initially planned. 8

17 CHAPTER 3 COVERAGE EVALUATION This chapter presents the estimated census coverage rates. These include the census omission rate, coverage rate, erroneous inclusions rate, true population, net coverage error, net coverage rate and gross coverage error rate. 3.1 Population estimates The population estimates presented in this section are the PES estimated population, the census enumerated population, the true estimated population and the census undercount. The PES population is the sum of non-movers and in-movers. The true population was estimated using the dual system, as explained in annex 2. The census undercount is the difference between the true population and the census enumerated population. The population estimates concern only the household population. Table 3.1 below presents the population estimates by sex, age groups, area of residence and region (domain). Table 3.1: Population estimates by Sex, Age, Area of Residence and Region Characteristic PES Population Census Enumerated Population True Population Census Undercount Sierra Leone 7,058,640 7,076,119 7,233, ,823 Age group 0-14 Years 2,866,186 2,892,240 2,957,398 65, Years 2,585,031 2,569,780 2,634,213 64, Years 1,362,048 1,367,815 1,391,718 23, Years + 245, , ,613 4,329 Sex Male 3,490,939 3,479,633 3,561,421 81,788 Female 3,567,701 3,596,486 3,672,521 76,035 Area of Residence Rural 4,471,980 4,182,489 4,240,111 57,622 Urban 2,586,660 2,893,630 2,993, ,201 Region Eastern 1,697,619 1,640,592 1,676,615 36,023 Northern 2,659,616 2,502,583 2,551,100 48,517 Southern 1,402,842 1,439,165 1,449,590 10,425 Western 1,298,563 1,493,779 1,556,637 62,858 Source: Computed. 9

18 At the national level, the PES population is estimated at 7,058,640 persons, 4,471,980 in rural area and 2,586,660 in urban area. The country s true population is estimated at 7,233,942, compared to 7,076,119 people enumerated by census. In the rural area, the true population is estimated at 4,240,111 people, compared to 4,182,489 enumerated by census. In urban area, the true population is estimated 2,993,831 against 2,893,630 given by the census enumeration. The census undercount is estimated at 157,823 persons at the national level - 81,788 males and 76,035 females. By area of residence, the estimates suggest that 57,622 persons were omitted by the census in rural area and almost the double in urban areas - 100,201 inhabitants. By region, the census undercount is estimated as follows: 36,023 people in Eastern region, 48,517 people in Northern region, 10,425 people in Southern region and 62,858 people in Western area. 3.2 Coverage rates The calculation of the Census coverage errors is one of the main PES objectives. Table 3.2 below summarizes the coverage errors indicators by sex, age, area of residence and region. At the national level, the Sierra Leone 2015 Census coverage rate is estimated at percent, which is considered as one of the highest coverage in the sub-saharan African region. By area of residence, the coverage rate is higher by almost two points in rural area, percent, compared to urban area percent. By domain, the highest coverage rate was registered in Southern region percent, followed by Northern region percent, Eastern region percent, and the lowest in the Western area percent. By characteristics, the results show no significant differences in coverage rate between males and females, percent and percent respectively. The results suggest that the coverage is higher for elders percent for population aged 65 years and over, compared to the youngest population percent for the population aged less than 15 years. For the remaining coverage errors, the Net Coverage Error Rate is estimated at 2.18 percent at the national level, the Erroneous Inclusion Rate at.09 percent and the Gross Coverage Error Rate at 2.25 percent. 10

19 Table 3.2: Coverage Error Rates by Sex, Age, Area of Residence and Region Characteristic Net Coverage Error Rate Erroneous Inclusion Rate Gross Coverage Error Rate Omission Rate Coverage Rate Sierra Leone Age group 0-14 Years Years Years Years Sex Male Female Area of Residence Rural Urban Region Eastern Northern Southern Western Source: Computed. 3.3 Coverage Errors As shown in table 3.3, the census erroneous inclusion is estimated 6,152 individuals, 1,766 in urban area, almost two and half times in higher in urban area - 4,386 people. About 65 percent of the census erroneous inclusions were in Western area - 4,024. By characteristic, out of the 6,152 erroneous inclusions in census, 53.9 percent are males, while 78.3 percent are aged less than 35 years. The Gross Coverage Error (GCE) is evaluated at 160,022 persons, 83,682 males and 76,340 females, while the Net Coverage Error (NCE) is 153,101-80,031 males and 73,070. By area of residence, the GCE reaches 64,980 people in rural area compared to 95,042 in urban area, while the NCE is 63,002 in rural area against 73,070 in urban area. 11

20 Table 3.3: Coverage Errors by Sex, Age, Area of Residence and Region Characteristic Erroneous Inclusion Gross Coverage Error Net Coverage Error Sierra Leone 6, , ,101 Age group 0-14 Years 2,376 66,047 63, Years 2,439 65,494 62, Years 1,276 24,470 22, Years ,011 4,013 Sex Male 3,313 83,682 80,031 Female 2,839 76,340 73,070 Area of Residence Rural 1,766 64,980 63,002 Urban 4,386 95,042 90,099 Region Eastern ,292 37,821 Northern ,834 49,806 Southern 1,308 12,352 10,093 Western 4,024 60,544 55,381 Source: Computed. 12

21 CHAPTER 4 CONTENT EVALUATION The evaluation of the census content is one of the key objectives for the Sierra Leone 2015 PES. The content evaluation was done by measuring the content error. The content error, also refeed to as response error, measures the variability -not the bias- between responses captured in PES and the ones in census for some selected variables. This chapter presents the findings of the content evaluation for the following key sociodemographic variables: relationship to head of household, sex, age, marital status and literacy. The content errors are estimated only for matched persons. To ensure the comparability between PES and the Census, the same methodology used for data collection in Census (wording, responses categories, concepts, definition training, etc.) was also applied for the PES exercise. 4.1 Content evaluation indicators Five indicators are commonly used to measure the content error (variability), namely: (i) net difference rate; (ii) index of inconsistency; (iii) aggregate index of inconsistency, (iv) the gross difference rate; and (v) the rate of agreement. The indicators definitions and mathematical formulas are: The Net difference rate (NDR) is the difference between the number of cases in the census and the number of cases in the PES that fall under each response category relative to the total number of reported persons in both the census and PES in all response categories. The formula used is: For the i th category, i= 1,2,,s where: x.i = unweighted census number of cases in the ith category. x i. = unweighted PES number of cases in the ith category. n = unweighted total number of reported persons in both census and PES. s = total number of response categories for characteristic x. The Index of Inconsistency (II): It is the ratio of the simple response variance to the total variance of the characteristic, including its variability in the population. It is calculated for each response category using the following formula: For the i th category, i= 1,2,,s Where: x ii = number of cases where category was given as response in both the census and the PES. 13

22 The Aggregate Index of Inconsistency (AII): It is a summary measure of the index of inconsistency (that is for all the response categories of the characteristic as a whole). The formula used is: c n xii ˆ i I = c n x i xi n i The Gross Difference Rate (GDR): It is the number of discrepancies between the census responses and the PES responses relative to the total number of persons matched. It is equivalent to the sum of all cells off the diagonal, for all categories, or the complement of the sum of the diagonal cells. The formula used is: n GDR = 1 n n s i sc i x ii x i The Rate of agreement (RA): It is the complement of the gross difference rate. A low rate of agreement indicates a high degree of variability, and vice versa. The formula used is: The standards for the interpretation of the different content error measures, as defined by UNSD, are presented in table 4.1 below: Table 4.1: Interpretation of Indices for Content Errors Measure Level Low Moderate High Index of inconsistency < > 50 Aggregate index of inconsistency Absolute Value of NDR ( NDR ) < > 50 < >0.05 Source: UNSD (2010), Post Enumeration Surveys Operational Guidelines 4.2 Content Analysis Results Over all, the assessment of the content error shows that the Sierra Leone 2015 Census is of a good data quality. For the five selected demographic variables, findings show high level of agreement and low or moderate level of inconsistency. The agreement rate ranges between 86.3 percent for the variable literacy, followed by age percent; marital status percent; relationship to head of household percent; and as high as 99.6 percent for sex. In addition, the results suggest low level of inconsistency (AII < 20) for four out of the five selected variables, and moderate inconsistency for the fifth variable - literacy (AII = 21.80). 14

23 The content error indictors, with 95 percent confidence intervals, for the five selected variables are presented in tables 4.2 to 4.5 below Content Analysis for Relationship to Head of Household For the characteristic relationship to head of household, the results reveal an overall moderate level for the net difference, low level of inconsistency and high agreement rate. The Net Difference Rate (NDR) is moderate (0.01< NDR <= 0.05) for categories uncle/aunt, grandparent, grandchild, step son/daughter and non-relative, while it is relatively high ( NDR > 0.05) for the remaining categories. The index of inconsistency varies between 4.64 for the category head to for the category uncle/aunt, while the Aggregate Index of Inconsistency is estimated at only 6.80, reflecting low level of inconsistency. The agreement rate reaches percent, as shown in table 4.1 below. Table 4.2: Net Difference Rate and Indices of Inconsistency for Relationship to Head of Household Category Number of Consistent Cases Number of Cases in Census Number of Cases in PES Net Difference Rate Index of Inconsistency Rate 95% Confidence Interval Index 95% Confidence Interval Lower Upper Lower Upper Head 24,440 25,403 25, Spouse 16,003 17,129 16, Son/Daughter 57,642 59,373 59, Sister/Brother 8,188 8,809 9, Nephew/Niece 5,861 6,555 6, Parent 1,586 1,766 1, Uncle/Aunt 1,201 1,439 1, In-Law 2,195 2,413 2, Grand Parent Grand Child 9,088 9,644 9, Step Son/ Daughter 2,005 2,354 2, Other 2,083 2,317 2, Non-Relative 1,608 1,806 1, Undetermined N/A N/A N/A N/A N/A N/A Total 132, , , Aggregate Index = Gross Difference Rate = 5.16% Agreement Rate = 94.84% Source: Computed. 15

24 4.2.2 Content Analysis for Sex The results show low level of net difference and inconsistency and very high agreement rate for the variable sex. The net difference rate (NDR) is.003 for category male and for the category female, which is considered low level of net difference ( NDR <.01). The index of inconsistency is estimated at.75 for both categories -male and female- and for the variable as a whole, which indicates low level of inconsistency. The agreement for is as high as 99.62%. Table 4.3: Net Difference Rate and Indices of Inconsistency for Sex Category Number of Consistent Cases Number of Cases in Census Number of Cases in PES Net Difference Rate Index of Inconsistency Rate 95% Confidence Interval Index 95% Confidence Interval Lower Upper Lower Upper Male 68,716 68,980 68, Female 70,510 70,769 70, Undetermined N/A N/A N/A N/A N/A N/A Total 139, , , Aggregate Index = Gross Difference Rate = 0.38% Agreement Rate = 99.62% Source: Computed Content Analysis for Age PES findings suggest low/moderate level of net difference and inconsistency, and high agreement rate for the variable age. As shown in table 4.3, the absolute value of the NDR is.05 for the upper age groups (45 years and above), while it is higher than.05 for the remaining age groups, reflecting a high level of net difference. The index of consistency values remains less than 20 for all age groups, indicating low level of inconsistency. The aggregate index of inconsistency is estimated at The agreement rate is percent. 16

25 Table 4.4: Net Difference Rate and Indices of Inconsistency for Age Category Number of Consistent Cases Number of Cases in Census Number of Cases in PES Net Difference Rate Index of Inconsistency Rate 95% Confidence Interval Index 95% Confidence Interval Lower Upper Lower Upper Under 5 years 17,004 18,471 17, years 37,033 39,048 39, years 14,855 17,030 16, years 22,441 24,766 24, years 20,759 22,703 23, years 11,508 12,805 12, or more ,903 4, Undetermined N/A N/A N/A N/A N/A N/A Total 128, , , Aggregate Index = Gross Difference Rate = 8.60% Agreement Rate = 91.39% Content Analysis for Marital Status Source: Computed. For marital status, the results reflect relatively moderate net difference and inconsistency and high agreement rate. The absolute value for NDR is less than.01 for two categories ( divorced and don t know ),.05 for the category separated and higher than.05 for the remaining categories. The index of consistency is at low level (II<20) for 7 categories out of the 10 marital status categories and moderate for the remaining three categories. The aggregate index of inconsistency is estimated at 9.32, reflecting low level of inconsistency. As to the agreement rate, it reaches percent, as shown in table 4.4 below. 17

26 Table 4.5: Net Difference Rate and Indices of Inconsistency f or Marital Status Category Number of Consistent Cases Number of Cases in Census Number of Cases in PES Net Difference Rate Index of Inconsistency Rate 95% Confidence Interval Index 95% Confidence Interval Lower Upper Lower Upper Never Married 41,336 42,421 42, Engaged 3,001 4,230 3, Married Monoga-mous Married Polyga-mous Co-habitation (< 5 years) Co-habitation (= >5 years) 32,612 34,478 35, ,084 6,835 7, Separated 1,059 1,229 1, Divorced Widowed 3,676 3,912 4, Don't know Undetermined N/A N/A N/A N/A N/A N/A Total 89,342 95,199 95, Aggregate Index = Gross Difference Rate = 6.15% Agreement Rate = 93.85% Source: Computed Content Analysis for Literacy The variable literacy initially comprised 33 categories of responses. To facilitate the interpretation of results, the 33 categories were grouped into four, as presented in table 4.5 below. The findings suggest high level of net difference, moderate level of consistency and an acceptable agreement rate for the variable literacy. The absolute value of the NDR is higher than.05 for all categories. The index of inconsistency registered value less than 20 (low) for two categories (illiterate and literate in English only), while it ranges from 20 to 50 (moderate) for the remaining categories. The aggregate index of inconsistency is evaluated at 21.80, reflecting moderate level of inconsistency. The agreement rate for literacy is percent. 18

27 Table 4.6: Net Difference Rate and Indices of Inconsistency for Literacy Category Number of Consistent Cases Number of Cases in Census Number of Cases in PES Net Difference Rate Index of Inconsistency Rate 95% Confidence Interval Index 95% Confidence Interval Lower Upper Lower Upper Illiterate 42,077 46,346 44, Literate in Local Language only Literate in English only 3,219 3,940 6, ,603 39,750 39, Other literate 3,824 4,484 6, Don't know Undetermined 0 4,673 3,007 N/A N/A N/A N/A N/A N/A Total 85,980 99,638 99, Aggregate Index = Gross Difference Rate = 13.71% Agreement Rate = 86.29% Source: Computed. 19

28 CHAPTER 5 CONCLUSION AND RECOMMANDATIONS 5.1 Conclusion The Sierra Leone 2015 PES was implemented successfully, with no major challenges. It was implemented by SSL, with the UNFPA technical support. In addition, the PES was monitored by independent and international monitors. All the PES activities were implemented within the recommended timing and according to international norms and standards. Maps reading and large EAs in some urban areas, and a delay of one month in matching activities were the main challenges encountered during the PES implementation. The challenges had no impact on the PES data quality. The PES results reveal that the Sierra Leone 2015 Census has a good coverage, and a good data quality. It is recommended to use the PES findings to improve the planning, implementation and data quality for future censuses and household surveys in the country. 5.2 Recommendations The PES key recommendations for future censuses are to: Improve on the process of deployment of field staff. The field deployment plan should ensure that field staff are deployed in their assigned areas well before commencement of fieldwork, field staff are assigned to EAs closer to their places of recruitment and adequate means of transport are provided for the supervision activities. Improve on the logistics plan for urban areas, particularly in the Western urban district, to ensure timely and sufficient supply of materials during data collection, as well as timely retrieval of completed questionnaires and other documents. 03 Ensure adequate training of the field staff on maps reading Ensure more quality control for maps field activities to avoid, as much as possible, large EA being assigned to one enumerator. Improve on the payment system for future censuses and surveys. 20

29 BIBLIOGRAPHY US Census Bureau, Evaluating censuses of Population and Housing, ISP-TR-5, US Census Bureau, UN Statistical Division, Post Enumeration Surveys Operational Guidelines, UN, New York, April Statistics South Africa (2004). Census 2001: Post-enumeration Survey: Results and Methodology. Statistics South Africa, Pretoria, South Africa. Report No (2001). 21

30 ANNEXES Annex 1: PES Concepts and definitions Section 1.01 Post enumeration survey (PES): Is a complete re-enumeration of a representative sample of a census population followed by matching each individual enumerated in the PES with information from the census enumeration (UN, 2008). Census reference night: The night between 4 and 5 December PES reference night: The night from 26 to 27 February Household: A household is a group of people who live together and provide themselves jointly with food or other essentials for living, or a single person who lives alone. For PES and census purposes, only people present in the household on the reference nights (census and PES) are included as part of the household. Enumeration area: An enumeration area (EA) is the smallest geographical unit into which the country is divided for census 2015 enumeration purposes. Each EA is expected to have clearly defined boundaries. EA number: The EA number is a unique ID number given to an EA for record-keeping and coding purposes. Enumeration: Enumeration is the process of counting all the members of a defined population and collecting demographic and other information about each person. This counting takes place by means of administering a PES questionnaire to all households in the sampled EAs. the PES, and who were also present on the night between 4 and 5 December 2015, that is, the reference night for the census, including babies, the elderly, visitors, and non-citizens. In-movers: Persons who were present in the household on the night between 26 and 27 February 2016, that is, the reference night for the PES, but who were absent on the night between 4 and 5 December 2015, that is, the reference night for the census, including babies, the elderly, visitors, and non- citizens. Out-movers: Persons who were absent from the household on the night between 26 and 27 February 2016, that is, the reference night for the PES, but who were present on the night between 4 and 5 December 2015, that is, the reference night for the census, including babies, the elderly, visitors, and non-citizens. Born after the Census: Babies who were present in the household on the night between 26 and 27 February 2016, that is, the reference night for the PES, but who were not yet born as of the night between 4 and 5 December 2015, that is, the reference night for the census. Even though these babies are included in the list of household members, they are different from the in- movers, because they are out of the scope of the target population. Out of scope: Persons who are not non-movers, in-movers or out-movers. Non-movers: Persons who were present in the household on the night between 26 and 27 February 2016, that is, the reference night for 22

31 Annex 2: Dual System Estimation Source: UNSD PES operational guidelines, New York, April 2010 The Dual System Estimation is implemented in PES to estimate the True Population of persons in households. The Chandrasekaran-Deming estimator, assuming independence, is expressed as follows: Nˆ ˆ N ˆ = + ++ N Nˆ 1 Correctly enumerated persons: In order to operationalise the dual system estimator there is need, as a first step, to define, in the census, the list of persons correctly enumerated. This is essential because since the general net coverage model assumes that events like duplications, non-existent or out-of-scope cases have been identified in both the census and PES and accounted for in the estimation. According to Mule, 2008, correctly enumerated has four aspects, namely, appropriateness, uniqueness, completeness and geographic correctness. In order, therefore, to come up with a good Dual System Estimates the PES organizers should try to ascertain correctly the enumerated population. Proportion of people captured in the census Having defined the set of correctly enumerated persons, the next step in the Dual System Estimation is to estimate census coverage. See the formula below. N 1 Census coverage rate (which is called the match rate) =. N Thus, matched population relative 1 to the PES population. + Steps followed in Dual System Estimation A number of census coverage estimates can be based on initial tabulations referred to in chapter 9. Population estimates are calculated for selected population parameters. The estimates are population estimates from the P sample and the E sample (Dauphin and Canamucio, 1993). It is helpful to identify all the elements that are essential in making Dual System Estimates. We hereby assign symbols to various estimates to facilitate the developments of compact standard formulas. In this case: a= total number of non-movers (estimated from P sample); b= estimated total number of out-movers (from P sample); c= estimated total number of in-movers (from P sample); d= estimated total number of matched non-movers (based on matched cases between census and P sample); e= the total number of matched out-movers (based on matched cases between census and P sample); f= total number of erroneous inclusions in the population (from the E sample) g= total number of census cases correctly enumerated in the census but missed in the PES (from E sample) Matched Population = Matched non-movers + Estimated matched out-movers ˆN 1 = d+e= 23

32 Census population estimate The census estimate is obtained as follows: Census Population = Matched non-movers + matched out-movers + population erroneously included in the census +population correctly enumerated in the census but missed in the PES Census population = d+e+f+g = PES sample estimate of total population PES population = Number of non-movers + in-movers PES population = a+c = True Population This is the population estimated from the PES multiplied by the population from the census after correcting for erroneous inclusions and divided by matched population between the census and the PES. True population = Net coverage error This is the difference between what should have been counted, thus, the True Population and what was counted in the census. Net coverage error = True population Census population Net coverage error rate PES Population The measure is the total net error relative to the Dual System Estimate of the True Population. It is an important indicator of the quality of census coverage. Net coverage error rate = True population Census population * 100 True Population Estimating census omissions ( Census Population Erroneous inclusions) Matched Population Net coverage error = True population Census population= Omissions Erroneous Inclusions Then, Omissions = True population Census population + Erroneous Inclusions Census omissions rate The census omission rate is the missed population relative to the PES population estimate. Omission rate = Omissions 100 TruePopulation 24

33 Coverage rate (match rate) Is the matched population between the census and PES relative to PES population. Coverage rate = Matchedpopulation 100 PESpopulation Erroneous Inclusions and its rate The erroneous inclusions as earlier stated include fabrications, out-of-scope, geographic misallocations, etc. It will be recalled that the objective of the E sample is to provide an estimate of erroneous inclusions. This facilitates the correction in the Dual System Estimate of the True Population. Erroneous inclusion rate = ErreneousInclusions CensusPopulation 100 Gross coverage error Some countries use it as an indicator of the operational quality of the census enumerations. It is the sum of omissions and erroneous inclusions. Gross coverage error = Omissions + Erroneous inclusions Gross coverage error rate per unit enumeration This is the absolute gross error relative to the census enumerated population Gross coverage error rate per unit of enumeration = (Omissions + Erroneous inclusions) * 100 Census Population 25

34 Annex 2: Matching Guidelines DEFINITION Matching is the process which entails the comparison of PES and census households and persons information to check whether everyone was enumerated during census. This will permit the calculation of coverage error and the determination of cases for which the content errors were calculated. In general, the basic process of matching involves comparing addresses, names and demographic characteristics between census and PES results. It is, therefore, an operation whereby households and persons enumerated during the census and PES are compared for similarities. Matching is conducted into two phases: Initial matching: Use information from the PES and census questionnaires to assign the initial match status. Final matching: Assigning the final match status based on reconciliation visits outcomes. There are two types of matching variables: Primary matching variables: The main variables used to determine a match status for persons; i.e. name, age and sex. Secondary matching variables: These variables are not reliable enough to determine a match status for persons. These include: relationship to head of household, place of birth, literacy and marital status. Moving status: An indication of a household or a person s presence based on the PES and census reference nights. Three categories of moving status: o o o 1 = Non-mover 2 = In-mover 3 = Out-of-scope Under Procedure C the matching is attempted for non-movers and out-movers. PES Enumeration Status PES Enumeration Status o Non movers o Out movers o In movers o Born After Census o Non movers o Out movers o In movers o Born After Census 26

35 MATCHING TASKS The matching process involves the following: (i) Gathering material: The following materials should be gathered before the start of the matching: Structures listing forms for both the PES and Census; Census questionnaires for the selected EAs; PES questionnaires; Maps for the PES EAs sample used for both Census and PES (ii) Determining the EA (or EAs) to be searched. (iii) Matching households: Searching for the census questionnaires for household(s) that matched with the corresponding PES household questionnaire for each selected EA. Matching of households involved comparing the names of Administrative units, census household numbers and the names of household members therein. Follow the following steps for households matching: Step 1: Take each PES HH questionnaire and the related census household questionnaires Step 2: status. Starting from the first listed household, check section II (Matching particulars) for enumeration status If in section II, P01 =1 and/or P02 = 1, then: - The supervisor should assign pairs of related (PES and Census) questionnaires to two matching clerks. One of the matching clerks should loudly read names of the household head and other names in the PES questionnaire, - The second matching clerk should thoroughly check for similar names in the census records. Since the sequence of the names may not be same in both questionnaires, all names should be checked for similarity. - The household head name and/or spouse could be adequate for deciding whether the household matched or not. - The structure number in P03 in section II in the PES questionnaire if provided can guide you in identifying the household. - If the name/names agree, complete (matching status) as = 1 and assign a sequence number (the same number) to the PES and census questionnaire: M01, M02, etc. in each of the census and PES questionnaire. - If the names do not agree, assign a different code to that household: NM01, NM02 If in section II, P01 = 2 and/or P02 = 2, then: - Check names of household head and other household members to determine whether the household was enumerated. If the names agree with those in census, code matching status as = 1and then continue as above. - If name/names in PES and census do not agree, complete section (matching status) as = 2, and put the census questionnaire and assign a sequence number (the same number) to the PES and census questionnaire: M01, M02, etc. in each of the census and PES questionnaire. 27

36 - If the name/names somehow agree, complete section (matching status) as = 3, and and assign a sequence number (the same number) to the PES and census questionnaire: PM01, PM02, etc. in each of the census and PES questionnaire. - All the un-matched questionnaires (Census and PES) should be handed over to the supervisors pending the reconciliation visits. EXAMPLES FOR MATCHED, NON MATECHED AND PARTIALLY MATCHED For all non-match census questionnaires: - Check related cartographic household lists to determine whether they are listed and if so, it will be entered into a separate form as correctly enumerated. If not, should be listed as requiring reconciliation - Check in adjacent EAs for similar names. If found, it will be entered into separate form as incorrectly enumerated. If not, should be listed as requiring reconciliation Step 3: Repeat the above steps until the fate of ALL questionnaires in the PES and Censuses are determined. Table 1: Conditions for identifying matching PES and census households Physical ID of DU (*) Sticker number (P02) Person match (1 or more matched) HH group Yes Yes Yes Household match Yes No Yes Household match No Yes Yes Household match Yes Yes No Possible match Yes No No Possible match No Yes No Possible match No No Yes Possible match No No No Non-match * Using structures listing forms (iv) Matching individuals This Section will be undertaken ONLY for matched households. The exact steps are: FOR SECTION III (ALL PERSONS WHO SPENT THE PES NIGHT IN THE HH) AND SECTION IV (OUT MOVERS) Step 1: Starting from the first listed household in PES questionnaires, match person by person record. Step 2: For all persons that match, write census information (relationship, sex, age, place of birth, marital status and literacy) in the appropriate gray boxes of the PES questionnaire. 28

37 Step 3: Complete question P10 (Moving Status) as given below: Non-mover = 1 In-mover = 2 Born after =3 Step 4: Complete P11 (Matching Status) as given below. The categories assigned to individuals in the matching operation are: Matched = 1 Non-Match = 2 Partly-Match = 3 Not Applicable = 4 Code 1 (matched) if: - Two names agree except for minor spelling differences OR, if the surname is common, the given name must agree except for minor differences. - Ages in PES and Census questionnaires correspond within acceptable limits in table 2 below: Table 2: Age tolerance (level 1)households - If Marital status is same (persons 10 years and above) Age (years) Under 10 Tolerance +/- 1 year - Sex in PES and census must agree exactly /- 2 years /- 3 years /- 4 years 60+ +/- 5 years Code 3 (partially matched) if: Conditions are the same as for the matched except for a contradiction in one of the following: - First name or surname, relationship, or sex. - Surname is the same but given name on either questionnaire is a recognized nickname of given name on the other questionnaire. - Conditions are the same as for the matched except that there is a larger age difference such as in table 3 below: Table 3: Age tolerance (level 2) - Conditions are the same as for the matched except that the family name differs. Age (years) Tolerance Under 10 +/- 3 year /- 4 years /- 5 years Over 40 +/- 6 years 29

38 Code 2 (Not-match) if: - The person cannot be matched even after considering the matched and partial matches Code 9 (not applicable) if: - P08 (Enumeration Status) is coded different than 1 (enumerated in this HH) - P10 (Moving Status) is coded different than 1 (Non movers) (v) Persons found in Census but not in PES: For persons who were counted during the census for a particular household (found in the Census questionnaires), but not counted during the PES (not in the PES questionnaire), for such cases, complete the form A1 in annexes. (vi) Sheet Section V to be completed from the Census questionnaire expect for the variable CENSUS ENUMERATION STATUS, which is completed after the reconciliation visits (Final matching). (vii) Supervision and Quality Control: There are two levels of supervision. The first level of supervision is done by the supervisors who have the oversight responsibility of editing and determining the final match status of questionnaires. The PES matching supervisors should verify all the matched cases. The second level of supervision is done by the team leader whose duties are to check and review work done by matchers and supervisors. 30

39

40 From the British people