Making Use of Benford s Law for the Randomized Response Technique. Andreas Diekmann ETH-Zurich

Transcription

1 Benford & RRT Making Use of Benford s Law for the Randomized Response Technique Andreas Diekmann ETH-Zurich

2 1. The Newcomb-Benford Law Imagine a little bet. The two betters bet on the first digit it of an unknown house number drawn at random. The loser has to pay one euro to the winner. Player A wins if the digit is in the range 1 to 4. Player B wins if the digit is to 9. Is this a fair bet?

3 1. The Newcomb-Benford Law Imagine a little bet. The two betters bet on the first digit it of an unknown house number drawn at random. The loser has to pay one euro to the winner. Player A wins if the digit is in the range 1 to 4. Player B wins if the digit is to 9. Is this a fair bet? It is not. Paradoxically, the bet is rather unfavourable to player B. The first digits of house numbers follow a logarithmic distribution known as Benford s law. The betters odds are :3 in terms of objective probabilities.

4 Hungerbühler 2

5 Benford s Law P(d 1 )=log 1 (1 + 1/d 1 ) P(D 1 = d 1,..., D k = d k ) = log 1 [ 1 + (Σd i 1 k-i ) -1 ] with d 1 = 1, 2,...,9 and d j =, 1,...,9 (j = 2,..., k).

6 Distribution of First Digits of OLS-Regressions Coefficients from Articles Published in the American Journal of Sociology First Digit Distribution.3.3 Fre equencies First Digit Actual Benford Upper Bound Lower Bound N = 14, Tables from AJS 14 / 1. Deviation from Benford is significant for α=.. Diekmann 2

7 Hungerbühler 2 Digits in the Bible Compilation of Digits in the Elberfelder Konkordanz

8 Hungerbühler 2 Digits in the Bible Compilation of Digits in the Elberfelder Konkordanz

9 Benford s Law and the number of votes for candidate Ahmadinejad (Roukema 29)

10 Sensitive Questions Allen H. Barton, 198. Asking the Embarrassing Question. Public Opinion i Quarterly 22: -88

11 Barton s (198) method for a very sensitive question

12

13 May be RRT is a better method for asking sensitive questions?

14 2. The Randomized Response Technique (RRT). A Method to Guarantee Full Anonymity for Sensitive Questions Subjects had to respond to either a sensitive question A (e.g. shoplifting, tax evasion etc.) or to a random question B (Was your mother s birthday in an even month?). Assignment to question A or B is by a random device (a dice, a coin etc.) The meaning of an individual answer cannot be identified. However, it is possible to estimate the proportion of shoplifting etc. and other statistics on the aggregate level.

15 Because the random mechanisms are known one can estimate the probability of answering yes to the sensitive question by Bayes formula. The RRT has the advantage of guaranteeing anonymity, but not without costs. The price is a loss in efficiency. In addition to sampling error, the probabilistic RRT device enlarges the variance of the estimated proportion of positive responses to the sensitive question.

16 In formal terms: p is the probability to answer the question of interest A, q =1-p is the probability to answer the random question B. π y = P( yes B) is the probability to response yes to the random question. Then, we are looking for an estimate of π x = P( yes A), the expected proportion of respondents answering yes to the question of interest. If we denote the overall proportion of yes in the sample by λ we have: λ = p π x + (1-p) π y. (λ, p,π y is known)

17 Solving for π x yields: π x = λ/p π y (1-p)/p. p and π y are determined ex ante by the researcher s RRT-design. A special case is the forced response design with π y = 1. In this case, a person is forced to respond yes to the random question. With variance: Var(π x ) = λ(1- λ)/np 2

18 3. The Benford distribution as a randomizing device In face-to-face interviews, a pack of cards, a dice, a coin or some other device may be used to generate randomized outcomes. For example, if a person tosses head he or she is instructed to answer the random question, if the result is tail the question of interest has to be answered. This technique has some difficulties in telephone interviews and is particularly problematic in selfadministered interviews such as mailed questionnaires or online-surveys. As an alternative, I suggest to make use of the Benford distribution.

19 House numbers (1st digit) 1,2,3,4 versus,,,8,9 The probability that digit 1, 2, 3 or 4 turns out is, therefore,.99 or roughly.. The probability to draw a first digit among the set of remaining digits is.3. The :3 rule provides a mechanism to split the sample in a set of respondents answering the question of interest A and respondents answering the random question B. For example, subjects are asked to think of the address of a friend and to keep the house number in mind. Depending on the first digit either belonging to the set {1,2,3,4} or belonging to the set {,,,8,9} a person has to answer question A or question B. Other sets may be constructed if a researcher prefers smaller or larger probabilities for the question of interest. However, first we should ask: Do house numbers follow the Benford distribution at all?

20 House numbers collected from the telephone directory of Zurich 3% 3% Per rcentage 2% 2% 1% 1% % % House number 29,99% 1,9% 13,1% 1,84% 8,4%,9% 4,% 4,4%,12% Benford 3,1% 1,1% 12,49% 9,9%,92%,9%,8%,12% 4,8% First digit

21 I i d bt d t S I am indebted to S. Wehrli for compiling the data.

22 4. The Benford illusion and other advantages of the method The price for the anonymity of the method is an increase in the variance of the estimator for the proportion of yes-responses (π x ) to the question of interest. The variance is (Fox and Tracy 198): Var(π x ) = λ(1- λ)/n(1-q) 2 It follows that the variance increases with the probability q = 1-p to arrive at the random question. On the other hand, the larger q the larger is the degree of anonymity. This is the formal expression for the conflict between efficiency and anonymity.

23 Benford Illusion To use the Benford distribution for the RRT has the advantage to diminish i i the conflict between efficiency and anonymity. The reason is that the perceived probabilities and the objective probabilities differ. Many people believe that the chance to pick a one, two, three or four is much smaller than percent. This discrepancy or Benford illusion has the positive effect that t the perceived q, and, therefore, the perceived anonymity is larger than the objective q. With the little trick of the Benford illusion, the anonymity can be increased without loss in efficiency.

24 There are other advantages, too. The method does not require any physical device such as a coin or a dice to generate random numbers. In most previous studies, the RRT is applied to sensitive questions in face-to- face interviews. However, it is unlikely that most people, asked to fill in online-surveys or mailed questionnaires, follow instructions properly if a coin or dice is required.

25 . Application Shoplifting Questionnaire Imagine a friend or relative who does not live in your house with an address known n to you. Keep in mind the house number s first digit. If the digit ist,,,8 or 9 skip over the next question and mark yes If the digit is 1,2,3,4, please, answer the following question: In the last five years, did you ever intentionally pick a shopping item without paying for it?

26 Study 1: Shoplifting RRT Experiment in Vorlesung SS Questionnaire in lecture M. Abraham, Bern 2 Ja = 88, Ja = 114 Nein = Ja = , = 2, Result: n =29 2, p (Ladendiebstahl) = 2,/2,/2 =,12 Nein = 181 n = 29 π x =.12 (SE =.4)

27 Study 2: Shoplifting Result: n = 93 π x = 9/ =.14 (SE =.3) Questionnaire in lecture Szydlick

28 . Do Subjects underestimate the probability of 1,2,3,4? ( Benford Illusion ) Schätzung der Häufigkeit der Hausnummern mit erster Ziffer 1,2,3, Percent 4 P N = 289, N 289, Mean = 1. Lecture M Schätzung der Häufigkeit der Hausnummern mit erster Ziffer 1,2,3,4 Abraham, Bern 2

29 Estimated t frequency of fhouse numbers starting with 1, 2, 3 or 4 in per cent Percentage e of answe ers Lecture Szydlik, n = 92, mean = 4

30 Underestimation of Objective Probability (student population) subjective (mean) objective Study 1, Bern 1 Study 2, Zurch 4

31 . Do subjects generate Benforddistributed house numbers? As we have seen, objective data follow the Benford distribution. However, are the digits produced by the respondents in accordance with Benford as well? This is a crucial assumption. Otherwise, This is a crucial assumption. Otherwise, the method wouldn t work.

32 . Do subjects generate Benforddistributed house numbers? I am indebted to B. Jann for compiling the data. Survey B. Jann, Wages in Switzerland, 2/2, N = 313