14 One Proportion

14.1 Comparing various confidence intervals

We consider a binomial experiments (i.e., an iid sequence of Bernoulli trials) and the task of building a confidence interval for the probability parameter.

n = 30 # sample size
theta = 0.1 # true value of the parameter
x = rbinom(n, 1, theta) # sample
y = sum(x) # number of 1's in the sample

The Clopper–Pearson interval is “exact”, and in fact conservative in terms of level. We compute below the two-sided (which is the default) 90% confidence interval.

test = binom.test(y, n, conf.level = 0.90) 
cat(test$conf.int)

0.06805557 0.3189712

The Wilson interval is “approximate” in terms of level, relying on the normal approximation to the binomial distribution. (The two intervals tend to be closer the larger the sample size is.)

test = prop.test(y, n, conf.level = 0.90) 
cat(test$conf.int)

0.07224992 0.3238618

We compare these two intervals in simulations.

n = 30
theta = 0.1
B = 1e3 # number of replicates
cp_in = 0 # stores the number of times that the CP interval contains the true value
cp_len = numeric(B) # stores the length of the CP interval
w_in = 0 # stores the number of times that the W interval contains the true value
w_len = numeric(B) # stores the length of the W interval
for (b in 1:B){
  y = rbinom(1, n, theta) # we generate y directly to save time
  cp = binom.test(y, n, conf.level = 0.90)$conf.int # CP interval
  cp_in = cp_in + (cp[1] <= theta)*(cp[2] >= theta)
  cp_len[b] = cp[2] - cp[1]
  w = prop.test(y, n, conf.level = 0.90)$conf.int # W interval
  w_in = w_in + (w[1] <= theta)*(w[2] >= theta)
  w_len[b] = w[2] - w[1]
}
paste("fraction of times that the 0.90 CP interval contained the parameter:", cp_in/B)

[1] "fraction of times that the 0.90 CP interval contained the parameter: 0.928"

paste("fraction of times that the 0.90 W interval contained the parameter:", w_in/B)

[1] "fraction of times that the 0.90 W interval contained the parameter: 0.979"

summary(cp_len)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
0.09503 0.18335 0.21078 0.20254 0.23276 0.29875 

summary(w_len)

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
 0.1110  0.1901  0.2148  0.2077  0.2349  0.2961