Home > Model checking & comparison > A Bayesian test of the equality of proportions

A Bayesian test of the equality of proportions

One problem with a p-value is that it is hard to interpret if you don’t know the sample size.  Andy Gelman talks about this problem on his blog.

Here I’ll contrast the message communicated by a p-value and a Bayesian test by looking at the following three tables:

Table A:  [32 18; 18 32]
Table B: [272 228; 228,272]
Table C: [2570 2430; 2430,2570]

Each of the tables has a Pearson chi-square value in the (7.4, 7.8) range and so each has (approximately) the same p-value.  But, following Gelman’s reasoning, the small p-value for the large sample size in Table C doesn’t have much significance and and the same p-value for Table A (small sample size) is more impressive.
Here is a reasonable Bayesian test of the equality of two proportions.  First, we define a class of priors indexed by the parameter K:
1.  p_1, p_2 are iid beta(K\eta, K(1-\eta))
2.  \eta has a uniform density
Testing H: p_1 = p_2, A: p_1 \neq p_2 is equivalent to testing H: K = \infty against K = 1 (I’ll talk more about this in class.)
Chapter 8 in BCWR gives an expression for the Bayes factor in support of A over H.   In the LearnBayes package, suppose the data is a matrix of two columns where the first column contains the counts of successes of the two samples and the second column contains the sample sizes.  Then the Bayes factor (on the log scale) is computed by the single line
s=laplace(bfexch,0,list(data=data,K=1))$int
Let’s illustrate using this for the three tables:
> laplace(bfexch,0,list(data=cbind(c(32,18),c(50,50)),K=1))$int
[1] 0.9436246
> laplace(bfexch,0,list(data=cbind(c(272,228),c(500,500)),K=1))$int
[1] -0.3035413
> laplace(bfexch,0,list(data=cbind(c(2570,2430),c(5000,5000)),K=1))$int
[1] -1.411366
So the Bayes factor against equality of proportions is exp(.94) = 2.56 for Table A, exp(-0.30) = 0.74 for Table B, and exp(-1.41) = 0.24 for Table C.  In contrast to the message communicated by a p-value, the Bayes factor calculation indicates that Table A is the most significant table.
Advertisements
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: