Gibbs sampling is very convenient for many “missing data” problems. To illustrate this situation, suppose we have the simple regression model
where the errors
We illustrate this situation by use of a picture inspired by a similar one in Martin Tanner’s book. We show a scatterplot of the covariate
To apply Gibbs sampling to this situation, we imagine a complete data set where all of the
The joint posterior of all unobservables (assuming the usual vague prior for regression) has the form
With the introduction of the complete data set, Gibbs sampling is straightforward.
Suppose one has initial estimates at the regression coefficients and the error variance. Then
(1) One simulates from the distribution of the missing data (the
(2) Given the complete data, one simulates values of the parameters using the usual approach for a normal linear regression model.
To do this on R, I have a couple of useful tools. The function rtruncated.R will simulate draws from an arbitrary truncated distribution
For example, if one wishes to simulate 20 draws of a normal(mean = 10, sd = 2) distribution that is truncated below by 4, one writes
rtruncated(20, 4, Inf, pnorm, qnorm, 10, 2)
Also the function blinreg.R in the LearnBayes package will simulate draws of a regression coefficient