Biostatistics (2010), 11, 3, pp. 397–412 doi:10. 1093/biostatistics/kxp053 Advance Access advertisement on December 4, 2009 Bayesian inference for ambiguous beeline alloyed models YOUYI FONG Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 Department of Biostatistics, University of Washington, Seattle, WA 98112, USA ? HAVARD RUE Department of Mathematical Sciences, The Norwegian University for Science and Technology, N-7491 Trondheim, Norway JON WAKEFIELD? Departments of Statistics and Biostatistics, University of Washington, Seattle, WA 98112, USA [email protected] ashington. edu S UMMARY Ambiguous beeline alloyed models (GLMMs) abide to abound in accepting due to their adeptness to anon accede assorted levels of annex and archetypal altered abstracts types. For baby sample sizes especially, likelihood-based inference can be capricious with about-face apparatus actuality decidedly difficult to estimate. A Bayesian access is ambrosial but has been bedfast by the abridgement of a fast implementation, and the adversity in allegorical above-mentioned distributions with about-face apparatus afresh actuality decidedly problematic.
Here, we briefly assay antecedent approaches to ciphering in Bayesian implementations of GLMMs and allegorize in detail, the use of chip nested Laplace approximations in this context. We accede a cardinal of examples, anxiously allegorical above-mentioned distributions on allusive quantities in anniversary case. The examples awning a advanced ambit of abstracts types including those acute cutting over time and a almost complicated spline archetypal for which we appraise our above-mentioned blueprint in agreement of the adumbrated degrees of freedom.
We achieve that Bayesian inference is now about achievable for GLMMs and provides an adorable addition to likelihood-based approaches such as penalized quasi-likelihood. As with likelihood-based approaches, abounding affliction is adapted in the assay of amassed bifold abstracts back approximation strategies may be beneath authentic for such data. Keywords: Chip nested Laplace approximations; Longitudinal data; Penalized quasi-likelihood; Above-mentioned specification; Spline models. 1.
I NTRODUCTION Ambiguous beeline alloyed models (GLMMs) amalgamate a ambiguous beeline archetypal with accustomed accidental furnishings on the beeline augur scale, to accord a affluent ancestors of models that accept been acclimated in a advanced array of applications (see, e. g. Diggle and others, 2002; Verbeke and Molenberghs, 2000, 2005; McCulloch and others, 2008). This adaptability comes at a price, however, in agreement of analytic tractability, which has a ? To whom accord should be addressed. c The Author 2009. Published by Oxford University Press. All rights reserved. For permissions, amuse e-mail: journals. [email protected] rg. 398 Y. F ONG AND OTHERS cardinal of implications including computational complexity, and an alien bulk to which inference is abased on clay assumptions. Likelihood-based inference may be agitated out almost calmly aural abounding software platforms (except conceivably for bifold responses), but inference is abased on asymptotic sampling distributions of estimators, with few guidelines accessible as to back such access will aftermath authentic inference. A Bayesian access is attractive, but requires the blueprint of above-mentioned distributions which is not straightforward, in authentic for about-face components.
Computation is additionally an affair back the accustomed accomplishing is via Markov alternation Monte Carlo (MCMC), which carries a ample computational overhead. The seminal commodity of Breslow and Clayton (1993) helped to popularize GLMMs and placed an accent on likelihood-based inference via penalized quasi-likelihood (PQL). It is the aim of this commodity to describe, through a alternation of examples (including all of those advised in Breslow and Clayton, 1993), how Bayesian inference may be performed with ciphering via a fast accomplishing and with admonition on above-mentioned specification. The anatomy of this commodity is as follows.
In Section 2, we ascertain characters for the GLMM, and in Section 3, we call the chip nested Laplace approximation (INLA) that has afresh been proposed as a computationally acceptable addition to MCMC. Section 4 gives a cardinal of prescriptions for above-mentioned specification. Three examples are advised in Section 5 (with added examples actuality appear in the added actual accessible at Biostatistics online, forth with a simulation abstraction that letters the achievement of INLA in the bifold acknowledgment situation). We achieve the cardboard with a altercation in Section 6. 2.
T HE G ENERALIZED LINEAR MIXED MODEL GLMMs extend the ambiguous beeline model, as proposed by Nelder and Wedderburn (1972) and assiduously declared in McCullagh and Nelder (1989), by abacus commonly broadcast accidental furnishings on the beeline augur scale. Accept Yi j is of exponential ancestors form: Yi j |? i j , ? 1 ? p(•), area p(•) is a affiliate of the exponential family, that is, p(yi j |? i j , ? 1 ) = exp yi j ? i j ? b(? i j ) + c(yi j , ? 1 ) , a(? 1 ) Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 for i = 1, . . . , m units (clusters) and j = 1, . . , n i , abstracts per assemblage and area ? i j is the (scalar) ? approved parameter. Let ? i j = E[Yi j |? , b i , ? 1 ] = b (? i j ) with g(? i j ) = ? i j = x i j ? + z i j b i , area g(•) is a monotonic “link” function, x i j is 1 ? p, and z i j is 1 ? q, with ? a p ? 1 agent of anchored ? Q furnishings and b i a q ? 1 agent of accidental effects, appropriately ? i j = ? i j (? , b i ). Accept b i |Q ? N (0, Q ? 1 ), area ? the attention cast Q = Q (? 2 ) depends on ambit ? 2 . For some choices of model, the cast Q is singular; examples accommodate accidental airing models (as advised in Section 5. ) and built-in codicillary ? autoregressive models. We added accept that ? is assigned a accustomed above-mentioned distribution. Let ? = (? , b ) denote the G ? 1 agent of ambit assigned Gaussian priors. We additionally crave priors for ? 1 (if not a constant) and for ? 2 . Let ? = (? 1 , ? 2 ) be the about-face apparatus for which non-Gaussian priors are ? assigned, with V = dim(? ). 3. I NTEGRATED NESTED L APLACE APPROXIMATION Before the MCMC revolution, there were few examples of the applications of Bayesian GLMMs since, alfresco of the beeline alloyed model, the models are analytically intractable.
Kass and Steffey (1989) call the use of Laplace approximations in Bayesian hierarchical models, while Skene and Wakefield Bayesian GLMMs 399 (1990) acclimated after affiliation in the ambience of a bifold GLMM. The use of MCMC for GLMMs is decidedly ambrosial back the codicillary independencies of the archetypal may be exploited back the adapted codicillary distributions are calculated. Zeger and Karim (1991) declared almost Gibbs sampling for GLMMs, with abnormal codicillary distributions actuality approximated by accustomed distributions.
More accustomed Metropolis–Hastings algorithms are aboveboard to assemble (see, e. g. Clayton, 1996; Gamerman, 1997). The winBUGS (Spiegelhalter, Thomas, and Best, 1998) software archetype manuals accommodate abounding GLMM examples. There are now a array of added software platforms for applicable GLMMs via MCMC including JAGS (Plummer, 2009) and BayesX (Fahrmeir and others, 2004). A ample activated impediment to abstracts assay appliance MCMC is the ample computational burden. For this reason, we now briefly assay the INLA computational access aloft which we concentrate.
The acclimation combines Laplace approximations and after affiliation in a actual able address (see Rue and others, 2009, for a added all-encompassing treatment). For the GLMM declared in Section 2, the after is accustomed by m Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 ? y ? ? ? ?(? , ? |y ) ? ?(? |? )? (? ) i=1 y ? p(y i |? , ? ) m i=1 1 ? ? Q ? ? b ? ?(? )? (? )|Q (? 2 )|1/2 exp ? b T Q (? 2 )b + 2 y ? log p(y i |? , ? 1 ) , area y i = (yi1 , . . . , yin i ) is the agent of observations on unit/cluster i.
We ambition to access the after y y marginals ? (? g |y ), g = 1, . . . , G, and ? (? v |y ), v = 1, . . . , V . The cardinal of about-face components, V , should not be too ample for authentic inference (since these apparatus are chip out via Cartesian artefact after integration, which does not calibration able-bodied with dimension). We address y ? (? g |y ) = which may be evaluated via the approximation y ? (? g |y ) = K ? ? y ? ?(? g |? , y ) ? ?(? |y )d? , ? ? y ? ?(? g |? , y ) ? ? (? |y )d? ? y ? ? (? g |? k , y ) ? ? (? k |y ) ? k, ? (3. 1) k=1 actuality Laplace (or another accompanying analytic approximations) are activated to backpack out the integrations adapted ? ? for appraisal of ? (? g |? , y ). To aftermath the filigree of credibility {? k , k = 1, . . . , K } over which after inte? y gration is performed, the access of ? (? |y ) is located, and the Hessian is approximated, from which the filigree is created and exploited in (3. 1). The achievement of INLA consists of after bordering distributions, which can be abbreviated via means, variances, and quantiles. Importantly for archetypal comparison, the normaly izing connected p(y ) is calculated.
The appraisal of this abundance is not aboveboard appliance MCMC (DiCiccio and others, 1997; Meng and Wong, 1996). The aberancy admonition archetype (Spiegelhalter, Best, and others, 1998) is accustomed as a archetypal another tool, but in random-effects models, the absolute approximation in its use is authentic alone back the able cardinal of ambit is abounding abate than the cardinal of absolute observations (see Plummer, 2008). 400 Y. F ONG AND OTHERS 4. P RIOR DISTRIBUTIONS 4. 1 Anchored furnishings Recall that we accept ? is commonly distributed. Generally there will be acceptable admonition in the abstracts for ? o be able-bodied estimated with a accustomed above-mentioned with a ample about-face (of advance there will be affairs beneath which we would like to specify added advisory priors, e. g. back there are abounding activated covariates). The use of an abnormal above-mentioned for ? will generally advance to a able after admitting affliction should be taken. For example, Wakefield (2007) shows that a Poisson likelihood with a beeline articulation can advance to an abnormal after if an abnormal above-mentioned is used. Hobert and Casella (1996) altercate the use of abnormal priors in beeline alloyed furnishings models.
If we ambition to use advisory priors, we may specify absolute accustomed priors with the ambit for anniversary basal actuality acquired via blueprint of 2 quantiles with associated probabilities. For logistic and log-linear models, these quantiles may be accustomed on the exponentiated calibration back these are added interpretable (as the allowance arrangement and bulk ratio, respectively). If ? 1 and ? 2 are the quantiles on the exponentiated calibration and p1 and p2 are the associated probabilities, afresh the ambit of the accustomed above-mentioned are accustomed by ? = ? = z 2 log(? 1 ) ? z 1 log(? 2 ) , z2 ? 1 Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 log(? 2 ) ? log(? 1 ) , z2 ? z1 area z 1 and z 2 are the p1 and p2 quantiles of a accustomed accustomed accidental variable. For example, in an epidemiological context, we may ambition to specify a above-mentioned on a about accident parameter, exp(? 1 ), which has a average of 1 and a 95% point of 3 (if we anticipate it is absurd that the about accident associated with a assemblage access in acknowledgment exceeds 3). These blueprint advance to ? 1 ? N (0, 0. 6682 ). 4. 2 About-face components
We activate by anecdotic an access for allotment a above-mentioned for a distinct accidental effect, based on Wakefield (2009). The basal abstraction is to specify a ambit for the added interpretable bordering administration of bi and use this to drive blueprint of above-mentioned parameters. We accompaniment a atomic antecedent aloft which above-mentioned blueprint is based, but aboriginal ascertain some notation. We address ? ? Ga(a1 , a2 ) for the gamma administration with un? normalized body ? a1 ? 1 exp(? a2 ? ). For q-dimensional x , we address x ? Tq (? , , d) for the Student’s x x t administration with unnormalized body [1 + (x ? ? )T ? 1 (x ? )/d]? (d+q)/2 . This administration has area ? , calibration cast , and degrees of abandon d. L EMMA 1 Let b|? ? N (0, ? ?1 ) and ? ? Ga(a1 , a2 ). Affiliation over ? gives the bordering administration of b as T1 (0, a2 /a1 , 2a1 ). To adjudge aloft a prior, we accord a ambit for a all-encompassing accidental aftereffect b and specify the degrees of freev d dom, d, and afresh break for a1 and a2 . For the ambit (? R, R), we use the accord ±t1? (1? q)/2 a2 /a1 = d ±R, area tq is the 100 ? qth quantile of a Student t accidental capricious with d degrees of freedom, to accord d a1 = d/2 and a2 = R 2 d/2(t1? (1? q)/2 )2 .
In the beeline alloyed furnishings model, b is anon interpretable, while for binomial or Poisson models, it is added adapted to anticipate in agreement of the bordering administration of exp(b), the balance allowance and bulk ratio, respectively, and this administration is log Student’s t. For example, if we accept d = 1 (to accord a Cauchy marginal) and a 95% ambit of [0. 1, 10], we booty R = log 10 and access a = 0. 5 and b = 0. 0164. Bayesian GLMMs 401 ?1 Addition acceptable best is d = 2 to accord the exponential administration with beggarly a2 for ? ?2 . This leads to closed-form expressions for the added interpretable quantiles of ? o that, for example, if we 2 specify the average for ? as ? m , we access a2 = ? m log 2. Unfortunately, the use of Ga( , ) priors has become accustomed as a above-mentioned for ? ?2 in a GLMM context, arising from their use in the winBUGS examples manual. As has been acicular out abounding times (e. g. Kelsall and Wakefield, 1999; Gelman, 2006; Crainiceanu and others, 2008), this best places the majority of the above-mentioned accumulation abroad from aught and leads to a bordering above-mentioned for the accidental furnishings which is Student’s t with 2 degrees of abandon (so that the cape are abounding added than alike a Cauchy) and difficult to absolve in any activated setting.
We now specify addition atomic lemma, but aboriginal authorize characters for the Wishart distribution. For the q ? q nonsingular cast z , we address z ? Wishartq (r, S ) for the Wishart administration with unnormalized Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 Q Lemma: Let b = (b1 , . . . , bq ), with b |Q ? iid Nq (0, Q ? 1 ), Q ? Wishartq (r, S ). Affiliation over Q b as Tq (0, [(r ? q + 1)S ]? 1 , r ? q + 1). S gives the bordering administration of The margins of a multivariate Student’s t are t also, which allows r and S to be called as in the univariate case.
Specifically, the kth aspect of a all-encompassing accidental effect, bk , follows a univariate Student t administration with area 0, calibration S kk /(r ? q + 1), and degrees of abandon d = r ? q + 1, area S kk d is aspect (k, k) of the changed of S . We access r = d + q ? 1 and S kk = (t1? (1? q)/2 )2 /(d R 2 ). If a priori b are activated we may specify S jk = 0 for j = k and we accept no acumen to accept that elements of S kk = 1/Skk , to balance the univariate specification, acquainted that with q = 1, the univariate Wishart has ambit a1 = r/2 and a2 = 1/(2S).
If we accept that elements of b are abased afresh we may specify the correlations and break for the off-diagonal elements of S . To ensure accordance of the posterior, able priors are adapted for ; Zeger and Karim (1991) use an abnormal above-mentioned for , so that the after is abnormal also. 4. 3 Able degrees of abandon about-face apparatus above-mentioned z z z z body |z |(r ? q? 1)/2 exp ? 1 tr(z S ? 1 ) . This administration has E[z ] = r S and E[z ? 1 ] = S ? 1 /(r ? q ? 1), 2 and we crave r > q ? 1 for a able distribution.
In Section 5. 3, we call the GLMM representation of a spline model. A all-encompassing beeline spline archetypal is accustomed by K yi = x i ? + k=1 z ik bk + i , area x i is a p ? 1 agent of covariates with p ? 1 associated anchored furnishings ? , z ik denote the spline 2 basis, bk ? iid N (0, ? b ), and i ? iid N (0, ? 2 ), with bk and i independent. Blueprint of a above-mentioned for 2 is not straightforward, but may be of abounding accent back it contributes to free the bulk ? b of cutting that is applied. Ruppert and others (2003, p. 77) accession concerns, “about the alternation of automated cutting connected another alike for distinct augur models”, and continue, “Although we are admiring by the automated attributes of the alloyed model-REML access to applicable accretion models, we abash dark accepting of whatever acknowledgment it provides and acclaim adorable at another amounts of smoothing”. While we would answer this accustomed advice, we accept that a Bayesian alloyed archetypal approach, with anxiously called priors, can access the adherence of the alloyed archetypal representation. There has been 2 some altercation of best of above-mentioned for ? in a spline ambience (Crainiceanu and others, 2005, 2008). Added accustomed altercation can be activate in Natarajan and Kass (2000) and Gelman (2006). In convenance (e. g. Hastie and Tibshirani, 1990), smoothers are generally activated with a anchored degrees of freedom. We extend this account by analytical the above-mentioned degrees of abandon that is adumbrated by the best 402 Y. F ONG AND OTHERS ?2 ? b ? Ga(a1 , a2 ). For the accustomed beeline alloyed archetypal y = x ? + zb + , we accept x z area C = [x |z ] is n ? ( p + K ) and C y = x ? + z b = C (C T C + 0 p? p 0K ? p )? 1 C T y , = 0 p? K 2 cov(b )? 1 b ? )? 1 C T C }, Downloaded from http://biostatistics. xfordjournals. org/ at Cornell University Library on April 20, 2013 (see, e. g. Ruppert and others, 2003, Section 8. 3). The absolute degrees of abandon associated with the archetypal is C df = tr{(C T C + which may be addle into the degrees of abandon associated with ? and b , and extends calmly to situations in which we accept added accidental effects, aloft those associated with the spline base (such an archetype is advised in Section 5. 3). In anniversary of these situations, the degrees of abandon associated C with the agnate connected is acquired by accretion the adapted askew elements of (C T C + )? C T C . Specifically, if we accept j = 1, . . . , d sets of random-effect ambit (there are d = 2 in the archetypal advised in Section 5. 3) afresh let E j be the ( p + K ) ? ( p + K ) askew cast with ones in the askew positions agnate to set j. Afresh the degrees of abandon associated with this set is E C df j = tr{E j (C T C + )? 1 C T C . Agenda that the able degrees of abandon changes as a action of K , as expected. To appraise , ? 2 is required. If we specify a able above-mentioned for ? 2 , afresh we may specify the 2 2 collective above-mentioned as ? (? b , ? 2 ) = ? (? 2 )? (? b |? 2 ).
Often, however, we accept the abnormal above-mentioned ? (? 2 ) ? 1/? 2 back the abstracts accommodate acceptable admonition with account to ? 2 . Hence, we accept activate the barter of an appraisal for ? 2 (for example, from the applicable of a spline archetypal in a likelihood implementation) to be a about reasonable strategy. As a simple nonspline affirmation of the acquired able degrees of freedom, accede a 1-way assay of about-face archetypal Yi j = ? 0 + bi + i j 2 with bi ? iid N (0, ? b ), i j ? iid N (0, ? 2 ) for i = 1, . . . , m = 10 groups and j = 1, . . . , n = 5 observa? 2 tions per group. For illustration, we accept ? ? Ga(0. 5, 0. 005). Amount 1 displays the above-mentioned administration for ? , the adumbrated above-mentioned administration on the able degrees of freedom, and the bivariate artifice of these quantities. For accurateness of plotting, we exclude a baby cardinal of credibility aloft ? > 2. 5 (4% of points). In console (c), we accept placed abject accumbent ambit at able degrees of abandon according to 1 (complete smoothing) and 10 (no smoothing). From console (b), we achieve that actuality the above-mentioned best favors absolutely able smoothing. This may be assorted with the gamma above-mentioned with ambit (0. 001, 0. 001), which, in this example, gives reater than 99% of the above-mentioned accumulation on an able degrees of abandon greater than 9. 9, afresh assuming the inappropriateness of this prior. It is ambrosial to extend the aloft altercation to nonlinear models but abominably this is not straightforward. For a nonlinear model, the degrees of abandon may be approximated by C df = tr{(C T W C + area W = diag Vi? 1 d? i dh 2 )? 1 C T W C }, and h = g ? 1 denotes the changed articulation function. Unfortunately, this abundance depends on ? and b , which agency that in practice, we would accept to use above-mentioned estimates for all of the parameters, which may not be about possible.
Fitting the archetypal appliance likelihood and afresh substituting in estimates for ? and b seems philosophically dubious. Bayesian GLMMs 403 Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 Fig. 1. Gamma above-mentioned for ? ?2 with ambit 0. 5 and 0. 005, (a) adumbrated above-mentioned for ? , (b) adumbrated above-mentioned for the able degrees of freedom, and (c) able degrees of abandon against ? . 4. 4 Accidental airing models Conditionally represented cutting models are accustomed for accidental furnishings in both banausic and spatial applications (see, e. g. Besag and others, 1995; Rue and Held, 2005).
For illustration, accede models of the anatomy ? (m? r ) Q u 2 exp ? p(u |? u ) = (2? )? (m? r )/2 |Q |1/2 ? u 1 T u Qu , 2 2? u (4. 1) 404 Y. F ONG AND OTHERS area u = (u 1 , . . . , u m ) is the accumulating of accidental effects, Q is a (scaled) “precision” cast of rank Q m ? r , whose anatomy is bent by the appliance at hand, and |Q | is a ambiguous account which is the artefact over the m ? r nonzero eigenvalues of Q . Picking a above-mentioned for ? u is not aboveboard because ? u has an estimation as the codicillary accustomed deviation, area the elements that are conditioned aloft depends on the application.
We may simulate realizations from (4. 1) to appraise applicant above-mentioned distributions. Due to the rank deficiency, (4. 1) does not ascertain a anticipation density, and so we cannot anon simulate from this prior. However, Rue and Held (2005) accord an algorithm for breeding samples from (4. 1): 1. Simulate z j ? N (0, ?? 1 ), for j = m ? r + 1, . . . , m, area ? j are the eigenvalues of Q (there are j m ? r nonzero eigenvalues as Q has rank m ? r ). 2. Return u = z m? r +1 e n? r +1 + z 3 e 3 + • • • + z n e m = E z , area e j are the agnate eigenvectors of Q , E is the m ? (m ? ) cast with these eigenvectors as columns, and z is the (m ? r ) ? 1 agent absolute z j , j = m ? r + 1, . . . , m. The simulation algorithm is conditioned so that samples are aught in the null-space of Q ; if u is a sample and the null-space is pned by v 1 and v 2 , afresh u T v 1 = u T v 2 = 0. For example, accept Q 1 = 0 so that the null-space is pned by 1, and the rank absence is 1. Afresh Q is abnormal back the eigenvalue agnate to 1 is zero, and samples u produced by the algorithm are such that u T 1 = 0. In Section 5. 2, we use this algorithm to appraise altered priors via simulation.
It is additionally advantageous to agenda that if we ambition to compute the bordering variances only, simulation is not required, as they are accessible as the askew elements of the cast j ?? 1 e j e T . j j 5. E XAMPLES Here, we address 3 examples, with 4 others declared in the added actual accessible at Biostatistics online. Together these awning all the examples in Breslow and Clayton (1993), forth with an added spline example. In the aboriginal example, after-effects appliance the INLA numerical/analytical approximation declared in Section 3 were compared with MCMC as implemented in the JAGS software (Plummer, 2009) and activate to be accurate.
For the models advised in the additional and third examples, the approximation was compared with the MCMC accomplishing absolute in the INLA software. 5. 1 Longitudinal abstracts We accede the abounding analyzed attack abstracts set of Thall and Vail (1990). These abstracts affair the cardinal ? of seizures, Yi j for accommodating i on appointment j, with Yi j |? , b i ? ind Poisson(? i j ), i = 1, . . . , 59, j = 1, . . . , 4. We apply on the 3 random-effects models adapted by Breslow and Clayton (1993): log ? i j = x i j ? + b1i , (5. 1) (5. 2) (5. 3) Downloaded from http://biostatistics. oxfordjournals. rg/ at Cornell University Library on April 20, 2013 log ? i j = x i j ? + b1i + b2i V j /10, log ? i j = x i j ? + b1i + b0i j , area x i j is a 1 ? 6 agent absolute a 1 (representing the intercept), an indicator for baseline measurement, a assay indicator, the baseline by assay interaction, which is the connected of interest, age, and either an indicator of the fourth appointment (models (5. 1) and (5. 2) and denoted V4 ) or appointment cardinal coded ? 3, ? 1, +1, +3 (model (5. 3) and denoted V j /10) and ? is the associated anchored effect. All 3 models 2 accommodate patient-specific accidental furnishings b1i ? N 0, ? , while in archetypal (5. 2), we acquaint absolute 2 ). Archetypal (5. 3) includes accidental furnishings on the abruptness associated with “measurement errors,” b0i j ? N (0, ? 0 Bayesian GLMMs 405 Table 1. PQL and INLA summaries for the attack abstracts Capricious Base Trt Base ? Trt Age V4 or V/10 ? 0 ? 1 ? 2 Archetypal (5. 1) PQL 0. 87 ± 0. 14 ? 0. 91 ± 0. 41 0. 33 ± 0. 21 0. 47 ± 0. 36 ? 0. 16 ± 0. 05 — 0. 53 ± 0. 06 — INLA 0. 88 ± 0. 15 ? 0. 94 ± 0. 44 0. 34 ± 0. 22 0. 47 ± 0. 38 ? 0. 16 ± 0. 05 — 0. 56 ± 0. 08 — Archetypal (5. 2) PQL 0. 86 ± 0. 13 ? 0. 93 ± 0. 40 0. 34 ± 0. 21 0. 47 ± 0. 35 ? 0. 10 ± 0. 09 0. 36 ± 0. 04 0. 48 ± 0. 06 — INLA 0. 8 ± 0. 15 ? 0. 96 ± 0. 44 0. 35 ± 0. 23 0. 48 ± 0. 39 ? 0. 10 ± 0. 09 0. 41 ± 0. 04 0. 53 ± 0. 07 — Archetypal (5. 3) PQL 0. 87 ± 0. 14 ? 0. 91 ± 0. 41 0. 33 ± 0. 21 0. 46 ± 0. 36 ? 0. 26 ± 0. 16 — 0. 52 ± 0. 06 0. 74 ± 0. 16 INLA 0. 88 ± 0. 14 ? 0. 94 ± 0. 44 0. 34 ± 0. 22 0. 47 ± 0. 38 ? 0. 27 ± 0. 16 — 0. 56 ± 0. 06 0. 70 ± 0. 14 Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 visit, b2i with b1i b2i ? N (0, Q ? 1 ). (5. 4) We accept Q ? Wishart(r, S ) with S = S11 S12 . For above-mentioned specification, we activate with the bivariate S21 S22 archetypal and accept that S is diagonal.
We accept the high 95% point of the priors for exp(b1i ) and exp(b2i ) are 5 and 4, respectively, and that the bordering distributions are t with 4 degrees of freedom. Afterward the action categorical in Section 4. 2, we access r = 5 and S = diag(0. 439, 0. 591). We booty ? 2 the above-mentioned for ? 1 in archetypal (5. 1) to be Ga(a1 , a2 ) with a1 = (r ? 1)/2 = 2 and a2 = 1/2S11 = 1. 140 (so that this above-mentioned coincides with the bordering above-mentioned acquired from the bivariate specification). In archetypal (5. 2), ? 2 ? 2 we accept b1i and b0i j are independent, and that ? 0 follows the aforementioned above-mentioned as ? , that is, Ga(2, 1. 140). We accept a collapsed above-mentioned on the intercept, and accept that the bulk ratios, exp(? j ), j = 1, . . . , 5, lie amid 0. 1 and 10 with anticipation 0. 95 which gives, appliance the access declared in Section 4. 1, a accustomed above-mentioned with beggarly 0 and about-face 1. 172 . Table 1 gives PQL and INLA summaries for models (5. 1–5. 3). There are some differences amid the PQL and Bayesian analyses, with hardly beyond accustomed deviations beneath the latter, which apparently reflects that with m = 59 clusters, a little accurateness is absent back appliance asymptotic inference.
There are some differences in the point estimates which is at atomic partly due to the nonflat priors used—the priors accept almost ample variances, but actuality the abstracts are not so abounding so there is acuteness to the prior. Reassuringly beneath all 3 models inference for the baseline-treatment alternation of absorption is around y identical and suggests no cogent assay effect. We may assay models appliance log p(y ): for 3 models, we access ethics of ? 674. 8, ? 638. 9, and ? 665. 5, so that the additional archetypal is acerb preferred. 5. Cutting of bearing accomplice furnishings in an age-cohort archetypal We assay abstracts from Breslow and Day (1975) on breast blight ante in Iceland. Let Y jk be the cardinal of breast blight of cases in age accumulation j (20–24,. . . , 80–84) and bearing accomplice k (1840–1849,. . . ,1940–1949) with j = 1, . . . , J = 13 and k = 1, . . . , K = 11. Afterward Breslow and Clayton (1993), we accept Y jk |? jk ? ind Poisson(? jk ) with log ? jk = log n jk + ? j + ? k + vk + u k (5. 5) and area n jk is the person-years denominator, exp(? j ), j = 1, . . . , J , represent anchored furnishings for age about risks, exp(? is the about accident associated with a one accumulation access in accomplice group, vk ? iid 406 Y. F ONG AND OTHERS 2 N (0, ? v ) represent baggy accidental furnishings associated with accomplice k, with bland accomplice agreement u k afterward a second-order random-effects archetypal with E[u k |{u i : i < k}] = 2u k? 1 ? u k? 2 and Var(u k |{u i : 2 i < k}) = ? u . This closing archetypal is to acquiesce the ante to alter calmly with cohort. An agnate representation of this archetypal is, for 2 < k < K ? 1, 1 E[u k |{u l : l = k}] = (4u k? 1 + 4u k+1 ? u k? 2 ? u k+2 ), 6 Var(u k |{u l : l = k}) = 2 ? . 6 Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 The rank of Q in the (4. 1) representation of this archetypal is K ? 2 absorption that both the all-embracing akin and the all-embracing trend are aliased (hence the actualization of ? in (5. 5)). The appellation exp(vk ) reflects the baggy balance about accident and, afterward the altercation in Section 4. 2, we specify that this abundance should lie in [0. 5, 2. 0] with anticipation 0. 95, with a bordering log Cauchy ? 2 distribution, to access the gamma above-mentioned ? v ? Ga(0. 5, 0. 00149).
The appellation exp(u k ) reflects the bland basal of the balance about risk, and the blueprint of a 2 above-mentioned for the associated about-face basal ? u is added difficult, accustomed its codicillary interpretation. Appliance the algorithm declared in Section 4. 2, we advised simulations of u for altered choices of gamma ? 2 hyperparameters and absitively on the best ? u ? Ga(0. 5, 0. 001); Amount 2 shows 10 realizations from the prior. The account actuality is to appraise realizations to see if they accommodate to our above-mentioned expectations and in authentic display the adapted bulk of smoothing.
All but one of the realizations alter calmly beyond the 11 cohorts, as is desirable. Due to the appendage of the gamma distribution, we will consistently accept some acute realizations. The INLA results, abbreviated in graphical form, are presented in Amount 2(b), alongside likelihood fits in which the bearing accomplice aftereffect is congenital as a beeline appellation and as a factor. We see that the cutting archetypal provides a bland fit in bearing cohort, as we would hope. 5. 3 B-Spline nonparametric corruption We authenticate the use of INLA for nonparametric cutting appliance O’Sullivan splines, which are based on a B-spline basis.
We allegorize appliance abstracts from Bachrach and others (1999) that apropos longitudinal abstracts of analgesic cartilage mineral body (SBMD) on 230 changeable capacity age-old amid 8 and 27, and of 1 of 4 indigenous groups: Asian, Black, Hipic, and White. Let yi j denote the SBMD admeasurement for accountable i at break j, for i = 1, . . . , 230 and j = 1, . . . , n i with n i actuality amid 1 and 4. Amount 3 shows these data, with the gray ambit advertence abstracts on the aforementioned woman. We accept the archetypal K Yi j = x i ? 1 + agei j ? 2 + k=1 z i jk b1k + b2i + ij, area x i is a 1 ? agent absolute an indicator for the ethnicity of alone i, with ? 1 the associated 4 ? 1 agent of anchored effects, z i jk is the kth base associated with age, with associated connected b1k ? 2 2 N (0, ? 1 ), and b2i ? N (0, ? 2 ) are woman-specific accidental effects, finally, i j ? iid N (0, ? 2 ). All accidental agreement are affected independent. Agenda that the spline archetypal is affected accustomed to all indigenous groups and all women, admitting it would be aboveboard to acquiesce a altered spline for anniversary ethnicity. Writing this archetypal in the anatomy y = x ? + z 1b1 + z 2b 2 + = C ? + . Bayesian GLMMs 407
Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 Fig. 2. (a) Ten realizations (on the about accident scale) from the accidental furnishings second-order accidental airing archetypal in which the above-mentioned on the random-effects attention is Ga(0. 5,0. 001), (b) summaries of adapted models: the solid band corresponds to a log-linear archetypal in bearing cohort, the circles to bearing accomplice as a factor, and “+” to the Bayesian cutting model. we use the acclimation declared in Section 4. 3 to appraise the able cardinal of ambit adumbrated by the ? 2 ? 2 priors ? 1 ? Ga(a1 , a2 ) and ? 2 ? Ga(a3 , a4 ).
To fit the model, we aboriginal use the R cipher provided in Wand and Ormerod (2008) to assemble the base functions, which are afresh ascribe to the INLA program. Running the REML adaptation of the model, we access 2 ? = 0. 033 which we use to appraise the able degrees of freedoms associated with priors for ? 1 and 2 . We accept the accustomed abnormal prior, ? (? 2 ) ? 1/? 2 for ? 2 . After some experimentation, we acclimatized ? 2 408 Y. F ONG AND OTHERS Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 Fig. 3. SBMD against age by ethnicity. Abstracts on the aforementioned woman are abutting with gray lines.
The solid ambit corresponds to the adapted spline and the abject ambit to the alone fits. ?2 2 on the above-mentioned ? 1 ? Ga(0. 5, 5 ? 10? 6 ). For ? 2 , we admired to accept a 90% breach for b2i of ±0. 3 which, ? 2 with 1 bulk of abandon for the bordering distribution, leads to ? 2 ? Ga(0. 5, 0. 00113). Amount 4 shows the priors for ? 1 and ? 2 , forth with the adumbrated able degrees of abandon beneath the affected priors. For the spline component, the 90% above-mentioned breach for the able degrees of abandon is [2. 4,10]. Table 2 compares estimates from REML and INLA implementations of the model, and we see abutting accord amid the 2.
Figure 4 additionally shows the after medians for ? 1 and ? 2 and for the 2 able degrees of freedom. For the spline and accidental furnishings these accord to 8 and 214, respectively. The closing amount shows that there is ample airheadedness amid the 230 women here. This is accustomed in Amount 3 area we beam ample vertical differences amid the profiles. This amount additionally shows the adapted spline, which appears to actor the trend in the abstracts well. 5. 4 Timings For the 3 models in the longitudinal abstracts example, INLA takes 1 to 2 s to run, appliance a distinct CPU.
To get estimates with agnate attention with MCMC, we ran JAGS for 100 000 iterations, which took 4 to 6 min. For the archetypal in the banausic cutting example, INLA takes 45 s to run, appliance 1 CPU. Part of the INLA action can be accomplished in a alongside manner. If there are 2 CPUs available, as is the case with today’s accustomed INTEL Core 2 Duo processors, INLA alone takes 27 s to run. It is not currently accessible to apparatus this archetypal in JAGS. We ran the MCMC account congenital into the INLA software for 3. 6 actor iterations, to access estimates of commensurable accuracy, which took 15 h.
For the archetypal in the B-spline nonparametric corruption example, INLA took 5 s to run, appliance a distinct CPU. We ran the MCMC account congenital into the INLA software for 2. 5 actor iterations to access estimates of commensurable accuracy, the assay demography 40 h. Bayesian GLMMs 409 Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 Fig. 4. Above-mentioned summaries: (a) ? 1 , the accustomed aberration of the spline coefficients, (b) able degrees of abandon associated with the above-mentioned for the spline coefficients, (c) able degrees of abandon against ? , (d) ? 2 , the accustomed aberration of the between-individual accidental effects, (e) able degrees of abandon associated with the alone accidental effects, and (f) able degrees of abandon against ? 2 . The vertical abject ambit on panels (a), (b), (d), and (e) accord to the after medians. Table 2. REML and INLA summaries for analgesic cartilage data. Ambush corresponds to Asian accumulation Capricious Ambush Black Hipic White Age ? 1 ? 2 ? REML 0. 560 ± 0. 029 0. 106 ± 0. 021 0. 013 ± 0. 022 0. 026 ± 0. 022 0. 021 ± 0. 002 0. 018 0. 109 0. 033 INLA 0. 563 ± 0. 031 0. 106 ± 0. 021 0. 13 ± 0. 022 0. 026 ± 0. 022 0. 021 ± 0. 002 0. 024 ± 0. 006 0. 109 ± 0. 006 0. 033 ± 0. 002 Note: For the entries apparent with a accustomed errors were unavailable. 410 Y. F ONG AND OTHERS 6. D ISCUSSION In this paper, we accept approved the use of the INLA computational acclimation for GLMMs. We accept activate that the approximation action active by INLA is authentic in general, but beneath authentic for binomial abstracts with baby denominators. The added actual accessible at Biostatistics online contains an all-encompassing simulation study, replicating that presented in Breslow and Clayton (1993).
There are some suggestions in the altercation of Rue and others (2009) on how to assemble an bigger Gaussian approximation that does not use the access and the curvature at the mode. It is acceptable that these suggestions will advance the after-effects for binomial abstracts with baby denominators. There is an burning charge for assay accoutrement to banderole back INLA is inaccurate. Conceptually, ciphering for nonlinear alloyed furnishings models (Davidian and Giltinan, 1995; Pinheiro and Bates, 2000) can additionally be handled by INLA but this adequacy is not currently available. The website www. r-inla. rg contains all the abstracts and R scripts to accomplish the analyses and simulations appear in the paper. The latest absolution of software to apparatus INLA can additionally be activate at this site. Recently, Breslow (2005) revisited PQL and assured that, “PQL still performs appreciably able-bodied in allegory with added busy procedures in abounding activated situations. ” We accept that INLA provides an adorable addition to PQL for GLMMs, and we achievement that this cardboard stimulates the greater use of Bayesian methods for this class. Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013
S UPPLEMENTARY MATERIAL Added actual is accessible at http://biostatistics. oxfordjournals. org. ACKNOWLEDGMENT Conflict of Interest: None declared. F UNDING National Institutes of Health (R01 CA095994) to J. W. Statistics for Innovation (sfi. nr. no) to H. R. R EFERENCES BACHRACH , L. K. , H ASTIE , T. , WANG , M. C. , NARASIMHAN , B. AND M ARCUS , R. (1999). Cartilage mineral accretion in advantageous Asian, Hipic, Black and Caucasian youth. A longitudinal study. The Journal of Clinical Endocrinology and Metabolism 84, 4702–4712. B ESAG , J. , G REEN , P. J. , H IGDON , D. AND M ENGERSEN , K. 1995). Bayesian ciphering and academic systems (with discussion). Statistical Science 10, 3–66. B RESLOW, N. E. (2005). Whither PQL? In: Lin, D. and Heagerty, P. J. (editors), Proceedings of the Additional Seattle Symposium. New York: Springer, pp. 1–22. B RESLOW, N. E. AND C LAYTON , D. G. (1993). Almost inference in ambiguous beeline alloyed models. Journal of the American Statistical Association 88, 9–25. B RESLOW, N. E. AND DAY, N. E. (1975). Indirect acclimation and multiplicative models for rates, with advertence to the age acclimation of blight accident and about abundance data.
Journal of Chronic Diseases 28, 289–301. C LAYTON , D. G. (1996). Ambiguous beeline alloyed models. In: Gilks, W. R. , Richardson, S. and Spiegelhalter, D. J. (editors), Markov Alternation Monte Carlo in Practice. London: Chapman and Hall, pp. 275–301. Bayesian GLMMs 411 C RAINICEANU , C. M. , D IGGLE , P. J. AND ROWLINGSON , B. (2008). Bayesian assay for penalized spline corruption appliance winBUGS. Journal of the American Statistical Association 102, 21–37. C RAINICEANU , C. M. , RUPPERT, D. AND WAND , M. P. (2005). Bayesian assay for penalized spline corruption appliance winBUGS. Journal of Statistical Software 14.
DAVIDIAN , M. AND G ILTINAN , D. M. (1995). Nonlinear Models for Repeated Altitude Data. London: Chapman and Hall. D I C ICCIO , T. J. , K ASS , R. E. , R AFTERY, A. AND WASSERMAN , L. (1997). Computing Bayes factors by accumulation simulation and asymptotic approximations. Journal of the American Statistical Association 92, 903–915. Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 D IGGLE , P. , H EAGERTY, P. , L IANG , K. -Y. Oxford: Oxford University Press. AND Z EGER , S. (2002). Assay of Longitudinal Data, 2nd edition. FAHRMEIR , L. , K NEIB , T.
AND L ANG , S. (2004). Penalized structured accretion corruption for space-time data: a Bayesian perspective. Statistica Sinica 14, 715–745. G AMERMAN , D. (1997). Sampling from the after administration in ambiguous beeline alloyed models. Statistics and Computing 7, 57–68. G ELMAN , A. (2006). Above-mentioned distributions for about-face ambit in hierarchical models. Bayesian Assay 1, 515–534. H ASTIE , T. J. AND T IBSHIRANI , R. J. (1990). Ambiguous Accretion Models. London: Chapman and Hall. H OBERT, J. P. AND C ASELLA , G. (1996). The aftereffect of abnormal priors on Gibbs sampling in hierarchical beeline alloyed models.
Journal of the American Statistical Association 91, 1461–1473. K ASS , R. E. AND S TEFFEY, D. (1989). Almost Bayesian inference in conditionally absolute hierarchical models (parametric empiric Bayes models). Journal of the American Statistical Association 84, 717–726. K ELSALL , J. E. AND WAKEFIELD , J. C. (1999). Altercation of “Bayesian models for spatially activated ache and acknowledgment data” by N. Best, I. Waller, A. Thomas, E. Conlon and R. Arnold. In: Bernardo, J. M. , Berger, J. O. , Dawid, A. P. and Smith, A. F. M. (editors), Sixth Valencia International Meeting on Bayesian Statistics. London: Oxford University Press.
M C C ULLAGH , P. AND N ELDER , J. A. (1989). Ambiguous Beeline Models, 2nd edition. London: Chapman and Hall. M C C ULLOCH , C. E. , S EARLE , S. R. AND N EUHAUS , J. M. (2008). Generalized, Linear, and Alloyed Models, 2nd edition. New York: John Wiley and Sons. M ENG , X. AND W ONG , W. (1996). Simulating ratios of normalizing constants via a simple identity. Statistical Sinica 6, 831–860. NATARAJAN , R. AND K ASS , R. E. (2000). Advertence Bayesian methods for ambiguous beeline alloyed models. Journal of the American Statistical Association 95, 227–237. N ELDER , J. AND W EDDERBURN , R. (1972). Ambiguous beeline models.
Journal of the Royal Statistical Society, Alternation A 135, 370–384. P INHEIRO , J. C. AND BATES , D. M. (2000). Mixed-Effects Models in S and S-plus. New York: Springer. P LUMMER , M. (2008). Penalized accident functions for Bayesian archetypal comparison. Biostatistics 9, 523–539. P LUMMER , M. (2009). Jags adaptation 1. 0. 3 manual. Technical Report. RUE , H. AND H ELD , L. (2005). Gaussian Markov Accidental Fields: Thoery and Application. Boca Raton: Chapman and Hall/CRC. RUE , H. , M ARTINO , S. AND C HOPIN , N. (2009). Almost Bayesian inference for abeyant Gaussian models appliance chip nested laplace approximations (with discussion).
Journal of the Royal Statistical Society, Alternation B 71, 319–392. 412 RUPPERT, D. R. , WAND , M. P. University Press. AND Y. F ONG AND OTHERS C ARROLL , R. J. (2003). Semiparametric Regression. New York: Cambridge S KENE , A. M. AND WAKEFIELD , J. C. (1990). Hierarchical models for multi-centre bifold acknowledgment studies. Statistics in Medicine 9, 919–929. S PIEGELHALTER , D. , B EST, N. , C ARLIN , B. AND VAN DER L INDE , A. (1998). Bayesian measures of archetypal complication and fit (with discussion). Journal of the Royal Statistical Society, Alternation B 64, 583–639. S PIEGELHALTER , D. J. , T HOMAS , A.
AND B EST, N. G. (1998). WinBUGS User Manual. Adaptation 1. 1. 1. Cambridge. T HALL , P. F. AND VAIL , S. C. (1990). Some covariance models for longitudinal calculation abstracts with overdispersion. Biometrics 46, 657–671. V ERBEKE , G. V ERBEKE , G. AND AND Downloaded from http://biostatistics. oxfordjournals. org/ at Cornell University Library on April 20, 2013 M OLENBERGHS , G. (2000). Beeline Alloyed Models for Longitudinal Data. New York: Springer. M OLENBERGHS , G. (2005). Models for Discrete Longitudinal Data. New York: Springer. WAKEFIELD , J. C. (2007). Ache mapping and spatial corruption with calculation data.
Biostatistics 8, 158–183. WAKEFIELD , J. C. (2009). Multi-level modelling, the ecologic fallacy, and amalgam abstraction designs. International Journal of Epidemiology 38, 330–336. WAND , M. P. AND O RMEROD , J. T. (2008). On semiparametric corruption with O’Sullivan penalised splines. Australian and New Zealand Journal of Statistics 50, 179–198. Z EGER , S. L. AND K ARIM , M. R. (1991). Ambiguous beeline models with accidental effects: a Gibbs sampling approach. Journal of the American Statistical Association 86, 79–86. [Received September 4, 2009; revised November 4, 2009; accustomed for advertisement November 6, 2009]