Importance of shrinkage in empirical bayes estimates for diagnostics: problems and solutions

Abstract

Empirical Bayes ("post hoc") estimates (EBEs) of etas provide modelers with diagnostics: the EBEs themselves, individual prediction (IPRED), and residual errors (individual weighted residual (IWRES)). When data are uninformative at the individual level, the EBE distribution will shrink towards zero (eta-shrinkage, quantified as 1-SD(eta (EBE))/omega), IPREDs towards the corresponding observations, and IWRES towards zero (epsilon-shrinkage, quantified as 1-SD(IWRES)). These diagnostics are widely used in pharmacokinetic (PK) pharmacodynamic (PD) modeling; we investigate here their usefulness in the presence of shrinkage. Datasets were simulated from a range of PK PD models, EBEs estimated in non-linear mixed effects modeling based on the true or a misspecified model, and desired diagnostics evaluated both qualitatively and quantitatively. Identified consequences of eta-shrinkage on EBE-based model diagnostics include non-normal and/or asymmetric distribution of EBEs with their mean values ("ETABAR") significantly different from zero, even for a correctly specified model; EBE-EBE correlations and covariate relationships may be masked, falsely induced, or the shape of the true relationship distorted. Consequences of epsilon-shrinkage included low power of IPRED and IWRES to diagnose structural and residual error model misspecification, respectively. EBE-based diagnostics should be interpreted with caution whenever substantial eta- or epsilon-shrinkage exists (usually greater than 20% to 30%). Reporting the magnitude of eta- and epsilon-shrinkage will facilitate the informed use and interpretation of EBE-based diagnostics.

Fig. 1

Deviation of empirical Bayes estimates (EBE) and individual weighted residual (IWRES) distribution from normality due to η- and ε-shrinkage, respectively, shown as a qq-plot (upper panels) and sum of squared distances from the true distributions (lower panels). This is a representative example of three simulations. EBE and IWRES were estimated using the true model

Fig. 2

Dependence of ETABAR on the time of the first and last sample for η _ka and η _CL distribution, respectively. Deviations of ETABAR from zero is a result of asymmetric empirical Bayes estimates shrinkage

Fig. 3

Empirical Bayes estimates (EBE) versus EBE plot indicating parameter correlation due to η-shrinkage when correlation is not truly present (upper panel) and hiding parameter correlation when correlation is truly present (lower panel)

Fig. 4

Left panel: relationship between (induced) correlation between η _Emax and η _EC50, η _ka–η _V and η _ka–η _CL, and η-shrinkage. Right panel: disappearance of correlation between empirical Bayes estimates with increased shrinkage. The average of the shrinkage in two studied parameters (e.g., η _Emax and η _EC50) is shown on the x-axes. Each symbol represents mean (SE) based on ten simulation–estimation procedures

Fig. 5

Empirical Bayes estimates versus covariate plot indicating parameter-covariate relationship due to η-shrinkage when relationship is not truly present

Fig. 6

Relationship between shrinkage extent and induced covariate (left panel) and hidden covariate relationship (right panel). The average of the η _ka- and η _V-shrinkage is shown on the x-axes of the left panel

Fig. 7

Individual prediction versus dependent variable plot for detection of structural model misspecification

Fig. 8

Power of individual weighted residual (|IWRES|) versus individual prediction (IPRED) to detect residual model misspecification. In absence of shrinkage, the regression line of |IWRES| versus IPRED clearly indicates residual model misspecification (left panel). With increased shrinkage, the slope of the regression line is diminishing (right panel)

Fig. 9

Relationship between η- and ε-shrinkage under different study designs