Bayesian Nonparametric Policy Search with Application to Periodontal Recall Intervals

Abstract

Tooth loss from periodontal disease is a major public health burden in the United States. Standard clinical practice is to recommend a dental visit every six months; however, this practice is not evidence-based, and poor dental outcomes and increasing dental insurance premiums indicate room for improvement. We consider a tailored approach that recommends recall time based on patient characteristics and medical history to minimize disease progression without increasing resource expenditures. We formalize this method as a dynamic treatment regime which comprises a sequence of decisions, one per stage of intervention, that follow a decision rule which maps current patient information to a recommendation for their next visit time. The dynamics of periodontal health, visit frequency, and patient compliance are complex, yet the estimated optimal regime must be interpretable to domain experts if it is to be integrated into clinical practice. We combine non-parametric Bayesian dynamics modeling with policy-search algorithms to estimate the optimal dynamic treatment regime within an interpretable class of regimes. Both simulation experiments and application to a rich database of electronic dental records from the HealthPartners HMO shows that our proposed method leads to better dental health without increasing the average recommended recall time relative to competing methods.

Keywords: Dirichlet process prior; dynamic treatment regimes; observational data; periodontal disease; practice-based setting; precision medicine; sequential optimization.

Fig. 1

The proportion of sites with unhealthy PPD (pocket depth exceeding 3mm) or missing tooth, denoted by PMU, over time, for 10 randomly chosen subjects.

Fig. 2

Density plots of actual time between visits (δ months) for each level of recommended recall interval (A months).

Fig. 3

Illustration of how to optimize a sequence of recall intervals under a given policy using the proposed method. In this hypothetical example, the risk score R_t is a linear combination of the current disease statue (Y_t) and a single covariate (X), and the two actions are to return in 3 or 9 months.

Fig. 4

Estimated optimal feature weights α_opt for the simulation study. The boxplots for the Gaussian and DPM methods show the estimated α_opt over the 100 simulated datasets; the solid points represent α_opt for the Oracle policy. The risk score is a linear combination the two baseline covariates (X₁ and X₂), noncompliance (“Non-comp”; log(| δ_t−1 − A_t−1 | +1)), and disease status (“Cur Y”; Y_t−1):

Fig. 5

Value and average recommended recall time for the 100 datasets for the policy (via α_opt) estimated using the Gaussian model or DPM model for data generated from a two-component mixture model with n = 1,000 subjects using reduction utility function. The value and average recommended recall time of the policy are evaluated using Monte Carlo samples under the true model used to generate the data and the estimated Gaussian model or DPM model.

Fig. 6

Posterior distribution (5th, 25th, 50th, 75th, 95th percentiles) of optimal feature weights α_opt for the HP data analysis. The features are age (standardized), diabetes (with diabetes=1 and without diabetes=0) non-compliance (log( | δ_t−1 − A_t−1 | +1)), and current response (Y_t−1).