Centered Partition Processes: Informative Priors for Clustering (with Discussion)

Abstract

There is a very rich literature proposing Bayesian approaches for clustering starting with a prior probability distribution on partitions. Most approaches assume exchangeability, leading to simple representations in terms of Exchangeable Partition Probability Functions (EPPF). Gibbs-type priors encompass a broad class of such cases, including Dirichlet and Pitman-Yor processes. Even though there have been some proposals to relax the exchangeability assumption, allowing covariate-dependence and partial exchangeability, limited consideration has been given on how to include concrete prior knowledge on the partition. For example, we are motivated by an epidemiological application, in which we wish to cluster birth defects into groups and we have prior knowledge of an initial clustering provided by experts. As a general approach for including such prior knowledge, we propose a Centered Partition (CP) process that modifies the EPPF to favor partitions close to an initial one. Some properties of the CP prior are described, a general algorithm for posterior computation is developed, and we illustrate the methodology through simulation examples and an application to the motivating epidemiology study of birth defects.

Keywords: Bayesian clustering; Bayesian nonparametrics; Dirichlet Process; centered process; exchangeable probability partition function; mixture model; product partition model.

Figure 1:

Hasse diagram for the lattice of set partitions of 4 elements. A line is drawn when two partitions have a covering relation. For example {1} {2, 3, 4} is connected with 3 partitions obtained by splitting the block {2, 3, 4} in every possible way, and with partition 1, obtained by merging the two clusters.

Figure 2:

Prior probabilities of the 52 set partitions of N = 5 elements for the CP process with uniform base EPPF. In each graph the CP process is centered on a different partition c₀ highlighted in blue. The cumulative probabilities across different values of the penalization parameter ψ are joined to form the curves, while the probability of a given partition corresponds to the area between the curves.

Figure 3:

Prior probabilities of the 52 set partitions of N = 5 elements for the CP process with Dirichlet process of α = 1 base EPPF. In each graph the CP process is centered on a different partition c₀ highlighted in blue. The cumulative probabilities across different values of the penalization parameter ψ are joined to form the curves, while the probability of a given partition corresponds to the area between the curves.

Figure 4:

Prior probabilities of the 52 set partitions of N = 5 elements for the CP process with Pitman-Yor process base EPPF with σ = 0.25 and α ≈ −0.004, such that the expected number of clusters equal to log(5) ≈ 1.6. In each graph the CP process is centered on a different partition c₀ highlighted in blue. The cumulative probabilities across different values of the penalization parameter ψ are joined to form the curves, while the probability of a given partition corresponds to the area between the curves.

Figure 5:

Prior probabilities of the 52 set partitions of N = 5 elements for the CP process with Pitman-Yor process base EPPF with σ = 0.75 and α ≈ −0.691, such that the expected number of clusters equal to log(5) ≈ 1.6. In each graph the CP process is centered on a different partition c₀ highlighted in blue. The cumulative probabilities across different values of the penalization parameter ψ are joined to form the curves, while the probability of a given partition corresponds to the area between the curves.

Figure 6:

Illustration of results from the local search algorithm based on the Hasse diagram of Π₄ starting from c₀ = {1}{2,3,4}. Partitions are colored according the exploration order following a dark-light gradient. Notice that after 3 iterations the space is entirely explored.

Figure 7:

Estimate of the cumulative prior probabilities assigned to different distances from c₀ for N = 12 and c₀ with configuration {3, 3, 3, 3}, under the CP process with uniform prior on the left and Dirichlet Process on the right. Black dots correspond to the base prior with no penalization, while dots from bottom-to-top correspond to increasing values of $\psi \in \left\{5,10,15,20\right\}$. Tables report the minimum distance values such that F (δ) ≥ 0.9.

Figure 8:

Results from grouped logistic regressions with DP(α = 1) prior and CP process prior with DP(α = 1) base EPPF for ψ = 15, 17, centered on the true partition. Heatmaps on the left side show the posterior similarity matrix. On the right side, boxplots show the distribution of deviations from the maximum likelihood baseline coefficients and posterior mean estimates for each defect i = 1,…, 12.

Figure 9:

Results from grouped logistic regression using CP process prior with DP(α = 1) base EPPF for ψ = 15 centered on partition ${\mathit{\text{c}}}_0^{\prime }=\left\{1,5,9\right\}\left\{2,6,10\right\}\left\{3,7,11\right\}\left\{4,8,12\right\}$ which has distance 3.16 from the true one. Heatmaps on the left side show the posterior similarity matrix. On the right side, boxplots show the distribution of deviations from the maximum likelihood baseline coefficients and posterior mean estimates for each defect i = 1,…, 12.

Figure 10:

Posterior allocation matrices obtained using the CP process with a DP (α = 1) prior for different values of ψ ∈ {0, 40, 80, 120}. On the y-axis labels are colored according base grouping information c₀, with dots on the diagonal highlighting differences between c₀ and the estimated partition ĉ.

Figure 11:

Comparison of significant odds ratio under ψ ∈ {0, 40, 80, 120, ∞} for some exposure factors and 4 selected heart defects in 4 different groups under c₀. Dots are in correspondence of significant mean posterior log-odds ratios (log-OR) at 95% with red encoding risk factors (log-OR > 0) and green protective factors (log-OR < 0).