Context-Sensitive Spelling Correction of Clinical Text via Conditional Independence

Abstract

Spelling correction is a particularly important problem in clinical natural language processing because of the abundant occurrence of misspellings in medical records. However, the scarcity of labeled datasets in a clinical context makes it hard to build a machine learning system for such clinical spelling correction. In this work, we present a probabilistic model of correcting misspellings based on a simple conditional independence assumption, which leads to a modular decomposition into a language model and a corruption model. With a deep character-level language model trained on a large clinical corpus, and a simple edit-based corruption model, we can build a spelling correction model with small or no real data. Experimental results show that our model significantly outperforms baselines on two healthcare spelling correction datasets.

Figure 4:

Beam search decoding results for several examples of the CSpell test set. For each example, we display the top 10 beam candidates. The column next to the candidate (Score) shows the final beam score for each candidate.

Figure 1:

Graphical model of our conditional independence model. The context and the typo are observed and the correct word is unobserved.

Figure 2:

Beam search of CIM on the Example 1 at time step $t=3$. The beam candidates are ranked by the sum of the language model score (LM) and the corruption model score (ED). The hyper-parameters of the corruption model are $C=5.0$ and $n=1$. The beam width is chosen to $B=1$ for clear visualization.

Figure 3:

Beam search decoding examples. For each example, we display the top 10 beam candidates. The column next to the candidate (Score) shows the final beam score for each candidate.