Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2010 Apr 12;1(1):6.
doi: 10.1186/2041-1480-1-6.

De-identifying Swedish clinical text - refinement of a gold standard and experiments with Conditional random fields

Hercules Dalianis  1 , Sumithra Velupillai
Affiliations

De-identifying Swedish clinical text - refinement of a gold standard and experiments with Conditional random fields

Hercules Dalianis et al. J Biomed Semantics. .

Abstract

Background: In order to perform research on the information contained in Electronic Patient Records (EPRs), access to the data itself is needed. This is often very difficult due to confidentiality regulations. The data sets need to be fully de-identified before they can be distributed to researchers. De-identification is a difficult task where the definitions of annotation classes are not self-evident.

Results: We present work on the creation of two refined variants of a manually annotated Gold standard for de-identification, one created automatically, and one created through discussions among the annotators. The data is a subset from the Stockholm EPR Corpus, a data set available within our research group. These are used for the training and evaluation of an automatic system based on the Conditional Random Fields algorithm. Evaluating with four-fold cross-validation on sets of around 4-6 000 annotation instances, we obtained very promising results for both Gold Standards: F-score around 0.80 for a number of experiments, with higher results for certain annotation classes. Moreover, 49 false positives that were verified true positives were found by the system but missed by the annotators.

Conclusions: Our intention is to make this Gold standard, The Stockholm EPR PHI Corpus, available to other research groups in the future. Despite being slightly more time-consuming we believe the manual consensus gold standard is the most valuable for further research. We also propose a set of annotation classes to be used for similar de-identification tasks.

PubMed Disclaimer

References

    1. Dalianis H, Hassel M, Velupillai S. The Stockholm EPR Corpus - Characteristics and Some Initial Findings. Proceedings of the 14th International Symposium for Health Information Management Research, Kalmar, Sweden. 2009. pp. 243–249.
    1. Artstein R, Poesio M. Inter-coder agreement for computational linguistics. Computational Linguistics. 2008;34(4):555–596. doi: 10.1162/coli.07-034-R2. - DOI
    1. HIPAA. Health Insurance Portability and Accountability (HIPAA), Privacy Rule and Public Health Guidance. From CD Cand the U.S. Department of Health and Human Services. 2003. http://www.cdc.gov/mmwr/preview/mmwrhtml/m2e411a1.htm (accessed October 11, 2009)
    1. Kokkinakis D, Thurin A. Proceedings of the 16th Nordic Conference of Computational Linguistics NODALIDA-2007. University of Tartu, Tartu; 2007. Identification of Entity References in Hospital Discharge Letters.
    1. Grishman R, Sundheim B. Message Understanding Conference-6: A Brief History. Proceedings of the 16th International Conference on Computational Linguistics (COLING), Copenhagen, Denmark. 1996. pp. 466–471.

LinkOut - more resources