Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Dec 15:10:100430.
doi: 10.1016/j.mlwa.2022.100430. Epub 2022 Nov 2.

A fully automatic framework for evaluating cosmetic results of breast conserving therapy

Affiliations

A fully automatic framework for evaluating cosmetic results of breast conserving therapy

Chenqi Guo et al. Mach Learn Appl. .

Abstract

The breast cosmetic outcome after breast conserving therapy is essential for evaluating breast treatment and determining patient's remedy selection. This prompts the need of objective and efficient methods for breast cosmesis evaluations. However, current evaluation methods rely on ratings from a small group of physicians or semi-automated pipelines, making the processes time-consuming and their results inconsistent. To solve the problem, in this study, we proposed: 1. a fully-automatic Machine Learning Breast Cosmetic evaluation algorithm leveraging the state-of-the-art Deep Learning algorithms for breast detection and contour annotation, 2. a novel set of Breast Cosmesis features, 3. a new Breast Cosmetic dataset consisting 3k+ images from three clinical trials with human annotations on both breast components and their cosmesis scores. We show our fully-automatic framework can achieve comparable performance to state-of-the-art without the need of human inputs, leading to a more objective, low-cost and scalable solution for breast cosmetic evaluation in breast cancer treatment.

Keywords: Breast Cosmesis scores; Breast cancer; Breast conserving therapy; Breast detection; Machine learning; Predictive model.

PubMed Disclaimer

Figures

Fig. 10.
Fig. 10.
Confusion matrices of our framework with 3 machine learning algorithms estimating 6 scores (1 global score and 5 categorical scores).
Fig. 1.
Fig. 1.
Overview of the proposed Breast Cosmesis evaluation system.
Fig. 2.
Fig. 2.
Score distributions for (a) individual categories and (b) global score.
Fig. 3.
Fig. 3.
Landmark annotations used in our study. There are 30 landmarks on each breast, 12 landmarks on each areola, and 6 landmarks on each nipple.
Fig. 4.
Fig. 4.
Image preprocessing to mask out tattoos or markers on breast skin. (a) Segmentation from landmarks. (b) A channel after thresholding. (c) A channel after correction. (d) Resulting RGB image after this preprocessing.
Fig. 5.
Fig. 5.
Here the areola is highlighted using the black contour. (a) This diagram illustrates the definitions of the width and height of areola, denoted as ωa and ha, respectively, when the areola exists. The black cross denotes the center of areola. If the areola does not exist but the nipple exists, compute these values from the nipple instead. (b) For cases where both the areola and nipple exist, this diagram illustrates the definitions of the nipple location relative to the areola, denoted as Dut, Dub, Dhl, and Dhr. The yellow cross denotes the center of nipple. If only the areola exists, then these relative locations are set to be semi-vertical and semi-horizontal axis length of the existing areola. If only the nipple exists, then these relative locations are set to the semi-vertical and semi-horizontal axis length of the existing nipple. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 6.
Fig. 6.
Example of breast mean shape warping and coordinate systems. To calculate EMD-XYLAB features, each breast of the patient needs to be warped to the mean breast shape of the same side. (a) A patient’s image before warping. (b) The same image after warping and masking. (c) mean breast shape used for warping and left (right) coordinate system for pixel location (x, y) used in the EMD-XYLAB features.
Fig. 7.
Fig. 7.
Color distribution differences between treated and untreated breast of a patient. The left breast of this patient is treated and the right is untreated. (a) Patient’s image in RGB color space. (b) 2D histogram in AB space for right breast. (c) 2D histogram in AB space for left breast. Comparing to the untreated breast, the pixels of treated breast are more linearly correlated between A and B channels. The color contour shows the breast pixels with corresponding color range in the histogram. Regions smaller than contour width are discarded for visual clarity. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)
Fig. 8.
Fig. 8.
Quantitative results for landmarks detection using within landmark evaluation. (a) Visualization of the mean and standard error of the normalized Euclidean distance for each landmark across images. The radius of each circle represents the average Euclidean distance (across images) between the ground truth and the detected breast landmark after equal arc sampling. The thickness of each circle equals 2 times the standard error. The half-inter-pit distance is defined as half of the distance between the starting landmarks of left breast and right breast, and both are defined at the armpit of each side. (b) within landmarks result of detection error for left breast, (c) right breast, (d) areolas and (e) nipples.
Fig. 9.
Fig. 9.
Bar plot of full pipeline performance with 10-fold Cross Validation, for each individual categorical score and global score. Bar heights represent average accuracy with error bar denoting standard error.

Similar articles

Cited by

References

    1. Althobaiti MM, Ashour AA, Alhindi NA, Althobaiti A, Mansour RF, Gupta D, et al. (2022). Deep transfer learning-based breast cancer detection and classification model using photoacoustic multimodal images. BioMed Research International. - PMC - PubMed
    1. Arjovsky M, Chintala S, & Bottou L (2017). Wasserstein GAN. arXiv:1701.07875.
    1. Brouwers P, Werkhoven E, Bartelink H, & Fourquet A (2016). Factors associated with patient-reported cosmetic outcome in the Young boost breast trial. Radiotherapy and Oncology. - PubMed
    1. Cardoso JS, & Cardoso MJ (2007). Towards an intelligent medical system for the aesthetic evaluation of breast cancer conservative treatment. Artificial Intelligence in Medicine. - PubMed
    1. Cardoso MJ, Cardoso JS, Wild T, Krois W, & Fitzal F (2009). Comparing two objective methods for the aesthetic evaluation of breast cancer conservative treatment. Breast Cancer Research and Treatment. - PubMed

LinkOut - more resources