Interlaboratory performance of a microarray-based gene expression test to determine tissue of origin in poorly differentiated and undifferentiated cancers

Abstract

Clinical workup of metastatic malignancies of unknown origin is often arduous and expensive and is reported to be unsuccessful in 30 to 60% of cases. Accurate classification of uncertain primary cancers may improve with microarray-based gene expression testing. We evaluated the analytical performance characteristics of the Pathwork tissue of origin test, which uses expression signals from 1668 probe sets in a gene expression microarray, to quantify the similarity of tumor specimens to 15 known tissues of origin. Sixty archived tissue specimens from poorly and undifferentiated tumors (metastatic and primary) were analyzed at four laboratories representing a wide range of preanalytical conditions (eg, personnel, reagents, instrumentation, and protocols). Cross-laboratory comparisons showed highly reproducible results between laboratories, with correlation coefficients between 0.95 to 0.97 for measurements of similarity scores, and an average 93.8% overall concordance between laboratories in terms of final tissue calls. Bland-Altman plots (mean coefficients of reproducibility of 32.48+/-3.97) and kappa statistics (kappa >0.86) also indicated a high level of agreement between laboratories. We conclude that the Pathwork tissue of origin test is a robust assay that produces consistent results in diverse laboratory conditions reflecting the preanalytical variations found in the everyday clinical practice of molecular diagnostics laboratories.

Figure 1

Study design. Sixty frozen tissue specimens from metastatic and primary tumors were assayed at four sites, three academic laboratories and one commercial laboratory, to adequately represent the variability found during routine clinical testing in molecular diagnostics laboratories. Microarray data files for samples passing quality control criteria only were analyzed by Pathwork Diagnostics, and reports were evaluated by pathologists (blinded). Data were compared between laboratories to determine reproducibility of the assay.

Figure 2

Example of Pathwork TOO report. Results shown are for study specimen 06-0360 from site 1. A SS ≥30 is the cutoff value to indicate a positive call, and an SS <5 confirms a negative call for each TOO.

Figure 3

Comparison of reproducibility of gene expression measurements and SS between test sites, as measured by the Pearson correlation coefficient. Data for 1668 genes normalized with the MAS5 and Pathwork TOO test algorithms show that correlation coefficients are significantly higher for the Pathwork SE value (Wilcoxon one-sided paired test, P = 0.01563) than those achieved with MAS5 normalization. Reproducibility across laboratories is further improved by the calculation of the SS from the SE data.

Figure 4

Bland-Altman plots graph the difference between the SS for the true TOO between sites (y axis) versus the average of the SS (x axis) of any given pair of sites. The mean and ± 1.96 SD intervals are shown by dotted and plain horizontal lines, respectively. The coefficients of reproducibility (CR) and percentage of outliers are indicated for each comparison. The number of samples included in each comparison were n = 51 (A), n = 54 (B), n = 51 (C), n = 53 (D), n = 51 (E), and n = 50 (F).