Demystifying EQA statistics and reports

Abstract

Reports act as an important feedback tool in External Quality Assessment (EQA). Their main role is to score laboratories for their performance in an EQA round. The most common scores that apply to quantitative data are Q- and Z-scores. To calculate these scores, EQA providers need to have an assigned value and standard deviation for the sample. Both assigned values and standard deviations can be derived chemically or statistically. When derived statistically, different anomalies against the normal distribution of the data have to be handled. Various procedures for evaluating laboratories are able to handle these anomalies. Formal tests and graphical representation techniques are discussed and suggestions are given to help choosing between the different evaluations techniques. In order to obtain reliable estimates for calculating performance scores, a satisfactory number of data is needed. There is no general agreement about the minimal number that is needed. A solution for very small numbers is proposed by changing the limits of evaluation. Apart from analyte- and sample-specific laboratory evaluation, supplementary information can be obtained by combining results for different analytes and samples. Various techniques are overviewed. It is shown that combining results leads to supplementary information, not only for quantitative, but also for qualitative and semi-quantitative analytes.

Keywords: Q-score; Z-score; external quality assessment; statistics.

Figure 1

Box plot for evaluating an individual result
The black rectangle reaches from the 25^th to the 75^th percentile; the vertical line inside the black rectangle is the median. The horizontal lines to the left and the right of the box plot (‘whiskers’) reach to the furthest values that are closer than 1.5 times the interquartile range from the 25^th and 75^th percentile. The blue rectangle reflects the limits for Q-scores, the green rectangle the Z-score limits. The bold vertical line represents the individual result of a laboratory. It has a good performance according to both limits.

Figure 2

Histogram for the same individual result as for Figure 1.
The highest bars represent all the data, the light grey bars represent the data of the peer group of the laboratory under interest. The bold vertical line represents the individual result of a laboratory. The blue rectangle reflects the limits for Q-scores, the green rectangle the Z-score limits.

Figure 3

A Youden plot based on reported values for one specific method
The thick black line represents a 99% robust confidence region, the thin grey line a 99% confidence region based on classical statistics of average and variance-covariance matrix. Points in zone A have a negative bias, while points in zone C have a positive bias. Points in zone B exhibit high intra-laboratory variability.