Inter-rater agreement in assessing occupational exposure in a case-control study

Abstract

The identification of occupational carcinogens in the workplace is a major concern of epidemiologists. A novel case-control approach has been developed which includes as a key component the assessment of a subject's occupational exposure history by a two stage process. Firstly, the subject is interviewed to obtain a detailed lifetime job history. Then a team of chemists and hygienists, hired and trained to do this work on a full time basis, translates each job into a list of potential occupational exposures. The present study investigated the inter-rater agreement in this type of retrospective exposure assessment. Six trials were carried out over a four year period with different raters and different sets of job files. Some trials involved only internal raters from the "in house" group whereas others involved comparisons between the internal raters and other external raters who had expertise in certain industries. In assessing exposure as simply present or absent, two summary indices of agreement were used: per cent with perfect agreement and Cohen's kappa. In most of the trials the per cent with perfect agreement among raters ranged from 95% to 98%, with kappa ranging from 0.5 to 0.7. The kappas were slightly higher for internal-internal comparisons than for internal-external ones. These results indicate a relatively high degree of inter-rater agreement and lend credibility to the validity of this type of retrospective exposure assessment.