Evaluation of the use of induction in the development of a medical expert system

Abstract

In an earlier study, two medical expert systems for diagnosing thyroid disorders, developed by the application of induction on a sample of previously diagnosed cases and on expert-generated rules, diagnosed a set of test cases better than an expert system developed by the more traditional method of collaboration between a knowledge engineer and an expert. In this paper, an alternative measure of the accuracy of diagnosis of each system is used to evaluate the systems. Diagnoses for every distinct case represented by a combination of indicating factors are compared with diagnoses that the expert made. The induced systems provide diagnoses for many more distinct cases, but a much higher proportion of these diagnoses are incorrect. It is argued that generalizing to unseen cases is an inappropriate use of induction algorithms. The systematic development of a decision table is a more appropriate method for devising a medical expert system.