Selecting treatment strategies with dynamic limited-memory influence diagrams

Abstract

Objective: The development of dynamic limited-memory influence diagrams as a framework for representing factorized infinite-horizon partially observable Markov decision processes (POMDPs), the introduction of algorithms for their (approximate) solution, and the application to a dynamic decision problem in clinical oncology.

Materials and methods: A dynamic limited-memory influence diagram for high-grade carcinoid tumor pathophysiology was developed in collaboration with an expert physician. Three algorithms, known as single policy updating, single rule updating, and simulated annealing have been examined for approximating the optimal treatment strategy from a space of 10(19) possible strategies.

Results: Single policy updating proved intractable for finding a treatment strategy for carcinoid tumors. Single rule updating and simulated annealing both found the treatment strategy that is applied by physicians in practice.

Conclusions: Dynamic limited-memory influence diagrams are a suitable framework for the representation of factorized infinite-horizon POMDPs, and the developed algorithms find acceptable solutions under the assumption of limited memory about past observations. The framework allows for finding reasonable treatment strategies for complex dynamic decision problems in medicine.