Current practices for QSP model assessment: an IQ consortium survey

Abstract

Quantitative Systems Pharmacology (QSP) modeling is increasingly applied in the pharmaceutical industry to influence decision making across a wide range of stages from early discovery to clinical development to post-marketing activities. Development of standards for how these models are constructed, assessed, and communicated is of active interest to the modeling community and regulators but is complicated by the wide variability in the structures and intended uses of the underlying models and the diverse expertise of QSP modelers. With this in mind, the IQ Consortium conducted a survey across the pharmaceutical/biotech industry to understand current practices for QSP modeling. This article presents the survey results and provides insights into current practices and methods used by QSP practitioners based on model type and the intended use at various stages of drug development. The survey also highlights key areas for future development including better integration with statistical methods, standardization of approaches towards virtual populations, and increased use of QSP models for late-stage clinical development and regulatory submissions.

Keywords: Decision Making; Drug Industry; Modeling and Simulation; Network Pharmacology.

Fig. 1

Responses to selected survey questions in the Demographics section. (a) UpSet [31, 32] plot showing the five largest categories of respondents’ educational background (set size) and their combinations (interaction size). (b) Circle graph showing work experience of survey takers. (c) Stacked bar graph illustrating where QSP simulation results are presented by survey responders. (d) Venn diagram illustrating experience of survey takers with different types of models

Fig. 2

Responses to selected survey questions in the Biology section. Each plot represents the distribution of answers with respect to the indicated frequencies (rarely, not often, sometimes, often, usually) and is normalized to the total number of responses. (a) Q14: Who are the stakeholders in defining your QSP question? (b) Q15: What types of data are incorporated into your QSP models? (c) Q17: What criteria do you use to include/exclude data? (d) Q21: What is the process for keeping documentation during model development?

Fig. 3

Deviations from selected responses in Implementation section based on subgroup demographic differences. Ticks on the left side of the plot define responder subgroups with number of total responders belonging to the subgroup in parentheses. Bars represent the difference between average response and subgroup responses. (a) Q28: Do you use QSP models that are developed externally? Answer: Yes (68% of all responders). Subgroup: Phase of drug development. (b) Q29: In order to use a QSP model developed externally, how often do you require markup language? Answer: Always (25% of all responders). Subgroup: Modeling experience. (c) Q37: In order to assess parameter sensitivity, how often do you use global sensitivity analysis on all parameters? Answer: Never (30% of all responders). Subgroup: Industry

Fig. 4

Responses to selected survey questions in the Simulation section, filtered by respondent preclinical or clinical focus. (a) Q42: How do you generate prediction intervals (i.e. range of likely values for a model outcome)? (b) Q47: When you publish a QSP model do you publish all the observations that informed the model (e.g. proprietary pre-clinical observations that informed design decisions)?

Fig. 5

Selected findings for the Robustness part of the survey. (a) Q51: For each stage, what methods do you use to ensure your model is robust? (b) Q52: How often do you evaluate model robustness using these criteria? (c) Q53: Do you focus on the model being able to describe the central tendency of the dynamics in question or the variability as well? (d) Q56: How do you assess parameter uncertainty? (e) Q57: How often have you repurposed a model outside its initial area of focus? Cent. tend. = central tendency, var. = variability, MOA = mechanism of action. ‘% Responses’ indicates the % of responses for that particular question, since not all questions were answered by all respondents