Evaluating the Practicality of Causal Inference From Non-randomized Observational Data in Small-Scale Clinical Settings: A Study on the Effects of Ninjin'yoeito

Abstract

Objective The primary goal of this study was to demonstrate the practical application of causal inference using non-randomized observational data, adapting this approach to smaller populations, such as those in hospitals or community healthcare. This adaptation seeks a more effective and practical research method than randomized controlled trials (RCTs), with the goal of revealing novel insights unexplored by traditional research and enhancing understanding within the realm of causal inference. Methods This study evaluated the effects of Ninjin'yoeito (NYT), a traditional Japanese Kampo medicine, on Overactive Bladder Symptom Score (OABSS) and the frailty scores. Employing new statistical methods, this study sought to illustrate the efficacy of estimating causal relationships from non-randomized data in a clinical setting. The database included 985 women aged 65-90 years who visited a clinic between November 2016 and November 2022. By utilizing various statistical techniques, including regression analysis, inverse probability of treatment weighting (IPTW), instrumental variable (IV), and difference-in-differences (DiD) analysis, this study aimed to provide insights beyond traditional methods, attempting to bridge the gap between theory and practice in causal inference. Results After applying propensity score matching, the NYT treatment group (220 participants) and non-treatment group (182 participants) were each adjusted to two groups of 159 individuals. NYT significantly improved OABSS and frailty scores. IPTW analysis highlighted that on average, the NYT treatment group showed an improvement of 0.8671 points in OABSS and 0.1339 points in the frailty scores, surpassing the non-treatment group (p<0.05). IV analysis indicated that NYT treatment is predicted to increase ΔOABSS by an average of approximately 4.86 points, highlighting its significant positive impact on OABSS improvement. The DiD analysis showed that the NYT treatment group demonstrated an average improvement of 0.5457 points in OABSS, which was significantly higher than that of the control group. The adjusted R² value for the model is 0.025. Conclusion This study successfully implemented a practical application of causal inference using non-randomized observational data in a relatively small population. NYT showed a significant improvement in OABSS and vulnerability, and this result was confirmed using a new statistical method. The relatively low adjusted R² of the model suggests the existence of other unmeasured variables that influence OABSS and vulnerability improvement. In particular, the use of diverse statistical techniques, including IPTW, IV, and DiD analysis, is an important step toward revealing the effectiveness of inferring causal relationships from non-randomized data and narrowing the gap between theory and practice. This study provides a valid and practical alternative to RCTs and reveals new insights that have not been explored in traditional research.

Keywords: causal inference; difference-in-differences; instrumental variable; inverse probability of treatment weighting; ninjin'yoeito; non-randomized observational data; overactive bladder symptom score; propensity score matching.

Figure 1. Directed Acyclic Graph Illustrating the Causal Relationship for ΔOABSS

B: Instrumental Variable (Behavioral Variable/Intervention Variable) Y: Outcome Variable (Y Variable) C1: Covariate 1 (Age) C2: Covariate 2 (BMI) C3: Covariate 3 (Grip Strength) ΔOABSS: The improvement rate in OABSS, calculated as the difference between post-treatment and pre-treatment OABSS, OABSS: Overactive Bladder Symptom Score, BMI: Body Mass Index

Figure 2. Directed Acyclic Graph Illustrating Causal Relationships Based on IPTW Analysis Covariate

I: IPTW (Inverse Probability of Treatment Weighting) T: Treatment Group (NYT Treatment) C: Control Group O: ΔOABSS F: ΔFRAILTY ΔOABSS: The improvement rate in OABSS, calculated as the difference between post-treatment and pre-treatment OABSS, OABSS: Overactive Bladder Symptom Score, ΔFRAILTY: The improvement rate in FRAILTY, calculated as the difference between post-treatment and pre-treatment FRAILTY, NYT: Ninjin'yoeito

Figure 3. Regression Coefficients and Their Statistical Significance

Vertical Axis (Y-axis): Regression Coefficients, Unit: Points Horizontal Axis (X-axis): Treatment and Medication Groups (NYT Treatment, Blue; Antimuscarinic: Yellow; β3 Adrenoceptor Agonist, Green; Combining Drug, Red), NYT: Ninjin'yoeito Numbers above bars indicate the p-values, representing the statistical significance of the regression coefficients for each treatment group

Figure 4. Assessment of Multicollinearity Using VIF

Y-Axis Label: VIF (No units) X-Axis Label: Predictors (Coded) a: Age, b: BMI, c: Grip Strength, d: NYT, e: Antimuscarinic, f: β3 Adrenoceptor Agonist, g: Combining Drug NYT: Ninjin'yoeito, VIF: Variance Inflation Factor The red dashed line at the top represents a VIF threshold, above which multicollinearity might distort the regression analysis

Figure 5. Weighted Average Improvement Rates for OABSS and FRAILTY

a: ΔOABSS, b: ΔFRAILTY Vertical axis: Weighted average improvement rate, unitless Horizontal axis: 0 represents the non-NYT group (non-treatment, blue bar), 1 represents the NYT treatment group (with treatment, orange bar) The numerical values displayed above each bar represent the specific improvement rates for each group ΔOABSS: The improvement rate in OABSS, calculated as the difference between post-treatment and pre-treatment OABSS, OABSS: Overactive Bladder Symptom Score, ΔFRAILTY: The improvement rate in FRAILTY, calculated as the difference between post-treatment and pre-treatment FRAILTY, NYT: Ninjin'yoeito