Results of a Geant4 benchmarking study for bio-medical applications, performed with the G4-Med system

Abstract

Background: Geant4, a Monte Carlo Simulation Toolkit extensively used in bio-medical physics, is in continuous evolution to include newest research findings to improve its accuracy and to respond to the evolving needs of a very diverse user community. In 2014, the G4-Med benchmarking system was born from the effort of the Geant4 Medical Simulation Benchmarking Group, to benchmark and monitor the evolution of Geant4 for medical physics applications. The G4-Med system was first described in our Medical Physics Special Report published in 2021. Results of the tests were reported for Geant4 10.5.

Purpose: In this work, we describe the evolution of the G4-Med benchmarking system.

Methods: The G4-Med benchmarking suite currently includes 23 tests, which benchmark Geant4 from the calculation of basic physical quantities to the simulation of more clinically relevant set-ups. New tests concern the benchmarking of Geant4-DNA physics and chemistry components for regression testing purposes, dosimetry for brachytherapy with a ${}^{125}I$ source, dosimetry for external x-ray and electron FLASH radiotherapy, experimental microdosimetry for proton therapy, and in vivo PET for carbon and oxygen beams. Regression testing has been performed between Geant4 10.5 and 11.1. Finally, a simple Geant4 simulation has been developed and used to compare Geant4 EM physics constructors and physics lists in terms of execution times.

Results: In summary, our EM tests show that the parameters of the multiple scattering in the Geant4 EM constructor G4EmStandardPhysics_option3 in Geant4 11.1, while improving the modeling of the electron backscattering in high atomic number targets, are not adequate for dosimetry for clinical x-ray and electron beams. Therefore, these parameters have been reverted back to those of Geant4 10.5 in Geant4 11.2.1. The x-ray radiotherapy test shows significant differences in the modeling of the bremsstrahlung process, especially between G4EmPenelopePhysics and the other constructors under study (G4EmLivermorePhysics, G4EmStandardPhysics_option3, and G4EmStandardPhysics_option4). These differences will be studied in an in-depth investigation within our Group. Improvement in Geant4 11.1 has been observed for the modeling of the proton and carbon ion Bragg peak with energies of clinical interest, thanks to the adoption of ICRU90 to calculate the low energy proton stopping powers in water and of the Linhard-Sorensen ion model, available in Geant4 since version 11.0. Nuclear fragmentation tests of interest for carbon ion therapy show differences between Geant4 10.5 and 11.1 in terms of fragment yields. In particular, a higher production of boron fragments is observed with Geant4 11.1, leading to a better agreement with reference data for this fragment.

Conclusions: Based on the overall results of our tests, we recommend to use G4EmStandardPhysics_option4 as EM constructor and QGSP_BIC_HP with G4EmStandardPhysics_option4, for hadrontherapy applications. The Geant4-DNA physics lists report differences in modeling electron interactions in water, however, the tests have a pure regression testing purpose so no recommendation can be formulated.

Keywords: Geant4; benchmarking; bio‐medical physics.

FIGURE 1

Number of scientific publications in bio‐medical physics found in the PubMed Central® archive, searching the general purpose Monte Carlo codes Geant4,, , and FLUKA,, , and MCNP and EGSnrc, in keywords, titles and citations of papers recorded in the PubMed archive. The figure contains data (up to 2018) published in Mancini‐Terracciano et al., Copyright Elsevier (2019).

FIGURE 2

Left: Radial dose rate distribution with respect to the distance from the source, calculated with the Brachytherapy test in the case of an Oncura^TM 6711 source. When not visible, the error bars are within the symbols. The dashed line represents the inverse square law distribution as is the case for a non interacting isotropic point source. Right: Ratio of the simulation and experimental data (Dolan et al. ⁴⁴ ). The shadowed area represents an agreement within 5% with the reference data. Geant4 11.1 is used. To note, the two plots have different scales on the x‐axis.

FIGURE 3

Schematic view of the simulation set‐up of the MV x‐ray radiotherapy test, which is released in the Geant4 medical_linac advanced example and based on the geometry of the Saturne 43 General Electric linear accelerator. The 10 × 10‐cm² field size is defined at 100 cm from the source, 10 cm below the phantom surface. The diagram is not to scale.

FIGURE 4

Results of the MV x‐ray radiotherapy test obtained with Geant4.11.1. Top left: longitudinal dose distribution in the water phantom. The curves are normalized to the dose value at a depth of 10 cm. Top right: transverse dose distribution at 10‐cm depth in the water phantom. The curves are normalized at the center; the uncertainties of the simulation results are within symbols. Bottom: ratios of simulated and experimental data for the longitudinal (left) and transverse (right) profile. The shadowed area represents an agreement within 5% with the reference data. Experimental data from Caccia et al.

FIGURE 5

Results of the $\gamma$ index test for all the tested EM constructors (Opt3, Opt4, Livermore, and Penelope). Top row: longitudinal profiles. Bottom row: transverse profiles. Dotted line signifies a $\gamma$ index of 1.

FIGURE 6

Absolute dose profiles in the water phantom, obtained with $2\times {10}^{11}$ histories and the four EM constructors. Left: longitudinal profile. Right: transverse profile at 10 cm of depth. Results obtained with Geant4.11.1.

FIGURE 7

Transverse dose profiles simulated with the full EM constructors Opt4, Penelope, and the same constructors using a different bremsstrahlung model: Opt4 with G4PenelopeBremsstrahlungModel, and Penelope with G4LivermoreBremsstrahlungModel. Results obtained with ${10}^{11}$ histories.

FIGURE 8

Schematic view of the simulation set‐up of the electron FLASH radiotherapy test. The different components are indicated in the figure: the position of the electron source, the plastic shaping applicator, and the water phantom where the dose distributions are computed.

FIGURE 9

Results of the electron FLASH radiotherapy test. Top: longitudinal dose distribution in the water phantom. When not visible, the error bars are within the symbols. Bottom: transverse dose distribution at 13.5 mm depth in the water phantom. The curves are shown in absolute dose normalized to the number of primary particles (left) and normalized to the maximum dose calculated with Opt4 (at 17 mm depth in the water phantom for the longitudinal distribution) (right). The plots in the last column show the ratio of the Geant4 simulations and reference data reported in DiMartino et al. The shadowed area represents an agreement within 5%. Results obtained with Geant4 11.1.

FIGURE 10

$\gamma$ index test results for the transversal (top) and longitudinal (bottom) dose profiles obtained with Livermore, Penelope, Opt3, and Opt4.

FIGURE 11

Results of the electron FLASH radiotherapy test, obtained with Geant4 11.2.1 and Opt3 EM constructor. The RangeFactor is 0.04 and the step limitation option is UseDistanceToBoundary (only two parameters that have been changed between Geant4 11.1 and 11.2.1 in Opt3). The shadowed area represents an agreement within 5%. When not visible, the error bars are within the symbols. Reference data from DiMartino et al.

FIGURE 12

Exemplary electron backscattering coefficients calculated for Al, Ta, and U targets, for different angles of incidence, compared against the Sandia Lab experimental data., The shadowed area represents a 5% agreement. When not visible, the error bars are within the symbols.

FIGURE 13

Mean relative error (MRE) calculated for Be, C, Al, Ti, Mo, Ta, and U targets, compared against the Sandia Lab experimental data., The MRE is calculated over the incident angles and electron energies under study. Here the uncertainties are provided as 1$\sigma$ (see Section 2.1).

FIGURE 14

Top: electron backscattering coefficients and ratios to reference data calculated for 30, 60, and 90 keV electrons, incident on Au, Bi, and Si targets. The reference data are experimental measurements of the project Exacrad of the European Space Agency. The shadowed area shows an agreement within 5%. Bottom: mean relative error (MRE), calculated for each EM physics constructor under study. Here the uncertainties are provided as 1$\sigma$ (see Section 2.1).

FIGURE 15

Top: fluence of forward‐scattered electrons obtained with a 13‐MeV electron beam incident on the following foils: 0.926 g/${\text{cm}}^2$ Be, 0.546 g/${\text{cm}}^2$ C, 0.14 g/${\text{cm}}^2$ Al, 0.0910 g/${\text{cm}}^2$ Ti, 0.0864‐g/${\text{cm}}^2$ Cu, 0.443 g/${\text{cm}}^2$ Ta, and 0.0312 g/${\text{cm}}^2$ Au. When not visible, the error bars are within the symbols. Middle: ratio of the fluences of the forward‐scattered electrons, plotted against the lateral position, for the different foils, for Geant4 10.5 and 11.1. Bottom: ratio of the fluences of the forward‐scattered electrons, for Opt3 with Geant4 versions 11.1 and 11.2.1. The shadowed area represents an agreement of 5%.

FIGURE 16

Top: Bremsstrahlung spectra obtained with a 15.18 MeV electron beam incident on targets of Al and Pb. Middle: ratio of the simulation results to the experimental reference data in log scale. Bottom: ratio of the simulation results to the experimental reference data on a linear scale. Note: Only every fifth data point is shown on the ratio plots for clarity and points outside the axis limits are not visible in the linear scaled ratio plot but are shown in the log scaled plot. The shadowed area represents an agreement within 5%.

FIGURE 17

Ratio of simulated dose and the theoretical value plotted against the dRoverRange parameter. The shadowed area represents a 1% agreement with the unity.

FIGURE 18

Total inelastic cross sections for the nuclear reactions under study. The Geant4 cross sections, obtained with QGSP_BIC_EMY and represented with red and blue lines for Geant4 10.5 and 11.1, respectively, do not have any statistical uncertainty because they are model‐based calculation. The shadowed area represents an agreement within 10%.

FIGURE 19

Total inelastic cross section, obtained with QGSP_BIC_EMY and represented with red and blue lines for Geant4 10.5 and 11.1, zoomed in the energy range between 1 and 100 MeV/u, together with the ratios (represented with gray continuous lines) of the cross sections calculated with Geant4 11.1 and 10.5.

FIGURE 20

production double‐differential cross sections, obtained with 62 MeV/u ions incident on a thin natural carbon target, calculated with Geant4 10.5 and 11.1. The simulation uncertainty is calculated, assuming a Poisson distribution, proportional to the square root of the number of events in each bin. Reference experimental measurements from De Napoli et al.

FIGURE 21

Ratio of the simulation and reference data for production double‐differential cross sections, obtained with 62 MeV/u ions incident on a thin natural carbon target. The shadowed area represents a 10% agreement. Reference experimental measurements from De Napoli et al.

FIGURE 22

Top: comparison of Geant4 simulation results obtained with Geant4 10.5 and 11.1 against experimental data for total and partial charge‐changing cross sections. The energy of the incident ion beam is 300 MeV/u. The error bars of the simulation results are within the symbols. Bottom: ratio of simulated and reference data for Geant4 10.5 and 11.1. The shadowed area represents an agreement within 10%.

FIGURE 23

${\overline{L}}_T$ curves obtained with Geant4 11.1, when using QGSP_BIC_HP (red), QGSP_BERT_HP (orange), and QGSP_BIC_AllHP (blue), against $y_F$ values obtained experimentally (Petringa et al. ²⁴ ). $L_T$ uncertainties have been calculated as the standard deviation of 10 simulations executed with different seeds, and they are of the order of 1%. The shadowed area represents an agreement within 10%.

FIGURE 24

Top row: positron yield per incident ion, with energy 148.5 MeV/u (left), 290.5 MeV/u (middle), and 350 MeV/u (right). Bottom row: ion, with energy 148 MeV/u (left) and 290 MeV/u (right). The positron yield is plotted with respect to the depth in the gelatin phantom. The yellow areas indicate the Bragg peak regions. The bottom plots in each row show the ratio between simulated and experimental data. The shadowed area represents an agreement within 10%, while the uncertainties (2$\sigma$) affecting the simulation results are shown as a shade around the curve. Experimental measurements from Chacon et al.

FIGURE 25

Top row: positron yield per incident ion, with energy 148.5 MeV/u (left), 290.5 MeV/u (middle), and 350 MeV/u (right). Bottom row: ion, with energy 148 MeV/u (left) and 290 MeV/u (right). The positron yield is plotted with respect to the depth in the PMMA phantom. The yellow areas indicate the Bragg peak regions. The bottom plots in each row show the ratio between simulated and experimental data. The shadowed area represents an agreement within 10%, while the uncertainties (2$\sigma$) affecting the simulation results are shown as a shade around the curve. Experimental measurements from Chacon et al.

FIGURE 26

Top row: positron yield per incident ion, with energy 148.5 MeV/u (left), 290.5 MeV/u (middle), and 350 MeV/u (right). Bottom row: ion, with energy 148 MeV/u (left) and 290 MeV/u (right). The positron yield is plotted with respect to the depth in the polyethylene phantom. The yellow areas indicate the Bragg peak regions. The bottom plots in each row show the ratio between simulated and experimental data. The shadowed area represents an agreement within 10%, while the uncertainties (2$\sigma$) affecting the simulation results are shown as a shade around the curve. Experimental measurements from Chacon et al.

FIGURE 27

Comparison of ${\mathrm{R}}_{80}$ and Gaussian spread ${\sigma }^{\ast }$ between Geant4 10.5/11.1 predictions and reference data, for a 67.5 MeV proton beam incident on a water phantom. Reference data from Faddegon et al.

FIGURE 28

Difference of ${\mathrm{R}}_{82}$ between Geant4 simulations and reference data. The differences are plotted against the energy of the incident particles (protons and ). The error bars are calculated at a 95% confidence level.

FIGURE 29

Integral neutron yields with respect to the angle of emission of neutrons, calculated with Geant4 and compared against reference data from Meier et al., for protons and from Satoh et al. for incident carbon ions. The error bars are within the symbols.

FIGURE 30

Neutron yields produced by protons in a Fe thick target. Top plot: 113 MeV protons incident on a 1.57 cm thick Fe target. Bottom plot: 256 MeV protons incident on a 8.0 cm thick Fe target. In each plot, top row: neutron yield; second row: ratio of Geant4 results and reference data; third row: ratio of the results obtained with Geant4 11.1 and 10.5. The shadowed area represents an agreement within 10%. When not visible, the error bars are within the symbols. Reference data: Meier et al.,

FIGURE 31

Fragment yield calculated with respect to depth. Experimental data from Haettner et al. The error bars of the simulation results are within the symbols.

FIGURE 32

Ratio of Geant4 simulation results and reference data in terms of fragment yields. Experimental data from Haettner et al. The shadowed area represents an agreement within 10%.

FIGURE 33

Mean relative error (MRE) calculated for H (left), He (middle), and B (right), for the calculation of the fragments yields. The MREs of fragment yields are calculated by averaging the ratio of experiment and simulation for all six water thicknesses for the particular fragment species. Here, the reported uncertainties correspond to 1$\sigma$ are provided as 1$\sigma$ (see Section 2.1).

FIGURE 34

Mean relative error (MRE) calculated for the angular (top) and kinetic energy (bottom) distributions of fragments, for both Geant4 10.5 (red) and 11.1 (blue). The MRE is calculated for individual distributions (35 angular and 151 energy distributions). Here, provided as 1$\sigma$ (see Section 2.1).

FIGURE 35

Top plots: Dose Point Kernel in water, obtained for 10, 15, and 100 keV electrons. Bottom plots: ratio of the results obtained with either Geant4‐DNA physics list Geant4‐DNA‐Opt4 or Geant4‐DNA‐Opt6 and Geant4‐DNA‐Opt2. Results obtained with the LowEElectDPK test and Geant4 11.1. The error bars of the simulation results are within the symbols. To note, for 100‐keV electrons, Geant4‐DNA‐Opt4 provides the same results as Geant4‐DNA‐Opt2.

FIGURE 36

Top plots: Dose–mean lineal energy $y_D$ with respect to the kinetic energy of the incident electrons, for a water sphere of 10 nm (left), 100 nm (middle), and 1 $\mu \mathrm{m}$ diameter (right). Bottom plots: ratio of the results obtained with either Geant4‐DNA‐Opt4 or Geant4‐DNA‐Opt6 with the default Geant4‐DNA‐Opt2. Results obtained with the microdosimetry test and Geant4 11.1. When not visible, the error bars are within the symbols.

FIGURE 37

G‐values of ${\mathrm{e}}_{aq}^{-}$, ${\mathrm{H}}_2{\mathrm{O}}_2$, and species calculated with the chemistry test, using the SBS, IRT‐sync, and IRT models. Results obtained with Geant4 11.1. The error bars of the simulation results are within the symbols.

FIGURE 38

Ratio of G‐values (plotted in Figure 37) obtained with IRT‐sync and IRT models, when compared to SBS. The shadowed area indicates an agreement within 5%.

FIGURE 39

Ratio of G‐values obtained with SBS, IRT, and IRT‐sync (Geant4 11.1) and experimental data., , The shadowed area indicates an agreement within 5%.