A physics-informed and data-driven framework for robotic welding in manufacturing

Abstract

The development of artificial intelligence (AI)-based industrial data-driven models is the driving force behind the digital transformation of manufacturing processes and the application of smart manufacturing. However, in real-world industrial applications, the intricate interplay among data quality, model accuracy, and generalizability poses significant challenges, hindering the effective deployment and scalability of data-driven models in complex manufacturing environments. To address this challenge, this paper proposes a universal Physics-informed Hybrid Optimization framework for Efficient Neural Intelligence (PHOENIX) in manufacturing, demonstrating its applicability in robotic welding scenarios. This framework systematically integrates physical principles into its input, model structure, and dynamic optimization processes, enabling proactive, real-time detection and predictive of welding instability. It achieves an accuracy of up to 98% for predictions within the next 50 ms and maintains an accuracy of 86% even for forecasts up to 1 s in advance. Through physics-informed data-driven modeling, the framework significantly reduces the dependence on high-cost data while maintaining the performance of the original model. By leveraging cloud-based optimization modules that integrate new data with historical experience, the framework enables autonomous model parameter optimization, ensuring its continuous adaptation to the complex and dynamic demands of industrial scenarios.

Fig. 1. The PHOENIX framework.

The PHOENIX framework is proposed to comprehensively integrate physical knowledge into data-driven models, enhancing their predictive performance and adaptability in welding-based manufacturing processes. This framework reduces the reliance on high-quality, large-scale datasets while significantly improving the prediction accuracy, recall, and generalization capabilities of the constructed model under complex operating conditions. a The PHOENIX framework systematically extracts physical information, including engineering expertise, welding knowledge, and conservation laws, to embed these insights into data-driven models. This approach effectively guides the model training and optimization processes, enhances the accuracy of the model and ensures consistency with physical information. b By leveraging physics-informed data-driven models, the framework substantially decreases the dependency on high-quality data and a large number of feature, addressing the challenges posed by small-scale and low-quality datasets in model applications. c During the training process, PHOENIX incorporates physical constraints to optimize the model normalization, parameter tuning, and loss function design procedures. Moreover, it translates physical laws into explicit model constraints, strengthening the ability of the framework to physically represent welding processes. d PHOENIX establishes an intergroup dynamic learning mechanism by integrating historical prior data with real-time collected data. This approach enhances the stability and reliability of the model in complex industrial scenarios, ensuring robust and dependable predictive results. e The framework enables the precise perception of transient and sequential behaviors in manufacturing processes by fusing physical information with data-driven models. This capability supports proactive predictions of melt pool dynamics, ensuring high stability and weld quality during the welding process. f The PHOENIX framework exhibits remarkable versatility and is applicable to real-time welding process monitoring, predictive melt pool state modeling, dynamic weld defect detection, and adaptive welding parameter regulation. Furthermore, it holds the potential for expansion to other welding methods and additive manufacturing technologies, offering a novel solution for implementing process optimization and quality control across diverse manufacturing techniques.

Fig. 2. Application of the PHOENIX framework in robotic VPPA welding.

Robotic VPPA welding represents a prototypical application scenario of the PHOENIX framework. Through three distinct pathways, the framework effectively integrates physical information to enable highly accurate and generalizable early melt pool instability predictions, even with small datasets. The blue pathway involves the development of a time-ahead melt pool instability prediction module based on the dynamic and morphological features of the melt pool. The red pathway concerns the establishment of a flow model for melt pool dynamics using quasistatic physical features as constraints, leveraging data-driven models to obtain precise dynamic feature predictions. The yellow pathway enables dynamic model hyperparameter tuning through intergroup incremental learning; this process integrates the historical data with the newly acquired information, enhancing the performance and adaptability of the model. a An in situ high-speed X-ray acquisition system captures high-precision melt pool dynamics data. While they are costly to obtain, these data provide a robust foundation for understanding melt pool behavior. b A highly adaptable machine vision module, which is equipped with transfer learning capabilities, extracts multisource image features in real time, delivering reliable data support for online melt pool monitoring. c An early instability detection module was developed; this module leverages physical information inputs to efficiently predict melt pool instability during robotic VPPA welding. d Melt pool flow trajectories and saddle point information at varying depths were recorded using a particle tracking method via an in situ high-speed X-ray system. e An industrial camera acquisition system was employed to collect low-cost morphological data from the melt pool, providing a viable solution for the rapid acquisition of cost-effective data. f A data-driven model constrained by quasistatic welding parameter features was developed to substitute expensive data with cost-effective alternatives, thereby reducing the reliance on high-quality data while maintaining strong model performance. g A schematic illustration of the full-position robotic VPPA welding and monitoring system under practical operational conditions. h Incremental learning was employed to integrate historical experience with the newly collected data, enabling proactive temporal information correction and dynamic model optimization.

Fig. 3. Robotic VPPA welding system with diverse welding conditions.

The VPPA welding technique has become crucial for manufacturing spacecraft components, particularly lightweight aluminum alloy structures. As the payload capacity requirements of commercial aerospace applications have increased, the performance requirements of aluminum alloys in terms of volume and weight have become more stringent. Simultaneously, the manufacturing industry, driven by customized solutions, demands greater flexibility, precision, and adaptability from welding technologies in complex scenarios. However, VPPA welding faces challenges, including multi-position welding, adaptation issues derived from preassembly errors in large components, and external disturbances that affect welding stability, necessitating process optimization and performance enhancement techniques. a The core principle of VPPA welding lies in the ionization of argon to form a stable plasma arc, thereby achieving efficient energy transfer and precise heat input control. This high-energy density arc provides robust technical support for aluminum alloy welding. b A distinctive feature of the VPPA welding process is the formation of a unique keyhole (through-hole) structure. The keyhole, which is achieved through the concentration of heat, facilitates efficient melting and penetration, maintaining its stability throughout the welding process. However, the instability of the molten metal flow can disrupt the maintenance of the keyhole structure, leading to weld cracking or incomplete fusion defects (see Supplementary Movie 3). c The application of robotic VPPA welding in large, complex, spatially curved aluminum alloy structures is exemplified, demonstrating the flexibility and precision of this technology under intricate geometric conditions. d During quasistable VPPA welding, the molten metal flows stably, forming a distinct rear saddle point. This process is typically accompanied by well-formed welds, reflecting the high stability and controllability of the welding process. e Under the nonstationary state, the saddle point of the melt pool exhibits some fluctuations, yet the weld formation remains relatively well formed, indicating the adaptability of the system to localized instabilities. f When the VPPA welding process enters an instability state, the saddle point of the melt pool is significantly impacted, and the keyhole structure becomes difficult to maintain, leading to incomplete weld closure or the occurrence of cutting defects.

Fig. 4. Physical feature-enhanced time-ahead prediction module.

In the guidance of the PHOENIX framework, the time-ahead prediction module built with physical information and integrated with machine vision significantly simplifies the process of constructing in situ online monitoring systems. The model achieved high welding state prediction accuracy and greatly reduced its missed detection rates through a feature engineering mechanism that incorporates engineers’ prior experience and welding knowledge. a The machine vision model constructed through transfer learning worked in synergy with the time-ahead prediction module based on physical information inputs, demonstrating an efficient operational flow. The model exhibited flexibility in acquiring and processing time series data. b Feature engineering, which is based on welding knowledge and engineers’ experience, significantly enhanced the generalizability of the model. During this process, expensive melt pool dynamics features derived from X-rays and cost-effective morphological features captured by industrial cameras were efficiently integrated via the machine vision module, supporting the multisource information processing ability of the model. c Performance evaluation results obtained with respect to the time-ahead prediction module in various data usage scenarios revealed significant prediction accuracy differences when all data, expensive data, cost-effective data, and physical models (replacing the expensive data) were used. These evaluations also demonstrated the dynamic relationship between the time length of the time series data accumulated in the sliding window and the predictions of future changes. The error bars represent the standard error of time-ahead prediction results across multiple independent data groups.

Fig. 5. Performance and visualization of physical feature-enhanced time-ahead prediction module.

Statistical analysis was conducted on the model to investigate the sensitivity of the time-ahead prediction module to different input features. a The accuracy and specificity of GRU-MLP, RNN-MLP, LSTM-MLP, and LSTM were compared. b A confusion matrix analysis was conducted on the task of predicting the welding state 0.05 s in the future using a 0.2 s sliding window. c Ablation experiments were implemented to visualize the importance rankings of the expensive and cost-effective features, showing the critical roles of these features under different welding states. d A t-SNE-based dimensionality reduction scheme was executed, and the result visualization clearly presents the temporal and spatial information capture capabilities of the time series prediction steps under different melt pool states.

Fig. 6. A double saddle point model in VPPA welding via a particle tracer method.

To further reveal the intrinsic mechanisms of melt pool instability and provide reliable data support for the construction of data-driven models, this study employed a combination of particle tracking and high-speed in situ X-ray acquisition systems to capture the flow behavior and flow channels of the molten metal contained within the melt pool. This method enabled the movement and flow paths the molten metal to be precisely tracked, thus providing crucial data for constructing physical models. a Particle tracking, which uses the high X-ray absorption capabilities of tungsten particles, reveals the flow behavior of the molten metal. This method accurately captures the microscopic flow features of the melt pool, offering new insights into the flow mechanisms and providing a novel perspective for constructing physical models. b Flow trajectories of tracer particles released at different depths in the melt pool, annotated via an image‑accumulation method (see Supplementary Movie 1 and Supplementary Movie 2). c The flow behavior and flow regions of the tracking particles significantly differ at various depths, and they present phenomena such as bifurcation at the front wall, a directional flow along the sidewalls, and complex merging and bifurcation behaviors at the rear wall. d The dual-saddle point flow model obtained from the experiments further reveals the key flow mechanisms in VPPA welding.

Fig. 7. Physics-constrained data-driven saddle point model for quasi-static information injection in welding scenarios.

By incorporating quasistatic welding parameters as constraints, a data-driven VPPA welding-based melt pool dynamics model was developed. This model successfully substitutes expensive data, allowing the constructed time series prediction model to deliver reliable results under lower-cost conditions. a A flowchart based on physical information constraints outlines the construction process of the data-driven model, providing a detailed explanation of its specific application in the time series prediction model. b CBN was used to embed physical static information, such as the welding current and ion gas, into the model. Upon combining the CBN and BPNN, the melt pool dynamics features were precisely modeled. c A comparative analysis indicated significant error performance differences between the MISO CBN-BPNN model (achieved through multiple training sessions) and the MIMO CBN-BPNN model (achieved through a single training session). Boxplots visually present the error distributions of the various melt pool dynamics features, further validating the superiority of the MIMO model. d A comparison with multiple machine learning methods (in terms of the MAE, MSE, RMSE, and R² metrics) demonstrates the significant advantages of the MIMO CBN-BPNN model, especially in terms of prediction accuracy and model robustness, providing a more competitive solution for melt pool dynamics modeling.

Fig. 8. Dynamic model parameter tuning via incremental learning with an integrated prior and new data.

To address the challenges posed by complex welding conditions and further enhance the generalizability of time-ahead prediction models, a dynamic model parameter tuning method was proposed based on the PHOENIX framework. This approach integrates prior knowledge with new experiential data through incremental learning, enabling the autonomous optimization of model parameters across different groups. This optimization ensures that the time-ahead prediction module maintains high efficiency and accuracy in dynamic and unpredictable industrial welding scenarios, effectively mitigating the issues related to environmental variability, tool misalignment, thermal strain, and other operational complexities. a Utilizing a distributed dual-edge and cloud-coordinated method, the proposed workflow rectifies the omissions and misclassifications arising in complex new scenarios. By fine-tuning the model through intergroup incremental learning, the generalization ability of the time-ahead prediction module is significantly enhanced. The theoretical foundation for this approach lies in the temporal causal relationships that are present among the melt pool, welding state, and weld seam during the welding process. b In practical applications, robotic VPPA welding often encounters unique environmental challenges, including different welding positions (e.g., horizontal or overhead welding), dynamic variations in the welding speed during directional changes, gaps caused by thermal strain in the base material, asymmetric heat dissipation leading to uneven thermal distributions, and anomalies in the wire feeding speed.

Fig. 9. Performance visualization of dynamic model tuning via incremental learning.

The figure illustrates the performance of dynamic model tuning via incremental learning, highlighting the impact of different layer-freezing strategies on the model’s adaptability, retention of prior knowledge, and overall predictive accuracy across evolving scenarios. a Experimental explored strategies for freezing different model parameter layers to preserve historical knowledge. Under the guidance of incremental learning, the model underwent adaptive optimization. Additionally, various sample replay strategies were evaluated, alongside the performance of the model in terms of adapting to complex new scenarios relative to its accuracy in familiar settings. The error bars represent the standard error of time-ahead prediction results across multiple independent data groups. b A comparison was conducted among three approaches for evaluating the cutting quality based on weld seam information: direct assessment using weld data, the time-ahead prediction module, and the time-ahead prediction module with incremental learning.