Digital twins as self-models for intelligent structures

Abstract

A self-model is an artificial intelligence that is able to create a continuously updated internal representation of itself. In this paper we use an agent-based architecture to create a 'digital twin self-model', using the example of a small-scale three-story building. The architecture is based on a set of heterogeneous digital components, each managed by an agent. The agents can be orchestrated to perform a specific workflow, or collaborate with a human user to perform requested tasks. The digital twin architecture enables multiple complex behaviors to be represented via a time-evolving dynamic assembly of the digital components, that also includes the encoding of a self-model in a knowledge graph as well as producing quantitative outputs. Four operational modes are defined for the digital twin and the example shown here demonstrates an offline mode that executes a predefined workflow with five agents. The digital twin has an information management system which is coordinated using a dynamic knowledge graph that encodes the self-model. Users can visualize the knowledge graph via a web-based user interface and also input natural language queries. Retrieval augmented generation is used to give a response to the queries using both the local knowledge graph and a large language model.

Keywords: Agent; Digital twin; Self-model; Structure.

Fig. 1

Creating a digital twin (DT) self-model from a predefined workflow for the three-story building example, showing a the DT components, b the predefined workflow, c the knowledge graph which encodes the self-model, d additional data sources, e the physical twin (PT), f a sample of the measured data from the PT, and (g) quantitative outputs. The solid arrows show initial data flows and the dashed lines show dynamically updated data/information flows into the knowledge graph.

Fig. 2

Screenshots of the user interface of the three-story building digital twin. a The web-page that forms the user-interface in this example, where a visualization of the knowledge graph is shown and the user can also input queries. Note that when the user’s cursor hovers over the agent node, the node dynamically enlarges, and the output generated by the agent is displayed adjacent to it (in this case the Gmsh image). This output visualization persists only during the hover interaction and is concealed once the cursor moves away from the node. The query system uses a form of retrieval augmented generation (RAG) to combine the local knowledge graph with a large language model (LLM). b A query which is answered using information from the local knowledge graph (the cursor response is RAG). c A query which is answered using information from the LLM (cursor shows as LLM).

Fig. 3

The overall architecture of the digital twin. N-agents are used to perform specific computational tasks within the digital twin. The agents communicate across a network. The information management system is coordinated with a dynamic knowledge graph, and the user interface allows the user to interact with the digital twin. The physical twin has local edge hardware such that data can be collected, and control actions taken to adjust the behavior of the physical twin. The numbered circles denote the main points of information exchange. Specifically is the data exchange between the physical and digital twin, is the exchange between the N agents and the IMS, is the UI interface with the agents, the UI and IMS and finally is the human user interaction with the digital twin.

Fig. 4

a Modes of operation for the digital twin as a combination of the connectivity modes and work modes. The cases are (1) online mode with agent orchestration, (2) online mode with the human user and agents collaborating, (3) offline mode with agent-based orchestration, and (4) offline mode with the human user and agents collaborating. b A schematic diagram of the structure of an intelligent agent within the digital twin.

Fig. 5

Sample pseudo-code for the algorithms for parts of the agent process. Shown here are: A Gmsh meshing process, and B workflow of the knowledge graph.

Fig. 6

Examples of the knowledge graph structure, showing a the three-story building knowledge graph with the primary entity of the graph (Layer 0) as the sand color entity, and the agents as green entities — note a number of entities and relationships have been removed from this Figure to make it readable. b A meta-graph, indicating a generic graph structure for a generic agent, with a representative example of the associated entities and relationships. c Evolution of the knowledge graph; the KG is initialized by defining the Layer 0 node, and it will subsequently be updated for each of the agent inputs (indicated by red arrows) and outputs (indicated by blue arrows) to the KG; the associated computational time between agent inputs and outputs is indicated in seconds (s). Note only Layers 0 and 1 are color coded in this figure.