Reinforcement learned adversarial agent (ReLAA) for active fault detection and prediction in space habitats

Abstract

With growing interest for human space tourism in the twenty-first century, much attention has been directed to the robust engineering of Environmental Control and Life Support Systems in space habitats. The stable, reliable operation of such a habitat is partly achieved with an ability to recognize and predict faults. For these two purposes, a reinforcement learning adversarial agent (ReLAA) is utilized in this work. A ReLAA is trained with experimental data to actively recognize and predict faults. These capabilities are achieved by proposing actions that activate known faults in a system. Instead of issuing these harmful actions to the actual hardware, a digital twin of the mock space habitat is simulated to discover vulnerabilities that would lead to faulted operation. The methods developed in this work will allow for the discovery of damaging latent behavior, and the reduction of false positive and negative fault identification.

Fig. 1. In the thermal control system (TCS), water is circulated through a closed loop to regulate the mock space habitat’s room temperature.

a A simplified schematic illustrates the operation of the TCS by showing how important components (heat exchanger, pump, chiller, and heater) are arranged in the loop. b The TCS was mostly assembled, installed, and instrumented on one wall within the mock space habitat (chiller not shown). (The annotated picture is provided by PacMar Technologies and used with permission.).

Fig. 2. In the grey water filtering system (GWS), grey water is sourced from a feed tank, filtered through a forward osmosis (FO) module, filtered through a reverse osmosis (RO) module, and finally fed into the product tank as potable water.

a A simplified schematic illustrates the operation of the GWS. Essentially, there are 3 loops in which water flows. The FO and RO modules are key components. b The GWS was mostly assembled, installed, and instrumented on one wall within the mock space habitat. The GWS’s feed tank (left) and product tank (right) are out of view in this picture. (The annotated picture is provided by PacMar Technologies and used with permission.).

Fig. 3

Multiple ReLAAs are trained before they are deployed to detect faults.

Fig. 4. Experimentally captured data from the mock space habitat illustrate the operation during normal and faulted conditions.

a Normal and faulted operation is shown for a variety of faults emulated in the TCS. b Pressures throughout the GWS change in response to normal and faulted conditions.

Fig. 5. KL divergence during adversarial agent training.

Average reward is represented by the dot color, with lighter being higher performers.

Fig. 6. ReLAA rollout shown for TCS flow meter 2.

A successfully trained ReLAA issues actions to the system which allow for faulted operation: the reduction in flow (outside of satisfactory operating bounds shown in red) throughout the TCS as shown in green.