Actuation-enhanced multifunctional sensing and information recognition by magnetic artificial cilia arrays

Abstract

Artificial cilia integrating both actuation and sensing functions allow simultaneously sensing environmental properties and manipulating fluids in situ, which are promising for environment monitoring and fluidic applications. However, existing artificial cilia have limited ability to sense environmental cues in fluid flows that have versatile information encoded. This limits their potential to work in complex and dynamic fluid-filled environments. Here, we propose a generic actuation-enhanced sensing mechanism to sense complex environmental cues through the active interaction between artificial cilia and the surrounding fluidic environments. The proposed mechanism is based on fluid-cilia interaction by integrating soft robotic artificial cilia with flexible sensors. With a machine learning-based approach, complex environmental cues such as liquid viscosity, environment boundaries, and distributed fluid flows of a wide range of velocities can be sensed, which is beyond the capability of existing artificial cilia. As a proof of concept, we implement this mechanism on magnetically actuated cilia with integrated laser-induced graphene-based sensors and demonstrate sensing fluid apparent viscosity, environment boundaries, and fluid flow speed with a reconfigurable sensitivity and range. The same principle could be potentially applied to other soft robotic systems integrating other actuation and sensing modalities for diverse environmental and fluidic applications.

Keywords: actuation-enhanced sensing mechanism; bioinspiration; fluidics; sensor-integrated cilia; soft robot.

Fig. 1.

Concept of the actuation-enhanced sensing mechanism and signal analysis. (A) The proposed application scenarios for the actuation-enhanced sensing mechanism of the SIC array. The SIC array could sense i) the size and distance of the boundary wall, ii) apparent viscosity of the liquid environment, and iii) the fluid flow with adjustable stiffness in a confined space. iv) The microcomputational tomography (micro-CT) image of the SIC array (6×6) with integrated sensors on top of a printed circuit board (PCB) for distributed sensing of liquid environment properties. v) The zoomed-in schematic of the SIC with a layer of magnetic composite and a layer of LIG. (B) The signals collected from SICs in different fluid environments are encoded as standard pulse code modulation (PCM) signals. To enable continuous prediction, features are extracted from the signals in both time and frequency domains and classified using linear discriminant analysis (LDA). The feature that contributes the most to the classification is used in the calibration of SICs, which further enables the prediction of unknown environmental properties. To efficiently classify signals from a large number of SICs, the corresponding spectrograms are generated using the wavelet packet decomposition (WPD) and processed using external recognition with deep-learning models.

Fig. 2.

Design, fabrication, and characterization of the SIC array. (A) The illustration of the proposed laser-based method for fabricating the SIC array. (B) The 3D profile of a single sensor-integrated cilium recorded by a laser surface profiler microscope. (C) The SEM image (recolored) of the bilayer structure after the LIG transfer process. On top is the LIG layer (green) and on bottom is the PDMS-NdFeB composite layer. The NdFeB particle is marked in blue (average diameter of 5 μm). (Scale bar, 20 μm.) (D) Optical image of cilia array after the laser-based fabrication process. The dark part is the patterned LIG sensor, and the gray part is the magnetic elastomer after engraving, which holds a rough surface with Sa = 1.092 μm. (Scale bar, 2 mm.) (E) The data plot of relative resistance changes $(\mathrm{\Delta }R/R_0)$ as a function of the cilium bending angle. The fitting curve from nonlinear regression holds R² = 0.995. (Scale bar, 1 mm.) (F) Signals with phase difference obtained from a SIC array (1×8) with the magnetic profiles in metachronal coordination. (G) The optical image and the image analysis of a SIC array with metachronal waves. (Scale bar, 2 mm.) (H) The signal amplitude of a SIC changed with different applied magnetic field strengths in air. (I) Durability tests of the SIC under over 10,000 cycles at different magnetic actuation frequencies, ranging from 1 Hz to 8 Hz. The inserted subfigure shows the signal in 1 s. (J) The scaled resistance changes of the LIG-based sensor as a function of the strain in a tensile test. The gauge factor reaches around 700 when the applied strain is less than 0.1% and around 3,500 when the strain is in the range of 1% to 4%.

Fig. 3.

Demonstration of sensing the viscosity of a liquid environment. (A) Illustration of the motions of the SIC in liquid environments of different viscosities. The proposed magnetically controlled cilium holds a larger bending angle in a low-viscosity environment than that in a high-viscosity environment with the same actuation signals. (B) The force analysis diagram of the infinitesimal element of the SIC. The movement of SIC is influenced by the fluid drag force $F_D$ , magnetic force $F_m$ and torque $\tau$ . (C) Stacked optical images of the SIC in one actuation cycle. Tested in air with $\begin{array}{ccc}{B}_m& =& 15 \mathrm{mT}\end{array}$ and $\begin{array}{ccc}f& =& 2 \mathrm{Hz}\end{array}$ . (Scale bar, 1 mm.) (D) Contours of the SIC actuated in different liquid environments with various viscosities, and the color is remapped from grayscale to “viridis” colormap. $\begin{array}{ccc}{B}_m& =& 30 \mathrm{mT}\end{array}$ ; $\begin{array}{ccc}f& =& 2 \mathrm{Hz}\end{array}$ . (E) Comparison among signals of a SIC under different viscosity environments. $\begin{array}{ccc}{B}_m& =& 15 \mathrm{mT}\end{array}$ ; $\begin{array}{ccc}f& =& 2 \mathrm{Hz}\end{array}$ . i) shows the original signal ( $\mathrm{\Delta }R/R_0$ ) and ii) shows the signals after first-order deviation. (F) Visualized classification result on the signal categories after dimensionality reduction with LDA. Each category corresponds to different viscosity liquid environments. The extracted features of the signals include both time- and frequency-domain. $\begin{array}{ccc}{B}_m& =& 30 \mathrm{mT}\end{array}$ ; $\begin{array}{ccc}f& =& 2 \mathrm{Hz}\end{array}$ . (G and H) The peak-to-peak value extracted from the signals of SIC under different actuation frequencies and different viscosity liquid environments. $\begin{array}{ccc}{B}_m& =& 30 \mathrm{mT}\end{array}$ ; $\begin{array}{ccc}f& =& 1 \mathrm{to} 7 \mathrm{Hz}\end{array}$ . (I) The fluid viscosities predicted by the proposed SIC at different actuation frequencies in non-Newtonian fluids (hyaluronic acid aqueous solutions, 6.3 mg/mL). (J and K) The simultaneous distributed dynamic viscosity perception and fluid manipulation when mixing low viscosity droplet (3 cP) into high viscosity environment (glycerol) by the proposed SIC array (array size: 2×4).

Fig. 4.

Demonstration of sensing the solid environmental boundaries. (A) Schematic of the distributed active sensing of boundary conditions by SIC array with metachronal coordination. (B) Comparison of signals of a cilium in a glycerol environment with different boundary distances. (C) Optical images of a 2×1 cilia array while sensing different boundaries with different distances. The marked area in (v) indicates the acceleration period of the SIC during the recovery stroke. (Scale bar, 2 mm.) (D) The two dimensional visualization of classification result of the different signal categories after dimensionality reduction with LDA. Each category corresponds to a different boundary distance, with a boundary size of 10 mm. The features of the signal include both time domain and frequency domain information. (E) The confusion matrix of classification results in boundary distance perception using the LDA method. (F) The changes in the Crest factor of the signals under different boundary size and distance conditions. Tested under glycerol environment with $\begin{array}{ccc}{B}_m& =& 30 \mathrm{mT}\end{array}$ and $\begin{array}{ccc}f& =& 4 \mathrm{Hz}\end{array}$ . (G) The setup for distributed boundary sensing with SIC array (array size, 6×6), in which D₁ = 5.5 mm, D₂ = 5.5 mm, d₁ = 2.5 mm, and d₂ = 2.5 mm. (H) Prediction results of the distributed sensing on the complex boundary. A SIC array with 3×3 sensing units is occupied, and each unit contains four cilia for high reliability. Each SIC holds its own LDA model that is trained by the test dataset.

Fig. 5.

Flow chart of the signal processing and the metrics of recognition-related performance in the sensing of viscosity and boundary conditions. (A) The basic process of using GoogLeNet to classify the dataset of signals collected from the SIC array. (B) The sample signals and corresponding spectrograms of SICs under different viscosity and boundary conditions. (C) A spectrogram of the signal in viscosity sensing, showing periodic features in different frequencies. $\begin{array}{ccc}{B}_m& =& 30 \mathrm{mT}\end{array}$ , $\begin{array}{ccc}f& =& 2 \mathrm{Hz}\end{array}$ . (D) Recognition accuracies of the signals in the mode of viscosity sensing, boundary sensing, and mixed sensing (independent SIC samples of n = 40 used in viscosity sensing, and n = 160 in boundary sensing). The full class recognition of viscosity includes 13 sets of different viscosity conditions ranging from ${10}^{-2}$ cP to ${10}^3$ cP, and 15 sets of different boundary conditions ranging from 200 μm to 7,000 μm. (E) The results of t-distributed stochastic neighbor embedding (t-SNE) for the signals in viscosity, boundary, and mixed sensing mode. (F) Confusion matrix of the mixed sensing mode with changes in both viscosity and boundary (overall accuracy of 90%).

Fig. 6.

Demonstration of sensing the environmental fluid flows with reconfigurable range and sensitivity. (A) The illustration of magnetic field-assisted flow sensing with adjustable stiffness. (B) The result of the stiffness test under different magnetic fields; the test sample has the same stiffness as the SIC we demonstrated before. (C) Images of the SIC inside a chamber for flow sensing. The dimension of the channel cross-section is $2.6 \mathrm{m}\mathrm{m} \times 1.5 \mathrm{m}\mathrm{m}$ ( $\mathrm{h}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\times \mathrm{w}\mathrm{i}\mathrm{d}\mathrm{t}\mathrm{h}$ ). (Scale bar, 5 mm.) (D–G) Signals of the SIC under different magnetic field strengths when sensing flow fields with different flow rates. The flow speeds are set as 10, 20, and 40 mL/min in D–F. The magnetic field with different strengths in each flow rate situation are 0, 11, 17.5, 30, and 40 mT. The signal shows a smaller amplitude (lower sensitivity) and smaller drift under the larger magnetic field when sensing the same flow rate. (H) The sensing signal of a flow at 600 mL/min, and the magnetic field strength are set as 0 mT and 40 mT. The signal is distorted and has more fluctuations in the absence of a magnetic field. (I–K) The signals of the SIC under the periodic oscillating flow generated by a heartbeat pump, and the flow speeds are set as 10, 20, and 40 mL/min at 80 bpm. Compared to a nonmagnetic field situation, the signal of the sensor-integrated cilium under 4 mT shows less drift, especially when sensing low-speed flows. (L and M) The detailed signals when sensing the mimetic heartbeat flows with 10 mL/min and 20 mL/min flow rate under magnetic field of 0 mT and 4 mT. The signals of SIC under magnetic field preserve the small peaks in the sensing of oscillatory flows which is mainly due to the magnetic torque applied that help to eliminate the response hysteresis of the ciliary soft body.