Multimodal Sensors with Decoupled Sensing Mechanisms

Abstract

Highly sensitive and multimodal sensors have recently emerged for a wide range of applications, including epidermal electronics, robotics, health-monitoring devices and human-machine interfaces. However, cross-sensitivity prevents accurate measurements of the target input signals when a multiple of them are simultaneously present. Therefore, the selection of the multifunctional materials and the design of the sensor structures play a significant role in multimodal sensors with decoupled sensing mechanisms. Hence, this review article introduces varying methods to decouple different input signals for realizing truly multimodal sensors. Early efforts explore different outputs to distinguish the corresponding input signals applied to the sensor in sequence. Next, this study discusses the methods for the suppression of the interference, signal correction, and various decoupling strategies based on different outputs to simultaneously detect multiple inputs. The recent insights into the materials' properties, structure effects, and sensing mechanisms in recognition of different input signals are highlighted. The presence of the various decoupling methods also helps avoid the use of complicated signal processing steps and allows multimodal sensors with high accuracy for applications in bioelectronics, robotics, and human-machine interfaces. Finally, current challenges and potential opportunities are discussed in order to motivate future technological breakthroughs.

Keywords: cross-sensitivity; decoupling sensing mechanisms; multimodal sensors; multiple input signals.

Figure 1

Schematic summarization of the development of multimodal sensors regarding decoupling mechanisms among different signals. a) Tactile sensor distinguishes different mechanical stimuli by signal patterns. Reproduced with permission.^{[ ]} Copyright 2018, Wiley‐VCH. b) Pressure‐insensitive strain sensor. Reproduced with permission.^{[ 14 ]} Copyright 2016, Nature Publishing Group. c) Integrated thermoelectric‐based temperature sensor and piezoresistive pressure sensor minimize interference with each other. Reproduced with permission.^{[ 15 ]} Copyright 2017, The Royal Society Chemistry. d) Polydimethylsiloxane (PDMS) bump‐based sensing array decouples 3D forces. e) Thermoelectric sensor to decouple pressure (P) and temperature (T) by different sensing principles. f) Multimodal sensor to decouple temperature (T) and pressure (P) by different response times. g) Bifunctional sensor based on a shared electrode to decouple different stimuli.

Figure 2

Asynchronous signal discrimination methods. a) Cloth‐based electronic skin (E‐skin) outputs different signal patterns under various forces. Reproduced with permission.^{[ 24 ]} Copyright 2016, Wiley‐VCH. i) Schematics of the cloth‐based sensor under pressure. ii) Changes in the relative current waveforms with various mechanical stimuli. b) Normal‐tangential force sensor with opposite resistance response. Reproduced with permission.^{[ 27 ]} Copyright 2018, Wiley‐VCH. i) Schematics of the sensor deformations upon normal pressure and lateral shear force. ii) Equivalent circuits, where R _S represents the resistance of the surface sublayer‐carbon nanotubes (CNTs)/graphene oxide (GO), and R _I represents the resistance of the inner sublayer‐GO/PDMS. c) Electronic fabric artificial skin that measures pressure, strain, and flexion by a combined analysis of different output signals from multiple electrodes. Reproduced with permission.^{[ 35 ]} Copyright 2015, Wiley‐VCH. i) Optical image of a sensor unit on a polyethylene terephthalate substrate. ii) Illustration of fibrous E‐skin and a single sensing unit. iii) The deformation at the contact point of the sensor unit with applied pressure, stretching, and flexion. A _p and d _p are the contact area and thickness with applied pressure, d _s is the thickness under stretching, and A _f is the contact area under flexion. iv) Changes in the relative resistances of the three electrodes over time for repeated pressure, strain, and flexion loadings. d) Multifunctional woven tactile sensor array. Reproduced under the terms of the Creative Commons CC BY License.^{[ 34 ]} Copyright 2017, The Authors, published by MDPI. i) Structural demonstration of the multifunctional woven sensor. ii) Schematic of the inner silver layer at resistant sensing mode for measuring strain. iii) Schematic illustration of the capacitor for pressure detection.

Figure 3

Decoupling target input by suppressing the influence of the other input signals. a) Pressure‐insensitive strain (PIS) sensor based on multiwalled carbon nanotubes (MWCNTs) and PDMS composite materials. Reproduced with permission.^{[ 14 ]} Copyright 2018, American Chemical Society. i) Scanning electron microscopy (SEM) images of the PIS sensor without (top) and with (bottom) tensile strain. ii) SEM images of the composite materials with a porous structure (top), and the zoomed version shows that most of the MWCNTs are embedded in the PDMS (bottom). iii) Photograph (top) and corresponding impedance analysis (bottom) of the PIS sensor under initial, pressured, and locally strained conditions. b) Transparent pressure sensor that is insensitive to bending. Reproduced with permission.^{[ 37 ]} Copyright 2016, Nature Publishing Group. i) Schematic diagram of the bending‐insensitive pressure sensor. ii) SEM image of the fiber layer. iii) SEM image of the fiber layer under tensile strain. iv) SEM image of the fiber layer under shear force. v) Sensor responds to pressure (0.4 and 1.6 g) at different bending radii (from 1.5 cm to 80 µm). c) E‐skin based on potentiometric mechanotransduction mechanism^{[ 39 ]} and its schematic illustration shown in (i). Two electrodes with reversible oxidation–reduction reactions are used to create a potential difference, where the microstructured ionic composite between electrodes alters the potential under an applied force. ii) Circuit model of the strain‐insensitive potentiometric sensor. iii) Almost invariable force sensing performance with or without 50% strain. Reproduced under the terms of the Creative Commons CC BY NC License.^{[ 39 ]} Copyright 2020, The Authors, published by The American Association for the Advancement of Science.

Figure 4

Integrated platforms with multiple sensors. a) Laminated nano‐cellulose tactile sensor. Reproduced with permission.^{[ 15 ]} Copyright 2017, The Royal Society Chemistry. i) Structural model with temperature (ii) and pressure (iii) sensing mechanisms. Responses from iv) temperature and v) pressure sensors to temperature and pressure. b) Silk‐derived E‐skin combined with pressure and temperature sensors. Reproduced with permission.^{[ 42 ]} Copyright 2017, American Chemical Society. i) Schematic showing an integrated temperature and pressure sensors (fabricated by intact continuous nanofiber structure and fractured nanofiber, respectively). ii) Relative resistance changes of the strain sensor versus strain. iii) Relative resistance changes of the temperature sensor versus pressure (not strain). Relative resistance changes of the iv) temperature and v) strain sensors versus temperature. c) Multifunctional E‐skin based on thermosensitive platinum. Reproduced with permission.^{[ 43 ]} Copyright 2017, Wiley‐VCH. i) Device structure of the E‐skin with the element descriptions. ii) Temperature response of the temperature sensor under different pressure. iii) Pressure response of the pressure sensor under different temperatures.

Figure 5

Array layout for decoupled multimodality. a) Flexible sensor based on CNT/PDMS composites to measure 3D force. Reproduced with permission.^{[ 65 ]} Copyright 2018, Springer Nature. i) Sensor in an exploded view and its demonstration to measure ii) pressure alone or iii,iv) shear forces in the x‐ or y‐axis with the pressure. b) Flexible capacitive sensor to measure pressure and shear forces.^{[ 68 ]} i) Schematic diagram, ii) layout, and its sensing principles for detecting shear (iii) and normal forces (iv). c) Bioinspired capacitive E‐skin to monitor the direction of force. Reproduced with permission.^{[ 69 ]} Copyright 2018, The American Association for the Advancement of Science. i) Optical image and ii) sensor placement of the E‐skin. Color maps show the real‐time discrimination of iii) normal force only, iv) shear force only, and v) combined normal and shear forces. d) Mirror‐stacked layer designed for decoupling bending from other stimuli such as temperature. Reproduced with permission.^{[ 70 ]} Copyright 2019, American Chemical Society. i) Schematic diagram of the sensor with ii) increased or iii) decreased length (and resistance) in the upper or lower part for iv) canceling the effect of bending.

Figure 6

Novel materials with multiple sensing principles in a single unit. a) Dual‐modal sensor built by a liquid droplet. Reproduced with permission.^{[ 75 ]} Copyright 2016, Wiley‐VCH. i) Sensing principle of the droplet sensor. ii) Variation of voltage over time when detecting temperature and pressure simultaneously. b) Dual‐parameter sensor based on thermoelectric materials^{[ 76 ]} at i) initial condition and after ii) temperature loading (with changed intercept) and iii) pressure loading (with shifted slope). iv) The sensor detects both the temperature and pressure with variations from the intercept and slope of the current‐voltage (I–V) curve. Reproduced under the terms of the Creative Commons CC BY License.^{[ 76 ]} Copyright 2015, The Authors, published by Springer Nature. c) Multiparameter sensor based on ionic aerogels. Reproduced under the terms of the Creative Commons CC BY License.^{[ 77 ]} Copyright 2019, The Authors, published by Wiley‐VCH. i) Schematic illustration of the pressure‐temperature‐humidity sensor with I–V curves exhibiting ii) different slopes for varying pressures and iii) different voltage axis intercepts for various temperatures. iv) Voltage as a function of time for different humidity values (ΔT = 10k added at 4 min). d) Artificial multimodal receptors that can differentiate strain and temperature. Reproduced with permission.^{[ 12 ]} Copyright 2020, The American Association for the Advancement of Science. i) Schematic illustration and ii) frequency‐dependent behavior of the ion conductor in different electric fields. iii) Bode plots for an ion conductor with a 5 wt% ion concentration at different temperatures. iv) Demonstration of the Bode plot with an increased (cut‐off) charge relaxation frequency (τ ⁻¹) and decreased resistance after applying temperature. v) Bode plot displaying a parallel downshift when the device is stimulated by tensile strain. vi) Changes in the charge relaxation time (ln (τ)) as a function of T ⁻¹ (T, temperature). vii) Relationship between the relative changes in capacitance and the tensile strain at various temperatures. e) Frequency‐enabled decouplable dual‐modal sensor. i) Schematic of the bimodal sensor based on chitosan. Reproduced with permission.^{[ 78 ]} Copyright 2020, IEEE. ii) Capacitance measurement of the sensor with different‐sized micropyramids under different frequencies at 80 kPa. iii) Temperature sensing performance varied under different frequencies. iv) Capacitance changes at 1 kHz (C _f1) and 500 Hz (C _f2) with both temperature and pressure stimuli. 3D map produced by linear interpolation to estimate pressure (v) and temperature (vi).

Figure 7

Materials with different response times to decouple temperature and force. a) Multifunctional tactile sensor based on ZnO/polyvinylidene fluoride (PVDF) film for detecting pressure and temperature. Reproduced under the terms of the Creative Commons CC BY License.^{[ 89 ]} Copyright 2015, The Authors, published by Springer Nature. i) Schematic of the sensor and ii) its recovery time changing with the applied pressure at different temperatures. iii) Color mapping to show the resistance changes as a function of recovery time, with iv) corresponding pressures applied to the sensor. b) Multi‐effect flexible tactile sensor in decoupling temperature and pressure. Reproduced with permission.^{[ 90 ]} Copyright 2019, Elsevier Ltd. i) Schematic of the tactile sensor for sensing pressure (top) and temperature (bottom). Voltage changes in the sensor when the pressure of 38 kPa is applied with a temperature of ii) 37 or iii) 15 °C. The enlarged view shows V ₁ generated by the temperature and pressure and V ₂ by the temperature only. c) Ferroelectric skins for detecting static/dynamic pressure and temperature. Reproduced under the terms of the Creative Commons CC BY NC License.^{[ 91 ]} Copyright 2015, The Authors, published by The American Association for the Advancement of Science. i) Schematic diagram of the ferroelectric skin and ii) its response (relative resistance changes) versus time for loading/unloading at various temperatures and pressures. d) Multifunctional E‐skin based on an interlocked quasi‐hemispherical micropattern array. Reproduced with permission.^{[ 92 ]} Copyright 2020 Elsevier B.V. ii) Response and recovery times with an object placed on top. iii) Change in relative resistance when the sensor is stimulated by both temperature and pressure. e) Dual‐mode sensor fabricated by an ionic hydrogel. Reproduced with permission.^{[ 93 ]} Copyright 2019, American Chemical Society. i) Pressure sensing mechanism and ii) its relative resistance changes upon pressure loading. iii) Sensing mechanism for temperature and iv) its change in relative resistance at 40 °C. v) Variation of relative resistance versus time upon both temperature and force loadings.

Figure 8

Combined structure and material innovations in a single sensing unit. a) Stretchable tactile sensor to distinguish multiple mechanical stimuli. Reproduced with permission.^{[ 96 ]} Copyright 2014 Wiley‐VCH. i) Schematic of the sensor and its responses in capacitance and resistance upon ii) pressure and iii) strain loading. b) Supramolecular E‐skin to decouple strain and temperature. Reproduced under the terms of the Creative Commons CC BY License.^{[ 97 ]} Copyright 2018, The Authors, published by Nature Publishing Group. i) Schematic of the sensor and its responses in capacitance and resistance. ii) Capacitance–strain curve of the sensor at different temperatures. iii) Resistance changes of the sensor upon varying strain and temperature. c) Linear bimodal pressure and temperature sensor^{[ 98 ]} with its i) schematic diagram and ii) simultaneous measurement of temperature and pressure. Reproduced with permission.^{[ 98 ]} Copyright 2018 Wiley‐VCH. d) Tactile E‐skin based on triboelectric and thermoresistive effects. Reproduced with permission.^{[ 99 ]} Copyright 2020 Elsevier Ltd. i) Schematic of the tactile sensor. ii) Relationship between the electrode resistance and temperature under different bending angles. iii) Thermal sensitivities of the sensor under different bending angles. Output voltage versus the applied pressure at iv) 40 and v) 60 °C, showing the temperature insensitivity. e) Dual‐parameter sensor with thermoelectric and piezoelectric sensing mechanisms. Reproduced with permission.^{[ 100 ]} Copyright 2019, The Royal Society of Chemistry. i) Schematic illustration of the thermoelectric and piezoelectric testing principles. Thermoelectric (red) and piezoelectric (black) responses of the sensor under ii) finger touch, iii) marker pen touch, and iv) non‐contact heating conditions. f) Mechanoreceptor based on a hybrid potentiometric‐triboelectric measuring principle. Reproduced with permission.^{[ 101 ]} Copyright 2020, Wiley‐VCH. i) Schematic of the mechanoreceptor and its ii) potentiometric, iii) triboelectric, and iv) hybrid modes.