Super-dense point clouds acquired by an ultralight 10 g solid-state single photon LiDAR

Abstract

Integration of photogrammetry and light detection and ranging (LiDAR) algorithms has garnered attention to enhance visual fidelity and geometric accuracy in three-dimensional (3D) modeling. Here we show an ultralight 10 g solid-state single-photon LiDAR that minimizes photon cost per measurement. Achieving a maximum indoor distance of 25 m and a point cloud density of ~3.16 Mpps, the LiDAR provides geometric fidelity, while maintaining fine structure information challenging for conventional LiDAR. Key embedded components include a Q-switching semiconductor laser, which emits a 50-ps pulse-width tail-free laser using bandgap renormalization. A four-channel time-to-digital converter achieves a 3 ps timing jitter per channel and features real-time time-walk error correction for Poisson-distributed photon counts. A low-Q two-dimensional (2D) MEMS mirror with a 20 mm² mirror size and precisely controlled feedforward-driven frequency enables non-repetitive scanning and super-dense point cloud generation. We present 3D modeling using the colored point clouds and discuss its characteristics and challenges.

Fig. 1. Principal and schematic of the ultralight solid-state single-photon light detection and ranging (SS-SPL).

a Simplified ranging principle. b Schematic of Lissajous trajectory generated by beam scanning of a 2D micro-electro-mechanical system (MEMS) mirror. c The large receiving aperture of 13.17 mm² comprises the large diameter 2D MEMS mirror and the freeform mirror. The transmitter (TX) unit comprises a single Q-switching semiconductor laser and small optics. The receiver (RX) unit includes the silicon photomultipliers (SiPMs), isolation transfers (~55 V on the input side, less than 1.0 V on the output side), and differential amplifiers (a part of the complementary metal-oxide-semiconductor (CMOS) chip). The four channels of the two-step time-to-digital converter in the CMOS chip are connected to the individual differential amplifiers. The field-programmable gate array (FPGA) is not included in the module. d Laser chip and CMOS driver are die-bonded on the small half cavity package with a volume of 0.033 cc (5.5 mm × 4.0 mm × 1.5 mm). e Ultralight module comprises TX on the left side, along with RX and time-to-digital converter (TDC) on the right side. The module comprises resin, resulting in a package volume of 11.7 cc (25 mm × 26 mm × 18 mm) and weight 9.97 g.

Fig. 2. Schematic and characteristics of edge-emitting, actively Q-switched semiconductor laser.

a The alternatively placed gain and Q-switch regions (52 pairs) in the cavity of 4 mm are electrically separated in the lateral direction. b A voltage (VLD) ranging from 4.45 to 5.04 V was applied to the gain region. Then, a pulsed current is input through a low-side n-type metal–oxide–semiconductor (NMOS) driver. Three states exist for the Q-switch anode: high impedance with parasitic capacitance, VDD (5 V) via a p-type metal–oxide–semiconductor (PMOS), or ground via an NMOS. Each timing is precisely controlled by a pulse signal generator embedded in the complementary metal-oxide-semiconductor (CMOS) driver with a least significant bit (LSB) equivalent to ~100 ps. c Streak camera (Hamamatsu Photonics C5680) images of the laser pulses at $\mathrm{\Delta }{t}_{\mathrm{lsd}}=3.17$ ns and$\mathrm{\Delta }{t}_{\mathrm{lsd}}=5.22$ ns. The rainbow scale represents the linear intensity. Notably, the time offsets observed in the images are induced by a jitter of the streak camera and do not reflect the actual timing of the pulsed laser oscillation. d Temporal waveform of the laser pulse at $\mathrm{\Delta }{t}_{\mathrm{lsd}}=5.22$ ns. e, f Peak power, average power, pulse width, wavelengths of pulse and tail, and ratio of tail depending on $\mathrm{\Delta }{t}_{\mathrm{lsd}}$. Pulse energy is calculated by dividing the average power by the repetition rate of 3.16 MHz. The peak power is determined from the laser pulse waveform. g Temporal waveform of seven laser pulses, demonstrating stable pulse generation. h Beam quality factor (M²) of a similar semiconductor laser not integrated into the module. Focal position shift is attributed to the large aspect ratio of the beam divergence angles. The distance from the collimator lens to an f = 40 mm lens for the M² measurement is ~250 mm. Therefore, the lens is positioned within the Rayleigh length for the vertical component and beyond the Rayleigh length for the horizontal component. The focal point along the horizontal axis is theoretically determined to be 47.6 mm, in agreement with the experimental results. For the module, the aspect ratio is adjusted using anamorphic optics.

Fig. 3. Characteristics of low Q-value micro-electro-mechanical system (MEMS) mirror and driver.

a Frequency response of optical scanning full angle for two samples. A 20 V peak-to-peak voltage is applied to lead-free piezoelectric actuators of the MEMS mirror to perform vertical scanning. b Optical scanning angle in the horizontal direction under the same conditions as the vertical direction. c Resonance amplitude decay scanned in vertical direction (blue), horizontal direction (red), and fitted curves (black). d Schematic for up-down direct digital synthesis (UD-DDS). e Sin wave generation by the UD-DDS. f Experimental frequency analysis of the UD-DDS. g Non-repetitive Lissajous trajectories. h Repetitive Lissajous trajectories. Both trajectories are captured at an exposure time of 198 ms. The field-of-view is 42° × 26° when a 28 V peak-to-peak voltage is applied.

Fig. 4. Characteristics of receiver (RX) and time-to-digital converter (TDC).

a Signal waveform measured at the node between the isolation transfer and the differential amplifier when the white paper is used as the target. b Distributions of time differences between Ch-1 and Ch-3, Ch-1 and Ch-4, and Ch-3 and Ch-4. c Probability $P\left(x\ge k\right)$ that more than $k$ photons are detected, depending on the distance to the target for single-photon (SP-mode) (black), double-photon (DP-mode) (red), and triple-photon (TP-mode) (blue). d Detection timing of Ch-4 against the difference of detection timing between Ch-4 and Ch-3 with the distance of 6.486 m, before (red dots) and after (blue dots) time–walk error correction. e Distance precision of single-shot after the time–walk error correction. Open circle represents the case corresponding to the minimum photoelectron for each mode. f Distance accuracy of the single-shot after the time–walk error correction. The values are calculated from 50,000 measurements at each distance.

Fig. 5. Validation of geometric accuracy in 3D point clouds.

Ground truth image (a) and 3D point clouds at distances of 3 m (b), 10 m (c), and 25 m (d). White lines on the point clouds indicate the outline of the ground truth. For each anisotropic voxel, assign a representative point at the center of the square cross-section (x, y) with z equal to the mean distance of the enclosed points. The number of points in an anisotropic voxel at the edge of the ground truth is 50% of that in the interior, and point clouds with values less than 30%, indicating regions outside the ground truth, are eliminated. The cross-section sizes are 2 mm, 4 mm, and 8 mm for distances of 3 m, 10 m, and 25 m, respectively. The target occupies 10.2, 0.92, and 0.15 % of the field of view for the three cases. e Dimensions of our original ground truth, (f–k). Means (black dots), standard deviations (gray lines), and the ground truth (red lines) along the cross-sections in x and y axis, as shown in the red dashed boxes in (a).

Fig. 6

Overview for reconstruction and fusion.

Fig. 7. 3D modeling by the fusion system and the solid-state single-photon light detection and ranging (SS-SPL).

a–c Stitched and colored point clouds of the building interior. Point clouds were captured indoors by the fusion system from different viewpoints and locations, with 114 shots stitched together to create the 3D model. Interior view of the restaurant (d) and fine texts on the white wall (e), including the ground truth on the left side and the point clouds on the right side. f–k Point clouds acquired outdoors by the fusion system under environmental illumination of ~3000 lx or 30,000 lx in single-photon (SP-mode), double-photon (DP-mode), or triple-photon (TP-mode). From left to right: the ground truth, the front view of the point cloud (same as the shooting direction), and the side view of the point cloud (perpendicular to the shooting direction). l Basketball shot photo sequence extracted from the side view of a point cloud video captured by the SS-SPL from behind a shooter.