Koushik A. Manjunatha, Morris Hsu and Rohit Kumar, Amazon Lab126, USA

The increasing popularity of wireless sensing applications has led to a growing demand for large datasets of realistic wireless data. However, collecting such wireless data is often timeconsuming and expensive. To address this challenge, we propose a synthetic data generation pipeline using human mesh generated from videos that can generate data at scale. The pipeline first generates a 3D mesh of the human in the video and then determines the initial synthetic Doppler of the human motion. The initial synthetic Doppler will be noisy due to the uncertainties involved during the mesh generation. To address this, we employ a trained Structured State Space for Sequence Modeling (S4) model to denoise the initial synthetic Doppler to match the Doppler signature from the radar device. We validated our pipeline on synthetic Doppler data from four hand gesture videos and found that the final synthetic Doppler closely resembles the real Doppler, outperforming existing U-Net-based models by 36% or more. Also, with smaller set of denoised synthetic Doppler, the gesture classification model performance increased from 89.5% to 92.7%.

Video, Mesh, mmWave Radar, Wireless, Artificial Intelligence