Headway

Real-time computing

Telecommunications

Computer science

Sensor fusion

Simulation

Asynchronous communication

Synchronization (alternating current)

Artificial intelligence

Channel (broadcasting)

Radar

Computer vision

IEEE Transactions on Intelligent Transportation Systems

Roadside sensors, such as camera and millimeter-wave (MMW) radar, provide traffic information beyond the visual range of intelligent vehicles in cooperative vehicle-infrastructure systems. Unlike onboard equipment, roadside sensors are affiliated with different systems and lack synchronization in both space and time. In this paper, we propose a novel spatio-temporal synchronization method of asynchronous roadside MMW radar–camera for sensor fusion, which utilizes features of the scenario to extract lane line corner points to pre-calibrate the camera. Based on the consistent time flow rate of the separate sensors, multiple virtual detection lines are set up to match the time headway of successive vehicles and conduct objective matching to track data. Finally, a synchronization optimization model is formulated and a constrained nonlinear minimization solver is applied to tune the parameters. Measure data from Donghai Bridge in Shanghai is applied to verify the feasibility and effectiveness of the method. The results determine that there are 33 frames (33*40 ms) of temporal deviation between the camera and the radar in this case. After the synchronization, the average spatial deviation is reduced from 2.47 m to 0.42 m in the X-direction and 64.06 m to 2.34 m in the Y-direction, respectively. This study provides an economical and effective way to solve the problem of spatio-temporal synchronization of roadside sensors.

A Novel Spatio-Temporal Synchronization Method of Roadside Asynchronous MMW Radar-Camera for Sensor Fusion