Construction of 540° Panoramic Image of Vehicles Based on Video Spatiotemporal Sequence Stitching

Abstract:
There are not only visual blind spots such as the front, rear, and A-pillars of the vehicle during driving, but also the environment under the vehicle chassis is also important for driving safety. Aiming at the problem that the traditional Around View Monitor (AVM) system only displays the environmental information around the vehicle body, a 540deg AVM system based on video spatiotemporal sequence image stitching is proposed. This paper aims to reproduce and display the blind spots around and at the bottom of the vehicle, based on image processing methods such as camera calibration, image registration fusion, and video sequence stitching. First, a group of fisheye cameras and wheel speed sensors are installed on the vehicle, and the perspective transformation matrix is calculated by calibrating the camera parameters and the relative position relationship of each camera, and then the original image is converted into a bird's-eye view. Then, the 360deg and 180deg panorama images are obtained by image stitching and fusion using the spatial relative position relationship of multiple cameras and video sequences at different moments from the same camera. Finally, the two are fused again to obtain a 540deg panoramic image. The experimental results show that the method can effectively eliminate the visual blind spot, display the surrounding environment of the vehicle without dead angle, and improve the driving safety.