High precision pose estimation of a UAV by integrating camera and laser scanner data with building models

Unmanned aerial vehicles or UAVs are being utilized more and more everyday due to their flexibility in surveying applications such as land inspection, monitoring, 3D reconstruction and also logistics, rescue missions and so on. In most scenarios, camera is the primary onboard sensor due to its flexibility and economic advantages. To be able to use the data gathered by the camera or cameras, the precise position and attitude of the sensor, also known as pose, is to be estimated. The aim of this project is the integrated, cm-precise and high frequency determination of the pose.

This is usually done using a sequence of overlapped images, the interior orientation parameters of the camera and additional data to define the scale and estimate the pose in a global coordinate system. This information can come from GNSS (Global Navigation Satellite System), GPS (Global Navigation System) being the most common example and an IMU (Inertial Measurement Unit). Aside from using additional sensors, one can set up GCPs (Ground Control Points) in the field. The coordinates of these points are known in the global coordinate system and so the sensor can be referenced in that system.

The GNSS/IMU solution faces difficulties in an urban environment, where the satellite signals may not reach the antenna in a direct way when the UAV is flying in a dense building zone. GCPs, on the other hand, are time consuming and costly to set out in the field. On this account, building models were chosen as ground control information for our urban scenarios. These models are freely available in level of detail 2 (LOD2) from the Lower Saxony Survey and Geoinformation Administration LGLN.

The processing starts with image matching and finding tie points to relate the images. subsequently, the data from images and building models are used in a so-called hybrid bundle adjustment to robustly estimate the pose. The novelty of this work lies in the addition of laser scanner measurements, which bring simultaneous data with cameras into this hybrid adjustment.

This project uses two cameras with non-overlapping views and a laser scanner. As a first step to use any multi-sensor system, calibration of the whole system is needed. assuming a stable platform, this system calibration is achieved by geometrically referencing and synchronizing the three sensors with respect to each other. In other words, the 6DoF or transformation and information about time synchronization for all sensors regarding each other must be determined. Figure above shows the platform mounted on the gimbal, which will be attached to the UAV DJI M600. The sensor platform was developed in cooperation with Stephan Schulz, Dept. of Mechanical Engineering and Production Management, HAW Hamburg.

This project is carried out in cooperation with the Geodetic Institute Hannover (GIH) and is funded by DFG.