Knowledge Based System for automatic Object Extraction (1999)

Funded by  German Science Foundation (Deutsche Forschungsgemeinschaft, DFG)

Cooperation with Institute of Communication Theory and Signal Processing
and Institute of Cartography

Research Group:  Automatic image analysis

Contact Person: Kian Pakzad

Background and goal
This project is part of the bundle project "Semantic Modeling and the Extraction of Spatial Objects from Spatial Objects from Images and Maps" funded by the German Science Foundation (Deutsche Forschungsgemeinschaft, DFG) supporting joint research of more than 10 research groups in Germany (see also Semantic_Modeling Homepage).

It was motivated  by    

• the need to foster cooperation between Photogrammetrists and Cartographers on one hand and researchers in Computer Vision and Pattern Recognition on the other hand.    
• the deficiencies in experience with automatic interpretation methods.    
• the practical demands in data acquisition from images and maps.

One aim of this project was the exploration of the potential of using explicit semantic knowledge for image and map interpretation.

The group of the University of Hanover examined all segments of this aim: The use of (1) explicit semantic knowledge for the interpretation of (2) images and (3) maps. For this the group consisted of three Institutes. Every one worked primarily on another segment: The Institute for Photogrammetry and Engineering Surveys concentrated on the interpretation of aerial images by using explicit semantic knowledge, the Institute of Communication Theory and Signal Processing worked on the development of the knowledge based interpretation system, which was used by the other two groups and the Institute of Cartography worked on the interpretation of maps, also by using explicit semantic knowledge.

Task und methods
The challenge of this project was to use explicit semantic knowledge for the interpretation. This have been done by using semantic nets as knowledge representation for the fact knowledge. Semantic nets consist of nodes and edges between the nodes. The nodes represent the objects expected in the scene while the edges or links of the semantic net model the relations between these objects.

The developed knowledge based interpretation system (AIDA (Automatic Image Data Analyser)) uses semantic nets as knowledge interpretation and works rule based. To make use of the knowledge represented in the semantic net control knowledge is required which states how and in which order the image interpretation has to proceed. The control knowledge is represented explicitly by a set of rules. The prior knowledge about the scene to be interpreted will be formulated in a concept semantic net. This concept net is used during the interpretation to build step by step an instance net, which provides at the end of interpretation a symbolic description of the scene.

The first task of the Institute of Photogrammetry was to do an interpretation of grayscale aerial images by using a Digital Landscape Model (DLM) as partial interpretation. For this the German DLM ATKIS Basis-DLM have been used to verify forest and river areas. During the interpretation the system took forest and river objects from the DLM directly through an developed automatic database connection. Because no information about the appearance of these objects (texture etc.) was provided, only the information about the position and shape of the objects at the acquisition time of the DLM, in a first step different texture parameters have been learned. After that this information was used to extract the true current areas of the object classes. Some results are depicted below (see Results).

In the second period of the project the task was to do an multitemporal interpretation of moorland from aerial images. Therefore CIR aerial images were used as well as grayscale images. The task was divided into parts: (1) The monotemporal interpretation of moorland and (2) the extension of the system to a multitemporal interpretation.

For the monotemporal interpretation moorland was divided into different relevant land use classes and described in a semantic net. For every class the obligatory parts were described. Obligatory parts are features and structures, which have to be found in the particular areas in order to assign the appropriate class to them. E.g. a class "area of peat working" was defined. For this class the obligatory parts are "harvester tracks" and "low vegetation density". The depiction of these parts in CIR aerial images are "parallel lines" for "harvester tracks" and "low NDVI-density" for "low vegetation density". During the interpretation special segment analysis operators verified the meaning of the obligatory parts in the aerial images.

Semantic net for monotemporal interpretation of moorland

The second part of the task was the extension to a multitemporal interpretation. The necessary temporal information for this part was formulated in a diagram, which describes the most probable state transitions. In the monitoring process after a monotemporal interpretation the state transition diagram was exploited to predict the possible land use changes. This led to a reduction of the search area and improved the multitemporal interpretation.

Results
The result of the DLM based forest verification shows, that the database boundaries, which are outdated and have a low accuracy, are updated correctly by using grayscale images. A particular area in the middle of the image has changed its land cover class and is correctly removed from the forest areas.

 
Result of DLM based forest verification
(left: forest in ATKIS Basis DLM, right: result of verification)

The results of the monotemporal interpretation using CIR aerial images led to correct results for the segments of our test area. Applying the interpretation on grayscale images led for the most segments to the same results. This is important because most existing aerial images of such areas are of grayscale type.

CIR-Images (1989), Resolution: 0.5m/pel

Result of Interpretation with color information

Below a result of multitemporal interpretation is depicted. As input data only grayscale images were used. The extension to a multitemporal interpretation enables the distinction between more land use classes, which could not be interpreted without the temporal knowledge. This applies e.g. to the distinction between the land use classes “area of regeneration” and “area of degeneration”. The depiction of both classes in aerial images can be very similar. To distinguish between these classes the temporal knowledge is necessary whether peat extraction had been carried out or not. The exploitation of this led also to a more robust interpretation of land use classes without color information. The use of temporal knowledge can therefore partly replace the need of color information.

Result of multitemporal interpretation

Links

Institute of Communication Theory and Signal Processing

Institute of Cartography

German Science Foundation (Deutsche Forschungsgemeinschaft, DFG)