|
|||||||||||||||||||||||||||||||||||||||||||
|
Fusing Aerial Multispectral Imagery and High-Resolution Photography by M. Bajic, Faculty, Geodesy University of Zagreb and
Scientific Council, CROMAC Introduction Aerial remote sensing is useful for spotting indicators in the survey of mine-suspected areas and minefields.1, 2, 3, 4, 5 Because the detection of landmines in the soil and under the vegetation cover is not yet practical, the minefield indicators and their electromagnetic signatures provide means for detection of the mine-suspected areas and minefields. The minefield indicators can be natural (e.g., vegetation cover) over a large spatial extent, or else artificial or man-made (e.g., trenches, bunkers, demolished objects) over a small spatial extent.1 The dimensions of artificial minefield indicators require use of multi-spectral sensors and aerial images with spatial resolution below one meter. Due to a mixture of natural and artificial minefield indicators, aerial survey should provide wide-area coverage, with very high spatial and spectral resolution. Two different approaches were applied to resolve this conflict. One approach used very high spatial resolution sensors (in the range of five to 10 cm), four wavelengths (green [G], red [R], near infrared [NIR] and thermal infrared), and image acquisition at low heights above terrain (starting with 30 m).4 Due to a large number of images, a serious problem was mosaicking and fusing the images acquired by used sensors (spatial accuracy and matching the radiometry). Another approach used three different sensors: multispectral line scanner (12 channels), synthetic aperture radar (Experimental SAR, four wavelengths) and photogrammetric camera RMK (color infrared photography [CIR], [G], [R] and [NIR]).6 Both cases used digital sensors, while aerial photography was used in one of them, but only as an auxiliary source of ground truth and contextual information. A current paper reports preliminary results of the fusion of aerial multispectral imagery and very high-resolution photography by principal components transformation, aimed at detecting the minefield indicators that are otherwise invisible. Two parameters were used as a measure of benefit: Image Quality (IQ) and estimated National Image Interpretation Rating Scale (NIIRS) in accordance with those established by Nill and Bouzas.7 Both a multispectral line scanner (spatial resolution one m, wide area coverage three by four km, 12 channels) and CIR photography (spatial resolution three cm, area coverage rectangle with sides 450 to 500 m, 3 channels) analyzed images for the project. Analysis shows the possibility of combining these different images in an efficient manner and provides data and information about important artificial minefield indicators that are otherwise invisible. This was not foreseen in a previous study6 but will be applied in the last phase of the Space and Airborne Mined Area Reduction Tools (SMART) project as a new tool for anomaly detection. Fusing the Aerial Multispectral Images and Very High Spatial Resolution Photography Through Principal Components Transformation The fusion of the multispectral images and very high spatial resolution photography was made by principal components transformation and results are reported in this paper, although the intensity, hue and saturation (IHS) method was analysed as well. An overview of methods and architectures of data fusion8 and a novel approach given by Thomas and Wald9 served as general guidance in analysis. Because the goal was to improve the possibility and benefits of data fusion to detect artificial minefield indicators that are otherwise invisible, the fine analysis of spatial and spectral quality was left for the next phase of the research. Typically, one set of the multispectral images of the surveyed area covers an area three by four km. Although the project entailed some 90 very high spatial resolution photographs available for the area, processing time and memory resources decreased the number of photographs that were actually developed. The a priori data and information about existing minefield indicators serve to guide the selection of the relevant photographs.
The considered photographs have three layers: G, R and NIR. The first processing step is to calculate principal components of the G, R and NIR layers, while only the first principal component (PC1CIR) will be utilized in the next process. Generally, only a relevant part of the PC1CIR should be registered on the geocoded multispectral image. Parts of the multispectral layers (channels) common to PC1CIR were extracted and resampled to the pixel size of the PC1CIR. The resampled layers (channels) have small pixels (3.95 cm) while the ground resolution is one m. The next phase of processing (often named sharpening) calculates the first principal component of the resampled multispectral layers (channels) and PC1CIR. The new images (fused channels) have the spatial resolution of the PC1CIR, which is 3.95 cm. The images obtained by the above process (fused channels or layers) are designed to be a new multispectral set for further interpretation. The processing was done by the software TNTmips.10
Very High Spatial Resolution Aerial Photography The aerial color infrared CIR photography (Figure 1, above) was acquired at the same time, at the same height above the terrain and on the same airplane as multispectral line scanner images, but the mapping modes were different. The first principal component PC1CIR was calculated using G, R and NIR layers of the CIR image and PC1CIR was used in data fusion. PC1CIR is a gray-scale image.
Multispectral Geocoded Images The multispectral images were collected by line scanner in 12 channels in visible, NIR and thermal infrared wavelengths (Figure 2). They were processed by the use of the digital elevation model and thus they are ortho images.2 Combined PC1CIR and multispectral images are shown on Figure 3 as well. The current article presents results for seven selected wavelengths in Table 1.
Comparison of the Original and Fused Images Comparison of the original multispectral images (one-meter resolution) and fused multispectral images of all channels (pixels 3.95 cm) shows that data fusion provided improvement in interpretability of minefield indicators (Figure 4 and Figure 5). Two examples were selected to be presented in the current paper. The first scene consists of the asphalt road, field road, ploughed land, land covered by low vegetation and land covered by trees (Figure 4).
The landcover on Figure 4 is as follows: an asphalt road crosses the image from the upper left to the lower right corner. The land below this road is covered with several trees and bushes. On the upper right side above the asphalt road is ploughed land, while in the left upper side is land covered with low vegetation. A thin field road divides the upper part of the scene. The same classification was applied on original and on fused channels; results are presented in Figure 4b and Figure 4d. The difference is evident in favor of fused channels.
The second example contains the remnants of the demolished house, an important example of minefield indicators (Figure 5). The scene of the second example is a simple target: the remnants of the demolished house, surrounded by several trees and grassy land (Figure 5). The original images (one-meter resolution) show the target could be detected (Figure 5a) but not classified as demolished (Figure 5b). Fused images enable detection (Figure 5c) and classification (Figure 5d) of the target as a demolished house. This example demonstrates the benefit that is enabled due to proposed fusion by principal component transformation. Improvement of the IQ and NIIRS The fusion of multispectral images (one-meter resolution) and the derivative of the aerial photography of very high spatial resolution (resolution of the original is three cm; derivative has pixels of 3.95 cm) indicated the benefits regarding the detection and classification of artificial minefield indicators. While further development and operation of the reported method are under way in project SMART, the preliminary quantitative analysis was performed by method7 and results are shown in Figure 6 and Figure 7. The greatest increase of IQ shows fused channels 6 and 7. The increase of NIIRS is significant and is more visible for channels 1 to 8. The increase of NIIRS is lowest for channels 10 and 12 in thermal infrared wavelengths.
Conclusion The preliminary results validated the benefits of fusing the aerial multispectral imagery and very high-resolution photography using principal components transformation to detect the minefield indicators that are otherwise invisible. IQ and estimated NIIRS were used as a measure of benefit. Both a multispectral line scanner (spatial resolution one meter, wide-area coverage three by four km, 12 channels) and CIR photography (spatial resolution three cm, area coverage rectangle with sides 450 to 500 m, three channels) analyzed images for the project. Analysis shows the possibility of combining these different images in an efficient manner and providing data and information about artificial minefield indicators that are otherwise invisible. *All figures courtesy of the author. Acknowledgements Research was conducted in a frame of the project EC IST SMART, 2000, SMART, EC IST-2000-25044. Discussion with Prof. L. Wald in May 2004 about different approaches to data fusion initiated the reported analysis. Endnotes
Contact Information Prof. D. Sc., Milan Bajic |
|||||||||||||||||||||||||||||||||||||||||||
