Automatic extraction of road seeds from high-resolution aerial images

EARTH SCIENCES

Automatic extraction of road seeds from high-resolution aerial images

Aluir P. Dal-Poz^I; Giovane M. Do Vale^II; Rodrigo B. Zanin^II

^IDepartamento de Cartografia - FCT/UNESP, Rua Roberto Simonsen, 305, 19060-900 Presidente Prudente, SP, Brasil

^IIAluno de Doutorado do Programa de Pós-Graduação em Ciências Cartográficas-FCT/UNESP Rua Roberto Simonsen, 305, 19060-900 Presidente Prudente, SP, Brasil

^{Correspondence}

ABSTRACT

This article presents an automatic methodology for extraction of road seeds from high-resolution aerial images. The method is based on a set of four road objects and another set of connection rules among road objects. Each road object is a local representation of an approximately straight road fragment and its construction is based on a combination of polygons describing all relevant image edges, according to some rules embodying road knowledge. Each one of the road seeds is composed by a sequence of connected road objects, in which each sequence of this type can be geometrically structured as a chain of contiguous quadrilaterals. Experiments carried out with high-resolution aerial images showed that the proposed methodology is very promising in extracting road seeds. This article presents the fundamentals of the method and the experimental results, as well.

Key words: road extraction, road objects, road seeds, digital photogrammetry.

RESUMO

Este artigo apresenta uma metodologia automática para extração de sementes de rodovia a partir de imagens aéreas de alta resolução. O método se baseia em um conjunto de quatro objetos de rodovia e em um conjunto de regras de conexão entre tais objetos. Cada objeto de rodovia é uma representação local de um fragmento de rodovia aproximadamente reto e sua construção é baseada na combinação de polígonos que descrevem todas as bordas relevantes da imagem, de acordo com algumas regras que incorporam conhecimento sobre a feição rodovia. Cada uma das sementes de rodovia é composta por uma sucessão de objetos de rodovia conectados, sendo que cada sucessão deste tipo pode ser geometricamente estruturada como uma cadeia de quadriláteros contíguos. Os experimentos realizados com imagens aéreas de alta resolução mostraram que a metodologia proposta é muito promissora na extração de sementes de rodovia. Este artigo apresenta os fundamentos do método, bem como os resultados experimentais.

Palavras-chave: extração de rodovia, objetos de rodovia, sementes de rodovia, fotogrametria digital.

INTRODUCTION

Road extraction is of fundamental importance in the context of spatial data capturing and updating for GIS (Geographic Information Systems) applications. Substantial work on road extraction has been accomplished since the 70's in computer vision and digital photogrammetry, with pioneering works by, e.g., Bajcsy and Tavakoli (1976) and Quam (1978). At times, the use of the term 'extraction' is vague, invoking varied meaning among a diverse image analysis community. In this context, the task of road extraction is related to two subtasks, i.e.: recognition and delineation. By convention, road extraction algorithm is categorized according to the extent to which it addresses either subtask, thereby implying the relative level of automation (Doucette et al. 2001). Usually road extraction methods, which in principle do not need human interaction, are categorized as automatic and the opposite as semi-automatic. Thus, automatic methods address both road extraction subtasks and semi-automatic methods address only the geometric delineation of the roads, leaving the high-level decisions (i.e., the recognition) to a human operator, who uses his natural skill to set the meaning to the object 'road'.

With regard to semi-automatic methods, probably some existing ones can already be used into operational workflows. Semi-automatic approaches may be divided into two broad categories. The first includes road-following approaches, in which the road is sequentially traced by using only local road information (McKeown and Denlinger 1988, Vosselman and de Knecht 1995, Dal-Poz and Silva 2002). These approaches are usually initialized by two close seed points on the road, being one a starting point and another a point to define the road direction. The second category includes active contour models (Kass et al. 1987, Neuenschwander et al. 1997, Grüen and Li 1997, Agouris et al. 2000), simulated annealing (Trinder et al. 2000), and dynamic programming optimization (Merlet and Zerubia 1996, Grüen and Li 1997), in which some type of simultaneous curve fitting is used. Usually, these approaches are initialized by a few seed points describing coarsely the road.

Until now, fully automatic methods for road extraction seem to be faraway from a mature state and, consequently, no such operational system is expected to be available in a near future. Fully automatic methods attempt to completely circumvent human intervention during the extraction process, in which two basic steps can be identified:

1. Road finding or road seed extraction. The whole area of interest is tested for local road property, and road segments (i.e., road seeds) are found (Zlotnick and Carnine 1993). Usually, the local road properties tested are geometric (e.g.: roads are smooth) and radiometric (e.g.: roads are usually lighter than the background) in sense. The results of this step are the road seeds or a fragmented road network.

2. Road network completion. This requires a skilful integration of contextual information (i.e., relations between roads and other objects like trees and buildings) and other a priori road knowledge into the road extraction methodology (Baumgartner et al. 1999). These high-level information are used in strategies to automatically complete the road network, including the ones to bridge gaps between road seeds and to extract road crossings. At the end of the road extraction process, the goal is to obtain a complete road network, both in geometrical and topological senses.

This paper only addresses the first phase of process for fully automatic road network extraction. The motivation is the fundamental importance of road seeds for the subsequent phase. The potential success of road network completion strategies is significantly determined by the quality of the results of the first phase. This paper is organized in four sections. Section 2 presents the proposed methodology for automatic road seed extraction, which is essentially based on radiometric and geometric road constraints. Preliminary results are presented and discussed in Section 3. Conclusion and future perspectives are provided in Section 4.

METHODOLOGY FOR AUTOMATIC ROAD SEED EXTRACTION

OVERVIEW

We propose a methodology for road seed extraction which is based on a set of four road objects and another set of connection rules among them. Each road object is a local representation of an approximately straight road fragment and its building is based on a combination of polygons describing all relevant image edges. The polygons representing image edges are extracted using well-known image processing algorithms as, e.g., the Canny edge detection method followed by an edge linking algorithm and the split and merge algorithm (Parker 1997). After the polygon extraction some steps are needed to build road objects. Firstly, two polygons that are candidates for representing edges of a same road are selected. Now taking one polygon as reference, the algorithm tries to combine their straight-line segments to the ones of other polygon. Two straight-line segments of the different polygons are accepted to form a road object when: 1 - only two endpoints of them are orthogonally projected onto opposite straight-line segment, whose results are shown as circles in figure 1; 2 - they are close to parallel and the distance between them is compatible to the road width (figure 1); and 3 - the region between them satisfies road knowledge as, for example, road surface is usually homogeneous, road surface has good contrast to its adjacent areas, road width does not change much, road edges are anti-parallels etc. In other words, as stated before a road object is a representation for an approximately straight road fragment. Moreover, road objects can be classified in four types, but additional details are given in the following sections. The road objects are sequentially connected to each other according to a rule set, allowing a road seed to be formed (Figure 1).

In the following, the extraction of road objects and the way they are combined to construct road seeds are described with enough details.

EXTRACTION OF ROAD OBJECTS

The road objects are defined using straight-linesegments belonging to two different polygons with characteristics that are compatible to a road.

Figure 2 shows four road objects found in any road seed. In the building of a road object, by convention the inferior straight-line segment is called base and the superior one is called candidate. For each road object case, the endpoints of both straight-line segments (base and candidate) are orthogonally projected from one to each other, resulting only in two points projected between endpoints. For example, in the figure 2(a) the endpoints of the candidate straight-line segment are projected into two points of the base straight-line segment. The opposite occurs with case 2 (figure 2(b)). In relation to cases 3 and 4, as respectively illustrated in figures 2(c) and 2(d), only one endpoint of a straight-line segment is projected between endpoints of other straight-line segment, and vice-versa. In all cases, two end points belonging to the base and/or candidate straight-line segment and two projected endpoints are combined to build quadrilaterals very similar in shape to rectangles. Each road object gives rise to a quadrilateral, being each one identified as crosshatched area in the figure 2. The axis of each quadrilateral coincides with a short road centerline.

The building of the four road objects is based on a rule set constructed from a priori road knowledge. The main rules used to identify and build road objects are described below:

1. Anti-parallelism rule: According to this rule, two image gradient vectors taken at two opposite road edge points, and belonging to the same road cross-section, are in approximately opposite directions. Beside this, they are approximately orthogonal to the road edges. This also means that if the road edges are approximated by polygons, the image gradient vectors, computed at edge pixels fitted to each straight-line segment (of a polygon), are approximately parallel to each other. Thus, a compact and effective representation for the image gradient vectors, computed for each straight-line segment, is the mean image gradient vector. This mean vector embodies an important road property (i.e., the anti-parallelism) and its compactness facilitates the image analysis process for road seed extraction. The mean image gradient vectors for the base and candidate straight-line segments are anti-parallels. By the anti-parallelism rule, two straight-line segments,base and candidate, would be compatible to a road fragment if it were satisfied. As shown in the figure 3, which is an illustrative example, several steps are necessary to compute the average image gradient vectors, i.e.: edge detection (figure 3(b)), vectorization or edge linking (figure 3(c)), and polygonization (by using e.g. the splitting and merging algorithm (Jain et al. 1995)) and computation of the average image gradient vectors for all straight-line segments of the polygons (figure 3(d)). The average image gradient vectors are plotted (figure 3(d)) at the center points of the straight-line segment belonging to the polygons;

2. Parallelism and proximity rule: by this rule, two straight-line segments, base and candidate, would be compatible to a road object if they were approximately parallel and sufficient close to each other;

3. Homogeneity rule: the road pixel gray levels do not vary too much, at least within short road segments. Thus, the area inside each quadrilateral must be approximately homogeneous. Moreover, as a road is usually lighter than the background, the average gray level inside each quadrilateral is significantly higher than the background. To be accepted as compatible, two straight-line segments, base and candidate, must satisfy the homogeneity rule;

4. Contrast rule: roads usually contrast sharply with the background. By the contrast rule, a base and candidate straight-line segments are compatible to form a road object when a high contrast between the respective quadrilateral and its background is verified;

5. Superposition rule: a base and candidate straight-line segments are compatible only if two of their endpoints can be orthogonally projected onto each other. It is just this rule that gives rise to four cases of road object depicted in the figure 2. For example, in case 1 the two endpoints of the candidate straight-line segment are orthogonally projected onto the base straight-line segment, giving rise to the quadrilateral of road object of case 1;

6. Fragmentation rule: as roads are usuallysmooth curves, polygons composed by short straight lines are not usually related to roads. For examples, image noise can generate short and isolate polygons. However, parts of polygons with very short straight-line segments can be extracted from road crossing where the curvature is much more accentuated. Another case is related to very perturbed road edges (byshadow or obstruction, for example), which may give rise to many short straight-line segments connected to form a polygon. In these places, road objects are difficult to be formed, as the two first rules are hardly satisfied at all. Thus, cases involving short straight-line segments are not considered and possible extraction fails (for examples, road crossings not extracted) are left to be handled by other strategies, which are based on previously extracted road seeds and other road knowledge, as e.g. context - relation between roads and other objects like trees and building.

The order of application of the rules presented above is important, mainly when the base and candidate straight-line segments are incompatible. It can avoid, in most cases, the verification of all rules for road object construction. The first rule to be applied is the sixth as it allows parts or whole polygons potentially not related with road objects to be eliminated. The next rule to be applied is the fifth, avoiding the use of another set of rules in the case this rule is not satisfied. In the following, the order of rule to be applied is the 2^nd rule, the 1^st rule, the 3^rd rule, and the 4^th rule. A road object will be accepted if all rules are satisfied.

Figure 4 demonstrates how the road seed,shown in figure 1, is decomposed using the four road objects defined above. From the right to the left, the following cases can be identified: first, second, fourth, and again the last one. The inverse problem, i.e., the reconstruction of the road seed using the road objects, is described in the following section.

ROAD SEGMENT EXTRACTION BY GROUPING ROAD OBJECTS

As described above, the road objects are constructed by combining the base and candidate straight-line segments, which in turn belong to polygons representing all relevant image edges. Each road object is a local representation for the longest straight segment of a road seed. Thus, the problem we have in hands is how to connect the road objects to form the road seeds.

Figure 5 shows the possible connections to the left and to the right between the road objects. Figure 5(a) shows that 1^st case road object can connect to the left with the 2^nd and 3^rd cases and to the right with the 2^nd and 4^th cases. The 2^nd case road object (figure 5(b)) can connect to the left with the 1^st and 4^th cases and to the left with the 1^st and 3^rd cases. Note that the 3^rd and 4^th cases (figures 5(c) and 5(d), respectively) can connect themselves to both, left and right cases.

In order to construct a road seed by combining road objects, two polygons are selected and their straight-line segments are combined two-by-two and the resulting road objects are connected sequentially. The advantage of using the connection rules is that the construction of any new road object is limited to one or two cases (figure 5). The great problem of the polygon combination is the large search space if no heuristic is used. For high-resolution images, an efficient way for drastically reducing the search space is to use strategies based on the space scale, which allow the elimination of most part of previously extracted polygons (Baumgartner et al. 1999).

Figure 6 shows the combination of polygons representing the edges of a same road. One of the polygons being combined is labeled base and another one is labeled candidate. As also shown in figure 6, the base polygon can be compatible with more than one candidate polygon. It means that a base polygon needs to be combined with candidate polygons until the whole base polygon is combined. Assuming that every combination gives true results, parts or whole polygons (base or candidate) are removed from the search space after they are combined.

The application of the methodology described above to the illustrative example of figure 6 would allow to obtain the result shown in figure 7.

Supposing that the base polygon is combined from the right to the left (figure 7(a)), two road objects are constructed by combining the base polygon with the first candidate polygon. The connection between these two road objects generates a first road seed (figure 7(b)), which in turn can be decomposed into three consecutive quadrilaterals. The vertices of these quadrilaterals allow the definition of a short road seed centerline. Now the combination of the base polygon with the second candidate polygon generates the second road seed constituted by only one quadrilateral. The straight-line segment of the base polygon not integrating to both road seeds extracted, may be a useful information for further analyses of the reasons for the missing road seed. It is an evidence that both extracted road seeds are related anyway. For example, there could be a "T" or "Y" road crossing. Therefore, everything providing information on road seed extraction problem should be preserved for further use in automatic completion strategies of the road network.

Finally, figure 8 shows the final result obtained from the application of the road seed extractionmethod to the image presented in figure 3(a). The polygons corresponding to the road seeds extracted are overlaid on the image as black lines.

EXPERIMENTAL RESULTS

In order to evaluate the potential of the methodology for road seed extraction, three experiments with real image data are carried out. As this methodology is suitable for road seed extraction from medium- and high- resolution images of rural scenes, one image of medium-resolution and two of high-resolution are used. Baumgartner et al. (1999) classify the image resolution in context of road extraction strategies into three categories: 1 - low-resolution images, i.e., pixel footprint greater than 2m; 2 - medium-resolution images, i.e., pixel footprint ranging from 0.7m to 2m; and 3 - high-resolution images, i.e., pixel footprint lesser than 0.7m.

The first experiment is carried out with amedium-resolution image (500 × 500 pixels), in which the main roads manifest as ribbons with 6-pixel width. Figure 9 shows this image and the results overlaid on it. Parts of polygons used to construct the road seeds and the centerlines of road seeds are overlaid on the input image. In this experiment, an average road width of 6 pixels is used to set up the road objects, meaning that only main road seeds are expected to be extracted. The results obtained can be considered close to the expected one. Most fails are expected due to the missing edges and other anomalies along both roads. For these places and especially for the road crossing no road objects can be constructed and, as a result, a fragmented road network, composed by isolate road seeds, is extracted. Two false negatives pointed out by short arrows are unexpected as they could be extracted. Probably, the fail closer to the road crossing is resulted from the low contrast around that road region. Two isolate and short road seeds are extracted along secondary roads, which are circled on the input image. These road seeds could be considered false positive, as the road objects are set up to the pattern of the main roads. In general, the method allows long road seeds, along main roads, to be extracted, facilitating the posterior automatic completion of the main road network. In terms of completeness, about 80% of the main road network are extracted.

The second experiment (figure 10) is carried out with a high-resolution image (498 × 535 pixels), in which the roads manifest as wide ribbons with 40-pixel width. Figure 10 shows that the results reflect those theoretically expected. In fact, due to the incompatibility with any road objects, the road crossing and few road parts, perturbed by illumination posts, are not extracted. These fails are needed to be treated by specific strategies embody other types of road knowledge, like the ones based on context and scale space (Baumgartner et al. 1999). Note that the vegetation edges adjacent to road edges do not cause false positives, showing the robustness of the method in these situations. The road crossing is not extracted as no road object can be constructed for that road region. As in previous experiment, the completeness is also about 80%.

The third experiment (figure 11) is also carried out with a high-resolution image (272 × 1168 pixels), but with lower resolution when compared to the resolution of the image used in the previous experiment. The test image of current experiment shows a road with mean road width of 20 pixels. The results obtained by the road seed extraction methodology are shown to be very satisfactory for the extraction of the two straight segments of the road, as long road seeds are recognized and delineated, especially the left one (which is approximately three times longer than the other road seed). These results are obtained as the road edges are well-defined along those regions. The opposite occurs with the curved segment of the road along which road edge anomalies are presented. As a result, many short road seeds are extracted in this road region, bringing additional difficulties for further reconstruction of that curve road segment. The completeness is about 80% as well.

CONCLUSIONS AND FUTURE PERSPECTIVES

This paper presented an automatic method forthe road seed extraction from medium- and high-resolution images of rural scenes. The innovation in the proposed methodology is the way the road objects and the connection rules between them are defined.

With a purpose of evaluating the method’s potential in extracting road seeds, three experiments were conducted using three test images: two of high-resolution and one of medium-resolution. In all cases the results obtained could be considered satisfactory. In most cases, they are in accordance to the ones theoretically expected. Some little disconnections are expected, as the edge detection is sensible to the irregularities along the road margins. As a result, no road objects can be constructed for such road parts, giving rise to the missing road segments. Also due to incompatibility with any road objects, road crossings have not been extracted by the proposed methodology. Some false positives have been also expected. Despite these theoretical expected fails, the methodology was able to extract about 80% of the road network. The automatic road network completion methodologies will be the focus of our future researches, whose basic input will be the road seeds extracted by the proposed methodology.

ACKNOWLEDGMENTS

This research has been supported by Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP), grant number 2001/01168-5, and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), grant number 301114/2003-0.

Manuscript received on June 18, 2004; accepted for publication on January 5, 2005; presented by ALCIDES N. SIAL

• AGOURIS P, GYFTAKIS S AND STEFANIDIS A. 2000. Uncertainty in Image-Based Change Detection. In: Accuracy, Amsterdam, Netherlands, p. 1-8.
• BAJCSY R AND TAVAKOLI M. 1976. Computer Recognition of Roads from Satellite Pictures. IEEE Trans Syst Man Cybern 6: 76-84.
• BAUMGARTNER A, STEGER C, MAYER H, ECKSTEIN W AND EBNER H. 1999. Automatic Road Extraction Based on Multi-Scale, Grouping, and Context. Photogramm Eng Remote Sensing 66: 777-785.
• CANNY JA. 1986. Computational Approach to Edge Detection. IEEE Trans Pattern Anal Mach Intell 8: 679-698.
• DAL-POZ AP AND SILVA MAO. 2002. Active Testing and Edge Analysis for Road Centreline Extraction. In: Int Arch Photogramm Remote Sensing, Graz, Germany, p. 44-47.
• DOUCETTE P, AGOURIS P, STEFANIDIS A AND MUSAVI M. 2001. Self-Organized Clustering for Road Extraction in Classified Imagery. ISPRS J Photogramm Remote Sensing 55: 347-358.
• GRÜEN A AND LI H. 1997. Semi-Automatic Linear Feature Extraction by Dynamic Programming andLSB-Snakes. Photogramm Eng Remote Sensing 63: 985-995.
• JAIN R, KASTURI R AND SCHUNCK BG. 1995. Machine Vision. New York: MIT Press and McGraw-Hill, 549 p.
• KASS M, WITKIN A AND TERZOPOULOS D. 1987. Snakes: Active Contour Models. In: INTER CONF COMPUT VISION, Tampa, USA, p. 259-268.
• MCKEOWN DM AND DENLINGER JL. 1988. Cooperative Methods for Road Tracking in Aerial Imagery. In: COMPUT VISION PATT RECOG, Ann Arbor, USA, p. 662-672.
• MERLET N AND ZERUBIA J. 1996. New Prospects in Line Detection by Dynamic Programming. IEEE Trans Patt Anal Mach Intell 18: 426-431.
• NEUENSCHWANDER WM, FUA P, IVERSON L, SZEKELY G AND KUBLER O. 1997. Ziplock Snakes. Int J Comput Vision 25: 191-201.
• PARKER JR. 1997. Algorithms for Image Processing and Computer Vision. J Wiley & Sons, NY, USA, 417 p.
• QUAM LH. 1978. Road Tracking and Anomaly Detection in Aerial Imagery. In: IMAGE UNDERSTANDING WORKSHOP, London, UK, p. 51-55.
• TRINDER JC, MAULIK U AND BANDYOPADHYAY S. 2000. Semi-Automated Feature Extraction Using Simulated Annealing. In: INT ARCH PHOTOGRAMM REMOTE SENSING, Amsterdam, Netherlands, p. 905-911.
• VOSSELMAN G AND DE KNECHT J. 1995. Road tracing by profile matching and Kalman filtering. In: AUTOMATIC EXTRACTION OF MAN-MADE OBJECTS FROM AERIAL AND SPACE IMAGES, Munich, Germany, p. 265-274.
• ZLOTNICK A AND CARMINE PD. 1993. Finding roads seeds in aerial images. CVGIP: Image Understanding 57: 243-260.

Publication Dates

• Publication in this collection
  24 Aug 2005
• Date of issue
  Sept 2005

History

• Accepted
  05 Jan 2005
• Received
  18 June 2004