Symbol spotting for architectural drawings: state-of-the-art and new industry-driven developments

2.1 Pixel-based descriptors

One of the earliest pixel-based signatures (compact representations) of line drawing images was proposed in [11] based on the idea of the \(\mathcal {F}\)-signature, which is a particular histogram of forces. This signature calculates exerted attraction forces, based on an attraction function, between different segments of a symbol in each direction θ. This description is invariant to fundamental geometric transformations such as scaling, translation, symmetry, and rotation and is also able to handle symbol degradation to some extent. Although this descriptor is fast to implement with a computational complexity equal to \(\mathcal {O}(p.n)\), where n is the number of image pixels and p the number of digitalization steps of angles in the histogram, the descriptor does not lend itself easily to recognizing symbols in context, embedded in a floor plan.

In [12], geometric constraints such as length ratios or angles between pair of pixels considering a third pixel as the reference point are summarized via histograms. These histograms are used to determine the similarity between symbols. Considering the constraints makes the descriptor rotation- and scale-invariant and also robust to degradation. One issue is that the complexity of the resulting algorithm is roughly equal to \(\mathcal {O}(n^{3})\), and like \(\mathcal {F}\)-signature, it is not easily applicable to recognition in context.

The Blurred Shape Model (BSM) descriptor was first introduced in [13] for recognizing handwritten symbols and further developed in [14]. To define BSM using skeleton points, the images are partitioned into a grid of n×n equal-sized sub-regions (where n×n identifies the blurring level allowed for the shapes). Each bin of the grid receives votes from the shape points it contains and also from the shape points in the neighboring bins. The most interesting property of this descriptor is its robustness to elastic and non-uniform distortions. However, it cannot cope with geometric transformations like rotation and scale changes. According to the experiments in [13], BSM outperformed several other descriptors (e.g., Zernike descriptors) in terms of accuracy and scalability. In terms of computational complexity, for a region of n×n pixels, k≤n×n skeleton points are considered to obtain the BSM with a cost of \(\mathcal {O}(k)\) simple operations, which is faster than the compared descriptors. To make BSM rotational invariant, an extension named Circular Blurred Shape Model (CBSM) was proposed in [15, 16]. CBSM is also able to handle irregular deformations and is fast to compute (\(\mathcal {O}(k)\) for k contour points). Using correlograms (radial distribution of sub-regions of the image) and shape features derived from the Canny edge detector, CBSM is defined based on a spatial arrangement of pixels (contour map). Figure 3 shows the grid structures being used for BSM and CBSM. The correlogram grid of CBSM makes its rotation invariant when the final result is rotated and aligned around the main diagonal of the correlogram with the highest density. Results show that CBSM can outperform many other descriptors, including the original BSM, for multi-class object categorization problems. Although CBSM was successful in making BSM invariant to rotation, occlusion and noise can highly affect the location of the main diagonal of the correlogram. Moreover, the calculation of the scale change remains a challenge.

Some symbol descriptors are inspired from the concept of visual words in text indexing/retrieval approaches. This concept has been studied in many works related to video/image retrieval (see for instance [17–19]). In [20], a new descriptor, known as Shape Context for Interest Points (SCIP), is proposed in order to describe a graphic symbol using visual words. The shape context of a contour point is defined based on the bivariate histogram (distance and angle) of the relative coordinates of neighboring contour points. Since the number of contour points may be huge in real documents, shape context is only computed on key points (for instance obtained via the difference of Gaussian operator, or DoG). SCIP can adaptively reflect the local geometry of symbols based on their complexity and details and also guarantee invariance to scaling and rotation for isolated symbols. Unfortunately, the rotation and scale invariance is accomplished via a normalization step at the symbol level, and it is therefore difficult to apply the descriptor at the document level, i.e., for recognition in context. For this reason, an extension of SCIP (sometimes called ESCIP in some papers, such as in [21]) for the document level was introduced in [22]. In that paper, ESCIP is used to describe a document with a library of visual words. This allows for floor plans to be processed as text documents; thus, text retrieval/indexing techniques can be applied. Although the experimental results of this method are promising, false detections are a major issue, due to spatial relations between visual words being ignored and to an instability of the key point detection in the case of symbols composed of curves.

2.2 Vectorial descriptors

Considering the fact that architectural floor plans are man-made and digitally born, symbols are usually composed of regular primitives such as lines, arcs, and polylines. For this reason, a vectorial representation of symbols may seem appropriate to describe their shape. Typically, a vectorial representation of the meaningful parts of document images and symbols is first obtained via primitives, which are then further grouped via a suitable vector-based descriptor. We further categorize vectorial descriptors into graph-based and other, due to the prominent role of graph structures.

2.2.1 Graph-based

Graph structures constitute one popular way of utilizing vector-based descriptors. Although they cannot be considered themselves as descriptors, they constitute a flexible and powerful tool for describing the relational information found in architectural symbols. For that reason, we introduce here some related background information on graph structures typically used in the description phase for symbol spotting in architectural floor plan images. The main advantage of using graphs is their capability to represent individual symbols with variable size and complexity. This is because a graph size (number of nodes and edges) does not have to be set a priori and can be adjusted depending on the data. Moreover, graphs can simultaneously encode both the numeric properties and the structural data of a symbol. This latter property is what makes graphs popular in structural pattern recognition. To define nodes and edge attributes, scale and rotation invariant descriptors, in addition to relational information, are typically used in a way that makes the final description scale and rotation invariant, to avoid putting an extra computational load on the graph matching step. A drawback of using a graph structures for describing objects (or symbols) is that pattern recognition (or symbol spotting) is converted to a subgraph matching (or isomorphism) problem which is NP-complete. Another disadvantage of graphs is their sensitivity to noise, as noise can affect the adjacency and Laplacian matrices of the graphs, and thus may negatively affect the final performance [23]. Nonetheless, graph structures remain highly popular for vector-based symbol description and spotting. Figure 4 shows several graph-based representations of a bed symbol.

In [24], vectorial primitives are considered as the nodes of a graph called Fully Visibility Graph (FVG), and edges represent the visibility between every pair of primitives. Figure 4c shows an example of FVG for the bed symbol of Fig. 4a. Visibility here refers to the existence of at least one straight line between two points on the primitives such that the line does not touch any other primitive in the drawing. Perceptual grouping of the primitives can then be done by applying clique detection to FVG. In [25], a structural representation of line drawing images is proposed based on polygonal and quadrilateral approximations of the symbol contours. After this vectorization step, a structural graph is constructed based on different types of interactions between vectors in order to organize the extracted vectors more appropriately. More specifically, the graph is an Attributed Relational Graph (ARG) as it extracts topological and geometric features. Figure 4d shows an example of ARG for the bed symbol considering the attributes defined in Fig. 4e. In this figure, L, T, and S show an L junction, a T junction, and two successive lines, respectively. The relative geometric features considered in ARG allow for invariance to rotation and scale changes and also make the graph robust to small variations in the symbols.

ARGs have also been utilized in hybrid symbol recognition approaches in order to integrate structural and statistical methods. In [26], a symbol is described by a set of spatio-structural descriptors (vocabulary) associated with visual primitives such as corners, circles, thick primitives, and extremities. This description is mainly inspired from [27, 28], where both complete symbols and symbols with missing parts are recognized using their vocabulary and grammar. The ARG-based spatio-structural description in [26] combines both topological and directional information specific to the symbol. The label assigned to each ARG vertex is the class of the corresponding extracted vocabulary, while the edge label is defined based on the spatial relation between two endpoint primitives. Two extensions of this method are proposed in [29] and [30] to include global shape signatures of each vocabulary class in the constructed ARG. Given the fact that graph nodes have unique labels and all instances of one specific vocabulary type are merged into one single node, the graph matching step for symbol recognition is not NP-hard. However, the underlying graph matching problem requires the symbols to have at least two different types of visual primitives in order to compute the spatial relations. Thus, the method cannot handle symbols containing only one primitive type [31].

One of the difficulties when working with graphs is their sensitivity to the vectorization step. Structural errors in this step can easily result in spurious nodes and edges, and in some disconnections between different parts of the graph, yielding to poor results. To cope with this problem, a hierarchical graph representation known as Hierarchical Plausibility Graphs (HPG) is introduced in [32, 33] to cover different possible vectorizations. HPG is able to solve three kinds of distortions: split nodes, dispensable nodes, and gaps. Although HPG can address the distortion issue under these circumstances, heavy distortion is still a major concern. In addition, building an HPG is time-consuming and the hierarchical matching algorithm sometimes fails to find local optima, yielding erroneous solutions.

Another type of graph representation called Region Adjacency Graph (RAG) can be created using regions of the floor plan images (e.g., based on connected components) as graph nodes and connecting the adjacent regions via graph edges (Fig. 4f). Typically, the characteristics of the region boundaries and the relational information between regions are used to label graph nodes and edges. Examples include boundary strings [34], Zernike moments for nodes and relative scale and distance for edges [35, 36], and histogram of distances between border points and centroids [37]. The main drawbacks of RAG is that it only considers closed regions as components of symbols and can fail to represent a symbol well in case of discontinuities in the boundaries. This problem is especially important when working with real-word architectural floor plan documents.

To tackle the problem of RAG with closed regions, the Near Convex Region Adjacency Graph (NCRAG) is proposed in [38], in which regions do not need to be clearly and continuously bounded. NCRAG rather focuses on the convexity of the different parts of a symbol, as convexity is one key symbol property. Even though NCRAG improves upon RAG, it still has limitations with respect to small variations of symbol instances in the architectural drawing, which can cause erroneous splits or merges between neighboring regions.

Since graph matching is a NP-complete problem, [39] proposed a Bag-of-GraphPaths (BoGP) descriptor that finds all acyclic paths between any two connected nodes of a graph representation of symbols and then used Zernike moments to describe these paths. The BoGP descriptor inherits the rotation invariance properties of graph paths. It is also unique in the sense that the descriptor converts the graph matching step to a statistical problem, which can be solved via hashing methods.

3.1 Sliding windows and correlation matching

Searching the entire floor plan image via a sliding window (with or without overlap) performs an exhaustive search but is very time-consuming. In addition, depending on the robustness of the symbol descriptor, it may be necessary to search the image using windows with varying scales and rotations. Correlation-based methods like [49] which proposed a variation of the Hit and Miss transform for template matching, and some descriptors like CBSM [15] (see Section 2.1) and vectorial signatures [50] use this method to locate regions of interest (ROIs) in input images.

One way of avoiding the unreasonable computational complexity of exhaustive searches is to limit the search area to the most relevant parts of image. For instance, [11] applies a text string separation technique and implements the matching step on connected components. Like other matching methods relying on some form of pre-segmentation, such as connected components, or trying to detect ROIs first, there is an assumption that the symbols can be found as one entity within the components or ROIs, which is not necessary true.

3.2 Geometric hashing

Indexing methods constitute another popular approach for spotting symbols within architectural drawings; indexation usually involves encoding the symbols’ and input documents’ primitives via hashing functions in the form of lookup tables. The existing literature on primitive indexing methods usually follows the idea of geometric hashing introduced by [51]. In [52], the contours of the closed regions composing a symbol are described by a chain of adjacent polylines, then transformed into attributed cyclic strings. Two polylines are thus compared and matched by taking into account the different string edit operations needed to transform a string into another. To build an indexing lookup table for locating symbols, representatives of similar strings are used as indexing keys. Another example of hashing approaches consists in choosing a digit descriptor obtained from Cassinian ovals to describe symbols and building an indexing hash table [47].

In [53], the authors propose a structural approach for indexing vectorial drawings. In their method, a proximity graph, which is built based on extracted primitives from the image, is used to save not only the spatial relationship between primitives both also their numerical description in a 2D hash table. This approach, called relational indexation, allows for the consideration of spatial relationships between primitives in the retrieval process. Considering spatial relationships makes this approach invariant to scale and rotation transforms. One downside is that spatial information is only limited to the proximity (adjacency) between primitives, which might yield the same representations for different symbols.

3.3 Geometric matching

Nayef and Breuel in [54] use a geometric matching technique [55, 56] known as RAST (Recognition by Adaptive Subdivision of Transformation space) to search the space of the transformations for the most likely transformation parameters, and thus, the location of symbols in the document. In order to find matches of the features of a symbol model within a floor plan image, RAST performs a branch-and-bound search which yields a globally optimal solution. The authors employ either sample points along all lines in the drawings and symbol images, or segments and their orientation as feature points. They improved their geometric matching framework in [57] so that it is able to deal with vectorial primitives such as lines and arcs in addition to pixels. As an additional improvement, non-matched features are also penalized in order to decrease the number of incorrect matches. This approach has a relatively low precision when dealing with noisy or older documents, so a solution is presented in [58], which adds a post-spotting module in order to refine results. In this module, candidate regions obtained from the geometric matching algorithm are classified as true and false matches using a support vector machine (SVM) classifier.

3.4 Graph matching

As mentioned in Section 2.2, graph-based representations convert the problem of symbol spotting to a subgraph-matching problem; however, finding the optimal solution is not feasible since it is an NP-complete problem. Accordingly, several solutions have been proposed in the literature to find a reasonable solution. In [59], a new scoring function is proposed to evaluate the similarity of two different graphs. This scoring function employs a greedy graph matching algorithm to avoid the implementation of an exhaustive search.

In [60], an ARG representation is integrated with the graph matching technique proposed in [61] to build a complete symbol spotting framework. To decrease the computational complexity of the spotting step, a set of hypotheses are considered by the authors for detecting ROIs in the input architectural drawing images (or equivalently, detecting the parts of the corresponding graph that may include a symbol). This way, the graph matching only compares the model with ROIs, instead of the entire document. Although the extraction of ROIs can considerably decrease the computational complexity, the final performance of the algorithm is limited by the quality of this localization step. Indeed, the more comprehensive the hypotheses are, the better precision and recall can be expected, so the hypotheses should be properly defined in a way that no symbol will be missed, while being able to cope with occlusion and noise.

In [36], the authors propose a substitution-tolerant subgraph isomorphism to solve symbol spotting in technical drawings. Here, substitution tolerance means that the matching can cope with attribute differences such as vertex and edge labels, but unfortunately, a one-to-one mapping has to exist between each vertex and each edge of the template graph (symbol) and the target graph (floor plan); thus, structural distortions are not handled. In their approach, architectural floor plan images are described with a RAG, and subgraph isomorphism is modeled via an integer linear program (ILP) optimization problem. To generalize this approach for handling structural distortions, [62] proposes a binary linear program (BLP) which allows for the deletion of vertices and/or edges in the template graph. Unfortunately, there is no polynomial-time algorithm for solving ILP and BLP. The authors utilize Mathematical Programming (MP), which provides tools for solving optimization problems. In order to reduce the search space, these programs use a branch-and-bound algorithm along with some heuristics. Experimental results show better matching performance for BLP as well as faster solving time. The main drawback of these subgraph isomorphism approaches is that they remain unsuitable for large databases, since subgraph matching requires the investigation of the entire graph corresponding to the floor plan image for each symbol. Indexation techniques have thus the advantage over subgraph matching to be able to describe the input drawing just once in an offline process, allowing for a faster online querying. Another disadvantage of subgraph isomorphism approaches is that solving the optimization problem only yields the optimal solution in each implementation, which means that only the best match can be found in each iteration. The entire procedure must therefore be repeated several times for each symbol, ignoring the previous optimal solutions.

3.5 Graph conversion matching

To reduce the computational cost of graph matching, several works focus on translating graph characteristics into a statistical space and then use a statistical method for locating symbols, as opposed to structural approaches. We refer to this group of techniques as graph conversion matching.

Fuzzy Graph Embedding (FGE) is one of the conversion methods used for symbol spotting in [63, 64]. Graph embedding is a dimensionality reduction method which aims to exploit significant structural and statistical details of an attributed graph for embedding it into a feature vector (point) in a vector space. Feature vectors should reflect the similarity of corresponding graphs, i.e., the more similar two graphs are, the smaller the distance between their corresponding feature vectors should be. Graph embedding has the interesting advantage of enabling graph-based representations to access the whole range of statistical classifiers and machine learning tools. The resulting feature vector in the FGE approach is termed as Fuzzy Structural Feature Vector (FSFV), containing graph, node, and edge level features. Graph level features include graph order and size; node level features include fuzzy histograms of node degrees and of values taken by node attributes; edge level features include a fuzzy histogram of values taken by edge attributes. FGE employs fuzzy overlapping intervals for minimizing the information loss while mapping from continuous graph space to discrete vector space. The main drawbacks of this method are twofold: first, the presence of some imperfections in the representation of crossing lines or lines with angular points and second, a potential unwanted decomposition of a line into several quadrilaterals. The interested reader can refer to [23] for more information about graph embedding approaches.

Another alternative to graph matching is converting the geometric information carried by a graph to one-dimensional structures (serialization), which are less computationally expensive [65–67]. In these works, graph nodes are the critical points detected in the vectorized graphical documents and the lines joining them are considered as the edges. Serialization is accomplished by factorizing the graph into a set of all the acyclic paths between each pair of connected nodes. After this factorization, the paths are described by a proper descriptor and symbol spotting can be done through hashing the shape descriptors of the graph paths. The factorization step introduces a lot of extra paths which can help the algorithm in handling distortion and noise, but on the other hand, it creates a lot of redundant information. In addition, since critical points are highly sensitive to noise, the serialization process can affect the final performance.

6.1 Theoretical background

where f(x,y) is the pixel intensity value at coordinates (x,y) of the source image, t(x−u,y−v) is the pixel intensity value at coordinates (x−u,y−v) of the template image, and (x,y) are incremented to cover all pixel coordinates of the region of the source image where the template image is superimposed.

One advantage of cross-correlation is that it can be computed at once for all locations over the source. However, it is not rotation—nor scale-invariant, which means that different angulations or scales of the template image compared to the source image will typically yield different levels of similarity. An architectural symbol can appear at various angles and scales on a floor plan. It is therefore necessary to compute the cross-correlation between the template image and the source image several times, each time with a modified template image (scaled and/or rotated), to cover all possibilities. Those modifications are considered as geometric transformations and are similar to those used in the geometric matching proposed by Nayef and Breuel [57], with the difference that translation transformations are not required.

While standard cross-correlation as in (1) allows for finding locations in a source image where matching with a template image is maximal, it does not allow for making a decision as to whether or not an object corresponding to the template has been located. As we cannot assume that there will be one and only one instance of the template in the source image, we cannot simply detect symbols based on the maximal cross-correlation coefficient. Moreover, cross-correlation coefficient values can range anywhere from 0 to an undetermined number, depending on the contents and the size of the template image. As template images can vary largely in contents and size, from one type of architectural symbol to another, standard cross-correlation with no known maximal coefficient value is of little use. Normalizing the coefficient values, between 0 and 1, would allow to make a decision based on a set threshold: for instance, we could consider that all coefficients that are larger than 0.75 indicate the location of a match. If a source image would yield coefficients that are all below this threshold, then no match would be found.

where \(\overline {t}\) is the mean value of the template image, and \(\overline {f}_{u,v}\) is the mean value of the source image in the region covered by the template.

One issue that might arise from (2) is that an instance of an architectural symbol on a floor plan might be connected to other elements of the floor plan, or might be overlapping other elements of the floor plan. In this scenario, to which we can refer as “real-life clutter,” the extraneous items that are part of the region of the source image (floor plan) that is covered by the template image will influence the normalization process as they will be taken into account in the mean values. Therefore, an instance of a symbol on a floor plan that is isolated will yield a different (higher) coefficient than an instance of the same symbol that is not isolated. Selecting a proper threshold to make a decision as to whether or not an instance is considered as matched might be difficult in this case, as the more clutter present, the lower the coefficient. Too low a threshold would allow for the detection of non-isolated or cluttered instances but would also probably cause many false detections, while a higher threshold might work better in term of not detecting incorrect instances but miss non-isolated or cluttered correct instances.

6.2 Proposed clutter-tolerant cross-correlation

With this formulation, only the “ON” pixels of the template count towards the computation of the correlation, which makes it clutter-insensitive. The computational complexity of (3) is the same as that of the standard cross-correlation (1). One drawback of this approach is that “ON” pixels of the template image can match “ON” pixels of the source image, whatever the shape of the elements contained on the source image. For instance, if the source image contains a solid region of “ON” pixels that is larger than the template—which rarely happens in architectural floor plans, then the template will yield matches over the solid region. This drawback can be addressed through pre-processing if necessary.

Our clutter-tolerant cross-correlation function bears similarities with the morphological-based Hit-or-Miss Transform extension proposed in [49] to deal with overlapping elements, but is faster and simpler in completely discarding template background information. Another important difference is that in [49], the authors change the threshold value for every single symbol query, while our method allows us to use the same correlation threshold value not only for all symbol queries, but also for all images (see Section 6.4 and Figs. 7 and 8).

6.3 Algorithm

Utilizing the proposed cross-correlation function (3), architectural symbols can be spotted in digital architectural floor plan images according to Algorithm 1. Steps 1 to 6 can be viewed as a representation/description phase, steps 7 to 11 as a spotting phase, and steps 12 to 15 as simple post-processing.

6.4 Experimental results

We evaluated our proposed method, implemented in MATLAB, on the architectural dataset used in the most recent edition of the GREC Symbol Recognition and Spotting Contest [5] (see Sections 1 and 4). The rationale for using the contest dataset as opposed to SESYD as some other methods do (see Table 3) is that the contest dataset is finite and well-defined. The test set in the architectural domain is comprised of 20 different synthetic floor plan images that contain symbol instances from 16 symbol models. Symbol instances can appear in the floor plans at any orientation across various scales. Various noise levels have been added to the 20 images to generate four subsets of 20 images each with varying degradation (ideal, level 1, level 2, and level 3).

We present the performance of our approach considering the correlation threshold TH as its main tunable parameter. Table 4 shows our results on the GREC contest dataset [5] for various TH values for the ideal case and compares them with Nayef and Breuel’s method [57], the sole participant in the contest. The rationale for focusing here on the ideal case for evaluation purposes is that it is representative of digitally born architectural floor plan documents, the industry standard. In Table 4, the precision (P), computed pixel-wise, represents the proportion of the detected symbol pixels that are part of actual symbols. The recall (R), also computed pixel-wise, represents the proportion of the actual symbol pixels that are detected as such. The F-score (F) is a combination of the precision and the recall (harmonic mean) into one single measure. The recognition rate (RecR), computed symbol-wise, is the percentage of symbols in the ground truth that are correctly identified. As per the contest instructions, a detected symbol is considered as recognized if there is an overlap of at least 75% between its area and that of the actual symbol in the ground truth. The average false positives (AveP), computed symbol-wise, is the average number of detected symbols with no correspondence in the ground truth per image over the test set. Figure 9 plots the precision-recall curve, showing the trade-off between a high recall and a high precision, with the best combination (highest F-score) highlighted.

Method		P	R	F	RecR	AveP
Proposed	TH = 0.950*	0.989	0.947	0.968	97.00%	0.00
	TH = 0.925	0.986	0.962	0.974	97.60%	2.40
	TH = 0.900	0.985	0.977	0.981	98.30%	2.60
	TH = 0.875	0.983	0.992	0.987	100.00%	3.60
	TH = 0.850	0.966	0.992	0.979	100.00%	4.30
Nayef and Breuel [57], as reported in [5]		0.620	0.990	0.760	99.31%	18.75

^*TH = correlation threshold

From Table 4, we can see that our proposed approach yields excellent results (high precision, high recall, high F-score, high recognition rate and low average false positives) and significantly outperforms the contest participant [57] in terms of precision, F-score, and average false positives, by 36 p.p., 22 p.p., and 15 fewer false positives on average, respectively (with similarly high recall and recognition rate). Interestingly, as the correlation threshold TH decreases from 0.95 to 0.85, the precision monotonically decreases and the recall monotonically increases, which adequately reflects the “strictness” of the threshold. Indeed, a higher, stricter threshold of 0.95 yields fewer albeit better detections, while a lower, looser threshold of 0.85 allows us to detect more symbols at the price of more false positives. The F-score is highest for a correlation threshold TH of 0.875, as can be seen in Fig. 9. The recognition rate monotonically increases as the threshold decreases, reaching the theoretical maximal value of 100% at TH = 0.875, also corresponding to the best F-score. The average false positives monotonically increases as the threshold decreases and is minimal (0) only at TH = 0.950, i.e., for the strictest threshold considered in the experiments. The monotonic behavior of the proposed algorithm, with respect to the correlation threshold value, is a highly desirable feature.

Figures 7 and 8 show sample visual results obtained using the best correlation threshold TH of 0.875, covering all five different layouts available in the dataset. Figure 7 illustrates ideal cases in which all symbol instances were correctly detected with no false positives, while Fig. 8 illustrates less-than-ideal cases where all true symbol instances are detected along with a few false positives. In the latter, all false positives (six in Fig. 8b, two in Fig. 8c, and three in Fig. 8d) reflect the downside of the clutter tolerance feature. The positioning of the symbols within the floor plans, as well as the floor plan structures, sometimes create regions that share similar features with a specific symbol to be spotted. For instance, in Fig. 8d, a kitchen sink symbol (sink2 or sink3 instances) is next to a wall, which creates a region that shares similar features with a specific window symbol (window1). A simple post-processing step looking at the proportion of ON pixels in the floor plan region covered by spotted symbols, i.e., its solidity, could be envisioned to discard such false positives.

Figures 10 and 11 show examples of symbol spotting on real-life architectural drawings used in industry to illustrate the claimed tolerance to clutter of our proposed approach. In Fig. 10, both examples are taken from a new construction project of a residential dwelling and contain an “intermediate” level of clutter. In the example on the left, the refrigerator (a) is correctly identified in (b) despite the additional layered information related to measurements and cupboards. In the example on the right, the two instances of the door symbol (c) are correctly identified in (d), despite the presence of layered text over one of the instances. In Fig. 11, both examples are taken from a renovation project of an old heritage building. Renovation project drawings typically contain a lot more layered data than new construction projects, making them ideal to showcase the clutter tolerance of our algorithm. Those additional data (notes, hatch lines, etc.) are necessary to indicate to the builder what to do with the existing conditions of the site. For instance, one pattern of hatch lines represents flooring infill, which indicates to the builder to patch/repair/level the floor in this area. Another pattern of hatch lines represents an area of a dropped ceiling to conceal piping from the unit above. In the example at the top of Fig. 11, bi-fold closet doors (a) are correctly identified in (b) in spite of the clutter that includes hatch lines at the same angle as one side of the closet doors. In the example at the bottom, two nightstands (lamp on top of a stand) (c) are correctly detected in (d) even though many different types of inscriptions and objects are in partial overlap with symbol instances. All symbol queries in Figs. 10 and 11 (a, c) are very different in terms of shape, topology, and symmetry, which also illustrates the genericness of the proposed method.

The proposed method has a number of advantages in addition to its monotonic behavior and simplicity. It is scalable both in terms of the number of symbol models and the number of symbol instances that it is able to recognize in a single pass. It does not require any off-line pre-processing of the architectural floor plans as in graph-based approaches (see Section 2.2.1). It is generic enough to be applicable to any symbol design and was designed to cope with clutter, which is a particularly important issue in real-world architectural floor plan images found in industry. One can also tune the correlation threshold TH to make the method less strict in its matching and tolerate some level of noise. Since our method is not inherently scale- nor rotation-invariant, several geometric transformations are necessary to spot all potential symbol instances. However, these transformations are easily integrated into the proposed spotting algorithm (see Section 6.3).