Use of Mathematical Morphology in Vehicle Plate Detection

Neeraja Mohanan, Afaq Ahmad, Sayyid Samir Al-Busaidi, Lazhar Khiriji, Amir Abdulghani and Muhammad Al Nadabi

Department of Electrical and Computer Engineering, Sultan Qaboos University, PO Box 33, Zip code 123, Muscat, Sultanate of Oman.

Corresponding Author E-mail: afaqahmad51@gmail.com

DOI : http://dx.doi.org/10.13005/ojcst11.04.04

ABSTRACT:

In the past couple of decades, the number of vehicles has increased radically. A statistic which presents the number of cars sold worldwide from 1990 through 2017, forecasts for 2018, some 81.6 million automobiles are expected to be sold by the end 2018. With this continuous increase, it is becoming very tedious to keep track of each vehicle for the purpose of security, law enforcement and traffic management. This phenomenon of rapidly increasing vehicles on the road highlights the importance for a vehicle number plate recognition system. By recognizing the car plates, the drivers of the vehicle can be identified from the database. Number plate detection system are used in various applications like traffic law maintenance, traffic control, automatic toll collection, parking systems, automatic gate openers. This paper presents a unique algorithmic procedure for detecting vehicle plate number which is based on the concept of mathematical morphology. The developed algorithm is simple, efficient and flexible. The algorithm is capable of working satisfactorily even in different constraints such as like rain, smoke and shadow. This user-friendly software tool is developed on MATLAB platform which is one of the common and efficient image processing analysis tools.

KEYWORDS: Algorithm; Image Processing; MATLAB; Mathematical Morphology; Set Theory; Vehicle Number Plate Recognition

Copy the following to cite this article: Mohanan N, Ahmad A, Al-Busaidi S. S, Khiriji L, Abdulghani A, Nadabi M. A. Use of Mathematical Morphology in Vehicle Plate Detection. Orient.J. Comp. Sci. and Technol;11(4).

Copy the following to cite this URL: Mohanan N, Ahmad A, Al-Busaidi S. S, Khiriji L, Abdulghani A, Nadabi M. A. Use of Mathematical Morphology in Vehicle Plate Detection. Orient.J. Comp. Sci. and Technol;11(4). Available from: https://bit.ly/2THLGfQ

Introduction

In recent years, research in developing an efficient vehicle number plate detection system has attracted a lot of attention. Every country has a unique number plate. Every number plate has a unique number which helps to distinguish one vehicle from another especially when the vehicles are of the same color, make and model. A vehicle number plate detection system can be designed to identify the number plate of the vehicle and extract the characters from the number plate. For further processing, the characters of the number plate can be used to recover information about the vehicle and its owner. In government sectors, for regulating authorities, vehicle number plate detection systems can be used in law enforcement, border protection, vehicle thefts, automatic toll collection along with the traffic monitoring and control. While in private sectors, vehicle number plate detection systems can be used to control the access or denial of entries in restricted complexes or campuses. Also, automatic parking control and guidance along with surveillance of the vehicle can be managed using vehicle number plate detection systems.

The rapid growth of a number of vehicles on road raises the importance of the development of a robust number plate recognition system for enforcing efficient laws and traffic rules for a steady traffic flow. A vehicle number plate detection system comprises of many processes such as image acquisition, localization of the number plate region, segmentation of the characters and character recognition. The first step in any vehicle number plate detection system is image acquisition. Image acquisition must be done with a high-resolution camera in order to get a clarion image. After acquiring the car image, the next important task is the localization of the license plate from the car image. To accomplish this task, it is essential that the license plate should exhibit sharp rectangular edges. The next step in vehicle number plate detection system is that the characters on the license plate must be segmented for character recognition. As the characters on the number plate are segmented, the task ahead remaining is to recognize the characters.^1,2 Every country has its unique vehicle identification systems for its transportation engineering. The size, color and format of the number plates vary for various vehicle types.

Background

Vehicle number plate detection can be done using many image processing techniques. One such technique is the mathematical morphology. Mathematical morphology is a powerful tool that is used to extract image components which help in representation and description of a particular shape or region of an image. It is used to investigate the interaction between an image and a certain chosen structuring element.

The vehicle number plate detection is a crucial process as the images of the number plates may undergo deteriorations like different weather conditions, size variations, tilt in the number plate position, and damages to the number plate. Humans can easily read the number plates under these circumstances, whereas, it is difficult for a machine. Therefore, efficient algorithms need to be developed to recognize the number plates of different series, the differently classified number plates, and the number plates subjected to different weather conditions and damages. The developed algorithms will make use of mathematical morphological transforms like dilation, erosion, closing, opening, etc. During the mathematical morphological transforms, the structuring elements for each of the developed algorithms are to be selected through a trial and error method using a statistical approach. 

Many algorithms were developed for vehicle number plate detection systems. However, for this paper an algorithm using the combination of mathematical morphology for pre-processing and localization, bounding box technique for segmentation and template matching for character recognition was never developed. The MATLAB software tools are used for simulation, test and validation of the developed algorithms

Mathematical Morphology

Mathematical morphology is a powerful tool that is used to extract image components which help in representation and description of a particular shape or region of an image. Mathematical morphology is developed using the set theory. In image processing, mathematical morphology is used to investigate the interaction between an image and a certain chosen Structuring Element (SE). The interaction can be the set operations like dilation and erosion. Dilation and erosion are the basic set operations and all other operations are derived from dilation and erosion. One of the most significant contributions of mathematical morphology is the morphological filters which are closing and opening set operations.^{3, 4}

To perform any mathematical morphological operation, the input image should always be able to be represented as a set of points. Once an image is represented by a point set, then all the morphological operations on the images are set operations on those point sets. For binary images, the set represents the two-dimensional integer space Z², where both the components give the coordinate of the pixel. For gray-scale images, the sets represent the three-dimensional integer space, where two components give the coordinate of the pixel and the third component give the discrete intensity value of the pixel.⁵

It is impossible to directly apply a morphological set operation on an image. A structuring element is needed which help to apply the desired set operation to the input image under study to obtain the properties of interest. A structuring element is a shape mask used in the basic morphological operations. The structuring elements are always smaller than the input image. It has an origin and a shape.

There are two types of Structuring elements, Flat structuring element and Non-flat structuring element. Flat structuring elements are used for 2-D images whereas non-flat structuring elements are used for 3-D images. The parameters of flat structuring elements are length and breadth whereas the parameters of non-flat structuring elements are length, breadth and height. Flat structuring elements include disk, circle, rectangle, square, line. Non-flat structuring elements include arbitrary and ball.^5-7

Application specific structuring elements created by the user. If the input image has sharp corners then square, diamond or line shape structuring elements are used. If the input image is arbitrary and smooth then, circular shape structuring elements are used. Depending on the input image and the properties of interest, the shape and size of the structuring element are decided. They can be of any shape and size that is digitally representable, and each has an origin. Some shapes of structuring elements are shown in Figure 1.

Basic Set Operations Used in Mathematical Morphology

Basic operations used in Mathematical Morphology for binary or valued images are based on logical or arithmetic calculations respectively. The operations are carried out on a local neighborhood of a pixel. The neighborhood is a subset of pixels defined by the shape of the structuring element, which has a corresponding origin, that determines the position of the calculated value in the output image.

The language of mathematical morphology is set theory. In context, if x = (x1, x2) is an element in X: x ∈ X, let rewrite some of the basic set theory operations to make better flow for the readers (see Table 1).

Figure 1: Some Shapes of Structuring Elements: (a) Box; (b) Hexagon; (c) Disk; (d) Any Shape Click here to View figure

 

Table 1: Some Basic Set Operations Click here to View figure

 

• Logical Expressions to Perform Set Operations

The logical expressions to perform basic set operations are depicted in Table 2.

Table 2: Relationship Between Set and Logical Operations Click here to View Table

 

The Developed Algorithmic Procedure

The developed algorithmic procedure for detecting vehicle plate number which is based on the concept of Mathematical Morphology is presented as follows:

1. Input the image
2. Pre-processing of the image

1. Resize image.
2. Convert input image into grey scale image.
3. Noise removal using morphological filters.

3. Localization of number plate

1. Extracting the number plate using morphological operations.

4. Segmentation of characters

1. Calculate number of connected components.
2. Perform bounding box techniques to segment the connected components.

5. Recognition of characters.

1. Resize image.
2. Initialize templates. Noise removal using morphological filters.
3. Convert the size of the segmented characters to the size of templates saved in database.
4. Find the correlation coefficient value of segmented character with each template image stored in the database and store that value in an array.
5. Find out the maximum correlation coefficient value in the array.
6. Find the letter which is similar to the maximum correlation value.
7. Store that letter in an array. Repeat the steps for every segmented character.
8. Extracting the number plate using morphological operations.

6. Display the letters stored in the array as result in a notepad file.

The flowchart concept of Mathematical Morphology is depicted in Figure 2.

Figure 2: The Flowchart Click here to View figure

 

Methodology

This section discusses the different steps in the vehicle plate detection system. The algorithm used for this consists of the following steps:

1. Image Acquisition
2. Pre-processing of the Input Image
3. Localization of the Number Plate
4. Segmentation of Characters
5. Recognition of segmented characters
6. Display of Results

Image Acquisition

The first step of the NPR system is the image acquisition. It is the process of obtaining or capturing the image before the image will be processed for the next step. During image acquisition the type of camera used, the resolution of the camera, distance of the camera from the vehicle, illumination characteristics must be taken into account.

Pre-Processing of the Input Image

Pre-processing consists of two steps: resizing of the image and removing noise using morphological filters. The input image must be resized keeping its aspect ratio same. For the image to fit the operational window, the image must be resized without changing the aspect ratio. The input image and the templates are converted into binary images and resized to the same size. Since we have many images, all the images must be resized to a particular size so that we have a common basis for each and every image.

In an input image, there can be various noises and small impurities that will affect the outcome of the output. These noises can be removed using morphological operations with specific structuring elements. The type of morphological operator and which structuring elements should be used is determined on the type of noise present in the image.

Localization of the Vehicle Plate

Localization of the number plate is performed to extract only the number plate from the image of the vehicle. Localization is done using morphological operations.

Segmentation of Characters

In segmentation of characters, the characters from the localized image are segmented. It is done by finding out the connected components. The connected components are found using the region props MATLAB command. Once the connected components are found, the bounding box command in MATLAB is used to draw rectangles around all the connected components found. This step provides us with the segmented characters.⁸

Recognition of Characters

Recognition of characters is done using template matching. The templates are normalized to 42 x 24 pixels and stored in the database. The templates should fit into a window without white spaces on all the four sides. Once the characters are segmented, we begin the character recognition by 2D correlation. We correlate each segmented character with either the alphabets or numeral templates from the database and then we choose the value of each character based on the result of correlation. The correlation is done in pixel by pixel basis. The result of character recognition is selected based on the maximum values of correlation for each character. Only the connected components showing the maximum correlation values will be selected, the remaining boxes will be discarded.^9-12

Display of Results

The recognised and extracted characters are saved in text file format. The results are displayed in a Notepad window.

Conclusion

This paper discusses about the algorithm developed for a successfully working vehicle number plate detection technique. The steps of the algorithm are reviewed. A detailed study of mathematical morphology, morphological operations, structuring elements and design of SEs were done for this paper. Various algorithms and techniques used in previous vehicle number plate detection systems and for other countries were studied. Mathematical morphology was used in the pre-processing step and localization step. Template matching technique was used for character recognition. The format, background colour or the number of characters of the number plate does not matter in this algorithm.

Only a few methods for adaptive morphological operators have been developed for the real-time applications.The reason might be due to relatively high computational complexity of computing adaptive structuring elements. Efficient algorithms for computing adaptive structuring elements need to be developed to exploit the extensive use of adaptive morphology. Our algorithmic design and approach is an addition to give the strength to the use of Morphology in image processing.

Acknowledgements

The authors would like to express their great appreciations and gratitude to Sultan Qaboos University, Sultanate of Oman for providing research facilities, technical supports and research environment that enabled us to complete this research task.

Conflict of Interest

All authors the authors declare that there is no conflict of interest.

Funding Source

This research received no external funding.

References

1. Sushama H. B., Gadicha, A. B. A review paper on Vehicle Number Plate Recognition (VNPR) using improved character segmentation method. International Journal of Scientific and Research Publications. 2013; Volume 3 (12): pp.1–2.
2. Amar B., Mohammed M. A., Ahmed M. T., Ahmed, M. A. Automatic number plate recognition system, Annals of the University of Craiova, Mathematics and Computer Science Series. 2011; Volume 38 (1): pp. 62–71.
3. Cris L., Structure Characterization Using Mathematical Morphology, Project Report, Delft University of Technology, January 2004.
4. Sreedhar K., Panlal B. Enhancement of images using morphological transformations, International Journal of Computer Science & Information Technology. 2012; Volume 4 (1): pp. 33-50.
5. Mugdha A. R., Fast Morphological Image Processing on GPU Using CUDA, College of Engineering, Pune, Dissertation 2013.
6. Vakken, Chapter 6, Mathematical Morphology, pp. 121-162. (Accessed on December 16, 2018)http://www.cs.uu.nl/docs/vakken/ibv/reader/chapter6.pdf
7. Owens, R. Mathematical morphology, computer vision lectures. (Accessed on December 16, 2018) http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT3/node3.html
8. Sai, K. P. Automatic number plate recognition by using MATLAB, International Journal of Innovative Research in Electronics and Communications. 2015; Volume 2 (4): pp. 1-7.
9. Jayanta K. B., Debnath B., Tai-hoon K. Use of artificial neural network in pattern recognition, International Journal of Software Engineering and Its Applications. 2010; Volume 4 (2): pp. 23-34.
10. Mohammed A. Q., Template matching method for recognition musnad characters based on correlation analysis, Department of Computer Science, Amran University, Yemen. (Accessed on December 16, 2018) https://repository.nauss.edu.sa/bitstream/handle/123456789/55486/Template%20Matching%20Method%20for%
    20Recognition%20Musnad%20Characters%20Based%20on%20Correlation%20Analysis.pdf?sequence=1&isAllowed=y
11. Muhammad F. H., Puteh S., Mohammad R. M. J., Shahrul N. Y. A Face recognition system using template matching and neural network classifier, In Proceedings: 1st International Workshop on Artificial Life and Robotics, Kolej Universiti Kejuruteraan Utara Malaysia, 2005; pp. 1-6. ISBN: 9834272413, 9789834272418.
12. Neeraja M., Afaq A., Amir A., Muhammed A., Lazhar K., Samir A. A brief description of pattern recognition techniques, Imperial Journal of Interdisciplinary Research. 2016; Volume 2 (8): pp. 717-72.

---

This work is licensed under a Creative Commons Attribution 4.0 International License.