One of the applications of machine vision is in crash detection. In fact, due to the high number of accidents in Iran and the need for the timely presence of relief and law enforcement centers, the existence of an intelligent system that can detect the accident and inform the relevant centers can be very useful in speeding up the accident detection. In this study, an attempt is made to present a new method for crash detection as well as diagnosis of damaged parts of the car. The Fig. 1 shows the general flowchart of the proposed method.
Initial car image segmentation
To identify different areas of the machine, it is necessary to perform the image segmentation step beforehand. In fact, by dividing the image and splitting the image into different parts. Then a label can be assigned to each of these parts in the next step, and the image classification step can be performed. The main innovation of this section is to provide a method to improve the segmentation of color images of accidents. The research tries to study the previous methods for color segmentation of color images to consider the strengths and weaknesses of each and to present a new method for color segmentation.
Previous segmentation-based methods in crash prediction have the following four major weaknesses:
-
Inefficiency for large images
-
Inability to process blurry images
-
Weakness to segment images in different lighting modes
-
Noise sensitivity
This article tries to provide solutions to all these problems.
In the first stage of image segmentation, the step of reducing the initial noise of the image is performed. In this article, Gaussian filler is used to reduce image noise. Gaussian filter is one of the conventional filters in reducing digital image noise that by adjusting the dimensions and variance of this filter, the effect of noise reduction and smoothing can be controlled. Since the noise added by the scanners is generally Gaussian, the use of this filter as a preprocessing step in image processing is quite common, which has also been used in the proposed algorithm. In the proposed method, using a Gaussian filter with a size of \(9\times 9\), the noise caused by imaging and digitizing it is reduced and its image information is extracted for the next parts.
After reducing the image noise, the image histogram is used to optimize the cluster centers in the image segmentation. A graph that shows the number of repetitions of different levels of gray in an image is called a histogram. In this diagram, the x-axis represents each of the gray planes and the y-axis represents the number of pixels corresponding to the different gray levels. To get the image histogram, it is enough to calculate the number of pixels of each brightness level by traversing the total pixels of the image.
The image histogram pan has peaks that can be considered as the center of the parts. Therefore, if the location of the separation of these peaks is detected, any part in the image can be identified. The histogram of each image is different and the location of the histogram peaks is not necessarily a specific region, and in some cases, the graph of each histogram increases with a slight decrease towards the other peak. So we need to use an algorithm to find the pixels around a peak to distinguish it from the existing peaks.
To calculate the competency function, we used a combination of two goals: the sum of the differences between the clustering centers and the pixels belonging to that class, as well as the distance between the centers. The smaller the difference between the clustering centers and the pixels belonging to that class and the greater the distance between the clustering centers, the higher the fit function. The objective function of the optimization algorithm is the sum of two different functions, one of which must be minimized and the other on should be maximized.
The sum function of the difference between the clustering centers, and the pixels belonging to the same class is shown below.
$$\sum_{i=1}^{c}\sum_{k=0}^{255}D({V}_{i},{X}_{k})\times h(k)$$
(1)
where, \(k\) is The range of intensity values from 0 (black) to 255 (white), and \(c\) is the number of clusters, \(D({V}_{i},{X}_{k})\) represents the distance of \(Pixe{l}_{k}\) from the \(Cente{r}_{i}\). Also, h (k) represents the normalized histogram of \(Light\_Intensit{y}_{k}\). As can be seen in the above relation, in this relation the smaller the total distance of different pixels from the center of their respective clusters, the smaller the objective function will be. In this regard, the histogram value of each pixel also affects the value of the objective function. The larger the histogram of a pixel, the greater the effect on the value of the objective function. In the proposed method in this study, it is preferred to select the centers of each piece close to the peaks of the histogram to reduce the segmentation error. Combining this goal, which is the error of clustering different parts to distance the centers of the clusters from each other, can provide a suitable objective function in segmenting images. In this paper, the particle swarm optimization (PSO) algorithm will be used to optimize the specified fit function. In fact, in this paper, the PSO is used to optimize the peaks as well as the image segmentation. PSO is a universal method of minimization that can be used to deal with problems whose answer is a point or surface in n-dimensional space. In such a space, an initial velocity is assigned to them, and communication channels between the particles are considered. These particles then move in the response space, and the results are calculated based on a “competency criterion” after each time interval. Over time, particles accelerate toward particles that have a higher competency standard and are in the same communication group. Although each method works well in a range of problems, the particle optimization algorithm of the method has shown great success in solving continuous optimization problems.
At each step of the PSO algorithm, the position and velocity of each particle are updated. Two criteria are used to do this. The first is the best position the particle has ever reached, known as the pbest, and the second is the best position ever obtained by the particle population. This position is displayed with gbest. In the particle optimization algorithm, each particle is displayed in the search space with the \({x}_{i}\) vector. It also has a velocity of \(v\), which is indicated by the vector \({v}_{i}\). In each repeated iteration, the velocity and location of each particle are measured using the following equations.
$$v=v\times \omega +{C}_{1}\times {r}_{1}\times \left(pbest-x\right)+ {C}_{2}\times {r}_{2}\times (gbest-x)$$
(3)
Here, \(x\) is the position of particle, \(v\) is velocity of particles, \(\omega\) is an input weight that determines the effect of velocity on the previous repetition on current velocity. Also, \({C}_{1}\) and \({C}_{2}\) are two random values between zero and one. Also in Eq. (3) the \(\times\) operator indicates a Vector Product between to matrix. In this equation, the position of the particles is updated. The parameter w is a constant weight of inertia, and for the classic version of PSO, this value is a constant positive value. In the classic version of PSO, the value of the parameter w is positive. This parameter is important for balancing global search, which is also called exploration (when higher values are set) and local search (when lower values are set). One of the most important differences between the classical PSO algorithm and other versions derived from this algorithm is the w parameter. The velocity that updates the first expression in the equation is the internal multiplication of the parameter w and the anterior velocity of the particle. Therefore the previous motion of the particle is displayed to the current motion. Hence, for example, if w = 1, the motion of the particle is completely affected by its previous motion; Therefore, the particle may continue to move in the same direction.
The pseudocode of PSO algorithm for this work is shown in Algorithm 1.
Classification of different areas of the car
After the segmentation stage, it is the turn of the image classification stage. The main purpose of many image processing algorithms is to extract specific features of the image, so that based on these properties, interpretations of the image appear or decisions are made. Such operations are frequently used, especially in the field of machine vision.
In image processing, image classification and retrieval are two very important operations. Image recovery methods are usually based on the color, texture, shape, and meaning of the image. Today, the key parts of image classification and retrieval are based on three aspects. (1) Different methods of calculating the degree of similarity, which include extracting the color feature and combining several features, (2) Reciprocal retrieval figures using the feedback grid, and (3) Image classification using the fuzzy clustering method. Recently, accuracy and recall rates have been used to evaluate the performance of image retrieval results.
In recent years, due to the growing need for optimal image classification in large databases, extensive research has been conducted in the field of image classification. The first ideas were put forward in the early 1990s, in which images were categorized without regard to their visual features and based solely on text annotations; In such a way that the concepts in the image were recognized by the operator and stored in the database as the keywords of the image. In this way, users had access to the relevant images using their desired keywords. This method is called text-based image classification [25].
In the field of image classification, various methods have been introduced to calculate the fuzzy similarity criterion, which uses the fuzzy part of the HSI color space. In this method, if there is no color in one image to calculate the degree of similarity of that color ratio between two images, by increasing the color ratio in the other image, the similarity criterion remains constant at zero; Although this method is generally logical, its implementation does not seem reasonable [26]. In addition, the color spectrum feature includes 360 elements that are divided into 8 sections. The number of these sections is not enough, and very large errors appear in the calculation of the complex image similarity criterion. Various things have been done in the field of image classification. In some previous work, to solve the problem of retrieving and classifying large-scale images, a fuzzy c-means algorithm has been proposed in which the initial condition of saving time is considered, but in this method, some images Are always considered as a single large cluster and therefore, the result of classification has not been very efficient [27, 28].
The HSI color model (hue, saturation, color intensity) is closely related to human interpretation of color. The HSI model also has the advantage of separating color and gray information in an image.
HSI color space is introduced when the user wants to specify color properties. This model needs a strong cluster of skin colors for better performance. Users can specify color properties by specifying this color model. For this model to perform better, a cluster of skin colors is necessary.
The HSI color model in a color image separates the light intensity components from the color carrier information (shade and saturation). As a result, this color space will be suitable for color-based image processing algorithms. Important components of the HSI color space are the axis of vertical light intensity, the length of the vector from the origin to the color point, and the angle that this vector has with the red axis.
In this paper, a method of classifying and retrieving different parts of an image using clustering and fuzzy similarity criteria is introduced. In this method, the criterion of similarity of two images, even when there is no color in one of the two images, seems reasonable. On the other hand, choosing a clustering center would be much more logical than the method mentioned in the previous methods, and thus, the classification result would be more efficient.
In most of the fuzzy clustering methods previously proposed for image retrieval, some images are always considered as just one large cluster, and therefore the classification result does not work well. In this paper, a method for classifying and retrieving a color image using clustering and fuzzy similarity criteria is introduced. In this method, the criterion of similarity of two images, even when there is no color in one of the two images, seems reasonable. On the other hand, choosing a clustering center would be much more logical than the method mentioned above, and thus the classification result would be more efficient.
In this paper, a method is proposed in which the similarity between two images is measured by considering the color characteristics of an image such as the color spectrum. The F-test criterion has also been used in this study; The criterion that finds the best threshold for finding fuzzy segmentation [29]. In this paper, a framework is introduced in which the images can be compared with each other concerning the fuzzy similarity criterion, which is calculated based on the color spectrum feature vector, and thus, the image can be retrieved. Image classification is also done using fuzzy clustering.
Final areas damage detection
At this stage of the proposed method, the final diagnosis should be made based on the training phase. The image pattern of the crash car, as well as the damaged parts of the car, should be identified. An important issue concerning data mining applications such as pattern recognition is the problem of large data sets in which the number of attributes is much larger than the number of patterns. For example, in recognizing control chart patterns that include very large datasets, the classifier parameters also increase. As a result, the performance of the classifier decreases dramatically. High-dimensional datasets reduce the performance of the classifier in two ways. On the one hand, with increasing the size of the data, the computational volume increases, and on the other hand, the model based on high-dimensional data has low generalizability, and the probability of over-fitting increases [30,31,32]. As a result, reducing the size of the problem can both reduce computational complexity and improve the performance of classification algorithms. It can be said that due to the special characteristics of the data that the data are independent and all-distributed, abnormal, autocorrelated, and multivariate, the traditional methods of detecting the control chart pattern for this data will not be very efficient. As a result, it is necessary to first reduce the data dimensions using a dimensional reduction method and then use an efficient classification algorithm for the final diagnosis.
In this paper, a new graph-based technique will be used for the feature extraction step. To do this, first, the initial features of the image are represented as a graph, and then based on the represented features and using the graph fitting technique, the final features are selected for the pattern prediction step.
Many existing methods use a single classification to identify the pattern of the accident [33,34,35]. Since each classification has its own characteristics, individual classifications have less combinability than a combination of multiple classifiers. In other words, multiple classifications are usually more accurate because of the variety of constructor classifiers and their function [36,37,38,39]. In this paper, an attempt is made to use a strategy based on the classifier combination using the majority voting mechanism to effectively combine the predictions from different classification algorithms [40,41,42]. For this purpose, in this paper, we used five classifiers: Support Vector Machine (SVM), decision tree (DT), nearest neighbor (KNN), simple Bayesian (NB) and Neural Network (NN). The subset of the feature selected in the previous input step constitutes the control pattern recognition model. Given that the final prediction of the crash pattern from the prediction combination of each of these classifiers is done using a learning mechanism, the final prediction model of the classifier is based on the majority vote. In this research, for the first time, a classifier combination based on majority voting and the graph fitting reduction technique [43] is used to improve the accuracy of accident detection and the detection of damaged areas.
