Mining and prioritization of association rules for big data: multi-criteria decision analysis approach

Abstract

Data mining techniques and extracting patterns from large datasets play a vital role in knowledge discovery. Most of the decision makers encounter a large number of decision rules resulted from association rules mining. Moreover, the volume of datasets brings a new challenge to extract patterns such as the cost of computing and inefficiency to achieve the relevant rules. To overcome these challenges, this paper aims to build a learning model based on FP-growth and Apache Spark framework to process and to extract relevant association rules. We also integrate the multi-criteria decision analysis to prioritize the extracted rules by taking into account the decision makers subjective judgment. We believe that this approach would be a useful model to follow, particularly for decision makers who are suffering from conflicts between extracted rules, and difficulties of building only the most interesting rules. Experimental results on road accidents analysis show that the proposed approach can be efficiently achieved more association rules with a higher accuracy rate and improve the response time of the proposed algorithm. The results make clear that the proposed approach performs well and can provide useful information that could help the decision makers to improve road safety.

Introduction

Nowadays, the ultra-connected world is generating massive volumes of data stored in a computer database and cloud environment. These large data need to be analyzed in order to extract useful knowledge and present it to decision makers for further use.

Data mining (non-trivial process of identifying valid, novel, potentially useful and ultimately understandable patterns in data [1, 2]) techniques are a vital part of many business analytics and predictive applications that come to complete systems that provide prediction techniques and necessary services of analysis. The term of association rules is a powerful technique of data mining for discovering correlation and relationships between objects in the database. It based on statistical analysis and artificial intelligence. This technique is particularly appropriate for analyzing the correlations between objects, because it considers conditional interaction among input data sets, and produce the decision rules of the form IF-THEN. An example of association rules extracted from a supermarket sales database: “$Cereals \wedge Sugar \to Milk\;(Support \, 7\% ,Confidence \, 50\% )$”. This rule means that the customers who buy cereals and sugar also tend to buy milk. The support of the rule is the proportion of customers, who bought the three articles, and the confidence defines as the proportion of customers who bought the milk from those who bought cereals and sugar. Association rules have been used successfully in many areas, including business planning, medical diagnosis, medical research, telecommunications and text mining. Although, the volume of data brings a new challenge to extract patterns such as processing storage and response time for iterative algorithms. Since the datasets are extremely large, parallel algorithms are required. Apache Spark stands as a powerful framework to process and to analyze big datasets by using machine learning algorithms such as FP-growth, this algorithm is a powerful technique for association rules extraction. Apache Spark is considered as the fast and general engine for large-scale data processing due to its speed and scalability. However, its performance decrease in terms of relevance. In addition, it produces a huge number of association rules. Indeed, the final stage of rule validation will let the user face the main difficulty: like the selection of the most interesting rules among the large number of extracted rules. Therefore, it is necessary to help the user in validation task by implementing a preliminary stage of post-processing of extracted rules. The post-processing task aims to reduce the number of rules potentially interesting for the user by using multi-criteria decision analysis. This task must take into account both preferences of decision makers and quality measurement.

The rest of this paper is organized as follows: “Related work” section describes the literature review. “Proposed methodology” section describes the methodology for mining association rules in big data, and its integration with multi-criteria decision analysis to prioritize the extracted rules. The results and discussion are introduced in “Empirical study: road accident analysis” section. In the last section, we concluded by summarizing the work done in the study by providing achievements of the study.

Related work

Data mining techniques are widely used in several research domains and provide useful results to guide the decision makers [3,4,5,6]. The results from these techniques, besides being of interest, provide guidance for decision makers. However, in most real cases, it requires many hardware resources to collect and analyze massive data.

Recently, with the rapid expansion of information technologies, data analysis is becoming increasingly complex. To overcome these challenges, many approaches have been proposed. Park et al. [7] proposed an approach based on classification to build a prediction model that can solve the problem of large data in transportation field by using Hadoop MapReduce framework [8]. MapReduce is a great solution for one-pass computations, but not very efficient for use cases that require multi-pass computations. It tends to be slow due to the huge space consumption by each job. Furthermore, Chen et al. [9] proposed an evolutionary algorithm namely Niche-Aided Gene Expression Programming (NGEP) to address the problem of the cost of computing and inefficiency to achieve the goal. Although, these methods have achieved a great deal in obtaining association rules. Moreover, many other research works [10,11,12] have been proposed but still, suffer from the accuracy and relevance of extracted rules. The motivation of this research is to propose an approach, which can address this issue, as, mentioned above. In this context, we proposed an approach based on Apache Spark [13] and multi-criteria decision analysis to extract the relevant association rules by using Parallel FP-growth algorithm [14]. This approach is based on four major steps: data preprocessing, frequent patterns extraction without candidate generation, association rules extraction, and the prioritization of extracted rules.

Proposed methodology

In this section, we discuss the various steps that construct our proposed methodology, we started by developing the association rules technique as follows:

Association rules mining

Association rules technique is a powerful data mining method for discovering the relationship between variables in large databases. It was initiated by Agrawal et al. [2], for the first time, to analyze transactional databases. An association rule is usually defined as an implication of the form: $A \to B$ such as $A,B \subset I$ and A ∩ B = ϕ. Every rule is composed of two different sets of items A and B, where A is called antecedent and B called consequent. For example $\left\{ {Driver} \right\} \to \left\{ {Vehicle} \right\}$, suggests that a strong relationship exists between two items $\left\{ {Driver, \, Vehicle} \right\}$.

To extract the association rules, two measures are required: the minimum support and the minimum confidence. The support is defined as the proportion of transaction in the database, which contains the items A, the formal definition is:

$$Supp(A \to B) = Supp(A \cup B) = \frac{|t(A \cup B)|}{t(A)}$$

(1)

The confidence determines how frequently items in B appear in transaction that contains A, ranges from 0 to 1, the formal definition is:

$$Confidence(A \to B) = \frac{Supp(A \cup B)}{Supp(A)}$$

(2)

The extraction algorithms of association rules can be classified into three large categories: frequent, maximum and closed algorithms. Many algorithms have been designed to extract frequent itemsets and generate association rules. However, the high number of these algorithms is itself an obstacle to the ability of choice of an expert. In this context, we have done a comparative study [15] based on ELECTRE method to choose the most appropriate algorithms from the large set proposed in the literature. The details of the result are given in Fig. 1.

Fig. 1

Outrank graph based on ELECTRE method

According to this graph, Apriori-Hybrid and Eclat are incomparable. AprioriTID outranks Partition, Apriori, and Dic. FP-growth outranks Close, and Dic. Dic outrank Relim, Pascal outranks Close and MaxMiner outranks Close. Based on this comparative study, FP-growth is the most appropriate algorithm according to the decision makers’ preferences.

FP-growth algorithm

FP-growth algorithm is an efficient and scalable algorithm for mining frequent patterns without using candidate generations, proposed by Han et al. [14]. This algorithm encodes the data set using a compact data structure called FP-Tree and extract frequent itemsets directly from this structure. This algorithm has the highest rate of success, it is much faster than Apriori, and use only two passes over datasets. The first step of FP-growth is to compress the input data and mapping each transaction onto the path in the FP tree. When the size of FP-Tree is small, we extract frequent itemsets directly from FP-Tree in memory instead of making multiples pass over the data stored on the disk. Figure 2 shows the dataset with teen transactions, after reading the first transaction the structure of FP-Tree is given in (i) each node contains the label of an item with a counter that shows the number of transactions mapped onto a given path.

Fig. 2

Construction of FP-Tree

Despite its performance, FP-growth algorithm has many drawbacks such as the complexity to extract frequent itemsets for big data and the production of a large number of association rules. To deal with these problems, the adaptation of FP-growth algorithm in big data environment using Apache Spark by integration of multi-criteria decision analysis (MCDA) stands as a powerful solution for mining and prioritization of association rules in big data.

Apache Spark

Apache Spark [13] is an open source framework built around speed, ease of use, and sophisticated analytics for big data processing. It was originally developed in 2009 in AMPLab of the University of California, Berkeley, and open sourced in 2010 as an Apache project. Spark takes MapReduce to the next level with less expensive shuffles in the data processing. With capabilities like in-memory data storage and real-time processing, the performance can be several times faster than other big data technologies. Apache Spark runs programs up to 100× faster than Hadoop MapReduce in memory, or 10× faster on the disk. Apache Spark has an RDD (Resilient Distributed Dataset) an immutable distributed collection of data, partitioned across nodes in a cluster that can be operated in parallel.

Spark architecture

Spark applications run on a cluster coordinated by spark context in the main program called driver program. Spark context can connect to several types of cluster managers, once connected spark acquire executors on nodes in the cluster which are processes that run computation and data storage. Afterward, it sends the app to the executors, and finally, spark context sends tasks to the executor to run (Fig. 3).

Fig. 3

Spark architecture

Multi-criteria decision analysis approach (MCDA)

MCDA is a sub-field of operational research, and management science, dedicated to the development of decision support tools in order to solve complex decision problems involving multi-criteria objectives. Whenever a real decision problem is modeled using multi-criteria analysis, three types of problems are distinguished: choice, sorting and ranking.

In data mining field, the extraction algorithms produce a large number of association rules that not allow the decision makers to make their own choice of the most interesting rules. To deal with this problem the integration of multi-criteria decision analysis, especially an existing method called PROMETHEE, provide the ability to rank the extracted rules according to the proposed relevant measures in the literature [2, 16,17,18,19,20,21].

PROMETHEE method developed by Brans [22, 23], has been applied in several situations thanks to its ability to simplify and to solve the complex decision problems of ranking type. This method is appropriate to treat the multi-criteria decision problem of the type: $max\left\{ {f_{1} \left( a \right), \ldots ,f_{n} \left( a \right)\left| {a \, A} \right.} \right\},$ by following the given steps:

First of all, it is necessary to determine the matrix of k criteria according to the n different alternatives, Let $A = \left\{ {a_{1} , \ldots ,a_{n} } \right\}$ the set of n alternatives, and $j = \left\{ {f_{1} , \ldots ,f_{q} } \right\}$ the set of q criteria, Table 1.

Table 1 Evaluation table

$$\begin{aligned} \forall a_{i} ,a_{j} \in A:d_{k} (a_{i} ,a_{j} ) = f_{k} (a_{i} ) - f_{k} (a_{j} ) \hfill \\ \pi (a_{i} ,a_{j} ) = P_{k} \left[ {d_{k} (a_{i} ,a_{j} )} \right] \hfill \\ \end{aligned}$$

(3)

The alternative a is better than alternative b according to criteria f, if $f\left( a \right) > f(b)$. The preference function can take values on the scale from 0 to 1. When the function of preference has been associated with each criteria by the decision maker, all comparisons between all pairs of actions can be performed for all criteria. A degree of preference is then calculated for each couple of action by the formula (4).

$$\pi (a,b) = \sum\limits_{k = 1}^{q} {P_{k} (a,b) \cdot w_{k} }$$

(4)

where w _k are weights associated with criteria (close to 1 if very important, close to 0 if very little significant).

The preference matrix is computed by the formula (5).

$$\forall a_{i} ,a_{j} \in A:\pi (a_{i} ,a_{j} ) = \sum\limits_{k = 1}^{q} {w_{k} \pi_{k} (a_{i} ,a_{j} )}$$

(5)

The flow score is computed by the formula (6).

$$\begin{aligned} \phi^{ + } (a_{i} ) & = \frac{1}{n - 1}\sum\limits_{b \in A} {\pi (a_{i} ,b)} \\ \phi^{ - } (a_{i} ) & = \frac{1}{n - 1}\sum\limits_{b \in A} {\pi (b,a_{i} )} \\ \phi (a_{i} ) & = \phi^{ + } (a_{i} ) - \phi^{ - } (a_{i} ) \\ \end{aligned}$$

(6)

The complete rankings based on the net flow score is given by the formula (7).

$$\begin{aligned} a_{i} Pa_{j} \Leftrightarrow \phi (a_{i} ) \succ \phi (a_{j} ) \hfill \\ a_{i} Ia_{j} \Leftrightarrow \phi (a_{i} ) = \phi (a_{j} ) \hfill \\ \end{aligned}$$

(7)

The partial ranking based on the positive and negative flow scores is given by the formula (7.1).

$$\begin{aligned} a_{i} Pa_{j} & \Leftrightarrow \left[ {\phi^{ + } (a_{i} ) \succ \phi^{ + } (a_{j} )} \right] \wedge \left[ {\phi^{ - } (a_{i} ) \le \phi^{ - } (a_{j} )} \right] \\ a_{i} Pa_{j} & \Leftrightarrow \left[ {\phi^{ + } (a_{i} ) \ge \phi^{ + } (a_{j} )} \right] \wedge \left[ {\phi^{ - } (a_{i} ) \prec \phi^{ - } (a_{j} )} \right] \\ a_{i} Ia_{j} & \Leftrightarrow \left[ {\phi^{ + } (a_{i} ) = \phi^{ + } (a_{j} )} \right] \wedge \left[ {\phi^{ - } (a_{i} ) = \phi^{ - } (a_{j} )} \right] \\ a_{i} Ja_{j} & \quad \;\,otherwise \\ \end{aligned}$$

(7.1)

PROMETHEE GAIA (geometrical analysis for interactive aid) computes the positive and negative preference flows for each alternative, where the positive flow expresses how much an alternative is dominating the other ones, and the negative flow expresses how much an alternative is dominated by the other ones.

GAIA plan (geometrical analysis for interactive aid)

We have:

$$\begin{aligned} \varPhi (a_{i} ) & = \frac{1}{n - 1}\sum\limits_{b \in A} {\sum\limits_{k = 1}^{q} {w_{k} \pi (a_{i} ,b) - \frac{1}{n - 1}} } \sum\limits_{b \in A} {\sum\limits_{k = 1}^{q} {w_{k} \pi_{k} (b,a_{i} )} } \\ & = \sum\limits_{k = 1}^{q} {w_{k} \frac{1}{n - 1}\sum\limits_{b \in A} {\pi_{k} (a_{i} ,b) - \pi_{k} (b,a_{i} )} } \\ & = \sum\limits_{k = 1}^{q} {w_{k} \phi_{k} (a_{i} )} \\ \end{aligned}$$

(8)

Every alternative can be represented by a vector in a space of q dimensions.

$$\vec{\phi }\left( a \right) = \left[ {\phi \left( {a_{i} } \right), \ldots ,\phi_{q} \left( {a_{i} } \right)} \right]$$

(8.1)

Quality measurements

To guide the data analyst identifying interesting rules, many objective measures have been proposed in the literature (Table 2).

Table 2 Quality measurements of association rules

Proposed approach

To have an adequate model for discovering association rules from big data, we think it is important to adapt FP-growth algorithm on Apache Spark to PFP-growth (Fig. 4) for mining frequent itemsets then generates association rules. Figure 5 shows the general scheme of the overall algorithm procedure, in this overall system, the association rules mining were generated by the PFP-growth algorithm and the prioritization of extracted rules by PROMEHEE method, the details of the proposed approach are described in Fig. 6 by the following steps:

Fig. 4

The overall PFP framework

Fig. 5

General algorithm procedure

Fig. 6

Steps of proposed approach

Preprocessing In this step, we refer to an ETL (extraction transformation loading) tool for preparing and cleansing data by transforming the data to a proper format and selecting only certain columns to load.

Frequent itemsets mining (FIM) Is one of the most intensively investigated problems in terms of computational and algorithmic development. It constitutes the major technique to extract frequent itemsets from datasets.

Association rules mining (ARM) Based on the comparative study [15], we used the FP-growth algorithm to extract association rules adapted (PFP-growth) to the context of big data. The first step of PFP-growth is to compute item frequencies and identify frequent items. The second step uses a suffix tree (FP-Tree) structure to encode transactions without generating candidate sets explicitly. In the final step, the frequent itemsets can be extracted from the FP-Tree by introducing the minimum support (for example, if an item appears 3 out of 5 transactions, it has a support of 3/5 = 0.6).

Association rules prioritization The process of association rule mining produces a large number of rules that not allow the decision makers to make the right choice of interesting rules. To tackle this problem, we used PROMETHEE method to select only the most interesting rules.

Visualization Data visualization is the presentation of data in a pictorial or graphical format. It enables decision makers to see analytics presented visually, in order, to that, they can grasp difficult concepts or identify new patterns. In this approach, we used GAIA plan to present the association rules preferences.

Empirical study: road accident analysis

Road accidents have emerged as an important public health problem in the world, according to World Health Organization [24], 1.24 million people die in road crashes each year and as many as 50 million are injured. In this study, the accident data were obtained from the Ministry of Equipment, Transport, and Logistics (METL) [25] in the province of Marrakech (Morocco). Moreover, 21 variables were used (Table 3) in order to identify the main factors that affect road accident [26,27,28,29,30]. The variables describe characteristics related to the accident (type and cause), the driver (age, sex, and experience), vehicle (age and type), road (condition and geometry), time, season, number of injuries/death, etc. In addition, the data model used is shown in Fig. 7, it is a CSV file of 200 MB.

Table 3 Attributes and factors of traffic accident

Fig. 7

Data mode

Results and discussion

At this stage, we present the analysis results of the treatments performed by the association rules, and we end by a multi-criteria analysis to assess the extracted association rules according to the decision makers’ preferences. We experiment our proposed approach using road accident data. The system was built on Spark by using single nodes. Our system environment is shown in Table 4.

Table 4 Experiment environment

Frequent itemsets extraction We chose FP-growth the classic algorithm that finds frequent itemsets without candidate generation based on the comparative study [15]. To make FP-growth works on massive data, we used parallel FP-growth version implemented on Apache Spark environment. PFP-growth algorithm takes a Resilient Distributed Dataset (RDD) of transactions, where each transaction is an array of items, Table 5.

Table 5 Frequent itemsets

Association rules generation This step, implement a parallel algorithm for building association rules that have a single item as the consequent. Two measures are required: the minimum support and the minimum confidence, Table 6.

Table 6 The extracted association rules

Data mining algorithms provide a substantial solution for extracting association rules. However, these algorithms produce a large number of rules, which do not allow the decision makers to make their own choice of the most interesting rules. To solve this problem, the integration of multi-criteria decision analysis within the big data platform would be practically useful for the decision makers who are suffering from a large number of extracted rules. In this approach, the huge number of extracted rules by the PFP-growth algorithm in Spark required the use of multi-criteria analysis ranking method that deals with a large number of alternatives. Therefore, we are interested in an existing method called PROMITHEE by using a set of previously extracted rules as the alternatives to be evaluated according to given criteria in Table 2.

Based on the decision makers’ preferences, Table 7 gives the evaluation table as a list of values in rows and columns that allow the analyst to identify the performance of relationships between sets of rules and measures. The evaluation table is used to describe a multi-criteria decision analysis problem where each alternative need to be evaluated on N criteria. Moreover, Table 8 represents weights of different criteria used. These weights are non-negative numbers, the higher the weight, the more important the criteria.

Table 7 Evaluation table

Table 8 Weights of relative importance

The next step is the computation of preference between pairwise (Eq. 4) this function expressing with which degree Rule _i is preferred to Rule _j .

Afterward, we compute the partial and global outranking flow (Table 9), then we present the final result of association rules ranking over all criteria used by the decision makers (Table 10). The graphic illustration of the result is obtained by using PROMETHEE GAIA (Fig. 8).

Table 9 Preference flow

Table 10 Interesting rules

Fig. 8

GAIA plan for association rules ranking

As shown in the results, after the integration of MCDA especially the PROMETHEE method, it is graphically confirmed that rule 12 (Summer, Car ≥ Day), has the strongest flow index. Consequently, it is the most relevant one. Eventually, the interesting rules according to the decision makers’ preferences are presented in Table 10 from Order 1 to Order 16. The success of the PROMETHEE method implementation in the process of decision making greatly depends on possibilities and experience of the decision makers.

Due to the huge number of frequent itemsets and extracted rules, we choose only a small set to present and to demonstrate the performance of our proposed approach.

According to the results of the proposed approach, the extracted rules (Table 10) suggest that a strong relationship exists between these attributes {fatal accident, weather conditions, driver sex, road condition, and the season of year}. Fatal accidents tend to occur when the weather is clear, especially in the summer season and when the road condition is collapse road, etc. The theoretical results are identical to those, which are provided by the minister of equipment and transport of Morocco, this result may help the decision makers to formulate new policies and strategies for improving road safety.

Previous studies [3, 5, 26, 27, 31,32,33,34] have found an association between drivers’ behaviors, weather conditions, light conditions, and accident severity. However, the size of database low processing speed which not explored further. The results of our study not only confirm an association between different variables but also show that the integration of PFP-growth algorithm with Apache Spark framework solve the problems of the large dataset and time consumption due to the capabilities of Apache Spark. Moreover, the integration of multi-criteria decision analysis allows the decision makers to extract the relevant association rules.

As explained, the use of the association rules mining algorithms performed well. However, it has several disadvantages, such as time-consuming and more resources are needed, depending on the data size. Consequently, it is very difficult for traditional algorithms to deal with big data. Indeed, the integration of association rules analysis technique within Apache Spark solves these problems by improving the processing time (Fig. 9) and the capacity of big data storage (Fig. 10) [35].

Fig. 9

Running times for different support levels

Fig. 10

Running times for different database size

The use of PFP-growth over big data produces a large number of association rules. Consequently, it is very difficult to select the most interesting rules. Indeed, the integration of multi-criteria decision analysis approach within the association rules process provides only the significant and interesting rules according to the decision makers’ preferences. In conclusion, the proposed approach has the flowing major strengths:

• Manage the complex decision situations by taking into account all the objective and subjective factors.

• Mining interesting association rules for big data.

• Improve the response time for iterative algorithms.

• Improve road safety.

Conclusion

This paper discusses the problem of association rules mining for big data through the road accident. It is clearly identified by using Apache Spark and machine learning, especially PFP-growth algorithm to extract frequent itemsets and generate association rules. Subsequently, we found that Apache Spark provided a faster execution engine for distributed processing and claimed that it is much faster than Hadoop MapReduce as it exploits the advantages of in-memory computations which is particularly more beneficial for iterative computations in the case of iterative algorithms. We performed several experiments on road accident data to measure the speed up and scale up of implementations of our proposed approach. We found out much better than expected results for our experiments. Furthermore, the integration of multi-criteria decision analysis within the association rules mining process solves the problem of a large number of extracted rules by selecting only the most interesting. The results demonstrate that the proposed approach is highly scalable and may assist the decision makers based on some hidden patterns to formulate new rules, strategies, and policies for improving road safety.

For further work, a new methodology should be addressed to process real-time data by using Apache KAFKA the distributed streaming platform, as well as the integration of fuzzy set theory to manage the accuracy of extracted association rules.