Table 3 Done experiments on IDS optimization using big data
Name | OSI layer | Time of detection | Data sources | Detection technique | Big data environment | Used data set |
|---|---|---|---|---|---|---|
Beehive [20] | 7 | Non-real | Proxy logs | k-means | Hadoop, Hive | Operational network |
Bumgardner and Marek [21] | 3, 4 | Real-time | Network flows | Threshold | Storm, HBase, Hadoop | Operational network |
Camacho et al. [22] | 7, 4, 3 | Non-real | Firewall and IDS logs | PCA | Custom | Public dataset |
Dromard et al. [23] | 4, 3 | Non-real | Network flows | DBSCAN | Spark | Operational network |
Giura and Wang [24] | 7, 4, 3 | Non-real | Network and application data | Threshold | Hadoop | Operational network |
Gupta and Kulariya [25] | 7, 4, 3 | Non-real | Network captures | Several feature extraction and classification algorithms | Spark | Public dataset |
Gonc¸alves et al. [26] | 3, 4, 7 | Non-real | DHCP, Authentication and Firewall logs | EM | Hadoop, Weka | Operational network |
Hashdoop [27] | Packet captures | Non-real | Network traffic | Â | Hadoop | Public dataset |
Iturbe et al. [19] | Â | Non-real | Network flows | Whitelisting | Elastics Search | Operational network |
Marchal et al. [28] | 3, 4, 7 | Non-real | Honeypot, DNS, HTTP, Network flow data | Threshold | Hadoop, Hive, Pig, Spark | Operational network |
MATATABI [29] | 3, 4, 7 | Non-real | DNS records, Network flows, Spam email | Multiple classification algorithms | Hive | Operational network |
Rathore et al. [30] | 3, 4 | Non-real | Network flows | C4.5, RepTree | Spark, Weka | Public dataset |
Ratner and Kelly [31] | Packet captures | Non-real | Network packets | Manual data querying | Hadoop | Operational network |
Therdphapiyanak and Piromsopa [32] | 7, 4, 3 | Non-real | Network logs | k-means | Hadoop, Mahout | Public dataset |
TADOOP [33] | 3, 4 | Non-real | Network flows | DTE-FP | Hadoop | Operational network |
Wang et al. [34] | 3, 4 | Real-time | Network flows | Intergroup entropy, LMS | Storm | Operational network |
Xu et al. [35] | 7 | Non-real | Console logs | PCA | Hadoop | Operational network |
Hadziosmanovic et al. [36] | Â | Non-real | SCADA logs | FP-Graph | Custom | Operational network |
Difallah et al. [37] | Â | Real-time | Process data | LISA | Storm | Simulated process data |
Wallace et al. [38] | Â | Real-time | Process data | Cumulative probability distribution | Spark | Operational network |
Kiss et al. [39] | Â | Non-real | Process data | k-means | Hadoop | Simulated process data |
Hurst et al. [40] | Â | Non-real | Process data | Multiple classification algorithms | Custom | Simulated process data |