Table 4 The tools used in the respective data analysis pipelines of each paper
From: Manufacturing process data analysis pipelines: a requirements analysis and survey
Paper | Ingestion | Communication | Storage | Analysis | Visualization |
|---|---|---|---|---|---|
[44] | Custom | − | HDFS, HBase, MongoDB Infinispan, | Hadoop, Hive, Pig, Elasticsearch | Custom |
[45] | Custom | − | HDFS, MySQL | Hadoop | − (\(\sim \)) |
[46] | WSO2 BAM | WSO2 ESB | HDFS, RDB (\(\sim \)), Cassandra (\(\sim \)) | Hadoop, WSO2 CEP | Custom (WSO2 UES) |
[47] | − | Kafka | HDFS | Hadoop, Storm | − |
[48] | − | − | HDFS, HBase, MongoDB Cassandra, | Hadoop, Hive | − |
[49] | − | − | HDFS | Hadoop, Mahout, Jena Elephas | − |
[50] | − | − | MySQL | Matlab, QuickCog | − |
[51] | Custom | − | Microsoft SQL 2012 | Custom | Custom |
[52] | Custom | Kafka | HDFS, HBase | Hadoop, Storm, Hive, Radoop, Rapidminer | − (\(\sim \)) |
[53] | Sqoop | − | HDFS, HBase | Hadoop, Hive, Impala | − |
[54] | Sqoop | Flume | HDFS, HBase, MySQL | Hadoop, Hive | Custom |
[55] | Custom | Custom | MongoDB | Custom | Custom |
[56] | Custom | − | MongoDB, PostgreSQL | RStudio, Watson Analytics, Qliksense | Custom |
[57] | Flume (\(\sim \)), Sqoop (\(\sim \)) | Custom | HDFS, HBase | Hadoop, Hive, Impala, Spark, Pig | Custom |
[58] | Custom | Custom | Cassandra | Spark | Zeppelin (\(\sim \)) |
[59] | Kafka | Kafka | Cassandra, OntoQUAD | Spark | Custom, Jupyter, Ontos Eiger |
[60] | Custom | − | HDFS | Hadoop, Hive, Spark | Custom |
[61] | Storm | Kafka | MongoDB | Storm | Custom |
[62] | Pig, Hive | Custom | HDFS | Hadoop, Hive, Pig | Flamingo, Custom |
[63] | ODI, Talend, Sqoop | Kafka | HDFS, HBase | Hadoop, Spark, IPython | Tableau, Microsoft BI |
[64] | Sqoop, Custom | Custom | HDFS, RDB \(\sim \) | Hadoop, Hive, Impala, Spark, Matlab | − |
[65] | Custom | − | − | Custom (\(\sim \)) | Custom |
[66] | Custom | Kafka, RabbitMQ | HDFS, HBase, Cassandra, PostgreSQL | Hadoop, Spark, Storm | Custom |
[67] | Custom | Custom | Microsoft SQL 2008R2 | Custom | Custom |
[68] | Custom | − | Cassandra | Spark | − |
[69] | Sqoop | − | HDFS | Spark | Custom |
[70] | Custom, Storm | Kafka | CouchDB | − | − |
[71] | Sqoop | − | HDFS, HBase | Hadoop, Hive, Pig | Custom |
[72] | Custom | − | MongoDB | Custom | − |
[73] | WSO2 ESB | WSO2 ESB | Alfresco CMS, Neo4j | Apache UIMA, WEKA | Custom |
[74] | − (\(\sim \)) | − | HDFS, HBase | Hadoop, Hive | − |
[75] | Spark | − | HDFS, HBase | R, Drools | Custom |
[76] | Custom | − | MySQL | Custom | Custom |
[77] | Custom | − | Microsoft SQL 2008R2 | Custom | Custom |
[78] | Flume, Sqoop | − | HDFS | Hadoop, Hive, Solr, RServe, Mahout | Custom |
[79] | Flume | Kafka | HDFS, HBase, MySQL | Hadoop, Hive, Storm | Custom |
[80] | − | Kafka | Cassandra | Spark | Custom |
[81] | Custom | RabbitMQ | HDFS | Hadoop | − |