Skip to main content

Table 2 Total sum of execution time for 100,000 messages dataset from “Performance results of data ingestion with and without data transformation”, “Performance results of intermediate data transformation using a MapReduce job” and “Performance results of a simple analytic computation with and without data transformation” sections

From: An efficient strategy for the collection and storage of large volumes of data for computation

Data transformation Performance results of data ingestion with and without data transformation” section execution time (s) Performance results of intermediate data transformation using a MapReduce job” section execution time (s) Performance results of a simple analytic computation with and without data transformation” section execution time (s) Total execution time (s)
pre-trans-avro-mr 100 0 43 143
raw-json-2-pre-trans-avro-mr 83 155 44 282
raw-avro-mr 80 0 51 131
pyth-traditional 166 0 224 390