Table 2 Total sum of execution time for 100,000 messages dataset from “Performance results of data ingestion with and without data transformation”, “Performance results of intermediate data transformation using a MapReduce job” and “Performance results of a simple analytic computation with and without data transformation” sections
From: An efficient strategy for the collection and storage of large volumes of data for computation
Data transformation | “Performance results of data ingestion with and without data transformation” section execution time (s) | “Performance results of intermediate data transformation using a MapReduce job” section execution time (s) | “Performance results of a simple analytic computation with and without data transformation” section execution time (s) | Total execution time (s) |
|---|---|---|---|---|
pre-trans-avro-mr | 100 | 0 | 43 | 143 |
raw-json-2-pre-trans-avro-mr | 83 | 155 | 44 | 282 |
raw-avro-mr | 80 | 0 | 51 | 131 |
pyth-traditional | 166 | 0 | 224 | 390 |