Skip to main content

Table 2 Mean computation times (seconds) for forecasting models training and evaluation

From: Concept and benchmark results for Big Data energy forecasting based on Apache Spark

Tests

Training

Evaluation

MLR

LASSO

Random forest

MLR

LASSO

Random forest

\(R_\text {1D}\)

0.014 (0.009)

0.407 (0.074)

1773.078 (18.274)

0.01 (0.008)

0.073 (0.026)

583.438 (16.824)

\(SC8_\text {1D}\)

0.208 (0.015)

0.223 (0.026)

22.122 (1.639)

0.192 (0.027)

0.188 (0.015)

20.905 (1.755)

\(SCl_\text {1D}\)

0.399 (0.022)

0.404 (0.040)

10.342 (0.202)

0.462 (0.029)

0.440 (0.020)

7.247 (0.199)

\(R_\text {1W}\)

0.07 (0.006)

2.7 (0.075)

OoM

0.072 (0.007)

0.27 (0.021)

OoM

\(SC8_\text {1W}\)

0.271 (0.013)

0.281 (0.013)

108.993 (6.285)

0.348 (0.022)

0.353 (0.012)

71.187 (7.316)

\(SCl_\text {1W}\)

0.390 (0.022)

0.409 (0.025)

28.418 (0.491)

0.475 (0.025)

0.476 (0.018)

14.416 (0.169)

\(R_\text {1M}\)

0.324 (0.056)

13.438 (0.247)

OoM

0.267 (0.074)

1.346 (0.223)

OoM

\(SC8_\text {1M}\)

0.627 (0.015)

0.638 (0.018)

454.759 (3.311)

1.058 (0.15)

1.074 (0.031)

243.008 (6.786)

\(SCl_\text {1M}\)

0.463 (0.021)

0.482 (0.032)

67.469 (2.125)

0.644 (0.042)

0.652 (0.036)

33.546 (1.908)

\(R_\text {6M}\)

1.862 (0.146)

115.960 (17.589)

OoM

3.206 (0.311)

13.610 (11.116)

OoM

\(SC8_\text {6M}\)

2.822 (0.060)

2.857 (0.077)

3101.202 (14.726)

5.420 (0.101)

5.457 (0.150)

1722.941 (36.603)

\(SCl_\text {6M}\)

1.133 (0.055)

1.115 (0.028)

357.388 (5.037)

1.747 (0.050)

1.757 (0.061)

161.561 (2.156)

\(R_\text {1Y}\)

4.061 (0.165)

OoM

OoM

6.903 (0.718)

OoM

OoM

\(SC8_\text {1Y}\)

5.588 (0.050)

5.604 (0.041)

6291.860 (43.074)

10.809 (0.049)

10.800 (0.053)

3246.918 (35.248)

\(SCl_\text {1Y}\)

1.934 (0.061)

1.903 (0.055)

784.018 (11.487)

1.934 (0.061)

1.903 (0.055)

304.891 (3.768)

\(R_\text {5Y}\)

OoM

OoM

OoM

OoM

OoM

OoM

\(SC8_\text {5Y}\)

41.464 (0.575)

42.520 (1.937)

NT

72.940 (1.726)

73.728 (2.644)

NT

\(SCl_\text {5Y}\)

16.104 (0.711)

15.474 (0.683)

IOF

26.528 (1.346)

26.380 (1.000)

IOF

\(R_\text {10Y}\)

OoM

OoM

OoM

OoM

OoM

OoM

\(SC8_\text {10Y}\)

NT

NT

NT

NT

NT

NT

\(SCl_\text {10Y}\)

38.997 (1.869)

39.300 (1.755)

IOF

63.044 (2.157)

63.127 (2.301)

IOF

  1. The values in parenthesis are the standard deviation values
  2. Italics: lowest computation time for a given data mining technique and a certain amount of training/evaluation data
  3. OoM out of memory, NT not tested, IOF integer overflow