Table 3 Tuning Spark parameters
From: Leveraging resource management for efficient performance of Apache Spark
Property name | Default value | Configured value |
|---|---|---|
– –driver-memory | 1 GB | 4 GB |
– –num-executors | allocated dynamically | 2 |
– –executor-memory | 1 GB | 5 GB |
– –executor-cores | 1 | 2 |