Skip to main content
Fig. 2 | Journal of Big Data

Fig. 2

From: HybSMRP: a hybrid scheduling algorithm in Hadoop MapReduce framework

Fig. 2

Spill and Merge phases in MapReduce [13]. The output produced by Map is not directly written to disk, it first writes it to its memory. It takes advantage of buffering writes in memory. Each map task has a circular buffer memory of about 100 MB by default. When the contents of the buffer reach a certain threshold size (mapreduce.map.sort.spill.percent, which has the default value 0.80, or 80%), a background thread will start to spill the contents

Back to article page