0
4.3kviews
What is the role of a "combiner" in the Map reduce framework? Explain with the help of an example.
1 Answer
0
266views

enter image description here

Combiner:

i. A Reduce function is associative and commutative. That is, the values to be combined can be combined in any order, with the same result.

ii. The addition performed in Example 1 is an example of an associative and commutative operation. It doesn’t matter how we group a list of numbers v1, v2, . . . , vn; the sum will be the same.

iii. When the Reduce function is associative and commutative, we can push some of what the reducers do to the Map tasks

iv. These key-value pairs would thus be replaced by one pair with key w and value equal to the sum of all the 1’s in all those pairs.

v. That is, the pairs with key w generated by a single Map task would be replaced by a pair (w, m), where m is the number of times that w appears among the documents handled by this Map task.

Please log in to add an answer.