FileKeyValue.java - File name and File value(list of words in the file)
WordKeyValue.java - Word name and word count
MapReducer.java - This does the bulk of the work.
- It creates a list of FileKeyValue objects for all the input files.
- Then threads out each FileKeyValue object to be processed in a Mapper.
- Waits for all the Mappers to finish.
- Then sorts the output of all the mappers by the outputKey(i.e word) and consolidates the output value from all the Mappers for that output key (i.e. creates the intermediate list).
- Threads out each unique combination of (outputKey, intermediate list) combination to a Reducer for reduction.
- Waits for all reducers to complete.
- Prints out the results
Reducer.java - Sums up the intermediate list values for a given word and passes it back to MapReducer class.Each Reducer runs as a separate Thread
No comments:
Post a Comment