Evaluating MapReduce for Multi-core and Multiprocessor Systems

TitleEvaluating MapReduce for Multi-core and Multiprocessor Systems
Publication TypeConference Paper
Year of Publication2007
AuthorsRanger, Colby., Kozyrakis, Christos., Raghuraman, Ramanan., Penmetsa, Arun., and Bradski, Gary
Conference NameHPCA
Date Published02/2007
Keywordsmachine learning, mapreduce, multi-core, multi-processor
Abstract

   This paper evaluates the suitability of the MapReduce
model for multi-core and multi-processor systems. MapRe-
duce was created by Google for application development
on data-centers with thousands of servers. It allows pro-
grammers to write functional-style code that is automati-
cally parallelized and scheduled in a distributed system.
   We describe Phoenix, an implementation of MapReduce
for shared-memory systems that includes a programming
API and an efficient runtime system. The Phoenix run-
time automatically manages thread creation, dynamic task
scheduling, data partitioning, and fault tolerance across
processor nodes. We study Phoenix with multi-core and
symmetric multiprocessor systems and evaluate its perfor-
mance potential and error recovery features. We also com-
pare MapReduce code to code written in lower-level APIs
such as P-threads. Overall, we establish that, given a care-
ful implementation, MapReduce is a promising model for
scalable performance on shared-memory systems with sim-
ple parallel code.

AttachmentSize
Ranger_etal_2007.pdf352.77 KB