The federation of cloud and big data activities is the next challenge where MapReduce should be modified to avoid (big) data migration across remote (cloud) sites. This is exactly our scope of research, where only the very essential data for obtaining the result is transmitted, reducing communication, processing and preserving data privacy as much as possible. We propose an algorithmic technique for MapReduce algorithms, called Meta-MapReduce, that decreases the communication cost by allowing us to process and move metadata to clouds and from the map to reduce phases.

1. Meta-MapReduce
A Technique for Reducing Communication in
MapReduce Computations
Foto N. Afrati1, Shlomi Dolev2, Shantanu Sharma2, and
Jeffrey D. Ullman3
1 National Technical University of Athens, Greece
2 Ben-Gurion University of the Negev, Israel
3 Stanford University, USA
17th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS 2017)
Canada (18-21 September 2015)

2. Communication Cost:
Join of two relations
2
Organization
A
Map
Phase
Reduce
PhaseOutputs Final
outputs
Organization
B

3. Do we need to send the
whole database to the cloud
before performing join
operations?
Problem Statement
3

4. Join of two relations
4
A B
a1 b1
a2 b1
a3 b2
B C
b1 c1
b1 c2
b3 c3
Mapper 1
Mapper 2
Mapper 3
Mapper 4
Mapper 5
Mapper 6
b1, a1
b1, a2
b2, a3
b1, c1
b1, c2
b3, c3
Reducer
for b1
Reducer
for b2
Reducer
for b3
A B C
a1 b1 c1
a1 b1 c2
a2 b1 c1
a2 b1 c2
The size of all B values is very small as
compared to values of A and C
Organization
B
Organization
A

5. • The amount of data required to move
– from the location of the user to the
location of the mappers
– from the map to the reduce phases in each
iteration of the job
Communication Cost
6

6. Do we need to send the whole
database to the cloud before
performing join operations?
NO
But then how to get answers??
Work on metadata
Problem Statement
7

7. Meta-MapReduce
• A new algorithmic approach for MapReduce
algorithms that decreases the communication
cost significantly
• Work on metadata, which varies
according to problems and very small in size
as compared to the original database
• Decreases the communication cost
8

8. Meta-MapReduce
9
Chunk1
Meta-
data
Original
input
data
Step 4: Call Function: Data
request and data
transmission
Step 2:
Meta-data
transmission
Split1
Split2
Splitm
Input
meta-data
split1
Mapper for 1st split
split2
Mapper for 2nd split
Mapper for mth split
splitm
Reducer
for k1
Reducer
for k2
Reducer
for kr
Output 1
Output 2
Master process
Step 1: MapReduce job
assignment
Step 3:
Read and Map tasks’ execution
Step 4:
Read and Reduce
tasks’ execution

9. • Users send their metadata
• Avoids the movement of data that does not
participate in the final output
• The final results now computed using
metadata and metadata avoids to upload the
whole database
Meta-MapReduce
10

10. • Amazon EMR
• Geographically distributed MapReduce
computations
• k-nearest-neighbors problem
• Shortest part problem in a social graph
• Multiway join
• Skyline queries
Applications
11

11. Foto Afrati1, Shlomi Dolev2, Shantanu Sharma2, and
Jeffrey D. Ullman3
1 School of Electrical and Computing Engineering, National Technical
University of Athens, Greece
afrati@softlab.ece.ntua.gr
2 Department of Computer Science, Ben-Gurion University of the
Negev, Israel
{dolev,sharmas}@cs.bgu.ac.il
3 Department of Computer Science, Stanford University, USA
ullman@cs.stanford.edu
Presentation is available at
http://www.cs.bgu.ac.il/~sharmas/publication.html