Beyond Map/Reduce: Getting Creative With Parallel Processing

Beyond Map/Reduce:  Getting Creative with Parallel Processing  " Ed Kohlwey @ekohlwey kohlwey_edmund@bah.com

Overview"•  Within the last year: –  Two cluster schedulers have been released –  Two BSP frameworks have been released –  An in-memory Map/Reduce has been released –  Accumulo has been released•  More importantly –  We have been given the tools to program in something besides Map/Reduce and MPI

What About…"•  This talk covers a few speciﬁc frameworks•  There’s lots more out there

Motivations for Schedulers" The cornerstone of new cluster computing environments

Different Tasks Have Different Needs" Host 7 Host 5 Host 3 Host 2CPU RAM Host 1 CPU RAM Host 1 CPU RAM Task A Task B Task C

Clusters Often Donʼt Accommodate This" Percentage of Cluster Expense of Hosts Required Load to Execute Task Task A Task B Task C Task A Task B Task C Types of Hosts In Cluster Type 1

This is How It Should Look" Percentage of Cluster Expense of Hosts Required Load to Execute Task Task A Task B Task C Task A Task B Task C Types of Hosts In Cluster Type 1 Type 2

Economic Reasons"Power Consumption Load

Simple Example: a Work Queue"•  Data scientists execute serial implementations of machine learning algorithms•  Some are expensive, some are not•  Scientists aren’t running analyses all the time•  Solution 1: –  Give all the analysts a big workstation•  Solution 2: –  Give the analysts all thin clients and let them share a cluster

Advantages for Moving to a Thin Client/Cluster Model"•  Scalability –  All analyst capabilities can be enhances by adding one host•  Increases resource utilization –  Workstations are expensive, and will be highly under-utilized•  Increase availability –  Using a distributed ﬁle system to store data

Desirable Scheduler Features" YARN Mesos Operate on heterogeneous clusters Y Y Highly Available Y Y Pluggable scheduling policies Y Y Authen9ca9on Y N Task ar9fact distribu9on Y P Scheduling policy based on mul9ple resources N Y (RAM, CPU) Mul9ple Queues Y N Fast accept/reject model N P Reusable method of describing resource Y N requirements Pluggable Isola9on N Y “Compute Units” N N

New Compute Environments" BSP, In-Memory Map/Reduce, and Streaming Processing

(Hadoop) Map/Reduce Pros & Cons"•  Map/Reduce implements partitioned, parallel sorting –  Many algorithms (relational) express well –  Creates O(n lg(n)) runtime constraints for some problems that wouldn’t otherwise have them•  Hadoop M/R is good for bulk jobs

In-Memory Map/Reduce"•  Memory is fast•  Often, after the map phase, a whole data set can ﬁt in the memory of the cluster•  Spark provides this, as well as a very succinct programming environment courtesy of Scala and it’s closures

In-Memory Performance" Logistic Regression Performance Comparison 4000 3000Time (s) 2000 Hadoop 1000 Spark 0 5 10 20 30 Iterations *Numbers taken from http://spark-project.org

Spark Wordcount"val file = spark.textFile("hdfs://...”) file.flatMap(line => line.split(" ")) .map(word => (word, 1)) .reduceByKey(_ + _)

Hadoop Wordcount"public class WordCount { public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); public static class TokenizerMapper String[] otherArgs = new GenericOptionsParser(conf, extends Mapper<Object, Text, Text, IntWritable>{ args).getRemainingArgs(); if (otherArgs.length != 2) { private final static IntWritable one = new IntWritable(1); System.err.println("Usage: wordcount <in> <out>"); private Text word = new Text(); System.exit(2); } public void map(Object key, Text value, Context context Job job = new Job(conf, "word count"); ) throws IOException, InterruptedException { job.setJarByClass(WordCount.class); StringTokenizer itr = new StringTokenizer(value.toString()); job.setMapperClass(TokenizerMapper.class); while (itr.hasMoreTokens()) { job.setCombinerClass(IntSumReducer.class); word.set(itr.nextToken()); job.setReducerClass(IntSumReducer.class); context.write(word, one); job.setOutputKeyClass(Text.class); } job.setOutputValueClass(IntWritable.class); } FileInputFormat.addInputPath(job, new Path(otherArgs[0])); } FileOutputFormat.setOutputPath(job, new Path(otherArgs[1])); System.exit(job.waitForCompletion(true) ? 0 : 1); public static class IntSumReducer } extends Reducer<Text,IntWritable,Text,IntWritable> { } private IntWritable result = new IntWritable(); public void reduce(Text key, Iterable<IntWritable> values, Context context ) throws IOException, InterruptedException { int sum = 0; for (IntWritable val : values) { sum += val.get(); } result.set(sum); context.write(key, result); } }

Streaming Processing: Accumulo"•  Accumulo is a BigTable implementation•  Idea: accumulate values in a column –  “map” using the ETL process•  Summarize values (stored in sorted order) at read-time –  “reduce” process•  No control over partitioning outside a row –  Accumulo doesn’t suffer from the column family problem that HBase has, so this is ok•  Less consistent than Map/Reduce because race conditions can occur with respect to the scan cursor•  Iterator programming environment allows you to compose “reduce” operations•  Implementing streaming Map/Reduce over a BigTable implementation is a hybrid of in-memory and disk based approaches•  Allows revision of ﬁgures due to data provenance issues

BSP"Generalizing Map/Reduce for graph processing

BSP"•  First proposed by Valiant in 1990•  Good at expressing iterative computation•  Good at expressing graph algorithms•  Concerned with passing messages between virtual processors•  Perhaps the most famous implementation is Pregel

MR Graph Traversal"Map Sort + Reduce Shuﬄe A n è B n è C n è

MR Graph Traversal"Map Sort + Reduce I want to send a Shuﬄe message to C!A n è B n è C n è

MR Graph Traversal"Map Sort + Reduce Shuﬄe A n è A C n, m B n è B nC n è C n

MR Graph Traversal"Map Sort + Reduce Shuﬄe A n è A C n, m An B n è B n B n C n è C n CC n, m

MR Graph Traversal"Map Sort + Reduce Shuﬄe A n è A C n, m An è A nB n è B n B n è B n I got it!C n è C n CC n, mè C n

MR Graph Traversal"Map Sort + Reduce Shuﬄe O((n+m) lg(n+m) ) A n è A C n, m A n è A nB n è B n B n è B nC n è C n CC n, mè C n

MR Graph Traversal"Map Sort + Reduce Shuﬄe This can be op9mized to O(m) A n è A C n, m A n è A nB n è B n B n è B nC n è C n CC n, mè C n

The BSP Version"Compute Exchange Synchronize Messages A n C m A nB n B n C B n è m C n

The BSP Version" No9ce A and C’s message Compute Exchange Synchronize exchange isn’t closely Messages coupled, providing beEer I/O u9liza9on A n C m A n B n B nC B n è m C n

The BSP Version" Also, no9ce we don’t necessarily Compute have to copy the en9re graph Exchange Synchronize state. We just send whatever Messages messages need to be sent A n C m A n B n B nC B n è m C n

BSP Implementations"•  Giraph –  Currently an Apache Incubator project –  Has a growing community –  Runs during the Hadoop Map phase•  GoldenOrb –  Not actively maintained since the summer•  Both implementations are in-memory, modeled after Pregel

Contact Info"Ed KohlweyBooz | Allen | Hamilton@ekohlweykohlwey_edmund@bah.com

Beyond Map/Reduce: Getting Creative With Parallel Processing

by Ed Kohlwey

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 22

Statistics

Beyond Map/Reduce: Getting Creative With Parallel Processing Presentation Transcript

http://lanyrd.com	19
https://si0.twimg.com	3