Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.

Slideshare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.

Successfully reported this slideshow.

Like this presentation? Why not share!

1,023 views

Published on

In this talk, I’ll discuss an alternate approach we’ve taken with Pyfora, an open-source platform for scalable machine learning and data science in Python. I’ll show how it produces efficient, large scale machine learning implementations directly from the source code of single-threaded Python programs. Instead of programming to a complex API, you can simply say what you mean and move on. I’ll show some classes of problem where this approach truly shines, discuss some practical realities of developing the system, and I’ll talk about some future directions for the project.

Published in:
Technology

No Downloads

Total views

1,023

On SlideShare

0

From Embeds

0

Number of Embeds

4

Shares

0

Downloads

13

Comments

0

Likes

1

No embeds

No notes for slide

- 1. Say What You Mean Braxton McKee, CEO & Founder Scaling up machine learning algorithms directly from source code
- 2. Q: Why should I have to rewrite my program as my dataset gets larger?
- 3. def sq_distance(p1,p2): return sum((p1[i]-p2[i])**2 for i in range(len(p1))) def index_of_nearest(p, points): return min((sq_distance(p, points[i]),i) for i in range(len(points)))[1] def nearest_center(points, centers): return [index_of_nearest(p, centers) for p in points] Example: Nearest Neighbors
- 4. Unfortunately, this is not fast.
- 5. A: You shouldn’t have to! Q: Why should I have to rewrite my program as my dataset gets larger?
- 6. Pyfora Automatically scalable Python for large-scale machine learning and data science 100% Open Source http://github.com/ufora/ufora http://docs.pyfora.com/
- 7. Goals of Pyfora • Provide identical semantics to regular Python • Easily use hundreds of CPUs / GPUs and TBs of RAM • Scale by analyzing source code, not by calling libraries No more complex frameworks or
- 8. Approaches to Scaling APIs and Frameworks • Library of functions for specific patterns of parallelism • Programmer (re)writes program to fit the pattern.
- 9. Approaches to Scaling APIs and Frameworks • Library of functions for specific patterns of parallelism • Programmer (re)writes program to fit the pattern. Programming Language • Semantics of calculation entirely defined by source- code • Compiler and Runtime are responsible for efficient execution.
- 10. Approaches to Scaling APIs and Frameworks • MPI • Hadoop • Spark Programming Languages •CUDA •CILK •SQL •Python with Pyfora
- 11. API Language Pros • More control over performance • Easy to integrate lots of different systems. • Simpler code • Much more expressive • Programs are easier to understand. • Cleaner failure modes • Much deeper optimizations are possible. Cons • More code • Program meaning obscured by implementation details • Hard to debug when something goes wrong • Very hard to implement
- 12. With a strong implementation, “language approach” should win • Any pattern that can be implemented in an API can be recognized in a language. • Language-based systems have the entire source code, so they have more to work with than API based systems. • Can measure behavior at runtime and use this to optimize.
- 13. Example: Nearest Neighbors def sq_distance(p1,p2): return sum((p1[i]-p2[i])**2 for i in range(len(p1))) def index_of_nearest(p, points): return min((sq_distance(p, points[i]),i) for i in xrange(len(points)))[1] def nearest_center(points, centers): return [index_of_nearest(p, centers) for p in points]
- 14. How can we make this fast? • JIT compile to make single-threaded code fast • Parallelize to use multiple CPUs • Distribute data to use multiple machines
- 15. Why is this tricky? Optimal behavior depends on the sizes and shapes of data. Centers Points If both sets are small, don’t bother to distribute.
- 16. Why is this tricky? Centers Points If “points” is tall and thin, it’s natural to split it across many machines and replicate “centers”
- 17. Why is this tricky? Centers Points If “points” and “centers” are really wide (say, they’re images), it would be better to split them horizontally, compute distances between all pairs in slices, and merge them.
- 18. Why is this tricky? You will end up writing totally different code for each of these different situations. The source code contains the necessary structure. The key is to defer decisions to runtime, when the system can actually see how big the datasets are.
- 19. Getting it right is valuable • Much less work for the programmer • Code is actually readable • Code becomes more reusable. • Use the language the way it was intended: For instance, in Python, the “row” objects can be anything that looks like a list.
- 20. What are some other common implementation problems we can solve this way?
- 21. Problem: Wrong-sized chunking • API-based frameworks require you to explicitly partition your data into chunks. • If you are running a complex task, the runtime may be really long for a small subset of chunks. You’ll end up waiting a long time for that last mapper. • If your tasks allocate memory, you can run out of RAM and crash.
- 22. Solution: Dynamically rebalance CORE #1 CORE #2 CORE #3 CORE #4 Splitting Adaptive Parallelism
- 23. Solution: Dynamically rebalance • This requires you to be able to interrupt running tasks as they’re executing. • Adding support for this to an API makes it much more complicated to use. • This is much easier to do with compiler support.
- 24. Problem: Nested parallelism Example: • You have an iterative model • There is lots of parallelism in each iteration • But you also want to search over many hyperparameters With API-based approaches, you have to manage this yourself, either by constructing a graph of subtasks, or figuring out how to flatten your workload into something that can be map-reduced.
- 25. sources of parallelism def fit_model(learning_rate, model, params): while not model.finished(params): params = model.update_params(learning_rate, params) return params fits = [[fit_model(rate, model, params) for rate in learning_rates] for model in models] Solution: infer parallelism from source
- 26. Problem: Common data is too big Example: • You have a bunch of datasets (say, for a bunch of products, the customers who bought that product) • You want to compute something on all pairs of sets (say, some average on common customers for both) • The whole set-of-sets is too big for memory [[some_function(s1,s2) for s1 in sets] for s2 in sets]
- 27. Problem: Common data is too big This creates problems because: • If you just do map-reduce on the outer loop, you still need to get to the data for all the other sets. • If you try to actually produce all pairs of sets, you’ll end up with something many many times larger than the original dataset. [[some_function(s1,s2) for s1 in sets] for s2 in sets]
- 28. Solution: infer cache locality • Think of each call to “f” as a separate task. • Break tasks into smaller tasks until each one’s active working set is a reasonable size. • Schedule tasks that use the same data on the same machine to minimize data movement. [[some_function(s1,s2) for s1 in sets] for s2 in sets]
- 29. Solution: infer cache locality f(s0,s0) f(s0,s1) f(s0,s2) f(s0,s3) f(s0,s4) f(s0,s5) f(s1,s0) f(s1,s1) f(s1,s2) f(s1,s3) f(s1,s4) f(s1,s5) f(s2,s0) f(s2,s1) f(s2,s2) f(s2,s3) f(s2,s4) f(s2,s5) f(s3,s0) f(s3,s1) f(s3,s2) f(s3,s3) f(s3,s4) f(s3,s5) f(s4,s0) f(s4,s1) f(s4,s2) f(s4,s3) f(s4,s4) f(s4,s5) f(s5,s0) f(s5,s1) f(s5,s2) f(s5,s3) f(s5,s4) f(s5,s5) f(s6,s0) f(s6,s1) f(s6,s2) f(s6,s3) f(s6,s4) f(s6,s5) f(s7,s0) f(s7,s1) f(s7,s2) f(s7,s3) f(s7,s4) f(s7,s5) f(s8, f(s8, f(s8, f(s8, f(s8, f(s8,
- 30. So how does Pyfora work? • Operate on a subset of Python that restricts mutability. • Built a JIT compiler that can “pop” code back into the interpreter • Can move sets of stackframes from one machine to another • Can rewrite selected stackframes to use futures if there is parallelism to exploit. • Carefully track what data a thread is using. • Dynamically schedule threads and data on machines to optimize for cache locality.
- 31. import pyfora executor = pyfora.connect(“http://...”) data = executor.importS3Dataset(“myBucket”,”myData.csv”) def calibrate(dataframe, params): #some complex model with loops and parallelism with executor.remotely: dataframe = parse_csv(data) models = [calibrate(dataframe, p) for p in params] print(models.toLocal().result())
- 32. What are we working on? • More libraries! • Better predictions on how long functions will take and what data they consume. This helps to make better scheduling decisions. • Compiler optimizations (immutable Python is a rich source of these) • Automatic compilation and scheduling of data and compute on GPU
- 33. Thanks! • Check out the repo: github.com/ufora/ufora • Follow me on Twitter and Medium: @braxtonmckee • Subscribe to “This Week in Data” (see top of ufora.com) • Email me: braxton@ufora.com

No public clipboards found for this slide

Be the first to comment