Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Functional Ideas fora Cloudy Future              Richard Minerich              @Rickasaurus              Senior Researcher...
Functional Programming?+                 =
Properties of FP?           (It Depends on Who You Ask)- First Class Functions- Currying, Composition, Combinators- Low Le...
The Spectrum of FunctionalConvenience “FP” ConstraintsMake Life Easy Now   Make Life Easy Later
Referential Transparency- Its all about scope!- Mutation only infects in so far as it’s scope- Global variables can be ok,...
What is Functional Programming?- Complementary convenience and constraints- A highly constrained set of approaches to  pro...
Program Scope over my Career• Largest scope was usually a process with one  thread• Then a process with a few threads• The...
We need to scale out• Desktop apps are going away• Hosted hardware is on the way out• No one cares about little data      ...
Thinking about Resource CostsFar Machines      Far Network    Machines       Network     Processes      Disk        Thread...
What is Cloud Computing?- More than just a sneaky way to charge a ton  for hosting- Paradigms that simply resource managem...
Papers Published Over Time(Microsoft Academic Search April 2012)                                         “Cloud Computing”...
Properties of Cloud Computing- Resources (Network, Disk, Memory, Cache)- What constraints can make this easier?  - Force e...
Cloud Computing Methodologies     (Warning, Gross oversimplifications ahead!)- MPI (Fixed Processes)     (OpenMPI, Tempest...
From: Flexible and Efficient Distributed Resolution of Large Entities(Molnar, et al.)
This is Word Count.Seriously.63 Lines!
Hadoop           (without losing your mind)-   Pig/Hive if your problem is simple-   Scoobi with Scala-   Scalding (on Cas...
Scoobi
Scalding
From the Pangool Website: http://pangool.net/benchmark.html
Which to Pick?MPI/Agents = Difficult to get right, Extremely PowerfulMapReduce = Limiting, Easier to use, Robust to failur...
Mesos: You don’t have to chooseSpark
This is Word Count.Seriously!?
10 Lines.That’s 53 Lessor ~15%
Cloud Computing is       Functional Programming- Can’t Escape Referential Transparency- Simple Composition is Key to Small...
Thanks for Listening!            Any Questions?Visit my blog for ants and rants:  RichardMinerich.comFollow me on Twitter:...
Functional Ideas for a Cloudy Future
Functional Ideas for a Cloudy Future
Functional Ideas for a Cloudy Future
Functional Ideas for a Cloudy Future
Upcoming SlideShare
Loading in …5
×

Functional Ideas for a Cloudy Future

1,637 views

Published on

Slides from my GLFPC 2012 Keynote

Published in: Technology, Business

Functional Ideas for a Cloudy Future

  1. 1. Functional Ideas fora Cloudy Future Richard Minerich @Rickasaurus Senior Researcher at Bayard Rock
  2. 2. Functional Programming?+ =
  3. 3. Properties of FP? (It Depends on Who You Ask)- First Class Functions- Currying, Composition, Combinators- Low Level Abstraction, Metaprogramming- Immutability, Fancy Types, Constraints- Fast Tail Recursion, Scope Minimization
  4. 4. The Spectrum of FunctionalConvenience “FP” ConstraintsMake Life Easy Now Make Life Easy Later
  5. 5. Referential Transparency- Its all about scope!- Mutation only infects in so far as it’s scope- Global variables can be ok, if your referential transparency scope is a process- This can be function, class, thread, process, or even a whole computer
  6. 6. What is Functional Programming?- Complementary convenience and constraints- A highly constrained set of approaches to programing- Where you lose in order to gain- Low level constraints that propagate upwards to the top level of your program
  7. 7. Program Scope over my Career• Largest scope was usually a process with one thread• Then a process with a few threads• Then a process with many threads• Then a few machines• Now a ton of machines
  8. 8. We need to scale out• Desktop apps are going away• Hosted hardware is on the way out• No one cares about little data - But! -• Old algorithms don’t generalize well• New tradeoffs between speed and scope• Too many costs to keep track of
  9. 9. Thinking about Resource CostsFar Machines Far Network Machines Network Processes Disk Threads Memory Instructions Cache
  10. 10. What is Cloud Computing?- More than just a sneaky way to charge a ton for hosting- Paradigms that simply resource management- You always lose in order to gain- High level constraints that propagate downward into your subtasks
  11. 11. Papers Published Over Time(Microsoft Academic Search April 2012) “Cloud Computing” “Type System”
  12. 12. Properties of Cloud Computing- Resources (Network, Disk, Memory, Cache)- What constraints can make this easier? - Force everything into one of a few styles of computation? - What if want we want to do is still possible but doesnt fit our cluster’s paradigm? - Wheres the escape hatch?
  13. 13. Cloud Computing Methodologies (Warning, Gross oversimplifications ahead!)- MPI (Fixed Processes) (OpenMPI, Tempest)- Agents (Dynamic Processes) (Erlang, Parallel Haskell, Akka)- MapReduce (More Like Collect-GroupBy-Fold) (Hadoop, Google)- Others/Hybrids (Iterative Map-Reduce, Mesos/Spark)
  14. 14. From: Flexible and Efficient Distributed Resolution of Large Entities(Molnar, et al.)
  15. 15. This is Word Count.Seriously.63 Lines!
  16. 16. Hadoop (without losing your mind)- Pig/Hive if your problem is simple- Scoobi with Scala- Scalding (on Cascading) on Scala- F# + .NET API once Microsoft Ships
  17. 17. Scoobi
  18. 18. Scalding
  19. 19. From the Pangool Website: http://pangool.net/benchmark.html
  20. 20. Which to Pick?MPI/Agents = Difficult to get right, Extremely PowerfulMapReduce = Limiting, Easier to use, Robust to failuresMiddle Road = Iterative MapReduce, Mesos/Spark
  21. 21. Mesos: You don’t have to chooseSpark
  22. 22. This is Word Count.Seriously!?
  23. 23. 10 Lines.That’s 53 Lessor ~15%
  24. 24. Cloud Computing is Functional Programming- Can’t Escape Referential Transparency- Simple Composition is Key to Small Programs- Object Oriented: a Square Peg in a Round Hole
  25. 25. Thanks for Listening! Any Questions?Visit my blog for ants and rants: RichardMinerich.comFollow me on Twitter: @RickasaurusCome to NYC for the SkillsMatter F# Tutorials June 5th and 6Th: is.gd/fsharptutorials

×