• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Chris Boos, arago AG: Big Data means new programs
 

Chris Boos, arago AG: Big Data means new programs

on

  • 625 views

Lightning Talk anlässlich des zweiten CloudCamp Frankfurt am 24.5.2012 in der Brotfabrik in Hausen.

Lightning Talk anlässlich des zweiten CloudCamp Frankfurt am 24.5.2012 in der Brotfabrik in Hausen.

Statistics

Views

Total Views
625
Views on SlideShare
625
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Chris Boos, arago AG: Big Data means new programs Chris Boos, arago AG: Big Data means new programs Presentation Transcript

    • ANALYZING BIG DATA IS PROGRAMMING FOR THE CLOUD Chris Boos (@boosc) boos@arago.de CloudCamp Frankfurt 24.5.2012Donnerstag, 24. Mai 12
    • Data, lots of itDonnerstag, 24. Mai 12
    • Even in simple datasets, common statistics fails - (avg, min, max, distribution)Donnerstag, 24. Mai 12
    • 79 times more CPU power than used in Apollo missions on one iPhoneDonnerstag, 24. Mai 12
    • Why you need big data You Are Here ! Yield 2010 s Systems Thinking Wisdom 2000 s Knowledge Ecology Intelligence 1990 s Knowledge Management Knowledge 1980 s Information Mangement Information 1970 s 1960 s 1950 s Data Processing DataDonnerstag, 24. Mai 12
    • Finding clusters, evaluating outliers and interpreting white noiseDonnerstag, 24. Mai 12
    • You are not looking for patterns, you are looking for anomaliesDonnerstag, 24. Mai 12
    • Cloud Computing 1.0 Is When the IT guys are finally able to explain to business people what they were talking about 20 years ago!Donnerstag, 24. Mai 12
    • =Donnerstag, 24. Mai 12
    • Computation on demand + Pay as you goDonnerstag, 24. Mai 12
    • Cloud Computing 2.0 Is When the IT guys realize that using this scalable ressource also calles for new ways of programmingDonnerstag, 24. Mai 12
    • =Donnerstag, 24. Mai 12
    • go beyond IaaS and start thinking parallelDonnerstag, 24. Mai 12
    • andDonnerstag, 24. Mai 12
    • BASE (Basically Available, Soft State, Eventual consistency) not ACID (Atomicity, Consistency, Isolation, Durability)Donnerstag, 24. Mai 12
    • How to scale (AWS Example) • Do not allocate instances manually • Each component needs to be independent • Plan for failure • Actively provoke failureDonnerstag, 24. Mai 12
    • Human Software • Click Workers and Mechanical Turks are not just cheap labour • They allow programmers to hand tasks to humans they are not able to handle algorithmically • Make use of it to • Do things too complicated for machine learning • Pre populate machine learning spacesDonnerstag, 24. Mai 12
    • Old Style (Imperative) Programming • Step by step explanation 1 what to do • Explaining WHAT to do rather than RESULTS you want 2 • Always necessary for basic algorithms 3Donnerstag, 24. Mai 12
    • One New Stly (Functional) Programming I • Combine results to 1 become a program 2 • Allows dynamic 3 distribution • Map-Reduce is only one way of doing it!Donnerstag, 24. Mai 12
    • Functional Programming II F ( G ( H ( A,B) , C), D) getMusicLikes(getFriends(facebookID) Instead of for i in getFriends(facebookID) getMusicLikes(i)Donnerstag, 24. Mai 12
    • Check out my tool list: http://www.hcboos.net/100-links/Donnerstag, 24. Mai 12
    • 2 ExamplesDonnerstag, 24. Mai 12
    • The AMP3 Platform at Senzari.com Adaptable Music Parallel Processing PlatformDonnerstag, 24. Mai 12
    • MARS-o-Matic at arago.de Big data based IT modelling and pricing appDonnerstag, 24. Mai 12
    • Thank You for Your TimeDonnerstag, 24. Mai 12
    • Credits • „Big Data Just Beginning to Explode“ by CSC http://www.csc.com/insights/flxwd/ 78931-big_data_just_beginning_to_explode • „Social media network connections among twitter users“ by Marc Smith http:// www.flickr.com/photos/marc_smith/ • Asteroid Datasets by Bruce Gary http:// brucegary.net/POVENMIRE/x.htmDonnerstag, 24. Mai 12