Cloud as a Data Platform
Upcoming SlideShare
Loading in...5
×
 

Cloud as a Data Platform

on

  • 784 views

My slides on how to use cloud as a data platform at BigDataWeek 2013 Romania

My slides on how to use cloud as a data platform at BigDataWeek 2013 Romania

http://www.eurocloud.ro/en/events/all-there-is-to-know-about-big-data/#.UXZFaUDvlVI

Statistics

Views

Total Views
784
Views on SlideShare
777
Embed Views
7

Actions

Likes
3
Downloads
2
Comments
0

3 Embeds 7

https://www.linkedin.com 3
https://twitter.com 2
http://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cloud as a Data Platform Cloud as a Data Platform Presentation Transcript

  • Cloud as a Data PlatformWhat is (Big) Data? Amazon Data Services
  • Andrei SavuFounder of Axemblr.comCo-organizer of Bucharest JUGLead of Apache ProvisionrPassion for Automation & Data AnalysisConnect with me on LinkedIn
  • @ AxemblrData Processing InfrastructureDeployment Automation on IaaS platformsProduct: Hadoop On-Demand ApplianceApache Provisionr (Open Source)Consulting & Professional Services
  • TopicsIntroduction on (Big)Data● Characteristics● In Practice● ValueAmazon Data Platform● Tools● How they fit
  • What is (Big)Data?Beyond the Hype (Source)
  • ... size & speed are relative
  • Characteristics #1Too big, Too fast, Unstructured
  • 1. Volume"Simple models work better with more data"The Unreasonable Effectiveness of DataAlon Halevy, Peter Norvig, and Fernando Pereira, GoogleChallenging from a technical perspectiveNeeds scalable storageDistributed query engines (massively parallel)
  • 2. VelocityNothing new for financial tradersTight feedback loop as competitive advantageComplex event processing (CEPs)Online stream summarization (estimation)Online aggregation (key-value stores)Long term storage for batch processing
  • 3. VarietyThe reality of data is messy and the formatevolves over timeEntity Resolution, Language Detection etc.Mantra: Detect Schema, Annotate, Enrich
  • Characteristics #2In Practice
  • (Big) data is messy80% efforts go into identifying sources,integration and cleaningMessy and disconnected: different systems,different networks, different departmentsConsider data-markets
  • (Big) data has gravityTends to attract processing servicesThe cost of moving may be large
  • Cloud or in-house?Cloud:● for development & exploration● low usage or variable capacity needsIn-house:● due to strict regulations● for performance and cost efficiency
  • People & Data ScienceYou need a team that combines: math,programming and scientific instinctBuilding data-science teamshttp://radar.oreilly.com/2011/09/building-data-science-teams.html
  • (Big)Data Value
  • ... answer them w/ Data
  • Enables New ProductsRecommendation engines (think Amazon,Netflix, Facebook, LinkedIn)Advanced advertising (more later)Advanced search & spelling suggestions(and many more)
  • Rule of thumb"Advice to businesses starting out with big data:first, decide what problem you want to solve." *Christer Johnson, IBM’s leader for advancedanalytics in North America* create data-driven business processes (more)
  • (Big)Data on AWShttp://aws.amazon.com/big-data/
  • Based on my work atMagnolia Labs Inc. http://magnolialabs.com/San Francisco, CA based company with R&Din RomaniaVarious products: RTB (real-time bidding),Secure Browsing etc.They are hiring! info@magnolialabs.com
  • Overview
  • Amazon S3Amazon S3
  • Amazon GlacierAmazon Glacier
  • Amazon EMR (Elastic MapReduce)
  • Amazon Data Pipeline
  • Amazon RedShift
  • Amazon DynamoDB
  • How they fit?
  • Thanks! Questions?Andrei Savu - asavu @ axemblr.con