• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Peter Elleby - Big Data, Big Noise, Big Hope - No Miracles
 

Peter Elleby - Big Data, Big Noise, Big Hope - No Miracles

on

  • 106 views

Peter Elleby from Greenlight's presentation from our Big Data breakfast conference

Peter Elleby from Greenlight's presentation from our Big Data breakfast conference

Statistics

Views

Total Views
106
Views on SlideShare
105
Embed Views
1

Actions

Likes
0
Downloads
4
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Peter Elleby - Big Data, Big Noise, Big Hope - No Miracles Peter Elleby - Big Data, Big Noise, Big Hope - No Miracles Presentation Transcript

    • Peter Elleby Greenlight ‘Big Data, Big Noise, Big Hope – No Miracles
    • Big Data, Big Noise, Big Hope – No Miracles 27/06/2013
    • Big Data - Volume, Velocity, Variety As American created about 4lb of rubbish every day. If the rest of the world produced as much, this would be 10M tons daily, or 4T tons annually.
    • How do you define “Big Data”? Applications involving collections of data of a size, that makes them impossible to process in a cost effective manner using traditional database management tools and data processing applications.
    • Traditional Data Management and Data Processing OLTP OLAP Application Operational Decision Support Horizon Days & Weeks Months & Years Refresh Immediate Periodic Data Model Entity-Relationship Multi-Dimensional Schema Normalized Star (de-normalized) Emphasis Update Retrieval Space Small Large (History)
    • Core Big Data Strategies • Distribution of Data • Network of Lower Cost Devices • Compression of Data • Using Processing Power to Reduce Bandwidth Requirements • Representation of Data • Focus on Algorithm rather than Data Model • Change of Emphasis • From Completeness to Relevancy
    • Big Data Application - Hydra
    • Big Data Application Characteristics - Hydra • Time Series Data • Storage of State versus Events • Data Aggregation • Statistical Significance • Dynamic Clustering • Ontologies of Keywords and Phrases • Data Refinement • Statistical Process Control and Regression Modelling
    • Brewers Theorem (the CAP Theorem) The CAP theorem states that any networked shared-data system can have at most two of three desirable properties: • consistency (C) equivalent to having a single up-to-date copy of the data • high availability (A) of that data (for updates) • tolerance to network partitions (P) “sacrifice consistency to gain faster responses in a more scalable manner”
    • A Practical Everyday Example S1 S2 SN ...
    • The Takeaways • The Aims of your Application determines whether you are dealing with Big Data • The frameworks or technologies best suited to achieve your goals are determined your application