Connecta Event: Big Query och dataanalys med Google Cloud Platform

555
-1

Published on

Avancerad dataanalys och ”big data” har under de senaste åren klättrat på trendlistorna och är nu ett av de mest prioriterade områdena i utvecklingen av nya tjänster och produkter för ledarföretag i det digitala landskapet.

Informationen som byggs upp i systemen när kundmötena digitaliseras har visat sig vara guld värt. Här finns allt vi behöver veta för att göra våra affärer mer effektiva.

Sedan sommaren 2013 har Connecta tillsammans med Google ett etablerat samarbete för att hjälpa våra kunder med övergången till moln-tjänster för bland annat avancerad dataanalys. För att göra oss själva redo att hjälpa våra kunder har vi under ett antal år utvecklat såväl kunskaper som skaffat oss erfarenheter kring Googles olika moln-produkter, som exempelvis ”Big Query”.

Big Query är ett molnbaserat analysverktyg och en del av Google Cloud Platform. Big Query gör det möjligt att ställa snabba frågor mot enorma dataset på bara någon sekund. Big Query och Google Cloud Platform erbjuder färdiga lösningar för att sätta upp och underhålla en infrastruktur som med enkla medel gör allt detta möjligt.

På Connecta Digital Consultings tredje event för våren introducerade vi våra kunder och partners i koncepten dataanalys och Big Query.

Under eventet berördes följande punkter:

- Big Data och Business Intelligence (BI)
- “The Google Big Data tools” – framgångsfaktorer och hur man kommer igång
- Google Cloud Platform och hur man genomför en framgångsrik molnsatsning

Vi presenterade case och berättade om viktiga lärdomar vi dragit i samarbetet med Google och våra kunder.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
555
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Connecta Event: Big Query och dataanalys med Google Cloud Platform

  1. 1. © Connecta - Confidential
  2. 2. © Connecta - Confidential Ett verkligt kundbehov – våra kunder upplever svårigheter att göra vettiga analyser En stor potential och affärsmöjlighet genom dagens enorma mängder data Innovations– och kunskapsutvecklingen går fort – och det nu händer nu! 1) 2) 3)
  3. 3. © Connecta - Confidential ■  What is Big Data? ■  The Google Cloud Platform ■  Big Data on the Google Cloud Platform - Big Query ■  Case study - Casual gaming ■  Demo - Swedish election with Big Query and Tableau ■  Summary - The benefits of Big Data Agenda
  4. 4. © Connecta - Confidential •  Svenskt konsultbolag som finns till för att förverkliga punkterna på ledningens agenda. Från strategi till transformation och värdeskapande •  Ca 700 konsulter inom -  Digital Consulting -  Management Consulting -  Enterprise Consulting -  AM och Infrastruktur •  Omsätter ca 800 MSEK och är noterade på Nordiska börsen. •  Vi gör våra kunderna mer konkurrenskraftigagenom att kombinera affärsstrategiskt tänkande, tekniska kunskaper och förmågan att gå från ord till handling.
  5. 5. © Connecta - Confidential “90% of the data in the world today was created in the last 2 years alone” http://www.forbes.com/sites/ciocentral/2013/01/15/big-data-get-ready-for-the-2013-big-bang/
  6. 6. © Connecta - Confidential
  7. 7. © Connecta - Confidential Big Data on the top of the agenda
  8. 8. © Connecta - Confidential Top technology priorityThe 2013 CIO agenda (and 2012, 2009, 2008, 2007…)
  9. 9. © Connecta - Confidential
  10. 10. © Connecta - Confidential data is the oilof the 21st century
  11. 11. © Connecta - Confidential What is Big Data?
  12. 12. © Connecta - Confidential ▪  “Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using traditional data processing applications” ▪  The 3 V’s of Big Data Introduction to Big Data
  13. 13. © Connecta - Confidential “Now that we have all this data we have to ask the pivotal question; can it be trusted? This is the essence of Veracity.” The 4:th V: Veracity Edd Dumbill. Planning for Big Data: A CIO’s Handbook to the Changing Data Landscape. O’Reilly Media, 2012
  14. 14. © Connecta - Confidential Big data is about the business value it provides ▪  Unless business needs are met the data and the plan it drives are missing the vital element of value ▪  Value comes when you find insights you wouldn’t have found otherwise and when you start making better decisions ▪  Try to quantify the value and communicate it across the organization
  15. 15. © Connecta - Confidential
  16. 16. © Connecta - Confidential
  17. 17. © Connecta - Confidential Challenges
  18. 18. © Connecta - Confidential
  19. 19. © Connecta - Confidential Key Challenges in Big Data Information Strategy: ■  What is your plan with Big Data? Enterprise & External Information Management: ■  Information is everywhere – volume, variety, velocity – and it keeps growing! Technical threshold and competence ■  How will you start the work and who will do it?
  20. 20. © Connecta - Confidential Solution
  21. 21. © Connecta - Confidential Information Strategy: ■  Make it a top management issue and make somebody take responsibility for the effort ■  Connect your corporate strategy with your information strategy ■  Transforming company culture to be data-driven Enterprise & External Information Management: ■  Ensuring reliable and consistent data by structured work with Master Data Management (MDM) ■  The information must be used in the organization, veracity is crucial Solution to Key Challenges in Big Data
  22. 22. © Connecta - Confidential Technical threshold and competence ■  Choose the technical solution that fits your needs and resources ■  Secure competences with an overall picture in order to start the work ■  Start with small pilot projects to show the business value it can bring Solution to Key Challenges in Big Data
  23. 23. Cloud Platform Big Data Session with Connecta, April 24 - 2014 Guillaume Leygues, Enterprise Cloud Platform Sales Engineer Benelux & Nordics André Hoekzema, Enterprise Cloud Platform Lead Benelux & Nordics
  24. 24. “Enabling Technology for Disruptive Business Models”
  25. 25. Agenda 25th, 2014 Google Cloud Platform Introduction, Gaining Momentum Big Data on Google Cloud Platform Discussion 1 2 3
  26. 26. - Google’s Mission Statement “Organize the world’s information and make it universally accessible and useful.”
  27. 27. Building Products that Scale Google Maps Gmail Google Drive
  28. 28. Developing at Google scale means encountering Google-sized challenges.
  29. 29. For the past 15 years, Google has been building out the world’s fastest, most powerful, highest quality cloud infrastructure on the planet. Images by Connie Zhou
  30. 30. Google has been running some of the world’s largest distributed systems with unique and stringent requirements. Images by Connie Zhou
  31. 31. A Network that Spans the Globe
  32. 32. Google's Global OpenFlow Network
  33. 33. Innovating Software & Driving Technology Forward SpannerDremelMapReduce Big Table Colossus 2012 20132002 2004 2006 2008 2010 GFS Compute Engine
  34. 34. Cloud Storage Cloud SQL Cloud Datastore Compute Compute Engine App Engine App Services BigQuery Cloud Endpoints Storage
  35. 35. May 2013 Google Compute Engine (Preview) PHP for App Engine (Preview) Big JOIN in BigQuery The Last Year in the Cloud Platform November 2013 Cloud Endpoints GA Dedicated Memcache GA August 2013 Layer 3 Load Balancing Encryption at Rest for Cloud Storage December 2013 Compute Engine GA Live Migration Persistent Disks July 2013 Dedicated Memcache Offline Disk Import February 2014 HIPAA Support Cloud SQL GA
  36. 36. Source: Google Internal Data 4.75 Million active applications
  37. 37. Investments in Cloud Platform
  38. 38. We can do better Lower and simplify pricing Make developers more productive
  39. 39. Prices are falling •  Public cloud prices have dropped 6-8% annually Source: Google Internal Data 20142006 Public Cloud Prices
  40. 40. But prices are not falling fast enough •  Hardware costs have dropped 20-30% annually Hardware Cost Public Cloud Prices•  Public cloud prices have dropped 6-8% annually Source: Google Internal Data 20142006
  41. 41. Pricing Updates (Effective April 1st, 2014) 35% price drop on Compute Engine, across all sizes, regions, and classes 37% price drop on App Engine frontend instance hours, 33% on Datastore writes and 50% on Dedicated Memcache 68% price drop on Cloud Storage On Demand pricing reduced by 85% - $5/TB
  42. 42. You should get the best price with... No Upfront Payments No Lock-in No Complexity
  43. 43. 100%0% 20% 40% 60% 80% Sustained Use Previous On Demand New On Demand $0.11 $0.10 $0.09 $0.08 $0.07 $0.06 $0.05 $0.04 $0.03 Sustained-use discountsNetPricePerHour
  44. 44. Sustained-Use Pricing 30% net reduction on Compute Engine instances with 24x7 use
  45. 45. •  Managed VMs •  The Flexibility of Compute Engine •  The productivity of App Engine •  Provides best of both worlds •  IaaS + PaaS Flexibility Managementand Managed VMs
  46. 46. Developer Productivity •  Use the tools you know and love •  Fast, reliable deployments •  Isolate and fix issues in production with Continuous Integration Developer Productivity Time to Market and Robust Design
  47. 47. 1000X BigQuery Streaming •  Near real-time analysis •  High fidelity, low latency •  Focus on results, not sharding and transforming $0.01 per 100,000 rows Real time availability of data100,000 rows per second
  48. 48. •  Deployment Manager •  Replica Pools •  Cloud DNS •  Windows Server, SuSE, RHEL support and so much more...
  49. 49. Agenda 25th, 2014 Google Cloud Platform Introduction, Gaining Momentum Big Data on Google Cloud Platform Discussion 2 3 1
  50. 50. http://www.google.org/ flutrends/ Detecting Flu Trends
  51. 51. Speech Recognition
  52. 52. •  Applications at the heart of business interactions •  Devices and sensors •  Lower cost of storage & ingestion •  New programming models •  New scale and capabilities for SQL •  Easily available software (Open Source) •  Easy on-ramp, cost effective experimentation •  Unlimited scale, low TCO •  Combine Open Source software and platform services Ability to process Cloud consumption modelData availability Key drivers in the growth of Big Data
  53. 53. Google Cloud Storage Mix and match storage and computation from OSS and Google Cloud Platform BigQuery and Datastore Connectors BigQueryDatastore Hadoop BigQuery Connector Datastore Connector Cloud Storage Connector HBase HivePig Hadoop Applications Hadoop, Pig, HBase, and Hive are trademarks of the Apache Software Foundation.
  54. 54. Q3, 2012 Q4,2012 Q1, 2013 Q2, 2013 TodayQ3, 2013 Q4, 2013Q2, 2012 Launch 1000x Streaming rate Table Views Table Wildcards JSON functions SQL Improvements BigQuery Innovation Momentum Google Analytics Integration Streaming API Table Decorators Large Query Results Query Caching Analytic functions Big JOIN Big Aggregates Timestamp JSON Import Nested / Repeated Fields Datastore ImportBatch Processing Excel Connector
  55. 55. BigQuery Ecosystem Chartio
  56. 56. Ease of use •  Simplified infrastructure for realtime use cases •  Stream events row-by-row via simple API Use cases •  Server Logs, Mobile apps, Gaming, In-App real time analytics BigQuery Streaming Low cost: $0.01 per 100,000 rows Real time availability of data100,000 rows per second Customer example:
  57. 57. Google Analytics + BigQuery Google Analytics Premium Platform Google BigQueryData Pipeline Native Data Pipeline to Load Data into BigQuery Project
  58. 58. Google Analytics + BigQuery Customers
  59. 59. BigQuery in Action " The interactive performance of Google BigQuery, combined with Tableau’s intuitive visualization tools, enabled our analysts to interactively explore huge quantities of data – hundreds of millions of rows – with incredible efficiency. Previously, analyses would require hours or days to complete, if they would even complete at all. With Google BigQuery it takes minutes, if that, to process. This time-to-insight was previously impossible" – Giovanni DeMeo Vice President Global Marketing and Analytics
  60. 60. " The simulation cluster ran for nearly two months as part of the ATLAS distributed compute grid, logging over 5 million core-hours, completing 458,000 computationally intensive jobs and processing about 214 million events. The cluster achieved sustained peak throughput of 15,000 jobs per day. “We had a great experience with Google Compute Engine … and think that it is modern cloud infrastructure that can serve as a stable, high performance platform for scientific computing”. – Dr. Panitkin CERN Atlas Project CERN Atlas Compute Grid Extended on GCE
  61. 61. •  1.5TB in 60 seconds •  8,412 cores •  Google Compute Engine MapR Breaks Minute Record Sort
  62. 62. Thank You
  63. 63. Agenda 25th, 2014 Google Cloud Introduction, Gaining Momentum Big Data on Google Cloud Platform Discussion 1 2 3
  64. 64. 28 Billion requests per day on App Engine
  65. 65. 6.3 Trillion Cloud Datastore operations per month
  66. 66. “[Google's] ability to build, organize, and operate a huge network of servers and fiber-optic cables with an efficiency and speed that rocks physics on its heels. This is what makes Google Google: its physical network, its thousands of fiber miles, and those many thousands of servers that, in aggregate, add up to the mother of all clouds.” - Wired Images by Connie Zhou
  67. 67. © Connecta - Confidential Big Data in practice - Understanding player behavior in a Casual game - Patrik Gottfridsson
  68. 68. © Connecta - Confidential ■  Simple rules, easy to learn ■  Play in short bursts ■  No long-term commitment ■  Targets a mass audience What is casual gaming?
  69. 69. © Connecta - Confidential Very small revenue per user ●  (Paid) ●  In-App Purchase ●  Ads Business model
  70. 70. © Connecta - Confidential ■  Measure 2nd day retention ■  Optimize across game versions Make it sticky Reactivate Encourage ■  Find the “stales” ■  Send a “miss you” push notification ■  Find the “spiders”, the socially connected players ■  Drop their rate of ad shows Facts based revenue optimization
  71. 71. © Connecta - Confidential BigData BigData BigData ■  Measure 2nd day retention ■  Optimize across game versions Make it sticky Reactivate Encourage ■  Find the “stales” ■  Send a “miss you” push notification ■  Find the “spiders”, the socially connected players ■  Drop their rate of ad shows Facts based revenue optimization
  72. 72. © Connecta - Confidential CSV upload Cron import Google spreadsheets High level technical solution
  73. 73. © Connecta - Confidential Quickly up and running Avoid upfront license costs Avoid on- premise hardware Process millions of events per day Challenges
  74. 74. © Connecta - Confidential Collect everything you can Segmentation of the data model Validate your analytical queries Visualize graphically (obviously) Success factors
  75. 75. © Connecta - Confidential Immediate discoveries about gamer behavior New campaigns launched to revive “stales” and encourage “spiders” Continous follow-up of player statistics at the board level All in all, better optimized games and an increased profitability Results
  76. 76. © Connecta - Confidential Demo How to make data useful using Google Cloud Platform
  77. 77. © Connecta - Confidential 60% potential increase in operating margins for retail
  78. 78. © Connecta - Confidential > 2x competitive advantage 5-6% higher productivity and profitability Significantly higher return on equity and market value Data-driven decisionmaking
  79. 79. © Connecta - Confidential What’s your next step?
  80. 80. © Connecta - Confidential Connecta offers: ■  BigQuery Quickstart - Initial analysis, workshops and a running BigQuery solution ■  Cloud Code Workshop - Get your team up to speed on the Google Cloud Platform ■  Cloud Assessment - Analysis, workshops and identification of where a Cloud solution would make your company more competitive What’s your next step?

×