Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2

2,113 views

Published on

AWS has been supporting companies across Australia and New Zealand to put their most innovative tools and technologies to work to achieve their business needs and goals. AWS and our ecosystem of partners has helped the likes of CP Mining, IntelliHQ, WesCEF, Oz Minerals, Woodside and many more to modernise their analytics and data architecture in order to successfully generate business value from their data.

This event series aimed to educate customers with a broader understanding of how to build next-gen data lakes and analytics platforms and make connections with AWS.

  • Be the first to comment

AWS Data-Driven Insights Learning Series_ANZ Sep 2019 Part 2

  1. 1. © 2019, Amazon Web Services, Inc. or its Affiliates. Tom McMeekin Solutions Architect AWS The Future of Cloud Data Warehousing
  2. 2. © 2019, Amazon Web Services, Inc. or its Affiliates. –
  3. 3. © 2019, Amazon Web Services, Inc. or its Affiliates.© 2019, Amazon Web Services, Inc. or its Affiliates.
  4. 4. © 2019, Amazon Web Services, Inc. or its Affiliates. every 5 years 15 years live for 1,000x scale >10x grows *IDC, Data Age 20215: The Evolution of Data to Life-Critical Don’t Focus on Big Data, Focus on the Data That’s Big, April 2017. How do I provide democratized access to data to enable informed decisions while at the same time enforce data governance and prevent mismanagement of the data? Hadoop Elasticsearch Presto Spark Democratization of data Governance & control Organic revenue growth *Aberdeen: Angling for Insight in Today’s Data Lake, Michael Lock, SVP Analytics and Business Intelligence more valuable 11 8 5 4
  5. 5. © 2019, Amazon Web Services, Inc. or its Affiliates.
  6. 6. © 2019, Amazon Web Services, Inc. or its Affiliates. Democratization of data Governance & control Hadoop Elasticsearch Presto Spark Apache Parquet
  7. 7. © 2019, Amazon Web Services, Inc. or its Affiliates. New use cases are emerging for the modern Data Warehouse clickstream improve ad targeting predict risk analysis, fraud detection Aggregate transactional in-game behavior personalized experiences
  8. 8. © 2019, Amazon Web Services, Inc. or its Affiliates. / / / / Broken view of your business and your customers
  9. 9. © 2019, Amazon Web Services, Inc. or its Affiliates.
  10. 10. © 2019, Amazon Web Services, Inc. or its Affiliates. process more than 2 Exabytes of data Most popular Fastest
  11. 11. © 2019, Amazon Web Services, Inc. or its Affiliates. Based on the cloud DW benchmark derived from TPS-DS 30 TB dataset, 4-node cluster Redshift Vendor 1 Vendor 2 Queries Per Hour (Higher is better)
  12. 12. © 2019, Amazon Web Services, Inc. or its Affiliates. For every 24 hours your main cluster is in use, we’ll provide a one-hour credit for concurrent cluster usage 97% of users will never see a charge for auto-scale resources 0 2000 4000 6000 8000 10000 12000 5 40 80 120 150 180 QueriesperHour (QpH) Number of concurrently active users Throughput scales linearly Amazon Redshift’s throughput scales linearly with concurrent users
  13. 13. © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon Redshift Concurrency Scaling Backup Redshift automatically adds transient clusters, in seconds, to serve sudden spike in concurrent requests with consistently fast performance. No hydration required. Caching Layer How it works: All queries go to the leader node, user only sees less wait for queries. When queries in designated WLM queue begin queuing, Redshift automatically routes them to the new clusters, enabling Concurrency Scaling automatically. Redshift automatically spins up a new cluster, processes waiting queries and automatically shuts down the Concurrency Scaling cluster. 1 2 3 For every 24 hours that your main cluster is in use, you accrue a one-hour credit for Concurrency Scaling. This means that Concurrency Scaling is free for > 97% of customers.
  14. 14. © 2019, Amazon Web Services, Inc. or its Affiliates. Enabling Concurrency scaling
  15. 15. © 2019, Amazon Web Services, Inc. or its Affiliates. to Redshift cluster in busy periods transition time compute and storage on-demand Scale up and down in minutes Redshift Cluster Compute nodes Redshift Managed S3 JDBC/ODBC Leader Node CN2CN1 CN3 CN4 Backup
  16. 16. © 2019, Amazon Web Services, Inc. or its Affiliates.
  17. 17. © 2019, Amazon Web Services, Inc. or its Affiliates. Fastest Most cost-effective up to 75% $758,845 average annual benefits per TB per year $319,300 higher revenue per 100TB per year 469% Less than the #2 cloud DW with on-demand pricing and 75% less with Reserved Instances (RIs) *based on IDC’s “ROI of Amazon Redshift paper”, 2017
  18. 18. © 2019, Amazon Web Services, Inc. or its Affiliates. Fastest Most cost-effective Integrates with your data lake
  19. 19. © 2019, Amazon Web Services, Inc. or its Affiliates. Security is built-in Select compliance certifications* 10 GigE (HPC) SQL Clients/BI Tools Customer VPC Internal VPC JDBC/ODBC Compute Nodes Leader Node Network isolation End-to-end encryption Integration with AWS Key Management Service Amazon S3 Fastest Most cost-effective Integrates with your data lake
  20. 20. © 2019, Amazon Web Services, Inc. or its Affiliates. Unicorn Nation UNICORN NATION UnicornNation is a global entertainment company that provides ticketing, merchandising and promotion of large concerts and events. In recent years, they have been collecting data through a number of disparate systems and have consolidated this data onto a modern data architecture This includes a large volume of data stored in the Unicorn Nation data lake, including both structured and unstructured data used by a wide range of users. In addition to a data lake, they have also created a data warehouse, which they use for dashboards and reports.
  21. 21. © 2019, Amazon Web Services, Inc. or its Affiliates. Unicorn Nation
  22. 22. © 2019, Amazon Web Services, Inc. or its Affiliates. Unicorn Nation
  23. 23. © 2019, Amazon Web Services, Inc. or its Affiliates.© 2019, Amazon Web Services, Inc. or its Affiliates. Demo
  24. 24. © 2019, Amazon Web Services, Inc. or its Affiliates. —
  25. 25. © 2019, Amazon Web Services, Inc. or its Affiliates. Visit the Redshift website Watch the Redshift sessions from re:Invent 2018 featuring Dow Jones, McDonald’s, Intuit, and Equinox Fitness Read the DW Modernization e-book Start free with the two- month trial Set up a POC with the guidebook Self-service: Migrate to Redshift using Data Migration Service and Schema Conversion tool Work with a partner
  26. 26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thank you Tom McMeekin tommcm@amazon.com
  27. 27. © 2019, Amazon Web Services, Inc. or its Affiliates. Eric Greene AI & ML Solutions Specialist AWS AI & ML and Data Lakes: A platform to build business outcomes from data
  28. 28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Data lakes enable analytics and machine learning Cost-effective Scalable and durable Secure Open and comprehensiveAnalyticsMachine learning Real-time data movement On-premises data movement Data lake on AWS
  29. 29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2019, Amazon Web Services, Inc. or its Affiliates. What is successful machine learning?
  30. 30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
  31. 31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 40% of digital transformation initiatives supported by AI in 2019 —IDC 2018 InnovationDecision making Customer experience C E N T E R P I E C E F O R D I G I TA L T R A N S F O R M AT I O N Business operations Competitive advantage
  32. 32. Our mission at AWS Put machine learning in the hands of every developer
  33. 33. 200 new features and services launched this last year alone Unmatched flexibility Broadest and deepest set of AI and ML services 70% cost reduction in data-labeling 10x faster performance 75% lower inference cost Accelerate your adoption of ML with SageMaker Built on the most comprehensive cloud platform AWS Named as a Leader in Gartner’s Infrastructure as a Service (IaaS) Magic Quadrant for the 9th Consecutive Year W H Y A W S F O R M L ?
  34. 34. AI Services Broadest and deepest set of capabilities T H E A W S M L S T A C K VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS ML Services P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D & C O M P R E H E N D M E D I C A L L E X F O R E C A S TR E K O G N I T I O N I M A G E R E K O G N I T I O N V I D E O T E X T R A C T P E R S O N A L I Z E Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment HostingAmazon SageMaker FRAMEWORKS INTERFACES INFRASTRUCTURE ML Frameworks + Infrastructure F P G A SE C 2 P 3 & P 3 D N E C 2 G 4 E C 2 C 5 I N F E R E N T I AG R E E N G R A S S E L A S T I C I N F E R E N C E D L C O N T A I N E R S & A M I s E L A S T I C K U B E R N E T E S S E R V I C E E L A S T I C C O N T A I N E R S E R V I C E
  35. 35. Modernize your contact center to improve customer service P U T M L TO W O R K F O R Y O U R B U S I N E S S conversational chat bots | call transcription | intelligent routing | sentiment analysis VoC analytics text-to speech | multilingual omni-channel communication POLLY TRANSCRIBE TRANSLATE COMPREHEND LEX
  36. 36. VOC analytics for customer service Amazon S3 Amazon Athena Amazon Translate Amazon Comprehend Amazon Transcribe sentiment, entities, and key phrases Call Recordings Transcribe and analyze historical call recordings to measure customer sentiment over time • specific topics • products and issues
  37. 37. Live call analysis with next best action and translation
  38. 38. Use AI services to strengthen safety and security P U T M L TO W O R K F O R Y O U R B U S I N E S S accurate facial analysis | identity protection | metadata extraction REKOGNITION IMAGE COMPREHEND & COMPREHEND MEDICAL REKOGNITION VIDEO
  39. 39. Fighting human trafficking Marinus Analytics uses Amazon Rekognition to find human trafficking victims, then help law enforcement prosecute the traffickers.
  40. 40. Automate media workflows to reduce costs and monetize content P U T M L TO W O R K F O R Y O U R B U S I N E S S content moderation | contextual ad insertion | searchable media library custom facial recognition | multi-language metadata search REKOGNITION IMAGE REKOGNITION VIDEO COMPREHEND TRANSCRIBE TRANSLATE TEXTRACT
  41. 41. Monitoring radio & TV content “At Isentia, we enable customers to analyze and monitor the media coverage for their brands. We create more than 13K summaries per day from radio and TV content. With Amazon Transcribe, we can transcribe all the audio/video content that we monitor and analyze the text data with Amazon Comprehend. Features like timestamps and punctuation make it very easy for us to search through the data and drill down and present key insights for our customers to review.” Andrea Walsh - CIO, Isentia
  42. 42. Reduce localization costs and improve accuracy P U T M L TO W O R K F O R Y O U R B U S I N E S S custom vocabulary | timestamp generation | secure real-time translation | language identification POLLY TRANSCRIBE TRANSLATE COMPREHEND
  43. 43. Scaling real-time translation Using Amazon Translate, Lionbridge is able to scale machine translation in order to localize content faster and in more languages. Using Translate, Lionbridge was able to reduce translation costs by 20 percent.
  44. 44. Accurately forecast future business outcomes P U T M L TO W O R K F O R Y O U R B U S I N E S S forecasting technology used by Amazon.com | multiple time-series data forecast scheduling and visualization | supply chain integration FORECAST
  45. 45. Personalize customer experiences with targeted recommendations P U T M L TO W O R K F O R Y O U R B U S I N E S S recommendation technology used by Amazon.com | context-aware recommendations sentiment analysis | VoC analytics PERSONALIZE REKOGNITION IMAGE REKOGNITION VIDEO COMPREHEND
  46. 46. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Personalizing customer experiences Domino’s uses Amazon Personalize to customize and scale relevant marketing communications to customers based on time, context, and content, thereby improving and enhancing their experience with the Domino’s brand.
  47. 47. Increase efficiency with automated document analysis P U T M L TO W O R K F O R Y O U R B U S I N E S S optical character recognition (OCR) | automatic data extraction | natural language processing intelligent search | text analytics TEXTRACT COMPREHEND & COMPREHEND MEDICAL
  48. 48. AI Services Broadest and deepest set of capabilities T H E AW S M L S TA C K VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS ML Services P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D & C O M P R E H E N D M E D I C A L L E X F O R E C A S TR E K O G N I T I O N I M A G E R E K O G N I T I O N V I D E O T E X T R A C T P E R S O N A L I Z E Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment HostingAmazon SageMaker FRAMEWORKS INTERFACES INFRASTRUCTURE ML Frameworks + Infrastructure F P G A SE C 2 P 3 & P 3 D N E C 2 G 4 E C 2 C 5 I N F E R E N T I AG R E E N G R A S S E L A S T I C I N F E R E N C E D L C O N T A I N E R S & A M I s E L A S T I C K U B E R N E T E S S E R V I C E E L A S T I C C O N T A I N E R S E R V I C E
  49. 49. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Smart statistics with machine learning Now on AWS, Formula 1 will extract critical performance statistics to make race predictions and deliver a next level viewing experience to its fans. With 65 years of historical race data, stored in Amazon S3, Formula 1 data scientists will train deep learning models using Amazon SageMaker to power F1 Insights. AWS will give Formula 1 the tools necessary to capture and deliver data at unmatched accuracy and speed.
  50. 50. The path to ML-driven insights and predictions While the power of ML is unrivaled, “data scientists spend around 80% of their time on preparing and managing data for analysis” … hence only 20% of their time is used to derive insights 6–18 months Fetch data Clean & format data Prepare & transform data Evaluate model Integrate with prod Monitor/ debug/ refresh Train model
  51. 51. Bringing machine learning to all developers A M A Z O N S A G E M A K E R Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment
  52. 52. Bringing machine learning to all developers A M A Z O N S A G E M A K E R Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Pre-built notebooks for common problems
  53. 53. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Successful models require high-quality data
  54. 54. Amazon SageMaker Ground Truth How it works
  55. 55. Bringing machine learning to all developers A M A Z O N S A G E M A K E R Collect and prepare training data Choose and optimize your ML algorithm Pre-built notebooks for common problems Built-in, high performance algorithms • K-Means Clustering • Principal Component Analysis • Neural Topic Modelling • Factorization Machines • Linear Learner (Regression) • BlazingText • Reinforcement learning • XGBoost • Topic Modeling (LDA) • Image Classification • Seq2Seq • Linear Learner (Classification) • DeepAR Forecasting
  56. 56. Over 200 algorithms and models Natural Language Processing Grammar & Parsing Text OCR Computer Vision Named Entity Recognition Video Classification Speech Recognition Text-to-Speech Speaker Identification Text Classification 3D Images Anomaly Detection Text Generation Object Detection Regression Text Clustering Handwriting Recognition Ranking A V A I L A B L E A L G O R I T H M S & M O D E L S S E L E C T E D V E N D O R S
  57. 57. Bringing machine learning to all developers A M A Z O N S A G E M A K E R Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training
  58. 58. Bringing machine learning to all developers A M A Z O N S A G E M A K E R Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training Optimization
  59. 59. Amazon SageMaker Neo: Train once, run anywhere Neo
  60. 60. Bringing machine learning to all developers A M A Z O N S A G E M A K E R Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training Optimization One-click deployment
  61. 61. Collect and prepare training data Choose and optimize your ML algorithm Set up and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment Pre-built notebooks for common problems Built-in, high performance algorithms One-click training Optimization One-click deployment Fully managed with auto-scaling, health checks, automatic handling of node failures, and security checks Bringing machine learning to all developers A M A Z O N S A G E M A K E R
  62. 62. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon SageMaker demo
  63. 63. FRAMEWORKS INTERFACES INFRASTRUCTURE AI Services Broadest and deepest set of capabilities T H E A W S M L S T A C K VISION SPEECH LANGUAGE CHATBOTS FORECASTING RECOMMENDATIONS ML Services ML Frameworks + Infrastructure P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D & C O M P R E H E N D M E D I C A L L E X F O R E C A S TR E K O G N I T I O N I M A G E R E K O G N I T I O N V I D E O T E X T R A C T P E R S O N A L I Z E Ground Truth Notebooks Algorithms + Marketplace Reinforcement Learning Training Optimization Deployment HostingAmazon SageMaker F P G A SE C 2 P 3 & P 3 D N E C 2 G 4 E C 2 C 5 I N F E R E N T I AG R E E N G R A S S E L A S T I C I N F E R E N C E D L C O N T A I N E R S & A M I s E L A S T I C K U B E R N E T E S S E R V I C E E L A S T I C C O N T A I N E R S E R V I C E
  64. 64. Introducing Reinforcement learning (RL) Reinforcement learning (RL) Supervised learning (ASR, computer vision) Unsupervised learning (Anomaly detection, identifying text topics) Amount of labeled training data required Complexityofdecisions
  65. 65. How does RL work?
  66. 66. Amazon SageMaker RL Reinforcement learning for every developer and data scientist Broad support for frameworks Broad support for simulation environments 2D & 3D physics environments and OpenGym support Support Amazon Sumerian, AWS RoboMaker and the open source Robotics Operating System (ROS) project Fully managed Example notebooks and tutorials K E Y F E A T U R E S
  67. 67. • Build machine learning models in Amazon SageMaker • Train, test, and iterate on the track using the AWS DeepRacer 3D racing simulator • Compete in the world’s first global autonomous racing league, to race for prizes and a chance to advance to win the coveted AWS DeepRacer Cup A fully autonomous 1/18th-scale race car designed to help you learn about reinforcement learning through autonomous driving AW S D E E P R A C E R
  68. 68. The world’s first deep learning- enabled video camera for developers AW S D E E P L E N S • Purpose-built for ML-skills development • Fully programmable and customizable • Build custom Amazon SageMaker models • 10-minutes to your first deep learning project
  69. 69. H O W W E C A N H E L P • Brainstorming • Custom modeling • Training • Work side-by-side with Amazon experts ML Solutions Lab • Practical education on ML for new and experienced practitioners • Based on the same material used to train Amazon developers Machine Learning Training and Certification
  70. 70. © 2019, Amazon Web Services, Inc. or its Affiliates. ml.aws Thank you!

×