Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Amazon Web Services


Published on

By Leveraging AWS Cloud and its services it not only help in reducing the cost but also brings agility and innovation. One of such service BigData provides a paradigm shift by putting smart in everything we do today including smart home, smart city, smart health, smart campus and many more. We will talk about how AWS services can help in reducing the cost and bring agility by leveraging Big Data to bring in innovation to campus.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Amazon Web Services

  1. 1. Amazon Web Services Amazon
  2. 2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Matt Johnson, Solutions Architect, UK Public Sector March 2017 Big Data Services on AWS JISC Digifest
  3. 3. Cloud is the New Normal
  4. 4. Equipment Resources and Administration Contracts Cost No Up Front Expense Pay for what you Use Improve Time to Market & Agility Scale Up and Down Self-Service Infrastructure Traditional Infrastructure AWS Cloud
  5. 5. What is Big Data?
  6. 6. Then…
  7. 7. When your data sets become so large and complex you have to start innovating around how to collect, store, process, analyze, and share it.
  8. 8. …Now
  9. 9. Scale as your data and business grows The volume, variety, and velocity at which data are being generated are leaving organisations with new questions:
  10. 10. Big Data is all about getting high value, actionable insights from your data assets
  11. 11. Unlocking Data Most organizations are embarking on ambitious innovation initiatives to unlock their data. The data already exists but goes unused or is locked away from complimentary data sets in isolated data silos.
  12. 12. Enter Data Lake Architectures Data Lake is a new and increasingly popular architecture to store and analyze massive volumes and heterogeneous types of data.
  13. 13. Benefits of a Data Lake – All Data in One Place Store and analyze all of your data, from all of your sources, in one centralized location. “Why is the data distributed in many locations? Where is the single source of truth ?”
  14. 14. Benefits of a Data Lake – Quick Ingest Quickly ingest data without needing to force it into a pre-defined schema. “How can I collect data quickly from various sources and store it efficiently?”
  15. 15. S3 is the Data Lake
  16. 16. Implementing a Data Lake using Amazon S3
  17. 17. AWS Direct Connect AWS Snowball ISV Connectors Amazon Kinesis Firehose S3 Transfer Acceleration AWS Storage Gateway Data Ingestion into Amazon S3
  18. 18. AWS Snowball: PB-scale Data Transport E-ink shipping label Ruggedized case “8.5G Impact” All data encrypted end-to-end 80TB & 100TB 10G network Rain & dust resistant Tamper-resistant case & electronics
  19. 19. Amazon Kinesis Streams Amazon Kinesis Firehose Amazon Kinesis Analytics Work with streaming data using Amazon Kinesis
  20. 20. Implementing IoT on AWS
  21. 21. Summary of AWS Analytics, Database & AI Tools Amazon Redshift Enterprise Data Warehouse Amazon EMR Hadoop/Spark Amazon Athena Clusterless SQL Amazon Glue Clusterless ETL Amazon Aurora Managed Relational Database Amazon Machine Learning Predictive Analytics Amazon Quicksight Business Intelligence/Visualization Amazon ElasticSearch Service ElasticSearch Amazon ElastiCache Redis In-memory Datastore Amazon DynamoDB Managed NoSQL Database Amazon Rekognition Deep Learning-based Image Recognition Amazon Lex Voice or Text Chatbots
  22. 22. Building a Big Data Application web clients mobile clients DBMS Amazon Redshift Amazon QuickSight AWS Cloud Transform your and explore your data at scale with Amazon EMR corporate data center Amazon EMR Structured Data In Amazon S3 Raw data In Amazon S3
  23. 23. Upload/user your existing data Discover & build models Make predictions over new data Integrated with AWS services for easy data access Model evaluation and interpretation tools Data visualization and exploration Machine Learning Algorithms APIs for batch and real-time predictions Modeling APIs Amazon Machine Learning
  24. 24. Building a Big Data Application web clients mobile clients DBMS Amazon Redshift Amazon QuickSight AWS Cloud React intelligently in real-time with Amazon Machine Learning corporate data center Amazon Kinesis Firehose Amazon Kinesis Analytics AWS Lambda Amazon Kinesis Streams Reference data in Amazon S3 Amazon Machine Learning Amazon SNS Amazon Athena
  25. 25. Amazon SQS apps Streaming KCL apps Amazon Redshift COLLECT STORE CONSUMEPROCESS / ANALYZE Amazon Machine Learning Presto Amazon EMR Amazon Elasticsearch Service Apache Kafka Amazon SQS Amazon Kinesis Streams Amazon Kinesis Firehose Amazon DynamoDB Amazon S3 Amazon ElastiCache Amazon RDS Amazon DynamoDB Streams HotHotWarm FastSlowFast SearchSQLNoSQLCacheFileMessageStream Amazon EC2 Amazon EC2 Mobile apps Web apps Devices Messaging Message Sensors & IoT platforms AWS IoT Data centers AWS Direct Connect AWS Import/Export Snowball Logging Amazon CloudWatch AWS CloudTrail RECORDS DOCUMENTS FILES MESSAGES STREAMS Amazon QuickSight Apps & Services Analysis&visualizationNotebooksIDEAPI LoggingIoTApplicationsTransportMessaging ETL BatchMessageInteractiveStreamML Amazon EMR AWS Lambda Amazon Kinesis Analytics Amazon Athena
  26. 26. Why AWS for Academic Computing?
  27. 27. University of Chicago Uses AWS to Reduce Costs and Provide Greater Than 99% Availability • The University of Chicago’s Computation Institute builds solutions for big data projects like genomics analysis and other scientific projects • Needed a cost-effective way to provide big-data analysis to labs around the world while providing always-on service • By using AWS, the Institute now experiences more than 99% availability and can focus on delivering capabilities to scientists rather than building and maintaining infrastructure
  28. 28. What sets AWS apart? Building and managing cloud since 2006 90+ services to support any cloud workload Over 2,000 customer-driven service/feature releases 16 regions, 42 availability zones, 70 edge locations 59 proactive price reductions to date Thousands of partners; 2,700+ Marketplace products Experience Service Breadth & Depth Pace of Innovation Global Footprint Pricing Philosophy Ecosystem *as of June 2016
  29. 29. JustGiving Supports 24 Million Users on Charity Site Using AWS • Needed a new platform to support general operations and new analytics service • Moved to AWS, using a wide range of services • Can scale system faster in response to unanticipated spikes in traffic • Receives query results in seconds compared to 30 minutes under old system • Obtains deeper insights into billions of data points, using information to deliver better services Using the new AWS tools, we can extract much finer-grained data points based on millions of donations and billions of visits, and then use that information to provide a better platform for our visitors. Richard Atkinson Chief Information Officer, JustGiving ” “ JustGiving is a major online platform that supports charitable giving. The organization is based in London.
  30. 30. Summary Build decoupled systems • Data → Store → Process → Store → Analyze → Answers Use the right tool for the job • Data structure, latency, throughput, access patterns Leverage AWS managed services • Scalable/elastic, available, reliable, secure, no/low admin Use log-centric design patterns • Immutable log, batch, interactive & real-time views Be cost-conscious • Big data ≠ big cost
  31. 31. Labs and training on cloud topics and AWS products Open course content by leading professors and AWS Grants for free usage of AWS services Communities that share best practices virtually and in person We invite you to join AWS Educate Accelerate cloud learning with AWS credits, cloud training, course content, and collaboration tools Not a student or educator? Help extend AWS grants to more students by inviting your network to participate (#awseducate). Learn more at:
  32. 32. Matt Johnson Solutions Architect, UK Public Sector E: T: @mhj_work