Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

February 2016 Webinar Series - 451 Research and AWS

2,792 views

Published on

Organizations often struggle to select and implement big data projects that produce meaningful results.

Learning from the success and failure of other organizations will help you identify common pitfalls and get more value from your big data initiatives. A new study from 451 research takes an in-depth look into six organizations and their cloud-based big data adoption efforts.

In this webinar, we will share some of the key findings from this research and see how organizations across a variety of industries use the Cloud to drive measurable value from big data. You will learn the challenges they faced, the tools they use to address these challenges, and the benefits of using AWS Cloud to develop and deploy big data solutions.

Learning Objectives:
Hear the experiences of organizations in a variety of industries, including a mobile technology analytics platform provider; a mobile application platform provider; a financial services regulator; a technology consultancy; a marketing strategy firm; and a mainstream financial services firm
Identify some of the challenges of deploying big data solutions
Learn 5 ways the Cloud delivers value for big data users
Understand the benefits of using the AWS Cloud to develop and deploy big data solutions
Who Should Attend:

Business & technical decision makers, architects and director-level or above of development for Big Data solutions, business analysts, data scientists, VP/Directors of engineering, CIOs, CTOs

Published in: Technology

February 2016 Webinar Series - 451 Research and AWS

  1. 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Roy Ben-Alta, Biz Dev Manager for Big Data Analytics, AWS 2/24/2016 Achieving Business Value with Big Data Matt Aslett, Research Director, 451 Research
  2. 2. Agenda Part I - Intro  Big data on AWS  Customer case study Part II – Report highlights, by Matt Aslett, 451 Research  Key findings  Business objectives  Users & use cases  Measuring success
  3. 3. Ever Increasing Big Data Volume Velocity Variety Veracity Value  What questions would help the business if we could answer them?  What data is available that could inform those answers?  What tools should be used to work with that data?  Aim to drive immediate business value with the first project Getting Started
  4. 4. Ever Increasing Big Data Volume Velocity Variety Veracity Value  Large capital expenditures  Long provisioning cycles  Too many tools to choose from  New & expensive skills  Bigger responsibility (sensitive data) Barriers to value
  5. 5. Big Data on AWS Immediate Availability. Deploy instantly. No hardware to procure, no infrastructure to maintain & scale Trusted & Secure. Designed to meet the strictest requirements. Continuously audited, including certifications such as ISO 27001, FedRAMP, DoD CSM, and PCI DSS. Broad & Deep Capabilities. Over 50 services and 100s of features to support virtually any big data application & workload Hundreds of Partners & Solutions. Get help from a consulting partner or choose from hundreds of tools and applications across the entire data management stack.
  6. 6. Simplify big data processing
  7. 7. Simplify big data Data Answers Collect Process Analyze Store Time to Answer (Latency) Throughput Cost
  8. 8. Big data workflow Data Answers Collect Process Analyze Store Time to Answer (Latency) Throughput Cost Data Collection and Storage Data Processing Event Processing Data Analysis
  9. 9. Big data workflow Data Answers Collect Process Analyze Store Data Collection and Storage Data Processing Event Processing Data Analysis Amazon S3 Amazon Kinesis Firehose Amazon DynamoDB Amazon RDS (Aurora) AWS Lambda Kinesis Streams Amazon EMR Amazon Redshift Amazon Machine Learning
  10. 10. AWS big data portfolio AnalyzeStoreCollect Amazon Machine Learning Amazon Kinesis Analytics AWS Import/Export AWS Direct Connect Amazon Kinesis Amazon Kinesis Firehose AWS Database Migration Amazon Glacier Amazon S3 Amazon CloudSearch Amazon Dynamo DB Amazon RDS, Aurora Amazon ElasticSearch AWS Data Pipeline Amazon Redshift Amazon EMR Amazon QuickSight Amazon EC2
  11. 11. Thousands of organizations use AWS for big data
  12. 12. Case Study: Hearst final data pipeline Buzzing API API Ready Data Amazon Kinesis S3 Storage Node.JS App- ProxyUsers to Hearst Properties Clickstream Data Science Application Amazon Redshift ETL on EMR 100 seconds 1G/day 30 seconds 5GB/day 5 seconds 1G/day Milliseconds 100GB/day LATENCY THROUGHPUT Models Agg Data
  13. 13. Data Science Amazon Redshift ETL Hearst: A “visual” representation of their pipeline Clickstream data Amazon Kinesis Results API
  14. 14. Achieving Business Value with Big Data Matt Aslett Research Director, Data Platforms and Analytics
  15. 15. © 2016 451 Research. All rights reserved 451 Research is a leading IT research & advisory company 16 Founded in 2000 250+ employees, including over 100 analysts 1,000+ clients: Technology & Service providers, corporate advisory, finance, professional services, and IT decision makers 50,000+ IT professionals, business users and consumers in our research community Over 52 million data points published each quarter and 4,500+ reports published each year 2,000+ technology & service providers under coverage 451 Research and its sister company, Uptime Institute, are the two divisions of The 451 Group Headquartered in New York City, with offices in London, Boston, San Francisco, Washington DC, Mexico, Costa Rica, Brazil, Spain, UAE, Russia, Taiwan, Singapore and Malaysia Research & Data Advisory Events Go 2 Market
  16. 16. © 2016 451 Research. All rights reserved A combination of research & data is delivered across fifteen channels aligned to the prevailing topics and technologies of digital infrastructure… from the datacenter core to the mobile edge. 3
  17. 17. Methodology Key Findings Business Objectives Users and Use Cases Measuring Success
  18. 18. © 2016 451 Research. All rights reserved The Cloud-Based Approach to Achieving Business Value From Big Data 19 Six in depth interviews conducted by 451 analysts with Big Data users whose primary deployment is cloud based. The report was commissioned by Amazon Web Services and users were sourced from customer lists provided by Amazon Web Services. The report was written based on a combination of the interviews and 451 Research’s ongoing customer and market analysis.
  19. 19. © 2016 451 Research. All rights reserved 20 The Cloud-Based Approach to Achieving Business Value From Big Data The enterprises interviewed represent a variety of industries: •mobile technology analytics platform provider •mobile application platform provider •financial services regulator •technology consultancy •marketing strategy firm •mainstream financial services firm •https://aws.amazon.com/big-data/business-value- big-data-learn-more/
  20. 20. Methodology Key Findings Business Objectives Users and Use Cases Measuring Success
  21. 21. © 2016 451 Research. All rights reserved 5 Ways the Cloud Delivers for Big-Data Users The study identified five fundamental ways that the combination of cloud computing and big data analytics are delivering business value. 22
  22. 22. © 2016 451 Research. All rights reserved 5 Ways the Cloud Delivers for Big-Data Users The study identified five fundamental ways that the combination of cloud computing and big data analytics are delivering business value. 1. Faster time to market: Query times for one organization improved by 4,000% over on-premises systems. 23
  23. 23. © 2016 451 Research. All rights reserved 5 Ways the Cloud Delivers for Big-Data Users The study identified five fundamental ways that the combination of cloud computing and big data analytics are delivering business value. 1. Faster time to market: Query times for one organization improved by 4,000% over on-premises systems. 2. Lower TCO: >50% cost savings over on-premises environments. 24
  24. 24. © 2016 451 Research. All rights reserved 5 Ways the Cloud Delivers for Big-Data Users The study identified five fundamental ways that the combination of cloud computing and big data analytics are delivering business value. 1. Faster time to market: Query times for one organization improved by 4,000% over on-premises systems. 2. Lower TCO: >50% cost savings over on-premises environments. 3. Reduced management overhead: >50% reduction in operating costs. 25
  25. 25. © 2016 451 Research. All rights reserved 5 Ways the Cloud Delivers for Big-Data Users The study identified five fundamental ways that the combination of cloud computing and big data analytics are delivering business value. 1. Faster time to market: Query times for one organization improved by 4,000% over on-premises systems. 2. Lower TCO: >50% cost savings over on-premises environments. 3. Reduced management overhead: >50% reduction in operating costs. 4. Improved developer agility: From concept to full production deployment in 24 hours. 26
  26. 26. © 2016 451 Research. All rights reserved 5 Ways the Cloud Delivers for Big-Data Users The study identified five fundamental ways that the combination of cloud computing and big data analytics are delivering business value. 1. Faster time to market: Query times for one organization improved by 4,000% over on-premises systems. 2. Lower TCO: >50% cost savings over on-premises environments. 3. Reduced management overhead: >50% reduction in operating costs. 4. Improved developer agility: From concept to full production deployment in 24 hours. 5. New revenue opportunities: Uncovering new revenue opportunities in minutes, not days. 27
  27. 27. Methodology Key Findings Business Objectives Users and Use Cases Measuring Success
  28. 28. © 2016 451 Research. All rights reserved Business Objectives for Big Data One of the key trends driving the adoption of big-data processing – both on-premises and in the cloud – is the ability to take advantage of inexpensive compute resources to perform data processing and analytics at a scale that was previously impossible due to cost and complexity. There are a variety of business opportunities that are enabled by big-data processing, but they largely fall into two key areas: •Improved operational efficiencies •Enabling and supporting new business initiatives 29
  29. 29. © 2016 451 Research. All rights reserved Business Objectives for Big Data Companies successfully taking advantage of big-data processing in the cloud are not simply enjoying incremental improvements. The benefits enabled by cloud-based big-data processing quickly become the heart of the business – enabling new applications and business processes through which the company can potentially gain competitive advantage. 30 The financial service regulator occasionally has to rerun ‘risk threat model’ analysis jobs based on large volumes of historical data covering 15 months or more. With Amazon EMR, the company has a more up- to-date and accurate picture of its historical and current threat risk.
  30. 30. © 2016 451 Research. All rights reserved Business Objectives for Big Data The companies unanimously described their cloud-based big-data processing initiatives as mission-critical in driving change for the business thanks to a variety of metrics, including improved time to market, lower data-processing costs, increased customer insight and improved customer services. 31 Querying indexed data for business analysts, some on a couple of zettabytes of data. With the company’s (on premises) data warehouse environment, such queries could take up to four hours to complete. That environment has now transitioned to Hadoop and HBase on Amazon EC2 with substantial performance improvements
  31. 31. Methodology Key Findings Business Objectives Users and Use Cases Measuring Success
  32. 32. © 2016 451 Research. All rights reserved The Analytics Landscape 33 Complexity AutomatedIT-driven PRESCRIPTIVE - Influence what happens MACHINE LEARNING DESCRIPTIVE - What is happening? PREDICTIVE - What will happen? User-driven VISUALIZATION STATISTICAL MODELING REPORTING - What happened ANALYSIS - Why did it happen?
  33. 33. © 2016 451 Research. All rights reserved Users and Use-Cases Business reporting and advanced analytics •The primary use-case for financial services firm is involves business analysts and financial analysts with domain expertise using reporting and visualization tools to search for illegal trading patterns. •The marketing strategy firm has business analysts querying customer data via internal reporting and trend-analysis tools so that it can better serve its customers. •The financial services regulator is running three primary internal workloads in the cloud for business analytics: batch analytics, interactive analytics and summary reporting. The fourth use case – ad hoc analytics – is currently still run on the on-premises data- warehousing infrastructure, but a plan is in place to migrate ad hoc analytics to the cloud. 34
  34. 34. © 2016 451 Research. All rights reserved Users and Use-Cases Data science •The marketing strategy firm has a team of data scientists doing modeling and predictive analysis on data in the cloud to identify new sources of data and competitive insights that can be rolled back into the products and services sold to its customers, including predictive analytics services. •The financial services regulator intends to develop the tools that will enable its data scientists to also take advantage of data stored in the cloud – using a combination of analysis tools and techniques, including statistical analysis, programmatic analysis and SQL-on-Hadoop – based on the advantages it has seen from its batch analytics, interactive analytics and summary reporting use cases. 35
  35. 35. © 2016 451 Research. All rights reserved Users and Use-Cases Data driven applications and services • The marketing strategy firm is making use of big-data cloud services to deliver insight to its customers in terms of how their marketing dollars are being spent, and with what results, especially when it comes to competitive analysis. • The technology consultancy is involved in a number of projects delivering applications taking advantage of big-data cloud services include transaction analysis and loyalty card analysis, as well as the generation of personalized offers. • The mobile technology analytics platform provider is providing mobile application developers with insights into their mobile application usage and performance. 36
  36. 36. © 2016 451 Research. All rights reserved Challenges – not generally cloud-specific There are multiple challenges associated with big-data projects, whether in the cloud or on- premises. The challenges highlighted are largely not specific to the cloud. As such the cloud itself does not represent a major challenge when it comes to big-data deployments. 37
  37. 37. © 2016 451 Research. All rights reserved Challenges – not generally cloud-specific • The shortage of data scientists is a significant challenge across the board, whether on- premises or in the cloud – although cloud less so than on-premises because cloud is set up to handle some of the complexity. To get the most value out of cloud-based big data quickly, developing a training plan is recommended. • Security (and the perception of security). The financial services firm subjected its cloud provider to a multi-month security review to ensure that it had the confidence to migrate data and services to the cloud. • Although the cloud provider passed the security review some data sets are still stored on- premises due to data-governance regulations. These big-data compliance concerns affect both on-premises and cloud-based instances. 38
  38. 38. Methodology Business Objectives Users and Use Cases Key Findings Measuring Success
  39. 39. © 2016 451 Research. All rights reserved Measuring Success The metrics used to measure the success of cloud-based big data projects varied from company to company and from role to role. Interviewees reported a variety of potential benefits including: • Faster time to market • Cost savings • Business performance • Developer agility 40
  40. 40. © 2016 451 Research. All rights reserved Faster Time to Market Faster time to market is often the driver for adopting big-data cloud services for startups and emerging industries, enabling them to move from concept to production without the need to design, procure, configure and maintain on-premises infrastructure. 41 “Faster time to market was by far the most important aspect. Being able to leverage these out-of-the-box, hands-on services that have built-in scalability and reasonable cost allowed us to create our service much more efficiently.” – Mobile application platform provider
  41. 41. © 2016 451 Research. All rights reserved Faster Time to Market The marketing strategy firm has also seen multiple benefits – cost savings, faster time to market, developer agility and revenue generation. For the marketing strategy firm’s CTO, however, the primary benefit is faster time to market. 42 “We could have an idea in the morning and then be developing it in the afternoon if need be. We’ve had things where we’ve turned around a full solution in 24 hours. It was unheard of before.” – Marketing strategy firm
  42. 42. © 2016 451 Research. All rights reserved Total Cost of Ownership Larger and established big-data users tend to be drawn to cloud services by the opportunity to reduce costs and improve operational efficiencies. 43 “Basically, given that total cost of ownership is one of our biggest driving factors, the business’s perception of that value is key to them embracing this whole thing.” “We definitely realized a substantial cost reduction when we moved that warehouse from on-premises [to the cloud]. It was 57% savings. ” – Financial services firm
  43. 43. © 2016 451 Research. All rights reserved Total Cost of Ownership Larger and established big-data users tend to be drawn to cloud services by the opportunity to reduce costs and improve operational efficiencies. 44 The financial services regulator discovered via an initial costing project that it could save 35- 40% of the cost of a comparative on-premises environment. Additionally, the company has found that by constantly evolving its cloud services to take advantage of innovative new services and technologies, it is able to fulfill its goal of reducing costs by 12-14% annually.
  44. 44. © 2016 451 Research. All rights reserved Total Cost of Ownership 45 “When we did the numbers and showed that a cloud-based system was going to cost less than half the legacy on-premises system, then the business said immediately, ‘Well, I want that. Just make sure that it’s going to work and my data is going to be secure.’ Once we were able to check all those boxes, that was a no-brainer.” – Financial services firm Larger and established big-data users tend to be drawn to cloud services by the opportunity to reduce costs and improve operational efficiencies.
  45. 45. © 2016 451 Research. All rights reserved Business Performance For the marketing firm, the key performance indicators are specific to the system – cost, uptime, application performance and security. These were viewed within the context of the larger concern of how the business as a whole is performing. For the mobile analytics platform provider the most important indicators are related to the business itself: how many customers are using the service; how much customer data it is handling; and how much value it can deliver to customers. 46 “If anything, our entire existence as a company is a measurement of our results of using the big-data initiatives.” – Mobile platform provider
  46. 46. © 2016 451 Research. All rights reserved Developer Agility 47 For the mobile application development platform provider, speeding time to market and the ability to modify and expand its platform in an agile manner is key to making a decision between whether an application should be deployed on-premises or in the cloud. In addition to cost considerations, agility is also a key factor for the financial services firm, specifically the ease with which it can provision new compute and storage resources in the cloud compared to the paperwork and hurdles required to provision server hardware on- premises. “If I want to provision a bunch of hardware [in the cloud], I can do that right now. If I want to provision a bunch of hardware in our datacenter, it is a multi-month extravaganza of paperwork and phone calls.” – Financial services firm
  47. 47. © 2016 451 Research. All rights reserved Developer Agility 48 “I think the biggest beneficiary is the IT infrastructure ops guys, because we have nothing that’s on-prem anymore…“the fact that an analyst or a data scientist could, within an hour of asking, have a complete environment set up means the business users are benefactors too.” – Marketing strategy firm Agility is also cited as a benefit by the marketing strategy firm, which notes that improved agility is a benefit for both the operations team and the business analysts.
  48. 48. © 2016 451 Research. All rights reserved Expectations 49 “As we move systems into [the cloud], we can retire the internal costs for that, which we can then use to save money and spend more on big data.” – Financial services firm The interviewees are unanimous in seeing big data as a critical component of the products and services they provide. They also noted that their investment in big data is expected to grow in the next 12-24 months. The unlimited capacity, agility and lower costs that come with hosting big data in the cloud are critical components of enabling that growth.
  49. 49. © 2016 451 Research. All rights reserved Conclusions • A broad array of organizations can realize the benefits of big data. The enterprises interviewed represented both startups and established vendors in a variety of industries. 50
  50. 50. © 2016 451 Research. All rights reserved Conclusions • A broad array of organizations can realize the benefits of big data. The enterprises interviewed represented both startups and established vendors in a variety of industries. • Underpinning all these early successful efforts to gain business value from big data is the ability to capture, store and process cloud data far less expensively than could ever be done in on-premises environments. 51
  51. 51. © 2016 451 Research. All rights reserved Conclusions • A broad array of organizations can realize the benefits of big data. The enterprises interviewed represented both startups and established vendors in a variety of industries. • Underpinning all these early successful efforts to gain business value from big data is the ability to capture, store and process cloud data far less expensively than could ever be done in on-premises environments. • This has allowed users to take advantage of advanced analytics techniques, especially the ability to analyze unstructured data to exploit the companies’ stored information. 52
  52. 52. © 2016 451 Research. All rights reserved Conclusions • A broad array of organizations can realize the benefits of big data. The enterprises interviewed represented both startups and established vendors in a variety of industries. • Underpinning all these early successful efforts to gain business value from big data is the ability to capture, store and process cloud data far less expensively than could ever be done in on-premises environments. • This has allowed users to take advantage of advanced analytics techniques, especially the ability to analyze unstructured data to exploit the companies’ stored information. • Cloud-based big data projects are not just incremental to the business, but rather, become the heart of the business. 53
  53. 53. © 2016 451 Research. All rights reserved Conclusions • A broad array of organizations can realize the benefits of big data. The enterprises interviewed represented both startups and established vendors in a variety of industries. • Underpinning all these early successful efforts to gain business value from big data is the ability to capture, store and process cloud data far less expensively than could ever be done in on-premises environments. • This has allowed users to take advantage of advanced analytics techniques, especially the ability to analyze unstructured data to exploit the companies’ stored information. • Cloud-based big data projects are not just incremental to the business, but rather, become the heart of the business. • The benefits of cloud-based big data are often measureable - in one organization, data queries to the cloud showed a 400-fold improvement over on-premises-based queries. 54
  54. 54. © 2016 451 Research. All rights reserved Conclusions • A broad array of organizations can realize the benefits of big data. The enterprises interviewed represented both startups and established vendors in a variety of industries. • Underpinning all these early successful efforts to gain business value from big data is the ability to capture, store and process cloud data far less expensively than could ever be done in on-premises environments. • This has allowed users to take advantage of advanced analytics techniques, especially the ability to analyze unstructured data to exploit the companies’ stored information. • Cloud-based big data projects are not just incremental to the business, but rather, become the heart of the business. • The benefits of cloud-based big data are often measureable - in one organization, data queries to the cloud showed a 400-fold improvement over on-premises-based queries. • Another company witnessed the time to execute a business-critical risk-threat analysis drop from 6-9 months to a week or less - a 98% improvement over on-premises systems. 55
  55. 55. © 2016 451 Research. All rights reserved 56 The Cloud-Based Approach to Achieving Business Value From Big Data • https://aws.amazon.com/big-data/business- value-big-data-learn-more/
  56. 56. © 2016 451 Research. All rights reserved Thank You! matthew.aslett@451research.com @maslett www.451research.com
  57. 57. Thank you!

×