AWS Big Data Analytics IP Expo 2013

•

1 like•1,257 views

Many companies recognize the use of data analytics as an opportunity to better understand their customers and gain a lead on their competition. The ability to get better insight from vast amounts of unstructured data, coming from a multitude of sources, can give businesses the advantage in an industry where even the smallest improvement can mean a big difference. Amazon Web Services offers a range of big data, analytics and storage solutions that are used by companies such as NASDAQ, Bankinter and S&P Capital to deliver a highly secure and agile platform. Join this session and learn how it allows customers to start on a small scale but grow as their business requires, giving them the agility they need to deliver cutting edge solutions to their customers without any upfront CAPEX investment.

Technology Business

Big Data Analytics
David de Santiago
Business Development Manager, Analytics EMEA

Overview

1. Introducing Big Data
2. From data to actionable information
3. Analytics and Cloud Computing

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Lower cost,
higher throughput

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Lower cost,
higher throughput

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

Highly
constrained

Data volume

Generated data

Available for analysis

Gartner: User Survey Analysis: Key Trends Shaping the Future of Data Center Infrastructure Through 2011
IDC: Worldwide Business Analytics Software 2012–2016 Forecast and 2011 Vendor Shares

Elastic and highly scalable
+
No upfront capital expense
+
Only pay for what you use
+
Available on-demand

=

Remove
constraints

Generation

Collection & storage
Accelerated

Analytics & computation

Collaboration & sharing

Big Data
Technologies and techniques for
working productively with data,
at any scale.

Per day:
3.5 billion records
13 TB of click stream logs
71 million unique cookies

User bought
recently a home
theatre system

Targeted Ad

And is now
looking at sport
games

Results:
500% return on ad spend
17,000% reduction in procurement time
“We couldn’t have done it”

Finding signal in the noise of logs
Identified early mobile usage
Invested heavily in mobile development

In January 2013
9,432,061 unique mobile devices
used the Yelp mobile app.
Other Features powered by EMR:
People Who Viewed this Also Viewed
Review highlights
Auto complete as you type on search
Search spelling suggestions
Top searches
Ads

Open web index.
3.4 billion records.
Available to all.

Tweeting about Flu

You Are What You Tweet: Analyzing Twitter for Public Health. M. J. Paul and M. Dredze, 2011

Full parse for impact of
social networks
300 lines of Ruby code.
14 hours.
$100.

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

S3, Glacier,
Storage Gateway,
DynamoDB,
Redshift, RDS,
HBase

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

EC2 &
Elastic MapReduce

Generation

Collection & storage

Analytics & computation

Collaboration & sharing

EC2 & S3,
CloudFormation,
Elastic MapReduce,
RDS, DynamoDB, Redshift

Amazon Redshift

Fully Managed Data Warehouse
Scales to 1.6PB

Faster, Simpler, Cheaper

Amazon Redshift

Effective
Hourly Price
Per TB

Effective
Annual Price
per TB

On-Demand

$ 0.425

$ 3,723

1 Year Reservation

$ 0.250

$ 2,190

3 Year Reservation

$ 0.114

$

999

“TOWARDS THE END OF
LAST YEAR OUR DATA
VOLUMES LITERALLY
BROKE THE EXISTING
DATABASE. WE WERE NO
LONG ABLE TO SCALE THE
DATABASE OR DO ANYTHING
USEFUL; LIKE RUNNING
QUERIES”
“Two months to migrate to Amazon Redshift.”
Greg Johnson, Head of Analytics, Nokia

Elastic Map Reduce: How does it work?

1. Put the data
into S3 (or HDFS)

S3

EMR Cluster

EMR

3. Get the
results

2. Launch your cluster.
Choose:
• Hadoop distribution
• How many nodes
• Node type (hi-CPU,
hi-memory, etc.)
• Hadoop apps (Hive,
Pig, HBase)

Elastic Map Reduce: How does it work?

EMR Cluster
S3

EMR

You can
easily resize
the cluster

Elastic Map Reduce: How does it work?

EMR Cluster
S3

EMR

Use Spot
nodes to
save time
and money

Elastic Map Reduce: How does it work?

EMR Cluster
S3

EMR

Launch parallel clusters
against the same data
source (tune for the
workload)

Elastic Map Reduce: How does it work?

S3

EMR Cluster

When the work is complete,
you can terminate the cluster
(and stop paying)

Thousands of Customers, 5+ Million Clusters

Give it a try.
Cost to run a 100-node EMR cluster:
£4.90 / hour

Generation

Collection & storage
AWS Data Pipeline

Analytics & computation

Collaboration & sharing

S3, Glacier,
Storage Gateway,
DynamoDB,
Redshift, RDS,
HBase

EC2 &
Elastic MapReduce

EC2 & S3,
CloudFormation,
Elastic MapReduce,
RDS, DynamoDB, Redshift

AWS Data Pipeline

Data-intensive orchestration and automation
Reliable and scheduled
Easy to use, drag and drop
Execution and retry logic
Map data dependencies
Create and manage temporary compute
resources

Thanks.
daviddes@amazon.co.uk

To Learn More:
aws.amazon.com/elasticmapreduce
aws.amazon.com/datapipeline
aws.amazon.com/big-data
aws.amazon.com/redshift
aws.amazon.com/rds

This changes everything. When it comes to data analytics, accuracy and data quality is crucial. Location Hub Analytics ® is the only self-service analytics engine that leverages Canada’s most robust, accurate and up-to-date location-based data for precise, compelling, unbiased results. CLEANSE Location Hub Analytics automatically validates, standardizes, and geocodes your address database. Each record is assigned a Unique Address Identifier (UAID®) ENRICH Location Hub Analytics enriches your data with Canadian demographics information for further analysis and greater customer intelligence. ANALYZE Location Hub Analytics quickly processes and analyzes your data, objectively revealing meaningful patterns and trends INFILL Location Hub Analytics helps you generate new prospect lists by infilling the addresses within a specific territory that are not in your current database. VISUALIZE Unlike other analytics engines, Location Hub Analytics allows you to visualize and interact with your results on a map for better data profiling SHARE Quickly and easily share your customized report with key stakeholders

Talend 6.1 - What's New in Talend?

Talend

Learn more - http://www.talend.com/products/talend-6 When you’re ready to move to Big Data, connect in the cloud, and across the Internet of Things, Talend 6 streamlines the process. Convert traditional data integration jobs and MapReduce jobs to Spark with the click of a button, and realize the potential of real-time data-driven decision making. Learn more about Talend and Spark. Talend 6 also brings continuous delivery, MDM REST API, plus data masking and semantic discovery to our products.

David Waxman Keynote

Data Con LA

David Waxman is the Managing Partner of Big Data Opportunity Stack which focuses on hardware, infrastructure, platforms, algorithms and applications related to AI. Within the next 10 years the world will shift to being AI-first according to Sundar Pichai, CEO of Google. TenOneTen Ventures specializes in industries with large proprietary data sources and aims to create value from that data through domain specialization and intimate industry knowledge.

Benchmarking Digital Readiness: Moving at the Speed of the Market

Apigee | Google Cloud

This document discusses how companies can benchmark their digital readiness and move faster in the digital market. It finds that digital leaders who adopt apps, APIs, and data analytics outperform digital laggards. To move up, companies need business and technology leadership. They should think strategically about customer experience, operations, data, and innovation to access new revenue channels beyond direct monetization. Technologically, companies should take a "cloud first" and "outside in" approach to deliver fast, differentiated customer experiences through systems of engagement built on APIs and backends.

Gartner peer forum sept 2011 orbitz

Raghu Kashyap

The document discusses how Orbitz Worldwide integrated Hadoop into its enterprise data infrastructure to handle large volumes of web analytics and transactional data. Some key points: - Orbitz used Hadoop to store and analyze large amounts of web log and behavioral data to improve services like hotel search. This allowed analyzing more data than their previous 2-week data archive. - They faced initial resistance but built a Hadoop cluster with 200TB of storage to enable machine learning and analytics applications. - The challenges now are providing analytics tools for non-technical users and further integrating Hadoop with their existing data warehouse.

Empower Splunk and other SIEMs with the Databricks Lakehouse for Cybersecurity

Databricks

Cloud, Cost, Complexity, and threat Coverage are top of mind for every security leader. The Lakehouse architecture has emerged in recent years to help address these concerns with a single unified architecture for all your threat data, analytics and AI in the cloud. In this talk, we will show how Lakehouse is essential for effective Cybersecurity and popular security use-cases. We will also share how Databricks empowers the security data scientist and analyst of the future and how this technology allows cyber data sets to be used to solve business problems.

Data Sharing with Snowflake

Snowflake Computing

Every day, businesses across a wide variety of industries share data to support insights that drive efficiency and new business opportunities. However, existing methods for sharing data involve great effort on the part of data providers to share data, and involve great effort on the part of data customers to make use of that data. However, existing approaches to data sharing (such as e-mail, FTP, EDI, and APIs) have significant overhead and friction. For one, legacy approaches such as e-mail and FTP were never intended to support the big data volumes of today. Other data sharing methods also involve enormous effort. All of these methods require not only that the data be extracted, copied, transformed, and loaded, but also that related schemas and metadata must be transported as well. This creates a burden on data providers to deconstruct and stage data sets. This burden and effort is mirrored for the data recipient, who must reconstruct the data. As a result, companies are handicapped in their ability to fully realize the value in their data assets. Snowflake Data Sharing allows companies to grant instant access to ready-to-use data to any number of partners or data customers without any data movement, copying, or complex pipelines. Using Snowflake Data Sharing, companies can derive new insights and value from data much more quickly and with significantly less effort than current data sharing methods. As a result, companies now have a new approach and a powerful new tool to get the full value out of their data assets.

mapr_case_study_experian

Erni Susanti

Experian is an international information services organization that relies on large amounts of consumer and business data. They were facing challenges processing increasing amounts of data within time constraints using their existing system. Experian upgraded to a Linux-based HPC cluster using MapR Distribution including Hadoop to gain more processing power and storage capacity. This allowed Experian to store more data faster and provide clients with enhanced insights, while reducing storage costs and improving speed to market for new services.

The document discusses best practices for big data and business intelligence (BI). It recommends focusing on business objectives, identifying needed data, using the right tools like Hadoop and fast databases, planning mixed architectures, distributing data widely, and tailoring delivery to audiences. It also demonstrates database performance benchmarks and visualization tips. The presentation aims to help organizations effectively use big data to drive value and action.

Web analyticsandbigdata techweek2011

Raghu Kashyap

The document discusses how Orbitz Worldwide uses Hadoop and big data to drive web analytics. It faces challenges with processing massive amounts of log data from millions of searches. Orbitz implemented a Hadoop infrastructure to provide long-term storage, access for developers and analysts, and rapid deployment of reporting applications. This allows Orbitz to aggregate data, run analysis jobs like traffic source mapping in minutes rather than hours, and generate over 25 million records per month. The implementation helps Orbitz shift analytics from innovation to mainstream use across business units.

Enterprise Data Hub: The Next Big Thing in Big Data

Cloudera, Inc.

If you missed Strata + Hadoop World, you missed quite a bit. This year's event was packed with Big Data practitioners across industries who shared their experiences and how they are driving new innovations like never before. Just because you weren't there, doesn't mean you missed out. In this session, we'll touch on a few of the key highlights from the show, including: Key trends in Big Data adoption The enterprise data hub How the enterprise data hub is used in practice

Great Expectations Presentation

Adam Doyle

Great Expectations is an open-source Python library that helps validate, document, and profile data to maintain quality. It allows users to define expectations about data that are used to validate new data and generate documentation. Key features include automated data profiling, predefined and custom validation rules, and scalability. It is used by companies like Vimeo and Heineken in their data pipelines. While helpful for testing data, it is not intended as a data cleaning or versioning tool. A demo shows how to initialize a project, validate sample taxi data, and view results.

Delivering Self-Service Analytics using Big Data and Data Virtualization on t...

Denodo

Watch full webinar here: [https://buff.ly/2FHWnMD] Headquartered in New York City, Guardian Life is one of the largest mutual life insurance companies in the United States. Guardian offerings range from life insurance, disability income insurance, annuities, and investments to dental and vision insurance and employee benefits. The Enterprise Data Program was initiated to modernize Guardian’s technology capabilities and transform how Guardian leverages data – the Enterprise Data Lake was implemented to democratize data and drive self-service analytics throughout the organization. Data virtualization has played a key role for delivering data services through Guardian’s Enterprise Data Marketplace, a centralized portal for analytics and reporting. Attend this session to learn: Who is Guardian and what were the key drivers for building a data lake? What are the data architectural patterns on the cloud? How data virtualization is powering analytics and reporting?

Big Data Real Time Analytics - A Facebook Case Study

Nati Shalom

Building Your Own Facebook Real Time Analytics System with Cassandra and GigaSpaces. Facebook's real time analytics system is a good reference for those looking to build their real time analytics system for big data. The first part covers the lessons from Facebook's experience and the reason they chose HBase over Cassandra. In the second part of the session, we learn how we can build our own Real Time Analytics system, achieve better performance, gain real business insights, and business analytics on our big data, and make the deployment and scaling significantly simpler using the new version of Cassandra and GigaSpaces Cloudify.

Real-time Analytics in Big data

Pratiksha Manan

DAMA Webinar: Turn Grand Designs into a Reality with Data Virtualization

Denodo

Watch full webinar here: https://buff.ly/2HMdbUp What started to evolve as the most agile and real-time enterprise data fabric, data virtualization is proving to go beyond its initial promise and is becoming one of the most important enterprise big data fabrics. Attend this session to learn: • What data virtualization really is, • How it differs from other enterprise data integration technologies • Real-world examples of data virtualization in action from companies such as Logitech, Autodesk and Festo.

Attributes of a Modern Data Warehouse - Gartner Catalyst

Jack Mardack

Most data-driven enterprises continue to struggle to generate the insights they need from their data. More data volumes from more data sources, combined with escalating user concurrency, have led to declining query throughput performance and skyrocketing data warehouse costs. Moreover, modern use cases such as customer-360 and hyper-personalization have blurred the boundaries between operational and analytics systems, making even greater demands on data warehouse solutions.

Data Engineering Efficiency @ Netflix - Strata 2017

Michelle Ufford

Slides from Strata 2017 talk, "Data Engineering Efficiency @ Netflix." Michelle Ufford explains how Netflix’s data engineering and analytics team is using data to find common patterns among the chaos that enable the company to automate repetitive and time-consuming tasks and discover ways to improve data quality, reduce costs, and quickly identify and respond to issues. Michelle provides a quick overview of Netflix’s analytics environment before diving into some of the major challenges facing the company’s data engineers. Along the way, Michelle shares how Netflix is building more intelligent data platform services and tools to improve data quality, automate data maintenance, alert on job optimization opportunities, and more.

Google BigQuery is the future of Analytics! (Google Developer Conference)

Rasel Rana

Rethink Analytics with an Enterprise Data Hub

Cloudera, Inc.

MapR Enterprise Data Hub Webinar w/ Mike Ferguson

MapR Technologies

This document discusses best practices for using Hadoop as an enterprise data hub. It provides an overview of how big data is driving new analytical workloads and the need for deeper customer insights. It discusses challenges with analyzing new sources of structured, unstructured and multi-structured data. It introduces the concept of a Hadoop enterprise data hub and data refinery to simplify access to new insights from big data. Key components of the data hub include a data reservoir to capture raw data from various sources, a data refinery to cleanse and transform the data, and publishing high value insights to data warehouses and other systems.

Talend winter 2017 overview webinar

Jean-Michel Franco

Talend Winter 17 enables IT to transform the data lake into qualified, clean data that anyone can use, so everyone can make more informed and faster decisions The presentations covers mostly three key areas how Talend helps you get the most from your data lake. Talend Data Preparation now has Big Data support so anyone can access trusted data in the lake and turn data into insight New Talend Data Stewardship app helps IT and Business to collaborate on data quality problems and guide resolution. It empowers the business to ensure data integrity at the source. 3. And we all know that there is an amazing amount of innovation going on in the market today. Talend enables you to stay on the cutting edge of big data and cloud innovation with the flexibility to leverage pretty much anything out there in the market, such as Spark 2.0, AWS, Salesforce, MapR and more ….

Introduction to Big Data

AmpoolIO

Hadoop World 2011: Advancing Disney’s Data Infrastructure with Hadoop - Matt ...

Cloudera, Inc.

This is the story of why and how Hadoop was integrated into the Disney data infrastructure. Providing data infrastructure for Disney’s, ABC’s and ESPN’s Internet presences is challenging. Doing so requires cost effective, performant, scalable and highly available solutions. Information requirements from the business add the need for these solutions work together; providing consistent acquisition, storage and access to data. Burdened with a heavily laden commercial RDBMS infrastructure, Hadoop provided an opportunity to solve some challenging use cases at Disney. The deployment of Hadoop helped Disney to address growing costs, scalability, and data availability. In addition, it provids our businesses with new data driven business to consumer opportunities.

Big Data in Action : Operations, Analytics and more

Softweb Solutions

democratization of data sql-konferenz

Jen Stirrup

"Don’t worry about people stealing an idea. If it’s original, you will have to ram it down their throats.” Howard Aiken, Founder of Harvard’s Computing Science Program. Data is moving so fast these days, and there is a shift whereby people are paying for value, not technology. This is where cloud computing comes in: it is very empowering, because anyone with an internet connection can access it. With Power BI in the cloud, small businesses are liberated with the ability to use the same tools and techniques to explore ideas as larger organisations. In this session, we will look at understanding the Power BI components and tools available in the cloud, including the Power BI Admin Center, Power Query, Power Pivot, Power View and Power Map. We will look at how to use them will accelerate ideas and help to clarify decisions, and related to this, discuss the roles within IT and the business in relation to these tools. We will also look at business puzzles versus business mysteries, a definition evoked by Malcolm Gladwell (Blink, Outliers) in relation to Power BI. “Out there in some garage is an entrepreneur who’s forging a bullet with your company’s name on it,” said Gary Hamel, a management guru. With Power BI, let’s see how you can translate your ideas in to a message that people can see, using cloud as an empowerment tool.

IBM Big Data in the Cloud

Rob Thomas

IBM InfoSphere BigInsights provides Hadoop on cloud computing platforms so that users can analyze large volumes of data without requiring large upfront investments in hardware, storage, and networking. It allows users to deploy their own Hadoop clusters on public clouds like Amazon or private clouds in under 30 minutes, paying only for the resources used on an hourly basis starting at $0.34 per node per hour. BigInsights can be deployed on IBM SmartCloud Enterprise with hourly charges starting at $0.30 per cluster per hour and a free trial during a fall promotion. It makes evaluating and learning Hadoop easy without needing to configure hardware or install software.

Big Data Analytics on the Cloud

Caserta

Caserta Concepts, Datameer and Microsoft shared their combined knowledge and a use case on big data, the cloud and deep analytics. Attendes learned how a global leader in the test, measurement and control systems market reduced their big data implementations from 18 months to just a few. Speakers shared how to provide a business user-friendly, self-service environment for data discovery and analytics, and focus on how to extend and optimize Hadoop based analytics, highlighting the advantages and practical applications of deploying on the cloud for enhanced performance, scalability and lower TCO. Agenda included: - Pizza and Networking - Joe Caserta, President, Caserta Concepts - Why are we here? - Nikhil Kumar, Sr. Solutions Engineer, Datameer - Solution use cases and technical demonstration - Stefan Groschupf, CEO & Chairman, Datameer - The evolving Hadoop-based analytics trends and the role of cloud computing - James Serra, Data Platform Solution Architect, Microsoft, Benefits of the Azure Cloud Service - Q&A, Networking For more information on Caserta Concepts, visit our website: http://casertaconcepts.com/

Scmp aws digitalmedia_2013

AWS Big Data Analytics IP Expo 2013

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to AWS Big Data Analytics IP Expo 2013

Similar to AWS Big Data Analytics IP Expo 2013 (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Recently uploaded

Recently uploaded (20)

AWS Big Data Analytics IP Expo 2013