Analytics on AWS:Structured, Unstructured and Streaming

•

6 likes•2,577 views

Data comes in a variety of forms and in order to gain insight from this data you need to have the right platform in place. AWS has the services to cover all types of data, whether you need databases for structured data, Hadoop for unstructured data or a streaming engine for high-velocity data. In this session we will cover the various data analytics services on AWS and when to use them.

Technology

Analytics on AWS
Structured, Unstructured and Streaming
Russell Nash
AWS Solutions Architect

Databases
Database
Data
Log
Data
High Velocity
Data
Analytics
Database
INGEST STORE
Devices
Web Servers
App Servers
Mobile
Amazon
Redshift
Amazon
RDS

Traditional Database
Choice of Engines
Low Admin
Amazon
RDS
(Relational
Database
Service)
Amazon Aurora

MPP SQL Database
Optimised for Analytics
Scalable
Amazon
Redshift

Amazon
Redshift
Amazon
RDS
ScalingVertical Horizontal
WorkloadMixed Analytical
VolumesLow to Medium Medium to High
TypeSQL Relational SQL Relational

Database
Data
ETL
INGEST STORE
High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Databases
Log
Data
Amazon
Redshift
Amazon
RDS

AWS Database
Migration Service
Amazon
Redshift
Source
Database
ETL
ETL Partners
Amazon
RDS

Storage
INGEST STORE
High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Database
Data
Databases
Log
Data
Search
Amazon
Elasticsearch
Amazon
Redshift
Amazon
RDS

Amazon
Elasticsearch
Search and Analytics
Scalable
Integrated – Logstash, Kibana

Database
Data
Storage
INGEST STORE
High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Databases
Log
Data
Amazon
Elasticsearch
Amazon
S3
Amazon
Redshift
Amazon
RDS

Amazon
S3
Object Storage
Low Cost
11 9’s of durability

CPU
c4 family
c3 family
Memory
x1 family
r3 family
Disk/IO
d2 family
i2 family
General
m4 family
m3 family
Instance Types
Batch Machine Spark and Large
process learning interactive HDFS

Cost & Time
# CPUs
Time
# CPUs
Time
Wall clock time: 1 hourWall clock time: 10 hours

Spot Price – M3.2XL
On-Demand Spot-Price
$0.08$0.75

Database
Data
INGEST STORE
Mobile High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Databases
Log
Data
Amazon
Elasticsearch
Amazon
S3
Stream
Processor
Amazon
Kinesis
NoSQL
Amazon
Redshift
Amazon
RDS

Availability
Zone
Availability
Zone
Availability
Zone
Data
Sources
Data
Sources
Data
Sources
Data
Sources
Data
Sources
S3
Redshift
Amazon Kinesis
Stream
AWS Lambda
KCL App
EMR
Elasticsearch

Database
Data
INGEST STORE
High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Databases
Log
Data
Amazon
Elasticsearch
Amazon
S3
Amazon
Kinesis
NoSQL
Amazon
DynamoDB
Amazon
Redshift
Amazon
RDS

scalability
data complexity
RDBMS
key/value
document
graph
NoSQL

NoSQL Database
Key/Value + Document
Very Low Latency
Amazon
DynamoDB

INGEST STORE PROCESS
Databases
Amazon
Redshift
Amazon
Kinesis
Amazon
S3
Impala
Amazon
Redshift
Amazon
EMR
Database
Data
InteractiveBatchStreaming
Hadoop
Amazon
Elasticsearch
Log
Data
Mobile High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Amazon
DynamoDB

INGEST STORE PROCESS
Impala
Amazon
Redshift
Database
Data
InteractiveBatch
PIG
Streaming
Amazon
EMR
Hadoop
Amazon
Kinesis
Amazon
S3
Amazon
Elasticsearch
Mobile High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Amazon
DynamoDB
Databases
Log
Data
Amazon
Redshift

INGEST STORE PROCESS
Impala
Amazon
Redshift
AWS
Lambda
Kinesis
Consumers
Database
Data
InteractiveBatchStreaming
PIG
Amazon
EMR
Hadoop
Amazon
Kinesis
Amazon
S3
Amazon
Elasticsearch
Mobile High Velocity
Data
Devices
Web Servers
App Servers
Mobile
Amazon
DynamoDB
Databases
Log
Data
Amazon
Redshift

Comparison of Query Engines
Amazon Redshift Amazon
Elasticsearch
Data Structure
Languages
Semi SemiSemi
Multiple Full Text SearchSQL
Full
SQL
Data Store S3/HDFS LocalS3/HDFS Local
Performance

Comparison of Query Engines
Amazon Redshift Amazon
Elasticsearch
General
Purpose
Processing
Engine
SQL Query
Engine for
S3/HDFS
Fully
Featured SQL
Analytics
Database
Full Text search
VisualizationUse Case

Visit bit.ly/aws-tech-chat
or download via SoundCloud or iTunes

Analytics on AWS:Structured, Unstructured and Streaming

In this session, you will learn best practices for implementing simple to advanced real-time streaming data use cases on AWS. First, we will review decision points on near real-time versus real time scenarios. Next, we will take a look at streaming data architecture patterns that include Amazon Kinesis Analytics, Amazon Kinesis Firehose, Amazon Kinesis Streams, Spark Streaming on Amazon EMR, and other open source libraries. Finally, we will dive deep into the most common of these patterns and cover design and implementation considerations.

ENT314 Automate Best Practices and Operational Health for Your AWS Resources

Amazon Web Services

"It can be challenging to optimize AWS resources across cost, performance, security and fault-tolerance, much less do it automatically. AWS Trusted Advisor is an online resource to help you do just that, by providing real time guidance to help you provision your resources following AWS best practices. In this session, we will go over how to safely automate these best practices using Amazon CloudWatch events and AWS Lambda along with samples for you to use. AWS Personal Health Dashboard (PHD) provides alerts and remediation guidance when AWS is experiencing events that may impact your AWS environment. The AWS Health API, the underlying service powering PHD integrates with Amazon CloudWatch Events, enabling you to trigger AWS Lambda functions to define automated remediation actions. We will also introduce you to AWS Health tools, a community-based source of tools to automate remediation actions and customize Health alerts. Come join us to see how you can implement automation of AWS best practice recommendations from Trusted Advisor and remediation from the AWS Health API on your AWS resources."

Modern Data Architectures for Business Insights at Scale

Amazon Web Services

Modern Data Architectures for Real Time Analytics & Engagement

Amazon Web Services

Full Stack Analytics on AWS - AWS Summit Cape Town 2017

Amazon Web Services

Building analytics applications requires more than just one good service. It requires the ability to capture a vast amount of data, and react to data changes in real time. It requires flexible tools which enable end users to work in the way they can be most productive, and which addresses the needs of both data consumers, as well as data scientists. This analysis won't just be about data exploration and reports, but must be able to support the largest scale, complex machine and deep learning models imaginable. Across it all, strong governance, security, and cataloguing is essential. In this session, come to hear about how to build a full stack analytics application using AWS Services. We'll see how to capture static and dynamic data in real time, and react to data changes. We'll see AWS Services which perform analytics from drag-and-drop, through simple query-on-files, and into exascale data science. At the end, we'll have a data lake architecture that will meet the demands of the most sophisticated analytics customers for many years to come. AWS Speaker: Ian Robinson, Specialist Solution Architect, Big Data and Analytics, EMEA - Amazon Web Services

Big Data Architectural Patterns and Best Practices

Amazon Web Services

ENT316 Keeping Pace With The Cloud: Managing and Optimizing as You Scale

Amazon Web Services

"With cloud maturity comes operational efficiencies and endless potential for innovation and business growth. However, the complexities of governing cloud infrastructure are impeding without the right strategy. Visibility, accountability, and actionable insights are some of the most invaluable considerations. The AWS cloud clearly enables convenience and cost savings for organizations that know how to leverage its full potential. Amazon EC2 Reserved Instances (RIs) in particular, present a tremendous opportunity when scaling to save significantly on capacity but there are many considerations to fully reaping the benefits of RIs. In this session, CloudCheckr CTO Patrick Gartlan will present issues that every organization runs into when scaling, provide best practices for how to combat them and help you show your boss how RIs help you save money and move faster. This session is brought to you by AWS Summit New York City sponsor, CloudCheckr. "

Data-driven companies have a need to make their data easily accessible to those who analyze it. Many organizations have adopted the Looker application, LookML on AWS, a centralized analytical database with a user-friendly interface that allows employees to ask and answer their own questions to make informed business decisions. Join our webinar to learn how our customer, Casper, an online mattress retailer, made the switch from a transactional database to Looker’s data analytics program on Amazon Redshift. Looker on Amazon Redshift can help you greatly reduce your analytics lifecycle with a simplified infrastructure and rapid cloud scaling. Join us to learn: • How to utilize LookML to build reusable definitions and logic for your data • Best practices for architecting a centralized analytical database • How Casper leveraged Looker and Amazon Redshift to provide all their employees access to their data and metrics Who should attend: Heads of Analytics, Heads of BI, Analytics Managers, BI Teams, Senior Analysts

Big Data & Analytics: End to End on AWS - Technical 101

Amazon Web Services

AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)

Amazon Web Services

For discovery-phase research, life sciences companies have to support infrastructure that processes millions to billions of transactions. The advent of a data lake to accomplish such a task is showing itself to be a stable and productive data platform pattern to meet the goal. We discuss how to build a data lake on AWS, using services and techniques such as AWS CloudFormation, Amazon EC2, Amazon S3, IAM, and AWS Lambda. We also review a reference architecture from Amgen that uses a data lake to aid in their Life Science Research.

AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...

Amazon Web Services

Building big data applications often requires integrating a broad set of technologies to store, process, and analyze the increasing variety, velocity, and volume of data being collected by many organizations. Using a combination of Amazon EMR, a managed Hadoop framework, and Amazon Redshift, a managed petabyte-scale data warehouse, organizations can effectively address many of these requirements. In this webinar, we will show how organizations are using Amazon EMR and Amazon Redshift to build more agile and scalable architectures for big data. We will look into how you can leverage Spark and Presto running on EMR, to address multiple data processing requirements. We will also share best practices and common use cases to integrate EMR and Redshift. Learning Objectives: • Best practices for building a big data architecture that includes Amazon EMR and Amazon Redshift • Understand how to use technologies such as Amazon EMR, Presto and Spark to complement your data warehousing environment • Learn key use cases for Amazon EMR and Amazon Redshift Who Should Attend: • Data architects, Data management professionals, Data warehousing professionals, BI professionals

Getting started with Amazon Kinesis

Amazon Web Services

Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017

Amazon Web Services

NEW LAUNCH! Intro to Amazon Athena. Easily analyze data in S3, using SQL.

Amazon Web Services

Amazon Athena is a new interactive query service that makes it easy to analyze data in Amazon S3, using standard SQL. Athena is serverless, so there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3. In this session, we will show you how easy is to start querying your data stored in Amazon S3, with Amazon Athena. First we will use Athena to create the schema for data already in S3. Then, we will demonstrate how you can run interactive queries through the built-in query editor. We will provide best practices and use cases for Athena. Then, we will talk about supported queries, data formats, and strategies to save costs when querying data with Athena.

AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...

Amazon Web Services

Join us for this lightning-round showcase of hot new brands and startup companies that are using AWS to play a really big game. You'll hear from experts like Mapbox CIO Will White, Ring Senior Engineer Jason Gluckman, Hudl Engineering Director Rob Hruska, and many others as they explain how they thought about the problems they faced and how they solved them in this TED-style session packed with lots of creative thinking.

The Best of re:invent 2016

Amazon Web Services

AWS re:Invent 2016 was AWS’ largest event yet with over 32,000 attendees, 400 breakout sessions, and two keynotes of new product announcements. In this talk, we’ll explore the core themes of AWS re:Invent 2016 such as serverless and artificial intelligence. We will also drill down into several of the services and features unveiled including AWS Batch, AWS Shield, Aurora for Postgres, X-Ray, Polly, Lex, Rekognition, AWS Step Functions. Light appetizers and refreshments will be provided.

AWS re:Invent 2016: Achieving Agility by Following Well-Architected Framework...

Amazon Web Services

The AWS Well-Architected Framework enables customers to understand best practices around security, reliability, performance, and cost optimization when building systems on AWS. This approach helps customers make informed decisions and weigh the pros and cons of application design patterns for the cloud. In this session, you'll learn how National Instruments used the Well-Architected Framework to follow AWS guidelines and best practices. By developing a strategy based on the AWS Well-Architected Framework, National Instruments was able to triple the number of applications running in the cloud without additional head count, significantly increase the frequency of code deployments, and reduce deployment times from two weeks to a single day. As a result, National Instruments was able to deliver a more scalable, dynamic, and resilient LabVIEW platform with agility.

Build a Website on AWS for Your First 10 Million Users

Amazon Web Services

ENT302 Deep Dive on AWS Management Tools and New Launches

Amazon Web Services

As companies shift workloads into the cloud, IT organizations are required to manage an increasing number of cloud resources. AWS provides a broad set of services that help IT organizations with provisioning, tracking, auditing, configuration management, and cost management of their AWS resources. In this session, we will explore the AWS Management Tools suite of services that support the lifecycle management of AWS resources at scale and enable IT governance and compliance. The Deep Dive on AWS Management Tools session will benefit both new and experienced IT administrators, systems administrators, and developers operating infrastructure on AWS and interested in learning about the AWS resource management capabilities.

ENT201 A Tale of Two Pizzas: Accelerating Software Delivery with AWS Develope...

Amazon Web Services

Software release cycles are now measured in days instead of months. Cutting edge companies are continuously delivering high-quality software at a fast pace. In this session, we will cover how you begin your DevOps journey by sharing best practices and tools by the "two pizza" engineering teams at Amazon. We will showcase how you can accelerate developer productivity by implementing continuous integration and delivery workflows. We will also cover an introduction to AWS CodeCommit, AWS CodeBuild, AWS CodePipeline, and AWS CodeDeploy, the services inspired by Amazon's internal devloper tools and DevOps practice.

Your First Data Lake on AWS_Simon Elisha

Helen Rogers

BDA309 Building Your Data Lake on AWS

Amazon Web Services

"Increasing demands to collect, store, and analyze massive amounts of data often means that the same tools and approaches that worked in the past, don't work anymore. That's why many organizations are shifting to a data lake architecture. A data lake is an architectural approach that allows you to store massive amounts of data into a central location, so it's readily available to be categorized, processed, analyzed and consumed by diverse groups within an organization. In this tech talk, we introduce key concepts for a data lake and present aspects related to its implementation. We highlight the core components of a data lake, such as storage, compute, analytics, databases, stream processing, data management, and security. We discuss how to choose the right technologies for each component of the data lake, based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. We also provide a reference architecture and recommendations to get started with a data lake implementation on AWS. Learning Objectives: Understand key concepts and architectural components of a data lake architecture Describe how and when to use a broad set of analytic and data management tools in a data lake architecture Get insights on how to get started with a data lake implementation on AWS"

Soluzioni di Database completamente gestite: NoSQL, relazionali e Data Warehouse

Amazon Web Services

BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis

Amazon Web Services

Thousands of services work in concert to deliver millions of hours of video streams to Netflix customers every day. These applications vary in size, function, and technology, but they all make use of the Netflix network to communicate. Understanding the interactions between these services is a daunting challenge both because of the sheer volume of traffic and the dynamic nature of deployments. In this talk, we’ll first discuss why Netflix chose Amazon Kinesis Streams over other streaming data solutions like Kafka to address these challenges at scale. We’ll then dive deep into how Netflix uses Amazon Kinesis Streams to enrich network traffic logs and identify usage patterns in real time. Lastly, we will cover how Netflix uses this system to build comprehensive dependency maps, increase network efficiency, and improve failure resiliency. From this talk, you’ll take away techniques and processes that you can apply to your large-scale networks and derive real-time, actionable insights.

Big Data Architectural Patterns and Best Practices on AWS

Amazon Web Services

In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. John Pignata, AWS Startup Solutions Architect, will discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. He will provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

Deep dive and best practices on real time streaming applications nyc-loft_oct...

Amazon Web Services

Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017

Amazon Web Services

Real-Time Streaming Analytics became popular amongst many verticals and use cases. In AdTech, Gaming, Financial Service and IoT, AWS customers are leveraging Amazon Kinesis platform to ingest billions of events every day and process them in real-time. In this session, we will discuss Amazon Kinesis Streams, Amazon Kinesis Firehose and Amazon Kinesis Analytics. We will show best practice and design patterns in integrating Amazon Kinesis platform with other services like Amazon EMR, Redshift, Amazon Elasticsearch and AWS lambda as well as 3rd party connectors like storm, Spark and more.

Real Time Analytics On AWS: Optimized Architectures

Amazon Web Services

Building a real-time analytics solution has never been faster or more cost-efficient. Most organizations are trying to find a way to improve customer experience and respond to business events in real time. Importantly, to do this quickly and at a fraction of the price of traditional approaches. In this session we will look at how to use the AWS services to best meet your real-time analytics needs.

Structured, Unstructured and Streaming Big Data on the AWS

Amazon Web Services

Using AWS has never been easier or more affordable to solve business problems and uncover new opportunities using data. Now, businesses of all sizes and across all industries can take advantage of big data technologies and easily collect, store, process, analyze, and share their data. Gain a thorough understanding of what AWS offers across the big data lifecycle and learn architectural best practices for applying these technologies to your projects. We will also deep dive into how to use AWS services such as Kinesis, DynamoDB, Redshift, and Quicksight to optimize logging, build real-time applications, and analyze and visualize data at any scale.

What's hot

A Data Culture with Embedded Analytics in Action

Amazon Web Services

Big Data & Analytics: End to End on AWS - Technical 101

Amazon Web Services

AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)

Amazon Web Services

AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...

Amazon Web Services

Getting started with Amazon Kinesis

Amazon Web Services

Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017

Amazon Web Services

NEW LAUNCH! Intro to Amazon Athena. Easily analyze data in S3, using SQL.

Amazon Web Services

AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...

Amazon Web Services

The Best of re:invent 2016

Amazon Web Services

AWS re:Invent 2016: Achieving Agility by Following Well-Architected Framework...

Amazon Web Services

Build a Website on AWS for Your First 10 Million Users

Amazon Web Services

ENT302 Deep Dive on AWS Management Tools and New Launches

Amazon Web Services

ENT201 A Tale of Two Pizzas: Accelerating Software Delivery with AWS Develope...

Amazon Web Services

Your First Data Lake on AWS_Simon Elisha

Helen Rogers

BDA309 Building Your Data Lake on AWS

Amazon Web Services

Soluzioni di Database completamente gestite: NoSQL, relazionali e Data Warehouse

Amazon Web Services

BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis

Amazon Web Services

Big Data Architectural Patterns and Best Practices on AWS

Amazon Web Services

Deep dive and best practices on real time streaming applications nyc-loft_oct...

Amazon Web Services

Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017

Amazon Web Services

What's hot (20)

A Data Culture with Embedded Analytics in Action

Big Data & Analytics: End to End on AWS - Technical 101

AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)

AWS March 2016 Webinar Series - Building Big Data Solutions with Amazon EMR a...

Getting started with Amazon Kinesis

Big Data adoption success using AWS Big Data Services - Pop-up Loft TLV 2017

NEW LAUNCH! Intro to Amazon Athena. Easily analyze data in S3, using SQL.

AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...

The Best of re:invent 2016

AWS re:Invent 2016: Achieving Agility by Following Well-Architected Framework...

Build a Website on AWS for Your First 10 Million Users

ENT302 Deep Dive on AWS Management Tools and New Launches

ENT201 A Tale of Two Pizzas: Accelerating Software Delivery with AWS Develope...

Your First Data Lake on AWS_Simon Elisha

BDA309 Building Your Data Lake on AWS

Soluzioni di Database completamente gestite: NoSQL, relazionali e Data Warehouse

BDA403 How Netflix Monitors Applications in Real-time with Amazon Kinesis

Big Data Architectural Patterns and Best Practices on AWS

Deep dive and best practices on real time streaming applications nyc-loft_oct...

Amazon Kinesis Platform – The Complete Overview - Pop-up Loft TLV 2017

Viewers also liked

Real Time Analytics On AWS: Optimized Architectures

Amazon Web Services

Structured, Unstructured and Streaming Big Data on the AWS

Amazon Web Services

All you need to know about Auto scaling - Pop-up Loft

Amazon Web Services

Adapting the capacity of your compute infrastructure to the demands of your applications is the domain of Auto Scaling. Adding and removing Amazon EC2 instances is only part of the story, though – there is more to it than first meets the eye. This session introduces the basics of how to use Auto Scaling before moving on to more advanced topics such as mixing Spot and On-Demand instances to optimize cost or strategies for blue/green deployments. If you have used Auto Scaling before, you can learn about useful new features like lifecycle hooks and step scaling policies that make Auto Scaling even more widely applicable.

Highly secure content delivery at global scale with amazon cloudfront

Amazon Web Services

February 2016 Webinar Series - Introduction to AWS Database Migration Service

Amazon Web Services

AWS Database Migration Service helps you migrate databases to AWS easily and securely with minimal downtime to the source database. AWS Database Migration Service can be used for both homogeneous and heterogeneous database migrations from on-premise to RDS or EC2 as well as EC2 to RDS. In this webinar, we will provide an introduction to AWS Database Migration Service and go through the details of how you can use it today for your database migration projects. We will also discuss AWS Schema Conversion Tool that help you convert your database schema and code for cross database (heterogeneous) migrations. Learning Objectives: Understand what is AWS Database Migration Service Learn how to start using AWS Database Migration Service Understand homogenous and heterogeneous migrations Learn about AWS Schema Conversions Tool Who Should Attend: IT Managers, DBAs, Solution Architects, Engineers and Developers

AWS re:Invent 2016: Configuration Management in the Cloud (DEV305)

Amazon Web Services

To ensure that your application operates in a predictable manner in both your test and production environments, you must vigilantly maintain the configuration of your resources. By leveraging configuration management solutions, Dev and Ops engineers can define the state of their resources across their entire lifecycle. In this session, we will show you how to use AWS OpsWorks, AWS CodeDeploy, and AWS CodePipeline to build a reliable and consistent development pipeline that assures your production workloads behave in a predictable manner.

Amazon RDS Deep Dive

Amazon Web Services

DDoS Resiliency

Amazon Web Services

Architecting a 24x7 Live Linear Broadcast for Availability on AWS

Amazon Web Services

Amazon DynamoDB Lessen's Learned by Beginner

Hirokazu Tokuno

Big Data Architectural Patterns and Best Practices on AWS

Amazon Web Services

The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

Big Data Architectural Patterns

Amazon Web Services

Technical Lessons on how to do Backup and Disaster Recovery in the Cloud

Amazon Web Services

Maintaining data integrity and guaranteeing business continuity is of utmost importance for any organization. However, in today's world, those systems have grown in complexity and cost, while the business demands IT agility and lower costs. In this talk, we will explore how organizations should approach backup and disaster recovery, and how these two aspects can be implemented in the cloud to improve efficiency and flexibility. The talk starts with general concepts, and then dives into technical details, culminating in real customer examples that showcase some tips and tricks and the benefits of a cloud-based approach.

Masterclass Live: Amazon EC2

Amazon Web Services

Amazon EC2 forms the backbone compute platform for hundreds of thousands of AWS customers, but how do you go beyond starting an instance and manually configuring it? In this webinar we will take you on a journey starting with the basics of key management and security groups and ending with an explanation of Auto Scaling and how you can use it to match capacity and costs to demand using dynamic policies.

AWS Webcast - Using the AWS Cloud for Disaster recovery_Public Sector

Amazon Web Services

State, Local and Education customers are using the AWS cloud to enable faster disaster recovery of their mission critical IT systems without incurring the infrastructure expense of a second physical site. This presentation shares informative on how AWS cloud supports many popular disaster recovery (DR) architectures from “pilot light” environments that are ready to scale up at a moment’s notice to “hot standby” environments that enable rapid failover. With infrastructure centers in 10 regions around the world, AWS provides a set of cloud-based DR services that enable rapid recovery of your IT infrastructure and data.

AWS Partner Presentation - Suse Linux Proven Cloud SuccessAmazon Web Services

AWS Webcast - Webinar Series for State and Local Government #3: Discover the ...

Amazon Web Services

This webinar will provide an overview of tools that help you deploy AWS resources easily and quickly using publicly available Amazon Machine Images (AMIs), SaaS solutions, and CloudFormation templates. This session will dig deeper into how to launch critical business applications on AWS such as deploy an emergency website, launch SharePoint server and more. The focus of the webinar will be on demonstrating vast ecosystem of AWS to help customers deploy business critical applications quickly without the steep learning curve.

REA Sydney Customer Appreciation Day

Amazon Web Services

Amazon federal registry 2.0Amazon Web Services

AWS Customer Presentation - The Guardian

Amazon Web Services

Viewers also liked (20)

Real Time Analytics On AWS: Optimized Architectures

Structured, Unstructured and Streaming Big Data on the AWS

All you need to know about Auto scaling - Pop-up Loft

Highly secure content delivery at global scale with amazon cloudfront

February 2016 Webinar Series - Introduction to AWS Database Migration Service

AWS re:Invent 2016: Configuration Management in the Cloud (DEV305)

Amazon RDS Deep Dive

DDoS Resiliency

Architecting a 24x7 Live Linear Broadcast for Availability on AWS

Amazon DynamoDB Lessen's Learned by Beginner

Big Data Architectural Patterns and Best Practices on AWS

Big Data Architectural Patterns

Technical Lessons on how to do Backup and Disaster Recovery in the Cloud

Masterclass Live: Amazon EC2

AWS Webcast - Using the AWS Cloud for Disaster recovery_Public Sector

AWS Partner Presentation - Suse Linux Proven Cloud Success

AWS Webcast - Webinar Series for State and Local Government #3: Discover the ...

REA Sydney Customer Appreciation Day

Amazon federal registry 2.0

AWS Customer Presentation - The Guardian

Similar to Analytics on AWS:Structured, Unstructured and Streaming

Big Data and Analytics

Amazon Web Services

Big Data and Analytics – End to End on AWS – Russell Nash

Amazon Web Services

Building a Modern Data Architecture on AWS - Webinar

Amazon Web Services

Amazon Web Services gives you fast access to flexible and low cost IT resources, so you can rapidly scale and build virtually any big data application including data warehousing, clickstream analytics, fraud detection, recommendation engines, event-driven ETL, serverless computing, and internet-of-things processing regardless of volume, velocity, and variety of data. https://aws.amazon.com/webinars/anz-webinar-series/

Big Data Architectural Patterns and Best Practices on AWS

Amazon Web Services

by Dario Rivera, Solutions Architect, AWS The world is producing an ever-increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

BDA303 Serverless big data architectures: Design patterns and best practices

Amazon Web Services

Serverless technologies let you build and scale applications and services rapidly without the need to provision or manage servers. But how can you incorporate serverless concepts into your big data architectures? In this session, we explore the key concepts and benefits of serverless architectures for big data, diving into design patterns to ingest, store, process, and visualize your data. Along the way, we explain when and how you can use serverless technologies to streamline data processing, minimize infrastructure management, and improve agility and robustness. We will share reference architectures using a combination of services that include AWS Lambda, Amazon Kinesis, Amazon Athena, Amazon QuickSight, and AWS Glue.

Building Data Warehouses and Data Lakes in the Cloud - DevDay Austin 2017 Day 2Amazon Web Services

Building A Modern Data Analytics Architecture on AWS

Amazon Web Services

Amazon Web Services gives you fast access to flexible and low cost IT resources, so you can rapidly scale and build virtually any big data and analytics application including data warehousing, clickstream analytics, fraud detection, recommendation engines, event-driven ETL, serverless computing, and internet-of-things processing regardless of volume, velocity, and variety of data. In this one-hour webinar, we will look at the portfolio of AWS Big Data services and how they can be used to build a modern data architecture. We will cover: Using different SQL engines to analyze large amounts of structured data Analysing streaming data in near-real time Architectures for batch processing Best practices for Data Lake architectures This session is suited for: Solution and enterprise architects Data architects/ Data warehouse owners IT & Innovation team members

BDA305 Building Data Lakes and Analytics on AWS

Amazon Web Services

Over 90% of today’s data was generated in the last 2 years, and the rate of data growth isn’t slowing down. In this session, we’ll step through the challenges and best practices on how to capture all the data that is being generated, understand what data you have, and start driving insights and even predict the future using purpose built AWS Services. We’ll frame the session and demonstrations around common pitfalls of building Data Lakes and how to successful drive analytics and insights from the data. This session will focus on the architecture patterns bringing together key AWS Services and rather than a deep dive on any single service. We’ll show how services such as Amazon S3, Amazon Glue, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, and Amazon Kinesis, and Amazon Machine Learning services are put together to build a successful data lake for various role including both data scientists and business users.

Building your first Data lake platform

Amazon Web Services

By using a Data Lake, you no longer need to worry about structuring or transforming data before storing it. A Data Lake on AWS enables your organization to more rapidly analyze data, helping you quickly discover new business insights. Join us for our webinar to learn about the benefits of building a Data Lake on AWS and how your organization can begin reaping their rewards. In this session, we will share methodology for implementing a Data Lake on AWS and best practices for getting the most from your Data Lake. Speaker: Russell Nash, APAC Solution Architect, DW, AWS APAC

Simplify Big Data with AWS

Julien SIMON

Understanding AWS Managed Database and Analytics Services | AWS Public Sector...

Amazon Web Services

The world is creating more data in more ways than ever before. The average internet user in 2017 generates 1.5GB of data per day, with the rate doubling every 18 months. A single autonomous vehicle can generate 4TB per day. Each smart manufacturing plant generates 1PB per day. Storing, managing, and analyzing this data requires integrated database and analytic services that provide reliability and security at scale. AWS offers a range of managed data services that let customers focus on making data useful, including Amazon Aurora, RDS, DynamoDB, Redshift, Spectrum, ElastiCache, Kinesis, EMR, Elasticsearch Service, and Glue. In this session, we discuss these services, share our vision for innovation, and show how our customers use these services today. Learn More: https://aws.amazon.com/government-education/

Understanding AWS Managed Database and Analytics Services | AWS Public Sector...

Amazon Web Services

AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...

Amazon Web Services

Building big data applications often requires integrating a broad set of technologies to store, process, and analyze the increasing variety, velocity, and volume of data being collected by many organizations. In this session, we show how you can build entire big data applications using a core set of managed services including Amazon S3, Amazon Kinesis, Amazon EMR, Amazon Elasticsearch Service, Amazon Redshift, and Amazon QuickSight. We walk you through the steps of building and securing a big data application using the AWS Big Data Platform. We also share best practices and common use cases for AWS big data services, including tips to help you choose the best services for your specific application.

Getting Started with Big Data and HPC in the Cloud - August 2015

Amazon Web Services

How can you use Big Data to grow your business and discover new opportunities? When organizations effectively capture, analyze, visualize and apply big data insights to their business goals, they differentiate themselves from their competitors and outperform them in terms of operational efficiency and the bottom line. With Amazon Web Services, businesses and researchers can easily fulfill their high performance computing (HPC) requirements with the added benefit of ad-hoc provisioning, pay-as-you-go pricing and faster time-to-results. Join this session to understand how to run HPC applications in AWS cloud, and about different AWS Big Data and Analytics services such as Amazon Elastic MapReduce (Hadoop), Amazon Redshift (Data Warehouse) and Amazon Kinesis (Streaming), when to use them and how they work together.

Database and Analytics on the AWS Cloud - AWS Innovate Toronto

Amazon Web Services

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

Amazon Web Services

In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes, and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.

AWS Analytics Immersion Day - Build BI System from Scratch (Day1, Day2 Full V...

Sungmin Kim

How to build Business Intelligence System from scratch on AWS (Day1, Day2) ------------------------------------------------------------------------------------------ 2020-03-18(수)~19(목) 2일 동안 온라인으로 진행한 Online AWS Analytics Immersion Day 전체 발표 자료 입니다. BI(Business Intelligence) 시스템을 설계하는 과정에서 AWS Analytics 서비스들을 어떻게 활용할 수 있는지 설명 드리고자 만든 자료 입니다. Target Audience ------------------- Online Analytics Immersion Day는 다음과 같은 고객을 대상으로 진행됩니다. - AWS Analytics Services (ex. Kinesis, Athena, Redshift, EMR, etc)의 기본 개념을 알고 있지만, 이러한 서비스 활용 방법 및 데이터 분석 시스템 구축 과정이 궁금하신 분 - 데이터 분석 시스템을 구축한 경험은 있지만, 자신이 만든 시스템을 아키텍처 관점에서 어떻게 평가하고 확인할 수 있는지 궁금하신 분

Building a Data Processing Pipeline on AWS - AWS Summit SG 2017

Amazon Web Services

AWS provides a broad platform of managed services to help you build, secure, and seamlessly scale end-to-end Big Data applications quickly and with ease. Want to get ramped up on how to use Amazon's big data web services? Learn when to use which service? Want to write your first big data application on AWS? Join us in this session as we discuss reference architecture, design patterns, and best practices for pulling together various AWS services to meet your big data challenges.

Implementing a Data Lake

Amazon Web Services

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

Amazon Web Services

In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.

Similar to Analytics on AWS:Structured, Unstructured and Streaming (20)

Big Data and Analytics

Big Data and Analytics – End to End on AWS – Russell Nash

Building a Modern Data Architecture on AWS - Webinar

Big Data Architectural Patterns and Best Practices on AWS

BDA303 Serverless big data architectures: Design patterns and best practices

Building Data Warehouses and Data Lakes in the Cloud - DevDay Austin 2017 Day 2

Building A Modern Data Analytics Architecture on AWS

BDA305 Building Data Lakes and Analytics on AWS

Building your first Data lake platform

Simplify Big Data with AWS

Understanding AWS Managed Database and Analytics Services | AWS Public Sector...

AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...

Getting Started with Big Data and HPC in the Cloud - August 2015

Database and Analytics on the AWS Cloud - AWS Innovate Toronto

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

AWS Analytics Immersion Day - Build BI System from Scratch (Day1, Day2 Full V...

Building a Data Processing Pipeline on AWS - AWS Summit SG 2017

Implementing a Data Lake

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Amazon Web Services

Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS. In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Amazon Web Services

La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup. Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti. Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.

Esegui pod serverless con Amazon EKS e AWS Fargate

Amazon Web Services

Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi

Costruire Applicazioni Moderne con AWS

Amazon Web Services

Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.

Come spendere fino al 90% in meno con i container e le istanze spot

Amazon Web Services

L’utilizzo dei container è in continua crescita. Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili. I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!

Open banking as a service

Amazon Web Services

In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th. Event Agenda : Open banking so far (short recap) • PSD2, OB UK, OB Australia, OB LATAM, OB Israel Intro to Open Finance marketplace • Scope • Features • Tech overview and Demo The role of the Cloud The Future of APIs • Complying with regulation • Monetizing data / APIs • Business models • Time to market One platform for all: a Strategic approach Q&A

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

Amazon Web Services

Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc. AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta. Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Amazon Web Services

Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity. AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet. Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Amazon Web Services

Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.

Computer Vision con AWS

Amazon Web Services

Database Oracle e VMware Cloud on AWS i miti da sfatare

Amazon Web Services

Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti. Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

Amazon Web Services

Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti. Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire. Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito. In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.

API moderne real-time per applicazioni mobili e web

Amazon Web Services

Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline. Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Amazon Web Services

Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali. In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.

Tools for building your MVP on AWSAmazon Web Services

How to Build a Winning Pitch DeckAmazon Web Services

Building a web application without serversAmazon Web Services

Fundraising EssentialsAmazon Web Services

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services

Introduzione a Amazon Elastic Container Service

Amazon Web Services

Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Esegui pod serverless con Amazon EKS e AWS Fargate

Costruire Applicazioni Moderne con AWS

Come spendere fino al 90% in meno con i container e le istanze spot

Open banking as a service

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Computer Vision con AWS

Database Oracle e VMware Cloud on AWS i miti da sfatare

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

API moderne real-time per applicazioni mobili e web

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Tools for building your MVP on AWS

How to Build a Winning Pitch Deck

Building a web application without servers

Fundraising Essentials

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Introduzione a Amazon Elastic Container Service

Recently uploaded

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Inflectra

In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring. Learn about: • The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks. • Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective. • Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification. • Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process. Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Albert Hoitingh

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams. Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Product School

Securing your Kubernetes cluster_ a step-by-step guide to success !

KatiaHIMEUR1

Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster. However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks. In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

JMeter webinar - integration with InfluxDB and Grafana

RTTS

Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application. In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics. Length: 30 minutes Session Overview ------------------------------------------- During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana: - What out-of-the-box solutions are available for real-time monitoring JMeter tests? - What are the benefits of integrating InfluxDB and Grafana into the load testing stack? - Which features are provided by Grafana? - Demonstration of InfluxDB and Grafana using a practice web application To view the webinar recording, go to: https://www.rttsweb.com/jmeter-integration-webinar

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

Assuring Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

Neuro-symbolic is not enough, we need neuro-*semantic*

Frank van Harmelen

Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”. All of this illustrated with link prediction over knowledge graphs, but the argument is general.

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

Paul Groth

Bits & Pixels using AI for Good.........

Alison B. Lowndes

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

Thierry Lestable

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Mission to Decommission: Importance of Decommissioning Products to Increase E...

Product School

UiPath Test Automation using UiPath Test Suite series, part 4

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap. The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies. Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques What will you get from this session? 1. Insights into SAP testing best practices 2. Heatmap utilization for testing 3. Optimization of testing processes 4. Demo Topics covered: Execution from the test manager Orchestrator execution result Defect reporting SAP heatmap example with demo Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

Recently uploaded (20)

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Designing Great Products: The Power of Design and Leadership by Chief Designe...

Securing your Kubernetes cluster_ a step-by-step guide to success !

When stars align: studies in data quality, knowledge graphs, and machine lear...

JMeter webinar - integration with InfluxDB and Grafana

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

PCI PIN Basics Webinar from the Controlcase Team

Assuring Contact Center Experiences for Your Customers With ThousandEyes

Neuro-symbolic is not enough, we need neuro-*semantic*

To Graph or Not to Graph Knowledge Graph Architectures and LLMs

Bits & Pixels using AI for Good.........

Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

Mission to Decommission: Importance of Decommissioning Products to Increase E...

UiPath Test Automation using UiPath Test Suite series, part 4

Monitoring Java Application Security with JDK Tools and JFR Events

Analytics on AWS:Structured, Unstructured and Streaming

3. Analytics on AWS Structured, Unstructured and Streaming Russell Nash AWS Solutions Architect

4. Ingest Store Process Analyse

5. EC2 Linux Windows

6. Databases Database Data Log Data High Velocity Data Analytics Database INGEST STORE Devices Web Servers App Servers Mobile Amazon Redshift Amazon RDS

7. Traditional Database Choice of Engines Low Admin Amazon RDS (Relational Database Service) Amazon Aurora

8. MPP SQL Database Optimised for Analytics Scalable Amazon Redshift

9. Amazon Redshift Amazon RDS ScalingVertical Horizontal WorkloadMixed Analytical VolumesLow to Medium Medium to High TypeSQL Relational SQL Relational

10.

11. Database Data ETL INGEST STORE High Velocity Data Devices Web Servers App Servers Mobile Databases Log Data Amazon Redshift Amazon RDS

12. AWS Database Migration Service Amazon Redshift Source Database ETL ETL Partners Amazon RDS

13. Storage INGEST STORE High Velocity Data Devices Web Servers App Servers Mobile Database Data Databases Log Data Search Amazon Elasticsearch Amazon Redshift Amazon RDS

14. Amazon Elasticsearch Search and Analytics Scalable Integrated – Logstash, Kibana

15.

16. Database Data Storage INGEST STORE High Velocity Data Devices Web Servers App Servers Mobile Databases Log Data Amazon Elasticsearch Amazon S3 Amazon Redshift Amazon RDS

17. Amazon S3 Object Storage Low Cost 11 9’s of durability

18. PIG HDFS

19. PIG Amazon EMR Amazon S3 EMRFS EMR

20. CPU c4 family c3 family Memory x1 family r3 family Disk/IO d2 family i2 family General m4 family m3 family Instance Types Batch Machine Spark and Large process learning interactive HDFS

21. Cost & Time # CPUs Time # CPUs Time Wall clock time: 1 hourWall clock time: 10 hours

22.

23. Spot Price – M3.2XL On-Demand Spot-Price $0.08$0.75

24.

25. Database Data INGEST STORE Mobile High Velocity Data Devices Web Servers App Servers Mobile Databases Log Data Amazon Elasticsearch Amazon S3 Stream Processor Amazon Kinesis NoSQL Amazon Redshift Amazon RDS

26. Availability Zone Availability Zone Availability Zone Data Sources Data Sources Data Sources Data Sources Data Sources S3 Redshift Amazon Kinesis Stream AWS Lambda KCL App EMR Elasticsearch

27. Database Data INGEST STORE High Velocity Data Devices Web Servers App Servers Mobile Databases Log Data Amazon Elasticsearch Amazon S3 Amazon Kinesis NoSQL Amazon DynamoDB Amazon Redshift Amazon RDS

28. scalability data complexity RDBMS key/value document graph NoSQL

29. NoSQL Database Key/Value + Document Very Low Latency Amazon DynamoDB

30. Low Latency

31.

32. INGEST STORE PROCESS Databases Amazon Redshift Amazon Kinesis Amazon S3 Impala Amazon Redshift Amazon EMR Database Data InteractiveBatchStreaming Hadoop Amazon Elasticsearch Log Data Mobile High Velocity Data Devices Web Servers App Servers Mobile Amazon DynamoDB

33. INGEST STORE PROCESS Impala Amazon Redshift Database Data InteractiveBatch PIG Streaming Amazon EMR Hadoop Amazon Kinesis Amazon S3 Amazon Elasticsearch Mobile High Velocity Data Devices Web Servers App Servers Mobile Amazon DynamoDB Databases Log Data Amazon Redshift

34. FAST RICH INDUSTRY SUPPORT FLEXIBLE

35. INGEST STORE PROCESS Impala Amazon Redshift AWS Lambda Kinesis Consumers Database Data InteractiveBatchStreaming PIG Amazon EMR Hadoop Amazon Kinesis Amazon S3 Amazon Elasticsearch Mobile High Velocity Data Devices Web Servers App Servers Mobile Amazon DynamoDB Databases Log Data Amazon Redshift

36. Comparison of Query Engines Amazon Redshift Amazon Elasticsearch Data Structure Languages Semi SemiSemi Multiple Full Text SearchSQL Full SQL Data Store S3/HDFS LocalS3/HDFS Local Performance

37. Comparison of Query Engines Amazon Redshift Amazon Elasticsearch General Purpose Processing Engine SQL Query Engine for S3/HDFS Fully Featured SQL Analytics Database Full Text search VisualizationUse Case

38. INGEST STORE PROCESS Impala Amazon Redshift AWS Lambda Kinesis Consumers Database Data InteractiveBatchStreaming PIG Amazon EMR Hadoop Amazon Kinesis Amazon S3 Amazon Elasticsearch Mobile High Velocity Data Devices Web Servers App Servers Mobile Amazon DynamoDB Databases Log Data Amazon Redshift ANALYSE BI Tools Amazon Machine Learning ML Amazon QuickSight

39. INGEST STORE PROCESS Impala Amazon Redshift AWS Lambda Kinesis Consumers Database Data InteractiveBatchStreaming PIG Amazon EMR Hadoop Amazon Kinesis Amazon S3 Amazon Elasticsearch Mobile High Velocity Data Devices Web Servers App Servers Mobile Amazon DynamoDB Databases Log Data Amazon Redshift ANALYSE BI Tools Amazon Machine Learning ML Amazon QuickSight

40. Visit bit.ly/aws-tech-chat or download via SoundCloud or iTunes

Analytics on AWS:Structured, Unstructured and Streaming

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Analytics on AWS:Structured, Unstructured and Streaming

Similar to Analytics on AWS:Structured, Unstructured and Streaming (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Recently uploaded

Recently uploaded (20)

Analytics on AWS:Structured, Unstructured and Streaming