What’s new in AWS
Philip Fitzsimons, Manager, Solutions Architecture
Amazon Web Services UK
What’s new in AWS
Amazon Kinesis

Managed Service for Real-time Big Data Processing

Amazon WorkSpaces

Fully Managed Desk...
Amazon Kinesis
Managed Service for
Real-time Big Data Processing
Introducing Amazon Kinesis
• Managed service that can scale without any down time
– Create a streaming data intake pipelin...
Why Amazon Kinesis?
Internal AWS experiences provided inspiration

Big Data

Real-time Big Data
•

CloudWatch metrics: wha...
Sample Use Cases of Amazon Kinesis
Financial Services Leader

Digital Advertising Tech. Pioneer

Maintain real-time audit ...
Clickstream Analytics with Amazon Kinesis

Clickstream Archive
Aggregate
Clickstream
Statistics

Clickstream Trend Analysi...
Simple Metering & Billing with Amazon Kinesis

Metering Record Archive
Incremental Bill
Computation

Billing Management Se...
Amazon Kinesis: Key Developer Benefits
Easy Administration
Managed service for real-time streaming
data collection, proces...
Amazon Kinesis
Managed Service for Real-Time Processing of Big Data
App.1

Data
Sources
Availability
Zone

Data
Sources

D...
Putting data into Kinesis
Managed Service for Ingesting Fast Moving Data
•

Streams are made of Shards
•

•

Each shard in...
Getting data out of Kinesis
Client library for fault-tolerant, at least-once, real-time processing
•

In order to keep up ...
Amazon Kinesis Resources
• Sign up for Limited Preview
– http://aws.amazon.com/kinesis/limited-preview/
– Get SDK and Endp...
Amazon WorkSpaces
Fully Managed Desktops in the Cloud
Sample Use Cases (there are many more)

• Mobile Device Access
• Secure WorkSpaces
• Remote Employees

• Seasonal Workers
...
Key Benefits
• Fully Managed
• Support Multiple
Devices
• Keep Data Secure
and Available

• Choose Software &
Hardware
• P...
Fully Managed

WorkSpaces

• Launch the number of WorkSpaces needed
• All heavy lifting taken care of by AWS
• Users recei...
Support Multiple Devices
•
•
•
•
•

iPad
Kindle Fire HDX (Keyboard & Mouse)
Android Tablet
Microsoft Windows
Mac
Keep Data Secure and Available

• No data stored on end-user device
• Only Pixels delivered to users (PCoIP)
• User volume...
Choose Software and Hardware
WorkSpaces Bundle

Hardware Resources

Applications

Standard

1 vCPU, 3.75 GiB Memory,
50 GB...
Pay as You Go
WorkSpaces Bundle
Standard

$35

Standard Plus

$50

Performance

$60

Performance Plus
•
•
•

Monthly Price...
Corporate Directory Integration

• Users: Get to use existing Enterprise Credentials
• IT: WorkSpaces control like regular...
Getting Started – What steps do customers take?

•
•
•
•
•
•

Integrate with Corporate Active Directory
Choose WorkSpaces ...
Availability
• Now: Limited Preview
• Q1 2014: Public Beta (Initially US-West & US-East)
• Q1/Q2 2014: Region Expansion
Resources
aws.amazon.com/WorkSpaces
•
•
•
•
•

Detail Page
Pricing
Limited Preview Sign-Up
Documentation will follow at Pu...
AWS CloudTrail (beta)

AWS CloudTrail is a web service that records AWS API
calls for your account and delivers log files ...
AWS CloudTrail Use Cases
• Security Analysis
–

You can use the AWS API call history produced by CloudTrail as an input in...
Features and Benefits
• Increased Visibility
– CloudTrail provides increased visibility into your user activity by
recordi...
Features and Benefits
• Durable and Inexpensive Log File Storage
– CloudTrail uses Amazon S3 for log file storage and deli...
Features and Benefits
• Easy Administration
– CloudTrail is a fully managed service.
– You simply turn on CloudTrail for y...
Features and Benefits
• Reliability
– CloudTrail continuously transports events from AWS services
using a highly available...
Features and Benefits
• Timely Delivery & Notification
– CloudTrail typically delivers events within 15 minutes of the API...
Features and Benefits
• Log File Aggregation
– CloudTrail can be configured to aggregate log files across
multiple account...
Features and Benefits
• Choice of Partner Solutions
– Multiple partners are available including AlertLogic, Boundary,
Logg...
Getting Started
CloudTrail can be turned on in as few as two clicks from the AWS
Management Console. CloudTrail generates ...
Amazon AppStream
Stream resource-intensive applications from the Cloud
Sample Use Cases
•
•
•
•

Games
Media and Entertainment
Simulation Software
3D Graphics Development
Key Benefits
• Remove Device
• Instant On
Constraints
• Improved Security
• Multi Platform Support • Automatic Scaling
• E...
Remove Device Constraints

• No longer a need to snap to the device capabilities
• AppStream helps reach broadest audience...
Multi Platform Support
•
•
•
•
•

Kindle Fire
Android
iOS
Windows 7
Mac OS (2014)
Easy Updates
• Provide a new application version to AppStream
• Immediately upgrade all users
• No action on users’ part
Instant On
• Users can start using application immediately
• No large file downloads
• No time consuming installations
Improved Security
• Create Entitlement Service to authorize connections
• Clients connect to Entitlement Service for acces...
Automatic Scaling
• Set the limits for scaling of your application
• The AppStream service takes care of scaling
• AppStre...
Getting Started – What steps do customers take?

•
•
•
•

Integrate AppStream SDK with application
Deploy Application to A...
Availability
• Now: Limited Preview
• Public Beta Date: TBD
Amazon RDS for PostgreSQL

Amazon RDS makes it easy to set up, operate,
and scale PostgreSQL deployments in the cloud.
Wit...
What does RDS for PostgreSQL enable?
•
•
•
•

Large-scale web applications
Internal and departmental applications
Excellen...
Benefits of RDS for PostgreSQL
• Easy, managed deployments
– Free up time from undifferentiated admin tasks
– Database ins...
Benefits of PostgreSQL for AWS Customers
• The preferred open-source database for many
enterprise developers and startups
...
Getting started with Amazon RDS for PostgreSQL

• Review the Getting Started Guide for RDS
http://docs.aws.amazon.com/gett...
Resources for Amazon RDS for PostgreSQL
• User Guide
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/
• Data Import ...
Details for Developers
• Launching with PostgreSQL 9.3.1
• Wide selection of available instances
– Including the new high-...
What’s new in AWS
Amazon Kinesis

Managed Service for Real-time Big Data Processing

Amazon WorkSpaces

Fully Managed Desk...
What's new in AWS?
Upcoming SlideShare
Loading in …5
×

What's new in AWS?

2,470 views

Published on

This presentation from the AWS Lab at Cloud Expo Europe 2014 contains details of newly announced services from Amazon Web Services, including Amazon Kinesis, Amazon WorkSpaces, AWS CloudTrail (beta), Amazon AppStream and Amazon RDS for PostgreSQL (beta)

  • Be the first to comment

What's new in AWS?

  1. 1. What’s new in AWS Philip Fitzsimons, Manager, Solutions Architecture Amazon Web Services UK
  2. 2. What’s new in AWS Amazon Kinesis Managed Service for Real-time Big Data Processing Amazon WorkSpaces Fully Managed Desktops in the Cloud AWS CloudTrail (beta) Get log files of API call made on your AWS account Amazon AppStream Stream resource-intensive applications from the Cloud Amazon RDS for PostgreSQL (beta) Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud
  3. 3. Amazon Kinesis Managed Service for Real-time Big Data Processing
  4. 4. Introducing Amazon Kinesis • Managed service that can scale without any down time – Create a streaming data intake pipeline and do analysis in near real time – Scale to hundreds of thousands of sensors or producers – Scale to gigabytes to terabytes of throughput per hour • Understand and take action based on data in seconds – Using Amazon Kinesis Client Library on Amazon EC2 instances to process data – Leverage pre-built connectors to store the data in S3, Redshift or DynamoDB • High availability of data streams at low cost – Data ingested in is stored in 3 different physical locations – Pay by the hour with no commitments or up front fees
  5. 5. Why Amazon Kinesis? Internal AWS experiences provided inspiration Big Data Real-time Big Data • CloudWatch metrics: what just went wrong now Weekly / Monthly Bill: What you spent this past billing cycle? • Real-time spending alerts/caps: guaranteeing you can’t overspend • Daily customer-preferences report from your website’s click stream: tells you what deal or ad to try next time • Real-time analysis: tells you what to offer the current customer now • Daily fraud reports: tells you if there was fraud yesterday • Real-time detection: blocks fraudulent use now • Daily business reports: tells me how customers used AWS services yesterday • Fast ETL into Amazon Redshift: how are customers using AWS services now • Hourly server logs: how your systems were misbehaving an hour ago •
  6. 6. Sample Use Cases of Amazon Kinesis Financial Services Leader Digital Advertising Tech. Pioneer Maintain real-time audit trail of every single market/ exchange order Generate real-time metrics, KPIs for online ads performance for advertisers Custom-built solutions operationally complex to manage, & not scalable End-of-day Hadoop based processing pipeline slow, & cumbersome Kinesis enables customer to ingest all market order data reliably, and build real-time auditing applications Kinesis enables customers to move from periodic batch processing to continual, real-time metrics and reports generation Accelerates time to market of elastic, real-time applications – while minimizing operational overhead Generates freshest analytics on advertiser performance to optimize marketing spend, and increases responsive to clients
  7. 7. Clickstream Analytics with Amazon Kinesis Clickstream Archive Aggregate Clickstream Statistics Clickstream Trend Analysis Clickstream Processing App
  8. 8. Simple Metering & Billing with Amazon Kinesis Metering Record Archive Incremental Bill Computation Billing Management Service Billing Auditors
  9. 9. Amazon Kinesis: Key Developer Benefits Easy Administration Managed service for real-time streaming data collection, processing and analysis. Simply create a new stream, set the desired level of capacity, and let the service handle the rest. S3, Redshift, & DynamoDB Integration Reliably collect, process, and transform all of your data in real-time & deliver to AWS data stores of choice, with Connectors for S3, Redshift, and DynamoDB. Real-time Performance High Throughput. Elastic Perform continual processing on streaming big data. Processing latencies fall to a few seconds, compared with the minutes or hours associated with batch processing. Seamlessly scale to match your data throughput rate and volume. You can easily scale up to gigabytes per second. The service will scale up or down based on your operational or business needs. Build Real-time Applications Low Cost Client libraries that enable developers to design and operate real-time streaming data processing applications. Cost-efficient for workloads of any scale. You can get started by provisioning a small stream, and pay low hourly rates only for what you use. 9
  10. 10. Amazon Kinesis Managed Service for Real-Time Processing of Big Data App.1 Data Sources Availability Zone Data Sources Data Sources Availability Zone S3 App.2 AWS Endpoint Data Sources Availability Zone [Aggregate & De-Duplicate] Shard 1 Shard 2 Shard N [Metric Extraction] DynamoDB App.3 [Sliding Window Analysis] Redshift Data Sources App.4 [Machine Learning]
  11. 11. Putting data into Kinesis Managed Service for Ingesting Fast Moving Data • Streams are made of Shards • • Each shard ingests up to 1MB/sec of data and up to 1000 TPS • All data is stored for 24 hours • • A Kinesis stream is composed of multiple Shards You scale Kinesis streams by adding or removing Shards Simple PUT interface to store data in Kinesis • Producers use a PUT call to store data in a stream • A Partition Key is used to distribute the PUTs across Shards • A unique Sequence # is returned to the Producer upon a successful PUT call
  12. 12. Getting data out of Kinesis Client library for fault-tolerant, at least-once, real-time processing • In order to keep up with the stream, your application must: • • Be fault tolerant, to handle failures in hardware or software • • Be distributed, to handle multiple shards Scale up and down as the number of shards increase or decrease Kinesis Client Library (KCL) helps with distributed processing: • • Automatically starts a Kinesis Worker for each shard • Changes number of Kinesis Workers as number of shards changes • • Simplifies reading by abstracting your code from individual shards Uses checkpoints to keep track of a Worker’s location in the stream Use the KCL with Auto Scaling Groups • Auto Scaling policies will restart EC2 instances if they fail • Automatically add EC2 instances when load increases
  13. 13. Amazon Kinesis Resources • Sign up for Limited Preview – http://aws.amazon.com/kinesis/limited-preview/ – Get SDK and Endpoint information after getting into Limited Preview • Getting Started Guide – http://docs.aws.amazon.com/kinesis/latest/dev/getting-started.html • Developer Guide – http://docs.aws.amazon.com/kinesis/latest/dev/introduction.html
  14. 14. Amazon WorkSpaces Fully Managed Desktops in the Cloud
  15. 15. Sample Use Cases (there are many more) • Mobile Device Access • Secure WorkSpaces • Remote Employees • Seasonal Workers • Student WorkSpaces • Developer WorkSpaces
  16. 16. Key Benefits • Fully Managed • Support Multiple Devices • Keep Data Secure and Available • Choose Software & Hardware • Pay as You Go • Corporate Directory Integration
  17. 17. Fully Managed WorkSpaces • Launch the number of WorkSpaces needed • All heavy lifting taken care of by AWS • Users receive email to install clients and connect
  18. 18. Support Multiple Devices • • • • • iPad Kindle Fire HDX (Keyboard & Mouse) Android Tablet Microsoft Windows Mac
  19. 19. Keep Data Secure and Available • No data stored on end-user device • Only Pixels delivered to users (PCoIP) • User volume backed by Amazon S3
  20. 20. Choose Software and Hardware WorkSpaces Bundle Hardware Resources Applications Standard 1 vCPU, 3.75 GiB Memory, 50 GB User Storage Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) Standard Plus 1 vCPU, 3.75 GiB Memory, 50 GB User Storage Microsoft Office Professional 2010, Trend Micro Worry-Free Business Security, Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) Performance 2 vCPU, 7.5 GiB Memory, 100 GB User Storage Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) Performance Plus 2 vCPU, 7.5 GiB Memory, 100 GB User Storage Microsoft Office Professional 2010, Trend Micro Worry-Free Business Security, Utilities (Adobe Reader, Internet Explorer 9, Firefox, 7-Zip, Adobe Flash, JRE) All WorkSpaces Bundles provide the Windows 7 Experience to users (provided by Windows Server 2008 R2 with RDS).
  21. 21. Pay as You Go WorkSpaces Bundle Standard $35 Standard Plus $50 Performance $60 Performance Plus • • • Monthly Price $75 No up-front commitment Delete WorkSpaces at any time Price includes infrastructure (compute, storage, bandwidth) and bundle’s software
  22. 22. Corporate Directory Integration • Users: Get to use existing Enterprise Credentials • IT: WorkSpaces control like regular desktops
  23. 23. Getting Started – What steps do customers take? • • • • • • Integrate with Corporate Active Directory Choose WorkSpaces Bundle Select Users to receive WorkSpaces Launch WorkSpaces Users receive email when provisioned Users connect to WorkSpaces
  24. 24. Availability • Now: Limited Preview • Q1 2014: Public Beta (Initially US-West & US-East) • Q1/Q2 2014: Region Expansion
  25. 25. Resources aws.amazon.com/WorkSpaces • • • • • Detail Page Pricing Limited Preview Sign-Up Documentation will follow at Public Beta Contact aws-bdms-workspaces@amazon.com – Prioritizing customers in Limited Preview, specific briefings
  26. 26. AWS CloudTrail (beta) AWS CloudTrail is a web service that records AWS API calls for your account and delivers log files to you. The recorded information includes: • • • • • The identity of the API caller The time of the API call The source IP address of the API caller The request parameters The response elements returned by the AWS service
  27. 27. AWS CloudTrail Use Cases • Security Analysis – You can use the AWS API call history produced by CloudTrail as an input into log management and analysis solutions to perform security analysis and to detect user behaviour patterns. • Track Changes to AWS Resources – You can use the AWS API call history produced by CloudTrail to track changes to AWS resources, including creation, modification, and deletion of AWS resources such as Amazon EC2 instances, Amazon VPC security groups and Amazon EBS • Troubleshoot Operational Issues – You can use the AWS API call history produced by CloudTrail to troubleshoot operational issues. For example, you can quickly identify the most recent changes made to resources in your environment. • Compliance Aid – CloudTrail makes it easier to ensure compliance with internal policies and regulatory standards by providing AWS API call history. Integrates with AWS Partner solutions like Alert Logic
  28. 28. Features and Benefits • Increased Visibility – CloudTrail provides increased visibility into your user activity by recording AWS API calls. – You can answer questions such as, what actions did a given user take over a given time period? For a given resource, which user has taken actions on it over a given time period? What is the source IP address of a given activity? Which activities failed due to inadequate permissions?
  29. 29. Features and Benefits • Durable and Inexpensive Log File Storage – CloudTrail uses Amazon S3 for log file storage and delivery, so log files are stored durably and inexpensively. – You can use Amazon S3 lifecycle configuration rules to further reduce storage costs. For example, you can define rules to automatically delete old log files or archive them to Amazon Glacier for additional savings.
  30. 30. Features and Benefits • Easy Administration – CloudTrail is a fully managed service. – You simply turn on CloudTrail for your account using the AWS Management Console, the Command Line Interface, or the CloudTrail SDK and start receiving CloudTrail log files in the Amazon Simple Storage Service (Amazon S3) bucket that you specify.
  31. 31. Features and Benefits • Reliability – CloudTrail continuously transports events from AWS services using a highly available and fault tolerant processing pipeline. – Turning on CloudTrail has no impact on performance of your AWS resources or API call latency.
  32. 32. Features and Benefits • Timely Delivery & Notification – CloudTrail typically delivers events within 15 minutes of the API call and can be configured to publish a notification for each log file delivered. – This feature enables you to automatically take action upon log file delivery. CloudTrail uses the Amazon Simple Notification Service (SNS) for notifications.
  33. 33. Features and Benefits • Log File Aggregation – CloudTrail can be configured to aggregate log files across multiple accounts and regions. – If you use multiple AWS regions, you can choose where log files are delivered for each region. For example, you can have a separate Amazon S3 bucket for each region, or you can aggregate log files from all regions in a single S3 bucket.
  34. 34. Features and Benefits • Choice of Partner Solutions – Multiple partners are available including AlertLogic, Boundary, Loggly, Splunk and Sumologic. – These partners offer integrated solutions to analyze CloudTrail log files. These solutions include features like change tracking, troubleshooting, and security analysis.
  35. 35. Getting Started CloudTrail can be turned on in as few as two clicks from the AWS Management Console. CloudTrail generates log files containing detailed information about API calls made, and periodically saves these files into an Amazon S3 bucket of your choosing. You can also choose to create an SNS topic to receive a notification every time a new log file is delivered. • • • To turn on CloudTrail, just provide a name for an Amazon S3 bucket where you want your log files delivered. If you use multiple AWS regions, you can choose where log files are delivered for each region. For example, you can have a separate Amazon S3 bucket for each region, or you can aggregate log files from all regions in a single S3 bucket. There is no additional charge for CloudTrail, but standard rates for Amazon S3 and Amazon SNS usage apply.
  36. 36. Amazon AppStream Stream resource-intensive applications from the Cloud
  37. 37. Sample Use Cases • • • • Games Media and Entertainment Simulation Software 3D Graphics Development
  38. 38. Key Benefits • Remove Device • Instant On Constraints • Improved Security • Multi Platform Support • Automatic Scaling • Easy Updates
  39. 39. Remove Device Constraints • No longer a need to snap to the device capabilities • AppStream helps reach broadest audience • Users get rich experience across devices
  40. 40. Multi Platform Support • • • • • Kindle Fire Android iOS Windows 7 Mac OS (2014)
  41. 41. Easy Updates • Provide a new application version to AppStream • Immediately upgrade all users • No action on users’ part
  42. 42. Instant On • Users can start using application immediately • No large file downloads • No time consuming installations
  43. 43. Improved Security • Create Entitlement Service to authorize connections • Clients connect to Entitlement Service for access • Your application is secured in AWS Cloud
  44. 44. Automatic Scaling • Set the limits for scaling of your application • The AppStream service takes care of scaling • AppStream uses the G2.2XLarge Instance – Support for more instance types coming in 2014 • Focus on your application, not infrastructure
  45. 45. Getting Started – What steps do customers take? • • • • Integrate AppStream SDK with application Deploy Application to AppStream Create Entitlement Service Create Client Apps
  46. 46. Availability • Now: Limited Preview • Public Beta Date: TBD
  47. 47. Amazon RDS for PostgreSQL Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud. With Amazon RDS, you can deploy scalable PostgreSQL deployments in minutes with costefficient and resizable hardware capacity.
  48. 48. What does RDS for PostgreSQL enable? • • • • Large-scale web applications Internal and departmental applications Excellent vehicle for ETL into analytics engines Geospatial and mobile applications
  49. 49. Benefits of RDS for PostgreSQL • Easy, managed deployments – Free up time from undifferentiated admin tasks – Database instances with pre-configured parameters, automated patching, monitoring, and notifications • Familiar environment for developers – All your existing PostgreSQL apps work • Fast, predictable performance – Use Provisioned IOPS to tune on the fly • Backup and recovery – Automated backups and database snapshots • High availability – Deploy in multiple Availability Zones Available in all regions
  50. 50. Benefits of PostgreSQL for AWS Customers • The preferred open-source database for many enterprise developers and startups • Support for geospatial queries using the PostGIS extensions • Support for full-text search • Support for advanced data types: JSON and key/value stores (“hstore”)
  51. 51. Getting started with Amazon RDS for PostgreSQL • Review the Getting Started Guide for RDS http://docs.aws.amazon.com/gettingstarted/lates t/awsgsg-intro/gsg-aws-intro.html • Review the RDS Free Tier http://aws.amazon.com/free • Create a PostgreSQL DB instance from the RDS console https://console.aws.amazon.com/rds/
  52. 52. Resources for Amazon RDS for PostgreSQL • User Guide http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/ • Data Import Guide http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/P ostgreSQL.Procedural.Importing.html • Common tasks, including setting up PostGIS geospatial extensions http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/A ppendix.PostgreSQL.CommonDBATasks.html • RDS forum https://forums.aws.amazon.com/forum.jspa?forumID=60
  53. 53. Details for Developers • Launching with PostgreSQL 9.3.1 • Wide selection of available instances – Including the new high-memory db.cr1.8xlarge • Choose Multi-AZ deployment for high availability – Synchronous replication to a secondary in a different AZ • Use Provisioned IOPS for predictable performance – Convert to IOPS with a brief availability impact – Then add IOPS or storage on the fly
  54. 54. What’s new in AWS Amazon Kinesis Managed Service for Real-time Big Data Processing Amazon WorkSpaces Fully Managed Desktops in the Cloud AWS CloudTrail (beta) Get log files of API call made on your AWS account Amazon AppStream Stream resource-intensive applications from the Cloud Amazon RDS for PostgreSQL (beta) Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the cloud Talk to someone from AWS or our partners at Stand 1070

×