How does McGraw-Hill Education use the AWS platform to scale and reliably receive 10,000 learning events per second? How do we provide near-real-time reporting and event-driven analytics for hundreds of thousands of concurrent learners in a reliable, secure, and auditable manner that is cost effective? MHE designed and implemented a robust solution that integrates AWS API Gateway, AWS Lambda, Amazon Kinesis, Amazon S3, Amazon Elasticsearch Service, Amazon DynamoDB, HDFS, Amazon EMR, Amazopn EC2, and other technologies to deliver this cloud-native platform across the US and soon the world. This session describes the challenges we faced, architecture considerations, how we gained confidence for a successful production roll-out, and the behind-the-scenes lessons we learned.
AWS re:Invent 2016: Big Data Architectural Patterns and Best Practices on AWS...Amazon Web Services
The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.
Accelerate your Business with SAP on AWS - AWS Summit Cape Town 2017 Amazon Web Services
From dev and test to large-scale HANA production deployments, enterprise usage of SAP on AWS is rapidly growing across all verticals and geographies. Gain insights on the AWS SAP offering and partnership and why a cloud first approach makes business, technical and financial sense for the numerous SAP solutions that are certified and ready to be deployed today.
AW Speaker: Michael Needham, Sr Mgr, Solutions Architecture - Amazon Web Services
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)Amazon Web Services
Which database is best suited for your use case? Should you choose a relational database or NoSQL or a data warehouse for your workload? Would a managed service like Amazon RDS, Amazon DynamoDB, or Amazon Redshift work better for you, or would it be better to run your own database on Amazon EC2? FanDuel has been running its fantasy sports service on Amazon Web Services (AWS) since 2012. You will learn best practices and insights from FanDuel’s successful migrations from self-managed databases on EC2 to fully-managed database services.
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Amazon Web Services
Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity, automates time-consuming database administration tasks, and provides you with six familiar database engines to choose from: Amazon Aurora, Oracle, Microsoft SQL Server, PostgreSQL, MySQL and MariaDB. In this session, we will take a close look at the capabilities of Amazon RDS and explain how it works. We’ll also discuss the AWS Database Migration Service and AWS Schema Conversion Tool, which help you migrate databases and data warehouses with minimal downtime from on-premises and cloud environments to Amazon RDS and other Amazon services. Gain your freedom from expensive, proprietary databases while providing your applications with the fast performance, scalability, high availability, and compatibility they need.
AWS Speaker: Andrew Kane, Solutions Architect - Amazon Web Services
AWS re:Invent 2016: JustGiving: Serverless Data Pipelines, Event-Driven ETL, ...Amazon Web Services
Organizations need to gain insight and knowledge from a growing number of Internet of Things (IoT), application programming interfaces (API), clickstreams, unstructured and log data sources. However, organizations are also often limited by legacy data warehouses and ETL processes that were designed for transactional data. Building scalable big data pipelines with automated extract-transform-load (ETL) and machine learning processes can address these limitations. JustGiving is the world’s largest social platform for online giving. In this session, we describe how we created several scalable and loosely coupled event-driven ETL and ML pipelines as part of our in-house data science platform called RAVEN. You learn how to leverage AWS Lambda, Amazon S3, Amazon EMR, Amazon Kinesis, and other services to build serverless, event-driven, data and stream processing pipelines in your organization. We review common design patterns, lessons learned, and best practices, with a focus on serverless big data architectures with AWS Lambda.
Amazon Kinesis provides services for you to work with streaming data on AWS. Learn how to load streaming data continuously and cost-effectively to Amazon S3 and Amazon Redshift using Amazon Kinesis Firehose without writing custom stream processing code. Get an introduction to building custom stream processing applications with Amazon Kinesis Streams for specialised needs.
Deep Dive on Elastic File System - February 2017 AWS Online Tech TalksAmazon Web Services
Organizations face significant challenges moving their applications to the cloud when they require a standard file system interface for accessing their cloud data. In this technical session, we will explore the world’s first cloud-scale file system and its targeted use cases. Attendees will learn about the Amazon Elastic File System (EFS) features and benefits, how to identify applications that are appropriate for use with Amazon EFS, and details about its performance and security models. We will highlight and demonstrate how to deploy Amazon EFS in one of our most common use cases and will share tips for success throughout.
Learning Objectives:
• Recognize why and when to use Amazon EFS
• Understand key technical/security concepts
• Learn how to leverage EFS’s performance
• See a demo of EFS in action
• Review EFS’s economics
Amazon Web Services provides startups with the low cost, easy to use infrastructure needed to scale and grow any size business. Attend this session and learn how to migrate your startup to AWS and make the most out of the platform.
AWS re:Invent 2016: Big Data Architectural Patterns and Best Practices on AWS...Amazon Web Services
The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.
Accelerate your Business with SAP on AWS - AWS Summit Cape Town 2017 Amazon Web Services
From dev and test to large-scale HANA production deployments, enterprise usage of SAP on AWS is rapidly growing across all verticals and geographies. Gain insights on the AWS SAP offering and partnership and why a cloud first approach makes business, technical and financial sense for the numerous SAP solutions that are certified and ready to be deployed today.
AW Speaker: Michael Needham, Sr Mgr, Solutions Architecture - Amazon Web Services
AWS re:Invent 2016: Introduction to Managed Database Services on AWS (DAT307)Amazon Web Services
Which database is best suited for your use case? Should you choose a relational database or NoSQL or a data warehouse for your workload? Would a managed service like Amazon RDS, Amazon DynamoDB, or Amazon Redshift work better for you, or would it be better to run your own database on Amazon EC2? FanDuel has been running its fantasy sports service on Amazon Web Services (AWS) since 2012. You will learn best practices and insights from FanDuel’s successful migrations from self-managed databases on EC2 to fully-managed database services.
Migrating Your Databases to AWS: Deep Dive on Amazon RDS and AWS Database Mig...Amazon Web Services
Amazon RDS allows you to launch an optimally configured, secure and highly available database with just a few clicks. It provides cost-efficient and resizable capacity, automates time-consuming database administration tasks, and provides you with six familiar database engines to choose from: Amazon Aurora, Oracle, Microsoft SQL Server, PostgreSQL, MySQL and MariaDB. In this session, we will take a close look at the capabilities of Amazon RDS and explain how it works. We’ll also discuss the AWS Database Migration Service and AWS Schema Conversion Tool, which help you migrate databases and data warehouses with minimal downtime from on-premises and cloud environments to Amazon RDS and other Amazon services. Gain your freedom from expensive, proprietary databases while providing your applications with the fast performance, scalability, high availability, and compatibility they need.
AWS Speaker: Andrew Kane, Solutions Architect - Amazon Web Services
AWS re:Invent 2016: JustGiving: Serverless Data Pipelines, Event-Driven ETL, ...Amazon Web Services
Organizations need to gain insight and knowledge from a growing number of Internet of Things (IoT), application programming interfaces (API), clickstreams, unstructured and log data sources. However, organizations are also often limited by legacy data warehouses and ETL processes that were designed for transactional data. Building scalable big data pipelines with automated extract-transform-load (ETL) and machine learning processes can address these limitations. JustGiving is the world’s largest social platform for online giving. In this session, we describe how we created several scalable and loosely coupled event-driven ETL and ML pipelines as part of our in-house data science platform called RAVEN. You learn how to leverage AWS Lambda, Amazon S3, Amazon EMR, Amazon Kinesis, and other services to build serverless, event-driven, data and stream processing pipelines in your organization. We review common design patterns, lessons learned, and best practices, with a focus on serverless big data architectures with AWS Lambda.
Amazon Kinesis provides services for you to work with streaming data on AWS. Learn how to load streaming data continuously and cost-effectively to Amazon S3 and Amazon Redshift using Amazon Kinesis Firehose without writing custom stream processing code. Get an introduction to building custom stream processing applications with Amazon Kinesis Streams for specialised needs.
Deep Dive on Elastic File System - February 2017 AWS Online Tech TalksAmazon Web Services
Organizations face significant challenges moving their applications to the cloud when they require a standard file system interface for accessing their cloud data. In this technical session, we will explore the world’s first cloud-scale file system and its targeted use cases. Attendees will learn about the Amazon Elastic File System (EFS) features and benefits, how to identify applications that are appropriate for use with Amazon EFS, and details about its performance and security models. We will highlight and demonstrate how to deploy Amazon EFS in one of our most common use cases and will share tips for success throughout.
Learning Objectives:
• Recognize why and when to use Amazon EFS
• Understand key technical/security concepts
• Learn how to leverage EFS’s performance
• See a demo of EFS in action
• Review EFS’s economics
Amazon Web Services provides startups with the low cost, easy to use infrastructure needed to scale and grow any size business. Attend this session and learn how to migrate your startup to AWS and make the most out of the platform.
AWS re:Invent 2016: AWS Database State of the Union (DAT320)Amazon Web Services
Raju Gulabani, vice president of AWS Database Services (AWS), discusses the evolution of database services on AWS and the new database services and features we launched this year, and shares our vision for continued innovation in this space. We are witnessing an unprecedented growth in the amount of data collected, in many different shapes and forms. Storage, management, and analysis of this data requires database services that scale and perform in ways not possible before. AWS offers a collection of such database and other data services like Amazon Aurora, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Amazon ElastiCache, Amazon Kinesis, and Amazon EMR to process, store, manage, and analyze data. In this session, we provide an overview of AWS database services and discuss how our customers are using these services today.
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)Amazon Web Services
Amazon EC2 Spot instances provide acceleration, scale, and deep cost savings to run time-critical, hyper-scale workloads for rapid data analysis. In this session, AOL and Metamarkets will present lessons learned and best practices from scaling their big data workloads using popular platforms like Presto, Spark and Druid.
AOL will present how they process, store, and analyze big data securely and cost effectively using Presto. AOL achieved 70% savings by separating compute and storage, dynamically resizing clusters based on volume and complexity, and using AWS Lambda to orchestrate processing pipelines. Metamarkets, an industry leader in interactive analytics, will present how they leverage Amazon EBS to persist 185 TiB of (compressed) state to run Druid historical nodes on EC2 Spot instances. They will also cover how they run Spark for batch jobs to process 1-4 PiB of data across 200 B to 1 T events/day, saving more than 60% in costs.
AWS re:Invent 2016: Workshop: Migrating Microsoft Applications to AWS (ENT216)Amazon Web Services
In this workshop, we will explore the different approaches to migrating Microsoft applications to AWS. We’ll walk through the concerns and considerations to take into account while planning a migration, and learn how to develop and implement a migration plan to move applications from on-premises (or traditional hosting) to AWS. This session will use a case study format to dive deep into the details of how to successfully plan an application migration. To keep it real, teams will work through planning a SharePoint migration that integrates in with an existing Active Directory.
(HLS402) Getting into Your Genes: The Definitive Guide to Using Amazon EMR, A...Amazon Web Services
The key to fighting cancer through better therapeutics is a deep understanding of the basic biology of this disease at a cellular and molecular level. Comprehensive analysis of cancer mutations in specific tumors or cancer cell lines by using Life Technologies sequencing and real-time PCR systems generates gigabytes to terabytes of data every day. Our customers bring together this data in studies that seek to discover the genetic fingerprint of cancer. The data typically translates to millions of records in databases that require complex algorithmic processing, cross-application analysis, and interactive visualizations with real-time response (2-3 seconds) to enable users to consume large volumes of complex scientific information.
We have chosen the AWS platform to bring this new era of data analysis power to our customers by using technologies such as Amazon S3, ElastiCache, and DynamoDB for storage and fast access and Amazon EMR for parallelizing complex computations. Our talk tells the story with rich details about challenges and roadblocks in building data-intense, highly interactive applications in the cloud. We also highlight enhanced customer workflows and highly optimized applications with orders of magnitude improvement in performance and scalability.
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...Amazon Web Services
Building big data applications often requires integrating a broad set of technologies to store, process, and analyze the increasing variety, velocity, and volume of data being collected by many organizations. In this session, we show how you can build entire big data applications using a core set of managed services including Amazon S3, Amazon Kinesis, Amazon EMR, Amazon Elasticsearch Service, Amazon Redshift, and Amazon QuickSight.
We walk you through the steps of building and securing a big data application using the AWS Big Data Platform. We also share best practices and common use cases for AWS big data services, including tips to help you choose the best services for your specific application.
Building analytics applications requires more than just one good service. It requires the ability to capture a vast amount of data, and react to data changes in real time. It requires flexible tools which enable end users to work in the way they can be most productive, and which addresses the needs of both data consumers, as well as data scientists. This analysis won't just be about data exploration and reports, but must be able to support the largest scale, complex machine and deep learning models imaginable. Across it all, strong governance, security, and cataloguing is essential. In this session, come to hear about how to build a full stack analytics application using AWS Services. We'll see how to capture static and dynamic data in real time, and react to data changes. We'll see AWS Services which perform analytics from drag-and-drop, through simple query-on-files, and into exascale data science. At the end, we'll have a data lake architecture that will meet the demands of the most sophisticated analytics customers for many years to come.
AWS Speaker: Ian Robinson, Specialist Solution Architect, Big Data and Analytics, EMEA - Amazon Web Services
Data-driven companies have a need to make their data easily accessible to those who analyze it. Many organizations have adopted the Looker application, LookML on AWS, a centralized analytical database with a user-friendly interface that allows employees to ask and answer their own questions to make informed business decisions.
Join our webinar to learn how our customer, Casper, an online mattress retailer, made the switch from a transactional database to Looker’s data analytics program on Amazon Redshift. Looker on Amazon Redshift can help you greatly reduce your analytics lifecycle with a simplified infrastructure and rapid cloud scaling.
Join us to learn:
• How to utilize LookML to build reusable definitions and logic for your data
• Best practices for architecting a centralized analytical database
• How Casper leveraged Looker and Amazon Redshift to provide all their employees access to their data and metrics
Who should attend: Heads of Analytics, Heads of BI, Analytics Managers, BI Teams, Senior Analysts
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRAmazon Web Services
Customers are migrating their analytics, data processing (ETL), and data science workloads running on Apache Hadoop, Spark, and data warehouse appliances from on-premise deployments to Amazon EMR in order to save costs, increase availability, and improve performance. Amazon EMR is a managed service that lets you process and analyze extremely large data sets using the latest versions of over 15 open-source frameworks in the Apache Hadoop and Spark ecosystems. This session will focus on identifying the components and workflows in your current environment and providing the best practices to migrate these workloads to Amazon EMR. We will explain how to move from HDFS to Amazon S3 as a durable storage layer, and how to lower costs with Amazon EC2 Spot instances and Auto Scaling. Additionally, we will go over common security recommendations and tuning tips to accelerate the time to production.
AWS re:Invent 2016: Getting Started with Amazon Aurora (DAT203)Amazon Web Services
Amazon Aurora is a MySQL-compatible relational database engine with the speed, reliability, and availability of high-end commercial databases at one-tenth the cost. This session introduces you to Amazon Aurora, explores the capabilities and features of Aurora, explains common use cases, and helps you get started with Aurora. Debanjan Saha, general manager for Aurora, explains how Aurora differs from other commonly available databases while staying compatible with MySQL and providing a high-end, cost-effective alternative to commercial and open-source database engines. In addition, Linda Xu, data architect at Ticketmaster, walks you through Ticketmaster's journey to Amazon Aurora, starting with evaluation through production migration of a critical Ticketmaster database to Amazon Aurora. Ticketmaster is one of the world's top 10 e-commerce companies and the global market leader in ticketing. In this session, Linda discusses how Aurora lets Ticketmaster provide better services to their fans, customers, and clients, and helps reduce the cost and operational burden while giving greater flexibility to support heavy traffic spikes.
Migration Recipes for Success - AWS Summit Cape Town 2017 Amazon Web Services
Now that you have earmarked workloads for migration, it's time to look at the various tools and methodologies that are available to help customers shift applications to AWS. This session highlights some of the key AWS tools, services and approaches that organisations are using to successfully migrate to the cloud.
AWS Speaker: Sven Hansen, Solution Architect - Amazon Web Services
Customer Speaker: Pieter Breed – Core Platform Engineer Zoona
Amazon Web Services (AWS) offers a wide range of database options to fit your application requirements. From database services that are fully managed and that can be launched in minutes with just a few clicks to self-managed databases running on EC2. AWS managed database services include Amazon Relational Database Service (Amazon RDS), with support for six commonly used database engines, Amazon Aurora, a MySQL and PostgreSQL-compatible relational database, Amazon DynamoDB, a NoSQL database service or Amazon Redshift, a petabyte-scale data warehouse service. AWS also provides the AWS Database Migration Service, a service which makes it easy and inexpensive to migrate your databases to AWS cloud.
In this webinar, we take a closer look at the AWS database offerings and learn how to quickly select, set up, operate, and scale your database in the cloud.
Learning Objectives:
• Gain insights into the AWS database offering and know which to select for your workload.
• Learn how the AWS Schema Conversion Tool (AWS SCT) and AWS Database Migration Service (AWS DMS) can facilitate and simplify migrating your business critical applications to Amazon Web Services.
• Learn how Amazon DynamoDB Accelerator (DAX) can reduce Amazon DynamoDB response times from milliseconds to microseconds, even at millions of requests per second.
• Hear from our partners like Version1 and Clckwrk who can help you in your journey towards Database freedom.
Introducing Amazon EMR Release 5.0 - August 2016 Monthly Webinar SeriesAmazon Web Services
Amazon EMR is a managed Hadoop service that makes it easy for customers to use big data frameworks and applications like Hadoop, Spark, and Presto to analyze data stored in HDFS or on Amazon S3 , Amazon’s highly scalable object storage service. In this webinar, we will introduce the latest release of Amazon EMR. With Amazon EMR release 5.0, customers can now launch the latest versions of popular open source frameworks including Apache Spark 2.0, Hive 2.1, Presto 0.151, Tez 0.8.4, and Apache Hadoop 2.7.2. We will walk through a demo to show you how to deploy a Hadoop environment within minutes. We will cover common use cases and best practices to lower costs using Amazon S3 as your data store and Amazon EC2 Spot Instances, which allow you to bid on space Amazon computing capacity.
Learning Objectives:
• Describe the new features and updated frameworks in Amazon EMR 5.0
• Learn best practices and real-world applications for Amazon EMR
• Understand how to use EC2 Spot pricing to save costs
• Explain the advantages of decoupling storage and compute with Amazon S3 as storage layer for EMR workloads
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
AWS re:Invent 2016: Case Study: How Monsanto Uses Amazon EFS with Their Large...Amazon Web Services
At Monsanto, we build and use technologies that support our data and also BI efforts that facilitate intelligent, data-driven decisions. In the past year, we've embarked on large-scale efforts to modernize our geospatial platform and improve our analytic processing capabilities by building out new cloud and open-source based services. We found using Amazon Elastic File System (Amazon EFS) gave us the flexibility and performance we were seeking while saving us significant time, effort, and cost. In this session, we discuss how Monsanto uses the Amazon EFS service to run our large scaling geospatial data sets such as raster, and to perform highly parallelized analytics for data scientists and business users. Topics include the technical architecture, how and why we chose EFS for handling data sets that are terabytes in size, our recommendations, and the lessons learned along the way.
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...Amazon Web Services
Media delivery requirements are continually changing, driven by accelerating mobile, tablet, smart TV, and set-top technology advances. Broadcasters need agile solutions to the changing media and entertainment landscape that don't require multiyear projects with large upfront investments. In this session, we walk through Discovery Communications' migration of its broadcast playout and channel origination to AWS. Discovery Communications is a leader in nonfiction media, reaching more than 3 billion cumulative viewers in 220 countries and territories. Traditionally, broadcast origination for content delivered to telecommunications companies, cable TV, and satellite has existed only in on-premises data centers. In this session, we walk through Discovery's migration of broadcast playout supporting hundreds of channels worldwide to AWS. We show how Discovery has not only reduced their TCO but also has improved their agility by launching new channels on demand. We also walk through how channel origination is being deployed in a secure, automated fashion, and with a level of high availability that exceeds what is possible in a traditional data center.
Managing Data with Amazon ElastiCache for Redis - August 2016 Monthly Webinar...Amazon Web Services
Many data sets, such as time-series collections or Internet of Things (IoT) deployments can include huge numbers of sensor reports and other data points, which can be a challenge to manage and aggregate. Amazon ElastiCache for Redis provides an on-demand managed service with the performance and scalability to turn big data into useful information. Join us to learn how to use Amazon ElastiCache to create serverless solutions that lets you rapidly make use of large and multisource data sets.
Learning Objectives:
• Learn how to ingest and analyze sensor data using Amazon ElastiCache for Redis and the AWS IoT Service
• Learn how to use ElastiCache Redis for Time-Series data
AWS re:Invent 2016: FINRA in the Cloud: the Big Data Enterprise (ENT313)Amazon Web Services
Large-scale enterprise migration can be a complex undertaking, especially for organizations that re-architect solutions to leverage the benefits of the Cloud. FINRA, which regulates US equities and options markets, recently completed a 2.5-year migration and re-architecture of its Big Data platform. Their platform consumes billions of market events every day. FINRA has developed scalable platforms and services on AWS that enable migrating enterprise applications and business functions to the Cloud quickly. Their data management platform takes advantage of AWS storage and compute products. In this session, IT influencers and decision makers will learn lessons from FINRA’s migration, including how to create an enterprise-class Cloud architecture and which technology skills are required for transitioning to the Cloud. We also share examples of the business value FINRA has realized.
BDA403 How Netflix Monitors Applications in Real-time with Amazon KinesisAmazon Web Services
Thousands of services work in concert to deliver millions of hours of video streams to Netflix customers every day. These applications vary in size, function, and technology, but they all make use of the Netflix network to communicate. Understanding the interactions between these services is a daunting challenge both because of the sheer volume of traffic and the dynamic nature of deployments. In this talk, we’ll first discuss why Netflix chose Amazon Kinesis Streams over other streaming data solutions like Kafka to address these challenges at scale. We’ll then dive deep into how Netflix uses Amazon Kinesis Streams to enrich network traffic logs and identify usage patterns in real time. Lastly, we will cover how Netflix uses this system to build comprehensive dependency maps, increase network efficiency, and improve failure resiliency. From this talk, you’ll take away techniques and processes that you can apply to your large-scale networks and derive real-time, actionable insights.
AWS re:Invent 2016: Billions of Rows Transformed in Record Time Using Matilli...Amazon Web Services
Billions of Rows Transformed in Record Time Using Matillion ETL for Amazon Redshift
GE Power & Water develops advanced technologies to help solve some of the world’s most complex challenges related to water availability and quality. They had amassed billions of rows of data on on-premises databases, but decided to migrate some of their core big data projects to the AWS Cloud. When they decided to transform and store it all in Amazon Redshift, they knew they needed an ETL/ELT tool that could handle this enormous amount of data and safely deliver it to its destination. In this session, Ryan Oates, Enterprise Architect at GE Water, shares his use case, requirements, outcomes and lessons learned. He also shares the details of his solution stack, including Amazon Redshift and Matillion ETL for Amazon Redshift in AWS Marketplace. You learn best practices on Amazon Redshift ETL supporting enterprise analytics and big data requirements, simply and at scale. You learn how to simplify data loading, transformation and orchestration on to Amazon Redshift and how build out a real data pipeline. Get the insights to deliver your big data project in record time.
AWS re:Invent 2016: Moving Mission Critical Apps from One Region to Multi-Reg...Amazon Web Services
In gaming, low latencies and connectivity are bare minimum expectations users have while playing online on PlayStation Network. Alex and Dustin share key architectural patterns to provide low latency, multi-region services to global users. They discuss the testing methodologies and how to programmatically map out a large dependency multi-region deployment with data-driven techniques. The patterns shared show how to adapt to changing bottlenecks and sudden, several million request spikes. You’ll walk away with several key architectural patterns that can service users at global scale while being mindful of costs.
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...Amazon Web Services
IT is evolving from a cost center to a source of continuous innovation for business. At the heart of this transition are modern, revenue-generating applications, based on dynamic architectures that constantly evolve to keep pace with end-customer demands. This dynamic application environment requires a new, comprehensive approach to traditional monitoring – one based on real-time, end-to-end visibility and analytics across the entire application lifecycle and stack, instead of monitoring by piecemeal. This presentation highlights practical advice on how developers and operators can leverage data and analytics to glean critical information about their modern applications. In this session, we will cover the types of data important for today’s modern applications. We’ll discuss visibility and analytics into data sources such as AWS services (e.g., Amazon CloudWatch, AWS Lambda, VPC Flow Logs, Amazon EC2, Amazon S3, etc.), development tool chain, and custom metrics, and describe how to use analytics to understand business performance and behaviors. We discuss a comprehensive approach to monitoring, troubleshooting, and customer usage insights, provide examples of effective data analytics to improve software quality, and describe an end-to-end customer use case that highlights how analytics applies to the modern app lifecycle and stack. Session sponsored by Sumo Logic.
AWS Competency Partner
AWS re:Invent 2016: AWS Database State of the Union (DAT320)Amazon Web Services
Raju Gulabani, vice president of AWS Database Services (AWS), discusses the evolution of database services on AWS and the new database services and features we launched this year, and shares our vision for continued innovation in this space. We are witnessing an unprecedented growth in the amount of data collected, in many different shapes and forms. Storage, management, and analysis of this data requires database services that scale and perform in ways not possible before. AWS offers a collection of such database and other data services like Amazon Aurora, Amazon DynamoDB, Amazon RDS, Amazon Redshift, Amazon ElastiCache, Amazon Kinesis, and Amazon EMR to process, store, manage, and analyze data. In this session, we provide an overview of AWS database services and discuss how our customers are using these services today.
AWS re:Invent 2016: Disrupting Big Data with Cost-effective Compute (CMP302)Amazon Web Services
Amazon EC2 Spot instances provide acceleration, scale, and deep cost savings to run time-critical, hyper-scale workloads for rapid data analysis. In this session, AOL and Metamarkets will present lessons learned and best practices from scaling their big data workloads using popular platforms like Presto, Spark and Druid.
AOL will present how they process, store, and analyze big data securely and cost effectively using Presto. AOL achieved 70% savings by separating compute and storage, dynamically resizing clusters based on volume and complexity, and using AWS Lambda to orchestrate processing pipelines. Metamarkets, an industry leader in interactive analytics, will present how they leverage Amazon EBS to persist 185 TiB of (compressed) state to run Druid historical nodes on EC2 Spot instances. They will also cover how they run Spark for batch jobs to process 1-4 PiB of data across 200 B to 1 T events/day, saving more than 60% in costs.
AWS re:Invent 2016: Workshop: Migrating Microsoft Applications to AWS (ENT216)Amazon Web Services
In this workshop, we will explore the different approaches to migrating Microsoft applications to AWS. We’ll walk through the concerns and considerations to take into account while planning a migration, and learn how to develop and implement a migration plan to move applications from on-premises (or traditional hosting) to AWS. This session will use a case study format to dive deep into the details of how to successfully plan an application migration. To keep it real, teams will work through planning a SharePoint migration that integrates in with an existing Active Directory.
(HLS402) Getting into Your Genes: The Definitive Guide to Using Amazon EMR, A...Amazon Web Services
The key to fighting cancer through better therapeutics is a deep understanding of the basic biology of this disease at a cellular and molecular level. Comprehensive analysis of cancer mutations in specific tumors or cancer cell lines by using Life Technologies sequencing and real-time PCR systems generates gigabytes to terabytes of data every day. Our customers bring together this data in studies that seek to discover the genetic fingerprint of cancer. The data typically translates to millions of records in databases that require complex algorithmic processing, cross-application analysis, and interactive visualizations with real-time response (2-3 seconds) to enable users to consume large volumes of complex scientific information.
We have chosen the AWS platform to bring this new era of data analysis power to our customers by using technologies such as Amazon S3, ElastiCache, and DynamoDB for storage and fast access and Amazon EMR for parallelizing complex computations. Our talk tells the story with rich details about challenges and roadblocks in building data-intense, highly interactive applications in the cloud. We also highlight enhanced customer workflows and highly optimized applications with orders of magnitude improvement in performance and scalability.
AWS re:Invent 2016: Building Big Data Applications with the AWS Big Data Plat...Amazon Web Services
Building big data applications often requires integrating a broad set of technologies to store, process, and analyze the increasing variety, velocity, and volume of data being collected by many organizations. In this session, we show how you can build entire big data applications using a core set of managed services including Amazon S3, Amazon Kinesis, Amazon EMR, Amazon Elasticsearch Service, Amazon Redshift, and Amazon QuickSight.
We walk you through the steps of building and securing a big data application using the AWS Big Data Platform. We also share best practices and common use cases for AWS big data services, including tips to help you choose the best services for your specific application.
Building analytics applications requires more than just one good service. It requires the ability to capture a vast amount of data, and react to data changes in real time. It requires flexible tools which enable end users to work in the way they can be most productive, and which addresses the needs of both data consumers, as well as data scientists. This analysis won't just be about data exploration and reports, but must be able to support the largest scale, complex machine and deep learning models imaginable. Across it all, strong governance, security, and cataloguing is essential. In this session, come to hear about how to build a full stack analytics application using AWS Services. We'll see how to capture static and dynamic data in real time, and react to data changes. We'll see AWS Services which perform analytics from drag-and-drop, through simple query-on-files, and into exascale data science. At the end, we'll have a data lake architecture that will meet the demands of the most sophisticated analytics customers for many years to come.
AWS Speaker: Ian Robinson, Specialist Solution Architect, Big Data and Analytics, EMEA - Amazon Web Services
Data-driven companies have a need to make their data easily accessible to those who analyze it. Many organizations have adopted the Looker application, LookML on AWS, a centralized analytical database with a user-friendly interface that allows employees to ask and answer their own questions to make informed business decisions.
Join our webinar to learn how our customer, Casper, an online mattress retailer, made the switch from a transactional database to Looker’s data analytics program on Amazon Redshift. Looker on Amazon Redshift can help you greatly reduce your analytics lifecycle with a simplified infrastructure and rapid cloud scaling.
Join us to learn:
• How to utilize LookML to build reusable definitions and logic for your data
• Best practices for architecting a centralized analytical database
• How Casper leveraged Looker and Amazon Redshift to provide all their employees access to their data and metrics
Who should attend: Heads of Analytics, Heads of BI, Analytics Managers, BI Teams, Senior Analysts
BDA302 Deep Dive on Migrating Big Data Workloads to Amazon EMRAmazon Web Services
Customers are migrating their analytics, data processing (ETL), and data science workloads running on Apache Hadoop, Spark, and data warehouse appliances from on-premise deployments to Amazon EMR in order to save costs, increase availability, and improve performance. Amazon EMR is a managed service that lets you process and analyze extremely large data sets using the latest versions of over 15 open-source frameworks in the Apache Hadoop and Spark ecosystems. This session will focus on identifying the components and workflows in your current environment and providing the best practices to migrate these workloads to Amazon EMR. We will explain how to move from HDFS to Amazon S3 as a durable storage layer, and how to lower costs with Amazon EC2 Spot instances and Auto Scaling. Additionally, we will go over common security recommendations and tuning tips to accelerate the time to production.
AWS re:Invent 2016: Getting Started with Amazon Aurora (DAT203)Amazon Web Services
Amazon Aurora is a MySQL-compatible relational database engine with the speed, reliability, and availability of high-end commercial databases at one-tenth the cost. This session introduces you to Amazon Aurora, explores the capabilities and features of Aurora, explains common use cases, and helps you get started with Aurora. Debanjan Saha, general manager for Aurora, explains how Aurora differs from other commonly available databases while staying compatible with MySQL and providing a high-end, cost-effective alternative to commercial and open-source database engines. In addition, Linda Xu, data architect at Ticketmaster, walks you through Ticketmaster's journey to Amazon Aurora, starting with evaluation through production migration of a critical Ticketmaster database to Amazon Aurora. Ticketmaster is one of the world's top 10 e-commerce companies and the global market leader in ticketing. In this session, Linda discusses how Aurora lets Ticketmaster provide better services to their fans, customers, and clients, and helps reduce the cost and operational burden while giving greater flexibility to support heavy traffic spikes.
Migration Recipes for Success - AWS Summit Cape Town 2017 Amazon Web Services
Now that you have earmarked workloads for migration, it's time to look at the various tools and methodologies that are available to help customers shift applications to AWS. This session highlights some of the key AWS tools, services and approaches that organisations are using to successfully migrate to the cloud.
AWS Speaker: Sven Hansen, Solution Architect - Amazon Web Services
Customer Speaker: Pieter Breed – Core Platform Engineer Zoona
Amazon Web Services (AWS) offers a wide range of database options to fit your application requirements. From database services that are fully managed and that can be launched in minutes with just a few clicks to self-managed databases running on EC2. AWS managed database services include Amazon Relational Database Service (Amazon RDS), with support for six commonly used database engines, Amazon Aurora, a MySQL and PostgreSQL-compatible relational database, Amazon DynamoDB, a NoSQL database service or Amazon Redshift, a petabyte-scale data warehouse service. AWS also provides the AWS Database Migration Service, a service which makes it easy and inexpensive to migrate your databases to AWS cloud.
In this webinar, we take a closer look at the AWS database offerings and learn how to quickly select, set up, operate, and scale your database in the cloud.
Learning Objectives:
• Gain insights into the AWS database offering and know which to select for your workload.
• Learn how the AWS Schema Conversion Tool (AWS SCT) and AWS Database Migration Service (AWS DMS) can facilitate and simplify migrating your business critical applications to Amazon Web Services.
• Learn how Amazon DynamoDB Accelerator (DAX) can reduce Amazon DynamoDB response times from milliseconds to microseconds, even at millions of requests per second.
• Hear from our partners like Version1 and Clckwrk who can help you in your journey towards Database freedom.
Introducing Amazon EMR Release 5.0 - August 2016 Monthly Webinar SeriesAmazon Web Services
Amazon EMR is a managed Hadoop service that makes it easy for customers to use big data frameworks and applications like Hadoop, Spark, and Presto to analyze data stored in HDFS or on Amazon S3 , Amazon’s highly scalable object storage service. In this webinar, we will introduce the latest release of Amazon EMR. With Amazon EMR release 5.0, customers can now launch the latest versions of popular open source frameworks including Apache Spark 2.0, Hive 2.1, Presto 0.151, Tez 0.8.4, and Apache Hadoop 2.7.2. We will walk through a demo to show you how to deploy a Hadoop environment within minutes. We will cover common use cases and best practices to lower costs using Amazon S3 as your data store and Amazon EC2 Spot Instances, which allow you to bid on space Amazon computing capacity.
Learning Objectives:
• Describe the new features and updated frameworks in Amazon EMR 5.0
• Learn best practices and real-world applications for Amazon EMR
• Understand how to use EC2 Spot pricing to save costs
• Explain the advantages of decoupling storage and compute with Amazon S3 as storage layer for EMR workloads
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
AWS re:Invent 2016: Case Study: How Monsanto Uses Amazon EFS with Their Large...Amazon Web Services
At Monsanto, we build and use technologies that support our data and also BI efforts that facilitate intelligent, data-driven decisions. In the past year, we've embarked on large-scale efforts to modernize our geospatial platform and improve our analytic processing capabilities by building out new cloud and open-source based services. We found using Amazon Elastic File System (Amazon EFS) gave us the flexibility and performance we were seeking while saving us significant time, effort, and cost. In this session, we discuss how Monsanto uses the Amazon EFS service to run our large scaling geospatial data sets such as raster, and to perform highly parallelized analytics for data scientists and business users. Topics include the technical architecture, how and why we chose EFS for handling data sets that are terabytes in size, our recommendations, and the lessons learned along the way.
AWS re:Invent 2016: Discovery Channel's Broadcast Workflows and Channel Origi...Amazon Web Services
Media delivery requirements are continually changing, driven by accelerating mobile, tablet, smart TV, and set-top technology advances. Broadcasters need agile solutions to the changing media and entertainment landscape that don't require multiyear projects with large upfront investments. In this session, we walk through Discovery Communications' migration of its broadcast playout and channel origination to AWS. Discovery Communications is a leader in nonfiction media, reaching more than 3 billion cumulative viewers in 220 countries and territories. Traditionally, broadcast origination for content delivered to telecommunications companies, cable TV, and satellite has existed only in on-premises data centers. In this session, we walk through Discovery's migration of broadcast playout supporting hundreds of channels worldwide to AWS. We show how Discovery has not only reduced their TCO but also has improved their agility by launching new channels on demand. We also walk through how channel origination is being deployed in a secure, automated fashion, and with a level of high availability that exceeds what is possible in a traditional data center.
Managing Data with Amazon ElastiCache for Redis - August 2016 Monthly Webinar...Amazon Web Services
Many data sets, such as time-series collections or Internet of Things (IoT) deployments can include huge numbers of sensor reports and other data points, which can be a challenge to manage and aggregate. Amazon ElastiCache for Redis provides an on-demand managed service with the performance and scalability to turn big data into useful information. Join us to learn how to use Amazon ElastiCache to create serverless solutions that lets you rapidly make use of large and multisource data sets.
Learning Objectives:
• Learn how to ingest and analyze sensor data using Amazon ElastiCache for Redis and the AWS IoT Service
• Learn how to use ElastiCache Redis for Time-Series data
AWS re:Invent 2016: FINRA in the Cloud: the Big Data Enterprise (ENT313)Amazon Web Services
Large-scale enterprise migration can be a complex undertaking, especially for organizations that re-architect solutions to leverage the benefits of the Cloud. FINRA, which regulates US equities and options markets, recently completed a 2.5-year migration and re-architecture of its Big Data platform. Their platform consumes billions of market events every day. FINRA has developed scalable platforms and services on AWS that enable migrating enterprise applications and business functions to the Cloud quickly. Their data management platform takes advantage of AWS storage and compute products. In this session, IT influencers and decision makers will learn lessons from FINRA’s migration, including how to create an enterprise-class Cloud architecture and which technology skills are required for transitioning to the Cloud. We also share examples of the business value FINRA has realized.
BDA403 How Netflix Monitors Applications in Real-time with Amazon KinesisAmazon Web Services
Thousands of services work in concert to deliver millions of hours of video streams to Netflix customers every day. These applications vary in size, function, and technology, but they all make use of the Netflix network to communicate. Understanding the interactions between these services is a daunting challenge both because of the sheer volume of traffic and the dynamic nature of deployments. In this talk, we’ll first discuss why Netflix chose Amazon Kinesis Streams over other streaming data solutions like Kafka to address these challenges at scale. We’ll then dive deep into how Netflix uses Amazon Kinesis Streams to enrich network traffic logs and identify usage patterns in real time. Lastly, we will cover how Netflix uses this system to build comprehensive dependency maps, increase network efficiency, and improve failure resiliency. From this talk, you’ll take away techniques and processes that you can apply to your large-scale networks and derive real-time, actionable insights.
AWS re:Invent 2016: Billions of Rows Transformed in Record Time Using Matilli...Amazon Web Services
Billions of Rows Transformed in Record Time Using Matillion ETL for Amazon Redshift
GE Power & Water develops advanced technologies to help solve some of the world’s most complex challenges related to water availability and quality. They had amassed billions of rows of data on on-premises databases, but decided to migrate some of their core big data projects to the AWS Cloud. When they decided to transform and store it all in Amazon Redshift, they knew they needed an ETL/ELT tool that could handle this enormous amount of data and safely deliver it to its destination. In this session, Ryan Oates, Enterprise Architect at GE Water, shares his use case, requirements, outcomes and lessons learned. He also shares the details of his solution stack, including Amazon Redshift and Matillion ETL for Amazon Redshift in AWS Marketplace. You learn best practices on Amazon Redshift ETL supporting enterprise analytics and big data requirements, simply and at scale. You learn how to simplify data loading, transformation and orchestration on to Amazon Redshift and how build out a real data pipeline. Get the insights to deliver your big data project in record time.
AWS re:Invent 2016: Moving Mission Critical Apps from One Region to Multi-Reg...Amazon Web Services
In gaming, low latencies and connectivity are bare minimum expectations users have while playing online on PlayStation Network. Alex and Dustin share key architectural patterns to provide low latency, multi-region services to global users. They discuss the testing methodologies and how to programmatically map out a large dependency multi-region deployment with data-driven techniques. The patterns shared show how to adapt to changing bottlenecks and sudden, several million request spikes. You’ll walk away with several key architectural patterns that can service users at global scale while being mindful of costs.
AWS re:Invent 2016: Effective Application Data Analytics for Modern Applicati...Amazon Web Services
IT is evolving from a cost center to a source of continuous innovation for business. At the heart of this transition are modern, revenue-generating applications, based on dynamic architectures that constantly evolve to keep pace with end-customer demands. This dynamic application environment requires a new, comprehensive approach to traditional monitoring – one based on real-time, end-to-end visibility and analytics across the entire application lifecycle and stack, instead of monitoring by piecemeal. This presentation highlights practical advice on how developers and operators can leverage data and analytics to glean critical information about their modern applications. In this session, we will cover the types of data important for today’s modern applications. We’ll discuss visibility and analytics into data sources such as AWS services (e.g., Amazon CloudWatch, AWS Lambda, VPC Flow Logs, Amazon EC2, Amazon S3, etc.), development tool chain, and custom metrics, and describe how to use analytics to understand business performance and behaviors. We discuss a comprehensive approach to monitoring, troubleshooting, and customer usage insights, provide examples of effective data analytics to improve software quality, and describe an end-to-end customer use case that highlights how analytics applies to the modern app lifecycle and stack. Session sponsored by Sumo Logic.
AWS Competency Partner
AWS re:Invent 2016: ElastiCache Deep Dive: Best Practices and Usage Patterns ...Amazon Web Services
In this session, we provide a peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns with our Redis and Memcached offerings and how customers have used them for in-memory operations to reduce latency and improve application throughput. During this session, we review ElastiCache best practices, design patterns, and anti-patterns.
AWS re:Invent 2016: High Performance Computing on AWS (CMP207)Amazon Web Services
High performance computing in the cloud is enabling high scale compute- and graphics-intensive workloads across industries, ranging from aerospace, automotive, and manufacturing to life sciences, financial services, and energy. AWS provides application developers and end users with unprecedented computational power for massively parallel applications, in areas such as large-scale fluid and materials simulations, 3D content rendering, financial computing, and deep learning. This session provides an overview of HPC capabilities on AWS, describes the newest generations of accelerated computing instances (including P2), as well as highlighting customer and partner use-cases across industries.
Attendees learn about best practices for running HPC workflows in the cloud, including graphical pre- and post-processing, workflow automation, and optimization. Attendees also learn about new and emerging HPC use cases: in particular, deep learning training and inference, large-scale simulations, and high performance data analytics.
AWS re:Invent 2016: What’s New with Amazon Redshift (BDA304)Amazon Web Services
In this session, you learn about the latest and hottest features of Amazon Redshift. Join Vidhya Srinivasan, General Manager of Amazon Redshift, to take a deep dive into the architecture and inner workings of Amazon Redshift. You discover how the recent availability, performance, and manageability improvements we’ve made can significantly enhance your end user experience. You also get a glimpse of what we are working on and our plans for the future.
AWS re:Invent 2016: How to Scale and Operate Elasticsearch on AWS (DEV307)Amazon Web Services
Elasticsearch has quickly become the leading open source technology for scaling search and building document services on. Many software providers have come to rely on it to serve the needs of high-performance, production applications.
In this talk, we’ll go deep on lessons learned from three years in production scaling from a few shards to more than 100 spread across 100s of nodes on AWS--to serve real-time queries against 100s of millions of documents.
Attendees will learn:
* How to capacity plan for ES on AWS
* How to scale and reshard on AWS with zero downtime
* What AWS and ES metrics to collect and alert on
* Tips on day to day ES operations
Session sponsored by SignalFx.
AWS re:Invent 2016: Case Study: How Startups like Mapbox, Ring, Hudl, and Oth...Amazon Web Services
Join us for this lightning-round showcase of hot new brands and startup companies that are using AWS to play a really big game. You'll hear from experts like Mapbox CIO Will White, Ring Senior Engineer Jason Gluckman, Hudl Engineering Director Rob Hruska, and many others as they explain how they thought about the problems they faced and how they solved them in this TED-style session packed with lots of creative thinking.
In this session, our automation engineers will talk about automation internals and demonstrate some new features like importing existing clusters into MMS, converting between storage engines, managing authentication, and interacting with our automation capabilities using the MMS API.
Increase Your Mission Critical Application Performance without Breaking the B...DataCore Software
In virtualized environments, mission critical applications get bogged down, leading to user complaints. Root cause analysis has shown that inadequate storage performance is the culprit. But, fixing these performance issues will cost 5 to 7 times your current storage.
In this presentation, learn about a revolutionary solution that combines Skyera’s advanced All Flash Arrays (AFA) with DataCore’s innovative Software-defined Storage platform. This solution will easily accelerate your SQL Servers at a price that fits your budget.
Webinar: Ensuring Zero Downtime for Your Mission Critical AppMongoDB
As a Database Administrator, you know just how stressful it can be to ensure your system is never down. MongoDB seeks to solve this problem by ensuring that no matter which database task you're performing, you maintain high availability.
This webinar will cover how to ensure zero downtime when managing a MongoDB cluster. We'll start by covering replica sets and then dive into automated software upgrades, and describe various deployment topologies including globally deployed clusters over multiple continents. We will also describe how MongoDB helps you scale on demand without requiring downtime, by automatically redistributing data across the newly provisioned hardware.
Topics covered include:
High Availability through Replica Sets
Automated Upgrades
Backups
Scaling - Sharding
Common maintenance operations
Mission Critical Applications Workloads on Amazon Web ServicesAmazon Web Services
In this session we will walk through practical examples of how Amazon Web Services customers operate heavily regulated workloads and mission critical applications in the cloud. Through real world customer examples we will apply security and governance controls which will provide you with increased visibility and control of your application and infrastructure for these workloads. You will learn how Enterprise secure and enable audit controls on their heavily regulated workloads in an Amazon Web Services Account. At the same time, extend your datacenter and control mechanisms to Amazon Web Services.
McGraw-Hill Education: Global Migration in Less than 2 Years (ENT211) | AWS r...Amazon Web Services
McGraw-Hill Education, a multi-billion-dollar publishing company, moved to digital on company-owned data centers, and now are migrating to AWS. This session details their two-year migration plan for the eventual global deployment of all MHE platforms on AWS. Learn about the business drivers as well as the technical and business challenges they have faced and overcome to date.
AWS re:Invent 2016: How Aptean uses AWS Marketplace storage solutions to back...Amazon Web Services
Aptean is a global enterprise software provider that uses AWS as the core of its infrastructure because it’s a solution that reliably backs up Aptean’s Amazon EC2 instances. Come to this session to learn what happened when Aptean needed a highly reliable, full backup solution that also allowed for ease of scale, automation, and instant recovery in case of a failure. In this session, you’ll learn how CPM helps take full advantage of AWS Snapshots, thus adding a management layer to control retention, automate recoveries, and allow live, application-aware backup of both Windows and Linux instances. Aptean will also why they selected N2W Cloud Protection Manager (CPM) in AWS Marketplace as its backup solution extending AWS services and supporting their diverse customer base. By the end of the session, you’ll have learned the details of how CPM helps Aptean properly manage its snapshots and recoveries, ensuring a resilient deployment that meets Aptean’s business continuity goals.
AWS re:Invent 2016: Automating Cloud Management and Deployment for a Diverse ...Amazon Web Services
Building scalable automation tools that work across heterogeneous application environments is challenging and can inhibit enterprise cloud migration efforts. Learn how the DevOps team at Infor, one of the world’s leading ISVs, manages dozens of enterprise applications built with a variety of technologies and application architectures. In addition to Infor’s approach to deployment and management automation, this session will cover the core tools they’ve developed on top of native AWS services such as AWS CloudFormation, AWS CodeDeploy, and AWS Lambda.
Running Mission Critical Workload for Financial Services Institutions on AWSAmazon Web Services
In this session we will walk through practical examples of how Financial Services Institutions (FSI) operate both common workloads and mission critical applications on AWS. Through real customer examples, we will show you how to leverage the AWS cloud platform to make your application more resilient, reliable and cost effective while increasing your visibility. You will also learn how FSI’s deploy, architect and secure their workloads on AWS and how to leverage platform features to extend and integrate your existing infrastructure with AWS.
AWS re:Invent 2016: How News UK Centralized Cloud Governance Through Policy M...Amazon Web Services
When you run a complex AWS environment with thousands of Amazon EC2 instances, more than half a petabyte of object storage, and support the largest daily newspapers in the UK, you need a world-class cloud management strategy. For companies like News Corp, implementing policies that automate infrastructure schedules, right-size workloads, and manage and modify reservations is critical. As you scale your cloud infrastructure, defining centralized governance rules while enabling decentralized management is key to running an optimized cloud.
This session is designed for advanced operations, infrastructure, and engineering teams to improve/deploy optimization strategies. It covers the five best cloud management practices, including automating Reserved Instance modifications, setting policies to ensure proper tagging, and scheduling lights-on/lights-off policies. Session sponsored by CloudHealth Technologies.
AWS re:Invent 2016: Best practices for running enterprise workloads on AWS (E...Amazon Web Services
Fortune 500 companies are increasingly using cloud services to run enterprise workloads to improve security, increase agility, and enable scale. Learn how OpenEye is running their AWS-native platform and workflow engine to support collaboration and data sharing at large pharmaceutical companies like Pfizer. In this session, OpenEye will share cloud best practiced around security controls, cross-departmental collaboration across the enterprise, and agility at scale. Attendees will gain practical tips for using AWS in the enterprise and healthcare industries.
What do companies with internal platforms have to change to succeed in the cloud? The four pillars at the heart of IT solutions in the cloud are reliability, performance efficiency, security, and cost optimization. This talk discusses cloud well-architected patterns and the tools that facilitate the development and automate the DevOps process. The talk also provides concrete examples of serverless architecture and migration adoption.
Getting Started with AWS Lambda and the Serverless Cloud - AWS Summit Cape T...Amazon Web Services
Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more. In this session, you’ll learn how to get started with serverless computing with AWS Lambda, which lets you run code without provisioning or managing servers. We’ll introduce you to the basics of building with Lambda and how you can benefit from features such as continuous scaling, built-in high availability, integrations with AWS and third-party apps, and subsecond metering pricing. We’ll also introduce you to the broader portfolio of AWS services that help you build serverless applications with Lambda, including Amazon API Gateway, Amazon DynamoDB, AWS Step Functions, and more.
AWS Speaker : Danilo Poccia, Technical Evangelist - Amazon Web Services
Understanding AWS Managed Databases and Analytic Services - AWS Innovate Otta...Amazon Web Services
• Overview of database services to elevate your applications, analytic services to engage your data, and migration services to help you reach database freedom.
• Survey of how Canadian and other organizations are using the cloud to make data scalable, reliable, and secure.
Join us to learn about the state of serverless computing from Dr. Tim Wagner, General Manager of AWS Lambda. Dr. Wagner discusses the latest developments from AWS Lambda and the serverless computing ecosystem. He talks about how serverless computing is becoming a core component in how companies build and run their applications and services, and he also discusses how serverless computing will continue to evolve.
Building a Real Time Dashboard with Amazon Kinesis, Amazon Lambda and Amazon ...Amazon Web Services
Organisations today need a way to manage the ever-increasing volume of data from numerous sources such as log systems, click streams or connected devices and be able to analyse this data in real-time. In this session we will walk through an architecture demonstration of how to leverage AWS services to meet these needs.
Speaker: Ganesh Raja, Solutions Architect, Amazon Web Services
Accenture Cloud Platform helps customers manage public and private enterprise cloud resources effectively and securely. In this session, learn how we designed and built new core platform capabilities using a serverless, microservices-based architecture that is based on AWS services such as AWS Lambda and Amazon API Gateway. During our journey, we discovered a number of key benefits, including a dramatic increase in developer velocity, a reduction (to almost zero) of reliance on other teams, reduced costs, greater resilience, and scalability. We describe the (wild) successes we’ve had and the challenges we’ve overcome to create an AWS serverless architecture at scale. Session sponsored by Accenture.
AWS Competency Partner
AWS has a large and growing portfolio of big data management and analytics services, designed to integrate into solution architectures to meet the needs of your business. In this session, we look at analytics through the eyes of a business intelligence analyst, a data scientist, and an application developer, to explore how to quickly leverage Amazon Redshift, Amazon QuickSight, RStudio, and Amazon Machine Learning to create powerful, yet straightforward, business solutions.
If you could not be one of the 60,000+ in attendance at Amazon AWS re:Invent, the yearly Amazon Cloud Conference, get the 411 on what major announcements that were made in Las Vegas. This presentation covers new AWS services & products, exciting announcements, and updated features.
2016-06 - Design your api management strategy - AWS - Microservices on AWSSmartWave
Morning session started with a presentation on working with a micro-services API gateway in hybrid architectures, by Jean-Pierre LeGoaller, Architect at AWS. We learned how to greatly reduce coding efforts, make applications far more efficient, and decrease errors all at the same time, using small and flexible Micro-services with an API Gateway. Jean-Pierre then illustrated the benefits of AWS lambda function to run seamlessly codes as a service in AWS high-availability compute infrastructure.
AWS Cloud Kata 2013 | Singapore - Getting to Scale on AWSAmazon Web Services
This session will focus on how to get from 'Minimum Viable Product' (MVP) to scale. It will also explain how to deal with unpredictable demand and how to build a scalable business. Attend this session to learn how to:
Scale web servers and app services with Elastic Load Balancing and Auto Scaling on Amazon EC2
Scale your storage on Amazon S3 and S3 Reduced Redundancy Storage
Scale your database with Amazon DynamoDB, Amazon RDS, and Amazon ElastiCache
Scale your customer base by reaching customers globally in minutes with Amazon CloudFront
With AWS Lambda, you can easily build scalable microservices for mobile, web, and IoT applications or respond to events from other AWS services without managing infrastructure. In this session, you’ll see demonstrations and hear more about newly launched features. We’ll show you how to use Lambda to build web, mobile, or IoT backends and voice-enabled apps, and we'll show you how to extend both AWS and third party services by triggering Lambda functions. We’ll also provide productivity and performance tips for getting the most out of your Lambda functions and show how cloud native architectures use Lambda to eliminate “cold servers” and excess capacity without sacrificing scalability or responsiveness.
With AWS Lambda, you can easily build scalable microservices for mobile, web, and IoT applications or respond to events from other AWS services without managing infrastructure. In this session, you’ll see demonstrations and hear more about newly launched features. We’ll show you how to use Lambda to build web, mobile, or IoT backends and voice-enabled apps, and we’ll show you how to extend both AWS and third party services by triggering Lambda functions. We’ll also provide productivity and performance tips for getting the most out of your Lambda functions and show how cloud native architectures use Lambda to eliminate “cold servers” and excess capacity without sacrificing scalability or responsiveness.
How to use Lambda to build web, mobile, or IoT backends and voice-enabled apps, and we'll show you how to extend both AWS and third party services by triggering Lambda functions.
With AWS Lambda, you can easily build scalable microservices for mobile, web, and IoT applications or respond to events from other AWS services without managing infrastructure. In this session, you’ll see demonstrations and hear more about newly launched features. We’ll show you how to use Lambda to build web, mobile, or IoT backends and voice-enabled apps, and we'll show you how to extend both AWS and third party services by triggering Lambda functions. We’ll also provide productivity and performance tips for getting the most out of your Lambda functions and show how cloud native architectures use Lambda to eliminate “cold servers” and excess capacity without sacrificing scalability or responsiveness.
Building and Managing Scalable Applications on AWS: 1 to 500K usersAmazon Web Services
This presentation session from the Cloud Management, Services and Applications Theatre at Cloud Expo Europe 2014 explores the techniques and AWS services that you can use in order to build high scalability web applications on AWS. It also features a great overview of a high-scalability mobile application built by Myriad Group, and AWS customer, that serves over 41 million users.
The AWS Lambda is now available in Singapore and we are excited to invite you to participate in a webinar to learn more about the service and ask questions live throughout the webinar and receive responses during the Q&A session. In this one hour session, you will get to understand key AWS Lambda features, learn the AWS Lambda programming model and get tips on getting the most out of AWS Lambda.
AWS Lambda is a new compute service that runs your code in response to events and automatically manages compute resources for you. In this webinar you’ll learn what you need to quickly begin building mobile, tablet, or IoT applications that use AWS Lambda as a serverless back-end. You’ll also hear about Amazon Web Service’s Event-Driven Compute strategy and see demonstrations that use Lambda to respond to events from Amazon S3 notifications and Amazon DynamoDB streams.
Similar to AWS re:Invent 2016: Event Handling at Scale: Designing an Auditable Ingestion and Persistence Architecture for 10K+ events/second (ARC306) (20)
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
2. What to Expect from the Session
• Business Background: Reporting and event-driven
analytics for hundreds of thousands of concurrent
learners in a reliable, secure, and auditable manner that
is cost effective
• Learning events
• Reporting & analytics architecture
• Architecture tradeoffs
• Challenges, built confidence, and lessons learned
3. Background
• McGraw-Hill Education is a digital learning company
• Learning Management Systems and Learning Analytics
• 2 million to 14 million students, initial load 10 thousand
events per second to scale to 15 million per second
• Cyclical and cost conscious nature of business – low
price point per student
• Service Level Agreements
• ‘Just-In-Time’ Insights; Example is Connect Insights
4. Background
• McGraw-Hill Education is a digital learning company
• Learning Management Systems and Learning Analytics
• 2 million to 14 million students, initial load 10 thousand
events per second to scale to 15 million per second
• Cyclical and cost conscious nature of business – low
price point per student
• Service Level Agreements
• ‘Just-In-Time’ Insights; Example is Connect Insights
17. Evolution of the Learning Analytics Platform
Elastic
Load
Balancing
Node.js
Cluster
MongoDB Connect Insights
For
Students
Amazon Route 53
LAP 1.0: Cluster of Node.js servers writing aggregations to MongoDB.
18. Evolution of the Learning Analytics Platform
Elastic
Load
Balancing
Amazon Route 53
Node.js
Cluster
MongoDB Connect Insights
For
Students
LAP 1.1: Put a queue in the middle.
Amazon
SQS
19. Evolution of the Learning Analytics Platform
Amazon
Elastic
Load
Balancing
Amazon Route 53
Node.js
Cluster MongoDB Connect Insights
For
Students
Amazon
SQS
LAP 1.2: Added S3 to pre-aggregate events and then load into MongoDB.
Amazon
S3
20. Evolution of the Learning Analytics Platform
Elastic
Load
Balancing
Amazon Route 53
Node.js
Cluster
Connect Insights
For
Students
Amazon
SQS
LAP 1.3: Replaced MongoDB with DynamoDB.
Amazon
S3
Amazon
DynamoDB
21. Evolution of the Learning Analytics Platform
Elastic
Load
Balancing
Amazon Route 53
Node.js
Cluster
Connect Insights
For
Students
Amazon
SQS
LAP 1.4 & LAP 1.5: Stabilized and fixed bugs.
Amazon
S3
Amazon
DynamoDB
27. Learning Analytics Platform - Services
Amazon DynamoDB
Amazon Relational
Database Service (RDS)
Amazon Simple
Storage Service (S3)
Amazon API Gateway
AWS Lambda
Amazon Kinesis
Streams
Amazon Elasticsearch
Service
28. Amazon API Gateway
Fully managed service for hosting
HTTPS APIs on top of AWS
• Support for standard HTTP methods
• Authenticate and authorize requests
• Highly scalable parallel processing
• DDoS protection and throttling for back-
end systems
• Support for standard HTTP methods
• Swagger Import/Export
• Custom domains
29. Benefits of Amazon API Gateway
Create a unified
API frontend for
multiple micro-
services
Authenticate and
authorize
requests to a
backend
DDoS protection
and throttling for
your backend
Throttle, meter,
and monetize API
usage by 3rd
party developers
30. API Gateway integrations
Internet
Mobile Apps
Websites
Services
AWS Lambda
functions
AWS
API Gateway
Cache
Endpoints on
Amazon EC2
All publicly
accessible
endpoints
Amazon
CloudWatch
Monitoring
Amazon
CloudFront
Any other
AWS service
31. AWS Lambda
• Runs your function code without you
managing or scaling servers
• Provides an API to trigger the execution
of your function
• Ensures function is executed when
triggered, in parallel, regardless of scale
• Provides additional capabilities for your
function (logging, monitoring).
Serverless, event-driven compute service
32. High performance at any scale;
Cost-effective and efficient
No Infrastructure to manage
Pay only for what you use: Lambda
automatically matches capacity to
your request rate. Purchase
compute in 100ms increments.
Bring Your Own Code
Lambda functions: Stateless, trigger-based code execution
Run code in a choice of standard
languages. Use threads, processes,
files, and shell scripts normally.
Focus on business logic, not
infrastructure. You upload code; AWS
Lambda handles everything else.
AWS Lambda Overview
33. How Lambda works
S3 event
notifications
DynamoDB
Streams
Amazon
Kinesis
events
Cognito
events
SNS
events
Custom
events
CloudTrail
events LambdaDynamoDB
Amazon
Kinesis S3
Any custom
Amazon
Redshift
SNS
Any AWS
34. Amazon Kinesis Streams
Fully managed service for real-time
processing of high-volume, streaming data
• Processes data in real-time
• Highly scalable parallel processing
• Open source libraries for sending data to
and reading data from a stream
• Synchronously replicates your data across
3 facilities
• Integrated with many AWS & third party
technologies
• Supports SSL and automatic encryption of
data once it is uploaded
35. Amazon Kinesis Streams
Easy administration: Simply create a new stream and set the desired level of capacity
with shards. Scale to match your data throughput rate and volume.
Build real-time applications: Perform continual processing on streaming big data using
Amazon Kinesis Client Library (KCL), Apache Spark/Storm, AWS Lambda, and more.
Low cost: Cost-efficient for workloads of any scale.
37. Elasticsearch
A powerful, real-time, distributed, open-source search and
analytics engine:
• Built on top of Apache Lucene
• Schema-free
• Developer-friendly RESTful API
38. Amazon Elasticsearch Service
Managed service that makes it easy to set
up, operate, and scale Elasticsearch
clusters in the cloud
• Built-in Kibana and Logstash plugin
• Modify clusters with no downtime
• Integrated with many AWS services like
CloudWatch Logs, Lambda, DynamoDB,
etc.
• Supports the ES API and is a drop in
replacement for your existing Elasticsearch
clusters
• Only pay for what you use
40. Integration with the AWS ecosystem
Arrow direction indicates general direction of data flow
EC2 instances
Logstash
cluster on EC2
VPC
Flow Logs
CloudTrail
Audit Logs
S3
Access
Logs
ELB
Access
Logs
CloudFront
Access
Logs
SNS
Notifications
DynamoDB
Streams
SES
Inbound
Email
Cognito
Events
Kinesis
Streams
CloudWatch
Events &
Alarms
Config
Rules
41. Amazon Simple Storage Service (S3)
Secure, durable, low cost, highly-scalable
object storage
• Easy to scale
• Designed for 99.999999999% durability and up
to 99.99% availability of objects over a given
year
• Cost-effective, pay only for the storage you
actually use
• Lifecycle policies can move objects to long term
storage or lower cost S3 Standard-IA
• Integrated with many AWS & third party
technologies
• Supports SSL and automatic encryption of data
once it is uploaded
42. Amazon Simple Storage Service (S3) for Big Data
• Scalable
• Virtually unlimited number of objects
• Very high bandwidth – no aggregate throughput limit
• Cost-Effective
• No need to run compute clusters for storage (unlike HDFS)
• Can run transient Hadoop clusters & Amazon EC2 Spot Instances
• Tiered storage (Standard, IA, Amazon Glacier) via life-cycle policy
• Flexible Access
• Direct access by big data frameworks (Spark, Hive, Presto)
• Shared access: Multiple (Spark, Hive, Presto) clusters can use the same data
43. Amazon Relational Database Service
Fully managed relational database service
• Simple and fast to deploy
• Fully managed = low admin
• Fast, predictable performance
• Easy to scale
• Cost-effective
• Open source engines: MySQL,
PostgreSQL, MariaDB
• Commercial engines: Oracle, SQLServer
• MySQL compatible engine: Aurora
Amazon
RDS
44. RDS PostgreSQL
Amazon RDS makes it easy to set up, operate, and scale PostgreSQL deployments in the
cloud. With Amazon RDS, you can deploy scalable PostgreSQL databases in minutes with
cost-efficient and resizable hardware capacity.
Key Features
Read Replicas (Same region and cross region)
High Availability with Multi-AZ
VPC and private subnet groups
Geospatial capabilities
Syntactically similar to Oracle
Amazon
RDS
45. Amazon DynamoDB
Non-Relational Managed NoSQL Database Service
• Schemaless data model
• Consistent, low-latency performance (single digit ms)
• Predictable provisioned throughput
• Seamless scalability
• Practically no storage limits
• High durability and availability (replication between 3
facilities)
• Easy administration – we scale for you!
• Low cost
• Cost modelling on throughput and size
DynamoDB
46. Amazon DynamoDB Scalability
• Virtually no limit in throughput
(reads/writes per second)
• Virtually no limit in storage
• DynamoDB automatically partitions
data
• Auto-partitioning occurs when:
• Data set growth
• Provisioned capacity increase
partitions
1 .. N
62. Architecture Tradeoffs (Amazon API Gateway)
• Compare with implementing code for HTTP/HTTPS
• Pro
• API Gateway is highly integrated service out of the box
• Automatically scales
• Handles thousands of concurrent calls
• Traffic management, authorization and access control,
monitoring, and API version management
• Con
• Does not currently support GZIP compression. Workaround is
to set up CloudFront server and enable compression
63. Architecture Tradeoffs (AWS Lambda)
• Compare with provisioning and managing EC2
instances
• Pro:
• No servers and instances to manage,
• Built-in automatic scaling
• Fixed cost model
• Don’t need a team of 7 DevOps resources to manage
• Con:
• Limited experience with Lambda
• Limited to CloudWatch and 6 MB data
• Debugging logs is time-consuming
64. Architectural Tradeoffs (Amazon Kinesis Streams)
• Compare to Kafka and Zookeeper
• Pro
• Able to process high-volume, streaming data
• 15 million records at peak load and growing
• Maintained and don’t have to predict storage
and volume
• Managed service with cross Availability Zone
replication
• Con
• May lose records depending on max
configuration setting (24 hours to 7 days)
65. Architectural Tradeoffs (Amazon S3)
• Compare with Rackspace CloudFiles and
OpenStack Swift
• Pro
• Secure, durable, highly-scalable cloud archive
• Managed service
• Easy to use, inexpensive, multiple means of
security content, backup of content, high availability
• Con
• SSL mismatch errors if you want to use own
domain name as domain name is
(bucketname).(region).amazonaws.com
66. Architectural Tradeoffs (Amazon ES)
• Compare with Lucene and Elasticsearch
company
• Pro
• Managed service that provides out of box
integrations with Amazon Kinesis and S3
• Use as data lake for learning events
• Con
• Release behind Elasticsearch so may not
have needed feature
67. Architectural Tradeoffs (Amazon DynamoDB)
• Compare to MongoDB
• Pro
• Experience
• Low cost
• Fast and flexible NoSQL data store
• Fully managed
• Con
• Limit – 400 KB row size, 1 MB queries
• Size is multiples of 4 KB for reads
68. Architectural Tradeoffs (RDS PostgreSQL)
• Compare to RDS Aurora and Amazon Redshift
• Pro
• Scales to 6 TB – within our immediate needs
• More concurrent connections than Amazon
Redshift
• Has full analytical engine
• Con
• Data volumes in future – address using archiving
and other strategies to reduce volume
69. AWS Estimated Cost Savings of 1 Billion Events
• Amazon Lambda
biggest cost saver
• Pay for what we use
• Auto-scales
• Additional capabilities
(logging, monitoring)
• Fewer DevOps
resources
• Gains in Agility
AWS Service Original
Cost
Estimated
Cost
Estimated
Savings
Amazon API
Gateway $4,319 $4,319 $0
AWS Lambda $0 $5,000 -$5,000
Amazon
Elasticsearch
Service $11,000 $11,000 $0
Amazon Kinesis $9,232 $9,232 $0
Amazon EC2 $410,000 $100,000 $310,000
Total $434,551 $129,551 $305,000
71. Challenges
• Lost events
• Elasticsearch performance
• Events to fail indexing in Elasticsearch
• Be aware of Elasticsearch limits
• Amazon Kinesis stream retention from 24 hours to 7 days
• DynamoDB hot spots
72. Challenges
• Lost events
• Elasticsearch performance
• Events to fail indexing in Elasticsearch
• Be aware of Elasticsearch limits
• Amazon Kinesis stream retention from 24 hours to 7 days
• DynamoDB hot spots
74. How We Built Confidence
• Built confidence thru robust testing strategy –
performance, failover, functional, and business
acceptance testing
75. How We Built Confidence (cont.)
• Caliper events sent to input API, validate persistence to
Elasticsearch and S3
• Tools used for testing
• Reconcile API – playback
• Monitoring of components
• Service Level Agreements established
76. Lessons Learned
• General
• Serverless framework
• CloudWatch and Sumologic
• Automated tests – about 80% coverage
• Custom dashboards
• Amazon API Gateway
• SigV4
77. Lessons Learned (cont.)
• AWS Lambda
• Cold start
• Great integration
• Sensitivity to EC2 and Auto Scaling issues and outages
• Need better debug tools
• Amazon Kinesis Streams
• Scaling up and down shards
• No purge functionality
78. Lessons Learned (cont.)
• Amazon Elasticsearch Service
• Scripts
• Queue capacity limits
• Performance tuning and monitoring
• AWS Enterprise Support
79. Summary of Actionable Takeaways
• Think about production scale
• Estimated costs including DevOps, engineers, architects
• Take time and resources to design the right architecture
– ilities – resiliency, redundancy, security, disaster
recovery, reliability, maintainability
• Amazon Enterprise Support
80. Summary of Actionable Takeaways
• Think about production scale
• Estimated costs including DevOps, engineers, architects
• Take time and resources to design the right architecture
– ilities – resiliency, redundancy, security, disaster
recovery, reliability, maintainability
• Amazon Enterprise Support