Data lakes allow organizations to store all types of data in a centralized repository at scale. AWS Lake Formation makes it easy to build secure data lakes by automatically registering and cleaning data, enforcing access permissions, and enabling analytics. Data stored in data lakes can be analyzed using services like Amazon Athena, Redshift, and EMR depending on the type of analysis and latency required.
Modern Data Architectures for Business Insights at Scale Amazon Web Services
Using AWS to design and build your data architecture has never been easier to gain insights and uncover new opportunities to scale and grow your business. Join this workshop to learn how you can gain insights at scale with the right big data applications.
Building a real-time analytics solution has never been faster or more cost-efficient. Most organizations are trying to find a way to improve customer experience and respond to business events in real time. Importantly, to do this quickly and at a fraction of the price of traditional approaches. In this session we will look at how to use the AWS services to best meet your real-time analytics needs.
Deep Dive and Best Practices for Real Time Streaming ApplicationsAmazon Web Services
Get answers to technical questions, frequently asked by those starting to work with streaming data. Learn best practices for building a real-time streaming data architecture on AWS with Amazon Kinesis, Spark Streaming, AWS Lambda, and Amazon EMR. First, we will focus on building a scalable, durable streaming data ingestion workflow from data producers like mobile devices, servers, or even web browsers. We will provide guidelines to minimize duplicates and achieve exactly-once processing semantics in your stream-processing applications. Then, we will show some of the proven architectures for processing streaming data using a combination of tools including Amazon Kinesis Stream, AWS Lambda, and Spark Streaming running on Amazon EMR.
Modern Data Architectures for Business Insights at Scale Amazon Web Services
Using AWS to design and build your data architecture has never been easier to gain insights and uncover new opportunities to scale and grow your business. Join this workshop to learn how you can gain insights at scale with the right big data applications.
Building a real-time analytics solution has never been faster or more cost-efficient. Most organizations are trying to find a way to improve customer experience and respond to business events in real time. Importantly, to do this quickly and at a fraction of the price of traditional approaches. In this session we will look at how to use the AWS services to best meet your real-time analytics needs.
Deep Dive and Best Practices for Real Time Streaming ApplicationsAmazon Web Services
Get answers to technical questions, frequently asked by those starting to work with streaming data. Learn best practices for building a real-time streaming data architecture on AWS with Amazon Kinesis, Spark Streaming, AWS Lambda, and Amazon EMR. First, we will focus on building a scalable, durable streaming data ingestion workflow from data producers like mobile devices, servers, or even web browsers. We will provide guidelines to minimize duplicates and achieve exactly-once processing semantics in your stream-processing applications. Then, we will show some of the proven architectures for processing streaming data using a combination of tools including Amazon Kinesis Stream, AWS Lambda, and Spark Streaming running on Amazon EMR.
AWS re:Invent 2016: What’s New with Amazon Redshift (BDA304)Amazon Web Services
In this session, you learn about the latest and hottest features of Amazon Redshift. Join Vidhya Srinivasan, General Manager of Amazon Redshift, to take a deep dive into the architecture and inner workings of Amazon Redshift. You discover how the recent availability, performance, and manageability improvements we’ve made can significantly enhance your end user experience. You also get a glimpse of what we are working on and our plans for the future.
NEW LAUNCH! Intro to Amazon Athena. Easily analyze data in S3, using SQL.Amazon Web Services
Amazon Athena is a new interactive query service that makes it easy to analyze data in Amazon S3, using standard SQL. Athena is serverless, so there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3.
In this session, we will show you how easy is to start querying your data stored in Amazon S3, with Amazon Athena. First we will use Athena to create the schema for data already in S3. Then, we will demonstrate how you can run interactive queries through the built-in query editor. We will provide best practices and use cases for Athena. Then, we will talk about supported queries, data formats, and strategies to save costs when querying data with Athena.
In this session, we will introduce Amazon RedShift, a new petabyte scale data warehouse service. We'll walk through the basics of the Redshift architecture, launching a new cluster and run SQL queries across a large scale, public dataset. After demonstrating how easy it is to get started with RedShift, we will show how to visualize and query large scale datasets, running queries, reports, and analytics against millions of rows of records in just a few seconds.
Understand the core concepts of “Cloud Computing” and how businesses around the world are running the infrastructure that supports their websites to lower costs, improve time-to-market, and enable rapid scalability matching resource to demands of users. Whether you are an enterprise looking for IT innovation, agility and resiliency or small and medium business who wants to accelerate growth without a big upfront investment in cash or time for technology, the AWS Cloud provides a complete set of services at zero upfront costs which are available with a few clicks and within minutes.
AWS re:Invent 2016: Driving Innovation with Big Data and IoT (GPSST304)Amazon Web Services
With unforeseen competitive threats and potential market disruptions, enterprises are seeking to innovate for the benefit of their customers. Business transformation in the digital age requires the successful use of new technologies including the cloud, IoT, and Big Data. Attend this session to learn more about how AWS can help organizations innovate faster around IoT and Big Data. We dive into specific opportunities and approaches for managing billions of connected devices and associated big data workloads on the cloud.
Running Lean Architectures: How to Optimize for Cost Efficiency Amazon Web Services
Whether you’re a cash-strapped startup or an enterprise optimizing spend, it pays to run cost-efficient architectures on AWS. This session reviews a wide range of cost planning, monitoring, and optimization strategies, featuring real-world experience from AWS customers. We’ll cover how you can effectively combine EC2 On-Demand, Reserved, and Spot instances to handle different use cases, leveraging auto scaling to match capacity to workload, choosing the most optimal instance type through load testing, taking advantage of multi-AZ support, and using CloudWatch to monitor usage and automatically shut off resources when not in use. We'll discuss taking advantage of tiered storage and caching, offloading content to Amazon CloudFront to reduce back-end load, and getting rid of your back end entirely, by leveraging AWS high-level services. We will also showcase simple tools to help track and manage costs, including the AWS Cost Explorer, Billing Alerts, and Trusted Advisor. This session will be your pocket guide for running cost effectively in the Amazon cloud.
Learning Objectives:
- Learn AWS architectural best practices
- Measure your architecture against pillars of security, reliability, cost, performance, and operations
- Build a plan to remediate and improve your architecture
Automatisierte Kontrolle und Transparenz in der AWS Cloud – Autopilot für Com...AWS Germany
Vortrag "Automatisierte Kontrolle und Transparenz in der AWS Cloud – Autopilot für Compliance Ihrer Cloud Ressourcen" von Philipp Behre beim AWS Cloud Web Day für Mittelstand und Großunternehmen. Alle Videos und Präsentationen finden Sie hier: http://amzn.to/1VUJZsT
In this session we will bring some clarity to the increasingly complex big data landscape and look at the common patterns for the ingest, storage, processing, and analysis of different types of data on the AWS platform.
Speaker: Russell Nash, Solutions Architect, Amazon Web Services
Featured Customer - TechnologyOne
AWS provides a broad platform of managed services to help you build, secure, and seamlessly scale end-to-end Big Data applications quickly and with ease. Want to get ramped up on how to use Amazon's big data web services? Learn when to use which service? Want to write your first big data application on AWS? Join us in this session as we discuss reference architecture, design patterns, and best practices for pulling together various AWS services to meet your big data challenges.
AWS re:Invent 2016: Deliver Engaging Experiences with Custom Apps Built on Sa...Amazon Web Services
Your developers are the most important part of transforming your customer interactions into engaging experiences. Salesforce App Cloud, which brings together Heroku, Force.com and Lightning, abstracts away infrastructure and devops complexity, so you can focus on what matters most: building differentiated experiences through apps. Reducing time to market and letting you iterate fast helps you rise above the competition and build lasting customer relationships. In this session, you hear from Zayo, a leading global communications infrastructure services provider, and how they are leveraging the power of integrating the Salesforce and AWS platforms to deliver highly engaging customer experiences, enhancing developer productivity and driving faster innovation cycles. We spotlight Heroku Connect, which makes it easy to extend and synchronize your customer data between Salesforce and AWS and enhance it in ways that empower your developers to do what they do best: innovate. Session sponsored by Salesforce.
Antoine Genereux takes us on a detailed overview of the Database solutions available on the AWS Cloud, addressing the needs and requirements of customers at all levels. He also discusses Business Intelligence and Analytics solutions.
Whether you’re a cash-strapped startup or an enterprise trying to optimizing spend, it pays to run cost-efficient architectures on AWS. Come learn about cost planning, monitoring, and optimization strategies, featuring real AWS customer use cases.
Organizations where cloud adoption has matured into broader enterprise deployment are facing the need to better manage and control their costs and expenditures. Cost optimization at scale is a process that involves a number of changes across the business, including technical, organizational and cultural transformation. In this session, you will learn the fundamentals of cost optimization and how this can be used to help your organization drive costs down and still being able to meet capacity, demand and organizational requirements. Key topics being discussed are right sizing services, optimizing purchase models and implementing a culture of cost management.
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
A data lake can be used as a source for both structured and unstructured data - but how? We'll look at using open standards including Spark and Presto with Amazon EMR, Amazon Redshift Spectrum and Amazon Athena to process and understand data.
Level: Intermediate
Speakers:
Tony Nguyen - Senior Consultant, ProServe, AWS
Hannah Marlowe - Consultant - Federal, AWS
AWS re:Invent 2016: What’s New with Amazon Redshift (BDA304)Amazon Web Services
In this session, you learn about the latest and hottest features of Amazon Redshift. Join Vidhya Srinivasan, General Manager of Amazon Redshift, to take a deep dive into the architecture and inner workings of Amazon Redshift. You discover how the recent availability, performance, and manageability improvements we’ve made can significantly enhance your end user experience. You also get a glimpse of what we are working on and our plans for the future.
NEW LAUNCH! Intro to Amazon Athena. Easily analyze data in S3, using SQL.Amazon Web Services
Amazon Athena is a new interactive query service that makes it easy to analyze data in Amazon S3, using standard SQL. Athena is serverless, so there is no infrastructure to setup or manage, and you can start analyzing your data immediately. You don’t even need to load your data into Athena, it works directly with data stored in S3.
In this session, we will show you how easy is to start querying your data stored in Amazon S3, with Amazon Athena. First we will use Athena to create the schema for data already in S3. Then, we will demonstrate how you can run interactive queries through the built-in query editor. We will provide best practices and use cases for Athena. Then, we will talk about supported queries, data formats, and strategies to save costs when querying data with Athena.
In this session, we will introduce Amazon RedShift, a new petabyte scale data warehouse service. We'll walk through the basics of the Redshift architecture, launching a new cluster and run SQL queries across a large scale, public dataset. After demonstrating how easy it is to get started with RedShift, we will show how to visualize and query large scale datasets, running queries, reports, and analytics against millions of rows of records in just a few seconds.
Understand the core concepts of “Cloud Computing” and how businesses around the world are running the infrastructure that supports their websites to lower costs, improve time-to-market, and enable rapid scalability matching resource to demands of users. Whether you are an enterprise looking for IT innovation, agility and resiliency or small and medium business who wants to accelerate growth without a big upfront investment in cash or time for technology, the AWS Cloud provides a complete set of services at zero upfront costs which are available with a few clicks and within minutes.
AWS re:Invent 2016: Driving Innovation with Big Data and IoT (GPSST304)Amazon Web Services
With unforeseen competitive threats and potential market disruptions, enterprises are seeking to innovate for the benefit of their customers. Business transformation in the digital age requires the successful use of new technologies including the cloud, IoT, and Big Data. Attend this session to learn more about how AWS can help organizations innovate faster around IoT and Big Data. We dive into specific opportunities and approaches for managing billions of connected devices and associated big data workloads on the cloud.
Running Lean Architectures: How to Optimize for Cost Efficiency Amazon Web Services
Whether you’re a cash-strapped startup or an enterprise optimizing spend, it pays to run cost-efficient architectures on AWS. This session reviews a wide range of cost planning, monitoring, and optimization strategies, featuring real-world experience from AWS customers. We’ll cover how you can effectively combine EC2 On-Demand, Reserved, and Spot instances to handle different use cases, leveraging auto scaling to match capacity to workload, choosing the most optimal instance type through load testing, taking advantage of multi-AZ support, and using CloudWatch to monitor usage and automatically shut off resources when not in use. We'll discuss taking advantage of tiered storage and caching, offloading content to Amazon CloudFront to reduce back-end load, and getting rid of your back end entirely, by leveraging AWS high-level services. We will also showcase simple tools to help track and manage costs, including the AWS Cost Explorer, Billing Alerts, and Trusted Advisor. This session will be your pocket guide for running cost effectively in the Amazon cloud.
Learning Objectives:
- Learn AWS architectural best practices
- Measure your architecture against pillars of security, reliability, cost, performance, and operations
- Build a plan to remediate and improve your architecture
Automatisierte Kontrolle und Transparenz in der AWS Cloud – Autopilot für Com...AWS Germany
Vortrag "Automatisierte Kontrolle und Transparenz in der AWS Cloud – Autopilot für Compliance Ihrer Cloud Ressourcen" von Philipp Behre beim AWS Cloud Web Day für Mittelstand und Großunternehmen. Alle Videos und Präsentationen finden Sie hier: http://amzn.to/1VUJZsT
In this session we will bring some clarity to the increasingly complex big data landscape and look at the common patterns for the ingest, storage, processing, and analysis of different types of data on the AWS platform.
Speaker: Russell Nash, Solutions Architect, Amazon Web Services
Featured Customer - TechnologyOne
AWS provides a broad platform of managed services to help you build, secure, and seamlessly scale end-to-end Big Data applications quickly and with ease. Want to get ramped up on how to use Amazon's big data web services? Learn when to use which service? Want to write your first big data application on AWS? Join us in this session as we discuss reference architecture, design patterns, and best practices for pulling together various AWS services to meet your big data challenges.
AWS re:Invent 2016: Deliver Engaging Experiences with Custom Apps Built on Sa...Amazon Web Services
Your developers are the most important part of transforming your customer interactions into engaging experiences. Salesforce App Cloud, which brings together Heroku, Force.com and Lightning, abstracts away infrastructure and devops complexity, so you can focus on what matters most: building differentiated experiences through apps. Reducing time to market and letting you iterate fast helps you rise above the competition and build lasting customer relationships. In this session, you hear from Zayo, a leading global communications infrastructure services provider, and how they are leveraging the power of integrating the Salesforce and AWS platforms to deliver highly engaging customer experiences, enhancing developer productivity and driving faster innovation cycles. We spotlight Heroku Connect, which makes it easy to extend and synchronize your customer data between Salesforce and AWS and enhance it in ways that empower your developers to do what they do best: innovate. Session sponsored by Salesforce.
Antoine Genereux takes us on a detailed overview of the Database solutions available on the AWS Cloud, addressing the needs and requirements of customers at all levels. He also discusses Business Intelligence and Analytics solutions.
Whether you’re a cash-strapped startup or an enterprise trying to optimizing spend, it pays to run cost-efficient architectures on AWS. Come learn about cost planning, monitoring, and optimization strategies, featuring real AWS customer use cases.
Organizations where cloud adoption has matured into broader enterprise deployment are facing the need to better manage and control their costs and expenditures. Cost optimization at scale is a process that involves a number of changes across the business, including technical, organizational and cultural transformation. In this session, you will learn the fundamentals of cost optimization and how this can be used to help your organization drive costs down and still being able to meet capacity, demand and organizational requirements. Key topics being discussed are right sizing services, optimizing purchase models and implementing a culture of cost management.
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
A data lake can be used as a source for both structured and unstructured data - but how? We'll look at using open standards including Spark and Presto with Amazon EMR, Amazon Redshift Spectrum and Amazon Athena to process and understand data.
Level: Intermediate
Speakers:
Tony Nguyen - Senior Consultant, ProServe, AWS
Hannah Marlowe - Consultant - Federal, AWS
Data Analytics Week at the San Francisco Loft
Using Data Lakes
A data lake can be used as a source for both structured and unstructured data - but how? We'll look at using open standards including Spark and Presto with Amazon EMR, Amazon Redshift Spectrum and Amazon Athena to process and understand data.
Speakers:
John Mallory - Principal Business Development Manager Storage (Object), AWS
Hemant Borole - Sr. Big Data Consultant, AWS
This overview presentation discusses big data challenges and provides an overview of the AWS Big Data Platform by covering:
- How AWS customers leverage the platform to manage massive volumes of data from a variety of sources while containing costs.
- Reference architectures for popular use cases, including, connected devices (IoT), log streaming, real-time intelligence, and analytics.
- The AWS big data portfolio of services, including, Amazon S3, Kinesis, DynamoDB, Elastic MapReduce (EMR), and Redshift.
- The latest relational database engine, Amazon Aurora— a MySQL-compatible, highly-available relational database engine, which provides up to five times better performance than MySQL at one-tenth the cost of a commercial database.
Created by: Rahul Pathak,
Sr. Manager of Software Development
Amazon Web Services proporciona una amplia gama de servicios que le ayudarán a crear e implementar aplicaciones de análisis de big data de forma rápida y sencilla. AWS ofrece un acceso rápido a recursos de TI económicos y flexibles, algo que permitirá escalar prácticamente cualquier aplicación de big data con rapidez, incluidos almacenamiento de datos, análisis de clics, detección de elementos fraudulentos, motores de recomendación, proceso ETL impulsado por eventos, informática sin servidor y procesamiento del Internet de las cosas. Con AWS no necesita hacer grandes inversiones iniciales de tiempo o dinero para crear y mantener la infraestructura. En su lugar, puede aprovisionar exactamente el tipo y el tamaño adecuado de los recursos que necesita para impulsar sus aplicaciones de análisis de big data. Puede obtener acceso a tantos recursos como necesite, prácticamente al instante, y pagar únicamente por los utilice.
A data lake can be used as a source for both structured and unstructured data - but how? We'll look at using open standards including Spark and Presto with Amazon EMR, Amazon Redshift Spectrum and Amazon Athena to process and understand data.
Speakers:
Neel Mitra - Solutions Architect, AWS
Roger Dahlstrom - Solutions Architect, AWS
FSI201 FINRA’s Managed Data Lake – Next Gen Analytics in the CloudAmazon Web Services
FINRA’s Data Lake unlocks the value in its data to accelerate analytics and machine learning at scale. FINRA's Technology group has changed its customer's relationship with data by creating a Managed Data Lake that enables discovery on Petabytes of capital markets data, while saving time and money over traditional analytics solutions. FINRA’s Managed Data Lake includes a centralized data catalog and separates storage from compute, allowing users to query from petabytes of data in seconds. Learn how FINRA uses Spot instances and services such as Amazon S3, Amazon EMR, Amazon Redshift, and AWS Lambda to provide the 'right tool for the right job' at each step in the data processing pipeline. All of this is done while meeting FINRA’s security and compliance responsibilities as a financial regulator.
Data Con LA 2020
Description
In this session, I introduce the Amazon Redshift lake house architecture which enables you to query data across your data warehouse, data lake, and operational databases to gain faster and deeper insights. With a lake house architecture, you can store data in open file formats in your Amazon S3 data lake.
Speaker
Antje Barth, Amazon Web Services, Sr. Developer Advocate, AI and Machine Learning
In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes, and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.
Data warehousing is a critical component for analysing and extracting actionable insights from your data. Amazon Redshift allows you to deploy a scalable data warehouse in a matter minutes and start to analyse your data right away using your existing business intelligence tools. It’s a fast, fully-managed, and cost-effective data warehousing system. You can analyse all your data using standard SQL and your existing Business Intelligence (BI) tools. Amazon Redshift also includes Redshift Spectrum, allowing you to directly run SQL queries against exabytes of unstructured data in Amazon S3. In this, you will learn how to migrate from existing data warehouses, optimise schemas and load data efficiently. We will also cover analytics tools to help you build visualisations, perform ad-hoc analysis and quickly get business insights from your data.
Learning Objectives:
• Discover best practices for building a data warehouse using Amazon Redshift
• Learn to use Amazon QuickSight for Business Intelligence and AWS Glue for ETL.
From Data Collection to Actionable Insights in 60 Seconds: AWS Developer Work...Amazon Web Services
From Data Collection to Actionable Insights in 60 Seconds: AWS Developer Workshop - Web Summit 2018
Columnar data formats such as Parquet and ORC are designed to optimize both query performance and costs for analytics scenarios. On the other hand, serverless computing platforms such as AWS Lambda allow you to run highly scalable applications without provisioning or managing servers. The combination of columnar storage and serverless computing can drastically simplify many of the pain points related to big data analytics, data collection, data exploration, and ETL orchestration, while at the same time reducing the total cost of ownership.
Speaker: Alex Casalboni - Technical Evangelist, AWS
AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)Amazon Web Services
For discovery-phase research, life sciences companies have to support infrastructure that processes millions to billions of transactions. The advent of a data lake to accomplish such a task is showing itself to be a stable and productive data platform pattern to meet the goal. We discuss how to build a data lake on AWS, using services and techniques such as AWS CloudFormation, Amazon EC2, Amazon S3, IAM, and AWS Lambda. We also review a reference architecture from Amgen that uses a data lake to aid in their Life Science Research.
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Amazon Web Services
The world is creating more data in more ways than ever before. The average internet user in 2017 generates 1.5GB of data per day, with the rate doubling every 18 months. A single autonomous vehicle can generate 4TB per day. Each smart manufacturing plant generates 1PB per day. Storing, managing, and analyzing this data requires integrated database and analytic services that provide reliability and security at scale. AWS offers a range of managed data services that let customers focus on making data useful, including Amazon Aurora, RDS, DynamoDB, Redshift, Spectrum, ElastiCache, Kinesis, EMR, Elasticsearch Service, and Glue. In this session, we discuss these services, share our vision for innovation, and show how our customers use these services today. Learn More: https://aws.amazon.com/government-education/
Understanding AWS Managed Database and Analytics Services | AWS Public Sector...Amazon Web Services
The world is creating more data in more ways than ever before. The average internet user in 2017 generates 1.5GB of data per day, with the rate doubling every 18 months. A single autonomous vehicle can generate 4TB per day. Each smart manufacturing plant generates 1PB per day. Storing, managing, and analyzing this data requires integrated database and analytic services that provide reliability and security at scale. AWS offers a range of managed data services that let customers focus on making data useful, including Amazon Aurora, RDS, DynamoDB, Redshift, Spectrum, ElastiCache, Kinesis, EMR, Elasticsearch Service, and Glue. In this session, we discuss these services, share our vision for innovation, and show how our customers use these services today. Learn More: https://aws.amazon.com/government-education/
Processing the volume and variety of data that today’s organizations produce can be both challenging and costly – especially with a legacy data warehouse. Combining the scale and performance of the cloud with AWS and APN Partner solutions for migration, integration, analysis, and visualization can help overcome these obstacles. With a modern data warehouse architecture, organizations can store, process, and analyze massive volumes of data of virtually any type. Register for this upcoming webinar, where Pearson - an education and media conglomerate - will share in detail how they built a scalable and flexible business intelligence platform on the cloud, with Tableau and AWS.
Learn how you can seamlessly load and transform data in Amazon Redshift with Matillion ETL and analyze it with Tableau. Hear how 47Lining and NorthBay can provide insights to guide you through migration with ease. Tableau will discuss best practices to analyze your data on AWS and share new insights throughout your organization.
En este webinar, aprenderá cómo las empresas pueden aprovechar la nube de AWS para automatizar los pipelines de desarrollo de software. Este enfoque permite que su equipo sea más ágil, mejorando su capacidad para entregar aplicaciones y servicios rápidamente.
Neste webinar, você aprenderá como as empresas podem se valer da nuvem da AWS para automatizar os pipelines de desenvolvimento de software. Essa abordagem permite que sua equipe seja mais ágil, melhorando sua capacidade para entregar aplicações e serviços mais rapidamente.
Las tecnologías como los contenedores y kubernetes pueden hacer que sus procesos de entrega de software sean más fáciles y más rápidos. En este webinar, hablaremos sobre cómo usar el Amazon Kubernetes Service (EKS) para construir aplicaciones modernas con grupos Kubernetes totalmente administrados.
Tecnologias como containers e Kubernetes podem tornar seus processos de entrega de software mais fáceis e rápidos. Neste webinar, falaremos sobre como usar o Amazon Elastic Kubernetes Service (EKS) para criar aplicativos modernos com clusters de Kubernetes totalmente gerenciados.
Ransomware é uma das ameaças de crescimento mais rápido para qualquer organização. Nenhuma empresa, grande ou pequena, está imune a ataques de cibercriminosos. Nesta sessão, mostramos como você pode aproveitar os serviços e recursos da nuvem AWS para proteger seus dados mais valiosos de ataques cibernéticos e acelerar a restauração de operações.
El ransomware es una de las amenazas de más rápido crecimiento para cualquier organización. Ninguna empresa, grande o pequeña, es inmune a los ataques de los ciberdelincuentes. En esta sesión, mostramos cómo puede aprovechar los servicios y las capacidades de la nube AWS para proteger sus datos más valiosos de los ataques cibernéticos y acelerar la restauración de las operaciones.
Ransomware é uma prática maliciosa que tem se popularizado nos últimos anos. Nessa sessão, mostraremos como através da Amazon Web Services nossos clientes podem desenvolver uma estratégia pró-ativa de mitigação a ataques de ransomware, tanto em cenários on-premises como operando na nuvem.
El ransomware es una práctica maliciosa que se ha popularizado en los últimos años. En esta sesión les mostraremos cómo desde Amazon Web Services nuestros clientes pueden desarrollar una estrategia proactiva de mitigación frente a ataques de ransomware, tanto en escenarios on-premises, como operando en la nube.
Al mover datos a la nube, los clientes deben comprender los métodos óptimos para los diferentes casos de uso, los tipos de datos que están moviendo y los recursos disponibles en la red, entre otros. Las soluciones de migración y transferencia de AWS contemplan desde la migración de datos con conectividad limitada, almacenamiento en la nube híbrida, transferencias frecuentes de archivos B2B, hasta transferencias de datos en línea y sin conexión. En esta sesión, le mostramos cómo puede acelerar la migración y transferencia de datos de manera simplificada desde y hacia la nube de AWS.
Ao mover dados para a nuvem, os clientes precisam entender os métodos ideais de movê-los para diferentes casos de uso, os tipos de dados que estão movendo e os recursos de rede disponíveis, entre outras considerações. As soluções de migração e transferência da AWS atendem desde a migração de dados com conectividade limitada, armazenamento em nuvem híbrida, transferências frequentes de arquivos B2B até transferências de dados online e offline. Nessa sessão, mostraremos como você pode simplificar e acelerar sua migração e transferência de dados de e para a nuvem AWS.
El almacenamiento de archivos tiene diversos casos de uso; como directorios de usuarios, datos de aplicaciones, archivos multimedia y almacenamiento compartido para cargas de trabajo de alto rendimiento. La administración del almacenamiento de archivos en instalaciones propias suele ser un trabajo pesado, indiferenciado, con altos costos de adquisición, carga operativa para configurar y administra, lo que conlleva a desafíos de escalabilidad. En esta sesión, le mostramos cómo puede aprovechar las soluciones de archivos totalmente administradas de AWS para dejar de preocuparse por la sobrecarga administrativa de configurar, proteger, mantener y realizar copias de seguridad de su infraestructura de archivos.
La visualización de datos analíticos es un reto al que se enfrentan muchas organizaciones, el poder crear tableros, alertas, agregar predicciones a sus datos y actuar de acuerdo a estas de manera rápida es una necesidad de todos los negocios actuales. Únase a nuestros arquitectos para aprender como Amazon QuickSight le permite agregar inteligencia de negocios a sus aplicaciones y crear predicciones a futuro de sus datos. Amazon QuickSight es un servicio de inteligencia de negocios escalable y serverless creado para la nube, a través del cual podrá explotar sus datos de negocio para convertirlos en insights para hacer decisiones informadas sobre su negocio sin preocuparse de la gestión, escalamiento y la disponibilidad de la infraestructura de cómputo.
A visualização de dados é um desafio que muitas organizações enfrentam hoje. Criar dashboards, alertas, fazer previsões e agir rapidamente de acordo com os insights dos dados é uma necessidade de todas as empresas. Junte-se aos nossos arquitetos para aprender como o Amazon QuickSight o ajudará a adicionar BI aos seus aplicativos. O Amazon Quicksight é um serviço de BI escalável e serverless criado para a nuvem. Com ele, você pode explorar seus dados para obter insights e tomar decisões embasadas em seus negócios, sem se preocupar em gerenciar e dimensionar servidores e manter a disponibilidade de sua infraestrutura.
Executar projetos de Big Data nunca foi tão simples. Com a AWS, você pode executar Hadoop, Spark, Hive, Flink e frameworks semelhantes de maneira mais rápida e econômica. Neste webinar, você aprenderá como melhorar o desempenho do processamento de seus dados e reduzir custos, especialmente quando comparado a um ambiente on-premises.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
7. Data
every 5 years
There is more data
than people think
15
years
live for
Data platforms need to
1,000x
scale
>10x
grows
8. There are more
people accessing data
And more
requirements for
making data available
Data Scientists
Analysts
Business Users
Applications
Secure Real time
Flexible Scalable
14. Analytics used to look like this
OLTP ERP CRM LOB
Data warehouse
Business intelligence
Relational data
TBs–PBs scale
Schema defined prior to data load
Operational reporting and ad hoc
Large initial CAPEX + $10K $50K/TB/Year
15. A data lake is a centralized repository that
allows you to store all your structured and
unstructured data at any scale
16. Why data lakes?
Data Lakes provide:
Relational and non-relational data
Scale-out to EBs
Diverse set of analytics and machine learning tools
Work on data without any data movement
Designed for low cost storage and analytics
OLTP ERP CRM LOB
Data Warehouse
Business
Intelligence
Data Lake
1001100001001010111001010
1011100101010000101111101
1010
0011110010110010110
0100011000010
Devices Web Sensors Social
Catalog
Machine
Learning
DW Queries Big data
processing
Interactive Real-time
17. • OLTP (Online Transaction Processing)
Characterized by a large number of short transactions (INSERT,
UPDATE, DELETE) that serve as persistence layer for applications.
e.g. Aurora, MySQL, PostgreSQL, etc. Typically a row-store
architecture
• OLAP (Online Analytical Processing)
Characterized by relatively low volume of transactions, and
queries are often complex and involve aggregations against large
historical datasets for data-driven decision making. e.g. Amazon
Redshift, Greenplum, etc. Typically a column-store architecture
• Data Lake
An architectural paradigm that allows customers to store all of
their data in a single unified place where they can collect and
store any data, at any scale, and at low cost. Data lakes
complement (not replace) other data stores such as data
warehouses. e.g. S3 data lake
OLTP
PostgreSQL
Amazon
Aurora
Amazon EC2
(Business Application)User
Applications
DataLake
Data Stores: What’s the Difference?
OLAP
ETL Tools
Amazon
QuickSight
Amazon Redshift
Amazon
Glue
BI Tools
OLTP ERP CRM LOBUser
Dashboards
18. Amazon S3 | AWS Glue
Any analytic
workload, any scale,
at the lowest possible
cost
AWS Direct Connect
AWS Snowball
AWS Snowmobile
AWS Database Migration Service
AWS IoT Core
Amazon Kinesis Data Firehose
Amazon Kinesis Data Streams
Amazon Kinesis Video Streams
On-premises
Data Movement
Amazon SageMaker
AWS Deep Learning AMIs
Amazon Rekognition
Amazon Lex
AWS DeepLens
Amazon Comprehend
Amazon Translate
Amazon Transcribe
Amazon Polly
Amazon Athena
Amazon EMR
Amazon Redshift
Amazon Elasticsearch Service
Amazon Kinesis
Amazon QuickSight
Analytics
Machine Learning
Real-time
Data Movement
Data Lake
19. There are lots of ingestion tools
Amazon S3
Process Consume
S3 Transfer
Acceleration
Data sources
Transactions
Web logs /
cookies
ERP
Connected
devices
20. Typical steps of building a data lake
Setup storage1
Move data2
Cleanse, prep, and
catalog data
3
Configure and enforce
security and compliance
policies
4
Make data available
for analytics
5
22. Data preparation accounts for ~80% of the work
Building training sets
Cleaning and organizing data
Collecting data sets
Mining data for patterns
Refining algorithms
Other
23. Sample of steps required
Find sources
Create Amazon Simple Storage Service (Amazon S3) locations
Configure access policies
Map tables to Amazon S3 locations
ETL jobs to load and clean data
Create metadata access policies
Configure access from analytics services
Rinse and repeat for other:
data sets, users, and end-services
And more:
manage and monitor ETL jobs
update metadata catalog as data changes
update policies across services as users and permissions change
manually maintain cleansing scripts
create audit processes for compliance
…
Manual | Error-prone | Time consuming
24. Enforce security policies
across multiple services
Gain and manage new
insights
Identify, ingest, clean,
and transform data
Build a secure data lake in days
AWS Lake Formation
26. Register existing data or import new
Amazon S3 forms the storage layer for
Lake Formation
Register existing S3 buckets that
contain your data
Ask Lake Formation to create required
S3 buckets and import data into them
Data is stored in your account. You have
direct access to it. No lock-in.
Data Lake Storage
Data
Catalog
Access
Control
Data import
Lake Formation
Crawlers ML-based
data prep
27. Easily load data to your data lake
logs
DBs
Blueprints
Data Lake Storage
Data
Catalog
Access
Control
Data import
Lake Formation
Crawlers ML-based
data prep
one-shot
incremental
28. With blueprints
You
1. Point us to the source
2. Tell us the location to load to
in your data lake
3. Specify how often you want to
load the data
Blueprints
1. Discover the source table(s)
schema
2. Automatically convert to the
target data format
3. Automatically partition the
data based on the
partitioning schema
4. Keep track of data that was
already processed
5. You can customize any of
the above
30. Secure once, access in multiple ways
Data Lake Storage
Data
Catalog
Access
Control
Lake Formation
Admin
31. Security permissions in Lake Formation
Control data access with simple
grant and revoke permissions
Specify permissions on tables and
columns rather than on buckets
and objects
Easily view policies granted to a
particular user
Audit all data access at one place
34. Serverless Query Processing
• Serverless query service for querying data in S3 using standard SQL with
no infrastructure to manage
• No data loading required; query directly from Amazon S3
• Use standard ANSI SQL queries with support for joins, JSON, and
window functions
• Support for multiple data formats include text, CSV, TSV, JSON, Avro,
ORC, Parquet
• Pay per query only when you’re running queries based on data scanned.
If you compress your data, you pay less and your queries run faster
Amazon
Athena
35. Querying it in Amazon Athena
Either Create a Crawler to
auto-generate schema
OR
Write a DDL on the Athena
console/API/ JDBC/ODBC
driver
Start Querying Data
36. Relational data warehouse
Massively parallel; Petabyte scale
Fully managed
HDD and SSD Platforms
$1,000/TB/Year; starts at $0.25/hour
Amazon
Redshift
a lot faster
a lot simpler
a lot cheaper
37. Amazon Redshift Speed: Three Highlights
Machine-learning based acceleration
1
2
Result-set Caching for Local & Data Lake Queries
(sub-second repeat
queries)
3
Constant improvements in performance for
real-world workloads
10x faster
than it was two years ago
Amazon Redshift is now
from over 200 features and enhancements
released due to lessons learnt from more than
10,000 customer deployments processing over 2 exabytes
of data every dayRedshift Spectrum caches
intermediate results that can
benefit different queries
38. The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical representation of Forrester's call on a
market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor, product, or service depicted in the Forrester Wave. Information is
based on best available resources. Opinions reflect judgment at the time and are subject to change.
The Forrester Wave™: Enterprise Data
Warehouse, Q4 2015
Forrester Wave™ Big Data Warehouse Q4 2018
AWS rated top in the
leader bracket and
received a score of
5/5 (the highest score
possible) in a number
of areas such as Use
Cases, Roadmap,
Market Awareness,
and Ability to Execute
AWS positioned as a
Leader in the Gartner
Magic Quadrant for
Data Management
Gartner Magic Quadrant, 2018
Solutions for
Analytics
39. Semi-structured/Unstructured Data Processing
• Hadoop, Hive, Presto, Spark, Tez, Impala etc.
• Release 5.2: Hadoop 2.7.3, Hive 2.1, Spark 2.02, Zeppelin, Presto, HBase 1.2.3 and HBase on
S3, Phoenix, Tez, Flink.
• New applications added within 30 days of their open source release
• Fully managed, Auto Scaling clusters with support for on-demand and
spot pricing
• Support for HDFS and S3 file systems enabling separated compute and
storage; multiple clusters can run against the same data in S3
• HIPAA-eligible. Support for end-to-end encryption, IAM/VPC, S3 client-
side encryption with customer managed keys and AWS KMS
Amazon EMR
44. Summary
• Data MUST be used in every organization
• Data lakes are very important to consume structured and
unstructured data
• Data lake governance
• Analyze data with the right tool
• Different type of consumers