Introdução ao AWS Data Pipeline

•

0 likes•549 views

This document introduces AWS Data Pipeline, which allows users to create workflows to move and transform data between different AWS services and on-premises systems. It provides examples of using Data Pipeline to move data between Amazon S3, DynamoDB, RDS, EMR and Redshift. It also discusses using Data Pipeline for ETL workflows and comparing the cost of running workflows on AWS versus on-premises systems.

Introdução ao AWS Data Pipeline:
ETL na Nuvem da AWS

José Papo
AWS Tech Evangelist
@josepapo

Amazon S3 Amazon
RDS

Amazon Amazon
DynamoDB Redshift

HDFS
On
(Amazon EMR)
Premise

Input Datanode

Activity

Output Datanode

Input Datanode com validação de pré-condição

Atividade com notificações de falha ou delay

Ouput Datanode

Custom Amazon RDS
Amazon S3 Amazon demographics
logs Precondition DynamoDB
event data
Hive
script
EMR usage-by-geo job

Amazon
Redshift
DW table
Amazon Redshift Amazon EC2
DW table report generation

Nós gerenciamos Você gerencia

EMR Clusters Instancias
Instancias
EC2
EC2

EMR Clusters
Recursos On Premise

Na AWS On Premises
Alta Frequencia $1/mês $2.50/mês
Baixa Frequencia $.60/mês $1.50/mês

OBRIGADO!
awshub.com.br
slideshare.net/AmazonWebServicesLATAM
José Papo
AWS Tech Evangelist
@josepapo

Amazon EMR enables fast processing of large structured or unstructured datasets, and in this presentation we'll show you how to setup an Amazon EMR job flow to analyse application logs, and perform Hive queries against it. We also review best practices around data file organisation on Amazon Simple Storage Service (S3), how clusters can be started from the AWS web console and command line, and how to monitor the status of a Map/Reduce job. Finally we take a look at Hadoop ecosystem tools you can use with Amazon EMR and the additional features of the service. See a recording of the webinar based on this presentation on YouTube here: Check out the rest of the Masterclass webinars for 2015 here: http://aws.amazon.com/campaigns/emea/masterclass/ See the Journey Through the Cloud webinar series here: http://aws.amazon.com/campaigns/emea/journey/

BDT201 AWS Data Pipeline - AWS re: Invent 2012

Amazon Web Services

Masterclass Webinar - Amazon Elastic MapReduce (EMR)Amazon Web Services

Deep Dive - Amazon Elastic MapReduce (EMR)

Amazon Web Services

Amazon Elastic MapReduce is one of the largest Hadoop operators in the world. Since its launch five years ago, AWS customers have launched more than 5.5 million Hadoop clusters. In this talk, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters and other Amazon EMR architectural patterns. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We also share best practices to keep your Amazon EMR cluster cost efficient. Speakers: Ian Meyers, AWS Solutions Architect Ian McDonald, IT Director, SwiftKey

Is your Cloud Bill soaring too high? In this presentation we will be exploring how to: -Drastically lower cloud spend and to more efficiently deploy, use, grow, and manage your cloud infrastructure. - Optimize storage to compute and everything in between. -Identify the best techniques, and how and when to use them. While the discussion will center around AWS, the same techniques can generally be applied to Azure, GCP, and others. You’ll leave this session ready to employ impactful cost optimization solutions to your environment that will start saving you money immediately.

AWS Perú Meetup - Arquitecting for HA by Raul Hugo

Carlos Cortez Senior Cloud Architect

AWS EMEA Online Summit - Blending Spot and On-Demand instances to optimizing ...

Cobus Bernard

Customer Case Study Containerised Bioinformatics on AWS How we Achieved 20x l...

Amazon Web Services

The Garvan Institute of Medical Research and Genome.One undertook a joint pilot project with AWS to optimise and customise our genomic analysis. We have run over 4,500 genomes through a new genomic pipeline, developed to the Broad Institute Best Practices leveraging Amazon’s ECS, SQS, RDS, CloudFormation, CloudWatch and Spot Instances to optimise specifically for the cloud. This session will present our architecture, learnings, and the cost reductions we’ve achieved. Speaker: Dr. Liviu Constantinescu, DevOps Team Lead, Genome.One Level: 300

Datadog jawsfesta2017 20171104

Masahiro Hattori

AWS Summit Stockholm 2014 – T3 – disaster recovery on AWS

Amazon Web Services

Implementation of a disaster recovery (DR) site is crucial for the business continuity of any enterprise. Due to the fundamental nature of features like elasticity, scalability and geographic distribution, DR implementation on AWS can be done at 10-50% of the conventional cost. In this session, we do a deep dive into proven DR architectures on AWS and the best practices, tools and techniques to get the most out of them. This session is recommended for attendees who wish to explore options for ensuring the continuity of their business.

AWS Summit Tel Aviv - Startup Track - Data Analytics & Big DataAmazon Web Services

How to Build Resilient & Scalable PHP Applications on Amazon Web Services

jamiebegin

SF Big Analytics: Machine Learning with Presto by Christopher Berner

Chester Chen

Talk 1: Machine Learning in Presto Presto is an open source distributed SQL query engine used by Facebook, in our Hadoop warehouse. It's typically about 10x faster than Hive, and can be extended to a number of other use cases. One of these extensions adds SQL functions to create and make predictions with machine learning models. The aim of this is to significantly reduce the time it takes to prototype a model, by moving the construction and testing of the model to the database. Bio: Christopher Berner works as a software engineer at Facebook on the Presto team. He wrote the ML functionality, and has worked on the query planner, type system, bytecode generator, and many other pieces of Presto. Before Presto he worked on the newsfeed ranking team developing machine learning models.

Introduction to AWS Batch

Amazon Web Services

AWS Batch is a fully-managed service that enables developers, scientists, and engineers to easily and efficiently run batch computing workloads of any scale on AWS. AWS Batch automatically provisions compute resources and optimizes the workload distribution based on the quantity and scale of the workloads. With AWS Batch, there is no need to install or manage batch computing software, allowing you to focus on analyzing results and solving problems. AWS Batch plans, schedules, and executes your batch computing workloads across the full range of AWS compute services and features, such as Amazon EC2, Spot Instances, and AWS Lambda. AWS Batch reduces operational complexities, saving time and reducing costs. In this session, you will learn core concepts behind AWS Batch and details of how the service functions.

Cost Optimisation on AWS

Amazon Web Services

The webinar based on this presentation discussed strategies that you can adopt to help you save money in the AWS Cloud. From turning systems off at night, to implementing bidding strategies on the spot market, there are many ways in which you can manage and reduce your costs with AWS. Dive into the differences between instance types; explain how you can reduce costs with Reserved Instances, the spot market and by architecting to reduce costs. We'll discuss how to combine on-demand pricing with spot pricing to perform cost effective big data analysis, and introduce customer examples to illustrate how AWS customers gain the most from AWS whilst at the same time managing their spend. Topics include: • Understand different cost optimisation strategies you can employ in the AWS Cloud • Learn how to take advantage of different instance types • Discover architectural principles behind cost optimisation in AWS • Learn about tools to help you keep on top of your AWS spend You can find a recording of this webinar on YouTube here: http://youtu.be/kId90Q7b6kY

Streaming Data Analytics with Amazon Redshift and Kinesis Firehose

Amazon Web Services

by Joyjeet Banerjee, Enterprise Solutions Architect, AWS Evolving your analytics from batch processing to real-time processing can have a major business impact, but ingesting streaming data into your data warehouse requires building complex streaming data pipelines. Amazon Kinesis Firehose solves this problem by making it easy to transform and load streaming data into Amazon Redshift so that you can use existing analytics and business intelligence tools to extract information in near real-time and respond promptly. In this session, we will dive deep using Amazon Kinesis Firehose to load streaming data into Amazon Redshift reliably, scalably, and cost-effectively. Level: 200

I want to

Bootcamp

Manual de administración de wordpress

Esus28

Apresentação Paisagem Horizonte

Miguel Lago

What's hot

Randall Hunt - AWS Midwest Community Day Keynote

AWS Chicago

Best Practices for Managing Hadoop Framework Based Workloads (on Amazon EMR) ...

Amazon Web Services

EC2 Pricing Model (deck 0307 of the InfiniteSkills AWS course at http://bit.l...

rICh morrow

Apresentação1Luiz Junior

Presto @ Netflix: Interactive Queries at Petabyte Scale

DataWorks Summit

This One Weird API Request Will Save You Thousands

Amazon Web Services

AWS Peru Meetup - recap reinvent 2016 (by Carlos Cortez)

Carlos Cortez Senior Cloud Architect

Bring Your Cloud Bill Back Down to Earth

Alan Hand

AWS Perú Meetup - Arquitecting for HA by Raul Hugo

Carlos Cortez Senior Cloud Architect

AWS EMEA Online Summit - Blending Spot and On-Demand instances to optimizing ...

Cobus Bernard

Customer Case Study Containerised Bioinformatics on AWS How we Achieved 20x l...

Amazon Web Services

Datadog jawsfesta2017 20171104

Masahiro Hattori

AWS Summit Stockholm 2014 – T3 – disaster recovery on AWS

Amazon Web Services

AWS Summit Tel Aviv - Startup Track - Data Analytics & Big DataAmazon Web Services

How to Build Resilient & Scalable PHP Applications on Amazon Web Services

jamiebegin

SF Big Analytics: Machine Learning with Presto by Christopher Berner

Chester Chen

Introduction to AWS Batch

Amazon Web Services

Cost Optimisation on AWS

Amazon Web Services

Streaming Data Analytics with Amazon Redshift and Kinesis Firehose

Amazon Web Services

What's hot (19)

Randall Hunt - AWS Midwest Community Day Keynote

Best Practices for Managing Hadoop Framework Based Workloads (on Amazon EMR) ...

EC2 Pricing Model (deck 0307 of the InfiniteSkills AWS course at http://bit.l...

Apresentação1

Presto @ Netflix: Interactive Queries at Petabyte Scale

This One Weird API Request Will Save You Thousands

AWS Peru Meetup - recap reinvent 2016 (by Carlos Cortez)

Bring Your Cloud Bill Back Down to Earth

AWS Perú Meetup - Arquitecting for HA by Raul Hugo

AWS EMEA Online Summit - Blending Spot and On-Demand instances to optimizing ...

Customer Case Study Containerised Bioinformatics on AWS How we Achieved 20x l...

Datadog jawsfesta2017 20171104

AWS Summit Stockholm 2014 – T3 – disaster recovery on AWS

AWS Summit Tel Aviv - Startup Track - Data Analytics & Big Data

How to Build Resilient & Scalable PHP Applications on Amazon Web Services

SF Big Analytics: Machine Learning with Presto by Christopher Berner

Introduction to AWS Batch

Cost Optimisation on AWS

Streaming Data Analytics with Amazon Redshift and Kinesis Firehose

Viewers also liked

I want to

Bootcamp

Manual de administración de wordpress

Esus28

Apresentação Paisagem Horizonte

Miguel Lago

Vidneo Presentationpanathinaikos131313

Olhando Para o Alto (OA) - MMGérson Fagundes da Cunha

Powerpoint foruseofcalculatorMichael Holtfrerich

Tarea321

09852314567

Viewers also liked (7)

I want to

Manual de administración de wordpress

Apresentação Paisagem Horizonte

Vidneo Presentation

Olhando Para o Alto (OA) - MM

Powerpoint foruseofcalculator

Tarea321

Similar to Introdução ao AWS Data Pipeline

(BDT308) Using Amazon Elastic MapReduce as Your Scalable Data Warehouse | AWS...

Amazon Web Services

In this presentation, we will demonstrate how to use Amazon Elastic MapReduce as your scalable data warehouse. Amazon EMR supports clusters with thousands of nodes and is used to access petabyte scale data warehouses. Amazon EMR is not only fast, but it is also easy to use for rapid development and adhoc analysis. We will show you how access the large scale data warehouses with emerging tools such as Hue, Hive, low latency SQL applications like Presto, and alternative execution engines like Apache Spark. We will also show you how these tools integrate directly with other AWS big data services such as Amazon S3, Amazon DynamoDB, and Amazon Kinesis.

slide share on aws data pipe line

KATTA ROHITHREDDY

(BDT310) Big Data Architectural Patterns and Best Practices on AWS | AWS re:I...

Amazon Web Services

The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.

Think Big Data, Think Cloud - AWS Presentation - AWS Cloud Storage for the En...

Amazon Web Services

Overview of Amazon Web ServicesAmazon Web Services

Scalable Media Workflows on the Cloud

Amazon Web Services

This session walks through approaches for media ingest, storage, processing and delivery scenarios on the AWS cloud. We cover solutions for high speed file transfer, cloud-based transcoding, tiered storage, content processing, and global low latency delivery, as well as the orchestration and management of the entire media workflow with the AWS Data Pipeline service. Attendees can expect to come away with an understanding of best practices for architecting and deploying cloud-based media workflows.

Deep Dive: Amazon Elastic MapReduce

Amazon Web Services

Amazon Elastic MapReduce (EMR) is a web service that allows you to easily and securely provision and manage your Hadoop clusters. In this talk, we will introduce you to Amazon EMR design patterns, such as using various data stores such as Amazon S3, how to take advantage of both transient and active clusters, as well as other Amazon EMR architectural patterns. We will dive deep on how to dynamically scale your cluster and address the ways you can fine-tune your cluster. We will discuss bootstrapping Hadoop applications from our partner ecosystem that you can use natively with Amazon EMR. Lastly, we will share best practices on how to keep your Amazon EMR cluster cost-effective.

Datadog jawsdays2017 lunch_lt

Masahiro Hattori

Building Fault Tolerant Applications in the cloud - AWS Summit 2012 - NYC Amazon Web Services

Best Practices in Architecting for the Cloud Webinar - Jinesh Varia

Amazon Web Services

AWS Lambda

Amazon Web Services

AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security. Lambda runs your code on high-availability compute infrastructure and performs all the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code and security patch deployment, and code monitoring and logging. All you need to do is supply the code. Presented by: Julien Simon, Principal Technical Evangelist, Amazon Web Services

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

Amazon Web Services

In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.

AWS Update | London - Performance Update and Provisioned IOPS

Amazon Web Services

Masterworks talk on Big Data and the implications of petascale scienceDeepak Singh

AWS Real-Time Event Processing

Amazon Web Services

Real-time event processing monitors the incoming data stream and initiates action based on detected events like fraud, error or performance degradation. These events are often used to issue alerts and notifications, take responsive action, or to populate a monitoring dashboard. In this session, we will walk through different use cases for event processing and demonstrate how to build a scalable pipeline for tracking IoT device status. AWS services to be covered include: AWS Lambda and the Kinesis Client Library (KCL).

Plenary Talk at ACAT 2010Deepak Singh

MED203 Scalable Media Processing - AWS re: Invent 2012

Amazon Web Services

Big Data and Analytics

Amazon Web Services

AWS May Webinar Series - Getting Started with Amazon EMR

Amazon Web Services

If you are interested to know more about AWS Chicago Summit, please use the following to register: http://amzn.to/1RooPPL Many AWS customers store vast amounts of data in Amazon S3, a low cost, scalable, and durable object store; Amazon DynamoDB, a NoSQL database; or Amazon Kinesis, a real time data stream processing service. With large datasets in various AWS services, how do you derive value from this information in a cost-effective way? Using Amazon Elastic MapReduce (Amazon EMR) with applications in the Apache Hadoop ecosystem, you can directly interact with data in each of these storage services for scalable analytics workloads or ad hoc queries. You can quickly and easily launch an Amazon EMR cluster from the AWS Management Console, and scale your cluster to match the compute and memory resources needed for your workflow, independent from the storage capacity used in your AWS storage services. The webinar will accelerate your use of Amazon EMR by showing you how to create and monitor Amazon EMR clusters, and provide several use cases and architectures for using Amazon EMR with different AWS data stores. Learning Objectives: • Recognize when to use Amazon EMR • Understand the steps required to set up and monitor an Amazon EMR cluster • Architect applications that effectively use Amazon EMR • Understand how to use HUE for ad hoc query of data in Amazon S3 Who Should Attend: • Developers, LOB owners, Continuous Integration & Continuous Delivery (CICD) practitioners

AWS SSA Webinar 9 - Getting Started on AWS: Storage

Cobus Bernard

Similar to Introdução ao AWS Data Pipeline (20)

(BDT308) Using Amazon Elastic MapReduce as Your Scalable Data Warehouse | AWS...

slide share on aws data pipe line

(BDT310) Big Data Architectural Patterns and Best Practices on AWS | AWS re:I...

Think Big Data, Think Cloud - AWS Presentation - AWS Cloud Storage for the En...

Overview of Amazon Web Services

Scalable Media Workflows on the Cloud

Deep Dive: Amazon Elastic MapReduce

Datadog jawsdays2017 lunch_lt

Building Fault Tolerant Applications in the cloud - AWS Summit 2012 - NYC

Best Practices in Architecting for the Cloud Webinar - Jinesh Varia

AWS Lambda

Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...

AWS Update | London - Performance Update and Provisioned IOPS

Masterworks talk on Big Data and the implications of petascale science

AWS Real-Time Event Processing

Plenary Talk at ACAT 2010

MED203 Scalable Media Processing - AWS re: Invent 2012

Big Data and Analytics

AWS May Webinar Series - Getting Started with Amazon EMR

AWS SSA Webinar 9 - Getting Started on AWS: Storage

More from Amazon Web Services LATAM

AWS para terceiro setor - Sessão 1 - Introdução à nuvem

Amazon Web Services LATAM

AWS para terceiro setor - Sessão 2 - Armazenamento e Backup

Amazon Web Services LATAM

AWS para terceiro setor - Sessão 3 - Protegendo seus dados.

Amazon Web Services LATAM

AWS para terceiro setor - Sessão 1 - Introdução à nuvem

Amazon Web Services LATAM

AWS para terceiro setor - Sessão 2 - Armazenamento e Backup

Amazon Web Services LATAM

AWS para terceiro setor - Sessão 3 - Protegendo seus dados.

Amazon Web Services LATAM

Automatice el proceso de entrega con CI/CD en AWS

Amazon Web Services LATAM

Automatize seu processo de entrega de software com CI/CD na AWS

Amazon Web Services LATAM

Cómo empezar con Amazon EKS

Amazon Web Services LATAM

Como começar com Amazon EKS

Amazon Web Services LATAM

Ransomware: como recuperar os seus dados na nuvem AWS

Amazon Web Services LATAM

Ransomware: cómo recuperar sus datos en la nube de AWS

Amazon Web Services LATAM

Ransomware: Estratégias de Mitigação

Amazon Web Services LATAM

Ransomware: Estratégias de Mitigación

Amazon Web Services LATAM

Aprenda a migrar y transferir datos al usar la nube de AWS

Amazon Web Services LATAM

Al mover datos a la nube, los clientes deben comprender los métodos óptimos para los diferentes casos de uso, los tipos de datos que están moviendo y los recursos disponibles en la red, entre otros. Las soluciones de migración y transferencia de AWS contemplan desde la migración de datos con conectividad limitada, almacenamiento en la nube híbrida, transferencias frecuentes de archivos B2B, hasta transferencias de datos en línea y sin conexión. En esta sesión, le mostramos cómo puede acelerar la migración y transferencia de datos de manera simplificada desde y hacia la nube de AWS.

Aprenda como migrar e transferir dados ao utilizar a nuvem da AWS

Amazon Web Services LATAM

Ao mover dados para a nuvem, os clientes precisam entender os métodos ideais de movê-los para diferentes casos de uso, os tipos de dados que estão movendo e os recursos de rede disponíveis, entre outras considerações. As soluções de migração e transferência da AWS atendem desde a migração de dados com conectividade limitada, armazenamento em nuvem híbrida, transferências frequentes de arquivos B2B até transferências de dados online e offline. Nessa sessão, mostraremos como você pode simplificar e acelerar sua migração e transferência de dados de e para a nuvem AWS.

Cómo mover a un almacenamiento de archivos administrados

Amazon Web Services LATAM

El almacenamiento de archivos tiene diversos casos de uso; como directorios de usuarios, datos de aplicaciones, archivos multimedia y almacenamiento compartido para cargas de trabajo de alto rendimiento. La administración del almacenamiento de archivos en instalaciones propias suele ser un trabajo pesado, indiferenciado, con altos costos de adquisición, carga operativa para configurar y administra, lo que conlleva a desafíos de escalabilidad. En esta sesión, le mostramos cómo puede aprovechar las soluciones de archivos totalmente administradas de AWS para dejar de preocuparse por la sobrecarga administrativa de configurar, proteger, mantener y realizar copias de seguridad de su infraestructura de archivos.

Simplifique su BI con AWS

Amazon Web Services LATAM

La visualización de datos analíticos es un reto al que se enfrentan muchas organizaciones, el poder crear tableros, alertas, agregar predicciones a sus datos y actuar de acuerdo a estas de manera rápida es una necesidad de todos los negocios actuales. Únase a nuestros arquitectos para aprender como Amazon QuickSight le permite agregar inteligencia de negocios a sus aplicaciones y crear predicciones a futuro de sus datos. Amazon QuickSight es un servicio de inteligencia de negocios escalable y serverless creado para la nube, a través del cual podrá explotar sus datos de negocio para convertirlos en insights para hacer decisiones informadas sobre su negocio sin preocuparse de la gestión, escalamiento y la disponibilidad de la infraestructura de cómputo.

Simplifique o seu BI com a AWS

Amazon Web Services LATAM

A visualização de dados é um desafio que muitas organizações enfrentam hoje. Criar dashboards, alertas, fazer previsões e agir rapidamente de acordo com os insights dos dados é uma necessidade de todas as empresas. Junte-se aos nossos arquitetos para aprender como o Amazon QuickSight o ajudará a adicionar BI aos seus aplicativos. O Amazon Quicksight é um serviço de BI escalável e serverless criado para a nuvem. Com ele, você pode explorar seus dados para obter insights e tomar decisões embasadas em seus negócios, sem se preocupar em gerenciar e dimensionar servidores e manter a disponibilidade de sua infraestrutura.

Os benefícios de migrar seus workloads de Big Data para a AWS

Amazon Web Services LATAM

More from Amazon Web Services LATAM (20)

AWS para terceiro setor - Sessão 1 - Introdução à nuvem

AWS para terceiro setor - Sessão 2 - Armazenamento e Backup

AWS para terceiro setor - Sessão 3 - Protegendo seus dados.

AWS para terceiro setor - Sessão 1 - Introdução à nuvem

AWS para terceiro setor - Sessão 2 - Armazenamento e Backup

AWS para terceiro setor - Sessão 3 - Protegendo seus dados.

Automatice el proceso de entrega con CI/CD en AWS

Automatize seu processo de entrega de software com CI/CD na AWS

Cómo empezar con Amazon EKS

Como começar com Amazon EKS

Ransomware: como recuperar os seus dados na nuvem AWS

Ransomware: cómo recuperar sus datos en la nube de AWS

Ransomware: Estratégias de Mitigação

Ransomware: Estratégias de Mitigación

Aprenda a migrar y transferir datos al usar la nube de AWS

Aprenda como migrar e transferir dados ao utilizar a nuvem da AWS

Cómo mover a un almacenamiento de archivos administrados

Simplifique su BI con AWS

Simplifique o seu BI com a AWS

Os benefícios de migrar seus workloads de Big Data para a AWS

Recently uploaded

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

UiPath Test Automation using UiPath Test Suite series, part 3

DianaGray10

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

Inflectra

In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring. Learn about: • The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks. • Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective. • Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification. • Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process. Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

Essentials of Automations: Optimizing FME Workflows with Parameters

Safe Software

Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place. Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects. Here’s what you’ll gain: - Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows. - Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy. - Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency. - Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity. We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic. Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.

UiPath Test Automation using UiPath Test Suite series, part 4

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap. The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies. Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques What will you get from this session? 1. Insights into SAP testing best practices 2. Heatmap utilization for testing 3. Optimization of testing processes 4. Demo Topics covered: Execution from the test manager Orchestrator execution result Defect reporting SAP heatmap example with demo Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Knowledge engineering: from people to machines and back

Elena Simperl

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Leading Change strategies and insights for effective change management pdf 1.pdf

OnBoard

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

Abida Shariff

How world-class product teams are winning in the AI era by CEO and Founder, P...

Product School

JMeter webinar - integration with InfluxDB and Grafana

RTTS

Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application. In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics. Length: 30 minutes Session Overview ------------------------------------------- During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana: - What out-of-the-box solutions are available for real-time monitoring JMeter tests? - What are the benefits of integrating InfluxDB and Grafana into the load testing stack? - Which features are provided by Grafana? - Demonstration of InfluxDB and Grafana using a practice web application To view the webinar recording, go to: https://www.rttsweb.com/jmeter-integration-webinar

Recently uploaded (20)

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

UiPath Test Automation using UiPath Test Suite series, part 3

Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

The Art of the Pitch: WordPress Relationships and Sales

Essentials of Automations: Optimizing FME Workflows with Parameters

UiPath Test Automation using UiPath Test Suite series, part 4

Knowledge engineering: from people to machines and back

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

Leading Change strategies and insights for effective change management pdf 1.pdf

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Accelerate your Kubernetes clusters with Varnish Caching

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

IOS-PENTESTING-BEGINNERS-PRACTICAL-GUIDE-.pptx

How world-class product teams are winning in the AI era by CEO and Founder, P...

JMeter webinar - integration with InfluxDB and Grafana

Introdução ao AWS Data Pipeline

1. Introdução ao AWS Data Pipeline: ETL na Nuvem da AWS José Papo AWS Tech Evangelist @josepapo

3. Amazon S3 Amazon RDS Amazon Amazon DynamoDB Redshift HDFS On (Amazon EMR) Premise

4. Amazon S3 Amazon RDS Amazon Amazon DynamoDB Redshift HDFS On (Amazon EMR) Premise

5. Amazon S3 Amazon RDS Amazon Amazon DynamoDB Redshift HDFS On (Amazon EMR) Premise

6. Amazon S3 Amazon RDS Amazon Amazon DynamoDB Redshift HDFS On (Amazon EMR) Premise

7. Input Datanode Activity Output Datanode

8. Input Datanode com validação de pré-condição Atividade com notificações de falha ou delay Ouput Datanode

10. Custom Amazon RDS Amazon S3 Amazon demographics logs Precondition DynamoDB event data Hive script EMR usage-by-geo job Amazon Redshift DW table Amazon Redshift Amazon EC2 DW table report generation

11. Custom Amazon RDS Amazon S3 Amazon demographics logs Precondition DynamoDB event data Hive script EMR usage-by-geo job Amazon Redshift DW table Amazon Redshift Amazon EC2 DW table report generation

12. Nós gerenciamos Você gerencia EMR Clusters Instancias Instancias EC2 EC2 EMR Clusters Recursos On Premise

13.

14. Na AWS On Premises Alta Frequencia $1/mês $2.50/mês Baixa Frequencia $.60/mês $1.50/mês

15. DEMO!

16. OBRIGADO! awshub.com.br slideshare.net/AmazonWebServicesLATAM José Papo AWS Tech Evangelist @josepapo

Introdução ao AWS Data Pipeline

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (7)

Similar to Introdução ao AWS Data Pipeline

Similar to Introdução ao AWS Data Pipeline (20)

More from Amazon Web Services LATAM

More from Amazon Web Services LATAM (20)

Recently uploaded

Recently uploaded (20)

Introdução ao AWS Data Pipeline