AWS offers a family of intelligent services that provide cloud-native machine learning and deep learning technologies to address different use cases and needs. This deck will help you to gain insight into practical use cases for Amazon Lex, Amazon Polly, and Amazon Rekognition, and learn about newly announced services Amazon Rekognition Video, Amazon Comprehend, Amazon Translate, and Amazon Transcribe. This presentation took place in Australia and New Zealand as part of the AWS Learning Series in 2018.
Ensuring Voice Quality for Amazon ConnectCarl Blume
Presentation by Chad Hart, callstats.io product manager, and Kentis Gopalla, AWS global consulting partner lead, delivered in the Amazon Connect booth at Enterprise Connect 2019
Managing AWS Costs with Anomaly Detection and Root Cause AnalysisYotascale
Learn to leverage AI to streamline your cloud operations with Anomaly Detection and Root Cause Analysis.
Use AI to maintain cloud governance, optimize cloud workloads, and simplify cloud operations. Find out more, request a demo, or set up a free trial at www.yotascale.com.
Detecting Financial Market Manipulation Using Machine Learning (AIM347) - AWS...Amazon Web Services
Researchers from the University of Michigan and Georgia Tech, in collaboration with the AWS Research Initiative, have developed new techniques to identify financial market manipulation in high-volume, high-velocity market data streams. They are using a combination of data-driven and model-based techniques to identify financial market manipulation. In this session, we discuss the use of machine learning using Amazon SageMaker to study, process, and analyze huge volumes of data to prevent financial market manipulation.
(SDD411) Amazon CloudSearch Deep Dive and Best Practices | AWS re:Invent 2014Amazon Web Services
Amazon CloudSearch is a fully-managed search service in the cloud that lets you quickly and easily set up and use a search solution for your application. The latest version of CloudSearch includes tons of new and advanced search and administrative features. This session covers how to design for high scale at low cost, as well as best practices for handling multiple languages, ranking your search results, securing your CloudSearch domains, achieving cost-effective multi-tenancy, sourcing from many different systems, and getting the most out of your CloudSearch instances.
Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) cloud
Can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage
Amazon EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic
AWS offers a family of intelligent services that provide cloud-native machine learning and deep learning technologies to address different use cases and needs. This deck will help you to gain insight into practical use cases for Amazon Lex, Amazon Polly, and Amazon Rekognition, and learn about newly announced services Amazon Rekognition Video, Amazon Comprehend, Amazon Translate, and Amazon Transcribe. This presentation took place in Australia and New Zealand as part of the AWS Learning Series in 2018.
Ensuring Voice Quality for Amazon ConnectCarl Blume
Presentation by Chad Hart, callstats.io product manager, and Kentis Gopalla, AWS global consulting partner lead, delivered in the Amazon Connect booth at Enterprise Connect 2019
Managing AWS Costs with Anomaly Detection and Root Cause AnalysisYotascale
Learn to leverage AI to streamline your cloud operations with Anomaly Detection and Root Cause Analysis.
Use AI to maintain cloud governance, optimize cloud workloads, and simplify cloud operations. Find out more, request a demo, or set up a free trial at www.yotascale.com.
Detecting Financial Market Manipulation Using Machine Learning (AIM347) - AWS...Amazon Web Services
Researchers from the University of Michigan and Georgia Tech, in collaboration with the AWS Research Initiative, have developed new techniques to identify financial market manipulation in high-volume, high-velocity market data streams. They are using a combination of data-driven and model-based techniques to identify financial market manipulation. In this session, we discuss the use of machine learning using Amazon SageMaker to study, process, and analyze huge volumes of data to prevent financial market manipulation.
(SDD411) Amazon CloudSearch Deep Dive and Best Practices | AWS re:Invent 2014Amazon Web Services
Amazon CloudSearch is a fully-managed search service in the cloud that lets you quickly and easily set up and use a search solution for your application. The latest version of CloudSearch includes tons of new and advanced search and administrative features. This session covers how to design for high scale at low cost, as well as best practices for handling multiple languages, ranking your search results, securing your CloudSearch domains, achieving cost-effective multi-tenancy, sourcing from many different systems, and getting the most out of your CloudSearch instances.
Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the Amazon Web Services (AWS) cloud
Can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage
Amazon EC2 enables you to scale up or down to handle changes in requirements or spikes in popularity, reducing your need to forecast traffic
Introducing AWS DataSync - Simplify, automate, and accelerate online data tra...Amazon Web Services
SFTP is used for the exchange of data across many industries, including financial services, healthcare, and retail. In this session, we will introduce you to AWS Transfer for SFTP, a service that helps you easily migrate file transfer workflows to AWS, without needing to modify applications or manage SFTP servers. We will demonstrate the product and talk about how to migrate your users so they continue to use their existing SFTP clients and credentials, while the data they access is stored in S3. You will also learn how FINRA is using this new service in conjunction with their Data Lake on AWS.N/A
Apache Spark Based Reliable Data Ingestion in Datalake with Gagan AgrawalDatabricks
Ingesting data from variety of sources like Mysql, Oracle, Kafka, Sales Force, Big Query, S3, SaaS applications, OSS etc. with billions of records into datalake (for reporting, adhoc analytics, ML jobs) with reliability, consistency, schema evolution support and within expected SLA has always been a challenging job. Also ingestion may have different flavors like full ingestion, incremental ingestion with and without compaction/de-duplication and transformations with their own complexity of state management and performance. Not to mention dependency management where hundreds / thousands of downstream jobs are dependent on this ingested data and hence data availability on time is of utmost importance. Most data teams end up creating adhoc ingestion pipelines written in different languages and technologies which adds operational overheads and knowledge is mostly limited to few.
In this session, I will talk about how we leveraged Sparks Dataframe abstraction for creating generic ingestion platform capable of ingesting data from varied sources with reliability, consistency, auto schema evolution and transformations support. Will also discuss about how we developed spark based data sanity as one of the core components of this platform to ensure 100% correctness of ingested data and auto-recovery in case of inconsistencies found. This talk will also focus how Hive table creation and schema modification was part of this platform and provided read time consistencies without locking while Spark Ingestion jobs were writing on the same Hive tables and how we maintained different versions of ingested data to do any rollback if required and also allow users of this ingested data to go back in time and read snapshot of ingested data at that moment.
Post this talk one should be able to understand challenges involved in ingesting data reliably from different sources and how one can leverage Spark’s Dataframe abstraction to solve this in unified way.
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...Amazon Web Services
In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes, and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.
MongoDB World 2019: Near Real-Time Analytical Data Hub with MongoDBMongoDB
Attendees will learn how to build an operational data hub that can be used as a silo-buster. In this session, I will show how we developed a data hub at TD Ameritrade to provide actionable 360 views of client data using MongoDB. I will also explain why this approach suited our use case better than a Hadoop-based data lake.
AWS provides a range of Compute Services, Amazon EC2, Amazon ECS, AWS Lambda, and AWS Elastic Beanstalk – allowing you to build everything from web applications, mobile backends to data processing applications.
In this session, we will provide an intro level overview of these services and highlight suitable use cases. We will discuss which service to choose to best get your applications up and running on AWS.
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon RedshiftAmazon Web Services
In this session, we take a deep dive on Amazon Redshift architecture and the latest performance enhancements that give you faster insights into your data. We also cover Redshift Spectrum, a feature of Redshift that enables you to analyze data across Redshift and your Amazon S3 data lake to deliver unique insights not possible by analyzing independent data silos. A customer is joining us to share how they were able to extend their data warehouse to their data lake to encompass multiple data sources and data formats. This modern architecture helps them tie together data sources to get actionable insights across their business units.
AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)Amazon Web Services
For discovery-phase research, life sciences companies have to support infrastructure that processes millions to billions of transactions. The advent of a data lake to accomplish such a task is showing itself to be a stable and productive data platform pattern to meet the goal. We discuss how to build a data lake on AWS, using services and techniques such as AWS CloudFormation, Amazon EC2, Amazon S3, IAM, and AWS Lambda. We also review a reference architecture from Amgen that uses a data lake to aid in their Life Science Research.
AWS Black Belt Tech Webinar 2015
Amazon Kinesis
次回のWebinarは、下記よりご確認ください。
http://aws.amazon.com/jp/about-aws/events/#webinar
★AWS Black Belt Tech Webinarとは
AWSのソリューションアーキテクト、TechメンバがAWSのプロダクト/ソリューションを深堀りして解説し、参加されている皆さまからの質問にお答えする無料のWebinar(Webセミナー)です。
毎週水曜日(祝日などを除く)、日本時間の18:00から約1時間にわたってお送りしています。
A closer look at the MySQL and PostgreSQL compatible relational database built for the cloud that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. We’ll explore how Aurora uses the AWS cloud to provide high reliability, high durability, and high throughput.
Speakers:
Steve Abraham - Principal Database Specialist Solutions Architect, AWS
Peter Dachnowicz - Sr. Technical Account Manager, AWS
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
Qu'est-ce qu'une estimation? Comment on fait ça? Pourquoi est-ce si difficile? Pourquoi estime-t-on? Ça sert à quoi et à qui? Est-ce que ça permet de générer plus de valeur? Serait-il pensable de ne pas estimer? Si on n'estime plus, on fait quoi pour répondre à la question : quand est-ce que ça sera fini? Existe-t-il d'autres approches?
Venez en débattre avec nous!!!
Pascal Roy
Stephane Lecuyer
Introducing AWS DataSync - Simplify, automate, and accelerate online data tra...Amazon Web Services
SFTP is used for the exchange of data across many industries, including financial services, healthcare, and retail. In this session, we will introduce you to AWS Transfer for SFTP, a service that helps you easily migrate file transfer workflows to AWS, without needing to modify applications or manage SFTP servers. We will demonstrate the product and talk about how to migrate your users so they continue to use their existing SFTP clients and credentials, while the data they access is stored in S3. You will also learn how FINRA is using this new service in conjunction with their Data Lake on AWS.N/A
Apache Spark Based Reliable Data Ingestion in Datalake with Gagan AgrawalDatabricks
Ingesting data from variety of sources like Mysql, Oracle, Kafka, Sales Force, Big Query, S3, SaaS applications, OSS etc. with billions of records into datalake (for reporting, adhoc analytics, ML jobs) with reliability, consistency, schema evolution support and within expected SLA has always been a challenging job. Also ingestion may have different flavors like full ingestion, incremental ingestion with and without compaction/de-duplication and transformations with their own complexity of state management and performance. Not to mention dependency management where hundreds / thousands of downstream jobs are dependent on this ingested data and hence data availability on time is of utmost importance. Most data teams end up creating adhoc ingestion pipelines written in different languages and technologies which adds operational overheads and knowledge is mostly limited to few.
In this session, I will talk about how we leveraged Sparks Dataframe abstraction for creating generic ingestion platform capable of ingesting data from varied sources with reliability, consistency, auto schema evolution and transformations support. Will also discuss about how we developed spark based data sanity as one of the core components of this platform to ensure 100% correctness of ingested data and auto-recovery in case of inconsistencies found. This talk will also focus how Hive table creation and schema modification was part of this platform and provided read time consistencies without locking while Spark Ingestion jobs were writing on the same Hive tables and how we maintained different versions of ingested data to do any rollback if required and also allow users of this ingested data to go back in time and read snapshot of ingested data at that moment.
Post this talk one should be able to understand challenges involved in ingesting data reliably from different sources and how one can leverage Spark’s Dataframe abstraction to solve this in unified way.
Building Data Lakes and Analytics on AWS; Patterns and Best Practices - BDA30...Amazon Web Services
In this session, we show you how to understand what data you have, how to drive insights, and how to make predictions using purpose-built AWS services. Learn about the common pitfalls of building data lakes, and discover how to successfully drive analytics and insights from your data. Also learn how services such as Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR, Amazon Kinesis, and Amazon ML services work together to build a successful data lake for various roles, including data scientists and business users.
MongoDB World 2019: Near Real-Time Analytical Data Hub with MongoDBMongoDB
Attendees will learn how to build an operational data hub that can be used as a silo-buster. In this session, I will show how we developed a data hub at TD Ameritrade to provide actionable 360 views of client data using MongoDB. I will also explain why this approach suited our use case better than a Hadoop-based data lake.
AWS provides a range of Compute Services, Amazon EC2, Amazon ECS, AWS Lambda, and AWS Elastic Beanstalk – allowing you to build everything from web applications, mobile backends to data processing applications.
In this session, we will provide an intro level overview of these services and highlight suitable use cases. We will discuss which service to choose to best get your applications up and running on AWS.
BDA306 Building a Modern Data Warehouse: Deep Dive on Amazon RedshiftAmazon Web Services
In this session, we take a deep dive on Amazon Redshift architecture and the latest performance enhancements that give you faster insights into your data. We also cover Redshift Spectrum, a feature of Redshift that enables you to analyze data across Redshift and your Amazon S3 data lake to deliver unique insights not possible by analyzing independent data silos. A customer is joining us to share how they were able to extend their data warehouse to their data lake to encompass multiple data sources and data formats. This modern architecture helps them tie together data sources to get actionable insights across their business units.
AWS re:Invent 2016: How to Build a Big Data Analytics Data Lake (LFS303)Amazon Web Services
For discovery-phase research, life sciences companies have to support infrastructure that processes millions to billions of transactions. The advent of a data lake to accomplish such a task is showing itself to be a stable and productive data platform pattern to meet the goal. We discuss how to build a data lake on AWS, using services and techniques such as AWS CloudFormation, Amazon EC2, Amazon S3, IAM, and AWS Lambda. We also review a reference architecture from Amgen that uses a data lake to aid in their Life Science Research.
AWS Black Belt Tech Webinar 2015
Amazon Kinesis
次回のWebinarは、下記よりご確認ください。
http://aws.amazon.com/jp/about-aws/events/#webinar
★AWS Black Belt Tech Webinarとは
AWSのソリューションアーキテクト、TechメンバがAWSのプロダクト/ソリューションを深堀りして解説し、参加されている皆さまからの質問にお答えする無料のWebinar(Webセミナー)です。
毎週水曜日(祝日などを除く)、日本時間の18:00から約1時間にわたってお送りしています。
A closer look at the MySQL and PostgreSQL compatible relational database built for the cloud that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. We’ll explore how Aurora uses the AWS cloud to provide high reliability, high durability, and high throughput.
Speakers:
Steve Abraham - Principal Database Specialist Solutions Architect, AWS
Peter Dachnowicz - Sr. Technical Account Manager, AWS
Summary of past Cassandra benchmarks performed by Netflix and description of how Netflix uses Cassandra interspersed with a live demo automated using Jenkins and Jmeter that created two 12 node Cassandra clusters from scratch on AWS, one with regular disks and one with SSDs. Both clusters were scaled up to 24 nodes each during the demo.
Qu'est-ce qu'une estimation? Comment on fait ça? Pourquoi est-ce si difficile? Pourquoi estime-t-on? Ça sert à quoi et à qui? Est-ce que ça permet de générer plus de valeur? Serait-il pensable de ne pas estimer? Si on n'estime plus, on fait quoi pour répondre à la question : quand est-ce que ça sera fini? Existe-t-il d'autres approches?
Venez en débattre avec nous!!!
Pascal Roy
Stephane Lecuyer
Web Applications on AWS: This session introduces AWS services that you can leverage to build a scalable web application architecture on AWS to handle large-scale flows.
面對日新月異的大數據工具,有時候很難跟上這節奏。有鑑於此,Amazon Web Services提供了廣泛而完善的雲端運算服務組合,幫助您構建、維護和部署大數據應用程式。
這場線上研討會,將為各位深入淺出介紹AWS 雲端平台提供的各種大數據選項,包括現正流行的大數據框架,如Hadoop、Spark、NoSQL數據庫等,同時透過使用案例來瞭解最佳實踐方式。最後,您將了解如何應用這些工具服務,將大數據導入您的現實應用程式中。
This webinar will familiarize you with the basics of AWS IoT and AWS Lambda and demonstrate how to build IoT backends using these services. AWS IoT is a managed cloud platform that lets connected devices easily and securely interact with cloud applications and other devices. AWS Lambda is a compute service that runs your code in response to triggers and automatically manages the compute resources for you.
We will discuss how you can leverage AWS IoT to receive data and send commands to IoT devices. You will learn how to setup device gateways that trigger AWS Lambda functions to process inbound data from IoT devices. You will also learn how to use Lambda to read and write to Amazon DynamoDB, a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale.
Learning Objectives:
Understand key AWS IoT and AWS Lambda and AWS IoT features
Learn how to set up a IoT backend using AWS IoT and AWS Lambda
Explore sample use cases, best practices, and tips on using AWS Lambda with AWS IoT
Who Should Attend:
IoT Developers
Build 1 trillion warehouse based on carbon databoxu42
Apache CarbonData & Spark Meetup
Build 1 trillion warehouse based on CarbonData
Huawei
Apache Spark™ is a unified analytics engine for large-scale data processing.
CarbonData is a high-performance data solution that supports various data analytic scenarios, including BI analysis, ad-hoc SQL query, fast filter lookup on detail record, streaming analytics, and so on. CarbonData has been deployed in many enterprise production environments, in one of the largest scenario it supports queries on single table with 3PB data (more than 5 trillion records) with response time less than 3 seconds!
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS.
In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup.
Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti.
Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.
Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi
Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
L’utilizzo dei container è in continua crescita.
Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili.
I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!
In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th.
Event Agenda :
Open banking so far (short recap)
• PSD2, OB UK, OB Australia, OB LATAM, OB Israel
Intro to Open Finance marketplace
• Scope
• Features
• Tech overview and Demo
The role of the Cloud
The Future of APIs
• Complying with regulation
• Monetizing data / APIs
• Business models
• Time to market
One platform for all: a Strategic approach
Q&A
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc.
AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta.
Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity.
AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet.
Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.
Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.
Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti.
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti.
Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire.
Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito.
In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.
Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline.
Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi.
La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.
In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.
Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.
13. 按查詢付款 - 掃描$ 5 / TB
• 支付每個查詢掃描的數據量
• 節省成本的方法
• 壓縮
• 轉換為Columnar格式
• 使用 partitioning
• 免費:DDL查詢,失敗的查詢
Dataset Size on Amazon S3 Query Run time Data Scanned Cost
Logs stored as Text
files
1 TB 237 seconds 1.15TB $5.75
Logs stored in
Apache Parquet
format*
130 GB 5.13 seconds 2.69 GB $0.013
Savings 87% less with Parquet 34x faster 99% less data scanned 99.7% cheaper
14. 轉換為ORC和PARQUET
• 您可以使用Hive CTAS轉換數據
• CREATE TABLE new_key_value_store
• STORED AS PARQUET
• As
• SELECT col_1,col2,col3 FROM noncolumartable
• SORT BY new_key,key_value_pair;
• 您也可以使用Spark將文件轉換為PARQUET / ORC
• 20行Pyspark代碼,將1TB的文本數據轉換為130 GB的PARQUET在EMR上運
行
• 快速轉換總成本$ 5
https://github.com/awslabs/aws-big-data-blog/tree/master/aws-blog-spark-parquet-conversion
15. 如何定義你的 partitions
CREATE EXTERNAL TABLE Employee (
Id INT,
Name STRING,
Address STRING
) PARTITIONED BY (year INT)
ROW FORMAT DELIMITED FIELDS
TERMINATED BY ','
LOCATION
‘s3://mybucket/athena/inputdata/’;
CREATE EXTERNAL TABLE Employee (
Id INT,
Name STRING,
Address STRING,
INT Year
) PARTITIONED BY (year INT)
ROW FORMAT DELIMITED FIELDS
TERMINATED BY ','
LOCATION
‘s3://mybucket/athena/inputdata/’;
16. 如何定義你的 partitions
s3://elasticmapreduce/impressions/
PRE dt=2009-04-12-13-00/
PRE dt=2009-04-12-13-05/
PRE dt=2009-04-12-13-10/
PRE dt=2009-04-12-13-15/
PRE dt=2009-04-12-13-20/
CREATE EXTERNAL TABLE impressions ( requestBeginTime string, ......)
PARTITIONED BY (dt string) LOCATION
's3://elasticmapreduce/samples/hive-ads/tables/impressions/' ;
PRE dt=2009-04-12-14-10/
MSCK REPAIR TABLE impressions
24. 遷移的關鍵和TCO考慮
• DO NOT LIFT AND SHIFT
• 透過S3,張存儲和計算分開
• 解構工作負載並映射到開源工具
• 短暫的群集和自動縮放
• 選擇實例類型和EC2 Spot實例
25. 分拆運算和存儲,使用S3去作為您的數據層
HDFS
S3 is designed for 11
9’s of durability and is
massively scalable
EC2 Instance
Memory
Amazon S3
Amazon EMR
Amazon EMR
Intermediates
stored on local
disk or HDFS
Local
30. 提交作業的選項
Amazon EMR
Step API
Submit a Spark
application
Amazon EMR
AWS Data Pipeline
Airflow, Luigi, or other
schedulers on EC2
Create a pipeline
to schedule job
submission or create
complex workflows
AWS Lambda
Use AWS Lambda to
submit applications to
EMR Step API or directly
to Spark on your cluster
Use Oozie on your
cluster to build
DAGs of jobs
33. Spot for
task nodes
Up to 80%
off EC2
On-Demand
pricing
On-demand for
core nodes
Standard
Amazon EC2
pricing for
on-demand
capacity
使用Spot和Reserved instance降低成本
以可預見的成本滿足SLA 以較低的成本超出SLA
37. DataXu – 180TB of Log Data per Day
CDN
Real Time
Bidding
Retargeting
Platform
Kinesis Attribution & ML
S3
Reporting
Data Visualization
Data
Pipeline
ETL(Spark SQL)
Ecosystem of tools and services
Amazon Athena
38. Petabytes of data generated
on-premises, brought to AWS,
and stored in S3
Thousands of analytical
queries performed on EMR
and Amazon Redshift.
Stringent security requirements
met by leveraging VPC, VPN,
encryption at-rest and in-
transit, CloudTrail, and
database auditing
Flexible
Interactive
Queries
Predefined
Queries
Surveillance
Analytics
Data Management
Data Movement
Data Registration
Version Management
Amazon S3
Web Applications
Analysts; Regulators
FINRA: Migrating from on-prem to AWS