SlideShare a Scribd company logo
1 of 23
© All rights reserved 2022 2
Basel AWS User Group
re:Invent re:Cap
Data & Analytics
© All rights reserved 2022
Cloudreach Service Lines
Cloudreach
Services
Advisory
Software Platform
Data & Analytics
Platform Development
Application Modernisation
Security Transformation
Migration
Management & Resale
© All rights reserved 2022
Our Cloud Partnerships
UK&I Consulting Partner
of the Year 2019
EMEA GSI Partner of the Year
2022
3x Golden Jackets!
900+ Certifications
15 Competency
Specializations
Data Analytics - Services Specialisation
GCP Security Partner
of the Year 2021
100+ Certifications
12 Competency
Specializations
Gold Data Analytics Competency
Gold Data Platform Competency
Microsoft UK Partner
of the Year 2020
500+ Certifications
12 Gold Certifications
15 Competency
Specializations
© All rights reserved 2022
Marketing
Achieved!
© All rights reserved 2022
The Big Data & Analytics
Announcements
© All rights reserved 2022
Zero-ETL Integration - Aurora with RedShift
Eliminate the need to build and maintain complex data pipelines to perform
extract, transform, and load (ETL) operations
Auto-Copy from S3
Load files from S3 as it arrives
Dynamic data masking
Protect sensitive information stored in your data warehouse and ensure that
only the relevant data is accessible by users based on their roles
Centralised access control with AWS Lake Formation
Simplify governance of data shared from Amazon Redshift and centrally manage
granular access across all data-sharing consumers
Integration with Apache Spark
Build and run Spark applications on Amazon Redshift and Redshift Serverless,
opening up the data warehouse for a broader set of AWS analytics and machine
learning solutions
Streaming Ingestion
Natively ingest hundreds of megabytes of data per second from Amazon Kinesis
Data Streams and Amazon MSK
Redshift
Glue 4.0
New and Updated Engines, More Data Formats, and More
This version of Glue includes Python 3.10 and Apache Spark 3.3.0, plus native
support for the Cloud Shuffle Service Plugin for Spark. It also includes Pandas
support and more.
Glue Data Quality
Analyse your tables and recommend a set of rules automatically based on what it
finds
Glue for Ray
Process large datasets with Python and popular Python libraries, serverless
option for data integration with Ray, a popular new open-source compute
framework that helps you scale Python workloads
Glue
© All rights reserved 2022
Built-in Data Preparation, Real-Time Collaboration, and
Notebook Automation
Improve data quality in minutes with the built-in data preparation capability, edit
the same notebooks with your teams in real time, and automatically convert
notebook code to production-ready jobs.
ML Governance Tools for Amazon SageMaker
Simplify Access Control and Enhance Transparency Over Your ML Projects
Build, Train, and Deploy ML Models Using Geospatial Data
This collection of features offers pre-trained deep neural network (DNN) models
and geospatial operators that make it easy to access and prepare large
geospatial datasets.
Support for Shadow Testing
Compare Inference Performance Between ML Model Variants
Deploying a model in shadow mode lets you conduct a more holistic test by
routing a copy of the live inference requests for a production model to the new
(shadow) model.
Real-Time and Batch Inference support in Data Wrangler
Reuse the data transformation flow which you created in SageMaker Data
Wrangler as a step in Amazon SageMaker inference pipelines.
SageMaker
Automated Data Preparation
Automated data preparation utilises machine learning to infer semantic
information about data and adds it to datasets as metadata about the columns
(fields), making it faster for you to prepare data in order to support natural
language questions.
Paginated Reports
Create and Share Operational Reports at Scale.
This feature allows customers to create and share highly formatted,
personalized reports containing business-critical data to hundreds of thousands
of end-users — without any infrastructure setup or maintenance, up-front
licensing, or long-term commitments.
New API Capabilities to Accelerate Your BI Transformation
New QuickSight API capabilities allow programmatic creation and management
of dashboards, analysis, and templates.
QuickSight
© All rights reserved 2022
Amazon Security Lake
A Purpose-Built Customer-Owned Data Lake Service. This new service
automatically centralises your organisation’s security data from cloud and
on-premises sources into a purpose-built data lake stored in your account.
Step Functions Distributed Map
A Serverless Solution for Large-Scale Parallel Data Processing. The new
distributed map state can launch up to ten thousand parallel workflows to
process data.
Amazon Athena for Apache Spark
Using Spark without having to manually provision the infrastructure. Run
Apache Spark workloads, use Jupyter Notebook as the interface to perform data
processing on Athena, and programmatically interact with Spark applications
using Athena APIs.
AWS Database Migration Service: Fully Managed Schema
Conversion
Streamlines database migrations by making schema assessment and conversion
available inside AWS DMS. You can now plan, assess, convert and migrate under
one central DMS service.
Other New Stuff
Amazon DataZone
Unlock data across organisational boundaries with built-in governance. Discover
and share data at scale across organisational boundaries with governance and
access controls.
Amazon Omics
Store, query, and analyze genomic, transcriptomic, and other omics data and
then generate insights from that data to improve health and advance scientific
discoveries.
AWS Clean Rooms
Securely match, analyse, and collaborate on combined datasets across
organisations without revealing the underlying data.
© All rights reserved 2022
So What is DataZone?
© All rights reserved 2022
Provides the technology infrastructure for pursuing
a data mesh architecture within an organisation
Key components:
Business Data Catalog
Make data assets discoverable and searchable with business
metadata, manageable by Data Stewards via metadata forms
Data Projects
Groups users and data assets within a business context where they
can collaborate and manage their output
Governance & Access Control
Includes self-service data access requests, data producer review
processes, and access control to manage where data assets are
used
Data Portal
A dedicated web portal (outside of the AWS console) for working
with DataZones
Domains
Groupings of the above which enable DataZone to map it’s
governance and access control structure to your organisational
structure
Basically; Data Mesh Tech Layer-as-a-Service Coming Soon
© All rights reserved 2022
So What is Omics?
© All rights reserved 2022
Basically; Genomic Research Infra-as-a-Service
Provides specialised storage and processing workflows for
omics data
Built for Petabyte scale storage and processing
Designed to be used in GDPR and HIPAA regulated
environments
Automatically manages underlying infrastructure for
analytics workflows on omics data
Enables utilisation of omics data via Athena, EMR,
QuickSight, and more
GA
© All rights reserved 2022
So What is Clean Rooms?
© All rights reserved 2022
Provides the plumbing for multiple
organisations to collaborate on their data,
without actually copying/transferring it
Can share any data in your Glue Data
Catalog / S3 Data Lake
Key terms:
Analysis Rules
Basically; Zero Transfer X-Org Analytics Preview
Define what types of SQL queries other collaborators can use with a data asset you associate with a clean room
E.g. limiting which SQL aggregations can be used
Output Restrictions
Define limits on the results other collaborators can receive from a data asset you associate with a clean room
E.g. setting a minimum result value for a COUNT DISTINCT query
Can restrict collaborators access to only rows that match an inner join on their own associated data assets
Can run queries on data whilst it remains encrypted - i.e. encryption in-use!
Optionally logs all queries performed against your data to CloudWatch Logs in your AWS account
Important T&C contractual limit:
You may not use AWS Clean Rooms or any information obtained from your use of AWS Clean Rooms to identify a person or associate such information with an identifiable
person, unless otherwise permitted by the applicable third party contributor of the data.
© All rights reserved 2022
My Favourite Session:
AWS IoT ExpressLink Workshop
© All rights reserved 2022
What is IoT ExpressLink?
Physical hardware for integration into IoT devices, built by AWS partners
Provides AWS connectivity without the need to write your own connectivity
code
Implements AWS security best practices for IoT connectivity
Effectively:
● Offload the undifferentiated heavy lifting of connectivity and
cryptography in IoT devices
● Simplify and accelerate AWS-integrated IoT device and service
development
● Rapid mass onboarding of IoT devices, e.g. via bluetooth
Configured for WiFi or 5G connections using serial AT commands e.g. (in
Python):
send_command(uart, f"AT+CONF SSID={wifi_ssid}")
send_command(uart, f"AT+CONF Passphrase={wifi_key}")
© All rights reserved 2022
We’ll use a freebie from the re:Invent
workshop to look at:
● IoT ExpressLink in use
● AWS IoT Core device shadows
● Wrapping device shadows in a
serverless API with AWS Lambda
● Building an AWS Amplify app on top
of that API
The demo badge contains:
● IoT ExpressLink chip (red circle)
● Raspberry Pi Pico-esque board
● A bunch of widgets!
Hardware details and image source here
Demo Time!
© All rights reserved 2022
IoT Core Device Shadow
© All rights reserved 2022
Basic Serverless Device Shadow API
© All rights reserved 2022
Device Shadow Web App
© All rights reserved 2022 22
Questions?
Perhaps Answers
© All rights reserved 2022 23
Thank you
Cloudreach Tech Blog: https://www.cloudreach.com/en/technical-blog
These Slides: https://bit.ly/3QFl7Xr
Demo Repos: https://bit.ly/3GLt3BH
https://bit.ly/3IRfXWg
Slides

More Related Content

What's hot

Scaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with DatabricksScaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with DatabricksDatabricks
 
AWS reInvent 2022 reCap AI/ML and Data
AWS reInvent 2022 reCap AI/ML and DataAWS reInvent 2022 reCap AI/ML and Data
AWS reInvent 2022 reCap AI/ML and DataChris Fregly
 
Best Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWSBest Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWSAmazon Web Services
 
Building a Modern Data Platform in the Cloud
Building a Modern Data Platform in the CloudBuilding a Modern Data Platform in the Cloud
Building a Modern Data Platform in the CloudAmazon Web Services
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019Randall Hunt
 
Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...
Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...
Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...Amazon Web Services
 
Speed up data preparation for ML pipelines on AWS
Speed up data preparation for ML pipelines on AWSSpeed up data preparation for ML pipelines on AWS
Speed up data preparation for ML pipelines on AWSData Science Milan
 
Accelerating Your Cloud Migration Journey with MAP
Accelerating Your Cloud Migration Journey with MAPAccelerating Your Cloud Migration Journey with MAP
Accelerating Your Cloud Migration Journey with MAPAmazon Web Services
 
ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...
ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...
ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...Amazon Web Services
 
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...Amazon Web Services
 
Amazon SageMaker Deep Dive for Builders
Amazon SageMaker Deep Dive for BuildersAmazon SageMaker Deep Dive for Builders
Amazon SageMaker Deep Dive for BuildersAmazon Web Services
 
Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...
Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...
Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...Amazon Web Services
 
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...Amazon Web Services
 

What's hot (20)

Building-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWSBuilding-a-Data-Lake-on-AWS
Building-a-Data-Lake-on-AWS
 
Ml ops on AWS
Ml ops on AWSMl ops on AWS
Ml ops on AWS
 
Scaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with DatabricksScaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with Databricks
 
AWS-Data-Migration-module3
AWS-Data-Migration-module3AWS-Data-Migration-module3
AWS-Data-Migration-module3
 
SaaS on AWS - ISV challenges
SaaS on AWS - ISV challengesSaaS on AWS - ISV challenges
SaaS on AWS - ISV challenges
 
AWS reInvent 2022 reCap AI/ML and Data
AWS reInvent 2022 reCap AI/ML and DataAWS reInvent 2022 reCap AI/ML and Data
AWS reInvent 2022 reCap AI/ML and Data
 
Best Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWSBest Practices for Building Your Data Lake on AWS
Best Practices for Building Your Data Lake on AWS
 
Building a Modern Data Platform in the Cloud
Building a Modern Data Platform in the CloudBuilding a Modern Data Platform in the Cloud
Building a Modern Data Platform in the Cloud
 
How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019How to Choose The Right Database on AWS - Berlin Summit - 2019
How to Choose The Right Database on AWS - Berlin Summit - 2019
 
Building a Data Lake on AWS
Building a Data Lake on AWSBuilding a Data Lake on AWS
Building a Data Lake on AWS
 
Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...
Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...
Reduce Costs and Build a Strong Operational Foundation with the AWS Migration...
 
Speed up data preparation for ML pipelines on AWS
Speed up data preparation for ML pipelines on AWSSpeed up data preparation for ML pipelines on AWS
Speed up data preparation for ML pipelines on AWS
 
Accelerating Your Cloud Migration Journey with MAP
Accelerating Your Cloud Migration Journey with MAPAccelerating Your Cloud Migration Journey with MAP
Accelerating Your Cloud Migration Journey with MAP
 
ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...
ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...
ABD318_Architecting a data lake with Amazon S3, Amazon Kinesis, AWS Glue and ...
 
Cloud Migration Workshop
Cloud Migration WorkshopCloud Migration Workshop
Cloud Migration Workshop
 
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
Effective Data Lakes: Challenges and Design Patterns (ANT316) - AWS re:Invent...
 
Are you Well-Architected?
Are you Well-Architected?Are you Well-Architected?
Are you Well-Architected?
 
Amazon SageMaker Deep Dive for Builders
Amazon SageMaker Deep Dive for BuildersAmazon SageMaker Deep Dive for Builders
Amazon SageMaker Deep Dive for Builders
 
Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...
Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...
Building Text Analytics Applications on AWS using Amazon Comprehend - AWS Onl...
 
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
An Overview of Best Practices for Large Scale Migrations - AWS Transformation...
 

Similar to Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf

How to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data AnalyticsHow to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data AnalyticsInformatica
 
Confluent_AWS_ImmersionDay_Q42023.pdf
Confluent_AWS_ImmersionDay_Q42023.pdfConfluent_AWS_ImmersionDay_Q42023.pdf
Confluent_AWS_ImmersionDay_Q42023.pdfAhmed791434
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluentconfluent
 
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdfSederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdfJazzy44
 
Best Practices for Cloud Migrations with Zero Disruption with AWS Marketplace
Best Practices for Cloud Migrations with Zero Disruption with AWS MarketplaceBest Practices for Cloud Migrations with Zero Disruption with AWS Marketplace
Best Practices for Cloud Migrations with Zero Disruption with AWS MarketplaceDenodo
 
Benefits of the Azure Cloud
Benefits of the Azure CloudBenefits of the Azure Cloud
Benefits of the Azure CloudCaserta
 
Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...
Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...
Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...Amazon Web Services
 
Module 3 - QuickSight Overview
Module 3 - QuickSight OverviewModule 3 - QuickSight Overview
Module 3 - QuickSight OverviewLam Le
 
101_Customer_Move and Modernize Siebel_07012021.pptx
101_Customer_Move and Modernize Siebel_07012021.pptx101_Customer_Move and Modernize Siebel_07012021.pptx
101_Customer_Move and Modernize Siebel_07012021.pptxBhagavathyPadmanabha1
 
Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017) Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017) Codit
 
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018Amazon Web Services
 
Transforming Enterprise IT - Virtual Transformation Day Feb 2019
Transforming Enterprise IT - Virtual Transformation Day Feb 2019Transforming Enterprise IT - Virtual Transformation Day Feb 2019
Transforming Enterprise IT - Virtual Transformation Day Feb 2019Amazon Web Services
 
Comment développer une stratégie Big Data dans le cloud public avec l'offre P...
Comment développer une stratégie Big Data dans le cloud public avec l'offre P...Comment développer une stratégie Big Data dans le cloud public avec l'offre P...
Comment développer une stratégie Big Data dans le cloud public avec l'offre P...Cloudera, Inc.
 
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...Amazon Web Services
 
Azure intelligent edge solutions overview
Azure intelligent edge solutions overviewAzure intelligent edge solutions overview
Azure intelligent edge solutions overviewCenk Ersoy
 
Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)
Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)
Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)Manoj Kumar
 
Cloud Computing Pascal Walschots
Cloud Computing   Pascal WalschotsCloud Computing   Pascal Walschots
Cloud Computing Pascal WalschotsPWalschots
 
IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...
IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...
IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...Amazon Web Services
 
Azure Overview Csco
Azure Overview CscoAzure Overview Csco
Azure Overview Cscorajramab
 
Benefits of the Azure cloud
Benefits of the Azure cloudBenefits of the Azure cloud
Benefits of the Azure cloudJames Serra
 

Similar to Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf (20)

How to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data AnalyticsHow to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
 
Confluent_AWS_ImmersionDay_Q42023.pdf
Confluent_AWS_ImmersionDay_Q42023.pdfConfluent_AWS_ImmersionDay_Q42023.pdf
Confluent_AWS_ImmersionDay_Q42023.pdf
 
Build real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with ConfluentBuild real-time streaming data pipelines to AWS with Confluent
Build real-time streaming data pipelines to AWS with Confluent
 
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdfSederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
Sederhanakan_integrasi_data_anda_dengan_AWS_Glue_handout.pdf
 
Best Practices for Cloud Migrations with Zero Disruption with AWS Marketplace
Best Practices for Cloud Migrations with Zero Disruption with AWS MarketplaceBest Practices for Cloud Migrations with Zero Disruption with AWS Marketplace
Best Practices for Cloud Migrations with Zero Disruption with AWS Marketplace
 
Benefits of the Azure Cloud
Benefits of the Azure CloudBenefits of the Azure Cloud
Benefits of the Azure Cloud
 
Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...
Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...
Accelerate your journey to AI: IBM Cloud Pak for Data on AWS - DEM18-S - New ...
 
Module 3 - QuickSight Overview
Module 3 - QuickSight OverviewModule 3 - QuickSight Overview
Module 3 - QuickSight Overview
 
101_Customer_Move and Modernize Siebel_07012021.pptx
101_Customer_Move and Modernize Siebel_07012021.pptx101_Customer_Move and Modernize Siebel_07012021.pptx
101_Customer_Move and Modernize Siebel_07012021.pptx
 
Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017) Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017)
 
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018
Realize Value of Your Microsoft Investments - Transformation Day Montreal 2018
 
Transforming Enterprise IT - Virtual Transformation Day Feb 2019
Transforming Enterprise IT - Virtual Transformation Day Feb 2019Transforming Enterprise IT - Virtual Transformation Day Feb 2019
Transforming Enterprise IT - Virtual Transformation Day Feb 2019
 
Comment développer une stratégie Big Data dans le cloud public avec l'offre P...
Comment développer une stratégie Big Data dans le cloud public avec l'offre P...Comment développer une stratégie Big Data dans le cloud public avec l'offre P...
Comment développer une stratégie Big Data dans le cloud public avec l'offre P...
 
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...
Optimize App Performance and Security by Managing Microsoft Workloads on AWS ...
 
Azure intelligent edge solutions overview
Azure intelligent edge solutions overviewAzure intelligent edge solutions overview
Azure intelligent edge solutions overview
 
Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)
Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)
Cloud Computing – Opportunities, Definitions, Options, and Risks (Part-1)
 
Cloud Computing Pascal Walschots
Cloud Computing   Pascal WalschotsCloud Computing   Pascal Walschots
Cloud Computing Pascal Walschots
 
IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...
IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...
IoT Building Blocks: From Edge Devices to Analytics in the Cloud - SRV204 - T...
 
Azure Overview Csco
Azure Overview CscoAzure Overview Csco
Azure Overview Csco
 
Benefits of the Azure cloud
Benefits of the Azure cloudBenefits of the Azure cloud
Benefits of the Azure cloud
 

Recently uploaded

社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token PredictionNABLAS株式会社
 
Solution manual for managerial accounting 8th edition by john wild ken shaw b...
Solution manual for managerial accounting 8th edition by john wild ken shaw b...Solution manual for managerial accounting 8th edition by john wild ken shaw b...
Solution manual for managerial accounting 8th edition by john wild ken shaw b...rightmanforbloodline
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancingmohamed Elzalabany
 
How to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsHow to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsBrainSell Technologies
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证acoha1
 
原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证pwgnohujw
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Klinik Aborsi
 
Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...
Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...
Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...mikehavy0
 
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...Voces Mineras
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...yulianti213969
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchersdarmandersingh4580
 
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.pptRachmaGhifari
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives23050636
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfRobertoOcampo24
 
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Klinik kandungan
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样jk0tkvfv
 
What is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationWhat is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationmuqadasqasim10
 
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...ThinkInnovation
 
obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di Ban...
obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di  Ban...obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di  Ban...
obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di Ban...siskavia95
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证pwgnohujw
 

Recently uploaded (20)

社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction社内勉強会資料_Object Recognition as Next Token Prediction
社内勉強会資料_Object Recognition as Next Token Prediction
 
Solution manual for managerial accounting 8th edition by john wild ken shaw b...
Solution manual for managerial accounting 8th edition by john wild ken shaw b...Solution manual for managerial accounting 8th edition by john wild ken shaw b...
Solution manual for managerial accounting 8th edition by john wild ken shaw b...
 
The Significance of Transliteration Enhancing
The Significance of Transliteration EnhancingThe Significance of Transliteration Enhancing
The Significance of Transliteration Enhancing
 
How to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data AnalyticsHow to Transform Clinical Trial Management with Advanced Data Analytics
How to Transform Clinical Trial Management with Advanced Data Analytics
 
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
如何办理(UPenn毕业证书)宾夕法尼亚大学毕业证成绩单本科硕士学位证留信学历认证
 
原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证原件一样伦敦国王学院毕业证成绩单留信学历认证
原件一样伦敦国王学院毕业证成绩单留信学历认证
 
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
Jual Obat Aborsi Bandung (Asli No.1) Wa 082134680322 Klinik Obat Penggugur Ka...
 
Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...
Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...
Abortion Clinic in Kempton Park +27791653574 WhatsApp Abortion Clinic Service...
 
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
Las implicancias del memorándum de entendimiento entre Codelco y SQM según la...
 
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
obat aborsi Tarakan wa 081336238223 jual obat aborsi cytotec asli di Tarakan9...
 
Bios of leading Astrologers & Researchers
Bios of leading Astrologers & ResearchersBios of leading Astrologers & Researchers
Bios of leading Astrologers & Researchers
 
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI  MANAJEMEN OF PENYAKIT TETANUS.pptMATERI  MANAJEMEN OF PENYAKIT TETANUS.ppt
MATERI MANAJEMEN OF PENYAKIT TETANUS.ppt
 
Displacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second DerivativesDisplacement, Velocity, Acceleration, and Second Derivatives
Displacement, Velocity, Acceleration, and Second Derivatives
 
Formulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdfFormulas dax para power bI de microsoft.pdf
Formulas dax para power bI de microsoft.pdf
 
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
Jual obat aborsi Bandung ( 085657271886 ) Cytote pil telat bulan penggugur ka...
 
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
如何办理(UCLA毕业证书)加州大学洛杉矶分校毕业证成绩单学位证留信学历认证原件一样
 
What is Insertion Sort. Its basic information
What is Insertion Sort. Its basic informationWhat is Insertion Sort. Its basic information
What is Insertion Sort. Its basic information
 
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
Identify Rules that Predict Patient’s Heart Disease - An Application of Decis...
 
obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di Ban...
obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di  Ban...obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di  Ban...
obat aborsi Banjarmasin wa 082135199655 jual obat aborsi cytotec asli di Ban...
 
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
原件一样(UWO毕业证书)西安大略大学毕业证成绩单留信学历认证
 

Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf

  • 1.
  • 2. © All rights reserved 2022 2 Basel AWS User Group re:Invent re:Cap Data & Analytics
  • 3. © All rights reserved 2022 Cloudreach Service Lines Cloudreach Services Advisory Software Platform Data & Analytics Platform Development Application Modernisation Security Transformation Migration Management & Resale
  • 4. © All rights reserved 2022 Our Cloud Partnerships UK&I Consulting Partner of the Year 2019 EMEA GSI Partner of the Year 2022 3x Golden Jackets! 900+ Certifications 15 Competency Specializations Data Analytics - Services Specialisation GCP Security Partner of the Year 2021 100+ Certifications 12 Competency Specializations Gold Data Analytics Competency Gold Data Platform Competency Microsoft UK Partner of the Year 2020 500+ Certifications 12 Gold Certifications 15 Competency Specializations
  • 5. © All rights reserved 2022 Marketing Achieved!
  • 6. © All rights reserved 2022 The Big Data & Analytics Announcements
  • 7. © All rights reserved 2022 Zero-ETL Integration - Aurora with RedShift Eliminate the need to build and maintain complex data pipelines to perform extract, transform, and load (ETL) operations Auto-Copy from S3 Load files from S3 as it arrives Dynamic data masking Protect sensitive information stored in your data warehouse and ensure that only the relevant data is accessible by users based on their roles Centralised access control with AWS Lake Formation Simplify governance of data shared from Amazon Redshift and centrally manage granular access across all data-sharing consumers Integration with Apache Spark Build and run Spark applications on Amazon Redshift and Redshift Serverless, opening up the data warehouse for a broader set of AWS analytics and machine learning solutions Streaming Ingestion Natively ingest hundreds of megabytes of data per second from Amazon Kinesis Data Streams and Amazon MSK Redshift Glue 4.0 New and Updated Engines, More Data Formats, and More This version of Glue includes Python 3.10 and Apache Spark 3.3.0, plus native support for the Cloud Shuffle Service Plugin for Spark. It also includes Pandas support and more. Glue Data Quality Analyse your tables and recommend a set of rules automatically based on what it finds Glue for Ray Process large datasets with Python and popular Python libraries, serverless option for data integration with Ray, a popular new open-source compute framework that helps you scale Python workloads Glue
  • 8. © All rights reserved 2022 Built-in Data Preparation, Real-Time Collaboration, and Notebook Automation Improve data quality in minutes with the built-in data preparation capability, edit the same notebooks with your teams in real time, and automatically convert notebook code to production-ready jobs. ML Governance Tools for Amazon SageMaker Simplify Access Control and Enhance Transparency Over Your ML Projects Build, Train, and Deploy ML Models Using Geospatial Data This collection of features offers pre-trained deep neural network (DNN) models and geospatial operators that make it easy to access and prepare large geospatial datasets. Support for Shadow Testing Compare Inference Performance Between ML Model Variants Deploying a model in shadow mode lets you conduct a more holistic test by routing a copy of the live inference requests for a production model to the new (shadow) model. Real-Time and Batch Inference support in Data Wrangler Reuse the data transformation flow which you created in SageMaker Data Wrangler as a step in Amazon SageMaker inference pipelines. SageMaker Automated Data Preparation Automated data preparation utilises machine learning to infer semantic information about data and adds it to datasets as metadata about the columns (fields), making it faster for you to prepare data in order to support natural language questions. Paginated Reports Create and Share Operational Reports at Scale. This feature allows customers to create and share highly formatted, personalized reports containing business-critical data to hundreds of thousands of end-users — without any infrastructure setup or maintenance, up-front licensing, or long-term commitments. New API Capabilities to Accelerate Your BI Transformation New QuickSight API capabilities allow programmatic creation and management of dashboards, analysis, and templates. QuickSight
  • 9. © All rights reserved 2022 Amazon Security Lake A Purpose-Built Customer-Owned Data Lake Service. This new service automatically centralises your organisation’s security data from cloud and on-premises sources into a purpose-built data lake stored in your account. Step Functions Distributed Map A Serverless Solution for Large-Scale Parallel Data Processing. The new distributed map state can launch up to ten thousand parallel workflows to process data. Amazon Athena for Apache Spark Using Spark without having to manually provision the infrastructure. Run Apache Spark workloads, use Jupyter Notebook as the interface to perform data processing on Athena, and programmatically interact with Spark applications using Athena APIs. AWS Database Migration Service: Fully Managed Schema Conversion Streamlines database migrations by making schema assessment and conversion available inside AWS DMS. You can now plan, assess, convert and migrate under one central DMS service. Other New Stuff Amazon DataZone Unlock data across organisational boundaries with built-in governance. Discover and share data at scale across organisational boundaries with governance and access controls. Amazon Omics Store, query, and analyze genomic, transcriptomic, and other omics data and then generate insights from that data to improve health and advance scientific discoveries. AWS Clean Rooms Securely match, analyse, and collaborate on combined datasets across organisations without revealing the underlying data.
  • 10. © All rights reserved 2022 So What is DataZone?
  • 11. © All rights reserved 2022 Provides the technology infrastructure for pursuing a data mesh architecture within an organisation Key components: Business Data Catalog Make data assets discoverable and searchable with business metadata, manageable by Data Stewards via metadata forms Data Projects Groups users and data assets within a business context where they can collaborate and manage their output Governance & Access Control Includes self-service data access requests, data producer review processes, and access control to manage where data assets are used Data Portal A dedicated web portal (outside of the AWS console) for working with DataZones Domains Groupings of the above which enable DataZone to map it’s governance and access control structure to your organisational structure Basically; Data Mesh Tech Layer-as-a-Service Coming Soon
  • 12. © All rights reserved 2022 So What is Omics?
  • 13. © All rights reserved 2022 Basically; Genomic Research Infra-as-a-Service Provides specialised storage and processing workflows for omics data Built for Petabyte scale storage and processing Designed to be used in GDPR and HIPAA regulated environments Automatically manages underlying infrastructure for analytics workflows on omics data Enables utilisation of omics data via Athena, EMR, QuickSight, and more GA
  • 14. © All rights reserved 2022 So What is Clean Rooms?
  • 15. © All rights reserved 2022 Provides the plumbing for multiple organisations to collaborate on their data, without actually copying/transferring it Can share any data in your Glue Data Catalog / S3 Data Lake Key terms: Analysis Rules Basically; Zero Transfer X-Org Analytics Preview Define what types of SQL queries other collaborators can use with a data asset you associate with a clean room E.g. limiting which SQL aggregations can be used Output Restrictions Define limits on the results other collaborators can receive from a data asset you associate with a clean room E.g. setting a minimum result value for a COUNT DISTINCT query Can restrict collaborators access to only rows that match an inner join on their own associated data assets Can run queries on data whilst it remains encrypted - i.e. encryption in-use! Optionally logs all queries performed against your data to CloudWatch Logs in your AWS account Important T&C contractual limit: You may not use AWS Clean Rooms or any information obtained from your use of AWS Clean Rooms to identify a person or associate such information with an identifiable person, unless otherwise permitted by the applicable third party contributor of the data.
  • 16. © All rights reserved 2022 My Favourite Session: AWS IoT ExpressLink Workshop
  • 17. © All rights reserved 2022 What is IoT ExpressLink? Physical hardware for integration into IoT devices, built by AWS partners Provides AWS connectivity without the need to write your own connectivity code Implements AWS security best practices for IoT connectivity Effectively: ● Offload the undifferentiated heavy lifting of connectivity and cryptography in IoT devices ● Simplify and accelerate AWS-integrated IoT device and service development ● Rapid mass onboarding of IoT devices, e.g. via bluetooth Configured for WiFi or 5G connections using serial AT commands e.g. (in Python): send_command(uart, f"AT+CONF SSID={wifi_ssid}") send_command(uart, f"AT+CONF Passphrase={wifi_key}")
  • 18. © All rights reserved 2022 We’ll use a freebie from the re:Invent workshop to look at: ● IoT ExpressLink in use ● AWS IoT Core device shadows ● Wrapping device shadows in a serverless API with AWS Lambda ● Building an AWS Amplify app on top of that API The demo badge contains: ● IoT ExpressLink chip (red circle) ● Raspberry Pi Pico-esque board ● A bunch of widgets! Hardware details and image source here Demo Time!
  • 19. © All rights reserved 2022 IoT Core Device Shadow
  • 20. © All rights reserved 2022 Basic Serverless Device Shadow API
  • 21. © All rights reserved 2022 Device Shadow Web App
  • 22. © All rights reserved 2022 22 Questions? Perhaps Answers
  • 23. © All rights reserved 2022 23 Thank you Cloudreach Tech Blog: https://www.cloudreach.com/en/technical-blog These Slides: https://bit.ly/3QFl7Xr Demo Repos: https://bit.ly/3GLt3BH https://bit.ly/3IRfXWg Slides