Industrialize Machine Learning Using CI/CD Techniques (FSV304-i) - AWS re:Invent 2018

•

4 likes•1,357 views

The document discusses using CI/CD techniques to industrialize machine learning workflows. It describes a typical ML workflow as being error-prone due to manual processes. The proposed solution is to use AWS services like SageMaker, CodeCommit, and S3 to create a flexible, collaborative, repeatable, auditable and secure ML workflow. A demo of the solution is shown and sample audit questions are provided that could be answered via AWS APIs.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Industrialize Machine Learning
Using CI/CD Techniques
Felix Candelario
Principal Global Accounts
Solutions Architect
AWS
F S V 3 0 4 - i
Harsha Sharma
Senior Technical Account Manager
AWS

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
The need for governing ML workflows
A typical ML workflow
Solution
Demo
Sample audit
Remarks

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Highly regulated
workloads
Marketplace Lenders
• Operating exclusively online
• Niche product focus
• User of non-traditional data sources

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Strong controls
required
Control objectives
• Govern the process while
maintaining flexibility
• Same inputs should result in same
outputs
• Significantly improve auditability
Risks
• Potentially greater exposure to bias
• Reproducibility is hard
• Field in great state of flux

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A typical ML workflow
On-Premise

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A typical ML workflow
Client
On-Premise
Idea
Network
drive
Source code
v13
GPU Cluster
Source code
v13

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A typical ML workflow
Client
On-Premise
Idea
Network
drive
GPU Cluster
Source code
v13

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
A typical ML workflow
Client
On-Premise
Idea
Network
drive
Model weightGPU Cluster Training job
Source code
v1
Source code
v2
Training job
v1.1
Training job
Error prone
• Manual orchestration
• Inputs aren’t versions
• Dataset
• Source code
• Outputs aren’t versions
• Weights
• Gathering evidence is manual
• Who made the change?
• When was the change made?
• What version of data was used?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solution

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solution
Region
AWS Cloud

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solution
Region
AWS Cloud
Source BucketBucket
Container
Training job
Input
Output
Dataset

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Solution
Region
AWS Cloud
BucketBucket
Container
Training job
Input
Output
Solution properties
• Flexible
• Collaborative
• Repeatable
• Auditable
• Secure

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Region
AWS Cloud
Source
Input
Output
Dataset

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Region
AWS Cloud
Source
Input
Output
Dataset
Contents of repo
• buildspec.yaml
• Dockerfile
• manifest.json
• train

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Region
AWS Cloud
Source BucketBucket
Input
Output
Dataset

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Region
AWS Cloud
BucketBucket
Container
Input
Output

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Demo
Region
AWS Cloud
BucketBucket
Container
Training job
Input
Output

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Sample audit
Region
AWS Cloud
Endpoints Inference
table
Apply for
loan
Trained model
Subsystem 1
Subsystem 2
Subsystem 3
Subsystem 4
Sample questions
• What version of the model produced
this inference?
• Who made the latest change to this
model?
• What version of training data was
exposed to this model?

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Remarks
Region
AWS Cloud
BucketBucket
Container
Training job
Input
Output
Solution properties
• Flexible
• Collaborative
• Repeatable
• Auditable
• Secure

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Related breakouts
Thursday, Nov 29
Build, Train, and Deploy ML Models Quickly and Easily with Amazon SageMaker, ft. Intuit
4:00 PM – 5:00 PM | Venetian, Level 3, San Polo 3405
Thursday, Nov 29
Build, Train, and Deploy ML Models with Amazon SageMaker
12:15 PM – 2:30 PM | Bellagio, Level 1, Grand Ballroom 9

Thank you!
© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Felix Candelario
fcandela@amazon.com

What's hot

Modernization involves implementing business processes and technology that provide your business applications with high availability, agility, and elasticity. Nowhere is this more important than in breaking apart the monolith. Modernizing an application as part of a migration can be extremely successful if you follow the AWS migration methodology of “discover, plan, migrate, and optimize” as you move that application to the cloud. In this session, we share what we learned from over 400 successful migrations. We also show you how to virtually break a monolith to a modernized architecture as part of the planning phase and accelerate your migration using container technologies and application discovery tools.

Breaking Up the Monolith While Migrating to AWS (GPSTEC320) - AWS re:Invent 2018

Amazon Web Services

Many enterprises want to drive faster cloud adoption and time to market (TTM) by allowing their developers, who have various skill sets, to onboard onto AWS quickly using self-service. However, enabling security, governance, and compliance while not compromising user experience can be a challenge. In this session, Verizon demonstrates how they use the AWS Service Catalog Connector for ServiceNow to create a robust self-service computing environment while meeting Verizon’s governance and security controls and achieving their goal of migrating > 30% of their applications onto AWS in a short timeframe.

How Verizon is Accelerating Cloud Adoption and Migration with the AWS Service...

Amazon Web Services

In this workshop, learn how to use machine learning to analyze the Deutsche Börse Public Dataset, which consists of trade data aggregated to one-minute intervals from the Tradex and Eurex engines, comprising a variety of equities, funds, and derivative securities. The public dataset provides initial price, lowest price, highest price, final price, and volume for every minute of the trading day, and for every tradeable security. Learn how to apply a variety of ML models to the data to find patterns and methods to predict price movements or identify trends in the market. Also learn how to interact with data staged for analysis in Amazon S3, use AWS Glue to transform data into analysis-ready formats, and use Amazon SageMaker and Amazon EC2 to experiment with a variety of ML models to derive insights from the data.

Market Prediction Using ML: Experiment with Amazon SageMaker and the Deutsche...

Amazon Web Services

The unique global cloud infrastructure offered by AWS helps customers build reliable, available, secure, scalable, and fault-tolerant applications. AWS has more experience operating global cloud infrastructures that enables customers to run business-critical workloads in the public cloud than anyone else. In this session, learn how AWS is continuously enhancing and expanding the AWS global infrastructure through more Regions and Availability Zones, custom hardware, purpose-built global network backbone, and innovative energy management systems to deliver to our customers lower latency, greater reliability, greater scalability, and operational efficiencies.

Foundations of AWS Global Cloud Infrastructure (ARC217) - AWS re:Invent 2018

Amazon Web Services

Amazon Kinesis Video Streams makes it easy to capture live video, play it back, and store it for real-time and batch-oriented ML-driven analytics. In this session, we first dive deep on the top five best practices for getting started and scaling with Amazon Kinesis Video Streams. Next, we demonstrate a streaming video from a standard USB camera connected to a laptop, and we perform a live playback on a standard browser within minutes. We also have on stage members of Amazon Go, who are building the next generation of physical retail store experiences powered by their "just walk out" technology. They walk through the technical details of their integration with Kinesis Video Streams and highlight their successes and difficulties along the way.

Serverless Video Ingestion & Analytics with Amazon Kinesis Video Streams (ANT...

Amazon Web Services

In this session, we explore landing zone considerations as they apply to compliance and auditing. We include such topics as a repeatable approach to SCP and IAM policy creation, internal separation of duty & "need to know", compliance scope ringfencing, Region scoping, scope of impact limitation, and mandatory access control. We review approaches for log and event analytics and log record lifecycle management (including redaction where necessary) and alerting. We also discuss how compliance assessment tools can be deployed in multi-account environments and their output sensibly interpreted. We encourage you to attend the full AWS Landing Zone track, including SEC303. Search for #awslandingzone in the session catalog.

Automate & Audit Cloud Governance & Compliance in Your Landing Zone (ENT315-R...

Amazon Web Services

In this workshop, we build and deploy an end-to-end analytics solution for monitoring the video streaming experience. We integrate an open source video player with Amazon Kinesis Data Streams to capture events in real time. We explore the data available for capture and a variety of use cases: from generating alerts on poor experience to content recommendations based on user behavior. We also show you how this real-time data can be archived in a data lake and further used to generate reports of aggregate performance and experience across a number of dimensions.

Build an AWS Analytics Solution to Monitor the Video Streaming Experience (MA...

Amazon Web Services

Your system is composed of highly decoupled, independent, fast, and modular microservices. But how can they share common configurations, dynamic endpoints, database references, and properly rotate secrets? Based on the size and complexity of your serverless system, you may use AWS Lambda environment variables, AWS Systems Manager's Parameter Store, and the recently announced AWS Secrets Manager. In this session, we showcase the best use cases for each solution, as well as corresponding best practices to achieve the highest standards for security and performance.

Configuration Management and Service Discovery with AWS Lambda (SRV338-R1) - ...

Amazon Web Services

With Amazon Connect, a cloud-based contact center service, businesses can create dynamic contact flows that provide personalized caller experiences by taking history and past context into consideration to anticipate callers’ needs. Join us to learn how customers are executing successful strategies using Amazon Lex to add NLU chatbots into their Amazon Connect customer experience workflows. Learn how using Amazon Lex, an AI service that enables you to create intelligent conversational chatbots that can turn your contact flows into natural conversations using the same technology that powers Amazon Alexa. Learn how to automate repeatable routine tasks such as password resets, order status, and balance inquiries without the need for an agent.

Bring the Power of AI to Your Amazon Connect Contact Center (BAP322-R1) - AWS...

Amazon Web Services

Learn about AWS Security Hub, and how it gives you a comprehensive view of your high-priority security alerts and your compliance status across AWS accounts. See how Security Hub aggregates, organizes, and prioritizes your alerts, or findings, from multiple AWS services, such as Amazon GuardDuty, Amazon Inspector, and Amazon Macie, as well as from AWS Partner solutions. We will demonstrate how you can continuously monitor your environment using compliance checks based on the AWS best practices and industry standards your organization follows.

[NEW LAUNCH!] Introduction to AWS Security Hub (SEC397) - AWS re:Invent 2018

Amazon Web Services

Deploying Your ONNX Deep Learning with Apache MXNet Model Server (AIM413) - A...

Amazon Web Services

In this session, we discuss best practices and approaches for managing your Microsoft Windows-based infrastructure on AWS. We describe the AWS services that can help you manage Windows servers at scale and realize the maximum benefit of the cloud. In addition, we show you how to build simple and effective solutions to manage logging, configuration drift, inventory, licensing, and more. Please join us for a speaker meet-and-greet following this session at the Speaker Lounge (ARIA East, Level 1, Willow Lounge). The meet-and-greet starts 15 minutes after the session and runs for half an hour.

Operationalizing Microsoft Workloads (WIN320) - AWS re:Invent 2018

Amazon Web Services

Edge computing is all about moving compute power to the source of the data instead of having to bring it to the cloud. The edge is a fundamental part of IoT, and it is not only about connecting things to the internet. In this sesssion, we discuss how AWS Greengrass, which is an IoT edge software, can power devices small and large, from a sensor all the way to a wind turbine. With AWS Greengrass, these IoT devices can securely gather data, keep device data in sync, and communicate with each other while still using the cloud for management, analytics, and durable storage. Join us to learn more about the edge of IoT.

Computing at the Edge with AWS Greengrass and Amazon FreeRTOS, ft. General El...

Amazon Web Services

Amazon SageMaker is a powerful tool that enables us to build, train, and deploy at scale our machine learning-based workloads. With help from AWS CI/CD tools, we can speed up this pipeline process. In this talk, we discuss how to integrate Amazon SageMaker into a CI/CD pipeline as well as how to orchestrate with other serverless components. This session is part of re:Invent Developer Community Day, a series led by AWS enthusiasts who share first-hand, technical insights on trending topics.

CI/CD for Your Machine Learning Pipeline with Amazon SageMaker (DVC303) - AWS...

Amazon Web Services

This talk dives into Trustpilot's journey to serverless compute. The journey starts at re:Invent 2016 and follows how the company fast-tracked its adoption within its engineering organization using a "serverless first" engineering principle. A representative from Trustpilot shares lessons learned and insights gained from running over 200 AWS Lambda functions with 12M invocations/day in production. Also covered are fun stories of what helped the company adopt serverless, how to make those stories actionable, a review of architectural patterns, and a discussion of why they choose serverless over traditional compute every day. This session is part of re:Invent Developer Community Day, a series led by AWS enthusiasts who share firsthand technical insights on trending topics.

Serverless:It All Started in Vegas (DVC306) - AWS re:Invent 2018

Amazon Web Services

In today’s environment of increasingly large data sets and resource-intensive algorithmic processing, the challenge of HPC includes keeping pace with the demands of researchers, scientists, engineers, and creative professionals so they can rapidly produce high-value answers to complex questions. AWS HPC solutions deliver significant leaps in compute performance, memory capacity and bandwidth and I/O scalability. The highly customizable computing platform and robust partner ecosystem enable your staff to imagine new approaches so they can fail forward faster, delivering more answers to more questions without the need for costly, on-premises upgrades. This session provides an overview of HPC capabilities on AWS, describes the newest generations of accelerated computing instances, as well as highlighting customer and partner use-cases across industries. Attendees will also learn how the steadily increasing interest in running HPC workloads on the cloud can be combined with the advances in AI/ML to make it a catalyst for sustained innovation in these industries.

High Performance Computing on AWS: Driving Innovation without Infrastructure ...

Amazon Web Services

Most of you can scale your applications on Amazon EC2 without needing any capacity reservations. For those of you who are risk averse and need the additional confidence, we now have Amazon EC2 Capacity Reservations. A Capacity Reservation enables you to reserve Amazon EC2 capacity for any duration without having to commit to a 1- or 3-year term, unlike RIs. In this session, we share the key features and benefits for using Capacity Reservations, and we explain how Capacity Reservations can help you manage capacity flexibly and independently of your financial commitments (RI discounts). We describe different use cases for Capacity Reservations, and we explore how they work with Regional RIs to get you the best of both worlds.

Reserve Amazon EC2 On-Demand Capacity for Any Duration with On-Demand Capacit...

Amazon Web Services

Learn about the latest and hottest features of Amazon Redshift. We’ll deep dive into the architecture and inner workings of Amazon Redshift and discuss how the recent availability, performance, and manageability improvements we’ve made can significantly enhance your user experience. We’ll also share glimpse of what we are working on and our plans for the future. McDonald's will join us to share how they leverage a data lake powered by Redshift, Redshift spectrum and Athena to get quick insights.

What's New with Amazon Redshift ft. McDonald's (ANT350-R1) - AWS re:Invent 2018

Amazon Web Services

Learn how Cox Automotive started its journey with GitHub Enterprise. Hear how the company improved its processes around managing GitHub Enterprise on AWS and its plans to streamline operations even further in the future. Millions of developers and thousands of businesses rely on GitHub to collaborate on code and build better software faster. GitHub Enterprise is the self-hosted solution for businesses that you can deploy and manage in your own secure environment, and what better place to do that than on AWS. This session is brought to you by AWS partner, GitHub.

How Cox Automotive Runs GitHub Enterprise on AWS (ENT356-S) - AWS re:Invent 2018

Amazon Web Services

Improve Accessibility Using Machine Learning (AIM332) - AWS re:Invent 2018

Amazon Web Services

What's hot (20)