Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance - 남궁영환 (AWS 데이터 사이언티스트 SA)
•
5 likes•2,316 views
인공지능, 머신 러닝은 비즈니스의 필수 기술이 되고 있습니다. 하지만 여전히 기술을 손쉽게 도입하기엔 어려움이 있습니다. 본 세션에서는 클라우드 상에서 머신러닝 기반 애플리케이션을 손쉽게 구현할 수 있는 AWS의 솔루션들에 대해 살펴봅니다. 다시보기 링크: https://youtu.be/UHvBYgCZiI4
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance - 남궁영환 (AWS 데이터 사이언티스트 SA)
Similar to Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance - 남궁영환 (AWS 데이터 사이언티스트 SA) (20)
More from Amazon Web Services Korea
More from Amazon Web Services Korea (20)
Recently uploaded
Recently uploaded (20)
Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance - 남궁영환 (AWS 데이터 사이언티스트 SA)
- 1. AWS Online Series: Data, Analytics, and ML Edition Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance 남궁영환, AWS 데이터 사이언티스트 SA
- 2. 강연 중 질문하는 방법 Go to Webinar “Questions” 창에 자신이 질문한 내역이 표시됩니다. 기본적으로 모든 질문은 공개로 답변 됩니다만 본인만 답변을 받고 싶으면 (비공개)라고 하고 질문해 주시면 됩니다. 본 컨텐츠는 고객의 편의를 위해 AWS 서비스 설명을 위해 온라인 세미나용으로 별도로 제작, 제공된 것입니다. 만약 AWS 사이트와 컨텐츠 상에서 차이나 불일치가 있을 경우, AWS 사이트(aws.amazon.com)가 우선합니다. 또한 AWS 사이트 상에서 한글 번역문과 영어 원문에 차이나 불일치가 있을 경우(번역의 지체로 인한 경우 등 포함), 영어 원문이 우선합니다. AWS는 본 컨텐츠에 포함되거나 컨텐츠를 통하여 고객에게 제공된 일체의 정보, 콘텐츠, 자료, 제품(소프트웨어 포함) 또는 서비스를 이용함으로 인하여 발생하는 여하한 종류의 손해에 대하여 어떠한 책임도 지지 아니하며, 이는 직접 손해, 간접 손해, 부수적 손해, 징벌적 손해 및 결과적 손해를 포함하되 이에 한정되지 아니합니다. 고지 사항(Disclaimer)
- 3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Agenda • Machine Learning on AWS • AWS Solutions and Machine Learning • Fraud Detection using Machine Learning • Predictive Maintenance using Machine Learning • Summary 3 out of 30
- 4. Machine Learning on AWS
- 5. The vision of AWS for Artificial Intelligence & Machine Learning “ 전세계 모든 데이터 과학자와 개발자가 손쉽게 활용할 수 있는 편리한 인공지능 & 머신러닝 환경을 제공”
- 6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS ML Stack: 가장 깊고 폭넓은 역량과 기술의 집약 ML FRAMEWORKS & INFRASTRUCTURE A I S E R V I C E S REKOGNITION IMAGE POLLY TRANSCRIBE TRANSLATE COMPREHEND L E X REKOGNITION VIDEO Vision Speech Language Chatbots AMAZON SAGEMAKER BUILD TRAIN FORECAST Forecasting TEXTRACT PERSONALIZE Recommendations DEPLOY Pre-built algorithms & notebooks Data labeling (GROUND TRUTH) One-click model training & tuning Optimization (N E O ) One-click deployment & hosting M L S E R V I C E S Frameworks Interfaces Infrastructure EC2 P3 & P3DN EC2 C5 FPGAs GREENGRASS ELASTIC INFERENCE Reinforcement learning Algorithms & models (AWS MARKETPLACE FOR MACHINE LEARNING) (App developers with little knowledge of ML) (ML developers and data scientists) (ML researchers and academics) INFERENTIA 6 out of 30
- 7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AI/ML on AWS: More ML happens on AWS than anywhere else 7 out of 30
- 8. AWS Solutions for Analytics and Machine Learning
- 9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Solutions - Machine Learning & Artificial Intelligence Help you solve common problems and build faster using the AWS platform. Fraud Detection Using Machine Learning Built by AWS Learn how to build an architecture that uses Amazon SageMaker to detect potentially fraudulent activity and flag that activity for review. Predictive Maintenance Using Machine Learning Built by AWS Learn how to build an architecture that uses Amazon SageMaker to detect potential equipment failures and provide recommended actions. AI-Driven Social Media Dashboard Built by AWS Deploy a solution that captures multi-language tweets in near real-time, translate them, and stores both the raw and enriched datasets durably in a data lake. Machine Learning for Telecommunication Built by AWS Learn how to build a framework for an end-to- end machine learning process, including ad-hoc data exploration, data processing and feature engineering, and model training and evaluation. Predictive data science with Amazon SageMaker and a Data Lake on AWS Built by Pariveda and AWS Builds a data lake environment for building, training, and deploying machine learning models with Amazon SageMaker. Media Analysis Solution Built by AWS Learn how to use serverless, AWS-native artificial intelligence services to automatically extract valuable metadata from your media files. and more … and more … 9 out of 30
- 10. Fraud detection using Machine Learning AWS Solutions
- 11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions Why detect fraud? Associated high cost Damaging customer trust To avoid Added operational burden How fraud detection has been typically done today Rules based • Static • Complex • Time Consuming 10 out of 30
- 12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions https://www.slideshare.net/awskorea/aws-sk-kb-aws-summit-seoul-2019 11 out of 30
- 13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions Why use machine learning? • Machine learning (ML) can provide a more flexible approach to fraud detection • ML models are trained to automatically recognize fraud patterns in datasets • With AWS, it is easy to implement and deploy an ML model for fraud detection ü Amazon SageMaker makes it easy to train and deploy ML models quickly https://aws.amazon.com/solutions/fraud-detection-using-machine-learning/ Implementation Guide https://s3.amazonaws.com/solutions-reference/fraud-detection-using-machine-learning/latest/fraud-detection-using-machine-learning.pdf (step by step) 12 out of 30
- 14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions End-to-end solution for fraud detection Automates detection of potentially fraudulent activity Includes model training on a sample dataset Leverage Amazon SageMaker for ML training and deployment Flags fraudulent activity for review and visualize processed events 13 out of 30
- 15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions Solution Architecture Amazon QuickSight Amazon S3 bucket (example dataset) Amazon S3 bucket (processed transactions) Amazon SageMaker Amazon Kinesis Data Firehose AWS Lambda function Amazon CloudWatch Event (time-based) Optional 14 out of 30
- 16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions Amazon SageMaker - Features LABEL Ground Truth BUILD Notebooks TRAIN & TUNE Jobs & Model Tuner EndpointsDEPLOY Training Environment Estimator Prediction Environment Predictor Training Data Trained Model Fraud request response .fit( ) .deploy( ) .predict( ) Notebook Instance endpoint Preprocessed event Amazon SageMaker SDK 15 out of 30
- 17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions Amazon SageMaker: Built-in Algorithms • Linear Learner • Image Classification Algorithm • Object Detection Algorithm • Factorization Machines • Principal Component Analysis (PCA) • K-Means Algorithm • K-Nearest Neighbors (KNN) • Latent Dirichlet Allocation (LDA) • Neural Topic Model (NTM) • Random Cut Forest • XGBoost Algorithm • BlazingText • Sequence2Sequence • DeepAR Forecasting Amazon SageMaker: Training and Deployment • The training code has been deployed to our AWS account • Let’s train the model and deploy the model live using Amazon SageMaker 16 out of 30
- 18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Fraud Detection using Machine Learning - AWS Solutions Enable the processing pipeline • The model is now trained and deployed to an endpoint. • Let’s trigger our AWS Lambda function to start processing real time events Visualizing the Predictions Amazon QuickSight AWS Lambda (Optional) 17 out of 30
- 19. Predictive Maintenance using Machine Learning - Predictive Maintenance at aws marketplace - Predictive Maintenance using Machine Learning (AWS Solutions) - Reference Architecture for Predictive Maintenance
- 20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Predictive Maintenance • Businesses are constantly seeking faster ways to take advantage of the value of sensor-based information and transform it into predictive maintenance insights that people can act on quickly • Captures the condition of industrial equipment in order to identify potential breakdowns before impacting production ü Predicting equipment failure ü Real-time anomaly detection ü Predicting pressure spikes ü Asset health monitoring • Industrial IoT: remote maintenance and monitoring • Smart home and city: device operation management Relevant Use cases Anomaly detection models Input gaussian periodic correlation independent Mahalanobis S-H-ESD One Class SVM Sparse Coding Recommended parameters Breakout LOF Gaussian Process Data Characteristics Algorithm Parameter Output Recommended parameters Recommended parameters Recommended parameters Recommended parameters Recommended parameters Recommended parameters 19 out of 30
- 21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Predictive Maintenance using Machine Learning - AWS Solutions Why use machine learning? • To enable you to execute automated data processing on your dataset • ML models detects potential equipment failures and provides recommended actions • Easy to implement and deploy an ML models with AWS ü Amazon SageMaker makes it easy and quick to train and deploy ML models to predict remaining useful life (RUL) ü Amazon CloudWatch Events, AWS lambda for triggering events https://aws.amazon.com/ko/solutions/predictive-maintenance-using-machine-learning/ Implementation Guide https://docs.aws.amazon.com/ko_kr/solutions/latest/predictive-maintenance-using-machine-learning/welcome.html https://s3.amazonaws.com/solutions-reference/predictive-maintenance-using-machine-learning/latest/predictive-maintenance-using-machine-learning.pdf 20 out of 30
- 22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Predictive Maintenance using Machine Learning - AWS Solutions Customization • By default, ‘turbofan degradation dataset’ used for model training • For modeling of your own dataset you must ü Modify the included notebook to point the model to our dataset ü Convert your dataset to an Apache MXNet Gluon dataset ü Modify the solutions’ AWS Lambda function to process and transform your sensor data during inference • CloudWatch Events rule can be modified to trigger an interval for your specific needs https://aws.amazon.com/ko/solutions/predictive-maintenance-using-machine-learning/ Regional Deployment • Please check the region where Amazon SageMaker is available 21 out of 30
- 23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Predictive Maintenance using Machine Learning - AWS Solutions Solution Architecture Amazon SageMaker MXNet model Amazon SageMaker Batch Transform AWS Lambda function Amazon CloudWatch Event (time-based) Amazon S3 bucket turbofan degradation dataset( ) Amazon SageMaker training instance Amazon SageMaker notebook instance transform create transform job training data trained model and source code model training code testdata predictions 22 out of 30
- 24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reference Architecture for Predictive Maintenance Amazon SNS 에서 anomaly detection 알림 토픽을 생성하고 여러분이 생성한 모델에서 트리거가 일어나도록 설정 8 AWS IoT Analytics 데이터의 분석 결과를 QuickSight 를 이용하여 시각화7 AWS IoT Analytics 에서 AWS IoT SiteWise Data store 로부터 컨테이너 데이터셋 생성 후 이를 Docker c컨테이너에 연결(link). 6 Docker 이미지 빌드 후 Amazon ECR 에 추가 5 AWS IoT SiteWise 에서 생성된 데이터셋을 위해 AWS IoT Analytics로부터 신규 Jupyter 노트북을 생성 (목적: Predictive Maintenance 머신 러닝 모델을 생성) 4 Factory Machines의 모니터링을 위한 메트릭 정의3 AWS IoT SiteWise 내에 뷰(View)를 생성하고 Factory Machines를 Assets 로 정의.2 Factory Machines OPC-UA 서버에 접속하기 위해 AWS IoT SiteWise Gateway 를 배포1 This reference architecture enables you to do Predictive Maintenance using AWS IoT SiteWise and AWS IoT Analytics. Factory AWS Cloud AWS IoT AnalyticsAWS IoT SiteWise AWS IoT Greengrass OPC-UA Factory Machines Amazon QuickSight Jupyter Notebook Amazon ECR docker container Amazon SNS Anomaly Notification 1 2 3 4 5 6 7 8AWS IoT SiteWise Connector Industrial Gateway https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-industrial-PdM-ML-RA.pdf 23 out of 30
- 25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reference Architecture for Predictive Maintenance Amazon SNS 에서 anomaly detection 알림 토픽을 생성하고 여러분이 생성한 모델에서 트리거가 일어나도록 설정 8 AWS IoT Analytics 데이터의 분석 결과를 QuickSight 를 이용하여 시각화7 AWS IoT Analytics 에서 AWS IoT SiteWise Data store 로부터 컨테이너 데이터셋 생성 후 이를 Docker c컨테이너에 연결(link). 6 Docker 이미지 빌드 후 Amazon ECR 에 추가 5 AWS IoT SiteWise 에서 생성된 데이터셋을 위해 AWS IoT Analytics로부터 신규 Jupyter 노트북을 생성 (목적: Predictive Maintenance 머신 러닝 모델을 생성) 4 Factory Machines의 모니터링을 위한 메트릭 정의3 AWS IoT SiteWise 내에 뷰(View)를 생성하고 Factory Machines를 Assets 로 정의.2 Factory Machines OPC-UA 서버에 접속하기 위해 AWS IoT SiteWise Gateway 를 배포1 This reference architecture enables you to do Predictive Maintenance using AWS IoT SiteWise and AWS IoT Analytics. https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-industrial-PdM-ML-RA.pdf Factory AWS Cloud AWS IoT AnalyticsAWS IoT SiteWise AWS IoT Greengrass OPC-UA Factory Machines Jupyter Notebook Amazon ECR docker container Amazon SNS Anomaly Notification 1 2 3 4 5 6 7 8 Amazon Kinesis Data Firehose Amazon S3 Greengrass Connector Kinesis Firehouse AWS IoT SiteWise Connector Industrial Gateway 9 Amazon QuickSight 9 Amazon S3 에 OPC-UA 태그가 없는 데이터를 전송하기 위해 AWS IoT Greengrass 상에서 Kinesis Firehose Greengrass Connector 의 환경 설정을 수행 24 out of 30
- 26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Reference Architecture for Predictive Maintenance Amazon Kinesis Data Analytics 에서 이상 탐지 애플리케이션을 생성8 AWS Athena 데이터의 결과를 QuickSight 를 이용하여 시각화7 AWS IoT GreenGrass Edge Gateway에 앞에서 만든 머신 러닝 모델을 배포 6 Amazon SageMaker를 이용하여 PdM 머신 러닝 모델 구축 5 Amazon S3 Data Lake 의 AWS Glue 데이터 카탈로그에서 Amazon Athena의 쿼리를 이용하여 데이터를 추축 4 Amazon S3 Data Lake에 Factory Machines data를 저장하기 위해 Kinesis Firehose 스트림 생성 3 Factory Machines의 MQTT topics를 이용하여 이벤트를 트리거하기 위해 AWS IoT Core 의 룰을 설정 2 Factory Machines 과 통신하기 위해 AWS IoT GreenGrass Connectors 를 이용하여 AWS IoT Greengrass 환경 설정 수행 1 This reference architecture enables you to do Predictive Maintenance using Machine Learning and/or anomaly detection with Amazon Kinesis Data Analytics https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-industrial-PdM-ML-RA.pdf 9 Kinesis Data Analytics의 결과를 Amazon SNS 가 이상 탐지 결과로 통보할 수 있도록 AWS Lambda 설정을 수행 Factory AWS Cloud AWS IoT Greengrass AWS IoT CoreFactory Machines Amazon SageMaker Amazon QuickSight Amazon Kinesis Data Firehose Amazon S3 Lambda function Amazon Athena Amazon Kinesis Data Analytics Amazon Kinesis Data Streams Amazon Kinesis Data Firehose Amazon SNSAWS Lambda ML Inference 1 2 3 4 5 6 7 8 9 25 out of 30
- 27. Summary AWS Solutions – Machine Learning
- 28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS Solutions – Machine Learning & AI • AWS 솔루션즈 아키텍트에 의한 검증 결과 • 운영 효율성, 안정성, 보안, 비용 효율성을 보장하도록 설계 • 상세 아키텍처, 배포 안내서, 수동/자동 배포용 지침도 함께 제공 • Fraud Detection using Machine Learning ü 잠재적 사기 행위의 탐지 및 검토 활동을 플래그 하는 아키텍처 구축 방법 • Predictive Maintenance using Machine Learning ü 장비의 이상 동작, 멈춤, 고장에 관한 잠재적 징후를 탐지하고 예방 및 조치에 권장하는 작업을 제공하는 아키텍처 구축 방법 “AWS의 플랫폼을 사용하여 공통된 문제의 해결과 빠른 구축을 지원” 27 out of 30
- 29. 감사합니다
- 30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. aws-korea-marketing@amazon.com twitter.com/AWSKorea facebook.com/amazonwebservices.ko youtube.com/user/AWSKorea slideshare.net/awskorea twitch.tv/aws 캠페인 온라인 세미나: Data, Analytics, and ML Edition 참석해주셔서 대단히 감사합니다. 저희가 준비한 내용, 어떻게 보셨나요? 더 나은 세미나를 위하여 설문을 꼭 작성해 주시기 바랍니다.