Download to read offline
Uploaded byVarun Manik
Canada DevOps Summit 2020 Presentation Nov_03_2020
Varun Kumar is a senior consultant at Deloitte with over 8 years of experience in cloud engineering and DevOps, holding several AWS certifications and a master's degree in computer science. He specializes in machine learning, cloud migrations, and automation, having recently led a cloud migration project for a media company in Southeast Asia. Varun is also an active technical blogger and trainer, contributing to the community through various platforms and training sessions.
More Related Content
Similar to Canada DevOps Summit 2020 Presentation Nov_03_2020
Canada DevOps Summit 2020 Presentation Nov_03_2020
- 3. Varun Kumar Senior Consultant, CloudEngineering Background Varun Kumar is a Senior Consultant with Deloitte SEA & AWS APN Ambassador. He is a cloud professional over 8 plus years of diversified experience in DevOps. • Varun is also a Research Scholar, published three research papers, including IEEE. His research area & topic was virtualization and Hypervisors. • He is a Technical Blogger, wrote many blogs about AWS cloud, DevOps. DevSecOps, MLOps and in general cloud computing topics. • He is a professional cloud trainer, delivered many training across the globe and share his knowledge toward community which includes collage students, fresher's and Corporate professionals. Please find more details: • https://www.linkedin.com/in/vkmanik/ • https://medium.com/@varunmanik1 • https://www.facebook.com/cloudvirtualization/ • https://twitter.com/varunkmanik/ • https://www.youtube.com/channel/UCcuMPYJ4Osax4528rgqQWrw
- 4. Qualifications and ProfessionalAffiliations Qualification: • Master Degree in Computer Science. Professional Certificate: • AWS Certified Solutions Architect - Professional • AWS Certified Developer – Associate • AWS Certified SysOps - Associate • AWS Certified Solution Architect – Associate • AWS Certified Practitioner • Red Hat Certificate of Expertise in Ansible Automation
- 5. Notable Recent Experiences •Cloud migration lead at a large media company based in South East Asia, providing ongoing consulting, improvement feedback, and cloud expertise to this team for implementation of DevOps practices and migration of their CRM/billing systems to cloud. • Building DevOps, DevSecOps, MLOps capabilities and Automation labs within the organization by implementing various tools for Continuous Integration (CI), Continuous Deployment (CD) and Continuous Assessment (CA) for multiple accounts. • Lead the automation team, Automate daily Cloud tasks in DevOps Tools, Create & configure automation of AWS resources with DevOps best practice to enhance the CICD (Continuous Integration and Continuous Delivery) process for the organization. • Built the DevOps, DevSecOps, DevFinOps practice within the organization and enabled a DevOps champion in every product team, leading to cohesive standards and capabilities in these areas.
- 6. Index - Introduction ofMachine learning • Using Jupyter Notebook 15 min • AWS SageMaker 15 min • Sentiment analysis 15 min • Break 10 min - Overview of a Use case • Model Deployment 15 min • Examining deployed model performance 10 min - Setup New Project Copilot • Setup and Run the Build CI/CD Pipeline 15 min • Setup the Release Pipeline 15 min • Break 10 min • Test Build and Release Pipelines 15 min • Testing the deployed solution 15 min
- 7. What is ML….? •What are the different types in machine learning? • What are the different algorithms available for developing machine learning models? • What tools are available for developing these models? • What are the programming language choices? • What platforms support development and deployment of Machine Learning applications? • How to quickly upgrade your skills in this important area?
- 8. Applications of MachinesLearning Machine Learning is the most rapidly growing technology and according to researchers we are in the golden year of AI and ML. It is used to solve many real-world complex problems which cannot be solved with traditional approach. Following are some real-world applications of ML − • Emotion analysis • Sentiment analysis • Error detection and prevention
- 9. Applications of MachinesLearning • Weather forecasting and prediction • Stock market analysis and forecasting • Speech synthesis • Speech recognition • Customer segmentation • Object recognition • Fraud detection • Fraud prevention • Recommendation of products to customer in online shopping
- 10. Types of MachineLearning Machine Learning Supervised Learning Unsupervised Learning Reinforcement Learning Deep Learning Deep Reinforcement Learning
- 11. Algorithms for SupervisedLearning There are several algorithms available for supervised learning. Some of the widely used algorithms of supervised learning are as shown below : • k-Nearest Neighbours • Decision Trees • Naive Bayes • Logistic Regression • Support Vector Machine
- 12. Sentiment Analysis byAmazon Comprehend Analyze text in real time by using built-in or custom models. With built-in models, you can recognize entities, extract key phrases, detect dominant languages, detect PII, determine sentiment, or analyze syntax. With custom models, you can detect entities that you define, or you can classify documents using your own categories or labels.
- 14.
- 15.
- 16. AWS Sagemaker SageMaker Studio,a machine learning Integrated Development Environment (IDE) for building, training, and debugging models, tracking experiments, deploying models, and monitoring their performance.
- 17.
- 18. SageMaker Benefits andfeatures • Labeling raw data with active learning • Highly accurate training datasets • Fully-managed notebook instances • Highly-optimized machine learning algorithms • One-click training • Deployment without engineering effort
- 19.
- 20.
- 21. Jupyter Notebook • JupyterLab • Note book Creation • File Option • Code vs Markdown • File Upload down load
- 22. Machine Learning Usecase • Machine Learning (and in mathematics) there are often three values that interests us: Mean - The average value Median - The mid point value Mode - The most common value • Random Data Distributions • Linear Regression • Polynomial Regression
- 23.
- 24.
- 25. Deep Learning usesANN Applications Deep Learning has shown a lot of success in several areas of machine learning applications. Self-driving Cars − The autonomous self-driving cars use deep learning techniques. They generally adapt to the ever changing traffic situations and get better and better at driving over a period of time. Speech Recognition − Another interesting application of Deep Learning is speech recognition. All of us use several mobile apps today that are capable of recognizing our speech. Apple’s Siri, Amazon’s Alexa, Microsoft’s Cortena and Google’s Assistant – all these use deep learning techniques. Mobile Apps − We use several web-based and mobile apps for organizing our photos. Face detection, face ID, face tagging, identifying objects in an image – all these use deep learning.
- 26. ML Platform &tools Language Choice • Python • R • Matlab • Octave • Julia • C++ • C IDEs • AWS Sagemaker • R Studio • Pycharm • iPython/Jupyter Notebook • Julia • Spyder • Anaconda • Rodeo • Google –Colab Platforms • Amazon • IBM • Microsoft Azure • Google Cloud • Mlflow
- 27.
- 28.
- 29. Micro Services Deploymentby AWS Copilot Part 2 DevOps
- 30. What is Copilot…? •The AWS Copilot command-line interface (CLI) provides application-first, high-level commands to simplify modeling, creating, releasing, and managing production-ready containerized applications on Amazon ECS from a local development environment. • Provisioned with application templates, infrastructure as code, and CI/CD pipeline options, the AWS Copilot CLI aligns with application workflows that support modern application best practices.
- 31. AWS ECS Copilot (AutoPilot Mode: Infra Creation and DevOps Deployment)
- 32. Copilot Building Block CopilotCLI Services Environment Application Pipelines Manifest File
- 33.
- 34. Open Source DevOpsTools
- 35. Docker Basics Docker Basic 1.Create EC2 Instance 2. Install Docker 3. Pull centos:centos6 4. Create index.html 5. Create Dockerfile 6. Build Docker image
- 36. Services • Creating aService • Choosing a Service Type • Load Balanced Web Service • Backend Service • Config and the Manifest • Deploying a Service • Digging into your Service • What's in your service? • What's your service status? • Where are my service
- 37. Environment • Creating anEnvironment • Deploying a Service • Environment Infrastructure • VPC and Networking • Load Balancers and DNS • Customize your Environment • Digging into your Environment • What's in your environment?
- 38. Application • Creating anApp • Additional App Configurations • App Infrastructure • ECR Repositories • Release Infrastructure • Digging into your App • What's in my application?
- 39. AWS Infrastructure • AWSVPC • Subnets • Security Group • ECS Cluster • ECS Service • Fargate task • Service Discovery • ECR Repositories • Application Load balancer • ACM Certs
- 40. Pipeline • Pipeline structure •GitHub Source • Build Stage • Deploy Stages • Creating a Pipeline in 3 steps • Preparing the pipeline structure. • Committing the generated buildspec.yml. • Creating the actual CodePipeline.
- 42. Final Deployment Testand Validation 1. Push the new version of code in code commit 2. It will automatically deploy the new task with new version 3. At last you can run the DNS ALB on your browser
- 43.
- 44.
- 45. Command Application • app init •app ls • app show • app delete Environment • env init • env ls • env show • env delete Services • svc init • svc ls • svc show • svc logs • svc status • svc package • svc deploy • svc delete • task run Pipeline • pipeline init • pipeline update • pipeline show • pipeline status • pipeline delete • deploy
- 46. Thanks…..! Please find moredetails: • https://www.linkedin.com/in/vkmanik/ • https://medium.com/@varunmanik1 • https://www.facebook.com/cloudvirtualization/ • https://twitter.com/varunkmanik/ • https://www.youtube.com/channel/UCcuMPYJ4Osax4 528rgqQWrw