The document provides an overview of a presentation on machine learning with ML.NET. It discusses machine learning concepts and workflows, how to use ML.NET to build models with .NET, and integrating ML.NET with other Microsoft technologies like Azure ML and TensorFlow. The agenda includes machine learning, ML.NET, the ML workflow, data preparation, model training, evaluation and deployment, and deep learning.
SQLDay 2021 PL AI Enrichment Azure Search.pptxLuis Beltran
This presentation introduces AI enrichment with Azure Cognitive Search. It discusses how cognitive skills can be used to extract text from images and other unstructured data sources during the indexing process. The key steps involve connecting data sources, applying cognitive skills to enrich documents, and creating a search index to enable query-based access to the enriched content. A demo is provided to illustrate combining Azure Cognitive Search with Azure Blob storage and Cognitive Services to build a skillset that indexes and enriches documents.
Data ANZ - Using database for ML.NET.pptxLuis Beltran
This document provides information about a machine learning conference called Data ANZ, including:
- A schedule for sessions and links to join virtual sessions on their website or mobile app.
- Details about prize drawings for attendees and requirements to participate.
- Information about dedicated time slots for sponsor sessions and how to join them.
- An agenda for a presentation on using databases for machine learning with ML.NET, including an introduction to machine learning, an overview of ML.NET, connecting data to ML.NET, and a demonstration.
Conversation Learner enables you to build task-oriented conversational interfaces that learn directly from example dialogues. Conversation Learner applies machine learning behind the scenes to decrease manual coding of dialogue control logic. Conversation Learner empowers developers to rapidly iterate to get to production quality and improve dialogues across multiple conversational channels.
• How to build a web app, and explore how Azure Search integrates DocumentDB?
• Build a Mobile App with Azure Mobile Apps and DocumentDB
• How it fits into the greater Azure ecosystem?
• Things you can do to get the best out of DocumentDB
Convert your sketches to code with microsoft aiMohit Chhabra
Sketch 2 Code is a tool that uses Microsoft AI to convert hand-drawn sketches into code. It utilizes various machine learning and cognitive services like computer vision, natural language processing and speech recognition through Azure Machine Learning and the Microsoft AI Platform. The tool allows for agile development of AI models and rapid experimentation through Jupyter notebooks and support for frameworks like TensorFlow, Caffe2 and Keras.
This document provides an overview of a Firebase workshop. It introduces Firebase services like Authentication, Realtime Database, Cloud Functions, and Cloud Storage. For each service, it discusses what it is, how it works, use cases, and recommendations for using it correctly. It also includes examples using Github repos. The workshop concludes with a Kahoot quiz and a request for feedback.
This document provides an overview of Firebase products and services for building, releasing, and engaging apps. It discusses how software engineers can use Firebase to accelerate development by using managed backend services like Cloud Firestore, Cloud Functions, and Authentication. It also explains how Firebase helps monitor apps and engage users through products like Crashlytics, Performance Monitoring, Predictions, and Cloud Messaging. Finally, it demonstrates some use cases of Firebase like enabling photo sharing, adding in-app chat, and optimizing ads based on user behavior.
SQLDay 2021 PL AI Enrichment Azure Search.pptxLuis Beltran
This presentation introduces AI enrichment with Azure Cognitive Search. It discusses how cognitive skills can be used to extract text from images and other unstructured data sources during the indexing process. The key steps involve connecting data sources, applying cognitive skills to enrich documents, and creating a search index to enable query-based access to the enriched content. A demo is provided to illustrate combining Azure Cognitive Search with Azure Blob storage and Cognitive Services to build a skillset that indexes and enriches documents.
Data ANZ - Using database for ML.NET.pptxLuis Beltran
This document provides information about a machine learning conference called Data ANZ, including:
- A schedule for sessions and links to join virtual sessions on their website or mobile app.
- Details about prize drawings for attendees and requirements to participate.
- Information about dedicated time slots for sponsor sessions and how to join them.
- An agenda for a presentation on using databases for machine learning with ML.NET, including an introduction to machine learning, an overview of ML.NET, connecting data to ML.NET, and a demonstration.
Conversation Learner enables you to build task-oriented conversational interfaces that learn directly from example dialogues. Conversation Learner applies machine learning behind the scenes to decrease manual coding of dialogue control logic. Conversation Learner empowers developers to rapidly iterate to get to production quality and improve dialogues across multiple conversational channels.
• How to build a web app, and explore how Azure Search integrates DocumentDB?
• Build a Mobile App with Azure Mobile Apps and DocumentDB
• How it fits into the greater Azure ecosystem?
• Things you can do to get the best out of DocumentDB
Convert your sketches to code with microsoft aiMohit Chhabra
Sketch 2 Code is a tool that uses Microsoft AI to convert hand-drawn sketches into code. It utilizes various machine learning and cognitive services like computer vision, natural language processing and speech recognition through Azure Machine Learning and the Microsoft AI Platform. The tool allows for agile development of AI models and rapid experimentation through Jupyter notebooks and support for frameworks like TensorFlow, Caffe2 and Keras.
This document provides an overview of a Firebase workshop. It introduces Firebase services like Authentication, Realtime Database, Cloud Functions, and Cloud Storage. For each service, it discusses what it is, how it works, use cases, and recommendations for using it correctly. It also includes examples using Github repos. The workshop concludes with a Kahoot quiz and a request for feedback.
This document provides an overview of Firebase products and services for building, releasing, and engaging apps. It discusses how software engineers can use Firebase to accelerate development by using managed backend services like Cloud Firestore, Cloud Functions, and Authentication. It also explains how Firebase helps monitor apps and engage users through products like Crashlytics, Performance Monitoring, Predictions, and Cloud Messaging. Finally, it demonstrates some use cases of Firebase like enabling photo sharing, adding in-app chat, and optimizing ads based on user behavior.
Introduction to Azure Functions.
An event-based serverless compute experience to accelerate your development. Scale based on demand and pay only for the resources you consume.
Elements of DDD with ASP.NET MVC & Entity Framework Code First v2Enea Gabriel
The document discusses Domain-Driven Design (DDD) with ASP.NET MVC and Entity Framework Code First. It covers challenges with traditional architecture like layers coupling and where to implement business rules. DDD is presented as a new default architecture where the database is not the primary focus, layers are loosely coupled, and business rules are within the application domain. Today's tools like ASP.NET MVC, dependency injection frameworks, and Entity Framework Code First are discussed. A demo is presented and conclusions recommend focusing on analyzing dependencies, designing the domain, and doing refactoring and unit testing.
The document discusses ADO.Net Data Services (Astoria) which enables exposing and consuming data as RESTful web services. It provides an overview of creating and hosting data services from various data sources, exploring the services using HTTP and consuming them from various client applications like web and desktop apps. Key concepts covered are entity data model, OData protocol, CRUD operations, querying and various client libraries.
What is going on - Application diagnostics on Azure - TechDays FinlandMaarten Balliauw
We all like building and deploying cloud applications. But what happens once that’s done? How do we know if our application behaves like we expect it to behave? Of course, logging! But how do we get that data off of our machines? How do we sift through a bunch of seemingly meaningless diagnostics? In this session, we’ll look at how we can keep track of our Azure application using structured logging, AppInsights and AppInsights analytics to make all that data more meaningful.
In this session, we will take a deep-dive into the DevOps process that comes with Azure Machine Learning service, a cloud service that you can use to track as you build, train, deploy and manage models. We zoom into how the data science process can be made traceable and deploy the model with Azure DevOps to a Kubernetes cluster.
At the end of this session, you will have a good grasp of the technological building blocks of Azure machine learning services and can bring a machine learning project safely into production.
Learn how to build a powerful social messaging app that leverages a range of AWS services. In this demo-heavy workshop, we show how you can build an app using Apple Swift and the AWS Mobile Hub. This is a step-by-step journey where you configure and add components to your architecture, then modify and test your components. In the end, you will have a mobile app with a backend running on AWS.
AWS DevDay San Francisco, June 21, 2016.
Presenter: John Burry, Senior Manager, Solutions Architecture
Dockerize your ML Models Data Science Summit.pptxicebeam7
This presentation discusses dockerizing machine learning models. It shows how an ML model in .pkl format can be served via an HTTP REST API using a WSGI HTTP server running in a container. The container also includes an HTTP server that connects to a database backend. A tunnel allows user requests to reach the container and interact with the ML model.
It is difficult to deploy interloop Kubernetes development in current state. Know these open-source projects that can save us from the burden of various tools and help in deploying microservices on Kubernetes cluster without saving secrets in a file.
This session will describe and demo methods to connect the Intel Edison to Amazon AWS in order to create a versatile IoT structure. The Intel Edison is a powerful system on chip module, the size of a postage stamp with powerful on board processing. It can be used as a sensor hub to gather data, a control board for actuators, and a gateway to connect to the cloud. When combined with the powerful services offered by AWS it can form the basis for many IoT solutions.
AWS DevDay San Francisco, June 21, 2016.
Presenter: Martin Kronberg, Intel oT Evengelist
The document discusses microservices architecture on Azure and provides an overview of microservices, containers, Docker, Service Fabric, and other Azure services for developing and deploying microservices applications. It covers the evolution of application architecture styles, benefits of microservices, and how technologies like Docker and Service Fabric on Azure can be used to build, deploy and manage microservices applications at scale. Examples of JSON files for deployment and links for further reading are also included.
Elements of DDD with ASP.NET MVC & Entity Framework Code FirstEnea Gabriel
The document discusses elements of Domain-Driven Design (DDD) when building applications with ASP.NET MVC and Entity Framework Code First. It emphasizes modeling the application around the business domain, using tools like EF Code First and dependency injection frameworks to decouple layers and enable unit testing. The presentation provides an overview of DDD concepts and techniques for applying them with ASP.NET MVC and EF Code First to build loosely coupled, testable architectures focused on the business domain.
MongoDB World 2019: Building Flexible and Secure Customer Applications with M...MongoDB
For enterprise software companies like Unqork, the NoSQL structure of MongoDB Atlas is a critical part of their infrastructure and business. This presentation will outline how MongoDB Atlas supports Unqork’s no-code, drag-and-drop infrastructure by providing a flexible and secure data environment.
Build a Text Enabled Keg-orator Robot with Alexa, AWS IoT & AWS LambdaAmazon Web Services
Learn how to build a text enabled robot that will take your beer order, serve your pint, and notify you when it is ready, all while keeping an eye on your consumption so that you wake up on time the next morning. In this demo-heavy workshop, we will use the Zipwhip Texterator as the platform on which we will show you how to use Alexa, AWS Lambda, and AWS IoT to build the ultimate beer serving device.
AWS DevDay San Francisco, June 21, 2016.
Presenter: John Rotach, SDE, AWS IoT
Firebase is a mobile and web app development platform owned by Google that provides tools and services to help developers build high-quality apps. It started as a startup called Envolve in 2011 that provided real-time data syncing across devices. After being acquired by Google in 2014, Firebase expanded its offerings and now integrates with various Google services. Firebase provides tools to help with app development, testing, analytics, cloud services, and more.
This document provides an overview of Azure Logic Apps. It defines Logic Apps as a workflow engine that allows for easy scaling and integration of Azure services without code. Logic Apps use connectors, triggers, and workflows (conditions and actions) to automate tasks and business processes. Examples of Logic App use cases include processing files uploaded to FTP and importing data into SQL Server, processing RSS feeds and sending summary emails, and creating tickets in Dynamics CRM from incoming emails. The document demonstrates building Logic Apps with triggers like schedules and HTTP requests, and includes actions like outputting to Google Drive and sending emails. It also provides references for additional Logic App integration scenarios.
Generating insights from IoT data using Amazon Machine LearningAmazon Web Services
If you knew the state of everything in the world, and could apply logic on top of the data, what problems could you solve?
AWS IoT Services help you collect and send data to the cloud, make it easy to load and analyse that information, and provide the ability to manage your devices, so you can focus on developing applications that fit your needs.
Amazon Machine Learning is a service that makes it easy for developers of all skill levels to use machine learning technology. Amazon Machine Learning provides visualisation tools and wizards that guide you through the process of creating machine learning (ML) models without having to learn complex ML algorithms and technology.
In this webinar, Neeraj explains how you can use AWS IoT and Amazon Machine Learning together to build smart IoT applications. We will demonstrate how to setup Amazon Machine Learning, create and train Machine Learning models for your applications. We will then use these models in our IoT Applications, in real time.
Learning objectives:
- Understand why you may use Amazon Machine Learning with IoT and how to set it up.
- Understand how to use IoT Rules Engine
Como construir suas aplicações escaláveis sem servidoresAlexandre Santos
This document discusses building scalable applications without servers using serverless architectures. It provides an overview of serverless concepts and components like AWS Lambda, API Gateway, DynamoDB, S3, and Cognito. It then demonstrates how to build a serverless web application including authentication and authorization using these services. The demo application architecture shows how unauthenticated and authenticated APIs can be secured using Cognito to obtain temporary credentials to call Lambda functions that interact with DynamoDB through API Gateway. Other options for authentication and authorization are also briefly discussed.
Everything you always wanted to know about API Management (but were afraid to...Massimo Bonanni
Azure API Management is an Azure service that allows developers to implement a consistent and secure access layer to their APIs. It provides features like throttling to prevent DOS attacks, JWT token validation for security, and a developer portal for API documentation and testing. The key components of API Management include the API gateway, publisher portal, developer portal, and policies for pre/post processing requests. Products are used to surface APIs to developers through subscriptions.
Sample Codes: https://github.com/davegautam/dotnetconfsamplecodes
Presentation on How you can get started with ML.NET. If you are existing .NET Stack Developer and Wanna use the same technology into Machine Learning, this slide focuses on how you can use ML.NET for Machine Learning.
Deep Learning in the Cloud at Scale: A Data Orchestration StoryAlluxio, Inc.
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
Deep Learning in the Cloud at Scale: A Data Orchestration Story
Mickey Zhang, Software Engineer (Microsoft)
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
Introduction to Azure Functions.
An event-based serverless compute experience to accelerate your development. Scale based on demand and pay only for the resources you consume.
Elements of DDD with ASP.NET MVC & Entity Framework Code First v2Enea Gabriel
The document discusses Domain-Driven Design (DDD) with ASP.NET MVC and Entity Framework Code First. It covers challenges with traditional architecture like layers coupling and where to implement business rules. DDD is presented as a new default architecture where the database is not the primary focus, layers are loosely coupled, and business rules are within the application domain. Today's tools like ASP.NET MVC, dependency injection frameworks, and Entity Framework Code First are discussed. A demo is presented and conclusions recommend focusing on analyzing dependencies, designing the domain, and doing refactoring and unit testing.
The document discusses ADO.Net Data Services (Astoria) which enables exposing and consuming data as RESTful web services. It provides an overview of creating and hosting data services from various data sources, exploring the services using HTTP and consuming them from various client applications like web and desktop apps. Key concepts covered are entity data model, OData protocol, CRUD operations, querying and various client libraries.
What is going on - Application diagnostics on Azure - TechDays FinlandMaarten Balliauw
We all like building and deploying cloud applications. But what happens once that’s done? How do we know if our application behaves like we expect it to behave? Of course, logging! But how do we get that data off of our machines? How do we sift through a bunch of seemingly meaningless diagnostics? In this session, we’ll look at how we can keep track of our Azure application using structured logging, AppInsights and AppInsights analytics to make all that data more meaningful.
In this session, we will take a deep-dive into the DevOps process that comes with Azure Machine Learning service, a cloud service that you can use to track as you build, train, deploy and manage models. We zoom into how the data science process can be made traceable and deploy the model with Azure DevOps to a Kubernetes cluster.
At the end of this session, you will have a good grasp of the technological building blocks of Azure machine learning services and can bring a machine learning project safely into production.
Learn how to build a powerful social messaging app that leverages a range of AWS services. In this demo-heavy workshop, we show how you can build an app using Apple Swift and the AWS Mobile Hub. This is a step-by-step journey where you configure and add components to your architecture, then modify and test your components. In the end, you will have a mobile app with a backend running on AWS.
AWS DevDay San Francisco, June 21, 2016.
Presenter: John Burry, Senior Manager, Solutions Architecture
Dockerize your ML Models Data Science Summit.pptxicebeam7
This presentation discusses dockerizing machine learning models. It shows how an ML model in .pkl format can be served via an HTTP REST API using a WSGI HTTP server running in a container. The container also includes an HTTP server that connects to a database backend. A tunnel allows user requests to reach the container and interact with the ML model.
It is difficult to deploy interloop Kubernetes development in current state. Know these open-source projects that can save us from the burden of various tools and help in deploying microservices on Kubernetes cluster without saving secrets in a file.
This session will describe and demo methods to connect the Intel Edison to Amazon AWS in order to create a versatile IoT structure. The Intel Edison is a powerful system on chip module, the size of a postage stamp with powerful on board processing. It can be used as a sensor hub to gather data, a control board for actuators, and a gateway to connect to the cloud. When combined with the powerful services offered by AWS it can form the basis for many IoT solutions.
AWS DevDay San Francisco, June 21, 2016.
Presenter: Martin Kronberg, Intel oT Evengelist
The document discusses microservices architecture on Azure and provides an overview of microservices, containers, Docker, Service Fabric, and other Azure services for developing and deploying microservices applications. It covers the evolution of application architecture styles, benefits of microservices, and how technologies like Docker and Service Fabric on Azure can be used to build, deploy and manage microservices applications at scale. Examples of JSON files for deployment and links for further reading are also included.
Elements of DDD with ASP.NET MVC & Entity Framework Code FirstEnea Gabriel
The document discusses elements of Domain-Driven Design (DDD) when building applications with ASP.NET MVC and Entity Framework Code First. It emphasizes modeling the application around the business domain, using tools like EF Code First and dependency injection frameworks to decouple layers and enable unit testing. The presentation provides an overview of DDD concepts and techniques for applying them with ASP.NET MVC and EF Code First to build loosely coupled, testable architectures focused on the business domain.
MongoDB World 2019: Building Flexible and Secure Customer Applications with M...MongoDB
For enterprise software companies like Unqork, the NoSQL structure of MongoDB Atlas is a critical part of their infrastructure and business. This presentation will outline how MongoDB Atlas supports Unqork’s no-code, drag-and-drop infrastructure by providing a flexible and secure data environment.
Build a Text Enabled Keg-orator Robot with Alexa, AWS IoT & AWS LambdaAmazon Web Services
Learn how to build a text enabled robot that will take your beer order, serve your pint, and notify you when it is ready, all while keeping an eye on your consumption so that you wake up on time the next morning. In this demo-heavy workshop, we will use the Zipwhip Texterator as the platform on which we will show you how to use Alexa, AWS Lambda, and AWS IoT to build the ultimate beer serving device.
AWS DevDay San Francisco, June 21, 2016.
Presenter: John Rotach, SDE, AWS IoT
Firebase is a mobile and web app development platform owned by Google that provides tools and services to help developers build high-quality apps. It started as a startup called Envolve in 2011 that provided real-time data syncing across devices. After being acquired by Google in 2014, Firebase expanded its offerings and now integrates with various Google services. Firebase provides tools to help with app development, testing, analytics, cloud services, and more.
This document provides an overview of Azure Logic Apps. It defines Logic Apps as a workflow engine that allows for easy scaling and integration of Azure services without code. Logic Apps use connectors, triggers, and workflows (conditions and actions) to automate tasks and business processes. Examples of Logic App use cases include processing files uploaded to FTP and importing data into SQL Server, processing RSS feeds and sending summary emails, and creating tickets in Dynamics CRM from incoming emails. The document demonstrates building Logic Apps with triggers like schedules and HTTP requests, and includes actions like outputting to Google Drive and sending emails. It also provides references for additional Logic App integration scenarios.
Generating insights from IoT data using Amazon Machine LearningAmazon Web Services
If you knew the state of everything in the world, and could apply logic on top of the data, what problems could you solve?
AWS IoT Services help you collect and send data to the cloud, make it easy to load and analyse that information, and provide the ability to manage your devices, so you can focus on developing applications that fit your needs.
Amazon Machine Learning is a service that makes it easy for developers of all skill levels to use machine learning technology. Amazon Machine Learning provides visualisation tools and wizards that guide you through the process of creating machine learning (ML) models without having to learn complex ML algorithms and technology.
In this webinar, Neeraj explains how you can use AWS IoT and Amazon Machine Learning together to build smart IoT applications. We will demonstrate how to setup Amazon Machine Learning, create and train Machine Learning models for your applications. We will then use these models in our IoT Applications, in real time.
Learning objectives:
- Understand why you may use Amazon Machine Learning with IoT and how to set it up.
- Understand how to use IoT Rules Engine
Como construir suas aplicações escaláveis sem servidoresAlexandre Santos
This document discusses building scalable applications without servers using serverless architectures. It provides an overview of serverless concepts and components like AWS Lambda, API Gateway, DynamoDB, S3, and Cognito. It then demonstrates how to build a serverless web application including authentication and authorization using these services. The demo application architecture shows how unauthenticated and authenticated APIs can be secured using Cognito to obtain temporary credentials to call Lambda functions that interact with DynamoDB through API Gateway. Other options for authentication and authorization are also briefly discussed.
Everything you always wanted to know about API Management (but were afraid to...Massimo Bonanni
Azure API Management is an Azure service that allows developers to implement a consistent and secure access layer to their APIs. It provides features like throttling to prevent DOS attacks, JWT token validation for security, and a developer portal for API documentation and testing. The key components of API Management include the API gateway, publisher portal, developer portal, and policies for pre/post processing requests. Products are used to surface APIs to developers through subscriptions.
Sample Codes: https://github.com/davegautam/dotnetconfsamplecodes
Presentation on How you can get started with ML.NET. If you are existing .NET Stack Developer and Wanna use the same technology into Machine Learning, this slide focuses on how you can use ML.NET for Machine Learning.
Deep Learning in the Cloud at Scale: A Data Orchestration StoryAlluxio, Inc.
Data Orchestration Summit 2020 organized by Alluxio
https://www.alluxio.io/data-orchestration-summit-2020/
Deep Learning in the Cloud at Scale: A Data Orchestration Story
Mickey Zhang, Software Engineer (Microsoft)
About Alluxio: alluxio.io
Engage with the open source community on slack: alluxio.io/slack
“A broad category of applications and technologies for gathering, storing, analyzing, sharing and providing access to data to help enterprise users make better business decisions” -Gartner
Automated machine learning (automated ML) automates feature engineering, algorithm and hyperparameter selection to find the best model for your data. The mission: Enable automated building of machine learning with the goal of accelerating, democratizing and scaling AI. This presentation covers some recent announcements of technologies related to Automated ML, and especially for Azure. The demonstrations focus on Python with Azure ML Service and Azure Databricks.
The document provides an overview of SQL Server 2008 business intelligence capabilities including SQL Server Analysis Services (SSAS) for online analytical processing (OLAP) cubes and data mining models. Key capabilities covered include new aggregation designer, simplified cube/dimension wizards in SSAS, improved time series and cross-validation algorithms in data mining, and the ability to use Excel as both an OLAP cube and data mining client and model creator.
The document discusses SQL Server 2008 data mining capabilities. It provides an overview of data mining concepts and scenarios, demonstrates the data mining lifecycle process using SQL Server tools, and highlights new features in SQL Server 2008 such as improved time series algorithms and holdout support for model validation. Resources for learning more about SQL Server data mining are also listed.
The document discusses SQL Server 2008 data mining capabilities. It provides an overview of data mining concepts and scenarios, demonstrates the data mining lifecycle process using SQL Server tools, and highlights new features in SQL Server 2008 such as improved time series algorithms and holdout support for model validation. Resources for learning more about SQL Server data mining are also listed.
This document provides an overview of querying and manipulating data using Entity Framework in .NET. It discusses Entity Framework concepts like Entity Data Models, Code First development, inheritance hierarchies, and querying. The document also covers ADO.NET connections, Entity Framework performance, and transactions. Key topics include creating EF data models, implementing POCO objects, querying with DbContext, and loading related data using lazy and eager loading.
Microsoft Entity Framework is an object-relational mapper that allows developers to work with relational data as domain-specific objects, and provides automated CRUD operations. It supports various databases and provides a rich query capability through LINQ. Compared to LINQ to SQL, Entity Framework has a full provider model, supports multiple modeling techniques, and continuous support. The Entity Framework architecture includes components like the entity data model, LINQ to Entities, Entity SQL, and ADO.NET data providers. Code First allows defining models and mapping directly through code.
This document discusses Oracle's machine learning capabilities. It provides an overview of the types of machine learning algorithms available in Oracle such as classification, clustering, regression, and time series analysis. It also describes how machine learning can be used directly from SQL and integrated with Oracle Autonomous Database and Oracle Database to build and deploy models. New algorithms like XGBoost and features for enhanced prediction are highlighted.
Machine Learning with ML.NET and Azure - Andy CrossAndrew Flatters
- The document discusses machine learning and ML.NET. It begins with an introduction of the speaker and their background in machine learning.
- Key topics that will be covered include machine learning, ML.NET, Parquet.NET, using machine learning in production, and relevant Azure tools for data and machine learning.
- Examples provided will demonstrate sentiment analysis, finding patterns in taxi fare data, image recognition, and more to illustrate machine learning algorithms and best practices.
201906 02 Introduction to AutoML with ML.NET 1.0Mark Tabladillo
ML.NET 1.0 release is the first major milestone of a great journey that started in May 2018 when we released ML.NET 0.1 as open source. ML.NET is an open-source and cross-platform machine learning framework for .NET developers. Using ML.NET, developers can leverage their existing tools and skillsets to develop and infuse custom AI into their applications by creating custom machine learning models for common scenarios like Sentiment Analysis, Recommendation, Image Classification and more.
“Automated ML” is a collection of new technologies from Microsoft to enhance the data science development process. Still in preview, Auto ML for ML.NET 1.0 will be demonstrated in a Deep Learning Virtual Machine running Windows Server 2016. Code examples are in C# and run in Visual Studio Community 2019.
This presentation is the second of four related to ML.NET and Automated ML. The presentation will be recorded with video posted to this YouTube Channel: http://bit.ly/2ZybKwI
Simplifying the Creation of Machine Learning Workflow Pipelines for IoT Appli...ScyllaDB
This document discusses using ScyllaDB as the data store for machine learning workflow pipelines processing IoT device data on Kubernetes. It describes SmartDeployAI's goal of creating reusable AI/ML pipelines and the challenges of previous approaches using Cassandra. ScyllaDB allows building cloud native ML pipelines that can efficiently run multiple workflows on Kubernetes and store model metadata, hyperparameters, and inference results for real-time analysis of IoT sensor data. Examples of computer vision pipelines for object detection and scene parsing are provided.
This document provides an overview and agenda for a session on SQL Server 2008 Data Mining. It discusses the objectives of understanding and learning the core functionality of SQL Server 2008 Data Mining. The session will examine what data mining is, compare cubes to data mining, demonstrate the data mining lifecycle process, and showcase new functionality in SQL Server 2008 such as improved time series algorithms and cross-validation capabilities. Data mining in SQL Server 2008 can leverage familiar Excel 2007 tools and supports the full data mining cycle from data understanding to deployment.
- The document provides a resume for Chandrajit Samanta including contact details, objectives, skills, experience and details of past roles. It summarizes his extensive experience with SQL Server databases, developing ETL processes in SQL Server Integration Services, data modeling, and building cubes and writing MDX queries in SQL Server Analysis Services. It details over 10 years of experience in database development, administration, and business intelligence roles for various companies.
Data mining by example forecasting and cross prediction using microsoft time ...Shaoli Lu
This document discusses using Microsoft Time Series for forecasting and cross prediction. It describes setting up a data mining project in SQL Server to create a time series mining structure and model using sales history from AdventureWorksDW. The model is processed, viewed, and queried to forecast trends and make predictions. Additional data can be added to the model using DMX, and cross prediction is demonstrated by creating new queries and mining structures. Microsoft Time Series provides an intuitive way to perform time series forecasting and analysis in SQL Server data mining projects.
This document discusses object-relational mapping (ORM) frameworks and the ADO.NET Entity Framework architecture specifically. It provides an overview of ORM and why it was developed, describes the key components of the Entity Framework including its programming model and mapping tools. It then discusses how the Entity Framework compiles mappings to generate bidirectional query and update views between the object and relational models to enable updating the database from object changes. The document evaluates the performance of the Entity Framework's mapping compiler.
Machine Learning on the Microsoft StackLynn Langit
This document provides an overview of machine learning solutions, including on-premise options using Excel add-ins, SQL Server, and R Studio, as well as cloud solutions on Azure and Predixion. It defines common machine learning roles and algorithms, discusses the R programming language, and compares features of the different solutions such as required infrastructure, complexity, costs, and capabilities.
Similar to PL SQLDay Machine Learning- Hands on ML.NET.pptx (20)
The document discusses improving reading fluency through pronunciation assessment using Microsoft's Speech Studio and Speech SDK. It provides links to demos and documentation for developing apps with pronunciation assessment capabilities, describes result parameters, and shows a demo of a mobile app using this feature. The presenter thanks the audience and provides links to learn more about pronunciation assessment and their own profile.
This document discusses machine learning and deep learning concepts like convolutional neural networks. It provides an overview of ML.NET, an open source machine learning framework, and shows how to build and train models with ML.NET including training a deep learning model to classify images into categories like rock, paper, or scissors. Examples of loading data, defining the model architecture, training the model, exporting it and using it for predictions are provided.
BI LATAM Summit 2022 - Creación de soluciones de automatización serverless-...Luis Beltran
Este documento describe una solución serverless para automatizar el procesamiento diario de webinars, incluyendo la emisión de diplomas y la recopilación de retroalimentación de los asistentes. La solución procesa los webinars cada 24 horas, genera diplomas para los asistentes y los envía junto con un enlace de encuesta. La retroalimentación recibida se almacena y genera reportes visuales para los organizadores del evento. La arquitectura incluye funciones serverless en Azure que procesan los datos de Teams, Graph API, blobs de al
CEIAAIT - Fundamentos y Aplicaciones de Deep Learning.pdfLuis Beltran
Este documento presenta una introducción al aprendizaje profundo y sus aplicaciones. Explica conceptos clave como redes neuronales, aprendizaje automático, aprendizaje profundo y sus diferencias. También describe cómo funcionan los modelos de aprendizaje profundo, incluidas las redes neuronales convolucionales y sus usos comunes en visión por computadora. Además, menciona ejemplos de aplicaciones de aprendizaje profundo en áreas como vehículos autónomos, asistentes de voz y reconocimiento facial, entre otras. Finalmente,
Computo en la Nube con Azure - AI Gaming Panama.pptxLuis Beltran
Este documento introduce los conceptos básicos de Azure. Explica que la nube es la entrega de servicios de cómputo a través de Internet, como servidores virtuales, almacenamiento y bases de datos. La nube ofrece escalabilidad y acceso a recursos casi ilimitados de forma económica, sin necesidad de adquirir hardware. Azure proporciona más de 100 servicios populares como almacenamiento, máquinas virtuales, redes y servicios cognitivos.
5StarsConf - Serverless Machine Learning con Azure Functions y ML.NET .pptxLuis Beltran
Este documento presenta sobre el uso de Azure Functions y ML.NET para implementar soluciones de machine learning sin servidores. Explica brevemente ML.NET, Azure Functions y cómo integrarlos mediante bindings y triggers para crear aplicaciones impulsadas por eventos que ejecutan modelos de ML cuando ocurren ciertos eventos. Finaliza con una demostración de esta integración.
Este documento presenta una introducción a Machine Learning. La agenda incluye una presentación del curso y los profesores, una introducción a Machine Learning, trabajando con Azure Machine Learning y conclusiones. Se explican conceptos clave como inteligencia artificial, machine learning, deep learning y marcos de trabajo como Azure Cognitive Services y Azure Machine Learning.
Latino NET - Integrando WhatsApp en nuestras apps .NET con Twilio.pptxLuis Beltran
Este documento proporciona información sobre WhatsApp, Twilio y Azure Functions. Explica que WhatsApp es una aplicación de mensajería gratuita, y que Twilio ofrece una API para enviar y recibir mensajes de WhatsApp de forma programática. También describe cómo Azure Functions permite ejecutar código de forma serverless mediante triggers de eventos.
Netcoreconf 2021 Realidad mixta en apps móviles con Azure Spatial Anchors y ...Luis Beltran
Este documento describe cómo usar Azure Spatial Anchors y Xamarin para crear aplicaciones de realidad mixta en dispositivos móviles. Explica qué es Xamarin, Azure Spatial Anchors y la realidad aumentada, y proporciona una demostración de cómo crear, localizar y compartir anclas espaciales entre dispositivos usando estas tecnologías.
ATG Puebla - El cementerio de Microsoft.pptxLuis Beltran
El documento presenta un resumen de varios productos de Microsoft que han sido descontinuados a lo largo del tiempo, incluyendo Microsoft Band, Windows Phone y Lumia, MSN Messenger, Encarta, CodePlex, Windows Movie Maker, Microsoft Kinect, Microsoft TechNet, Windows 10 IoT Core, Microsoft XNA y Visual Basic. Explica brevemente las funciones y años de cada producto.
Real NET Docs Show - Serverless Machine Learning v3.pptxLuis Beltran
Slides of my presentation about Serverless Machine Learning using Azure Functions, Twilio APIs, and Cognitive Services for text and image processing of WhatsApp messages at .NET Docs Show weekly community event organized by Microsoft
XamarinExpertDay - Creating PDF files in mobile apps with PdfSharpCore and Mi...Luis Beltran
Luis Beltrán discusses options for creating PDF files in mobile apps using .NET, including PdfSharp and MigraDoc, which allow creation and modification of PDFs. He highlights PdfSharpCore and MigraDocCore, which are open source and free ports of these libraries that work with Xamarin and .NET Standard, allowing developers to generate PDFs in both Android and iOS apps. Beltrán then provides a demo and thanks attendees for their time.
Latam Space Week - Clasificación de rocas espaciales por medio de IA.pptxLuis Beltran
Este documento describe cómo la inteligencia artificial puede ayudar a los astronautas a clasificar rocas espaciales recolectadas. Explica que la IA puede extraer características como el texto, tamaño y color de imágenes de rocas y luego asociar esas características con diferentes tipos de rocas para predecir la clasificación de nuevas rocas desconocidas. El proceso involucra obtener datos, extraer características, buscar asociaciones y predecir tipos de rocas. También recomienda bibliotecas de Python como Mat
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Infrastructure Challenges in Scaling RAG with Custom AI modelsZilliz
Building Retrieval-Augmented Generation (RAG) systems with open-source and custom AI models is a complex task. This talk explores the challenges in productionizing RAG systems, including retrieval performance, response synthesis, and evaluation. We’ll discuss how to leverage open-source models like text embeddings, language models, and custom fine-tuned models to enhance RAG performance. Additionally, we’ll cover how BentoML can help orchestrate and scale these AI components efficiently, ensuring seamless deployment and management of RAG systems in the cloud.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
3. Luis Beltrán
• Researcher - Tomas Bata University in Zlín, Czech Republic.
• Lecturer - Tecnológico Nacional de México en Celaya,
Mexico.
• Xamarin, Azure and Artificial Intelligence
@darkicebeam
luis@luisbeltran.mx
4. AGENDA
• Machine Learning
• ML.NET
• ML Workflow with Ml.NET
• Data
• Train
• Evaluate
• Save model
• Consume model
• Deep Learning
• MLOps
SQLDay 2021
5. Objectives
• Build, train, evaluate, and consume machine learning algorithms in your .NET apps
using ML.NET.
• Understand how TensorFlow (and ONNX) models can be integrated into a pipeline
for deep learning.
• Set up model lifecycle automation using MLOps.
SQLDay 2021
7. Artificial Intelligence
The ability of a computer to perform tasks
commonly associated with intelligent beings
(reason, discover meaning, generalize, learn
from past experience)
Artificial Intelligence
• Typically starts as rule or
logic-based system
• Traditional AI techniques
can be difficult to scale
SQLDay 2021
8. Machine Learning
Machine Learning
Getting computers to make predictions
without being explicitly programmed
• Computers find patterns in
data and learn from
experience to act on new
data
• Used to solve problems
that are difficult or
impossible to solve with
rules-based programming
SQLDay 2021
10. Artificial Intelligence vs. Machine Learning
Artificial Intelligence
Machine Learning
Rules
Data
Data
Answers
Answers
Rules
SQLDay 2021
11. Deep learning
Deep Learning
Subset of ML based on
artificial neural networks
which imitate the way the
human brain learns, thinks,
and processes data.
• Neural networks form
many layers
• Scenarios include image
classification, object
detection, speech
recognition, NLP
SQLDay 2021
12. AI + ML + Deep learning
Artificial
Intelligence
Machine Learning
Deep
Learning
SQLDay 2021
13. Mapping business problems to ML Tasks
What
problem
are you
looking to
solve?
Find outliers
• Anomaly detection
Predict a
number
• Regression
• Forecasting
Find
relationships
• Clustering
Categorize
items
• Binary
classification
• Multiclass
classification
•Image
classification
Make
suggestions
•Recommendation
SQLDay 2021
14. Machine Learning Workflow
Prepare the data Evaluate
Train Deploy
Model Training Model Consumption
Inferencing
Get the data
SQLDay 2021
15. Get and prepare the data
Data Source Pipeline Environment Data exploration
SQLDay 2021
17. Train the model
Previous Grade (A-F) Hours Studied Pass
B 5 Y
D 2 N
A 20 Y
Features Label / Target
F(PreviousGrade, HoursStudied)
=
Pass
Model
SQLDay 2021
24. Machine learning landscape
At Microsoft
Training custom models Model consumption Requires ML knowledge
ML.NET Yes Yes - ML.NET, TensorFlow, ONNX No
Azure Cognitive Services Limited to some services Yes – consume via API/SDK No
Azure ML Yes Yes – register models & consume via
web service
Somewhat
WinML No Yes - ONNX No
SQLDay 2021
26. An open source and cross-platform
machine learning framework for .NET
Windows Linux macOS
SQLDay 2021
27. Built for
.NET
Can use existing
C# and F# skills to
integrate ML into
.NET apps
Data science &
ML experience
not required
Developers
SQLDay 2021
28. ML.NET Tooling + AutoML
ML.NET API
(Microsoft.ML)
AutoML.NET API
(Microsoft.ML.AutoML)
Model Builder ML.NET CLI
SQLDay 2021
29. Model Builder & ML.NET CLI
• Easily build custom ML models with AutoML
• Generates code for training and consumption
• Model Builder
• Currently in Visual Studio only (ships with VS 16.6)
• Integration with Azure ML (image classification)
• ML.NET CLI
• Cross platform
SQLDay 2021
30. Supported ML tasks in ML.NET
Classification Regression Image classification
Anomaly detection
Forecasting
Object detection
Clustering Recommendation
Ranking
SQLDay 2021
31. Integration with other ML tech @ Microsoft
Azure Cognitive Services
(Custom Vision)
Train
Image classification or
object detection
Consume
In .NET app using
ML.NET
Export
To ONNX
Azure Cognitive Services – Custom Vision
SQLDay 2021
32. Integration with other ML tech @ Microsoft
Train
Using Azure AutoML
Start in Model Builder
Choose Scenario, Training
Environment, & Data
Consume
In .NET app using Model
Builder & ML.NET
Azure Machine Learning
Azure ML
Model Builder in VS
SQLDay 2021
33. Integration with other ML tech @ Microsoft
Train
Using ML.NET
Consume
With WinML in Windows
Desktop Apps
WinML
Export
To ONNX
WinML
SQLDay 2021
34. • Want to stay in .NET ecosystem for Machine Learning
• Don’t want to worry about low-level complexities of ML
• Want to train a custom model
• Want to consume a pre-trained model
When should you use ML.NET?
When you…
SQLDay 2021
36. • We will create a web app that allows users to input in data about a taxi
trip and returns how much they will pay (taxi fare).
• Regression task (value prediction scenario)
• App details:
• Train regression model in .NET core console app with given dataset
• Consume model in ASP.NET Core web app
Problem to solve: Taxi fare prediction
SQLDay 2021
38. • MLContext = starting point for all ML.NET operations
• Provides ways to create components for
• Data preparation
• Feature engineering
• Training
• Prediction
• Model evaluation
• Logging
• Execution control
• Seeding
MLContext
SQLDay 2021
39. Task
1. Add the Microsoft.ML NuGet package to your console project
2. Initialize a new MLContext in your console app
SQLDay 2021
40. Data in ML.NET represented as IDataView
IDataView
High-dimensional Lazy + memory efficient Immutable
SQLDay 2021
41. • DataViewSchema = Data schema of IDataView = set of columns, their
names, types, & other annotations
• Before loading data, must define how schema of data will look (column
names & column types)
• Use class definitions to define IDV schemas
Data schema
Class definition of
schema
Dataset
Label SepalLength SepalWidth PetalLength PetalWidth
Iris-setosa 5.1 3.5 1.4 0.2
Iris-versicolor 7.0 3.2 4.7 1.4
Iris-setosa 4.9 3.0 1.5 0.1
…
IDataView
SQLDay 2021
42. File loaders
• Load data from sources like text,
binary, and image files to IDV
• Can load from single or multiple
files
• Supported:
• Text: .csv, .tsv, .txt
• Images: .png, .jpg, .bmp
Data loaders & sources
Database loaders
• Load and train data directly
from relational database
• Supports:
• SQL Server, Azure SQL Database,
Oracle, SQLite, PostgreSQL,
Progress, IBM DB2, + many more
Other sources
• Load from Enumerable (in-
memory collections)
• Supports:
• JSON/XML
• Everything else
SQLDay 2021
43. Task
1. Create class for Model Input based on the provided taxi trip dataset
2. Load data from file to IDataView
SQLDay 2021
44. Preparing your data
Filter data Convert data types Normalize the data
Split data Feature engineering
SQLDay 2021
46. Normalization
• Min-Max
• Binning
• Mean variance
Missing Values
• Indicate
• Replace
ColumnMapping
• Concatenate
• Copy columns
• Drop columns
Type Conversion
• Convert type
• Map value to
key
• Hash
Text Transforms
• Featurize text
• Remove stop
words
• N-grams
• Word bags
Data transforms
SQLDay 2021
48. IEstimatorChain = Collection of Data Transforms + Algorithms
Training pipeline
IDataView
IEstimatorChain Model
Drop columns Normalize
Naïve Bayes
Algorithm
SQLDay 2021
49. Pipeline executed when Fit() method is called
Fit() the model
ITransformer model = pipeline.Fit(trainingData)
SQLDay 2021
50. Task
1. Split data into train and test datasets
2. Add data transformations to the pipeline
3. Choose an algorithm and add to the pipeline
4. Train the model
SQLDay 2021
51. Evaluation metrics
ML Task
Most common evaluation
metric
Look for
Classification
Binary: Accuracy
Multi-class: Micro-Accuracy
Closer to 1.0, the better the
quality
Regression R-Squared
Closer to 1.0, the better the
quality
Recommendation R-Squared
Closer to 1.0, the better the
quality
Clustering Average Distance Values closer to 0 are better.
Ranking Discounted Cumulative Gains Higher values are better
Anomaly detection Area Under ROC Curve Values closer to 1 are better.
SQLDay 2021
52. Underfitting & Overfitting a model
Underfitting
Model is too simple and can’t
capture the underlying trend of
the data
Overfitting
Model doesn’t generalize well
from training data to unseen
data
To prevent:
• Feature engineering
• Remove noise from data
• Try different algorithms
To prevent:
• More training data
• Remove features
• Cross validation
SQLDay 2021
53. • Training and model evaluation technique
• Folds the data into n-partitions and trains multiple algorithms on these
partitions
• Improves robustness by holding out data from training process
Cross validation
Partition 1 Partition 2 Partition 3 Partition 4 Partition 5
SQLDay 2021
54. • Global and local explanations
• Global = entire model (What features does the model give more importance to?)
• Local = individual predictions (Why was Bob rejected for a loan?)
• Techniques:
Model explainability
Permutation Feature Importance (PFI)
• Used for Classification and Regression models
• Shuffles data one feature at a time and calculates
how much the performance metric of interest
decreases; the larger the change, the more
important the feature
Feature Contribution Calculation (FCC)
• Used for Classification and Regression models
• Shows which features are most influential for a
model’s prediction on a particular and individual
data sample
SQLDay 2021
55. • Provide more training data
• Filter missing values and outliers
• Feature engineering
• Select different features
• Choose a different algorithm
• Tune algorithm hyperparameters
• Cross validation
Improving your model
SQLDay 2021
56. • Use AutoML to speed up the experimentation process
• Use Model Builder in VS or cross-platform ML.NET CLI
Tooling + AutoML
ML Task Tooling Local / Azure AutoML
Text-based classification Model Builder, CLI Local
Value prediction
(Regression)
Model Builder, CLI Local
Image classification Model Builder Local + Azure
Recommendation Model Builder, CLI Local
SQLDay 2021
57. Task
1. Evaluate your model and print out the metrics
2. Optional: Try training with different algorithms to see if your
evaluation metrics change
SQLDay 2021
60. 1. Create model output schema
How to consume model in ML.NET
Task Model Output
Binary
classification
Predicted Label: Class predicted by model (true or false)
Score: Positive score = true, negative score = false
Probability: Probability of having true as label
Multiclass
classification
Predicted Label: Class predicted by model
Score (vector): Scores of all classes; highest score = predicted
label
Regression Score: Predicted value
Recommendation Score: Predicted rating
Clustering
Predicted Label: Closest cluster’s index predicted by model
Score: Distances of data point to clusters’ centroid
Ranking Score: Predicted rank
Anomaly
detection
<Alert (Boolean), Raw Score, P-value (likelihood of anomaly)>
OR
Predicted Label: Anomaly vs. not anomaly predicted by model
Score: Likelihood of anomaly
Forecasting
Forecasting values
Confidence lower bounds
Confidence upper bounds
SQLDay 2021
61. 2. Load your model
How to consume model in ML.NET
SQLDay 2021
62. How to consume model in ML.NET
• Prediction Engine = convenience
API for making single
predictions
Make single predictions
Prediction Engine
• Prediction Engine not thread-
safe
• Use dependency injection +
Prediction Engine Pool in multi-
threaded apps (e.g. web apps
and services)
• Creates ObjectPool of
PredictionEngine objects for
application use
Make single predictions scalable
Prediction Engine Pool
• Takes in data, makes the
transformations (such as,
making predictions), and
outputs the data
• Can load unknown data into
IDataView, use Transform to
predict, receive IDataView of
predicted values, and use
GetColumn to get the Prediction
column
Make batch predictions
Transform
3. Choose one of the below:
SQLDay 2021
64. Task
1. Load the model from a file to the web app
2. Create a Prediction Engine
3. Use the model and prediction engine to make predictions on new
sample data (e.g. consume the model)
SQLDay 2021
67. Deep Learning
• Deep learning is a subfield of Machine Learning
concerned with algorithms inspired by the structure
and function of the brain called artificial neural
networks.
• It is exceptionally effective in discovering patterns.
• Algorithms learn through a multi-layered hierarchy.
• If you supply the system with tons of information, it
will begin to understand and respond in helpful
ways.
SQLDay 2021
68. Deep learning has an inbuilt automatic multi stage feature learning
process that learns rich hierarchical representations (i.e. features).
Low-level
features
Mid-level
features
Output (e.g. exterior,
interior)
High-level
features
Trainable
Classifier
SQLDay 2021
69. • Image
Pixel Edge Texture Motif Part Object
• Text
Character Word Word-group Clause Sentence Story
• Each module in Deep Learning transforms its input representation into a
higher-level one, in a way similar to human cortex.
Low Level
Features
Mid Level
Features Output
High
Level
Features
Trainable
Classifier
Input
SQLDay 2021
71. Convolution
Input Image Convolved Image
(Feature Map)
a b c d
e f g h
i j k l
m n o p
w1 w2
w3 w4
Filter
h1 h2
ℎ1 = 𝑓 𝑎 ∗ 𝑤1 + 𝑏 ∗ 𝑤2 + 𝑒 ∗ 𝑤3 + 𝑓 ∗ 𝑤4
ℎ2 = 𝑓 𝑏 ∗ 𝑤1 + 𝑐 ∗ 𝑤2 + 𝑓 ∗ 𝑤3 + 𝑔 ∗ 𝑤4
SQLDay 2021
73. Pooling
• Max pooling: reports the maximum output within a rectangular
neighborhood.
• Average pooling: reports the average output of a rectangular
neighborhood.
1 3 5 3
4 2 3 1
3 1 1 3
0 1 0 4
MaxPool with 2X2 filter with
stride of 2
Input Matrix Output Matrix
4 5
3 4
SQLDay 2021
74. Convolutional Neural Network
Feature Extraction Architecture
64
64
128
128
256
256
256
512
512
512
512
512
512
Filter
Max
Pool
Fully Connected
Layers
Living Room
Bed Room
Kitchen
Bathroom
Outdoor
Maxpool
Output
Vector
SQLDay 2021
76. Deep learning in ML.NET
• Model Training
Image Classification API
• Train custom image
classification models via
Image Classification API
• Uses transfer learning
• Built on TensorFlow.NET
• Can use local GPU for
training
• Model Consumption
ML.NET API
• Consume pre-trained
TensorFlow and ONNX
models
Model Training
Model Builder in VS
• Train custom image
classification models
• Can train locally or in Azure
(Azure ML)
SQLDay 2021
77. Task
• Task: Local image training with Image Classification API
SQLDay 2021
82. Main program
Loading data for
supervised learning
(images include tags)
Training and Validation sets
Load pipeline:
Images loaded in memory
Training options:
ImageClassificationTrainer
chosen, based on the
InceptionV3 architecture
Training pipeline:
Trying to predict a
category
Both pipelines are combined
SQLDay 2021
83. Perform training
Model precision is validated
using validation dataset
Model Metrics calculated
Test the classification model using the new images
Prepare new images for validation
Export the model
Consume the model
SQLDay 2021
84. ConsumingModel
Load a previously trained
classification model and prepare test
images that were not used before in
the training and validation stages
ClassifyImages: Test the model with new images
SQLDay 2021
92. Machine Learning Operations (MLOps) applies DevOps principles &
practices (e.g. continuous integration, delivery, and deployment) to the
ML process
What is ML Ops?
SQLDay 2021
AI = machines imitating human abilities and behaviors
Classification:
Categorize e-mail as spam or not spam
Automatically add labels to GitHub issues
Predict of sentiment of a comment is positive or negative
Diagnose if a patient has a disease or not
Categorize hotel reviews into location, price, cleanliness, etc.
Regression
Predict the price of a house based on house features
Predict sales of a product based on advertising budgets
Predict number of goals a player will scare in upcoming match based on previous perf
Recommendation
Recommend products to shoppers
Recommends movies or songs to users
Forecasting
Predict future sales based on past sales
Predictive maintenance
Weather forecasting
Product demand forecasting
Anomaly detection
Identify fraudulent transactions
Alert when there are predicted network intrusions
Find abnormal cluster of patients
Clustering
Customer segmentation
Ranking
Search results ranking
Data Source
Database
Files
Pipeline
Streaming
Batch
Environment
On-Premises
Cloud
Exploratory Data Analysis
Visualization
Cleanup
Validation
Date exploration
Distribution of the data
What are the ranges (age?)
Data cleaning
Remove null values
Visualization
Try to identify patterns in your data
Training a model consists of providing data examples to an algorithm that learns the mappings from the inputs to outputs. This is like a function that maintains an internal state. You can then reuse this model to make predictions on new data.
Metrics – These are there to see whether your model is performing accurately
Explainability – How easy is it to interpret the model
Training effort – Does it take too long to prepare the data for the model or does it take too long to train the model? If it takes a week to gather the data for a particular model maybe it might be best to trade accuracy for ease of use and training.
Deployment can mean many things
Deployment can mean uploading a file
This also means that you can then deploy the model to multiple deployment targets
ML is very experimental / trial and error
Preparing the data
Training
Evaluating is an art as much as a science
You can’t just say, “classification, use Naïve Bayes”. Your data has a lot to do with what works and doesn’t work.
Scikit Learn – Traditional / Classical Machine Learning
TensorFlow – ML Deep Learning
PyTorch – ML Deep Learning
Keras – Higher level API for building ML that can use a variety of backends to actually build the models and computations
Classification – sentiment analysis, Categorizing e-mail as spam or not spam, classify law documents
Regression – price prediction, predict sales of product based on advertising budgets
Forecasting - Predicting future sales based on past sales, Predictive maintenance, Weather forecasting
Anomaly detection - Identifying transactions that are potentially fraudulent., Learning patterns that indicate that a network intrusion has occurred., Finding abnormal clusters of patients., Checking values entered into a system.
IDV = Flexible, efficient way of describing tabular data
The IDataView component provides a very efficient, compositional processing of tabular data (columns and rows) especialy made for machine learning and advanced analytics applications. It is designed to efficiently handle high dimensional data and large data sets. It is also suitable for single node processing of data partitions belonging to larger distributed data sets.
Immutable – can’t change it – have to create a copy – not directly accessing the idv and changing it – making new copy of it
Lazy – as needed go through it, not all in memory
https://github.com/dotnet/machinelearning/blob/master/docs/code/IDataViewDesignPrinciples.md
FlowerType is what we’re trying to predict. This becomes the Label property. This Label is the label or target variable. In an IDataView the column names are the names of the properties. The way in which data is loaded is determined by LoadColumn which specifies the index where to find that data point in the file.
https://github.com/dotnet/machinelearning/blob/master/docs/code/IDataViewDesignPrinciples.md
https://github.com/dotnet/machinelearning/blob/master/docs/code/SchemaComprehension.md
https://docs.microsoft.com/en-us/dotnet/machine-learning/resources/transforms#feature-transformations
https://docs.microsoft.com/en-us/dotnet/machine-learning/resources/transforms#feature-transformations
Estimators and transformers: https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetHighLevelConcepts.md#transformer
https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetHighLevelConcepts.md#estimator
Estimators
Untrained transformer
Definition of the operations that are to take place
Transformers
Component that realizes the transformations defined by the estimators
With ML.NET, the same algorithm can be applied to different tasks. For example, Stochastic Dual Coordinated Ascent can be used for Binary Classification, Multiclass Classification, and Regression. The difference is in how the output of the algorithm is interpreted to match the task.
https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-choose-an-ml-net-algorithm
A statistical model or a machine learning algorithm is said to have underfitting when it cannot capture the underlying trend of the data. Its occurrence simply means that our model or the algorithm does not fit the data well enough..
The word overfitting refers to a model that models the training data too well. Instead of learning the genral distribution of the data, the model learns the expected output for every data point.
This is the same a memorizing the answers to a maths quizz instead of knowing the formulas. Because of this, the model cannot generalize. Everything is all good as long as you are in familiar territory, but as soon as you step outside, you’re lost.
Overfitting can be pretty discouraging because it raises your hopes just before brutally crushing them.
https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/train-machine-learning-model-ml-net
PFI measures feature importance by asking the question, “What would the effect on the model be if the values for a feature were set to a random value (permuted across the set of examples)?”
FCC works by determining the amount each feature contributed to the model’s score for that particular data sample
https://docs.microsoft.com/en-us/dotnet/machine-learning/how-to-guides/explain-machine-learning-model-permutation-feature-importance-ml-net
https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetCookBook.md#i-want-to-look-at-my-models-coefficients
https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetCookBook.md#i-want-to-look-at-my-models-coefficients
https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetCookBook.md#i-want-to-look-at-my-models-coefficients
https://github.com/dotnet/machinelearning/blob/master/docs/code/MlNetCookBook.md#how-do-i-look-at-the-local-feature-importance-per-example
Deep learning can be considered as a subset of machine learning. It is a field that is based on learning and improving on its own by examining computer algorithms. While machine learning uses simpler concepts, deep learning works with artificial neural networks, which are designed to imitate how humans think and learn. Until recently, neural networks were limited by computing power and thus were limited in complexity. However, advancements in Big Data analytics have permitted larger, sophisticated neural networks, allowing computers to observe, learn, and react to complex situations faster than humans. Deep learning has aided image classification, language translation, speech recognition. It can be used to solve any pattern recognition problem and without human intervention.
Artificial neural networks, comprising many layers, drive deep learning. Deep Neural Networks (DNNs) are such types of networks where each layer can perform complex operations such as representation and abstraction that make sense of images, sound, and text.
Low-level features are minor details of the image, like lines or dots, that can be picked up by, say, a convolutional filter (for really low-level things) or (for more abstract things like edges). High-level features are built on top of low-level features to detect objects and larger shapes in the image.
Convolutional neural networks use both types of features: the first couple convolutional layers will learn filters for finding lines, dots, curves etc. while the later layers will learn to recognize common objects and shapes.
Convolution is a general purpose filter effect for images
In Convolutional Neural Networks, Filters detect spatial patterns such as edges in an image by detecting the changes in intensity values of the image.
In terms of an image, a high-frequency image is the one where the intensity of the pixels changes by a large amount, whereas a low-frequency image is the one where the intensity is almost uniform. Usually, an image has both high and low frequency components. The high-frequency components correspond to the edges of an object because at the edges the rate of change of intensity of pixel values is high.
Convolution is a simple mathematical operation which is fundamental to many common image processing operators. Convolution provides a way of `multiplying together' two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality.
3. Now when you apply a set of filters on top of that (pass it through the 2nd conv. layer), the output will be activations that represent higher-level features. Types of these features could be semicircles (a combination of a curve and straight edge) or squares (a combination of several straight edges). As you go through the network and go through more conv. layers, you get activation maps that represent more and more complex features.
ML.NET uses TensorFlow through the low-level bindings provided by the TensorFlow.NET library. The TensorFlow.NET library is an open source and low-level API library that provides the .NET Standard bindings for TensorFlow. That library is part of the open source SciSharp stack libraries.
To train image classification models, using the ML.NET API, use the ImageClassification API.
You can also train custom deep learning models in Model Builder. The process is generally the same, but in addition to training locally, you can also leverage Azure to train models in GPU enabled compute instances.