Transfer learning enables you to use pretrained deep neural networks trained on various large datasets (ImageNet, CIFAR, WikiQA, SQUAD, and more) and adapt them for various deep learning tasks (e.g., image classification, question answering, and more).
Wee Hyong Tok and Danielle Dean share the basics of transfer learning and demonstrate how to use the technique to bootstrap the building of custom image classifiers and custom question-answering (QA) models. You’ll learn how to use the pretrained CNNs available in various model libraries to custom build a convolution neural network for your use case. In addition, you’ll discover how to use transfer learning for question-answering tasks, with models trained on large QA datasets (WikiQA, SQUAD, and more), and adapt them for new question-answering tasks.
Topics include:
An introduction to convolution neural networks and question-answering problems
Using pretrained CNNs and the last fully connected layer as a featurizer (Once the features are extracted, any existing classifier can be used for image classification, using the extracted features as inputs.)
Fine-tuning the pretrained models and adapting them for the new images
Using pretrained QA models trained on large QA datasets (WikiQA, SQUAD) and applying transfer learning for QA tasks
How to use transfer learning to bootstrap image classification and question a...Wee Hyong Tok
#theaiconf SFO 2018
Session by Danielle Dean, WeeHyong Tok
Transfer learning enables you to use pretrained deep neural networks trained on various large datasets (ImageNet, CIFAR, WikiQA, SQUAD, and more) and adapt them for various deep learning tasks (e.g., image classification, question answering, and more).
Wee Hyong Tok and Danielle Dean share the basics of transfer learning and demonstrate how to use the technique to bootstrap the building of custom image classifiers and custom question-answering (QA) models. You’ll learn how to use the pretrained CNNs available in various model libraries to custom build a convolution neural network for your use case. In addition, you’ll discover how to use transfer learning for question-answering tasks, with models trained on large QA datasets (WikiQA, SQUAD, and more), and adapt them for new question-answering tasks.
https://conferences.oreilly.com/artificial-intelligence/ai-ca/public/schedule/detail/68527
Deep learning is receiving phenomenal attention due to breakthrough results in several AI tasks and significant research investment by top technology companies like Google, Facebook, Microsoft, IBM. For someone who has not been introduced to this technology, it may be daunting to learn several concepts such as feature learning, Restricted Boltzmann Machines, Autoencoders, etc all at once and start applying it to their own AI applications. This presentation is the first of several in this series that is intended at practitioners.
This slide set on convolutional neural networks is meant to be supplementary material to the slides from Andrej Karpathy's course. In this slide set we explain the motivation for CNN and also describe how to understand CNN coming from a standard feed forward neural networks perspective. For detailed architecture and discussions refer the original slides. I might post more detailed slides later.
Automatic gender and age classification has become quite relevant in the rise of social media platforms. However, the existing methods have not been completely successful in achieving this. Through this project, an attempt has been made to determine the gender and age based on a frame of the person. This is done by using deep learning, OpenCV which is capable of processing the real-time frames. This frame is given as input and the predicted gender and age are given as output. It is difficult to predict the exact age of a person using one frame due the facial expressions, lighting, makeup and so on so for this purpose various age ranges are taken, and the predicted age falls in one of them. The Adience dataset is used as it is a benchmark for face photos and includes various real-world imaging conditions like noise, lighting etc.
How to use transfer learning to bootstrap image classification and question a...Wee Hyong Tok
#theaiconf SFO 2018
Session by Danielle Dean, WeeHyong Tok
Transfer learning enables you to use pretrained deep neural networks trained on various large datasets (ImageNet, CIFAR, WikiQA, SQUAD, and more) and adapt them for various deep learning tasks (e.g., image classification, question answering, and more).
Wee Hyong Tok and Danielle Dean share the basics of transfer learning and demonstrate how to use the technique to bootstrap the building of custom image classifiers and custom question-answering (QA) models. You’ll learn how to use the pretrained CNNs available in various model libraries to custom build a convolution neural network for your use case. In addition, you’ll discover how to use transfer learning for question-answering tasks, with models trained on large QA datasets (WikiQA, SQUAD, and more), and adapt them for new question-answering tasks.
https://conferences.oreilly.com/artificial-intelligence/ai-ca/public/schedule/detail/68527
Deep learning is receiving phenomenal attention due to breakthrough results in several AI tasks and significant research investment by top technology companies like Google, Facebook, Microsoft, IBM. For someone who has not been introduced to this technology, it may be daunting to learn several concepts such as feature learning, Restricted Boltzmann Machines, Autoencoders, etc all at once and start applying it to their own AI applications. This presentation is the first of several in this series that is intended at practitioners.
This slide set on convolutional neural networks is meant to be supplementary material to the slides from Andrej Karpathy's course. In this slide set we explain the motivation for CNN and also describe how to understand CNN coming from a standard feed forward neural networks perspective. For detailed architecture and discussions refer the original slides. I might post more detailed slides later.
Automatic gender and age classification has become quite relevant in the rise of social media platforms. However, the existing methods have not been completely successful in achieving this. Through this project, an attempt has been made to determine the gender and age based on a frame of the person. This is done by using deep learning, OpenCV which is capable of processing the real-time frames. This frame is given as input and the predicted gender and age are given as output. It is difficult to predict the exact age of a person using one frame due the facial expressions, lighting, makeup and so on so for this purpose various age ranges are taken, and the predicted age falls in one of them. The Adience dataset is used as it is a benchmark for face photos and includes various real-world imaging conditions like noise, lighting etc.
Shou-de Lin is currently a full professor in the CSIE department of National Taiwan University. He holds a BS in EE department from National Taiwan University, an MS-EE from the University of Michigan, and an MS in Computational Linguistics and PhD in Computer Science both from the University of Southern California. He leads the Machine Discovery and Social Network Mining Lab in NTU. Before joining NTU, he was a post-doctoral research fellow at the Los Alamos National Lab. Prof. Lin's research includes the areas of machine learning and data mining, social network analysis, and natural language processing. His international recognition includes the best paper award in IEEE Web Intelligent conference 2003, Google Research Award in 2007, Microsoft research award in 2008, merit paper award in TAAI 2010, best paper award in ASONAM 2011, US Aerospace AFOSR/AOARD research award winner for 5 years. He is the all-time winners in ACM KDD Cup, leading or co-leading the NTU team to win 5 championships. He also leads a team to win WSDM Cup 2016 Champion. He has served as the senior PC for SIGKDD and area chair for ACL. He is currently the associate editor for International Journal on Social Network Mining, Journal of Information Science and Engineering, and International Journal of Computational Linguistics and Chinese Language Processing. He receives the Young Scholars' Creativity Award from Foundation for the Advancement of Outstanding Scholarship and Ta-You Wu Memorial Award.
Our research lines on Model-Driven Engineering and Software EngineeringJordi Cabot
Highlighting some of our research lines (March 2015 Edition)
Learn more about what we do on : http://modeling-languages.com , http://som-research.uoc.edu and http://jordicabot.com
For the full video of this presentation, please visit:
https://www.edge-ai-vision.com/2021/02/introducing-machine-learning-and-how-to-teach-machines-to-see-a-presentation-from-tryolabs/
Facundo Parodi, Research and Machine Learning Engineer at Tryolabs, presents the “Introduction to Machine Learning and How to Teach Machines to See” tutorial at the September 2020 Embedded Vision Summit.
What is machine learning? How can machines distinguish a cat from a dog in an image? What’s the magic behind convolutional neural networks? These are some of the questions Parodi answers in this introductory talk on machine learning in computer vision.
Parodi introduces machine learning and explores the different types of problems it can solve. He explains the main components of practical machine learning, from data gathering and training to deployment. He then focuses on deep learning as an important machine learning technique and provides an introduction to convolutional neural networks and how they can be used to solve image classification problems. Parodi will also touches on recent advancements in deep learning and how they have revolutionized the entire field of computer vision.
“Automatically learning multiple levels of representations of the underlying distribution of the data to be modelled”
Deep learning algorithms have shown superior learning and classification performance.
In areas such as transfer learning, speech and handwritten character recognition, face recognition among others.
(I have referred many articles and experimental results provided by Stanford University)
Recently I gave a talk at UC Berkeley regarding the transition from academia to industry in the context of Machine Learning and Data Science related roles. I based most of my slides on my own transition from being an Astrophysicist to a Machine Learning Expert. I hope this will be useful to many. Feedback is welcome!
Scene classification using Convolutional Neural Networks - Jayani WithanawasamWithTheBest
Scene Classification is used in Convolutional Neural Networks (CNNs). We seek to redefine computer vision as an AI problem, understand the importance of scene classification as well as challenges, and the difference between traditional machine learning and deep learning. Additionally, we discuss CNNs, using caffe for implementing CNNs and importact reosources to imorove.
CNNs
Jayani Withanawasam
Please don’t make me draw!
Lesson learned during the development of a software
to support early analysis of object-oriented systems.
This paper describes the development of a software tool to support rich pictures creation for Object Oriented Analysis (OOA). This software should be useful both as an e-learning tool for bachelor-level students, as well as for practitioners working with agile methodologies. Since the transposition of manual rich picture practise into software proved difficult, we decided to follow a user-centered approach, design and implement a prototype with basic functionalities, then run a usability test with a few students and professionals. The feedback collected in the test validated the design of our prototype, and forced us to re-consider the relationship between concrete examples and abstract concepts in rich pictures and in our tool. This unexpectedly helped us realize how to implement support for behavioral description (i.e. events), an elusive feature before the test. Moreover we gained a deeper insight on programmers’ perspective on their practise.
At a more general level we realized how modern object-oriented development methodologies, such as agile methods, are informed by design, and sometimes assume design skills that programmers do not have or do not value. An important lesson to consider carefully to keep our tool usable.
Building Large-scale Real-world Recommender Systems - Recsys2012 tutorialXavier Amatriain
There is more to recommendation algorithms than rating prediction. And, there is more to recommender systems than algorithms. In this tutorial, given at the 2012 ACM Recommender Systems Conference in Dublin, I review things such as different interaction and user feedback mechanisms, offline experimentation and AB testing, or software architectures for Recommender Systems.
In this talk we explore how to build Machine Learning Systems that can that can learn "continuously" from their mistakes (feedback loop) and adapt to an evolving data distribution.
The youtube link to video of the talk is here:
https://www.youtube.com/watch?v=VtBvmrmMJaI
From Conventional Machine Learning to Deep Learning and Beyond.pptxChun-Hao Chang
In this slide, Deep Learning are compared with Conventional Learning and the strength of DNN models will be explained.
The target audience are people who have the knowledge of Machine Learning or Data Mining but not familiar with Deep Learning.
Crafting Recommenders: the Shallow and the Deep of it! Sudeep Das, Ph.D.
I present a brief review, and an outlook on the rapid changes happening in the field of recommendation engine research on the heels of the deep learning revolution!
ODSC East: Effective Transfer Learning for NLPindico data
Presented by indico co-founder Madison May at ODSC East.
Abstract: Transfer learning, the practice of applying knowledge gained on one machine learning task to aid the solution of a second task, has seen historic success in the field of computer vision. The output representations of generic image classification models trained on ImageNet have been leveraged to build models that detect the presence of custom objects in natural images. Image classification tasks that would typically require hundreds of thousands of images can be tackled with mere dozens of training examples per class thanks to the use of these pretrained reprsentations. The field of natural language processing, however, has seen more limited gains from transfer learning, with most approaches limited to the use of pretrained word representations. In this talk, we explore parameter and data efficient mechanisms for transfer learning on text, and show practical improvements on real-world tasks. In addition, we demo the use of Enso, a newly open-sourced library designed to simplify benchmarking of transfer learning methods on a variety of target tasks. Enso provides tools for the fair comparison of varied feature representations and target task models as the amount of training data made available to the target model is incrementally increased.
Easy path to machine learning (2023-2024)wesley chun
1-hr tech talk introducing Machine Learning and the GCP ML APIs and other Google Cloud developer tools to a technical audience:
Easier onramp to getting into AI/ML by using GCP AI/ML APIs (Vision, Video Intelligence, Natural Language, Speech-to-Text, Text-to-Speech, Translation) backed by single-task pre-trained models found in Vertex AI, AutoML for finetuning those pre-trained models, and other "friends of AI/ML" Google dev tools & platforms that can help: BigQuery (data warehouse & analysis), Cloud SQL+AlloyDB & Firestore (SQL & NoSQL databases), serverless platforms (App Engine, Cloud Functions, Cloud Run), and introducing the Gemini API (from both Google AI and GCP Vertex AI)
Shou-de Lin is currently a full professor in the CSIE department of National Taiwan University. He holds a BS in EE department from National Taiwan University, an MS-EE from the University of Michigan, and an MS in Computational Linguistics and PhD in Computer Science both from the University of Southern California. He leads the Machine Discovery and Social Network Mining Lab in NTU. Before joining NTU, he was a post-doctoral research fellow at the Los Alamos National Lab. Prof. Lin's research includes the areas of machine learning and data mining, social network analysis, and natural language processing. His international recognition includes the best paper award in IEEE Web Intelligent conference 2003, Google Research Award in 2007, Microsoft research award in 2008, merit paper award in TAAI 2010, best paper award in ASONAM 2011, US Aerospace AFOSR/AOARD research award winner for 5 years. He is the all-time winners in ACM KDD Cup, leading or co-leading the NTU team to win 5 championships. He also leads a team to win WSDM Cup 2016 Champion. He has served as the senior PC for SIGKDD and area chair for ACL. He is currently the associate editor for International Journal on Social Network Mining, Journal of Information Science and Engineering, and International Journal of Computational Linguistics and Chinese Language Processing. He receives the Young Scholars' Creativity Award from Foundation for the Advancement of Outstanding Scholarship and Ta-You Wu Memorial Award.
Our research lines on Model-Driven Engineering and Software EngineeringJordi Cabot
Highlighting some of our research lines (March 2015 Edition)
Learn more about what we do on : http://modeling-languages.com , http://som-research.uoc.edu and http://jordicabot.com
For the full video of this presentation, please visit:
https://www.edge-ai-vision.com/2021/02/introducing-machine-learning-and-how-to-teach-machines-to-see-a-presentation-from-tryolabs/
Facundo Parodi, Research and Machine Learning Engineer at Tryolabs, presents the “Introduction to Machine Learning and How to Teach Machines to See” tutorial at the September 2020 Embedded Vision Summit.
What is machine learning? How can machines distinguish a cat from a dog in an image? What’s the magic behind convolutional neural networks? These are some of the questions Parodi answers in this introductory talk on machine learning in computer vision.
Parodi introduces machine learning and explores the different types of problems it can solve. He explains the main components of practical machine learning, from data gathering and training to deployment. He then focuses on deep learning as an important machine learning technique and provides an introduction to convolutional neural networks and how they can be used to solve image classification problems. Parodi will also touches on recent advancements in deep learning and how they have revolutionized the entire field of computer vision.
“Automatically learning multiple levels of representations of the underlying distribution of the data to be modelled”
Deep learning algorithms have shown superior learning and classification performance.
In areas such as transfer learning, speech and handwritten character recognition, face recognition among others.
(I have referred many articles and experimental results provided by Stanford University)
Recently I gave a talk at UC Berkeley regarding the transition from academia to industry in the context of Machine Learning and Data Science related roles. I based most of my slides on my own transition from being an Astrophysicist to a Machine Learning Expert. I hope this will be useful to many. Feedback is welcome!
Scene classification using Convolutional Neural Networks - Jayani WithanawasamWithTheBest
Scene Classification is used in Convolutional Neural Networks (CNNs). We seek to redefine computer vision as an AI problem, understand the importance of scene classification as well as challenges, and the difference between traditional machine learning and deep learning. Additionally, we discuss CNNs, using caffe for implementing CNNs and importact reosources to imorove.
CNNs
Jayani Withanawasam
Please don’t make me draw!
Lesson learned during the development of a software
to support early analysis of object-oriented systems.
This paper describes the development of a software tool to support rich pictures creation for Object Oriented Analysis (OOA). This software should be useful both as an e-learning tool for bachelor-level students, as well as for practitioners working with agile methodologies. Since the transposition of manual rich picture practise into software proved difficult, we decided to follow a user-centered approach, design and implement a prototype with basic functionalities, then run a usability test with a few students and professionals. The feedback collected in the test validated the design of our prototype, and forced us to re-consider the relationship between concrete examples and abstract concepts in rich pictures and in our tool. This unexpectedly helped us realize how to implement support for behavioral description (i.e. events), an elusive feature before the test. Moreover we gained a deeper insight on programmers’ perspective on their practise.
At a more general level we realized how modern object-oriented development methodologies, such as agile methods, are informed by design, and sometimes assume design skills that programmers do not have or do not value. An important lesson to consider carefully to keep our tool usable.
Building Large-scale Real-world Recommender Systems - Recsys2012 tutorialXavier Amatriain
There is more to recommendation algorithms than rating prediction. And, there is more to recommender systems than algorithms. In this tutorial, given at the 2012 ACM Recommender Systems Conference in Dublin, I review things such as different interaction and user feedback mechanisms, offline experimentation and AB testing, or software architectures for Recommender Systems.
In this talk we explore how to build Machine Learning Systems that can that can learn "continuously" from their mistakes (feedback loop) and adapt to an evolving data distribution.
The youtube link to video of the talk is here:
https://www.youtube.com/watch?v=VtBvmrmMJaI
From Conventional Machine Learning to Deep Learning and Beyond.pptxChun-Hao Chang
In this slide, Deep Learning are compared with Conventional Learning and the strength of DNN models will be explained.
The target audience are people who have the knowledge of Machine Learning or Data Mining but not familiar with Deep Learning.
Crafting Recommenders: the Shallow and the Deep of it! Sudeep Das, Ph.D.
I present a brief review, and an outlook on the rapid changes happening in the field of recommendation engine research on the heels of the deep learning revolution!
ODSC East: Effective Transfer Learning for NLPindico data
Presented by indico co-founder Madison May at ODSC East.
Abstract: Transfer learning, the practice of applying knowledge gained on one machine learning task to aid the solution of a second task, has seen historic success in the field of computer vision. The output representations of generic image classification models trained on ImageNet have been leveraged to build models that detect the presence of custom objects in natural images. Image classification tasks that would typically require hundreds of thousands of images can be tackled with mere dozens of training examples per class thanks to the use of these pretrained reprsentations. The field of natural language processing, however, has seen more limited gains from transfer learning, with most approaches limited to the use of pretrained word representations. In this talk, we explore parameter and data efficient mechanisms for transfer learning on text, and show practical improvements on real-world tasks. In addition, we demo the use of Enso, a newly open-sourced library designed to simplify benchmarking of transfer learning methods on a variety of target tasks. Enso provides tools for the fair comparison of varied feature representations and target task models as the amount of training data made available to the target model is incrementally increased.
Easy path to machine learning (2023-2024)wesley chun
1-hr tech talk introducing Machine Learning and the GCP ML APIs and other Google Cloud developer tools to a technical audience:
Easier onramp to getting into AI/ML by using GCP AI/ML APIs (Vision, Video Intelligence, Natural Language, Speech-to-Text, Text-to-Speech, Translation) backed by single-task pre-trained models found in Vertex AI, AutoML for finetuning those pre-trained models, and other "friends of AI/ML" Google dev tools & platforms that can help: BigQuery (data warehouse & analysis), Cloud SQL+AlloyDB & Firestore (SQL & NoSQL databases), serverless platforms (App Engine, Cloud Functions, Cloud Run), and introducing the Gemini API (from both Google AI and GCP Vertex AI)
Cutting Edge Computer Vision for EveryoneIvo Andreev
Microsoft offers a wide range of tools and advanced solutions to support you in managing computer vision related tasks.
From purely coding approaches with ML.NET, through zero-code ComputerVision.ai to advanced and flexible AI service in Azure ML, there is a solution for every need and each type of person.
From running on premises, through managed infrastructure to completely cloud services the speed of getting to the desired results and the return of investment are guaranteed.
Join this session to get insights about the options, deployment, pricing, pros and cons compared and select the most appropriate tech for your business case.
Certification Study Group - NLP & Recommendation Systems on GCP Session 5gdgsurrey
This session features Raghavendra Guttur's exploration of "Atlas," a chatbot powered by Llama2-7b with MiniLM v2 enhancements for IT support. ChengCheng Tan will discuss ML pipeline automation, monitoring, optimization, and maintenance.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
OReilly AI Transfer Learning
1. O'Reilly Artificial Intelligence Conference San Francisco 2018
How to use transfer learning to bootstrap image
classification and question answering (QA)
Danielle Dean PhD, Wee Hyong Tok PhD
Principal Data Scientist Lead
Microsoft
@danielleodean | @weehyong
Inspired by “Transfer Learning: Repurposing ML Algorithms from Different Domains to Cloud Defense” , Mark Russinovich, RSA Conference 2018
2. Textbook ML development
Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
3. Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing
Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
Fact | Industry grade ML solutions are highly exploratory
Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing
Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing
Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing
Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing
Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
Choosing the
Learning Task
Defining Data
Input
Applying Data
Transforms
Choosing the
Learner
Choosing
Output
Choosing Run
Options
View Results
Debug and
Visualize Errors
Analyze Model
Predictions
Attempt 1 Attempt 2 Attempt 3
Attempt 4 Attempt n
4. Traditional versus Transfer learning
Learning
system
Learning
system
Learning
system
Different tasks
Traditional Machine Learning Transfer Learning
Source tasks
Learning
system
Target task
Source: "A survey on transfer learning." , Pan, Sinno Jialin, and Qiang Yang. IEEE Transactions on knowledge and data engineering
5. Why are we talking about transfer learning ?
Commercial
success
Time 2016
Supervised
learning
Transfer
learning
Unsupervised
learning
Reinforcement
learning
Drivers of ML success in industry
Source: “Transfer Learning - Machine Learning's Next Frontier” , Ruder, Sebastian,
6. Transfer Learning in Computer Vision
Can we leverage knowledge of processing images to help with new
tasks?
• What’s in the picture?
• Where is the bike located?
• Can you find a similar bike?
• How many bikes are there?
7. Before Deep Learning
• Researchers took a traditional machine learning approach
• Manual creation of a variety of different visual feature extractors
• Followed by traditional ML classifiers
• Features not very generalizable to other vision tasks – not easy to transfer
• Example: HoG Detectors
- Histogram of oriented
gradients (HoG) features
- Sliding window detector
- SVM Classifier
- Very fast OpenCV
implementation (<100ms)
10. Transfer Learning for Computer Vision
Train a model
using data from
ImageNet Retail
Manufacturing
Deep Learning
Model for
Computer
Vision
Apply the
model to
other domains
11. Example – Visualizing the different layers
Source: Olah, et al., "Feature Visualization", Distill, 2017
https://distill.pub/2017/feature-visualization/
Another fun site:
https://deepart.io/nips/submissions/random/
http://cs231n.stanford.edu/
12. Example – Visualizing the different layers
Source: Olah, et al., "Feature Visualization", Distill, 2017
https://distill.pub/2017/feature-visualization/
Check out these sites -
https://deepart.io/nips/submissions/random/
http://cs231n.stanford.edu/
15. Transfer Learning – How to get started?
Type How to Initialize
Featurization
Layers
Output
Layer
Initialization
How is Transfer Learning
used?
How to Train?
Standard DNN Random Random None Train featurization and output
jointly
Headless DNN Learn using
another task
Separate ML
algorithm
Use the features learned
on a related task
Use the features to train a
separate classifier
Fine Tune DNN Learn using
another task
Random Use and fine tune
features learned on a
related task
Train featurization and output
jointly with a small learning rate
Multi-Task DNN Random Random Learned features need to
solve many related tasks
Share a featurization network
across both tasks. Train all
networks jointly with a loss
function (sum of individual task
loss function)
16. Pre-Built CNN from General Task on Millions of Images
Output
Layer
Stripped
cat? YES
dog? NO
car? NO
Classi
fier
e.g.
SVM
dotted?
Complex
Objects &
Scenes
(people, animals,
cars, beach
scene, etc.)
Low-Level Features
(lines, edges,
color fields, etc.)
High-Level Features
(corners, contours,
simple shapes)
Object Parts
(wheels, faces,
windows, etc.)
Outputs of penultimate layer of ImageNet Trained CNN
provide excellent general purpose image features
17. Pre-Built CNN from General Task on Millions of Images
Output
Layer
Stripped
Using a pre-trained DNN, an accurate
model can be achieved with thousands (or
less) of labeled examples instead of millions
cat? YES
dog? NO
car? NO
dotted?
Train one or more
layers in new network
18. Transfer Learning Results - Texture Dataset
DNN featurization
Input Image Size: 224x224 pixels
Area Under Curve: 0.59
Classification Accuracy: 69.0%
Fine-tuning (full CNN)
Input Image Size: 224x224 pixels
Area Under Curve: 0.76
Classification Accuracy: 77.4%
Fine-tuning (full CNN)
Input Image Size: 896x886 pixels
Area Under Curve: 0.83
Classification Accuracy: 88.2%
25. Aerial Use Classification ESmart – Connected Drone Jabil – Defect Inspection
Example Applications in Computer Vision
Lung Cancer Detection
Distributed deep domain
adaptation for automated
poacher detection
28. Read more details: https://www.microsoft.com/en-us/research/blog/using-
transfer-learning-to-address-label-noise-for-large-scale-image-classification/
Label Noise
29. Read more details: https://www.microsoft.com/en-us/research/blog/using-
transfer-learning-to-address-label-noise-for-large-scale-image-classification/
Traditional Method: Manual Verification
30. Read more details: https://www.microsoft.com/en-us/research/blog/using-
transfer-learning-to-address-label-noise-for-large-scale-image-classification/
Applying Transfer Learning
31. Computer Vision is not a “solved problem”
The knowledge being “transferred” can be very useful but not the same as
how humans learn to see
32. Recap: Transfer Learning for Image Classification
Define the
Learning Task
Identify a pre-
trained model
Decide whether to
further fine-tune
or use it as a
headless DNN
Freeze top layers,
re-train the
classifier
Validate the model
Deploy the model
33. Audio Spectrograms
Images
Rich, high-dimensional datasets
Rich, high-dimensional datasets
Text
Spare data (depends on the encoding)I s e e a b I g c a t
Deep Learning on Different Types of Data
36. Transfer Learning for Text
Define the
Learning Task
Identify a pre-
trained model
Decide whether to
further fine-tune
Freeze top layers,
re-train the
classifier
Validate the model
Deploy the model
What does the top
layer encode?
What kind of pre-
trained model?
37. Word Embeddings
Male - Female Verb Tense Country - Capital
Source: Tensorflow Tutorial - https://www.tensorflow.org/tutorials/representation/word2vec
39. Using Pre-trained Embeddings
Text Classification using 20 Newsgroup dataset
Source: https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html
embeddings_index = {}
f = open(os.path.join(GLOVE_DIR, 'glove.6B.100d.txt'))
for line in f:
values = line.split()
word = values[0]
coefs = np.asarray(values[1:], dtype='float32')
embeddings_index[word] = coefs
f.close()
Compute an index
mapping words to
known embeddings
embedding_matrix = np.zeros((len(word_index) + 1, EMBEDDING_DIM))
for word, i in word_index.items():
embedding_vector = embeddings_index.get(word)
if embedding_vector is not None:
# words not found in embedding index will be all-zeros.
embedding_matrix[i] = embedding_vector
Compute Embedding
Matrix
40. Using Pre-trained Embeddings
Text Classification using 20 Newsgroup dataset
Source: https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html
from keras.layers import Embedding
embedding_layer = Embedding(len(word_index) + 1,
EMBEDDING_DIM,
weights=[embedding_matrix],
input_length=MAX_SEQUENCE_LENGTH,
trainable=False)
Load the Embedding
Matrix into an
Embedding Layer
Prevent weights from being
updated during training
41. Using Pre-trained Embeddings
Text Classification using 20 Newsgroup dataset
Source: https://blog.keras.io/using-pre-trained-word-embeddings-in-a-keras-model.html
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(128, 5, activation='relu')(embedded_sequences)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(5)(x)
x = Conv1D(128, 5, activation='relu')(x)
x = MaxPooling1D(35)(x) # global max pooling
x = Flatten()(x)
x = Dense(128, activation='relu')(x)
preds = Dense(len(labels_index), activation='softmax')(x)
model = Model(sequence_input, preds)
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['acc'])
model.fit(x_train, y_train, validation_data=(x_val, y_val),
epochs=2, batch_size=128)
Build a small 1D
convnet to solve the
classification problem
42. From initializing the first layers to pre-
training the entire model
(and learning higher level semantic concepts)
43. Transfer Learning for NLP - ULMFiT
Source: Universal Language Model Fine-tuning for Text Classification, Jeremy Howard, Sebastian Ruder, ACL 2018
Train a Language Model
using Large General
Domain Corpus
Fine-tune the
Language Model
Fine-tune Classifier
44. Transfer Learning for NLP - ELMo
Source: Deep contextualized word representations, Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt
Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer., NAACL 2018
ELMo ELMo ELMo
have a nice
Corpus
Train
biLMs
Enhancing
Inputs with ELMos
Usual
Inputs
46. Using ELMo with TensorFlow Hub
Source: https://www.tensorflow.org/hub/modules/google/elmo/2
elmo = hub.Module("https://tfhub.dev/google/elmo/2",
trainable=True)
embeddings = elmo(
["the cat is on the mat", "dogs are in the fog"],
signature="default",
as_dict=True)["elmo"]
elmo = hub.Module("https://tfhub.dev/google/elmo/2", trainable=True)
tokens_input = [["the", "cat", "is", "on", "the", "mat"],
["dogs", "are", "in", "the", "fog", ""]]
tokens_length = [6, 5]
embeddings = elmo(
inputs={
"tokens": tokens_input,
"sequence_len": tokens_length
},
signature="tokens",
as_dict=True)["elmo"]
ELMo
Untokenized Sentences
Tokens
Or Dictionary
• Character-based word representation
• First LSTM Hidden State
• Second LSTM Hidden State
• elmo (weighted sum of 3 layers)
• Fixed mean-pooling of contextualized
word representation
47. Transfer Learning for MRC tasks
Source:
Transfer Learning for Machine Reading Comprehension - https://bit.ly/2Cmiffy
48. Transfer Learning for MRC
Train a model
using data from
WikiPedia
News Articles
Customer Support Data
MRC
Model Apply the
model to
other domains
49. SQUAD
Stanford Question Answering Dataset (SQuAD)
Reading comprehension dataset
Based on Wikipedia articles
Crowdsource questions
Answer is Text Segment, or span, from
the corresponding reading passage, or the no
answers found.
Question Answer Pairs
51. Transfer Learning for MRC using SynNet
Train using a large
MRC Dataset (e.g.
SQuAD)
Apply the pre-
trained model to a
new domain (e.g.
NewsQA)
Validate
the model
Deploy the model
Transfer Learning for MRC –Survey - https://bit.ly/2JAt1h0
More comparisons between different MRC Approaches
52. SynNet
Stage 1- Answer Synthesis module
uses a bi-directional LSTM to predict
IOB tags on the input paragraph.
Marks out semantic concept that are
likely answer
Stage 2 – Question Synthesis module
uses a uni-directional LSTM to
generate the questions
Source: ACL 2017, https://www.microsoft.com/en-us/research/publication/two-stage-synthesis-networks-transfer-learning-machine-comprehension/
54. O'Reilly Artificial Intelligence Conference San Francisco 2018
How to use transfer learning to
bootstrap image classification and
question answering (QA)
Summary
1. Transfer Learning and
Applications
2. How to use Transfer Learning for
Image Classification
3. How to use Transfer Learning for
NLP tasks
55. O'Reilly Artificial Intelligence Conference San Francisco 2018
How to use transfer learning to
bootstrap image classification and
question answering (QA)
Danielle Dean PhD, Wee Hyong Tok PhD
Principal Data Scientist Lead
Microsoft
@danielleodean | @weehyong
Thank You!