This document provides an overview of deep learning applications and techniques for implementing deep neural networks. It discusses:
- Various use cases for deep learning in industries like computer vision, natural language processing, and speech recognition.
- Frameworks for implementing deep networks like Caffe and Keras, which allow finetuning pre-trained models or building custom networks.
- Examples of applying deep learning to tasks like classifying bee photos and search queries, using techniques like convolutional neural networks, word embeddings, and XGBoost classifiers.
- Tips for optimizing neural network architecture like starting simple and gradually increasing complexity.
The document shares results from several classification problems to demonstrate deep learning methods.
Deep Learning Frameworks Using Spark on YARN by Vartika SinghData Con LA
Abstract:- Traditional machine learning and feature engineering algorithms are not efficient enough to extract complex and nonlinear patterns hallmarks of big data. Deep learning, on the other hand, helps translate the scale and complexity of the data into solutions like molecular interaction in drug design, the search for subatomic particles and automatic parsing of microscopic images. Co-locating a data processing pipeline with a deep learning framework makes data exploration/algorithm and model evolution much simpler, while streamlining data governance and lineage tracking into a more focused effort. In this talk, we will discuss and compare the different deep learning frameworks on Spark in a distributed mode, ease of integration with the Hadoop ecosystem, and relative comparisons in terms of feature parity.
Presentation of few recent papers on Deep Learning ... in particular Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding, Song Han, Huizi Mao, William J. Dally International Conference on Learning Representations ICLR2016
Improving Hardware Efficiency for DNN ApplicationsChester Chen
Speaker: Dr. Hai (Helen) Li is the Clare Boothe Luce Associate Professor of Electrical and Computer Engineering and Co-director of the Duke Center for Evolutionary Intelligence at Duke University
In this talk, I will introduce a few recent research spotlights by the Duke Center for Evolutionary Intelligence. The talk will start with the structured sparsity learning (SSL) method which attempts to learn a compact structure from a bigger DNN to reduce computation cost. It generates a regularized structure with high execution efficiency. Our experiments on CPU, GPU, and FPGA platforms show on average 3~5 times speedup of convolutional layer computation of AlexNet. Then, the implementation and acceleration of DNN applications on mobile computing systems will be introduced. MoDNN is a local distributed system which partitions DNN models onto several mobile devices to accelerate computations. ApesNet is an efficient pixel-wise segmentation network, which understands road scenes in real-time, and has achieved promising accuracy. Our prospects on the adoption of emerging technology will also be given at the end of this talk, offering the audiences an alternative thinking about the future evolution and revolution of modern computing systems.
This presentation focuses on Deep Learning (DL) concepts, such as neural networks, backprop, activation functions, and Convolutional Neural Networks. You'll also learn how to incorporate Deep Learning in Android applications. Basic knowledge of matrices is helpful for this session, which is targeted primarily to beginners.
Squeezing Deep Learning Into Mobile PhonesAnirudh Koul
A practical talk by Anirudh Koul aimed at how to run Deep Neural Networks to run on memory and energy constrained devices like smart phones. Highlights some frameworks and best practices.
Deep learning on mobile - 2019 Practitioner's GuideAnirudh Koul
The 2019 Guide to Deep Learning on Mobile, from Inference to Training on iOS and Android smartphones. Featuring CoreML, Tensorflow Lite, MLKit, Fritz, AutoML Approaches (Hardware Aware Neural Architecture Search) to make models more efficient, and lots of videos. Presented by Anirudh Koul, Siddha Ganju and Meher Anand Kasam. More details at PracticalDL.ai in the upcoming O'Reilly Book 'Practical Deep Learning on Cloud & Mobile'
A practical talk by Anirudh Koul aimed at how to run Deep Neural Networks to run on memory and energy constrained devices like smartphones. Highlights some frameworks and best practices.
Some resources how to navigate in the hardware space in order to build your own workstation for training deep learning models.
Alternative download link: https://www.dropbox.com/s/o7cwla30xtf9r74/deepLearning_buildComputer.pdf?dl=0
Deep Learning Frameworks Using Spark on YARN by Vartika SinghData Con LA
Abstract:- Traditional machine learning and feature engineering algorithms are not efficient enough to extract complex and nonlinear patterns hallmarks of big data. Deep learning, on the other hand, helps translate the scale and complexity of the data into solutions like molecular interaction in drug design, the search for subatomic particles and automatic parsing of microscopic images. Co-locating a data processing pipeline with a deep learning framework makes data exploration/algorithm and model evolution much simpler, while streamlining data governance and lineage tracking into a more focused effort. In this talk, we will discuss and compare the different deep learning frameworks on Spark in a distributed mode, ease of integration with the Hadoop ecosystem, and relative comparisons in terms of feature parity.
Presentation of few recent papers on Deep Learning ... in particular Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding, Song Han, Huizi Mao, William J. Dally International Conference on Learning Representations ICLR2016
Improving Hardware Efficiency for DNN ApplicationsChester Chen
Speaker: Dr. Hai (Helen) Li is the Clare Boothe Luce Associate Professor of Electrical and Computer Engineering and Co-director of the Duke Center for Evolutionary Intelligence at Duke University
In this talk, I will introduce a few recent research spotlights by the Duke Center for Evolutionary Intelligence. The talk will start with the structured sparsity learning (SSL) method which attempts to learn a compact structure from a bigger DNN to reduce computation cost. It generates a regularized structure with high execution efficiency. Our experiments on CPU, GPU, and FPGA platforms show on average 3~5 times speedup of convolutional layer computation of AlexNet. Then, the implementation and acceleration of DNN applications on mobile computing systems will be introduced. MoDNN is a local distributed system which partitions DNN models onto several mobile devices to accelerate computations. ApesNet is an efficient pixel-wise segmentation network, which understands road scenes in real-time, and has achieved promising accuracy. Our prospects on the adoption of emerging technology will also be given at the end of this talk, offering the audiences an alternative thinking about the future evolution and revolution of modern computing systems.
This presentation focuses on Deep Learning (DL) concepts, such as neural networks, backprop, activation functions, and Convolutional Neural Networks. You'll also learn how to incorporate Deep Learning in Android applications. Basic knowledge of matrices is helpful for this session, which is targeted primarily to beginners.
Squeezing Deep Learning Into Mobile PhonesAnirudh Koul
A practical talk by Anirudh Koul aimed at how to run Deep Neural Networks to run on memory and energy constrained devices like smart phones. Highlights some frameworks and best practices.
Deep learning on mobile - 2019 Practitioner's GuideAnirudh Koul
The 2019 Guide to Deep Learning on Mobile, from Inference to Training on iOS and Android smartphones. Featuring CoreML, Tensorflow Lite, MLKit, Fritz, AutoML Approaches (Hardware Aware Neural Architecture Search) to make models more efficient, and lots of videos. Presented by Anirudh Koul, Siddha Ganju and Meher Anand Kasam. More details at PracticalDL.ai in the upcoming O'Reilly Book 'Practical Deep Learning on Cloud & Mobile'
A practical talk by Anirudh Koul aimed at how to run Deep Neural Networks to run on memory and energy constrained devices like smartphones. Highlights some frameworks and best practices.
Some resources how to navigate in the hardware space in order to build your own workstation for training deep learning models.
Alternative download link: https://www.dropbox.com/s/o7cwla30xtf9r74/deepLearning_buildComputer.pdf?dl=0
On-device machine learning: TensorFlow on AndroidYufeng Guo
Machine learning has traditionally been the solely performed on servers and high performance machines. But there is great value is having on-device machine learning for mobile devices. Doing ML inference on mobile devices has huge potential and is still in its early stages. However, it's already more powerful than most realize.
In this demo-oriented talk, you will see some examples of deep learning models used for local prediction on mobile devices. Learn how to use TensorFlow to implement a machine learning model that is tailored to a custom dataset, and start making delightful experiences today!
This is an 1 hour presentation on Neural Networks, Deep Learning, Computer Vision, Recurrent Neural Network and Reinforcement Learning. The talks later have links on how to run Neural Networks on
Faster deep learning solutions from training to inference - Michele Tameni - ...Codemotion
Intel Deep Learning SDK enables using of optimized open source deep-learning frameworks, including Caffe and TensorFlow through a step-by-step wizard or iPython interactive notebooks. It includes easy and fast installation of all depended libraries and advanced tools for easy data pre-processing and model training, optimization and deployment, providing an end-to-end solution to the problem. In addition, it supports scale-out on multiple computers for training, as well as using compression methods for deployment of the models on various platforms, addressing memory and speed constraints.
Comparing TensorFlow and MxNet from a cloud engineering and production app building perspective. Looks at adoption, support, deployment and cloud support cross vendors.
For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/wavecomp/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-nicol
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Chris Nicol, CTO at Wave Computing, presents the "New Dataflow Architecture for Machine Learning" tutorial at the May 2017 Embedded Vision Summit.
Data scientists have made tremendous advances in the use of deep neural networks (DNNs) to enhance business models and service offerings. But training DNNs can take a week or more using traditional hardware solutions that rely on legacy architectures that are limited in performance and scalability. New innovations that can reduce training time for both image-centric and text-centric deep neural networks will lead to an explosion of new applications. Dr. Chris Nicol, Wave Computing’s Chief Technology Officer, examines the performance challenge faced by data scientists today. Nicol outlines the technical factors underlying this bottleneck for systems relying on CPUs, GPUs, FPGAs and ASICs, and introduces a new dataflow-centric approach to DNN training.
An introduction to Keras, a high-level neural networks library written in Python. Keras makes deep learning more accessible, is fantastic for rapid protyping, and can run on top of TensorFlow, Theano, or CNTK. These slides focus on examples, starting with logistic regression and building towards a convolutional neural network.
The presentation was given at the Austin Deep Learning meetup: https://www.meetup.com/Austin-Deep-Learning/events/237661902/
Yinyin Liu presents at SD Robotics Meetup on November 8th, 2016. Deep learning has made great success in image understanding, speech, text recognition and natural language processing. Deep Learning also has tremendous potential to tackle the challenges in robotic vision, and sensorimotor learning in a robotic learning environment. In this talk, we will talk about how current and future deep learning technologies can be applied for robotic applications.
Josh Patterson, Advisor, Skymind – Deep learning for Industry at MLconf ATL 2016MLconf
DL4J and DataVec for Enterprise Deep Learning Workflows: Applications in NLP, sensor processing (IoT), image processing, and audio processing have all emerged as prime deep learning applications. In this session we will take a look at a practical review of building practical and secure Deep Learning workflows in the enterprise. We’ll see how DL4J’s DataVec tool enables scalable ETL and vectorization pipelines to be created for a single machine or scale out to Spark on Hadoop. We’ll also see how Deep Networks such as Recurrent Neural Networks are able to leverage DataVec to more quickly process data for modeling.
Tom Peters, Software Engineer, Ufora at MLconf ATL 2016MLconf
Say What You Mean: Scaling Machine Learning Algorithms Directly from Source Code: Scaling machine learning applications is hard. Even with powerful systems like Spark, Tensor Flow, and Theano, the code you write has more to do with getting these systems to work at all than it does with your algorithm itself. But it doesn’t have to be this way!
In this talk, I’ll discuss an alternate approach we’ve taken with Pyfora, an open-source platform for scalable machine learning and data science in Python. I’ll show how it produces efficient, large scale machine learning implementations directly from the source code of single-threaded Python programs. Instead of programming to a complex API, you can simply say what you mean and move on. I’ll show some classes of problem where this approach truly shines, discuss some practical realities of developing the system, and I’ll talk about some future directions for the project.
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLoveNETWAYS
In the beginning there was CFEngine, and the learning curve was high, then came Puppet , Chef and the learning curve was still high.
Now we have Ansible , for everyone that wasn't smart enough to learn the original tools. Or wasn't that the problem ?
For some people Infrastructure as Code became a goal alone, not caring about the infrastructure, Junior people wanted to learn Puppet, but forgot about the service they were configuring. Too Complex, Too much effort, .. And then containers came.
This talk is about why I believe having the ability to write tools and/or scripts can help elevate a Pen Testers game to the next level.
The talk is case study driven by the different scenarios I've encountered on assessments and the scripts or tools that have been developed as a result.
On-device machine learning: TensorFlow on AndroidYufeng Guo
Machine learning has traditionally been the solely performed on servers and high performance machines. But there is great value is having on-device machine learning for mobile devices. Doing ML inference on mobile devices has huge potential and is still in its early stages. However, it's already more powerful than most realize.
In this demo-oriented talk, you will see some examples of deep learning models used for local prediction on mobile devices. Learn how to use TensorFlow to implement a machine learning model that is tailored to a custom dataset, and start making delightful experiences today!
This is an 1 hour presentation on Neural Networks, Deep Learning, Computer Vision, Recurrent Neural Network and Reinforcement Learning. The talks later have links on how to run Neural Networks on
Faster deep learning solutions from training to inference - Michele Tameni - ...Codemotion
Intel Deep Learning SDK enables using of optimized open source deep-learning frameworks, including Caffe and TensorFlow through a step-by-step wizard or iPython interactive notebooks. It includes easy and fast installation of all depended libraries and advanced tools for easy data pre-processing and model training, optimization and deployment, providing an end-to-end solution to the problem. In addition, it supports scale-out on multiple computers for training, as well as using compression methods for deployment of the models on various platforms, addressing memory and speed constraints.
Comparing TensorFlow and MxNet from a cloud engineering and production app building perspective. Looks at adoption, support, deployment and cloud support cross vendors.
For the full video of this presentation, please visit:
https://www.embedded-vision.com/platinum-members/wavecomp/embedded-vision-training/videos/pages/may-2017-embedded-vision-summit-nicol
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Chris Nicol, CTO at Wave Computing, presents the "New Dataflow Architecture for Machine Learning" tutorial at the May 2017 Embedded Vision Summit.
Data scientists have made tremendous advances in the use of deep neural networks (DNNs) to enhance business models and service offerings. But training DNNs can take a week or more using traditional hardware solutions that rely on legacy architectures that are limited in performance and scalability. New innovations that can reduce training time for both image-centric and text-centric deep neural networks will lead to an explosion of new applications. Dr. Chris Nicol, Wave Computing’s Chief Technology Officer, examines the performance challenge faced by data scientists today. Nicol outlines the technical factors underlying this bottleneck for systems relying on CPUs, GPUs, FPGAs and ASICs, and introduces a new dataflow-centric approach to DNN training.
An introduction to Keras, a high-level neural networks library written in Python. Keras makes deep learning more accessible, is fantastic for rapid protyping, and can run on top of TensorFlow, Theano, or CNTK. These slides focus on examples, starting with logistic regression and building towards a convolutional neural network.
The presentation was given at the Austin Deep Learning meetup: https://www.meetup.com/Austin-Deep-Learning/events/237661902/
Yinyin Liu presents at SD Robotics Meetup on November 8th, 2016. Deep learning has made great success in image understanding, speech, text recognition and natural language processing. Deep Learning also has tremendous potential to tackle the challenges in robotic vision, and sensorimotor learning in a robotic learning environment. In this talk, we will talk about how current and future deep learning technologies can be applied for robotic applications.
Josh Patterson, Advisor, Skymind – Deep learning for Industry at MLconf ATL 2016MLconf
DL4J and DataVec for Enterprise Deep Learning Workflows: Applications in NLP, sensor processing (IoT), image processing, and audio processing have all emerged as prime deep learning applications. In this session we will take a look at a practical review of building practical and secure Deep Learning workflows in the enterprise. We’ll see how DL4J’s DataVec tool enables scalable ETL and vectorization pipelines to be created for a single machine or scale out to Spark on Hadoop. We’ll also see how Deep Networks such as Recurrent Neural Networks are able to leverage DataVec to more quickly process data for modeling.
Tom Peters, Software Engineer, Ufora at MLconf ATL 2016MLconf
Say What You Mean: Scaling Machine Learning Algorithms Directly from Source Code: Scaling machine learning applications is hard. Even with powerful systems like Spark, Tensor Flow, and Theano, the code you write has more to do with getting these systems to work at all than it does with your algorithm itself. But it doesn’t have to be this way!
In this talk, I’ll discuss an alternate approach we’ve taken with Pyfora, an open-source platform for scalable machine learning and data science in Python. I’ll show how it produces efficient, large scale machine learning implementations directly from the source code of single-threaded Python programs. Instead of programming to a complex API, you can simply say what you mean and move on. I’ll show some classes of problem where this approach truly shines, discuss some practical realities of developing the system, and I’ll talk about some future directions for the project.
OSDC 2015: Kris Buytaert | From ConfigManagementSucks to ConfigManagementLoveNETWAYS
In the beginning there was CFEngine, and the learning curve was high, then came Puppet , Chef and the learning curve was still high.
Now we have Ansible , for everyone that wasn't smart enough to learn the original tools. Or wasn't that the problem ?
For some people Infrastructure as Code became a goal alone, not caring about the infrastructure, Junior people wanted to learn Puppet, but forgot about the service they were configuring. Too Complex, Too much effort, .. And then containers came.
This talk is about why I believe having the ability to write tools and/or scripts can help elevate a Pen Testers game to the next level.
The talk is case study driven by the different scenarios I've encountered on assessments and the scripts or tools that have been developed as a result.
Introduction to Deep Learning | CloudxLabCloudxLab
( Machine Learning & Deep Learning Specialization Training: https://goo.gl/goQxnL )
This CloudxLab Deep Learning tutorial helps you to understand Deep Learning in detail. Below are the topics covered in this tutorial:
1) What is Deep Learning
2) Deep Learning Applications
3) Artificial Neural Network
4) Deep Learning Neural Networks
5) Deep Learning Frameworks
6) AI vs Machine Learning
SDEC2011 Mahout - the what, the how and the whyKorea Sdec
Mahout is an open source machine learning library from Apache. From its humble beginnings at Apache Lucene, the project has grown into a active community of developers, machine learning experts and enthusiasts. With v0.5 released recently, the project has been focussing full steam on developing stable APIs with an eye on our major milestone of v1.0. The speaker has been with Mahout from his days in college as a computer science student. The talk will focus on the major use cases of Mahout. The design decisions, things that worked, things that didn't, and things to expect in the future releases.
http://sdec.kr/
TonY: Native support of TensorFlow on HadoopAnthony Hsu
Anthony Hsu, Jonathan Hung, and Keqiu Hu offer an overview of TensorFlow on YARN (TonY), a framework to natively run TensorFlow on Hadoop. TonY enables running TensorFlow distributed training as a new type of Hadoop application. Its native Hadoop connector, together with other features, aims to run TensorFlow jobs as reliably and flexibly as other applications on Hadoop.
Video: https://youtu.be/sIfnsU-5jHM
The content was modified from Google Content Group
Eric ShangKuan(ericsk@google.com)
---
TensorFlow Lite guide( for mobile & IoT )
TensorFlow Lite is a set of tools to help developers run TensorFlow models on mobile, embedded, and IoT devices. It enables on-device machine learning inference with low latency and small binary size.
TensorFlow Lite consists of two main components:
The TensorFlow Lite interpreter:
- optimize models on many different hardware types, like mobile phones, embedded Linux devices, and microcontrollers.
The TensorFlow Lite converter:
- which converts TensorFlow models into an efficient form for use by the interpreter, and can introduce optimizations to improve binary size and performance.
---
Event: PyLadies TensorFlow All-Around
Date: Sep 25, 2019
Event link: https://www.meetup.com/PyLadies-Berlin/events/264205538/
Linkedin: http://linkedin.com/in/mia-chang/
OSDC 2018 | Migrating to the cloud by Devdas BhagatNETWAYS
This is an experience report of a migration from self-hosted services to running in the cloud. While there have been plenty of business case studies showing the benefits of a cloud migration, there are very few reports on the IT side of the migration. This talk covers the migration of Spilgames (a small Dutch games publisher) from a self-hosted Openstack and hardware based infrastructure to Google cloud, challenges, tooling (and lack thereof). This migration is still work in progress, and the talk will cover as much detail as possible.
How to Choose a Deep Learning FrameworkNavid Kalaei
The trend of neural networks has been attracted a huge community of researchers and practitioners. However, not all of the upfront runners are masters of deep learning and the colorful frameworks could be confusing, especially for the newcomers. In this presentation, I demystified the mystery of the leading frameworks of deep learning and provided a guideline on how to choose the most suitable option.
This presentation walks the reader through implementing a simple web application and its tests using Python, Flask, and the Pytest testing framework. Emphasis is placed on following the process of test-driven development (TDD) in creating the application.
Intro - End to end ML with Kubeflow @ SignalConf 2018Holden Karau
There are many great tools for training machine learning tools, ranging from sci-kit to Apache Spark, and tensorflow. However many of these systems largely leave open the question how to use our models outside of the batch world (like in a reactive application). Different options exist for persisting the results and using them for live training, and we will explore the trade-offs of the different formats and their corresponding serving/prediction layers.
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014Austin Ogilvie
Applied Data Science: Building a Beer Recommender | Data Science MD - Oct 2014
-----------
Slides from a talk by Greg Lamp, CTO of Yhat, about building recommendation systems using Python and deploying them to production.
Similar to Deep Learning Applications (dadada2017) (20)
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Safalta Digital marketing institute in Noida, provide complete applications that encompass a huge range of virtual advertising and marketing additives, which includes search engine optimization, virtual communication advertising, pay-per-click on marketing, content material advertising, internet analytics, and greater. These university courses are designed for students who possess a comprehensive understanding of virtual marketing strategies and attributes.Safalta Digital Marketing Institute in Noida is a first choice for young individuals or students who are looking to start their careers in the field of digital advertising. The institute gives specialized courses designed and certification.
for beginners, providing thorough training in areas such as SEO, digital communication marketing, and PPC training in Noida. After finishing the program, students receive the certifications recognised by top different universitie, setting a strong foundation for a successful career in digital marketing.
Normal Labour/ Stages of Labour/ Mechanism of LabourWasim Ak
Normal labor is also termed spontaneous labor, defined as the natural physiological process through which the fetus, placenta, and membranes are expelled from the uterus through the birth canal at term (37 to 42 weeks
2. About me
● Chief Data Scientist @ Boost AI
● Machine learning enthusiast
● Kaggle junkie (highest world rank #3)
● Interested in:
○ Automatic machine learning
○ Large scale classification of text data
○ Chatbots
I like big data
and
I cannot lie
3. Agenda
● Brief introduction to deep learning
● Implementation of deepnets
● Fine-tuning of pre-trained networks
● 4 different industrial use cases
● No maths!!!!
23. How can I implement my own DeepNets?
● Implement them on your own
24. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
25. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
26. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
27. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
● Save yourself some time and finetune
28. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
● Save yourself some time and finetune
○ Convert data
29. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
● Save yourself some time and finetune
○ Convert data
○ Define net
30. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
● Save yourself some time and finetune
○ Convert data
○ Define net
○ Define solver
31. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
● Save yourself some time and finetune
○ Convert data
○ Define net
○ Define solver
○ Train
32. How can I implement my own DeepNets?
● Implement them on your own
○ Decompose into smaller parts
○ Implement layers
○ Start training
● Save yourself some time and finetune
○ Convert data
○ Define net
○ Define solver
○ Train
● Caffe (caffe.berkeleyvision.org)
● Keras (www.keras.io)
41. What do you need for Caffe?
● Convert data
● Define a network (prototxt)
42. What do you need for Caffe?
● Convert data
● Define a network (prototxt)
● Define a solver (prototxt)
43. What do you need for Caffe?
● Convert data
● Define a network (prototxt)
● Define a solver (prototxt)
● Train the network (with or without pre-trained weights)
48. Training a net using Caffe
/PATH_TO_CAFFE/caffe train --solver=solver.prototxt
49. Fine Tuning!
● Fine tuning using GoogleNet
● Why?
○ It has Google in its name
○ It won ILSVRC 2014
○ It’s complicated and I wanted to play with it
● Caffe model zoo offers a lot of pretrained nets, including GoogleNet
● Model Zoo: https://github.com/BVLC/caffe/wiki/Model-Zoo
87. Why classify search queries?
● For businesses
○ Find out user-intent
○ Track keywords according to transactional buying cycle of user
○ Optimize website content and focus on smaller keyword set
88. Why classify search queries?
● For business
○ Find out user-intent
○ Track keywords according to transactional buying cycle of user
○ Optimize website content and focussing on smaller keyword set
● For data scientists
○ 100s of millions of unlabeled keywords to play with
○ Why Not!
101. Representing Queries as Images
David Villa
Word2Vec
representations of
the top search
result titles
Apple juice
Irish
102. I don’t see much
difference!
Guild Wars or Apple Juice
103.
104. Machine Learning Models
● Boosted trees
○ Word2vec embeddings
○ Titles from top results
○ Additional features of the SERP page
○ TF-IDF
○ XGBoost!!!! (https://github.com/dmlc/xgboost)
105. Machine Learning Models
● Convolutional Neural Networks:
○ Using images directly
○ Using random crops from the image
106. Machine Learning Models
● Convolutional Neural Networks:
○ Using images directly
○ Using random crops from the image
Convolutional Neural Network
107. Machine Learning Models
● Convolutional Neural Networks:
○ Using images directly
○ Using random crops from the image
Convolutional Neural Network
Convolutional Neural Network
108. Neural Networks with Keras
Convolutional Neural Network
https://github.com/fchollet/keras
111. Approaching “any” ML problem
AutoCompete: A Framework for Machine Learning Competitions, A.Thakur and A Krohn-Grimberghe, ICML AutoML Workshop, 2015
117. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
118. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
119. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
120. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
● Add new layer:
121. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
● Add new layer:
○ 1200-1500 neurons
122. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
● Add new layer:
○ 1200-1500 neurons
○ High dropout: 40-50%
123. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
● Add new layer:
○ 1200-1500 neurons
○ High dropout: 40-50%
● Very big network:
124. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
● Add new layer:
○ 1200-1500 neurons
○ High dropout: 40-50%
● Very big network:
○ 8000-10000 neurons in each layer
125. Selecting NNet Architecture
● Always use SGD or Adam (for fast convergence)
● Start low:
○ Single layer with 120-500 neurons
○ Batch normalization + ReLU
○ Dropout: 10-20%
● Add new layer:
○ 1200-1500 neurons
○ High dropout: 40-50%
● Very big network:
○ 8000-10000 neurons in each layer
○ 60-80% dropout
131. What are clickbaits?
● 10 things Apple didn’t tell you about the new iPhone
● What happened next will surprise you
● This is what the actor/actress from 90s looks like now
● What did Donald Trump just say about Obama and Clinton
● 9 things you must have to be a good data scientist
@abhi1thakur
133. What are clickbaits?
● Interesting titles
● Frustrating titles
● Seldomly good enough content
● Google penalizes clickbait content
● Facebook does the same
@abhi1thakur
134. The data
● Crawl buzzfeed, clickhole
● Crawl new york times, cnn
● ~10000 titles
○ Clickbaits: buzzfeed, clickhole
○ Non-clickbaits: new york times, cnn
○ ~5000 from either categories
@abhi1thakur
135. Good old TF-IDF
● Very powerful
● Used both character and word analyzers
@abhi1thakur
141. Is that it?
● No!
● Model predictions:
○ “Donald Trump” : 15% Clickbait
○ “Barack Obama”: 80% Clickbait
● Something was very wrong!
● TF-IDF didn’t capture the meanings
@abhi1thakur
142. Word2Vec
● Shallow neural networks
● Generates vectors of high dimension for every word
● Every word gets a position in space
● Similar words cluster together
@abhi1thakur
147. Does word2vec capture everything?
Do we have all we need only from titles?
What if content of website isn’t clickbait-y?
@abhi1thakur
148. The data
● Crawl Buzzfeed, NYT, CNN, clickhole, etc.
● Too much work
● Simple models
● Doubts about results
● Crawl public Facebook pages:
○ Buzzfeed
○ CNN
○ The New York Times
○ Clickhole
○ StopClickBaitOfficial
○ Upworthy
○ Wikinews
Facebook page scrapper is available here:
https://github.com/minimaxir/facebook-page-post-scraper
@abhi1thakur
149. The data
● link_name (the title of the URL shared)
● status_type (whether it’s a link, photo or a video)
● status_link (the actual URL)
@abhi1thakur
152. Feature Generation
● Size of the HTML (in bytes)
● Length of HTML
● Total number of links
● Total number of buttons
● Total number of inputs
● Total number of unordered lists
● Total number of ordered lists
● Total number of lists (ordered +
unordered)
@abhi1thakur
● Total Number of H1 tags
● Total Number of H2 tags
● Full length of all text in all H1
tags that were found
● Full length of all text in all H2
tags that were found
● Total number of images
● Total number of html tags
● Number of unique html tags
167. The Problem
➢ ~ 13 million questions (as of March, 2017)
➢ Many duplicate questions
➢ Cluster and join duplicates together
➢ Remove clutter
➢ First public data release: 24th January, 2017
168. Duplicate Questions
➢ How does Quora quickly mark questions as needing improvement?
➢ Why does Quora mark my questions as needing improvement/clarification
before I have time to give it details? Literally within seconds…
➢ What practical applications might evolve from the discovery of the Higgs
Boson?
➢ What are some practical benefits of discovery of the Higgs Boson?
➢ Why did Trump win the Presidency?
➢ How did Donald Trump win the 2016 Presidential Election?
169. Non-Duplicate Questions
➢ Who should I address my cover letter to if I'm applying for a big company like
Mozilla?
➢ Which car is better from safety view?""swift or grand i10"".My first priority is
safety?
➢ Mr. Robot (TV series): Is Mr. Robot a good representation of real-life hacking
and hacking culture? Is the depiction of hacker societies realistic?
➢ What mistakes are made when depicting hacking in ""Mr. Robot"" compared
to real-life cybersecurity breaches or just a regular use of technologies?
➢ How can I start an online shopping (e-commerce) website?
➢ Which web technology is best suitable for building a big E-Commerce
website?
170. The Data
➢ 400,000+ pairs of questions
➢ Initially data was very skewed
➢ Negative samples from related questions
➢ Not real distribution on Quora’s website
➢ Noise exists (as usual)
https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs
171. The Data
➢ 255045 negative samples (non-duplicates)
➢ 149306 positive samples (duplicates)
➢ 40% positive samples
172. The Data
➢ Average number characters in question1: 59.57
➢ Minimum number of characters in question1: 1
➢ Maximum number of characters in question1: 623
➢ Average number characters in question2: 60.14
➢ Minimum number of characters in question2: 1
➢ Maximum number of characters in question2: 1169
173. Basic Feature Engineering
➢ Length of question1
➢ Length of question2
➢ Difference in the two lengths
➢ Character length of question1 without spaces
➢ Character length of question2 without spaces
➢ Number of words in question1
➢ Number of words in question2
➢ Number of common words in question1 and question2
175. Fuzzy Features
➢ pip install fuzzywuzzy
➢ Uses Levenshtein distance
➢ QRatio
➢ WRatio
➢ Token set ratio
➢ Token sort ratio
➢ Partial token set ratio
➢ Partial token sort ratio
➢ etc. etc. etc.
https://github.com/seatgeek/fuzzywuzzy
177. TF-IDF
➢ TF(t) = Number of times a term t appears in a document / Total number of
terms in the document
➢ IDF(t) = log(Total number of documents / Number of documents with term t in
it)
➢ TF-IDF(t) = TF(t) * IDF(t)
tfidf = TfidfVectorizer(min_df=3, max_features=None,
strip_accents='unicode', analyzer='word',token_pattern=r'w{1,}',
ngram_range=(1, 2), use_idf=1, smooth_idf=1, sublinear_tf=1,
stop_words = 'english')
179. Fuzzy Features
➢ Also known as approximate string matching
➢ Number of “primitive” operations required to convert string to exact match
➢ Primitive operations:
○ Insertion
○ Deletion
○ Substitution
➢ Typically used for:
○ Spell checking
○ Plagiarism detection
○ DNA sequence matching
○ Spam filtering
185. Word2Vec Features
➢ Multi-dimensional vector for all the words in any dictionary
➢ Always great insights
➢ Very popular in natural language processing tasks
➢ Google news vectors 300d
186. Word2Vec Features
➢ Representing words
➢ Representing sentences
def sent2vec(s):
words = str(s).lower().decode('utf-8')
words = word_tokenize(words)
words = [w for w in words if not w in stop_words]
words = [w for w in words if w.isalpha()]
M = []
for w in words:
M.append(model[w])
M = np.array(M)
v = M.sum(axis=0)
return v / np.sqrt((v ** 2).sum())
187. W2V Features: WMD
Kusner, M., Sun, Y., Kolkin, N. & Weinberger, K.. (2015). From Word Embeddings To Document Distances.
188. W2V Features: Skew
➢ Skew = 0 for normal distribution
➢ Skew > 0: more weight in left tail
189. W2V Features: Kurtosis
➢ 4th central moment over the square of variance
➢ Types:
○ Pearson
○ Fisher: subtract 3.0 from result such that result is 0 for normal distribution
198. LSTM
➢ Long short term memory
➢ A type of RNN
➢ Learn long term dependencies
➢ Used two LSTM layers
199. 1D CNN
➢ One dimensional convolutional layer
➢ Temporal convolution
➢ Simple to implement:
for i in range(sample_length):
y[i] = 0
for j in range(kernel_length):
y[i] += x[i-j] * h[j]
201. Time Distributed Dense Layer
➢ TimeDistributed wrapper around dense layer
➢ TimeDistributed applies the layer to every temporal slice of input
➢ Followed by Lambda layer
➢ Implements “translation” layer used by Stephen Merity (keras snli model)
model1 = Sequential()
model1.add(Embedding(len(word_index) + 1,
300,
weights=[embedding_matrix],
input_length=40,
trainable=False))
model1.add(TimeDistributed(Dense(300, activation='relu')))
model1.add(Lambda(lambda x: K.sum(x, axis=1), output_shape=(300,)))
202. GloVe Embeddings
➢ Count based model
➢ Dimensionality reduction on co-occurrence counts matrix
➢ word-context matrix -> word-feature matrix
➢ Common Crawl
○ 840B tokens, 2.2M vocab, 300d vectors
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation
203. Basis of Deep Learning Model
➢ Keras-snli model: https://github.com/Smerity/keras_snli
204. Before Training DeepNets
➢ Tokenize data
➢ Convert text data to sequences
tk = text.Tokenizer(nb_words=200000)
max_len = 40
tk.fit_on_texts(list(data.question1.values) + list(data.question2.values.astype(str)))
x1 = tk.texts_to_sequences(data.question1.values)
x1 = sequence.pad_sequences(x1, maxlen=max_len)
x2 = tk.texts_to_sequences(data.question2.values.astype(str))
x2 = sequence.pad_sequences(x2, maxlen=max_len)
word_index = tk.word_index
205. Before Training DeepNets
➢ Initialize GloVe embeddings
embeddings_index = {}
f = open('data/glove.840B.300d.txt')
for line in tqdm(f):
values = line.split()
word = values[0]
coefs = np.asarray(values[1:], dtype='float32')
embeddings_index[word] = coefs
f.close()
206. Before Training DeepNets
➢ Create the embedding matrix
embedding_matrix = np.zeros((len(word_index) + 1, 300))
for word, i in tqdm(word_index.items()):
embedding_vector = embeddings_index.get(word)
if embedding_vector is not None:
embedding_matrix[i] = embedding_vector
218. Time to Train the DeepNet
➢ Total params: 174,913,917
➢ Trainable params: 60,172,917
➢ Non-trainable params: 114,741,000
➢ NVIDIA Titan X
219.
220. Combined Results
The deep network was trained on
an NVIDIA TitanX and took
approximately 300 seconds for
each epoch and took 10-15 hours
to train. This network achieved
an accuracy of 0.848 (~0.85).
221. Improving Further
➢ Cleaning the text data, e.g correcting mis-spellings
➢ POS tagging
➢ Entity recognition
➢ Combining deepnet with traditional ML models
222. Conclusion & References
➢ The deepnet gives near state-of-the-art result
➢ BiMPM model accuracy: 88%
Some reference:
➢ Zhiguo Wang, Wael Hamza and Radu Florian. "Bilateral Multi-Perspective
Matching for Natural Language Sentences," (BiMPM)
➢ Matthew Honnibal. "Deep text-pair classification with Quora's 2017 question
dataset," 13 February 2017. Retreived at
https://explosion.ai/blog/quora-deep-text-pair-classification
➢ Bradley Pallen’s work:
https://github.com/bradleypallen/keras-quora-question-pairs
223.
224. Natural Language
Processing
Pre-trained domain
knowledge
Classification of intent
Identify entities
(extracting information)
API
Analytics
Delegation to customer support
Delegation to back-end robots
INSTANT PROCESSING and END-TO-END AUTOMATION
Monitoring and AI training
Chat
Avatar
Text
(Speech)
225. Pre-defined replyEnquiry
Intent classificationPre-processing of enquiry
Stemming
Cross-language
Misspellings algorithm
1. Insurance
2. Vehicle
3. Car
4.Rules for practice driving
Conversation without API
You don’t need to adjust your car
insurance when practise driving with
a learner’s permit. In case of damage
it’s the supervisor with a full driver’s
license that shall write and sign the
insurance claim
Hey you, do you
knoww if my car
insruacne covers
practice driving??
226. Hi James, what’s the weather in
Berlin on Thursday?
Thursday’s forecast for Berlin is
partly sunny and mostly clouds.
Required value
- Location
Optional value
- Date
Conversation with API
Redirect to API
- Weather
227.
228. Thank you!
Questions / Comments?
All The Code:
❖ github.com/abhishekkrthakur
Get in touch:
➢ E-mail: abhishek4@gmail.com
➢ LinkedIn: bit.ly/thakurabhishek
➢ Kaggle: kaggle.com/abhishek
➢ Twitter: @abhi1thakur
If everything fails, use Xgboost