A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
A fast-paced introduction to Deep Learning concepts, such as activation functions, cost functions, back propagation, and then a quick dive into CNNs. Basic knowledge of vectors, matrices, and derivatives is helpful in order to derive the maximum benefit from this session.
The presentation briefly answers the questions:
1. What is Machine Learning?
2. Ideas behind Neural Networks?
3. What is Deep Learning? How different is it from NN?
4. Practical examples of applications.

For more information:
https://www.quora.com/How-does-deep-learning-work-and-how-is-it-different-from-normal-neural-networks-and-or-SVM
http://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma
https://www.youtube.com/watch?v=n1ViNeWhC24 - presentation by Ng
http://techtalks.tv/talks/deep-learning/58122/ - deep learning tutorial and slides - http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf
Deep learning for NLP - http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial
papers: http://www.cs.toronto.edu/~hinton/science.pdf
http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_ErhanCBV10.pdf
http://arxiv.org/pdf/1206.5538v3.pdf
http://arxiv.org/pdf/1404.7828v4.pdf
More recommendations - https://www.quora.com/What-are-the-best-resources-to-learn-about-deep-learning
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...Simplilearn
This Deep Learning Presentation will help you in understanding what is Deep learning, why do we need Deep learning, applications of Deep Learning along with a detailed explanation on Neural Networks and how these Neural Networks work. Deep learning is inspired by the integral function of the human brain specific to artificial neural networks. These networks, which represent the decision-making process of the brain, use complex algorithms that process data in a non-linear way, learning in an unsupervised manner to make choices based on the input. This Deep Learning tutorial is ideal for professionals with beginners to intermediate levels of experience. Now, let us dive deep into this topic and understand what Deep learning actually is.
Below topics are explained in this Deep Learning Presentation:
1. What is Deep Learning?
2. Why do we need Deep Learning?
3. Applications of Deep Learning
4. What is Neural Network?
5. Activation Functions
6. Working of Neural Network
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you’ll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms.
There is booming demand for skilled deep learning engineers across a wide range of industries, making this deep learning course with TensorFlow training well-suited for professionals at the intermediate to advanced level of experience. We recommend this deep learning online course particularly for the following professionals:
1. Software engineers
2. Data scientists
3. Data analysts
4. Statisticians with an interest in deep learning
A comprehensive tutorial on Convolutional Neural Networks (CNN) which talks about the motivation behind CNNs and Deep Learning in general, followed by a description of the various components involved in a typical CNN layer. It explains the theory involved with the different variants used in practice and also, gives a big picture of the whole network by putting everything together.
Next, there's a discussion of the various state-of-the-art frameworks being used to implement CNNs to tackle real-world classification and regression problems.
Finally, the implementation of the CNNs is demonstrated by implementing the paper 'Age ang Gender Classification Using Convolutional Neural Networks' by Hassner (2015).
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
A fast-paced introduction to Deep Learning concepts, such as activation functions, cost functions, back propagation, and then a quick dive into CNNs. Basic knowledge of vectors, matrices, and derivatives is helpful in order to derive the maximum benefit from this session.
The presentation briefly answers the questions:
1. What is Machine Learning?
2. Ideas behind Neural Networks?
3. What is Deep Learning? How different is it from NN?
4. Practical examples of applications.

For more information:
https://www.quora.com/How-does-deep-learning-work-and-how-is-it-different-from-normal-neural-networks-and-or-SVM
http://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma
https://www.youtube.com/watch?v=n1ViNeWhC24 - presentation by Ng
http://techtalks.tv/talks/deep-learning/58122/ - deep learning tutorial and slides - http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf
Deep learning for NLP - http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial
papers: http://www.cs.toronto.edu/~hinton/science.pdf
http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_ErhanCBV10.pdf
http://arxiv.org/pdf/1206.5538v3.pdf
http://arxiv.org/pdf/1404.7828v4.pdf
More recommendations - https://www.quora.com/What-are-the-best-resources-to-learn-about-deep-learning
What Is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutori...Simplilearn
This Deep Learning Presentation will help you in understanding what is Deep learning, why do we need Deep learning, applications of Deep Learning along with a detailed explanation on Neural Networks and how these Neural Networks work. Deep learning is inspired by the integral function of the human brain specific to artificial neural networks. These networks, which represent the decision-making process of the brain, use complex algorithms that process data in a non-linear way, learning in an unsupervised manner to make choices based on the input. This Deep Learning tutorial is ideal for professionals with beginners to intermediate levels of experience. Now, let us dive deep into this topic and understand what Deep learning actually is.
Below topics are explained in this Deep Learning Presentation:
1. What is Deep Learning?
2. Why do we need Deep Learning?
3. Applications of Deep Learning
4. What is Neural Network?
5. Activation Functions
6. Working of Neural Network
Simplilearn’s Deep Learning course will transform you into an expert in deep learning techniques using TensorFlow, the open-source software library designed to conduct machine learning & deep neural network research. With our deep learning course, you’ll master deep learning and TensorFlow concepts, learn to implement algorithms, build artificial neural networks and traverse layers of data abstraction to understand the power of data and prepare you for your new role as deep learning scientist.
Why Deep Learning?
It is one of the most popular software platforms used for deep learning and contains powerful tools to help you build and implement artificial neural networks.
Advancements in deep learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in deep learning models, learn to operate TensorFlow to manage neural networks and interpret the results.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms.
There is booming demand for skilled deep learning engineers across a wide range of industries, making this deep learning course with TensorFlow training well-suited for professionals at the intermediate to advanced level of experience. We recommend this deep learning online course particularly for the following professionals:
1. Software engineers
2. Data scientists
3. Data analysts
4. Statisticians with an interest in deep learning
Tijmen Blankenvoort, co-founder Scyfer BV, presentation at Artificial Intelligence Meetup 15-1-2014. Introduction into Neural Networks and Deep Learning.
Artificial Intelligence, Machine Learning, Deep Learning
The 5 myths of AI
Deep Learning in action
Basics of Deep Learning
NVIDIA Volta V100 and AWS P3
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Simplilearn
This presentation about Deep Learning is designed for beginners who want to learn Deep Learning from scratch. We will look at where Deep Learning is applied and what exactly this term means. We'll see how Deep Learning, Machine Learning, and AI are different and why Deep Learning even came into the picture. We will then proceed to look at Neural Networks, which are the core of Deep Learning. Before we move into the working of Neural Networks, we'll cover activation and cost functions. The video will also introduce you to the most popular Deep Learning platforms. We wrap it up with a demo in TensorFlow to predict if a person receives a salary above or below 50k. Now, let us get started and understand Deep Learning in detail.
Below topics are explained in this Deep Learning presentation:
1. Applications of Deep Learning
2. What is Deep Learning
3. Why is Deep Learning important
4. What are Neural Networks
5. Activation function
6. Cost function
7. How do Neural Networks work
8. Deep Learning platforms
9. Introduction to TensorFlow
10. Use case implementation using TensorFlow
Why Deep Learning?
It is one of the most popular software platforms used for Deep Learning and contains powerful tools to help you build and implement artificial Neural Networks.
Advancements in Deep Learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in Deep Learning models, learn to operate TensorFlow to manage Neural Networks and interpret the results. According to payscale.com, the median salary for engineers with Deep Learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
1. Understand the concepts of TensorFlow, its main functions, operations and the execution pipeline
2. Implement Deep Learning algorithms, understand Neural Networks and traverse the layers of data abstraction which will empower you to understand data like never before
3. Master and comprehend advanced topics such as convolutional Neural Networks, Recurrent Neural Networks, training deep networks and high-level interfaces
4. Build Deep Learning models in TensorFlow and interpret the results
5. Understand the language and fundamental concepts of Artificial Neural Networks
6. Troubleshoot and improve Deep Learning models
Learn more at https://www.simplilearn.com/deep-learning-course-with-tensorflow-training
Learn the fundamentals of Deep Learning, Machine Learning, and AI, how they've impacted everyday technology, and what's coming next in Artificial Intelligence technology.
The presentation is made on CNN's which is explained using the image classification problem, the presentation was prepared in perspective of understanding computer vision and its applications. I tried to explain the CNN in the most simple way possible as for my understanding. This presentation helps the beginners of CNN to have a brief idea about the architecture and different layers in the architecture of CNN with the example. Please do refer the references in the last slide for a better idea on working of CNN. In this presentation, I have also discussed the different types of CNN(not all) and the applications of Computer Vision.
Machine Learning and Real-World ApplicationsMachinePulse
This presentation was created by Ajay, Machine Learning Scientist at MachinePulse, to present at a Meetup on Jan. 30, 2015. These slides provide an overview of widely used machine learning algorithms. The slides conclude with examples of real world applications.
Ajay Ramaseshan, is a Machine Learning Scientist at MachinePulse. He holds a Bachelors degree in Computer Science from NITK, Suratkhal and a Master in Machine Learning and Data Mining from Aalto University School of Science, Finland. He has extensive experience in the machine learning domain and has dealt with various real world problems.
Deep Learning - Overview of my work IIMohamed Loey
Deep Learning Machine Learning MNIST CIFAR 10 Residual Network AlexNet VGGNet GoogleNet Nvidia Deep learning (DL) is a hierarchical structure network which through simulates the human brain’s structure to extract the internal and external input data’s features
Tijmen Blankenvoort, co-founder Scyfer BV, presentation at Artificial Intelligence Meetup 15-1-2014. Introduction into Neural Networks and Deep Learning.
Artificial Intelligence, Machine Learning, Deep Learning
The 5 myths of AI
Deep Learning in action
Basics of Deep Learning
NVIDIA Volta V100 and AWS P3
Deep Learning Tutorial | Deep Learning Tutorial For Beginners | What Is Deep ...Simplilearn
This presentation about Deep Learning is designed for beginners who want to learn Deep Learning from scratch. We will look at where Deep Learning is applied and what exactly this term means. We'll see how Deep Learning, Machine Learning, and AI are different and why Deep Learning even came into the picture. We will then proceed to look at Neural Networks, which are the core of Deep Learning. Before we move into the working of Neural Networks, we'll cover activation and cost functions. The video will also introduce you to the most popular Deep Learning platforms. We wrap it up with a demo in TensorFlow to predict if a person receives a salary above or below 50k. Now, let us get started and understand Deep Learning in detail.
Below topics are explained in this Deep Learning presentation:
1. Applications of Deep Learning
2. What is Deep Learning
3. Why is Deep Learning important
4. What are Neural Networks
5. Activation function
6. Cost function
7. How do Neural Networks work
8. Deep Learning platforms
9. Introduction to TensorFlow
10. Use case implementation using TensorFlow
Why Deep Learning?
It is one of the most popular software platforms used for Deep Learning and contains powerful tools to help you build and implement artificial Neural Networks.
Advancements in Deep Learning are being seen in smartphone applications, creating efficiencies in the power grid, driving advancements in healthcare, improving agricultural yields, and helping us find solutions to climate change. With this Tensorflow course, you’ll build expertise in Deep Learning models, learn to operate TensorFlow to manage Neural Networks and interpret the results. According to payscale.com, the median salary for engineers with Deep Learning skills tops $120,000 per year.
You can gain in-depth knowledge of Deep Learning by taking our Deep Learning certification training course. With Simplilearn’s Deep Learning course, you will prepare for a career as a Deep Learning engineer as you master concepts and techniques including supervised and unsupervised learning, mathematical and heuristic aspects, and hands-on modeling to develop algorithms. Those who complete the course will be able to:
1. Understand the concepts of TensorFlow, its main functions, operations and the execution pipeline
2. Implement Deep Learning algorithms, understand Neural Networks and traverse the layers of data abstraction which will empower you to understand data like never before
3. Master and comprehend advanced topics such as convolutional Neural Networks, Recurrent Neural Networks, training deep networks and high-level interfaces
4. Build Deep Learning models in TensorFlow and interpret the results
5. Understand the language and fundamental concepts of Artificial Neural Networks
6. Troubleshoot and improve Deep Learning models
Learn more at https://www.simplilearn.com/deep-learning-course-with-tensorflow-training
Learn the fundamentals of Deep Learning, Machine Learning, and AI, how they've impacted everyday technology, and what's coming next in Artificial Intelligence technology.
The presentation is made on CNN's which is explained using the image classification problem, the presentation was prepared in perspective of understanding computer vision and its applications. I tried to explain the CNN in the most simple way possible as for my understanding. This presentation helps the beginners of CNN to have a brief idea about the architecture and different layers in the architecture of CNN with the example. Please do refer the references in the last slide for a better idea on working of CNN. In this presentation, I have also discussed the different types of CNN(not all) and the applications of Computer Vision.
Machine Learning and Real-World ApplicationsMachinePulse
This presentation was created by Ajay, Machine Learning Scientist at MachinePulse, to present at a Meetup on Jan. 30, 2015. These slides provide an overview of widely used machine learning algorithms. The slides conclude with examples of real world applications.
Ajay Ramaseshan, is a Machine Learning Scientist at MachinePulse. He holds a Bachelors degree in Computer Science from NITK, Suratkhal and a Master in Machine Learning and Data Mining from Aalto University School of Science, Finland. He has extensive experience in the machine learning domain and has dealt with various real world problems.
Deep Learning - Overview of my work IIMohamed Loey
Deep Learning Machine Learning MNIST CIFAR 10 Residual Network AlexNet VGGNet GoogleNet Nvidia Deep learning (DL) is a hierarchical structure network which through simulates the human brain’s structure to extract the internal and external input data’s features
Deep Learning - The Past, Present and Future of Artificial IntelligenceLukas Masuch
In the last couple of years, deep learning techniques have transformed the world of artificial intelligence. One by one, the abilities and techniques that humans once imagined were uniquely our own have begun to fall to the onslaught of ever more powerful machines. Deep neural networks are now better than humans at tasks such as face recognition and object recognition. They’ve mastered the ancient game of Go and thrashed the best human players. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new hype? How is Deep Learning different from previous approaches? Let’s look behind the curtain and unravel the reality. This talk will introduce the core concept of deep learning, explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why “deep learning is probably one of the most exciting things that is happening in the computer industry“ (Jen-Hsun Huang – CEO NVIDIA).
Automatic Attendace using convolutional neural network Face Recognitionvatsal199567
Automatic Attendance System will recognize the face of the student through the camera in the class and mark the attendance. It was built in Python with Machine Learning.
MachinaFiesta: A Vision into Machine Learning 🚀GDSCNiT
🕵️♂️ Embark on an exhilarating journey into the realm of Machine learning and Generative AI with MachinaFiesta! 🚀. Join us for MachinaFiesta, a two-hour event exploring the fascinating world of machine learning and generative AI where you can Vision, Innovate and learn new technologies.
Slide contets:
🎤 Brief introduction to the agenda and speakers of the event
🌐 Get to know the importance and future prospects of machine learning
🧠 Interactive session on core machine learning concepts
🚀 Exploration of cutting-edge generative AI advancements
🤖 Introduction to Gemini, the open-source factual language model
🤔Discussion on Gemini's capabilities and potential applications in research and development
Traditional Machine Learning had used handwritten features and modality-specific machine learning to classify images, text or recognize voices. Deep learning / Neural network identifies features and finds different patterns automatically. Time to build these complex tasks has been drastically reduced and accuracy has exponentially increased because of advancements in Deep learning. Neural networks have been partly inspired from how 86 billion neurons work in a human and become more of a mathematical and a computer problem. We will see by the end of the blog how neural networks can be intuitively understood and implemented as a set of matrix multiplications, cost function, and optimization algorithms.
Deep Learning: concepts and use cases (October 2018)Julien SIMON
An introduction to Deep Learning theory
Neurons & Neural Networks
The Training Process
Backpropagation
Optimizers
Common network architectures and use cases
Convolutional Neural Networks
Recurrent Neural Networks
Long Short Term Memory Networks
Generative Adversarial Networks
Getting started
Understanding Deep Learning & Parameter Tuning with MXnet, H2o Package in RManish Saraswat
Simple guide which explains deep learning and neural network with hands on experience in R using MXnet and H2o package. It also explains gradient descent and backpropagation algorithm.
Complete tutorial: http://blog.hackerearth.com/understanding-deep-learning-parameter-tuning-with-mxnet-h2o-package-r
Deep learning is a part of machine learning, which involves the use of computer algorithms to learn, improve and evolve on its own. Deep learning may be considered similar to machine learning. However, while machine learning works with simple concepts, deep learning uses artificial neural networks, which imitate the way humans learn and think.
A neural network is a method in artificial intelligence that teaches computers to process data in a way that is inspired by the human brain. It is a type of machine learning process, called deep learning, that uses interconnected nodes or neurons in a layered structure that resembles the human brain.
Machine learning workshop using Orange datamining frameworkAmr Rashed
Machine learning workshop using Orange
youtube video
https://youtu.be/wpgQY5f2hOo
Topic: Data mining, analysis, and visualization Using Python-Orange
Start Time : Mar 27, 2021 08:30 PM
Meeting Recording:
https://zoom.us/rec/share/esp-FwuaZs3ekc-yYNK74EV7Jn-TSM1TpmT2fTbe8Oy99MKmsdDhQigRneEyQaM-.JNssJnqQqtrAVgQO
Sachpazis:Terzaghi Bearing Capacity Estimation in simple terms with Calculati...Dr.Costas Sachpazis
Terzaghi's soil bearing capacity theory, developed by Karl Terzaghi, is a fundamental principle in geotechnical engineering used to determine the bearing capacity of shallow foundations. This theory provides a method to calculate the ultimate bearing capacity of soil, which is the maximum load per unit area that the soil can support without undergoing shear failure. The Calculation HTML Code included.
Saudi Arabia stands as a titan in the global energy landscape, renowned for its abundant oil and gas resources. It's the largest exporter of petroleum and holds some of the world's most significant reserves. Let's delve into the top 10 oil and gas projects shaping Saudi Arabia's energy future in 2024.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
Automobile Management System Project Report.pdfKamal Acharya
The proposed project is developed to manage the automobile in the automobile dealer company. The main module in this project is login, automobile management, customer management, sales, complaints and reports. The first module is the login. The automobile showroom owner should login to the project for usage. The username and password are verified and if it is correct, next form opens. If the username and password are not correct, it shows the error message.
When a customer search for a automobile, if the automobile is available, they will be taken to a page that shows the details of the automobile including automobile name, automobile ID, quantity, price etc. “Automobile Management System” is useful for maintaining automobiles, customers effectively and hence helps for establishing good relation between customer and automobile organization. It contains various customized modules for effectively maintaining automobiles and stock information accurately and safely.
When the automobile is sold to the customer, stock will be reduced automatically. When a new purchase is made, stock will be increased automatically. While selecting automobiles for sale, the proposed software will automatically check for total number of available stock of that particular item, if the total stock of that particular item is less than 5, software will notify the user to purchase the particular item.
Also when the user tries to sale items which are not in stock, the system will prompt the user that the stock is not enough. Customers of this system can search for a automobile; can purchase a automobile easily by selecting fast. On the other hand the stock of automobiles can be maintained perfectly by the automobile shop manager overcoming the drawbacks of existing system.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
Welcome to WIPAC Monthly the magazine brought to you by the LinkedIn Group Water Industry Process Automation & Control.
In this month's edition, along with this month's industry news to celebrate the 13 years since the group was created we have articles including
A case study of the used of Advanced Process Control at the Wastewater Treatment works at Lleida in Spain
A look back on an article on smart wastewater networks in order to see how the industry has measured up in the interim around the adoption of Digital Transformation in the Water Industry.
Hybrid optimization of pumped hydro system and solar- Engr. Abdul-Azeez.pdffxintegritypublishin
Advancements in technology unveil a myriad of electrical and electronic breakthroughs geared towards efficiently harnessing limited resources to meet human energy demands. The optimization of hybrid solar PV panels and pumped hydro energy supply systems plays a pivotal role in utilizing natural resources effectively. This initiative not only benefits humanity but also fosters environmental sustainability. The study investigated the design optimization of these hybrid systems, focusing on understanding solar radiation patterns, identifying geographical influences on solar radiation, formulating a mathematical model for system optimization, and determining the optimal configuration of PV panels and pumped hydro storage. Through a comparative analysis approach and eight weeks of data collection, the study addressed key research questions related to solar radiation patterns and optimal system design. The findings highlighted regions with heightened solar radiation levels, showcasing substantial potential for power generation and emphasizing the system's efficiency. Optimizing system design significantly boosted power generation, promoted renewable energy utilization, and enhanced energy storage capacity. The study underscored the benefits of optimizing hybrid solar PV panels and pumped hydro energy supply systems for sustainable energy usage. Optimizing the design of solar PV panels and pumped hydro energy supply systems as examined across diverse climatic conditions in a developing country, not only enhances power generation but also improves the integration of renewable energy sources and boosts energy storage capacities, particularly beneficial for less economically prosperous regions. Additionally, the study provides valuable insights for advancing energy research in economically viable areas. Recommendations included conducting site-specific assessments, utilizing advanced modeling tools, implementing regular maintenance protocols, and enhancing communication among system components.
Quality defects in TMT Bars, Possible causes and Potential Solutions.PrashantGoswami42
Maintaining high-quality standards in the production of TMT bars is crucial for ensuring structural integrity in construction. Addressing common defects through careful monitoring, standardized processes, and advanced technology can significantly improve the quality of TMT bars. Continuous training and adherence to quality control measures will also play a pivotal role in minimizing these defects.
2. Deep Learning
DEEP LEANING in Bioinformatics, Conclusion
RECURRENT NN,DEEP LEARNING TOOLS
Types of Networks , Convolution Neural Networks
DEEP NN ARCHITECTURE, PROBLEM SPACE
WHAT IS DEEP LEARNING, DEEP LEARNING BASICS
BIG PLAYERS, APPLICATIONS
A Brief History, MACHINE LEARNING BASICS
MOTIVATIONS, WHY DEEP NN
AGENDA
3. Motivations
I have worked all my life in machine learning, and I’ve
never seen one algorithm knock over benchmarks like
deep learning . Andrew Ng(Stanford & Baidu)
Deep learning is an algorithm which has no theoretical
limitations of what it can learn ;the more data you give
and the more computational time you provide ;the
better it is-Geoffrey Hilton(Google)
Human-level artificial intelligence has the potential to
help humanity thrive more than any invention that has
come before it –Dileep George(Co-Founder Vicarious)
For a very long time it will be a complementary tool that
human scientists and human experts can use to help
them with the things that humans are not naturally
good- Demis Hassabis (Co-Founder Deep Mind)
6. The Problem Space
0 Image classification is the task of taking an input image and
outputting a class (a cat, dog, etc) or a probability of classes that
best describes the image.
0 For humans, this task of recognition is one of the first skills we
learn from the moment we are born .
0 Without even thinking twice, we’re able to quickly and seamlessly
identify the environment and objects that surround us.
0 When we see an image we are able to immediately characterize the
scene and give each object a label, all without even consciously
noticing.
0 These skills of being able to quickly recognize patterns, generalize
from prior knowledge, and adapt to different image environments
are ones that we do not share with our fellow machines.
7. The Problem Space : inputs & outputs
0 Depending on the resolution and size of the image, it will see a 32 x
32 x 3 array of numbers
0 These numbers, while meaningless to us when we perform image
classification, are the only inputs available to the computer.
0 The idea is that you give the computer this array of numbers and it
will output numbers that describe the probability of the image
being a certain class (.80 for cat, .15 for dog, .05 for bird, etc).
8. The Problem Space :What We Want the Computer to Do
0 To be able to differentiate between all the images it’s given
and figure out the unique features that make a dog a dog or
that make a cat a cat.
0 This is the process that goes on in our minds
subconsciously as well.
0 When we look at a picture of a dog, we can classify it as
such if the picture has identifiable features such as paws or
4 legs.
0 In a similar way, the computer is able perform image
classification by looking for low level features such as
edges and curves, and then building up to more abstract
concepts through a series of convolutional layers.
9. Why Deep Neural Network
- Imagine you have extracted features from sensors .
- The dimension of each sample is around 800
- You have 70,000 samples (trial)
- What method would you apply?
- You may have several ways to do it
0 Reduce the dimension from 800 to 40 by using a feature selection or
dim. reduction technique
0 What you did here is “Finding a good representation”
0 - Then, you may apply a classification methods to classify 10 classes
But, what if
- You have no idea for feature selection?
- The dimension is much higher than 800 and you have more classes.
10. Why Deep Neural Network
0 Drawbacks -Back-Propagation:
0 Scalability -does not scale well over multiple layers.
0 very slow to converge.
0“Vanishing gradient problem “ errors shrink exponentially with the
number of layers
0Thus makes poor use of many layers.
0This is the reason most feed forward NN have only three layers.
0 It got stuck at local optima
0 When the network outputs have not got their desired signals, the
activation functions in the hidden layer are saturating.
0These were often surprisingly good but there was no good theory.
0 Traditional Machine Learning doesn’t work well when
features can’t be explicitly defined.
11. A Brief History
0 2012 was the first year that neural nets grew to prominence
as Alex Krizhevsky used them to win that year’s ImageNet
competition (basically, the annual Olympics of computer
vision), dropping the classification error record from 26% to
15%, an astounding improvement at the time.
12. Machine Learning –Basics
Introduction
Machine Learning is a type of Artificial
Intelligence that provides computers with the
ability to learn without being explicitly
programmed. Provides various techniques that
can learn from and make predictions on data.
13. Supervised Learning : Learning with a labeled
training set
Example : e-mail spam detector with training set of
already labeled emails
Unsupervised Learning :Discovering patterns in
unlabeled data
Example :cluster similar documents based on the text
content
Reinforcement learning : learning based on feedback
or reward.
Example :learn to play chess by wining or losing
Machine Learning –Basics
Learning Approach
15. Big Players :Superstar Researcher
Jeoferry Hilton : University of Toronto
&Google
Yann Lecun : New Yourk University
&Facebook
Andrew Ng : Stanford & Baidu
Yoshua Bengio : University of Montreal
Jurgen schmidhuber : Swiss AI Lab &
NNAISENSE
20. What is Deep Learning(DL)
Part or a powerful class of the machine learning field of
learning representations of data. Exceptional effective at
learning patterns.
Utilize learning algorithms that derive meaning out of data
by using hierarchy of multiple layers that mimic the neural
networks of our brain.
If you provide the system tons of information , it begins to
understand it and respond in useful ways.
Stacked “Neural Network”. Is usually indicates “Deep Neural Network”.
21. What is Deep Learning (DL)
• Collection of simple, trainable mathematical functions.
• Learning methods which have deep architecture.
• It often allows end-to-end learning.
• It automatically finds intermediate representation. Thus, it can
be regarded as a representation learning.
22. Deep Learning Basics :Neuron
An artificial neuron contains a nonlinear activation function and has several
incoming and outgoing weighted connections.
Neurons are trained filters and detect specific features or patterns (e.g.
edge, nose) by receiving weighted input, transforming it with the activation
function and passing it to the outgoing connections
23. Commonalities with real brains:
● Each neuron is connected to a small subset of other neurons.
● Based on what it sees, it decides what it wants to say.
● Neurons learn to cooperate to accomplish the task.
• The vental pathway in the visual cortex has multiple stages
• There exist a lot of intermediate representations
Deep Learning Basics :Neuron
24. Deep Learning Basics :Training process
Learns by generating an error signal that measures the difference between the
predictions of the network and the desired values and then using this error signal
to change the weights (or parameters) so the predictions get more accurate.
25. Deep Learning Basics :Gradient Descent
Gradient Descent/optimization finds the (local)the minimum of the cost function(used
to calculate the output error) and is used to adjust the weights.
Curve represents is the network's error relative to the position of a single weight.
X axis weight Y axiserror
Oversimplified Gradient Descent:
Calculate slope at current position
• If slope is negative, move right
• If slope is positive, move left
• (Repeat until slope == 0)
26. Deep Learning Basics :Gradient Descent Problems
Problem 1: When slopes are too big Solution 1: Make Slopes Smaller
Problem 2: Local Minimums-Solution 2: Multiple Random Starting States
27. Deep Neural Network :Architecture
A Deep Neural Network consists of a hierarchy of layers, whereby each layer
transforms the input data into more abstract representations (e.g edge -
>nose -> face). The output layer combines those features to make predictions.
28. Deep Neural Network :Architecture
Consists of one input ,one output and multiple fully-connected hidden layers in
between. Each layer is represented as a series of neurons and progressively extracts
higher and higher-level features of the input until the final layer essentially makes a
decision about what the input shows. The more layer the network has, the higher-level
features it will learn.
30. Types of Networks used for Deep Learning
Convolutional Neural Networks(Convnet,CNN)
Recurrent Neural Networks(RNN)
Long Short Term Memory (LSTM) networks
Deep/Restricted Boltzmann Machines (RBM)
Deep Q-networks
Deep Belief Networks(DBN)
Deep Stacking Networks
31. Convolution Neural Networks(CNN):Basic Idea
0 This idea was expanded upon by a fascinating experiment by
Hubel and Wiesel in 1962 where they showed that some
individual neuronal cells in the brain responded (or fired)
only in the presence of edges of a certain orientation.
0 For example, some neurons fired when exposed to vertical
edges and some when shown horizontal or diagonal edges.
Hubel and Wiesel found out that all of these neurons were
organized in a columnar architecture and that together, they
were able to produce visual perception.
0 This idea of specialized components inside of a system
having specific tasks (the neuronal cells in the visual cortex
looking for specific characteristics) is one that machines use
as well, and is the basis behind CNNs.
32. Convolution Neural Networks(CNN)
• Convolutional neural networks learn a complex representation of visual data
using vast amounts of data .they are inspired by human visual system and learn
multiple layers of transformations , which are applied on top of each other to
extract progressively more sophisticated representation of the input .
DEFENITION
• Inspired by the visual cortex and Pioneered by Yann Lecun (NYU).
• CNN have multiple types of layers ,the first of which is the Convolutional
layer.
Notes
34. CNN Structure :First Layer – Convolution
Convolution layer is a feature detector that auto magically learns to filter out
not needed information from an input by using convolution kernel.
37. Examples of Actual Visualizations
End-to-End Learning
A pipeline of successive Layers
• Layers produce features of higher and higher abstractions
– Initial Layer capture low-level features (e.g. edges or corners)
– Middle Layer capture mid-level features (object parts e.g. circles, squares, textures)
– Last Layer capture high level, class specific features (object model e.g. face detector)
• Preferably, input as raw as possible
– Pixels for computer vision, words for NLP.
38. Batch Normalization Layer
0 Batch Normalization is a technique to provide any layer in a Neural Network with
inputs that are zero mean/unit variance .
0 Use batch normalization layers between convolutional layers and nonlinearities such as
ReLU layers to speed up network training and reduce the sensitivity to network
initialization.
0 The layer first normalizes the activations of each channel by subtracting the mini-batch
mean and dividing by the mini-batch standard deviation. Then, the layer shifts the input
by an offset β and scales it by a scale factor γ. β and γ are themselves learnable
parameters that are updated during network training.
39. Nonlinear Activation Function(Relu)
•Rectified Linear Unit (ReLU) module .
•Activation function 𝑎=ℎ(𝑥)=max(0,𝑥).
Most deep networks use ReLU –max(0,x)-nowadays for hidden
layers ,since it trains much faster ,is more expressive than logistic
function and prevents the gradient vanishing problem.
Non-linearity is needed to learn complex (non
linear)representations of data ,otherwise the NN would be just a
linear function .
40. Vanishing Gradient Problem
0 It is a difficulty founds in training ANN with gradient-based learning
methods and backpropagation.
0 In such methods, each of the neural network's weights receives an
update proportional to the gradient of the error function with respect
to the current weight in each iteration of training.
0 Traditional activation functions such as the hyperbolic
tangent function have gradients in the range (−1, 1), and
backpropagation computes gradients by the chain rule.
0 This has the effect of multiplying n of these small numbers to
compute gradients of the "front" layers in an n-layer network.
0 This means that the gradient (error signal) decreases exponentially
with n while the front layers train very slowly.
41. Local Response Normalization Layer
0 we want to have some kind of inhibition scheme.
0 Neurobiology concept “lateral inhibition”.
0 Useful when we are dealing with ReLU neurons(unbounded activations ).
0 Detect high frequency features with a large response.
0 If we normalize around the local neighborhood of the excited neuron, it
becomes even more sensitive as compared to its neighbors.
0 Inhibit the responses that are uniformly large in any given local
neighborhood.
0 If all the values are large, then normalizing those values will decrease all
of them.
0 Inhibition and boost the neurons with relatively larger activations.
0 Two types of normalizations available in Caffe (same and cross channel)
where K, α, and β are the hyperparameters in the
normalization, and ss is the sum of squares of the
elements in the normalization window.
42. Max and AVG Pooling Layer
0 Pooling layers compute the max or average value of a particular feature over a
region of the input data (downsizing of input images).
0 Also helps to detect objects in some unusual places and reduces memory size.
0 Aggregate multiple values into a single value
0 – Reduces the size of the layer output/input to next layer ->Faster computations
0 – Keeps most important information for the next layer
0 • Max pooling/Average pooling
43. Over Fitting
0 Over Fitting. This term refers to when a model is so tuned to the training
examples that it is not able to generalize well for the validation and test sets.
0 A symptom of over Fitting is having a model that gets 100% or 99% on the
training set, but only 50% on the test data.
0 Implement dropout layers in order to combat the problem of over fitting to the
training data.
44. Dropout Layers
0 after training, the weights of the network are so tuned to the
training examples they are given that the network doesn’t
perform well when given new examples.
0 The idea of dropout is simplistic in nature. This layer “drops out”
a random set of activations in that layer by setting them to zero.
Simple as that.
0 Benefits
0 it forces the network to be redundant. By that the network should
be able to provide the right classification or output for a specific
example even if some of the activations are dropped out.
0 It makes sure that the network isn’t getting too “fitted” to the
training data and thus helps alleviate the over fitting problem.
0 An important note is that this layer is only used during training,
and not during test time.
45. Output Layer –Soft-max Activation Function
0 For classification problems the sigmoid function can only handle two classes, which
is not what we expect.
0 The softmax function squashes the outputs of each unit to be between 0 and 1, just
like a sigmoid function.
0 But it also divides each output such that the total sum of the outputs is equal to 1 .
0 The output of the softmax function is equivalent to a categorical probability
distribution, it tells you the probability that any of the classes are true.
46. Output Layer –Regression
0 You can also use ConvNets for regression problems, where the
target (output) variable is continuous.
0 In such cases, a regression output layer must follow the final
fully connected layer. You can create a regression layer using
the regressionLayer function.
0 The default loss function for a regression layer is the mean
squared error.
0 where ti is the target output, and yi is the network’s prediction
for the response variable corresponding to observation i.
47. Transfer Learning
0 Transfer learning is the process of taking a pre-trained model (the
weights and parameters of a network that has been trained on a
large dataset by somebody else) and “fine-tuning” the model with
your own dataset.
0 Pre-trained model will act as a feature extractor.
0 Freeze the weights of all the other layers .
0 Remove the last layer of the network and replace it with your own
classifier.
0 Train the network normally.
48. Cont.:Transfer Learning
• Transfer from A (image recognition) to B (radiology images)
• When transfer learning makes sense
1. Task A and B have same input X.
2. You have a lot more data for Task A than Task B.
3. Low level features from A could be helpful for learning B.
4. Weights initialization.
• When transfer learning doesn’t makes sense
1. You have a lot more data for Task B than Task A.
49. Data Augmentation Techniques
0 Approaches that alter the training data in ways that
change the array representation while keeping the label
the same.
0 They are a way to artificially expand your dataset. Some
popular augmentations people use are gray scales ,
horizontal flips, vertical flips, random crops, color jitters,
translations, rotations, and much more .
ZCA Whitening Random Rotations Random shift Random flipExample MNIST
images
50. Recurrent Neural Network (RNN)
RNNs are general computers which can learn algorithms to map
input sequences to output sequences (flexible –sized vectors).
The output vector’s contents influenced by the entire history of
input.
Applications
• In time series prediction , adaptive robotics, handwriting
recognition ,image classification, speech recognition,
stock market prediction, and other sequence learning
problems .every thing can be processed sequentially
53. Usage Requirements
Large data set with good quality (input –output
mapping)
Measurable and describable goals (define the
cost)
Enough computing power(AWS GPU Instance)
Excels in tasks where the basic unit (pixel, word)
has very little meaning in itself, but the
combination of such units has a useful meaning.
54. Deep Learning : Benefits
Robust
• No need to design the features ahead of time –features are
automatically learned to be optimal for the task at hand
• Robustness to natural variations in the data is automatically learned
Generalizable
• The same neural network approach can be used for many different
applications and data types
Scalable
• Performance improves with more data ,method is massively
parallelizable
55. Deep Learning: Weakness
1
• Deep learning requires a large dataset, hence long training period.
2
• In term of cost, Machine learning methods like SVM and other tree
ensembles are very easily deployed even by relative machine learning
novices and can usually get you reasonably good results
3
• Deep learning methods tends to learn everything. It’s better to encode
prior knowledge about structure of images (or audio or text).
4
• The learned features are often difficult to understand. Many vision
features are also not really human-understandable (e.g,
concatenations/combinations of different features).
5
• Requires a good understanding of how to model multiple modalities
with traditional tools.
56. Pre-trained Deep Learning models
• Authors :Alex Krizhevsky, Ilya Sutskever,
and Geoffrey Hinton
• Winner 2012.
• Trained the network on ImageNet data,
which contained over 15 million
• used for classification with 1000 possible
categories
• Use 11x11 sized filters in the first layer.
• Used ReLU for the nonlinearity functions .
• Used data augmentation techniques that
consisted of image translations,
horizontal reflections, and patch
extractions.
• Implemented dropout layers.
• Trained the model using batch stochastic
gradient descent, with specific values for
momentum and weight decay.
• Trained on two GTX 580 GPUs for five to
six days
• This model achieved an 15.4% error rate.
built by Matthew Zeiler and Rob Fergus
from NYU.
Winner 2013.
Very similar architecture to AlexNet, except
for a few minor modifications.
ZF Net trained on only 1.3 million images.
ZF Net used filters of size 7x7
Used ReLUs for their activation functions,
cross-entropy loss for the error function,
and trained using batch stochastic gradient
descent.
Trained on a GTX 580 GPU for twelve days.
Developed a visualization technique named
Deconvolutional Network, which helps to
examine different feature activations and
their relation to the input space. Called
“deconvnet” because it maps features to
pixels (the opposite of what a convolutional
layer does).
This model achieved an 11.2% error rate
AlexNet (2012) ZF Net (2013)
57. Pre-trained Deep Learning models
• Built by Karen Simonyan and Andrew Zisserman
of the University of Oxford
• Not a winner .
• 19 layer CNN , used 3x3 sized filters with stride
and pad of 1 , 2x2 max pooling layers with stride 2.
• The combination of two 3x3 conv layers has an
effective receptive field of 5x5.
• Decrease in the number of parameters.
• Also, with two conv layers, we’re able to use two
ReLU layers instead of one. 3 conv layers back to
back have an effective receptive field of 7x7.
• Interesting to notice that the number of filters
doubles after each max pool layer. This reinforces
the idea of shrinking spatial dimensions, but
growing depth.
• Worked well on both image classification and
localization tasks.
• used a form of localization as regression Built
model with the Caffe toolbox.
• Used scale jittering as one data augmentation
technique during training.
• Used ReLU layers after each conv layer and trained
with batch gradient descent.
• Trained on 4 Nvidia Titan Black GPUs for two to
three weeks.
• This achieved an 7.3% error rate
• GoogLeNet is a 22 layer CNN.
• Winner 2014
• Used 9 Inception modules in the whole
architecture, with over 100 layers in
total.
• No use of fully connected layers! They
use an average pool instead, to go from a
7x7x1024 volume to a 1x1x1024
volume.
• Uses 12x fewer parameters than
AlexNet.
• During testing, multiple crops of the
same image were created, fed into the
network, and the softmax probabilities
were averaged to give us the final
solution.
• Utilized concepts from R-CNN for their
detection model.
• This model places notable consideration
on memory and power usage
• Trained on “a few high-end GPUs within
a week”.
• This achieved an 6.7% error rate
VGG Net (2014) Google Net (2015)
59. Pre-trained Deep Learning models
Microsoft ResNet (2015)
input x go through conv-relu-conv series.
152 layer, “Ultra-deep” – Yann LeCun.
Winner 2015.
After only the first 2 layers, the spatial size gets compressed from an input volume of 224x224 to a 56x56
volume.
increase of layers in plain nets result in higher training and test error (author claims).
The authors believe that “it is easier to optimize the residual mapping than to optimize the original,
unreferenced mapping
The group tried a 1202-layer network, but got a lower test accuracy, presumably due to over-fitting.
Trained on an 8 GPU machine for two to three weeks.
Achieved an 3.6 % error rate.
60. 2016 LSVRC winner(CUImage team)
0 Compared with CUImage submission in ILSVRC 2015, the new components are as follows.
(1) The models are pretrained for 1000-class object detection task using the approach in [a] but adapted to
the fast-RCNN for faster detection speed.
(2) The region proposal is obtained using the improved version of CRAFT in [b].
(3) A GBD network [c] with 269 layers is fine-tuned on 200 detection classes with the gated bidirectional
network (GBD-Net), which passes messages between features from different support regions during both
feature learning and feature extraction. The GBD-Net is found to bring ~3% mAP improvement on the
baseline 269 model and ~5% mAP improvement on the Batch normalized GoogleNet.
(4) For handling their long-tail distribution problem, the 200 classes are clustered. Different from the
original implementation in [d] that learns several models, a single model is learned, where different
clusters have both shared and distinguished feature representations.
(5) Ensemble of the models using the approaches mentioned above lead to the final result in the provided
data track.
(6) For the external data track, we propose object detection with landmarks. Comparing to the standard
bounding box centric approach, our landmark centric approach provides more structural information and
can be used to improve both the localization and classification step in object detection. Based on the
landmark annotations provided in [e], we annotate 862 landmarks from 200 categories on the training set.
Then we use them to train a CNN regressor to predict landmark position and visibility of each proposal in
testing images. In the classification step, we use the landmark pooling on top of the fully convolutional
network, where features around each landmark are mapped to be a confidence score of the corresponding
category. The landmark level classification can be naturally combined with standard bounding box level
classification to get the final detection result.
(7) Ensemble of the models using the approaches mentioned above lead to the final result in the external
data track. The fastest publicly available multi-GPU caffe code is our strong support [f].
61. 2017 LSVRC winner(BDAT team)
0 Adaptive attention[1] and deep combined convolutional
models[2,3] are used for LOC task.
0 Deep residual learning for image recognition.
0 Scale[4,5,6], context[7], sampling and deep combined
convolutional networks[2,3] are considered for DET task.
0 Object density estimation is used for score re-rank
63. Solution 1:Exhaustively search for objects.
0 Problem: Extremely slow, must process tens of thousands
of candidate objects.
64. Solution 2:
Running a scanning detector is cheaper than running a recognizer, so
do that first.
1. Exhaustively search for candidate objects with a generic detector.
2. Run recognition algorithm only on candidate objects.
Problem: What about oddly-shaped
objects? Will we need to scan with windows of many different shapes?
65. Segmentation
0 Idea: If we correctly segment the image before running object
recognition, we can use our segmentations as candidate objects.
0 Advantages: Can be efficient, makes no assumptions about
object sizes or shapes.
66. Segmentation is Hard
0 As we saw in Project 1, it’s not always clear what separates an
object.
67. Segmentation is Hard
0 As we saw in Project 1, it’s not always clear what separates
an object.
68. Selective Search
Goals:
1. Detect objects at any scale.
a. Hierarchical algorithms are good at this.
2. Consider multiple grouping criteria.
a. Detect differences in color, texture, brightness, etc.
3. Be fast.
Idea: Use bottom-up grouping of image regions to generate
a hierarchy of small to large regions.
69. Selective Search
Step 1: Generate initial sub-segmentation
Goal: Generate many regions, each of which belongs to at
most one object.
Using the method described by Felzenszwalb et al. from
week 1 works well.
70. Selective Search
Step 2: Recursively combine similar regions into
larger ones.
Greedy algorithm:
1. From set of regions, choose two that are most similar.
2. Combine them into a single, larger region.
3. Repeat until only one region remains.
This yields a hierarchy of successively larger regions, just
like we want.
71. Similarity
What do we mean by “similarity”?
Goals:
1. Use multiple grouping criteria.
2. Lead to a balanced hierarchy of small to large objects.
3. Be efficient to compute: should be able to quickly combine
measurements in two regions.
Two-pronged approach:
1. Choose a color space that captures interesting things.
a. Different color spaces have different invariants, and different
responses to changes in color.
2. Choose a similarity metric for that space that captures
everything we’re interested: color, texture, size, and shape.
72. Similarity
0 RGB (red, green, blue) is a good baseline, but changes
in illumination (shadows, light intensity) affect all three
channels.
73. Similarity
0 HSV (hue, saturation, value) encodes color information in the
hue channel, which is invariant to changes in lighting.
Additionally, saturation is insensitive to shadows, and value is
insensitive to brightness changes.
74. Similarity
0 Lab uses a lightness channel and two color channels (a and
b). It’s calibrated to be perceptually uniform. Like HSV, it’s also
somewhat invariant to changes in brightness and shadow.
78. Generative Adversarial Networks
0 According to Yann LeCun, these networks could be the next big development.
0 The idea is to simultaneously train two neural nets.
0 The first one, called the Discriminator D(Y) takes an input (e.g. an image) and
outputs a scalar that indicates whether the image Y looks “natural” or not.
0 The second network is called the generator, denoted G(Z), where Z is generally
a vector randomly sampled in a simple distribution (e.g. Gaussian). The role of
the generator is to produce images so as to train the D(Y) function to take the
right shape (low values for real images, higher values for everything else).
79. Generative Adversarial Networks : Importance
0 The discriminator now is aware of the “internal
representation of the data” because it has been trained to
understand the differences between real images from the
dataset and artificially created ones.
0 It can be used as a feature extractor for CNN.
0 You can just create really cool artificial images that look
pretty natural to me.
80. Generating Image Descriptions
0 What happens when you combine CNNs with RNNs. you do get one
really amazing application.
0 Combination of CNNs and bidirectional RNNs to generate natural
language descriptions of different image regions
81. Spatial Transformer Network
0 The basic idea is that this module transforms the input image in a way so
that the subsequent layers have an easier time making a classification.
0 The module consists of:
0 A localization network which takes in the input volume and outputs
parameters of the spatial transformation that should be applied.
0 The creation of a sampling grid that is the result of warping the regular grid
with the affine transformation (theta) created in the localization network.
0 A sampler whose purpose is to perform a warping of the input feature map.
82. Spatial Transformer Network
0 This Network implements the simple idea of making affine
transformations to the input image in order to help models
become more invariant to translation, scale, and rotation.
83. Conclusion
Significant advances in deep reinforcement and
unsupervised learning
Bigger and more complex architectures based on
various interchangeable modules/techniques
Deeper models that can learn from much fewer
training cases
Harder problems such as video understanding
and natural language processing will be
successfully tackled by deep learning algorithms
84. Conclusion
Machines that learn to represent the world from
experience
Deep learning is no magic ! Just statistics in a black
box , but exceptional effective at learning patterns.
We haven’t figured out creativity and human-
empathy.
Transitioning from research to consumer products
.will make the tools you use every day work better,
faster and smarter
AlexNet,8 layers(ILSVRC 2012),VGG,19 layers(ILSVRC 2014),GoogleNet,22 layers(ILSVRC 2014)
ImageNet Large Scale Visual Recognition (ILSVRC )
https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html
He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
A Beginner's Guide To Understanding Convolutional Neural Networks – Adit Deshpande – CS Undergrad at UCLA ('19)
A Beginner's Guide To Understanding Convolutional Neural Networks – Adit Deshpande – CS Undergrad at UCLA ('19)
A Beginner's Guide To Understanding Convolutional Neural Networks – Adit Deshpande – CS Undergrad at UCLA ('19)
Paw=hand
Convolutional neural networks. Sounds like a weird combination of biology and math with a little CS sprinkled in, but these networks have been some of the most influential innovations in the field of computer vision.
if we computed the network's error for every possible value of a single weight, it would generate the curve you see above. We would then pick the value of the single weight that has the lowest error (the lowest part of the curve). I say single weight because it's a two-dimensional plot. Thus, the x dimension is the value of the weight and the y dimension is the neural network's error when the weight is at that position.
https://iamtrask.github.io/2015/07/27/python-network-part2/
Problem 3: When Slopes are Too Small -Solution 3: Increase the Alpha
Convolutional neural networks(Convnet,CNN)
ConvNet has shown outstanding performance in recognition tasks (image, speech,object)
ConvNet contains hierarchical abstraction process called pooling.
Recurrent neural networks(RNN)
RNN is a generative model: It can generate new data.
Long Short term memory (LSTM) networks[Hochreiter and Schmidhuber, 1997]
RNN+LSTM makes use of long-term memory → Good for time-series data.
Deep/Restricted Boltzmann machines (RBM)
It helps to avoid local minima problem(It regularizes the training data).
But it is not necessary when we have large amount of data.(Drop-out is enough for regularization)
A Beginner's Guide To Understanding Convolutional Neural Networks – Adit Deshpande – CS Undergrad at UCLA ('19)
Every layer of a CNN takes a 3D volume of numbers and outputs a 3D volume of numbers.
However, let’s talk about what this convolution is actually doing from a high level. Each of these filters can be thought of as feature identifiers. When I say features, I’m talking about things like straight edges, simple colors, and curves. Think about the simplest characteristics that all images have in common with each other. Let’s say our first filter is 7 x 7 x 3 and is going to be a curve detector. (In this section, let’s ignore the fact that the filter is 3 units deep and only consider the top depth slice of the filter and the image, for simplicity.)As a curve detector, the filter will have a pixel structure in which there will be higher numerical values along the area that is a shape of a curve (Remember, these filters that we’re talking about as just numbers!).
examples of actual visualizations of the filters of the first conv layer of a trained network. Nonetheless, the main argument remains the same. The filters on the first layer convolve around the input image and “activate” (or compute high values) when the specific feature it is looking for is in the input volume.
https://www.mathworks.com/help/nnet/ug/layers-of-a-convolutional-neural-network.html#mw_ad6f0a9d-9cc7-4e57-9102-0204f1f13e99
https://prateekvjoshi.com/2016/04/05/what-is-local-response-normalization-in-convolutional-neural-networks/
“lateral inhibition”
This refers to the capacity of an excited neuron to subdue its neighbors. We basically want a significant peak so that we have a form of local maxima. This tends to create a contrast in that area, hence increasing the sensory perception. Increasing the sensory perception is a good thing! We want to have the same thing in our CNNs.
Big Data: large data sets are available
– Imagenet: 14M+ labeled images http://www.image-net.org/
– YouTube-8M: 7M+ labeled videos https://research.google.com/youtube8m/
– AWS public data sets: https://aws.amazon.com/public-datasets/
Baidu’s Chinese speech recognition: 4TB of training data, +/- 10 Exaflops
Computations
Neural networks can now be trained on a Raspberry Pi or GPUs because DL model is lightweight
https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html
a] "Deep Residual Learning for Image Recognition", Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun. Tech Report 2015. [b] "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks", Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun. NIPS 2015.
http://image-net.org/challenges/LSVRC/2016/results
[a] W. Ouyang, X. Wang, X. Zeng, S. Qiu, P. Luo, Y. Tian, H. Li, S. Yang, Z. Wang, C. Loy, X. Tang, “DeepID-Net: Deformable Deep Convolutional Neural Networks for Object Detection,” CVPR 2015. [b] Yang, B., Yan, J., Lei, Z., Li, S. Z. "Craft objects from images." CVPR 2016. [c] X. Zeng, W. Ouyang, B. Yang, J. Yan, X. Wang, “Gated Bi-directional CNN for Object Detection,” ECCV 2016. [d] Ouyang, W., Wang, X., Zhang, C., Yang, X. Factors in Finetuning Deep Model for Object Detection with Long-tail Distribution. CVPR 2016. [e] Wanli Ouyang, Hongyang Li, Xingyu Zeng, and Xiaogang Wang, "Learning Deep Representation with Large-scale Attributes", In Proc. ICCV 2015. [f] https://github.com/yjxiong/caffe
http://image-net.org/challenges/LSVRC/2017/results
[1] Wang F, Jiang M, Qian C, et al. Residual Attention Network for Image Classification[J]. arXiv preprint arXiv:1704.06904, 2017. [2] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778. [3] Szegedy C, Ioffe S, Vanhoucke V, et al. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning[C]//AAAI. 2017: 4278-4284. [4] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[J]. arXiv preprint arXiv:1505.04597, 2015. [5] Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[J]. arXiv preprint arXiv:1612.03144, 2016. [6] Shrivastava A, Sukthankar R, Malik J, et al. Beyond skip connections: Top-down modulation for object detection[J]. arXiv preprint arXiv:1612.06851, 2016. [7] Zeng X, Ouyang W, Yan J, et al. Crafting GBD-Net for Object Detection[J]. arXiv preprint arXiv:1610.02579, 2016.
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
[N. Dalal and B. Triggs. “Histograms of oriented gradients for human detection.” In CVPR, 2005.]
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
[B. Alexe, T. Deselaers, and V. Ferrari. “Measuring the objectness of image windows.” IEEE transactions on Pattern Analysis and Machine Intelligence, 2012.]
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
Texture :بنية او نسيج او تركيب
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
[P. F. Felzenszwalb and D. P. Huttenlocher. “Efficient Graph-Based Image Segmentation.” IJCV, 59:167–181, 2004.]
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
[P. F. Felzenszwalb and D. P. Huttenlocher. “Efficient Graph-Based Image Segmentation.” IJCV, 59:167–181, 2004.]
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith
presentation :Selective Search for Object Recognition Uijlings et al. Schuyler Smith