Downloaded 18 times
![THE MACHINE LEARNING LANDSCAPE: CAPSULES
Transforming Auto-encoders
G. E. Hinton, A. Krizhevsky & S. D. Wang Department of Computer Science, University of Toronto
Abstract. The artificial neural networks that are used to recognize shapes typically use one or more layers of learned feature detectors
that produce scalar outputs. By contrast, the computer vision community uses complicated, hand-engineered features, like SIFT [6],
that produce a whole vector of outputs including an explicit representation of the pose of the feature. We show how neural
networks can be used to learn features that output a whole vector of instantiation parameters and we argue that this
is a much more promising way of dealing with variations in position, orientation, scale and lighting than the methods currently
employed in the neural networks community. It is also more promising than the hand- engineered features currently used in computer
vision because it provides an efficient way of adapting the features to the domain.
This paper argues that convolutional neural networks are misguided in what they are trying to achieve. Instead of aiming for
viewpoint invariance in the activities of “neurons” that use a single scalar output to summarize the activities of a
local pool of replicated feature detectors, artificial neural networks should use local “capsules” that perform
some quite complicated internal computations on their inputs and then encapsulate the results of these
computations into a small vector of highly informative outputs.](https://image.slidesharecdn.com/bowlessmartdata32018machinelearningupdatefinal-180313030218/85/Smart-Data-Webinar-Machine-Learning-Update-17-320.jpg)
Uploaded byDATAVERSITY
Smart Data Webinar: Machine Learning Update
The document provides an overview of machine learning technology, detailing the evolution of various AI concepts, including deep learning and neural networks. It discusses the maturity of machine learning approaches, highlighting their applications and the importance of explainability, as well as recognizing key players in the market such as Amazon AWS, IBM Watson, and Google Cloud. Additionally, it emphasizes the shift towards innovative solutions like capsule networks for better feature recognition in AI systems.
More Related Content
![Tiger graph 2021 corporate overview [read only]](https://cdn.slidesharecdn.com/ss_thumbnails/tigergraph2021corporateoverviewread-only-211014093106-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Smart Data Webinar: Machine Learning Update
Smart Data Webinar: Machine Learning Update
- 1. MARCH 8, 2018 MachineLearning Update An Overview of Technology Maturity and Product Vendors Adrian J Bowles, PhD Founder, STORM Insights, Inc. Lead Analyst, AI, Aragon Research info@storminsights.com
- 2. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. FIRST, DEFINE TERMS Artificial Intelligence Machine Learning Deep Learning Data Science
- 3. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. Machine Learning Deep Learning Artificial Intelligence Data Science Each discipline has algorithms and models. FIRST, DEFINE TERMS
- 4. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. #MODERN AI: ARTIFICIAL, AUTOMATED, AUGMENTED, AMPLIFIED…INTELLIGENCE PERCEPTION UNDERSTANDING LEARNING Big Data Classic AI Deep Learning
- 5. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. Systems Controls LearnReason Understand Model Data Mgmt Human Machine Input Output Gestures Emotions Language Narrative Generation Visualization Reports Haptics Sensors (IOT) Systems Controls ML IN THE MODERN AI LANDSCAPE
- 6. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. Human Input Gestures Language Context Learn Reason Understand Model Data Mgmt Detected by Human Senses Derived ImagesSee Hear Touch Smell Taste Sounds Objects Emotions Meaning Concepts Intent Emotions Meaning Concepts Intent Context ML IN THE MODERN AI LANDSCAPE
- 7. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. FUNDAMENTAL DESIGN CHOICE: SYMBOLS VS STATISTICS Symbolic Logic Representations Reasoning Concepts Statistical Models Mechanical Theorem Proving
- 8. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. PROXIMITY/DISTANCE ALGORITHMS Mapped with vectors, proximity algorithm based on purpose. Mapping for autocorrect/complete vs Mapping for meaning Boy Bay Map Mop Man Nay May Mope Buy Hop Hope Boy Bay Map Mop Man Nay May Mope BuyHop HopeSimilar structure -> similar meaning in vision, not always in language. Memory-Based Reasoning
- 9. MACHINE LEARNING FOCUSCONTINUES TO EVOLVE Copyright (c) 2018 by STORM Insights Inc. All Rights Reserved. DATA More Data + Faster HW make Deep Learning Practical Deep Learning Success With Recognition Spurs Investment ALGORITHMS & RULES Caution for Applications Where Transparency is Critical Investment Leads to Investigation Broaden the Scope of Applications New “Explainability” Research Emerges Hybrid Solutions to Augment Intelligence Will Thrive for Critical Applications
- 10. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. RECOGNIZING CONCEPTS - DISCOVERY <> UNDERSTANDING Courtesy of LoopAI Labs.
- 11. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. Supervised Unsupervised Deep General Reinforcement Learning by example, using training data. Strategies based on performance feedback. Discovers patterns based on experience with data. Biologically-inspired ML approach. Leverages simple processing units - analogous to neurosynaptic elements organized in layers that collaborate to solve complex problems. ML MATURING RAPIDLY - ALREADY WELL OVER THE USABILITY THRESHOLD
- 12. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. MACHINE LEARNING - ARTIFICIAL NEURAL NETS Input Output Highly Connected Neural Processors A digital representation of the state of the input domain. Scalars, Vectors, Equations…
- 13. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. DEEP LEARNING Visible Layer Hidden Layer Hidden Layer Output Layer Hidden Layer Input: Observable Variables HIGHABSTRACTIONLOW Output Pixels Depth of the Model Edges Object Shapes/Parts Object Class Brightness/ Contrast Geometry Rules Features to Extract Methods
- 14. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. LIMITATIONS: HOW IMPORTANT IS IT TO BE ABLE TO EXPLAIN REASONING?
- 15. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. LOOKING FOR FEATURES: WHICH ONE IS NOT LIKE THE OTHERS? Edges are easy Objects are easy What are the distinguishing features? Context is King for Discovery
- 16. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. WHAT CAN A DL SYSTEM “LEARN” FROM THIS PICTURE?
- 17. THE MACHINE LEARNINGLANDSCAPE: CAPSULES Transforming Auto-encoders G. E. Hinton, A. Krizhevsky & S. D. Wang Department of Computer Science, University of Toronto Abstract. The artificial neural networks that are used to recognize shapes typically use one or more layers of learned feature detectors that produce scalar outputs. By contrast, the computer vision community uses complicated, hand-engineered features, like SIFT [6], that produce a whole vector of outputs including an explicit representation of the pose of the feature. We show how neural networks can be used to learn features that output a whole vector of instantiation parameters and we argue that this is a much more promising way of dealing with variations in position, orientation, scale and lighting than the methods currently employed in the neural networks community. It is also more promising than the hand- engineered features currently used in computer vision because it provides an efficient way of adapting the features to the domain. This paper argues that convolutional neural networks are misguided in what they are trying to achieve. Instead of aiming for viewpoint invariance in the activities of “neurons” that use a single scalar output to summarize the activities of a local pool of replicated feature detectors, artificial neural networks should use local “capsules” that perform some quite complicated internal computations on their inputs and then encapsulate the results of these computations into a small vector of highly informative outputs.
- 18. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. Maturity/ Refinement Initial Neural Networks Rules CapsulesAd Hoc ML TECHNOLOGIES MATURITY OVERVIEW Utility: Demonstrated reliability & validity ML Technologies/Approaches: Arrow Width Indicates Estimated Future Development/Potential
- 19. THE MACHINE LEARNINGMARKET BIG 4 CLOUD-NATIVE, SCALABLE Copyright (c) 2018 by STORM Insights Inc. All Rights Reserved. Amazon AWS - Model, Vision, Language services… IBM Watson. Watson Machine Learning Google Cloud Machine Learning Engine Managed service for ML models Microsoft Azure Machine Learning Studio Ease of Use Breadth of Services Depth of Services LinkedIn Data Weather Data
- 20. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. THE MACHINE LEARNING MARKET: NOTEWORTHY ML platform supports business users and “citizen data scientists” Private deployment & subscription models (virtual private cloud on AWS, Azure, Google) H2O Compute Engine - Open Source Platform Cognitive Scale: Augmented Intelligence Platform with industry-optimized “CortexAI Systems” (IBM Watson & Microsoft Partners)
- 21. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. Develops custom DL solutions THE MACHINE LEARNING MARKET: NOTEWORTHY Skymind - Skymind Intelligence Layer (SKIL) Leverages Spark to help users “productionize” TensorFlow, Keras, DL4J Skytree - ML platform, MLaaS for data scientists LoopAILabs Loop Q Platform, Natural language-independent reasoning
- 22. Copyright (c) 2018by STORM Insights Inc. All Rights Reserved. THE MACHINE LEARNING MARKET: NOTEWORTHY ALTERNATIVE MODELS Developer of the Hierarchical Temporal Memory model based on the human neocortex. Intel Saffron - Bio-inspired Associative memory model
- 23. adrian@storminsights.com Twitter @ajbowles Skype ajbowles KEEPIN TOUCH Upcoming SmartData Webinar Dates & Topics April 12 Knowledge as a Service: An Introduction to the Emerging Pre-Built Knowledge Market May 10 Case Studies: Transforming Industries with AI (Manufacturing & Retail) June 14 Natural Language Processing: From Chatbots to Artificial Understanding with Affective I/O COMING SOON… AGEOFREASONING.COM BOOK, VIDEOS, PROFESSIONAL SERVICES WWW.AGEOFREASONING.COM
- 24. CAPSULE REFERENCES Copyright (c)2018 by STORM Insights Inc. All Rights Reserved. https://medium.com/ai³-theory-practice-business/understanding-hintons-capsule-networks- part-i-intuition-b4b559d1159b https://openreview.net/pdf?id=HJWLfGWRb https://hackernoon.com/capsule-networks-are-shaking-up-ai-heres-how-to-use-them- c233a0971952