This document discusses using neuroscience to inspire artificial intelligence algorithms and using AI to analyze neuroscience data. Specifically:
1. AI can be used to analyze neuroimaging data to discover biomarkers for mental disorders and predict mental states.
2. Dynamical models inspired by neuroscience, like the van der Pol oscillator model, can capture nonlinear dynamics in brain data and predict future activity.
3. Computational psychiatry aims to use AI to develop objective clinical tests and automated therapeutic agents like conversational agents to improve mental healthcare accessibility.
Artificial Intelligence (AI) is shaping and reshaping every industry under the sun. The Healthcare industry is not any exception.
In this presentation, I have discussed the basics of AI as well as how it is being used in various branches of the healthcare industry. I presented this topic in my departmental seminar in October 2021 and received appreciation as well as positive feedback in this regard.
Artificial intelligence in health care by Islam salama " Saimo#BoOm "Dr-Islam Salama
A Lecture about basics and concepts of Artificial Intelligence in health care & there applications
محاضرة عامة حول الذكاء الإصطناعي وأساسياته في الرعاية الصحية والطبية وتطبيقاته
Artificial Intelligence (AI) is shaping and reshaping every industry under the sun. The Healthcare industry is not any exception.
In this presentation, I have discussed the basics of AI as well as how it is being used in various branches of the healthcare industry. I presented this topic in my departmental seminar in October 2021 and received appreciation as well as positive feedback in this regard.
Artificial intelligence in health care by Islam salama " Saimo#BoOm "Dr-Islam Salama
A Lecture about basics and concepts of Artificial Intelligence in health care & there applications
محاضرة عامة حول الذكاء الإصطناعي وأساسياته في الرعاية الصحية والطبية وتطبيقاته
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
Talk @ ACM SF Bayarea Chapter on Deep Learning for medical imaging space.
The talk covers use cases, special challenges and solutions for Deep Learning for Medical Image Analysis using Tensorflow+Keras. You will learn about:
- Use cases for Deep Learning in Medical Image Analysis
- Different DNN architectures used for Medical Image Analysis
- Special purpose compute / accelerators for Deep Learning (in the Cloud / On-prem)
- How to parallelize your models for faster training of models and serving for inferenceing.
- Optimization techniques to get the best performance from your cluster (like Kubernetes/ Apache Mesos / Spark)
- How to build an efficient Data Pipeline for Medical Image Analysis using Deep Learning
- Resources to jump start your journey - like public data sets, common models used in Medical Image Analysis
Performance analysis of automated brain tumor detection from MR imaging and CT scan using basic image processing techniques based on various hard and soft computing has been performed in our work. Moreover, we applied six traditional classifiers to detect brain tumor in the images. Then we applied CNN for brain tumor detection to include deep learning method in our work. We compared the result of the traditional one having the best accuracy (SVM) with the result of CNN. Furthermore, our work presents a generic method of tumor detection and extraction of its various features.
The presentation briefly answers the questions:
1. What is Machine Learning?
2. Ideas behind Neural Networks?
3. What is Deep Learning? How different is it from NN?
4. Practical examples of applications.

For more information:
https://www.quora.com/How-does-deep-learning-work-and-how-is-it-different-from-normal-neural-networks-and-or-SVM
http://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma
https://www.youtube.com/watch?v=n1ViNeWhC24 - presentation by Ng
http://techtalks.tv/talks/deep-learning/58122/ - deep learning tutorial and slides - http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf
Deep learning for NLP - http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial
papers: http://www.cs.toronto.edu/~hinton/science.pdf
http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_ErhanCBV10.pdf
http://arxiv.org/pdf/1206.5538v3.pdf
http://arxiv.org/pdf/1404.7828v4.pdf
More recommendations - https://www.quora.com/What-are-the-best-resources-to-learn-about-deep-learning
Artificial intelligence in practice- part-1GMR Group
Summary is made in 5 parts-
This is Part -1
Cyber-solutions to real-world business problems Artificial Intelligence in Practice is a fascinating look into how companies use AI and machine learning to solve problems. Presenting 50 case studies of actual situations, this book demonstrates practical applications to issues faced by businesses around the globe.
• The rapidly evolving field of artificial intelligence has expanded beyond research labs and computer science departments and made its way into the mainstream business environment.
• Artificial intelligence and machine learning are cited as the most important modern business trends to drive success.
• It is used in areas ranging from banking and finance to social media and marketing.
• This technology continues to provide innovative solutions to businesses of all sizes, sectors and industries.
• This engaging and topical book explores a wide range of cases illustrating how businesses use AI to boost performance, drive efficiency, analyse market preferences and many others.
• This detailed examination provides an overview of each company, describes the specific problem and explains how AI facilitates resolution.
• Each case study provides a comprehensive overview, including some technical details as well as key learning summaries:
o Understand how specific business problems are addressed by innovative machine learning methods Explore how current artificial intelligence applications improve performance and increase efficiency in various situations
o Expand your knowledge of recent AI advancements in technology
o Gain insight on the future of AI and its increasing role in business and industry
o Artificial Intelligence in Practice: How 50 Successful Companies Used Artificial Intelligence to Solve Problems is an insightful and informative exploration of the trans-formative power of technology in 21st century commerce
Computational Neuroscience - The Brain - Computer Science InterfaceChristopher Currin
Understanding intelligence is one of the most challenging scientific problems faced by humanity.
This talk will provide an introduction to the multi-disciplinary field of Computational Neuroscience: the questions it seeks to answer and some of the (mathematical & computational) techniques used to investigate how we fundamentally think.
Currently doing his Computational Neuroscience PhD at the University of Cape Town, Chris has a wonderfully weird background in machine learning, neuroscience, and psychology. He is fascinated by how we think and learn, sometimes to a fault, and how this works in both biological and artificial intelligence.
It’s long ago, approx. 30 years, since AI was not only a topic for Science-Fiction writers, but also a major research field surrounded with huge hopes and investments. But the over-inflated expectations ended in a subsequent crash and followed by a period of absent funding and interest – the so-called AI winter. However, the last 3 years changed everything – again. Deep learning, a machine learning technique inspired by the human brain, successfully crushed one benchmark after another and tech companies, like Google, Facebook and Microsoft, started to invest billions in AI research. “The pace of progress in artificial general intelligence is incredible fast” (Elon Musk – CEO Tesla & SpaceX) leading to an AI that “would be either the best or the worst thing ever to happen to humanity” (Stephen Hawking – Physicist).
What sparked this new Hype? How is Deep Learning different from previous approaches? Are the advancing AI technologies really a threat for humanity? Let’s look behind the curtain and unravel the reality. This talk will explore why Sundar Pichai (CEO Google) recently announced that “machine learning is a core transformative way by which Google is rethinking everything they are doing” and explain why "Deep Learning is probably one of the most exciting things that is happening in the computer industry” (Jen-Hsun Huang – CEO NVIDIA).
Either a new AI “winter is coming” (Ned Stark – House Stark) or this new wave of innovation might turn out as the “last invention humans ever need to make” (Nick Bostrom – AI Philosoph). Or maybe it’s just another great technology helping humans to achieve more.
Talk @ ACM SF Bayarea Chapter on Deep Learning for medical imaging space.
The talk covers use cases, special challenges and solutions for Deep Learning for Medical Image Analysis using Tensorflow+Keras. You will learn about:
- Use cases for Deep Learning in Medical Image Analysis
- Different DNN architectures used for Medical Image Analysis
- Special purpose compute / accelerators for Deep Learning (in the Cloud / On-prem)
- How to parallelize your models for faster training of models and serving for inferenceing.
- Optimization techniques to get the best performance from your cluster (like Kubernetes/ Apache Mesos / Spark)
- How to build an efficient Data Pipeline for Medical Image Analysis using Deep Learning
- Resources to jump start your journey - like public data sets, common models used in Medical Image Analysis
Performance analysis of automated brain tumor detection from MR imaging and CT scan using basic image processing techniques based on various hard and soft computing has been performed in our work. Moreover, we applied six traditional classifiers to detect brain tumor in the images. Then we applied CNN for brain tumor detection to include deep learning method in our work. We compared the result of the traditional one having the best accuracy (SVM) with the result of CNN. Furthermore, our work presents a generic method of tumor detection and extraction of its various features.
The presentation briefly answers the questions:
1. What is Machine Learning?
2. Ideas behind Neural Networks?
3. What is Deep Learning? How different is it from NN?
4. Practical examples of applications.

For more information:
https://www.quora.com/How-does-deep-learning-work-and-how-is-it-different-from-normal-neural-networks-and-or-SVM
http://stats.stackexchange.com/questions/114385/what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma
https://www.youtube.com/watch?v=n1ViNeWhC24 - presentation by Ng
http://techtalks.tv/talks/deep-learning/58122/ - deep learning tutorial and slides - http://www.cs.nyu.edu/~yann/talks/lecun-ranzato-icml2013.pdf
Deep learning for NLP - http://www.socher.org/index.php/DeepLearningTutorial/DeepLearningTutorial
papers: http://www.cs.toronto.edu/~hinton/science.pdf
http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS2010_ErhanCBV10.pdf
http://arxiv.org/pdf/1206.5538v3.pdf
http://arxiv.org/pdf/1404.7828v4.pdf
More recommendations - https://www.quora.com/What-are-the-best-resources-to-learn-about-deep-learning
Artificial intelligence in practice- part-1GMR Group
Summary is made in 5 parts-
This is Part -1
Cyber-solutions to real-world business problems Artificial Intelligence in Practice is a fascinating look into how companies use AI and machine learning to solve problems. Presenting 50 case studies of actual situations, this book demonstrates practical applications to issues faced by businesses around the globe.
• The rapidly evolving field of artificial intelligence has expanded beyond research labs and computer science departments and made its way into the mainstream business environment.
• Artificial intelligence and machine learning are cited as the most important modern business trends to drive success.
• It is used in areas ranging from banking and finance to social media and marketing.
• This technology continues to provide innovative solutions to businesses of all sizes, sectors and industries.
• This engaging and topical book explores a wide range of cases illustrating how businesses use AI to boost performance, drive efficiency, analyse market preferences and many others.
• This detailed examination provides an overview of each company, describes the specific problem and explains how AI facilitates resolution.
• Each case study provides a comprehensive overview, including some technical details as well as key learning summaries:
o Understand how specific business problems are addressed by innovative machine learning methods Explore how current artificial intelligence applications improve performance and increase efficiency in various situations
o Expand your knowledge of recent AI advancements in technology
o Gain insight on the future of AI and its increasing role in business and industry
o Artificial Intelligence in Practice: How 50 Successful Companies Used Artificial Intelligence to Solve Problems is an insightful and informative exploration of the trans-formative power of technology in 21st century commerce
Computational Neuroscience - The Brain - Computer Science InterfaceChristopher Currin
Understanding intelligence is one of the most challenging scientific problems faced by humanity.
This talk will provide an introduction to the multi-disciplinary field of Computational Neuroscience: the questions it seeks to answer and some of the (mathematical & computational) techniques used to investigate how we fundamentally think.
Currently doing his Computational Neuroscience PhD at the University of Cape Town, Chris has a wonderfully weird background in machine learning, neuroscience, and psychology. He is fascinated by how we think and learn, sometimes to a fault, and how this works in both biological and artificial intelligence.
How can Big Data help upgrade brain care?SharpBrains
Current standards of brain and mental care often rely on trials of insufficient scale, which not only limits our ability to diagnose, prevent, treat and personalize care but often leads to incorrect conclusions and undesirable results. What tools and data are becoming available via large-scale web-based and mobile applications, and how can researchers, innovators and practitioners connect with these initiatives?
- Chair: Alvaro Fernandez, CEO of SharpBrains, YGL Class of 2012
- Daniel Sternberg, Data Scientist at Lumosity
- Joan Severson, President of Digital Artefacts
- Robert Bilder, Chief of Medical Psychology-Neuropsychology at UCLA Semel Institute for Neuroscience
NYAI #23: Using Cognitive Neuroscience to Create AI (w/ Dr. Peter Olausson)Maryam Farooq
Dr. Peter Olausson started his career as a cognitive neuroscientist and spent over a decade at Yale University researching how our memories, motivation and cognitive control together affect decision-making. Before starting COGNITUUM, Peter was focusing on new breakthroughs in the information solutions that shape the human experience, including cognitive computing, data analytics, neuromanagement, and knowledge networks. Peter received his PhD in neuropharmacology at the University of Gothenburg in Sweden and his postdoctoral training at Yale University.
COGNITUUM has developed a general intelligence framework that provides a viable pathway towards human-level machine intelligence. The platform features continuous and real-time learning from any data source.
Role of artificial intellegence (a.i) in radiology department nitish virmaniNitish Virmani
Machine Learning and Deep Learning is the key to Artificial Intelligence. Future of Radiology with Artificial Intelligence and advancements of Radiology Equipments
How to address privacy, ethical and regulatory issues: Examples in cognitive ...SharpBrains
How to address privacy, ethical and regulatory issues: Examples in cognitive enhancement, depression and ADHD
Dr. Karen Rommelfanger, Director of the Neuroethics Program at Emory University
Dr. Anna Wexler, Assistant Professor at the Perelman School of Medicine at UPenn
Jacqueline Studer, Senior VP and General Counsel of Akili Interactive Labs
Chaired by: Keith Epstein, Healthcare Practice Leader at Blue Heron
Slidedeck supporting presentation and discussion during the 2019 SharpBrains Virtual Summit: The Future of Brain Health (March 7-9th). Learn more at:
https://sharpbrains.com/summit-2019/
Frankie Rybicki slide set for Deep Learning in Radiology / MedicineFrank Rybicki
These are my #AI slides for medical deep learning using #radiology and medical imaging examples. Please use them & modify to teach your own group about medical AI.
June 2018 version
How deep learning reshapes medicine
- Brief deep learning
- Recent applications
- Specific researches
- Perspectives and future directions
Dr. Pablo Paredes. Faculty at the Radiology, Psychiatry and Population Health Departments, Stanford University.
Pablo Paredes earned his PhD in Computer Science from the University of California, Berkeley in 2015. He is faculty in Radiology and Psychiatry at Stanford University. Prior to joining the School of Medicine, he was a Postdoctoral Researcher in Computer Science at Stanford University for two years. During his PhD career, he held internships on behavior change and affective computing at Microsoft Research and Google. Before 2010 he was a senior strategic manager with Intel in Sao Paulo, Brazil, a lead product manager with Telefonica in Quito, Ecuador and an Entrepreneur in his natal Cuenca, Ecuador. In these roles, he has had the opportunity to closely evaluate designers, engineers, business people and researchers in telecommunications, product development, and ubiquitous computing (Ubicomp). He has advised several PhD, masters and undergrad students.
Link to video: https://youtu.be/iNxf6vzjSGk
https://cars.stanford.edu/people/pablo-paredes
How can we make a Radiologist more efficient?
Increased Imaging for Chronic Diseases and Emergencies raise the demand for radiologists globally & AI could definitely assist them in increasing their efficiency & meet the requirements.
Life is hard, but it offers progress and satisfaction. It's normal to feel many emotions, and you can cope and feel better. You're supported.
Our mental health app offers 24/7 professional assistance!
We can create a positive mental health graphic by analysing these patterns with machine learning techniques.
These insights can help mental health professionals diagnose, plan, and track treatment.
Our app helps you manage your mental health with guided meditations, mood tracking, and personalised self-care routines.
Our mood tracking function helps you recognise patterns and make positive changes by keeping track of your emotions.
Our guided meditation function allows you to find the right audio and video meditations for you.
Our personalised self-care plans provide information and advice on stress management, sleep quality, and healthy living.
If you need help, our app lets you contact with a licenced therapist.
Our convenient scheduling system lets you pick a time for your session.
Our mental health app helps you to improve your mental health daily.
Download our app today to improve your mental health!
A major challenge facing healthcare organizations (hospitals, medical centers) is
the provision of quality services at affordable costs. Quality service implies diagnosing
patients correctly and administering treatments that are effective. Poor clinical decisions
can lead to disastrous consequences which are therefore unacceptable. Hospitals must
also minimize the cost of clinical tests. They can achieve these results by employing
appropriate computer-based information and/or decision support systems.
Most hospitals today employ some sort of hospital information systems to manage
their healthcare or patient data.
These systems are designed to support patient billing, inventory management and generation of simple statistics. Some hospitals use decision support systems, but they are largely limited. Clinical decisions are often made based on doctors’ intuition and experience rather than on the knowledge rich data hidden in the database.
This practice leads to unwanted biases, errors and excessive medical costs which affects the quality of service provided to patients.
Similar to AI for Neuroscience and Neuroscience for AI (20)
Jamila Smith-Loud - Understanding Human Impact: Social and Equity Assessments...MLconf
Understanding Human Impact: Social and Equity Assessments for AI Technologies
Social and Equity Impact Assessments have broad applications but can be a useful tool to explore and mitigate for Machine Learning fairness issues and can be applied to product specific questions as a way to generate insights and learnings about users, as well as impacts on society broadly as a result of the deployment of new and emerging technologies.
In this presentation, my goal is to advocate for and highlight the need to consult community and external stakeholder engagement to develop a new knowledge base and understanding of the human and social consequences of algorithmic decision making and to introduce principles, methods and process for these types of impact assessments.
Ted Willke - The Brain’s Guide to Dealing with Context in Language UnderstandingMLconf
The Brain’s Guide to Dealing with Context in Language Understanding
Like the visual cortex, the regions of the brain involved in understanding language represent information hierarchically. But whereas the visual cortex organizes things into a spatial hierarchy, the language regions encode information into a hierarchy of timescale. This organization is key to our uniquely human ability to integrate semantic information across narratives. More and more, deep learning-based approaches to natural language understanding embrace models that incorporate contextual information at varying timescales. This has not only led to state-of-the art performance on many difficult natural language tasks, but also to breakthroughs in our understanding of brain activity.
In this talk, we will discuss the important connection between language understanding and context at different timescales. We will explore how different deep learning architectures capture timescales in language and how closely their encodings mimic the brain. Along the way, we will uncover some surprising discoveries about what depth does and doesn’t buy you in deep recurrent neural networks. And we’ll describe a new, more flexible way to think about these architectures and ease design space exploration. Finally, we’ll discuss some of the exciting applications made possible by these breakthroughs.
Justin Armstrong - Applying Computer Vision to Reduce Contamination in the Re...MLconf
Applying Computer Vision to Reduce Contamination in the Recycling Stream
With China’s recent refusal of most foreign recyclables, North American waste haulers are scrambling to figure out how to make on-shore recycling cost-effective in order to continue providing recycling services. Recyclables that were once being shipped to China for manual sorting are now primarily being redirected to landfills or incinerators. Without a solution, a nearly $5 billion annual recycling market could come to a halt.
Purity in the recycling stream is key to this effort as contaminants in the stream can increase the cost of operations, damage equipment and reduce the ability to create pure commodities suitable for creating recycled goods. This market disruption as a result of China’s new regulations, however, provides us the chance to re-examine and improve our current disposal & collection habits with modern monitoring & artificial intelligence technology.
Using images from our in-dumpster cameras, Compology has developed an ML-based process that helps identify, measure and alert for contaminants in recycling containers before they are picked-up, helping keep the recycling stream clean.
Our convolutional neural network flags potential instances of contamination inside a dumpster, enabling garbage haulers to know which containers have the wrong type of material inside. This allows them to provide targeted, timely education, and when appropriate, assess fines, to improve recycling compliance at the businesses and residences they serve, helping keep recycling services financially viable.
In this presentation, we will walk through our ML-based contamination measurement and scoring process by showing how Waste Management, a national waste hauler, has experienced 57% contamination reduction in nearly 2,000 containers over six months, This progress shows significant strides towards financially viable recycling services.
Igor Markov - Quantum Computing: a Treasure Hunt, not a Gold RushMLconf
Quantum Computing: a Treasure Hunt, not a Gold Rush
Quantum computers promise a significant step up in computational power over conventional computers, but also suffer a number of counterintuitive limitations --- both in their computational model and in leading lab implementations. In this talk, we review how quantum computers compete with conventional computers and how conventional computers try to hold their ground. Then we outline what stands in the way of successful quantum ML applications.
Josh Wills - Data Labeling as Religious ExperienceMLconf
Data Labeling as Religious Experience
One of the most common places to deploy a production machine learning systems is as a replacement for a legacy rules-based system that is having a hard time keeping up with new edge cases and requirements. I'll be walking through the process and tooling we used to help us design, train, and deploy a model to replace a set of static rules we had for handling invite spam at Slack, talk about what we learned, and discuss some problems to solve in order to make these migrations easier for everyone.
Vinay Prabhu - Project GaitNet: Ushering in the ImageNet moment for human Gai...MLconf
Project GaitNet: Ushering in the ImageNet moment for human Gait kinematics
The emergence of the upright human bipedal gait can be traced back 4 to 2.8 million years ago, to the now extinct hominin Australopithecus afarensis. Fine grained analysis of gait using the modern MEMS sensors found on all smartphones not just reveals a lot about the person’s orthopedic and neuromuscular health status, but also has enough idiosyncratic clues that it can be harnessed as a passive biometric. While there were many siloed attempts made by the machine learning community to model Bipedal Gait sensor data, these were done with small datasets oft collected in restricted academic environs. In this talk, we will introduce the ImageNet moment for human gait analysis by presenting 'Project GaitNet', the largest ever planet-sized motion sensor based human bipedal gait dataset ever curated. We’ll also present the associated state-of-the-art results in classifying humans harnessing novel deep neural architectures and the related success stories we have enjoyed in transfer-learning into disparate domains of human kinematics analysis.
Jekaterina Novikova - Machine Learning Methods in Detecting Alzheimer’s Disea...MLconf
Machine Learning Methods in Detecting Alzheimer’s Disease from Speech and Language
Alzheimer's disease affects millions of people worldwide, and it is important to predict the disease as early and as accurate as possible. In this talk, I will discuss development of novel ML models that help classifying healthy people from those who develop Alzheimer's, using short samples of human speech. As an input to the model, features of different modalities are extracted from speech audio samples and transcriptions: (1) syntactic measures, such as e.g. production rules extracted from syntactic parse trees, (2) lexical measures, such as e.g. features of lexical richness and complexity and lexical norms, and (3) acoustic measures, such as e.g. standard Mel-frequency cepstral coefficients. I will present the ML model that detects cognitive impairment by reaching agreement among modalities. The resulting model is able to achieve state of the art performance in both supervised and semi-supervised manner, using manual transcripts of human speech. Additionally, I will discuss potential limitations of any fully-automated speech-based Alzheimer's disease detection model, focusing mostly on the analysis of the impact of a not-so-accurate automatic speech recognition (ASR) on the classification performance. To illustrate this, I will present the experiments with controlled amounts of artificially generated ASR errors and explain how the deletion errors affect Alzheimer's detection performance the most, due to their impact on the features of syntactic and lexical complexity.
Meghana Ravikumar - Optimized Image Classification on the CheapMLconf
Optimized Image Classification on the Cheap
In this talk, we anchor on building an image classifier trained on the Stanford Cars dataset to evaluate two approaches to transfer learning -fine tuning and feature extraction- and the impact of hyperparameter optimization on these techniques. Once we define the most performant transfer learning technique for Stanford Cars, we will double the size of the dataset through image augmentation to boost the classifier’s performance. We will use Bayesian optimization to learn the hyperparameters associated with image transformations using the downstream image classifier’s performance as the guide. In conjunction with model performance, we will also focus on the features of these augmented images and the downstream implications for our image classifier.
To both maximize model performance on a budget and explore the impact of optimization on these methods, we apply a particularly efficient implementation of Bayesian optimization to each of these architectures in this comparison. Our goal is to draw on a rigorous set of experimental results that can help us answer the question: how can resource-constrained teams make trade-offs between efficiency and effectiveness using pre-trained models?
Noam Finkelstein - The Importance of Modeling Data CollectionMLconf
The Importance of Modeling Data Collection
Data sets used in machine learning are often collected in a systematically biased way - certain data points are more likely to be collected than others. We call this "observation bias". For example, in health care, we are more likely to see lab tests when the patient is feeling unwell than otherwise. Failing to account for observation bias can, of course, result in poor predictions on new data. By contrast, properly accounting for this bias allows us to make better use of the data we do have.
In this presentation, we discuss practical and theoretical approaches to dealing with observation bias. When the nature of the bias is known, there are simple adjustments we can make to nonparametric function estimation techniques, such as Gaussian Process models. We also discuss the scenario where the data collection model is unknown. In this case, there are steps we can take to estimate it from observed data. Finally, we demonstrate that having a small subset of data points that are known to be collected at random - that is, in an unbiased way - can vastly improve our ability to account for observation bias in the rest of the data set.
My hope is that attendees of this presentation will be aware of the perils of observation bias in their own work, and be equipped with tools to address it.
The Uncanny Valley of ML
Every so often, the conundrum of the Uncanny Valley re-emerges as advanced technologies evolve from clearly experimental products to refined accepted technologies. We have seen its effects in robotics, computer graphics, and page load times. The debate of how to handle the new technology detracts from its benefits. When machine learning is added to human decision systems a similar effect can be measured in increased response time and decreased accuracy. These systems include radiology, judicial assignments, bus schedules, housing prices, power grids and a growing variety of applications. Unfortunately, the Uncanny Valley of ML can be hard to detect in these systems and can lead to degraded system performance when ML is introduced, at great expense. Here, we'll introduce key design principles for introducing ML into human decision systems to navigate around the Uncanny Valley and avoid its pitfalls.
Sneha Rajana - Deep Learning Architectures for Semantic Relation Detection TasksMLconf
Deep Learning Architectures for Semantic Relation Detection Tasks
Recognizing and distinguishing specific semantic relations from other types of semantic relations is an essential part of language understanding systems. Identifying expressions with similar and contrasting meanings is valuable for NLP systems which go beyond recognizing semantic relatedness and require to identify specific semantic relations. In this talk, I will first present novel techniques for creating labelled datasets required for training deep learning models for classifying semantic relations between phrases. I will further present various neural network architectures that integrate morphological features into integrated path-based and distributional relation detection algorithms and demonstrate that this model outperforms state-of-the-art models in distinguishing semantic relations and is capable of efficiently handling multi-word expressions.
Anoop Deoras - Building an Incrementally Trained, Local Taste Aware, Global D...MLconf
Building an Incrementally Trained, Local Taste Aware, Global Deep Learned Recommender System Model
At Netflix, our main goal is to maximize our members’ enjoyment of the selected show by minimizing the amount of time it takes for them to find it. We try to achieve this goal by personalizing almost all the aspects of our product -- from what shows to recommend, to how to present these shows and construct their home-pages to what images to select per show, among many other things. Everything is recommendations for us and as an applied Machine Learning group, we spend our time building models for personalization that will eventually increase the joy and satisfaction of our members. In this talk we will primarily focus our attention on a) making a global deep learned recommender model that is regional tastes and popularity aware and b) adapting this model to changing taste preferences as well as dynamic catalog availability.
We will first go through some standard recommender system models that use Matrix Factorization and Topic Models and then compare and contrast them with more powerful and higher capacity deep learning based models such as sequence models that use recurrent neural networks. We will show what it entails to build a global model that is aware of regional taste preferences and catalog availability. We will show how models that are built on simple Maximum Likelihood principle fail to do that. We will then describe one solution that we have employed in order to enable the global deep learned models to focus their attention on capturing regional taste preferences and changing catalog.In the latter half of the talk, we will discuss how we do incremental learning of deep learned recommender system models. Why do we need to do that ? Everything changes with time. Users’ tastes change with time. What’s available on Netflix and what’s popular also change over time. Therefore, updating or improving recommendation systems over time is necessary to bring more joy to users. In addition to how we apply incremental learning, we will discuss some of the challenges we face involving large-scale data preparation, infrastructure setup for incremental model training as well as pipeline scheduling. The incremental training enables us to serve fresher models trained on fresher and larger amounts of data. This helps our recommender system to nicely and quickly adapt to catalog and users’ taste changes, and improve overall performance.
Vito Ostuni - The Voice: New Challenges in a Zero UI WorldMLconf
Vito Ostuni - The Voice: New Challenges in a Zero UI World
The adoption of voice-enabled devices has seen an explosive growth in the last few years and music consumption is among the most popular use cases. Music personalization and recommendation plays a major role at Pandora in providing a daily delightful listening experience for millions of users. In turn, providing the same perfectly tailored listening experience through these novel voice interfaces brings new interesting challenges and exciting opportunities. In this talk we will describe how we apply personalization and recommendation techniques in three common voice scenarios which can be defined in terms of request types: known-item, thematic, and broad open-ended. We will describe how we use deep learning slot filling techniques and query classification to interpret the user intent and identify the main concepts in the query.
We will also present the differences and challenges regarding evaluation of voice powered recommendation systems. Since pure voice interfaces do not contain visual UI elements, relevance labels need to be inferred through implicit actions such as play time, query reformulations or other types of session level information. Another difference is that while the typical recommendation task corresponds to recommending a ranked list of items, a voice play request translates into a single item play action. Thus, some considerations about closed feedback loops need to be made. In summary, improving the quality of voice interactions in music services is a relatively new challenge and many exciting opportunities for breakthroughs still remain. There are many new aspects of recommendation system interfaces to address to bring a delightful and effortless experience for voice users. We will share a few open challenges to solve for the future.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
2. NEURO-AI @ IBM
Matt Riemer
Sadhana
Kumaravel
Brian Kingsbury
Ronny Luss
Roger Traub
Yuhai Tu
Djallel
Bouneffouf
Guillermo Cecchi
David Cox
Dmitry Krotov Steve Heisig
Irina Rish
James Kozloski
3. AI Neuro4
Neuroscience 4 AI:
Neuroscience-inspired AI Algorithms
AI for Neuroscience:
Mental State Recognition and Biomarker Discovery
4. [Cecchi et al, NIPS 2009]
[Rish et al, PLOS One, 2013
[Gheiratmand et al, Nature PJ
Schizophrenia 2017]
[Carroll et al, Neuroimage 2009]
[Scheinberg&Rish, ECML 2010]
Schizophrenia classification: 74% to 93% accuracy
symptom severity prediction
[Rish et al, Brain Informatics 2010]
[Rish et al, SPIE Med.Imaging 2012]
[Cecchi et al, PLOS Comp Bio 2012]
Mental states in videogames: sparse regression, 70-95%
Pain perception: sparse regression,
70-80% accuracy, “holographic” patterns
[Honorio et al, AISTATS 2012]
[Rish et al, SPIE Med.Imaging 2016]
Cocaine addiction: sparse Markov net biomarkers;
MPH effects analysis (“stimulant 4 stimulant”)
[Bashivan et al, ICLR 2016]
Cognitive load prediction: 91% w/ recurrent ConvNets
AI 4 NEURO:
NEUROIMAGING DATA ANALYSIS
“Statistical biomarkers”:
+
++
+
- -
- --
Predictive Model
mental
disorder
healthy
[Abrevaya et al, 2018, submitted] Nonlinear dynamical models of CaI and fMRI
5. very high dimensionality:
~100,000 time series (neurons),
even in a collapsed 2D version
(average) of the original 3D data
oscillatory behavior
Larval zebrafish calcium imaging data
(Nature Methods, 2013) M. Ahrens (Janelia Farm/HHMI)
high temporal (0.8Hz) and high
spatial (few voxels/neuron)
resolution (unlike fMRI/EEG)
Can we learn a model capturing the underlying dynamics of this system?
Can this model predict the temporal evolution of brain activity?
Can it be interpretable – i.e., relate to prior neuroscientific knowledge?
MODELING
NONLINEAR BRAIN DYNAMICS
6. Why use van der Pol?
1. Oscillatory behavior is common in
all neural systems; van der Pol is
simplest nonlinear oscillator with
rich enough behavior.
2. Voltage-like x1 (activity), and
recovery-like x2 (excitability),
similarly to neuro literature
[Izhikevich 2007]
DYNAMICAL MODEL:
VAN DER POL OSCILLATOR
i-th observed variable
i-th hidden variable
parameters
W – interaction “network"
alphas – determine the bifurcation diagram of the system
Observed
(activity)
Hidden
(excitability)
How to best estimate oscillator parameters from data?
Not a mainstream problem - neither in machine learning, nor in physics!
7. Parameter fitting:
stochastic search + variable-projection optimization [Aravkin et al, 2016]
VAN DER POL MODEL:
DATA FIT + INTERPRETABILITY
Interpretability: learned W
gives Interaction strength
among brain areas
0.7-0.8 correlation between the
model and the actual data
8. PREDICTING FUTURE BRAIN ACTIVITY
WITH VAN DER POL AND LSTM
• Issue: van der Pol prediction is so-so; also, LSTM suffers from small data problem
• Solution: hybrid approach outperforms both methods (and baseline VAR model) -
data-augmented LSTM with fitted van der Pol simulating more data
CorrelationRMSE(rootmean
squareerror)
Calcium Imaging (zebrafish) Functional MRI (people)
Hybrid outperforms LSTM, van der Pol and VAR on both calcium and fMRI data
9. AI 4 NEURO:
GENERAL LESSONS ON WHAT IS IMPORTANT
▪ Interpretability is as important as predictive accuracy
▪ Domain knowledge must be combined with data-driven models
❖ Feature engineering (still not fully replaced by representation learning)
❖ Domain-specific priors (regularization)
❖ Combining data-driven and analytical models
▪ Deep learning faces specific challenges in neuroimaging:
❖ Small data (e.g., 100 to 1000 samples) – regularization? Data augmentation?
❖ interpretability challenge: e.g. deconvolution
10. “Psychiatric research is in crisis”
- Wiecki, Poland, Frank, 2015
“Imagine going to a doctor because of chest pain that has been bothering you for a
couple of weeks. The doctor would sit down with you, listen carefully to your
description of symptoms, and prescribe medication to lower blood pressure in case
you have a heart condition. After a couple of weeks, your pain has not subsided.
The doctor now prescribes medication against reflux, which finally seems to help.
In this scenario, not a single medical analysis (e.g., electrocardiogram, blood
work, or a gastroscopy) was performed, and medication with potentially severe
side effects was prescribed on a trial-and-error basis.
…This scenario resembles much of contemporary psychiatry
diagnosis and treatment.”
Psychiatry lacks objective clinical tests routinely used in other medical fields!
AI 4 NEURO:
COMPUTATIONAL PSYCHIATRY
11. • As predicted by the World Health Organization, by 2030 the amount
of worldwide disability and life loss attributable to depression may
become greater than for any other condition, including cancer, stroke,
heart disease, accidents, and war
• However, many people do not receive an adequate treatment; one of
the major factors here is limited availability of mental health
professionals, compared to the number of potential patient
• Goal: easily accessible, round-the-clock therapeutic services provided
by a conversational agent (“PsyBot”?)
AI 4 NEURO:
(SEMI-)AUTOMATED THERAPIST?
Dialogue Modeling via Hash Functions (submitted)
13. • patient’s responses can be very long
• therapist’s responses are much shorter,
high-level, ‘label’-like summarizing and
confirming patient’s response
• what constitutes a “good” response?
Difficult to measure output of a therapy
session, it can take many sessions to
see an “improvement” in patient’s
condition
• Some notion of relevance/coherence
(mutual information?) may help describe
“successful” dialogue
construct hash codes of responses
optimize hashing model to maximize
mutual information between patient
and therapist
learn a predictive model to infer
therapist’s response to patient
Patien
t
Therapist
INFO-THEORETIC HASHING MODELS
14. HASHING MODELS OUTPERFORM
DEEP NETS ON SMALL(ER) DATA
<patient> I did lose three pounds, did you notice that?
<therapist> That's great, I did notice that.
---------------------- hashing methods:
<RMM> Yes, you did.
<RkNN> To help you with your diet?
<RLSTM> Okay what's the explanation?
------------------------ deep network methods:
<LSTM> I don't want to be a lot of a lot of a lot of a lot of a lot of
a lot of a lot of a lot of a lot of a lot of a lot of a lot of a lot of a lot
of a lot of a lot of a lot of a lot of a lot of a lot of a lot of a lot of a
lot of a lot of a lot of a lot of a lot of a lot of a lot of a lot of a lot
of the way
<HRED> Yeah. You
<VHRED> Uh-huh. you
15. <patient> Which is stupid because like, ya know, (sniffles) I guess if it's not
what they like, then it's not what they like and they can like do something
else, but like (sniffles) ya know there's some psychiatrists who charge like $300
an hour. If you did that you would really have to work like...if you worked an
hour a day five days a week that would be fifteen hundred dollars a week.
Like...you wouldn't have to work at all...like barely. I mean, though, most
psychiatrists probably don't charge $300 an hour...yeah...it's just kinda
ridiculous that...
<therapist> I'm sorry you're having such trouble getting...getting access to
the care that you need (pause).
---------------------- hashing methods:
<RMM> You got to do.
<RkNN> Oh yeah, you never tried Lithium?
<RLSTM> And you ought not to just shove it away out of your life because you
will need it again.
------------------------ deep network methods:
<LSTM> Uh-huh. you
<HRED> Yeah. you
<VHRED> Uh-huh. you
HASHING MODELS OUTPERFORM
DEEP NETS ON SMALL(ER) DATA
16. FUTURE:
VIRTUAL THERAPIST?
A virtual AI assistant on a smartphone which implements the following
four main steps: (1) data collection; (2) mental state recognition;
(3) taking action to improve the mental state; (4) receiving feedback
from a person to improve future actions
Sensor data:
Text, Audio
Video
EEG signal
Temperature
Heart-rate
AI Algorithms:
classification
of mental states,
detection of
emotional and
cognitive changes
Decision-
Making:
choosing best
feedback or another
action (call a friend?
Tell a joke? Send a
reminder?)
Take an actionObtain feedback from a person
Users
Roles:
24/7 personal coach, assistant, therapist, caretaker, or just a “digital friend”
17. • Successful examples: reinforcement learning, deep learning
• Still, artificial brains are way behind the natural ones:
• brains develop from a single cell via neurogenesis and plasticity,
while artificial NNs (ANNs) are manually constructed
• brains can easily adapt to very different environments and new
tasks over lifetime, ANNs are still highly specialized and inflexible
• Attention, memory, learning mechanism (backprop) in ANNs can
be improved by more biologically plausible implementations
• Brain is a dynamical system changing even without the input, in
resting-state, while machine-learning models are mainly “static”
Neuro 4 AI:
Why Neuroscience-Inspired AI?
18. Our Long-Term Goal
Current Focus:
• lifelong learning systems with continuous model adaptation
• complex dynamical systems modeling: oscillators + LSTM
Next-generation AI based on better understanding of brain functioning
including plasticity, attention, memory, reward processing, motivation, and
beyond, while approaching both brain and AI as non-equilibrium
stochastic dynamical systems rather than just predictors.
19. Adult Neurogenesis
• Adult neurogenesis (AN): generation of new
neurons in adult brains throughout life
(balanced by neuronal death)
• In dentate gyrus of the hippocampus (in humans)
• Increased AN is associated with better adaptation to new
environments. But why is it necessary, besides the usual synaptic
plasticity (i.e. learning weights in neural nets)?
• Can a computational model of AN support this observation?
Can it lead to an adaptive representation learning algorithm?
Dentate gyrus of the hippocampus
20. • Current neuroscience theories suggest that the hippocampus
functions as an autoenconder to create and evoke memories
• A simple autoencoder model: single-hidden-layer sparse linear
autoencoder (classical sparse coding of Olshausen & Field, 1996),
also known as dictionary learning model:
• Solved via l1-regularized optimization:
nsamples
p variables
~~ mbasisvectors
(dictionary)
sparse
representation
input x
output x’
reconstructed x
hidden nodes c
encoded x
link weights
‘dictionary’ D
Baseline: Sparse Autoencoder
X C D
21. Online Dictionary Learning (Mairal et al 2009)
Surrogate
objective:
Surrogate objective in online setting vs batch objective: codes of
past samples are not updated when dictionary is updated on next
samples
Converges to the “true” batch-objective function (Mairal09)
Data
Dictionary
Code
Sparsity
on codes
Optional sparsity
on dictionary
elements
22. Neurogenetic Online DL (NODL):
Neuronal Birth and Death
Compute
Code
Dictionary Update
+ Neuronal Death
via group sparsity
Memory
Update
Reconstruction error too high on
new samples?
Neuronal birth
(new random
elements)
Encode
new samples
yes
no
23. Experiments in Non-Stationary Environments:
Switching Between Different Domains
Images:
from urban
(“Oxford”)
to nature
(flowers,
dogs,cats)
NODL improves reconstruction accuracy of ODL on both old
and new data and learns more compact representations
NODL adapts to change, while not forgetting the past (‘memory’ matrices)
24. Extension to Deep Neural Nets
Alternative to Backpropagation: breaking gradient chains
with auxiliary activation variables:
• Neural Net (deterministic) Bayes Net
• Relaxing nonlinear activations to noisy
(Gaussian) linear activations followed by
nonlinearity (e.g., ReLU)
• Alternating minimization over activations
and weights:
a1
X1 XN
YK
am
Y1
…
…
…
c1 cm
…
W
1
W
L+1
25. AltMin Advantages over Backpropagation
(1) no vanishing gradients
(2) handles nondifferentiable functions
(3) weight updates parallelized across layers
speedup
(4) local updates: more bio-plausible than backprop
(5) co-activation matrices serve as memory in lifelong
learning
26. Online Alternating Minimization w/ co-Activation Memory
• At each layer: co-activation, or covariance matrix A
and between-layer cross-covariance matrix B
27. Results on MNIST
Better than SGD in small(er)-data
regime, comparable otherwise
Outperforms offline AltMin
and similar methods (e.g., Taylor 2016)
Continual Learning with AltMin: work in progress
Online AltMin vs Backprop-SGD vs offline AltMin (Taylor et al )
29. Decision: e.g., image
class,type of medication,
ad on webpage etc.
“External” attention:
input selection [IJCAI 2017]
• Inspiration: visual attention (foveation,
sequence of glimpses)
• Recent neuroscience literature suggests that
attention is a reward-driven mechanism which
in turn drives the future reward
Reward
Reward-Driven Attention: External and Internal
input
x1
x2
1st level
embeddings
f11
f21
fn1
x
N
f12
f22
fn2
f1m
f2m
fnm
. . .
Last level
embeddings
output
r
y1
y2
y
m
“Internal” attention as dynamic execution / network path selection [AAMAS 2018]
30. CONCLUSIONS
▪ In brain imaging and computational psychiatry applications
❖ Datasets are relatively small, thus regularization is crucial
❖ Important: model interpretability and feature engineering, combined with
automated feature extraction (representation learning)
❖ “Cheap” sensors (wearables, speech, text etc.) can be a rich source of
beyond-the-scanner data for mental-state detection, tracking and improvement
▪ Brain-inspired algorithms:
❖ Neurogenesis, attention, memory and many other brain phenomena can serve
as inspiration for better AI algorithms
❖ Challenge: deeper understanding and better modeling of such phenomena