A Psychological background on how we think and store memory to explain the motivation behind the Autoencoders and then comparing the performance, in terms of reconstruction error, of the PCA against the Autoencoders.
Video Games Flow Distraction and LudologyJerry Pickard
This study examined the effects of distraction on flow experience while playing video games. 13 participants played Little Big Planet for 30 minutes with or without distraction. Those with distraction had papers shuffled. Questionnaires before and after measured flow, skill level, narratology and ludology. Results showed distraction did not affect flow, and flow correlated positively with ludology. Gender did not affect narratology or ludology preferences as other studies found. Skill level correlated with ability to experience distraction. Larger samples and varying games were suggested for future research.
This document provides an overview of statistics and the role of statisticians. It begins by introducing the author, Corey Chivers, who is a PhD student studying biological invasions using statistics. It then defines a statistician as someone who turns data into insights, answers questions about the world, and isn't necessarily fun to talk to at parties. The document discusses how statisticians assume the world is boring under the null hypothesis and look for evidence against it. It provides examples of descriptive statistics like variance, standard deviation, and the mean. It also introduces hypothesis testing and the Student's t-test for comparing two groups and determining if any differences could be due to chance.
The document discusses machine learning techniques for classification problems. It introduces classification using raw pixel data before discussing feature extraction to represent images more effectively using features like wheels and seas. Neural networks can also learn features through techniques like autoencoders, where the network is pre-trained before being used for classification. Deep learning networks take advantage of unlabeled data, avoid overfitting through techniques like dropout, and use computational efficiency through GPU processing to effectively perform tasks like classifying images of handwritten digits and galaxies.
The document introduces autoencoders, which are neural networks that compress an input into a lower-dimensional code and then reconstruct the output from that code. It discusses that autoencoders can be trained using an unsupervised pre-training method called restricted Boltzmann machines to minimize the reconstruction error. Autoencoders can be used for dimensionality reduction, document retrieval by compressing documents into codes, and data visualization by compressing high-dimensional data points into 2D for plotting with different categories colored separately.
Техносфера Mail.ru Group, МГУ им. М.В. Ломоносова. Курс "Алгоритмы интеллектуальной обработки больших объемов данных", Лекция №8 "Методы снижения размерности пространства"
Лектор - Владимир Гулин
Проблема проклятия размерности. Отбор и выделение признаков. Методы выделения признаков (feature extraction). Метод главных компонент (PCA). Метод независимых компонент (ICA). Методы основанные на автоэнкодерах. Методы отбора признаков (feature selection). Методы основанные на взаимной корреляции признаков. Метод максимальной релевантность и минимальной избыточности (mRMR). Методы основанные на деревьях решений.
Видео лекции курса https://www.youtube.com/playlist?list=PLrCZzMib1e9pyyrqknouMZbIPf4l3CwUP
Deep Learning - What's the buzz all aboutDebdoot Sheet
Deep learning is a genre of machine learning algorithms that attempt to solve tasks by learning abstraction in data following a stratified description paradigm using non-linear transformation architectures.
When put in simple terms, say you want to make the machine recognize some Mr. X with Mt. E in the background, then this task is a stratified or hierarchical recognition task. At the base of the recognition pyramid would be kernels which can discriminate flats, lines, curves, sharp angles, color; higher up will be kernels which use this information to discriminate body parts, trees, natural scenery, clouds, etc.; higher up will use this knowledge to recognize humans, animals, mountains, etc.; and higher up will learn to recognize Mr. X and Mt. E and finally the apex lexical synthesizer module would say that Mr. X is standing in front of Mt. E. Deep learning is all about how you make machines synthesize this hierarchical logic and also learn these representative kernels all by itself.
Deep learning has been extensively used to efficiently solve these kinds of problems from handwritten character recognition (NYU, U Toronto), speech recognition (Microsoft, Google Voice), lexical ordered speech synthesis (Google Voice, iPhone Siri), object and poster recognition (Cortica), image retrieval (Baidu), content filtering (Youtube, Metacafe, Twitter), product visibility tracking (GazeMetrix), computational medical imaging (IITKgp).
This talk will focus on the buzz around this topic and how firm does the buzz hold on to the claims it boasts of?
This document provides an overview and literature review of unsupervised feature learning techniques. It begins with background on machine learning and the challenges of feature engineering. It then discusses unsupervised feature learning as a framework to learn representations from unlabeled data. The document specifically examines sparse autoencoders, PCA, whitening, and self-taught learning. It provides details on the mathematical concepts and implementations of these algorithms, including applying them to learn features from images. The goal is to use unsupervised learning to extract features that can enhance supervised models without requiring labeled training data.
This document provides an overview of autoencoders and their use in unsupervised learning for deep neural networks. It discusses the history and development of neural networks, including early work in the 1940s-1980s and more recent advances in deep learning. It then explains how autoencoders work by setting the target values equal to the inputs, describes variants like denoising autoencoders, and how stacking autoencoders can create deep architectures for tasks like document retrieval, facial recognition, and signal denoising.
Video Games Flow Distraction and LudologyJerry Pickard
This study examined the effects of distraction on flow experience while playing video games. 13 participants played Little Big Planet for 30 minutes with or without distraction. Those with distraction had papers shuffled. Questionnaires before and after measured flow, skill level, narratology and ludology. Results showed distraction did not affect flow, and flow correlated positively with ludology. Gender did not affect narratology or ludology preferences as other studies found. Skill level correlated with ability to experience distraction. Larger samples and varying games were suggested for future research.
This document provides an overview of statistics and the role of statisticians. It begins by introducing the author, Corey Chivers, who is a PhD student studying biological invasions using statistics. It then defines a statistician as someone who turns data into insights, answers questions about the world, and isn't necessarily fun to talk to at parties. The document discusses how statisticians assume the world is boring under the null hypothesis and look for evidence against it. It provides examples of descriptive statistics like variance, standard deviation, and the mean. It also introduces hypothesis testing and the Student's t-test for comparing two groups and determining if any differences could be due to chance.
The document discusses machine learning techniques for classification problems. It introduces classification using raw pixel data before discussing feature extraction to represent images more effectively using features like wheels and seas. Neural networks can also learn features through techniques like autoencoders, where the network is pre-trained before being used for classification. Deep learning networks take advantage of unlabeled data, avoid overfitting through techniques like dropout, and use computational efficiency through GPU processing to effectively perform tasks like classifying images of handwritten digits and galaxies.
The document introduces autoencoders, which are neural networks that compress an input into a lower-dimensional code and then reconstruct the output from that code. It discusses that autoencoders can be trained using an unsupervised pre-training method called restricted Boltzmann machines to minimize the reconstruction error. Autoencoders can be used for dimensionality reduction, document retrieval by compressing documents into codes, and data visualization by compressing high-dimensional data points into 2D for plotting with different categories colored separately.
Техносфера Mail.ru Group, МГУ им. М.В. Ломоносова. Курс "Алгоритмы интеллектуальной обработки больших объемов данных", Лекция №8 "Методы снижения размерности пространства"
Лектор - Владимир Гулин
Проблема проклятия размерности. Отбор и выделение признаков. Методы выделения признаков (feature extraction). Метод главных компонент (PCA). Метод независимых компонент (ICA). Методы основанные на автоэнкодерах. Методы отбора признаков (feature selection). Методы основанные на взаимной корреляции признаков. Метод максимальной релевантность и минимальной избыточности (mRMR). Методы основанные на деревьях решений.
Видео лекции курса https://www.youtube.com/playlist?list=PLrCZzMib1e9pyyrqknouMZbIPf4l3CwUP
Deep Learning - What's the buzz all aboutDebdoot Sheet
Deep learning is a genre of machine learning algorithms that attempt to solve tasks by learning abstraction in data following a stratified description paradigm using non-linear transformation architectures.
When put in simple terms, say you want to make the machine recognize some Mr. X with Mt. E in the background, then this task is a stratified or hierarchical recognition task. At the base of the recognition pyramid would be kernels which can discriminate flats, lines, curves, sharp angles, color; higher up will be kernels which use this information to discriminate body parts, trees, natural scenery, clouds, etc.; higher up will use this knowledge to recognize humans, animals, mountains, etc.; and higher up will learn to recognize Mr. X and Mt. E and finally the apex lexical synthesizer module would say that Mr. X is standing in front of Mt. E. Deep learning is all about how you make machines synthesize this hierarchical logic and also learn these representative kernels all by itself.
Deep learning has been extensively used to efficiently solve these kinds of problems from handwritten character recognition (NYU, U Toronto), speech recognition (Microsoft, Google Voice), lexical ordered speech synthesis (Google Voice, iPhone Siri), object and poster recognition (Cortica), image retrieval (Baidu), content filtering (Youtube, Metacafe, Twitter), product visibility tracking (GazeMetrix), computational medical imaging (IITKgp).
This talk will focus on the buzz around this topic and how firm does the buzz hold on to the claims it boasts of?
This document provides an overview and literature review of unsupervised feature learning techniques. It begins with background on machine learning and the challenges of feature engineering. It then discusses unsupervised feature learning as a framework to learn representations from unlabeled data. The document specifically examines sparse autoencoders, PCA, whitening, and self-taught learning. It provides details on the mathematical concepts and implementations of these algorithms, including applying them to learn features from images. The goal is to use unsupervised learning to extract features that can enhance supervised models without requiring labeled training data.
This document provides an overview of autoencoders and their use in unsupervised learning for deep neural networks. It discusses the history and development of neural networks, including early work in the 1940s-1980s and more recent advances in deep learning. It then explains how autoencoders work by setting the target values equal to the inputs, describes variants like denoising autoencoders, and how stacking autoencoders can create deep architectures for tasks like document retrieval, facial recognition, and signal denoising.
This document provides an overview of an introductory lecture on artificial intelligence and expert systems. It discusses the Turing Test, definitions of artificial intelligence, a brief history of AI including important figures and milestones, and examples of what current AI systems can and cannot do.
http://www.create-learning.com
Creativity to Innovation program.
People that wish to remain competitive in the today’s environment must develop their capacity to generate creative ideas and then use their talent well to transfer these ideas into innovative practices. This leads to new processes and improved methods for the best use of existing resources, and increases the ability to solve problems and implement solutions that enhance their lives and work. In addition to broadening their personal capacity for creativity and innovation, leaders are better able to implement innovative ideas into their existing practices.
http://www.create-learning.com Creativity to Innovation program at Syracuse University. People that wish to remain competitive in the today’s environment must develop their capacity to generate creative ideas and then use their talent well to transfer these ideas into innovative practices. This leads to new processes and improved methods for the best use of existing resources, and increases the ability to solve problems and implement solutions that enhance their lives and work. In addition to broadening their personal capacity for creativity and innovation, leaders are better able to implement innovative ideas into their existing practices.
An Introduction to Deep Learning (May 2018)Julien SIMON
This document provides an introduction to deep learning, including common network architectures and use cases. It defines artificial intelligence, machine learning, and deep learning. It discusses how neural networks are trained using stochastic gradient descent and backpropagation to minimize loss and optimize weights. Common network types are described, such as convolutional neural networks for image recognition and LSTM networks for sequence prediction. Examples of deep learning applications include machine translation, object detection, segmentation, and generation of images, text, and video. Resources for learning more about deep learning are provided.
Machine Learning can often be a daunting subject to tackle much less utilize in a meaningful manner. In this session, attendees will learn how to take their existing data, shape it, and create models that automatically can make principled business decisions directly in their applications. The discussion will include explanations of the data acquisition and shaping process. Additionally, attendees will learn the basics of machine learning - primarily the supervised learning problem.
Evolution of Deep Learning and new advancementsChitta Ranjan
Earlier known as neural networks, deep learning saw a remarkable resurgence in the past decade. Neural networks did not find enough adopters in the past century due to its limited accuracy in real world applications (due to various reasons) and difficult interpretation. Many of these limitations got resolved in the recent years, and it was re-branded as deep learning. Now deep learning is widely used in industry and has become a popular research topic in academia. Learning about the passage of its evolution and development is intriguing. In this presentation, we will learn about how we resolved the issues in last generation neural networks, how we reached to the recent advanced methods from the earlier works, and different components of deep learning models.
This document provides information about a machine learning course including logistics, content, and expectations. The course consists of 2 lectures and 1 lab session per week over 10 weeks. Assessments include a coursework, exam, and problem sets completed during lab sessions. Students are encouraged to attend all sessions, complete assignments, ask questions, and provide feedback. The course will cover key topics in deep learning including supervised, unsupervised, and reinforcement learning using neural networks applied to domains like computer vision, natural language processing, and more. Landmarks in the field and the 2018 Turing Award winners are also mentioned.
TRIZ is a theory of inventive problem solving developed by Genrich Altshuller based on an analysis of patents. It involves systematically analyzing problems to find underlying contradictions, and provides tools like the contradiction matrix and separation principles to resolve those contradictions and generate innovative solutions. The goal of TRIZ is to make the inventive process more structured and less reliant on trial and error.
From Research Objects to Reproducible Science TalesBertram Ludäscher
University of Southampton. Electronics & Computer Science. Research Seminar (Invited Talk).
TITLE: From Research Objects to Reproducible Science Tales
ABSTRACT. Rumor has it that there is a reproducibility crisis in science. Or maybe there are multiple crises? What do we mean by reproducibility and replicability anyways? In this talk I will first make an attempt at sorting out some of the terminological confusion in this area, focusing on computational aspects. The PRIMAD model is another attempt to describe different aspects of reproducibility studies by focusing on the "delta" between those studies and the original study. In addition to these more theoretical investigations, I will discuss practical efforts to create more reproducible and more transparent computational platforms such as the one developed by the Whole-Tale project: here 'tales' are executable research objects that may combine data, code, runtime environments, and narratives (i.e., the traditional "science story"). I will conclude with some thoughts about the remaining challenges and opportunities to bridge the large conceptual gaps that continue to exist despite the recognition of problems of reproducibility and transparency in science.
ABOUT the Speaker. Bertram Ludäscher is a professor at the School of Information Sciences at the University of Illinois, Urbana-Champaign and a faculty affiliate with the National Center for Supercomputing Applications (NCSA) and the Department of Computer Science at Illinois. Until 2014 he was a professor at the Department of Computer Science at the University of California, Davis. His research interests range from practical questions in scientific data and workflow management, to database theory and knowledge representation and reasoning. Prior to his faculty appointments, he was a research scientist at the San Diego Supercomputer Center (SDSC) and an adjunct faculty at the CSE Department at UC San Diego. He received his M.S. (Dipl.-Inform.) in computer science from the University of Karlsruhe (now K.I.T.), and his PhD (Dr. rer. nat.) from the University of Freiburg, in Germany.
This document provides an overview of machine learning concepts including:
1. It defines data science and machine learning, distinguishing machine learning's focus on letting systems learn from data rather than being explicitly programmed.
2. It describes the two main areas of machine learning - supervised learning which uses labeled examples to predict outcomes, and unsupervised learning which finds patterns in unlabeled data.
3. It outlines the typical machine learning process of obtaining data, cleaning and transforming it, applying mathematical models, and using the resulting models to make predictions. Popular models like decision trees, neural networks, and support vector machines are also briefly introduced.
The document provides an introduction to deep learning, including common network architectures and use cases. It defines artificial intelligence, machine learning, and deep learning. Deep learning uses neural networks to teach machines to learn from complex data without explicitly programmed features. Common network types discussed are convolutional neural networks, LSTM networks, and GANs. Applications mentioned include object detection, segmentation, translation, generation, and more. The document also covers concepts like activation functions, training, optimization methods, and resources for learning more.
This document provides an overview of game theory and its applications to neural networks. It begins by discussing deductive and inductive reasoning, and how algorithms like weighted majority and gradient descent can be understood through the lens of game theory. Specifically, it notes that gradient descent achieves low regret when viewed as playing against an adversarial environment. It then discusses how neural networks achieve superhuman performance despite being non-convex problems, which required decades of engineering tweaks. Finally, it suggests game theory can provide insights into modeling populations of neural networks or "experts" that distribute knowledge effectively.
Survey of the current trends, and the future in Natural Language Generation Yu Sheng Su
Small talk on NLG/TG. In recent years, NLG has been widely used in many fields. Today, we'll start with a brief introduction to NLG methods including Pipeline, Autoregressive: Seq2seq, Transformer, GAN, and Non-Autoregressive models. Besides, we will roughly talk about NLG tasks/scenarios and some issues we encounter. Nowadays, Neural incorporates Symbolic is a powerful paradigm in future AI, as Bengio mentioned in AAAI 2020 conference. We will introduce how knowledge guides NLG as well.
The approaches to Artificial Intelligence (AI) in the last century may be labelled as (a) trying to understand and copy (human) nature, (b) being based on heuristic considerations, (c) being formal but from the outset (provably) limited, (d) being (mere) frameworks that leave crucial aspects unspecified. This decade has spawned the first theory of AI, which (e) is principled, formal, complete, and general. This theory, called Universal AI, is about ultimate super-intelligence. It can serve as a gold standard for General AI, and implicitly proposes a formal definition of machine intelligence. After a brief review of the various approaches to (general) AI, I will give an introduction to Universal AI, concentrating on the philosophical, mathematical, and computational aspects behind it. I will also discuss various implications and future challenges.
This document provides an overview of machine learning, including the different types of machine learning problems and algorithms. It discusses supervised learning problems like classification and regression where labels are provided, unsupervised learning problems where the goal is to find structure in unlabeled data like clustering and dimensionality reduction, and reinforcement learning problems where the learning signal comes from rewards. It also covers topics like generalization, learning as compression, nearest neighbor classification, Bayes' rule, Naive Bayes classifiers, and Bayesian networks as graphical models for representing complex relationships between variables.
This document summarizes key concepts related to thinking, language, and intelligence from Chapter 8. It discusses thinking as the manipulation of information through visual imagery and concepts. It also covers heuristics, lateral thinking, framing effects, creativity, theories of language acquisition, the linguistic relativity hypothesis, intelligence testing principles, theories of intelligence from Spearman and Gardner, and the heritability of intelligence. Problem-solving examples like the nine-dot problem and water jar problems are provided to illustrate different thinking strategies.
The document provides an introduction to cognitive psychology. It discusses that cognitive psychology is the study of mental processes, including attention, learning, memory, language, and emotions. It notes that cognitive psychology informs other areas of psychology and has real-world applications in areas like attention while driving, improving learning techniques, and designing understandable text. The document also summarizes common frameworks for explaining cognition, such as the information processing approach, production systems, semantic networks, and connectionism.
The document provides an introduction to cognitive psychology. It discusses that cognitive psychology is the study of mental processes, including attention, learning, memory, language, and emotions. It notes that cognitive psychology informs other areas of psychology and has real-world applications in areas like attention while driving, improving learning techniques, and designing understandable text. The document also summarizes common frameworks for explaining cognition, such as the information processing approach, production systems, semantic networks, and connectionism.
This document discusses language development and structure. It begins by outlining the learning objectives, which are to describe language structure and its flaws, identify stages of language development, and distinguish between Chomsky and Skinner's views of language development. It then defines key parts of language structure, such as phonemes, morphemes, grammar, and discusses Chomsky's views on surface structure and deep structure. The document also outlines flaws in language semantics, syntax, and developmental stages of language learning in children. It concludes by contrasting Chomsky and Skinner's theories of language development.
Diagnosing cancer with Computational IntelligenceSimon van Dyk
An introduction to key Computational Intelligence (CI) concepts, using Hello World as an introductory example, and moving onto Diagnosing Cancer with Neural Networks.
The problem of diagnosing cancer is actually a very simple problem for CI to solve, yet it's impact can be large. It really is just up to what kind of data we have access to, that will determine our creativity in the problems we can solve with CI.
Enjoy
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...Sérgio Sacani
Context. With a mass exceeding several 104 M⊙ and a rich and dense population of massive stars, supermassive young star clusters
represent the most massive star-forming environment that is dominated by the feedback from massive stars and gravitational interactions
among stars.
Aims. In this paper we present the Extended Westerlund 1 and 2 Open Clusters Survey (EWOCS) project, which aims to investigate
the influence of the starburst environment on the formation of stars and planets, and on the evolution of both low and high mass stars.
The primary targets of this project are Westerlund 1 and 2, the closest supermassive star clusters to the Sun.
Methods. The project is based primarily on recent observations conducted with the Chandra and JWST observatories. Specifically,
the Chandra survey of Westerlund 1 consists of 36 new ACIS-I observations, nearly co-pointed, for a total exposure time of 1 Msec.
Additionally, we included 8 archival Chandra/ACIS-S observations. This paper presents the resulting catalog of X-ray sources within
and around Westerlund 1. Sources were detected by combining various existing methods, and photon extraction and source validation
were carried out using the ACIS-Extract software.
Results. The EWOCS X-ray catalog comprises 5963 validated sources out of the 9420 initially provided to ACIS-Extract, reaching a
photon flux threshold of approximately 2 × 10−8 photons cm−2
s
−1
. The X-ray sources exhibit a highly concentrated spatial distribution,
with 1075 sources located within the central 1 arcmin. We have successfully detected X-ray emissions from 126 out of the 166 known
massive stars of the cluster, and we have collected over 71 000 photons from the magnetar CXO J164710.20-455217.
This document provides an overview of an introductory lecture on artificial intelligence and expert systems. It discusses the Turing Test, definitions of artificial intelligence, a brief history of AI including important figures and milestones, and examples of what current AI systems can and cannot do.
http://www.create-learning.com
Creativity to Innovation program.
People that wish to remain competitive in the today’s environment must develop their capacity to generate creative ideas and then use their talent well to transfer these ideas into innovative practices. This leads to new processes and improved methods for the best use of existing resources, and increases the ability to solve problems and implement solutions that enhance their lives and work. In addition to broadening their personal capacity for creativity and innovation, leaders are better able to implement innovative ideas into their existing practices.
http://www.create-learning.com Creativity to Innovation program at Syracuse University. People that wish to remain competitive in the today’s environment must develop their capacity to generate creative ideas and then use their talent well to transfer these ideas into innovative practices. This leads to new processes and improved methods for the best use of existing resources, and increases the ability to solve problems and implement solutions that enhance their lives and work. In addition to broadening their personal capacity for creativity and innovation, leaders are better able to implement innovative ideas into their existing practices.
An Introduction to Deep Learning (May 2018)Julien SIMON
This document provides an introduction to deep learning, including common network architectures and use cases. It defines artificial intelligence, machine learning, and deep learning. It discusses how neural networks are trained using stochastic gradient descent and backpropagation to minimize loss and optimize weights. Common network types are described, such as convolutional neural networks for image recognition and LSTM networks for sequence prediction. Examples of deep learning applications include machine translation, object detection, segmentation, and generation of images, text, and video. Resources for learning more about deep learning are provided.
Machine Learning can often be a daunting subject to tackle much less utilize in a meaningful manner. In this session, attendees will learn how to take their existing data, shape it, and create models that automatically can make principled business decisions directly in their applications. The discussion will include explanations of the data acquisition and shaping process. Additionally, attendees will learn the basics of machine learning - primarily the supervised learning problem.
Evolution of Deep Learning and new advancementsChitta Ranjan
Earlier known as neural networks, deep learning saw a remarkable resurgence in the past decade. Neural networks did not find enough adopters in the past century due to its limited accuracy in real world applications (due to various reasons) and difficult interpretation. Many of these limitations got resolved in the recent years, and it was re-branded as deep learning. Now deep learning is widely used in industry and has become a popular research topic in academia. Learning about the passage of its evolution and development is intriguing. In this presentation, we will learn about how we resolved the issues in last generation neural networks, how we reached to the recent advanced methods from the earlier works, and different components of deep learning models.
This document provides information about a machine learning course including logistics, content, and expectations. The course consists of 2 lectures and 1 lab session per week over 10 weeks. Assessments include a coursework, exam, and problem sets completed during lab sessions. Students are encouraged to attend all sessions, complete assignments, ask questions, and provide feedback. The course will cover key topics in deep learning including supervised, unsupervised, and reinforcement learning using neural networks applied to domains like computer vision, natural language processing, and more. Landmarks in the field and the 2018 Turing Award winners are also mentioned.
TRIZ is a theory of inventive problem solving developed by Genrich Altshuller based on an analysis of patents. It involves systematically analyzing problems to find underlying contradictions, and provides tools like the contradiction matrix and separation principles to resolve those contradictions and generate innovative solutions. The goal of TRIZ is to make the inventive process more structured and less reliant on trial and error.
From Research Objects to Reproducible Science TalesBertram Ludäscher
University of Southampton. Electronics & Computer Science. Research Seminar (Invited Talk).
TITLE: From Research Objects to Reproducible Science Tales
ABSTRACT. Rumor has it that there is a reproducibility crisis in science. Or maybe there are multiple crises? What do we mean by reproducibility and replicability anyways? In this talk I will first make an attempt at sorting out some of the terminological confusion in this area, focusing on computational aspects. The PRIMAD model is another attempt to describe different aspects of reproducibility studies by focusing on the "delta" between those studies and the original study. In addition to these more theoretical investigations, I will discuss practical efforts to create more reproducible and more transparent computational platforms such as the one developed by the Whole-Tale project: here 'tales' are executable research objects that may combine data, code, runtime environments, and narratives (i.e., the traditional "science story"). I will conclude with some thoughts about the remaining challenges and opportunities to bridge the large conceptual gaps that continue to exist despite the recognition of problems of reproducibility and transparency in science.
ABOUT the Speaker. Bertram Ludäscher is a professor at the School of Information Sciences at the University of Illinois, Urbana-Champaign and a faculty affiliate with the National Center for Supercomputing Applications (NCSA) and the Department of Computer Science at Illinois. Until 2014 he was a professor at the Department of Computer Science at the University of California, Davis. His research interests range from practical questions in scientific data and workflow management, to database theory and knowledge representation and reasoning. Prior to his faculty appointments, he was a research scientist at the San Diego Supercomputer Center (SDSC) and an adjunct faculty at the CSE Department at UC San Diego. He received his M.S. (Dipl.-Inform.) in computer science from the University of Karlsruhe (now K.I.T.), and his PhD (Dr. rer. nat.) from the University of Freiburg, in Germany.
This document provides an overview of machine learning concepts including:
1. It defines data science and machine learning, distinguishing machine learning's focus on letting systems learn from data rather than being explicitly programmed.
2. It describes the two main areas of machine learning - supervised learning which uses labeled examples to predict outcomes, and unsupervised learning which finds patterns in unlabeled data.
3. It outlines the typical machine learning process of obtaining data, cleaning and transforming it, applying mathematical models, and using the resulting models to make predictions. Popular models like decision trees, neural networks, and support vector machines are also briefly introduced.
The document provides an introduction to deep learning, including common network architectures and use cases. It defines artificial intelligence, machine learning, and deep learning. Deep learning uses neural networks to teach machines to learn from complex data without explicitly programmed features. Common network types discussed are convolutional neural networks, LSTM networks, and GANs. Applications mentioned include object detection, segmentation, translation, generation, and more. The document also covers concepts like activation functions, training, optimization methods, and resources for learning more.
This document provides an overview of game theory and its applications to neural networks. It begins by discussing deductive and inductive reasoning, and how algorithms like weighted majority and gradient descent can be understood through the lens of game theory. Specifically, it notes that gradient descent achieves low regret when viewed as playing against an adversarial environment. It then discusses how neural networks achieve superhuman performance despite being non-convex problems, which required decades of engineering tweaks. Finally, it suggests game theory can provide insights into modeling populations of neural networks or "experts" that distribute knowledge effectively.
Survey of the current trends, and the future in Natural Language Generation Yu Sheng Su
Small talk on NLG/TG. In recent years, NLG has been widely used in many fields. Today, we'll start with a brief introduction to NLG methods including Pipeline, Autoregressive: Seq2seq, Transformer, GAN, and Non-Autoregressive models. Besides, we will roughly talk about NLG tasks/scenarios and some issues we encounter. Nowadays, Neural incorporates Symbolic is a powerful paradigm in future AI, as Bengio mentioned in AAAI 2020 conference. We will introduce how knowledge guides NLG as well.
The approaches to Artificial Intelligence (AI) in the last century may be labelled as (a) trying to understand and copy (human) nature, (b) being based on heuristic considerations, (c) being formal but from the outset (provably) limited, (d) being (mere) frameworks that leave crucial aspects unspecified. This decade has spawned the first theory of AI, which (e) is principled, formal, complete, and general. This theory, called Universal AI, is about ultimate super-intelligence. It can serve as a gold standard for General AI, and implicitly proposes a formal definition of machine intelligence. After a brief review of the various approaches to (general) AI, I will give an introduction to Universal AI, concentrating on the philosophical, mathematical, and computational aspects behind it. I will also discuss various implications and future challenges.
This document provides an overview of machine learning, including the different types of machine learning problems and algorithms. It discusses supervised learning problems like classification and regression where labels are provided, unsupervised learning problems where the goal is to find structure in unlabeled data like clustering and dimensionality reduction, and reinforcement learning problems where the learning signal comes from rewards. It also covers topics like generalization, learning as compression, nearest neighbor classification, Bayes' rule, Naive Bayes classifiers, and Bayesian networks as graphical models for representing complex relationships between variables.
This document summarizes key concepts related to thinking, language, and intelligence from Chapter 8. It discusses thinking as the manipulation of information through visual imagery and concepts. It also covers heuristics, lateral thinking, framing effects, creativity, theories of language acquisition, the linguistic relativity hypothesis, intelligence testing principles, theories of intelligence from Spearman and Gardner, and the heritability of intelligence. Problem-solving examples like the nine-dot problem and water jar problems are provided to illustrate different thinking strategies.
The document provides an introduction to cognitive psychology. It discusses that cognitive psychology is the study of mental processes, including attention, learning, memory, language, and emotions. It notes that cognitive psychology informs other areas of psychology and has real-world applications in areas like attention while driving, improving learning techniques, and designing understandable text. The document also summarizes common frameworks for explaining cognition, such as the information processing approach, production systems, semantic networks, and connectionism.
The document provides an introduction to cognitive psychology. It discusses that cognitive psychology is the study of mental processes, including attention, learning, memory, language, and emotions. It notes that cognitive psychology informs other areas of psychology and has real-world applications in areas like attention while driving, improving learning techniques, and designing understandable text. The document also summarizes common frameworks for explaining cognition, such as the information processing approach, production systems, semantic networks, and connectionism.
This document discusses language development and structure. It begins by outlining the learning objectives, which are to describe language structure and its flaws, identify stages of language development, and distinguish between Chomsky and Skinner's views of language development. It then defines key parts of language structure, such as phonemes, morphemes, grammar, and discusses Chomsky's views on surface structure and deep structure. The document also outlines flaws in language semantics, syntax, and developmental stages of language learning in children. It concludes by contrasting Chomsky and Skinner's theories of language development.
Diagnosing cancer with Computational IntelligenceSimon van Dyk
An introduction to key Computational Intelligence (CI) concepts, using Hello World as an introductory example, and moving onto Diagnosing Cancer with Neural Networks.
The problem of diagnosing cancer is actually a very simple problem for CI to solve, yet it's impact can be large. It really is just up to what kind of data we have access to, that will determine our creativity in the problems we can solve with CI.
Enjoy
Similar to Auto-Encoders and PCA, a brief psychological background (20)
EWOCS-I: The catalog of X-ray sources in Westerlund 1 from the Extended Weste...Sérgio Sacani
Context. With a mass exceeding several 104 M⊙ and a rich and dense population of massive stars, supermassive young star clusters
represent the most massive star-forming environment that is dominated by the feedback from massive stars and gravitational interactions
among stars.
Aims. In this paper we present the Extended Westerlund 1 and 2 Open Clusters Survey (EWOCS) project, which aims to investigate
the influence of the starburst environment on the formation of stars and planets, and on the evolution of both low and high mass stars.
The primary targets of this project are Westerlund 1 and 2, the closest supermassive star clusters to the Sun.
Methods. The project is based primarily on recent observations conducted with the Chandra and JWST observatories. Specifically,
the Chandra survey of Westerlund 1 consists of 36 new ACIS-I observations, nearly co-pointed, for a total exposure time of 1 Msec.
Additionally, we included 8 archival Chandra/ACIS-S observations. This paper presents the resulting catalog of X-ray sources within
and around Westerlund 1. Sources were detected by combining various existing methods, and photon extraction and source validation
were carried out using the ACIS-Extract software.
Results. The EWOCS X-ray catalog comprises 5963 validated sources out of the 9420 initially provided to ACIS-Extract, reaching a
photon flux threshold of approximately 2 × 10−8 photons cm−2
s
−1
. The X-ray sources exhibit a highly concentrated spatial distribution,
with 1075 sources located within the central 1 arcmin. We have successfully detected X-ray emissions from 126 out of the 166 known
massive stars of the cluster, and we have collected over 71 000 photons from the magnetar CXO J164710.20-455217.
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...Travis Hills MN
Travis Hills of Minnesota developed a method to convert waste into high-value dry fertilizer, significantly enriching soil quality. By providing farmers with a valuable resource derived from waste, Travis Hills helps enhance farm profitability while promoting environmental stewardship. Travis Hills' sustainable practices lead to cost savings and increased revenue for farmers by improving resource efficiency and reducing waste.
What is greenhouse gasses and how many gasses are there to affect the Earth.moosaasad1975
What are greenhouse gasses how they affect the earth and its environment what is the future of the environment and earth how the weather and the climate effects.
Unlocking the mysteries of reproduction: Exploring fecundity and gonadosomati...AbdullaAlAsif1
The pygmy halfbeak Dermogenys colletei, is known for its viviparous nature, this presents an intriguing case of relatively low fecundity, raising questions about potential compensatory reproductive strategies employed by this species. Our study delves into the examination of fecundity and the Gonadosomatic Index (GSI) in the Pygmy Halfbeak, D. colletei (Meisner, 2001), an intriguing viviparous fish indigenous to Sarawak, Borneo. We hypothesize that the Pygmy halfbeak, D. colletei, may exhibit unique reproductive adaptations to offset its low fecundity, thus enhancing its survival and fitness. To address this, we conducted a comprehensive study utilizing 28 mature female specimens of D. colletei, carefully measuring fecundity and GSI to shed light on the reproductive adaptations of this species. Our findings reveal that D. colletei indeed exhibits low fecundity, with a mean of 16.76 ± 2.01, and a mean GSI of 12.83 ± 1.27, providing crucial insights into the reproductive mechanisms at play in this species. These results underscore the existence of unique reproductive strategies in D. colletei, enabling its adaptation and persistence in Borneo's diverse aquatic ecosystems, and call for further ecological research to elucidate these mechanisms. This study lends to a better understanding of viviparous fish in Borneo and contributes to the broader field of aquatic ecology, enhancing our knowledge of species adaptations to unique ecological challenges.
BREEDING METHODS FOR DISEASE RESISTANCE.pptxRASHMI M G
Plant breeding for disease resistance is a strategy to reduce crop losses caused by disease. Plants have an innate immune system that allows them to recognize pathogens and provide resistance. However, breeding for long-lasting resistance often involves combining multiple resistance genes
Phenomics assisted breeding in crop improvementIshaGoswami9
As the population is increasing and will reach about 9 billion upto 2050. Also due to climate change, it is difficult to meet the food requirement of such a large population. Facing the challenges presented by resource shortages, climate
change, and increasing global population, crop yield and quality need to be improved in a sustainable way over the coming decades. Genetic improvement by breeding is the best way to increase crop productivity. With the rapid progression of functional
genomics, an increasing number of crop genomes have been sequenced and dozens of genes influencing key agronomic traits have been identified. However, current genome sequence information has not been adequately exploited for understanding
the complex characteristics of multiple gene, owing to a lack of crop phenotypic data. Efficient, automatic, and accurate technologies and platforms that can capture phenotypic data that can
be linked to genomics information for crop improvement at all growth stages have become as important as genotyping. Thus,
high-throughput phenotyping has become the major bottleneck restricting crop breeding. Plant phenomics has been defined as the high-throughput, accurate acquisition and analysis of multi-dimensional phenotypes
during crop growing stages at the organism level, including the cell, tissue, organ, individual plant, plot, and field levels. With the rapid development of novel sensors, imaging technology,
and analysis methods, numerous infrastructure platforms have been developed for phenotyping.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...University of Maribor
Slides from talk:
Aleš Zamuda: Remote Sensing and Computational, Evolutionary, Supercomputing, and Intelligent Systems.
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Inter-Society Networking Panel GRSS/MTT-S/CIS Panel Session: Promoting Connection and Cooperation
https://www.etran.rs/2024/en/home-english/
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Ana Luísa Pinho
Functional Magnetic Resonance Imaging (fMRI) provides means to characterize brain activations in response to behavior. However, cognitive neuroscience has been limited to group-level effects referring to the performance of specific tasks. To obtain the functional profile of elementary cognitive mechanisms, the combination of brain responses to many tasks is required. Yet, to date, both structural atlases and parcellation-based activations do not fully account for cognitive function and still present several limitations. Further, they do not adapt overall to individual characteristics. In this talk, I will give an account of deep-behavioral phenotyping strategies, namely data-driven methods in large task-fMRI datasets, to optimize functional brain-data collection and improve inference of effects-of-interest related to mental processes. Key to this approach is the employment of fast multi-functional paradigms rich on features that can be well parametrized and, consequently, facilitate the creation of psycho-physiological constructs to be modelled with imaging data. Particular emphasis will be given to music stimuli when studying high-order cognitive mechanisms, due to their ecological nature and quality to enable complex behavior compounded by discrete entities. I will also discuss how deep-behavioral phenotyping and individualized models applied to neuroimaging data can better account for the subject-specific organization of domain-general cognitive systems in the human brain. Finally, the accumulation of functional brain signatures brings the possibility to clarify relationships among tasks and create a univocal link between brain systems and mental functions through: (1) the development of ontologies proposing an organization of cognitive processes; and (2) brain-network taxonomies describing functional specialization. To this end, tools to improve commensurability in cognitive science are necessary, such as public repositories, ontology-based platforms and automated meta-analysis tools. I will thus discuss some brain-atlasing resources currently under development, and their applicability in cognitive as well as clinical neuroscience.
The ability to recreate computational results with minimal effort and actionable metrics provides a solid foundation for scientific research and software development. When people can replicate an analysis at the touch of a button using open-source software, open data, and methods to assess and compare proposals, it significantly eases verification of results, engagement with a diverse range of contributors, and progress. However, we have yet to fully achieve this; there are still many sociotechnical frictions.
Inspired by David Donoho's vision, this talk aims to revisit the three crucial pillars of frictionless reproducibility (data sharing, code sharing, and competitive challenges) with the perspective of deep software variability.
Our observation is that multiple layers — hardware, operating systems, third-party libraries, software versions, input data, compile-time options, and parameters — are subject to variability that exacerbates frictions but is also essential for achieving robust, generalizable results and fostering innovation. I will first review the literature, providing evidence of how the complex variability interactions across these layers affect qualitative and quantitative software properties, thereby complicating the reproduction and replication of scientific studies in various fields.
I will then present some software engineering and AI techniques that can support the strategic exploration of variability spaces. These include the use of abstractions and models (e.g., feature models), sampling strategies (e.g., uniform, random), cost-effective measurements (e.g., incremental build of software configurations), and dimensionality reduction methods (e.g., transfer learning, feature selection, software debloating).
I will finally argue that deep variability is both the problem and solution of frictionless reproducibility, calling the software science community to develop new methods and tools to manage variability and foster reproducibility in software systems.
Exposé invité Journées Nationales du GDR GPL 2024
2. •How do Humans Learn? And why not replicating that?
•How do babies think?
Long Term
Slide 2 of 77
3. •“We might expect that babies would have really powerful learning mechanisms. And in fact, the baby's brain seems to be the most powerful learning computer on the planet.
•But real computers are actually getting to be a lot better. And there's been a revolution in our understanding of machine learning recently. And it all depends on the ideas of this guy, the Reverend Thomas Bayes, who was a statistician and mathematician in the 18th century.”
Alison Gopnik is an American professor of psychology and affiliate professor of philosophy at the University of California, Berkeley.
How do babies think
Slide 3 of 77
4. •“And essentially what Bayes did was to provide a mathematical way using probability theory to characterize, describe, the way that scientists find out about the world.
•So what scientists do is they have a hypothesis that they think might be likely to start with. They go out and test it against the evidence.
•The evidence makes them change that hypothesis. Then they test that new hypothesis and so on and so forth.”
Alison Gopnik is an American professor of psychology and affiliate professor of philosophy at the University of California, Berkeley.
How do babies think
Slide 4 of 77
5. •푃휔푋∝푃푋휔∗푃(휔)
•Posterior ∝ Likelihood * Prior
•If this is how our brain work, why not continue in this way !
Bayes’ Theorem
Slide 5 of 77
7. •푃휔푋∝푃푋휔∗푃(휔)
•To build the likelihood, we need tons of data (The Law of Large Numbers)
Bayes’ Theorem – Issues
Slide 6 of 77
8. •푃휔푋∝푃푋휔∗푃(휔)
•To build the likelihood, we need tons of data (The Law of Large Numbers)
•Not any data, labeled data !
Bayes’ Theorem – Issues
Slide 6 of 77
9. •푃휔푋∝푃푋휔∗푃(휔)
•To build the likelihood, we need tons of data (The Law of Large Numbers)
•Not any data, labeled data !
•We need to solve for features.
Bayes’ Theorem – Issues
Slide 6 of 77
10. •푃휔푋∝푃푋휔∗푃(휔)
•To build the likelihood, we need tons of data (The Law of Large Numbers)
•Not any data, labeled data !
•We need to solve for features.
•How should we decide on which features to use ?
Bayes’ Theorem – Issues
Slide 6 of 77
22. On Computer Perception
•The Adult visual system computes an incredibly complicated function of the input.
Slide 22 of 77
23. On Computer Perception
•The Adult visual system computes an incredibly complicated function of the input.
•We can try to implement most of this incredibly complicated function (hand- engineer features)
Slide 22 of 77
24. On Computer Perception
•The Adult visual system computes an incredibly complicated function of the input.
•We can try to implement most of this incredibly complicated function (hand- engineer features)
•OR, we can learn this function instead.
Slide 22 of 77
28. Feature Learning via Sparse Coding
• , 푋(2),…, 푋(푚) (each in 푅푛∗푛 )
• , Φ2,…, Φ푘 (also in 푅푛∗푛 ), so that each input X can be approximately decomposed as:
• 푎푗휑푗 푘푗 =1 , s.t. 푎푗 are mostly zero (“sparse”)
Slide 25 of 77
29. Feature Learning via Sparse Coding
•Sparse coding (Olshausen & Field,1996). Originally developed to explain early visual processing in the brain (edge detection).
• , 푋(2),…, 푋(푚) (each in 푅푛∗푛 )
• , Φ2,…, Φ푘 (also in 푅푛∗푛 ), so that each input X can be approximately decomposed as:
• 푎푗휑푗 푘푗 =1 , s.t. 푎푗 are mostly zero (“sparse”)
Slide 25 of 77
30. Feature Learning via Sparse Coding
•1) 푋 (1) , 푋 (2) 푋푋 푋 (2) (2) 푋 (2) ,…, 푋 (푚) 푋푋 푋 (푚) (푚푚) 푋 (푚) (each in 푅 푛∗푛 푅푅 푅 푛∗푛 푛푛∗푛푛 푅 푛∗푛 )
•Sparse coding (Olshausen & Field,1996). Originally developed to explain early visual processing in the brain (edge detection).
•Input: Images 푋 ( (1) (1) , 푋(2),…, 푋(푚) (each in 푅푛∗푛 )
• , Φ2,…, Φ푘 (also in 푅푛∗푛 ), so that each input X can be approximately decomposed as:
• 푎푗휑푗 푘푗 =1 , s.t. 푎푗 are mostly zero (“sparse”)
Slide 25 of 77
31. Feature Learning via Sparse Coding
• Φ 1 1 Φ 1 , Φ 2 Φ Φ 2 2 Φ 2 ,…, Φ 푘 Φ Φ 푘 푘푘 Φ 푘 (also in 푅 푛∗푛 푅푅 푅 푛∗푛 푛푛∗푛푛 푅 푛∗푛 ), so that each input X can be approximately decomposed as:
•1) 푋 (1) , 푋 (2) 푋푋 푋 (2) (2) 푋 (2) ,…, 푋 (푚) 푋푋 푋 (푚) (푚푚) 푋 (푚) (each in 푅 푛∗푛 푅푅 푅 푛∗푛 푛푛∗푛푛 푅 푛∗푛 )
•Sparse coding (Olshausen & Field,1996). Originally developed to explain early visual processing in the brain (edge detection).
•Learn: Dictionary of bases Φ 1 , Φ2,…, Φ푘 (also in 푅푛∗푛 ), so that each input X can be approximately decomposed as:
• , Φ2,…, Φ푘 (also in 푅푛∗푛 ), so that each input X can be approximately decomposed as:
• 푎푗휑푗 푘푗 =1 , s.t. 푎푗 are mostly zero (“sparse”)
Slide 25 of 77
32. Feature Learning via Sparse Coding
• 푗=1 푘 푎 푗 휑 푗 푗푗=1 푗=1 푘 푎 푗 휑 푗 푘푘 푗=1 푘 푎 푗 휑 푗 푎 푗 푎푎 푎 푗 푗푗 푎 푗 휑 푗 휑휑 휑 푗 푗푗 휑 푗 푗=1 푘 푎 푗 휑 푗 , s.t. 푎 푗 푎푎 푎 푗 푗푗 푎 푗 are mostly zero (“sparse”)
• Φ 1 1 Φ 1 , Φ 2 Φ Φ 2 2 Φ 2 ,…, Φ 푘 Φ Φ 푘 푘푘 Φ 푘 (also in 푅 푛∗푛 푅푅 푅 푛∗푛 푛푛∗푛푛 푅 푛∗푛 ), so that each input X can be approximately decomposed as:
•1) 푋 (1) , 푋 (2) 푋푋 푋 (2) (2) 푋 (2) ,…, 푋 (푚) 푋푋 푋 (푚) (푚푚) 푋 (푚) (each in 푅 푛∗푛 푅푅 푅 푛∗푛 푛푛∗푛푛 푅 푛∗푛 )
•Sparse coding (Olshausen & Field,1996). Originally developed to explain early visual processing in the brain (edge detection).
•X ≈ 푎푗휑푗 푘푗 =1 , s.t. 푎푗 are mostly zero (“sparse”)
• , Φ2,…, Φ푘 (also in 푅푛∗푛 ), so that each input X can be approximately decomposed as:
• 푎푗휑푗 푘푗 =1 , s.t. 푎푗 are mostly zero (“sparse”)
Slide 25 of 77
44. Brain Operation Modes
Slide 37 of 77
•Professor Daniel Khaneman, the Hero of Psychology.
•Won in 2002, the Nobel Prize in economics.
•Now he is teaching psychology in Princeton.
50. System One: Memory
Slide 43 of 77
•By the age of three we all learned that “Big things can’t go inside small things”.
51. System One: Memory
Slide 43 of 77
•By the age of three we all learned that “Big things can’t go inside small things”.
•All of us have tried to save their favorite movie on the computer and we know that those two hours requires gabs of space.
53. System One: Memory
Slide 45 of 77
•How do we cram the vast universe of our experience in a relatively small storage compartment between our ears?
54. System One: Memory
Slide 45 of 77
•How do we cram the vast universe of our experience in a relatively small storage compartment between our ears?
•We Cheat !
•Compress memories into critical thread and key features.
•Ex: “Dinner was disappointing”, “Tough Steak”
55. System One: Memory
Slide 45 of 77
•How do we cram the vast universe of our experience in a relatively small storage compartment between our ears?
•We Cheat !
•Compress memories into critical thread and key features.
•Ex: “Dinner was disappointing”, “Tough Steak”
•Later when we want to remember our experience, our brains reweave, and not retrieve, the scenes using the extracted features.
56. System One: Memory
Slide 46 of 77 Daniel Todd Gilbert is Professor of Psychology at Harvard University.
In this experiment two groups of people set down to watch a set of slides, the question group and the now question group. The slides were about two cars approaching a yield sign, one car turns right and then the two cars collide.
57. System One: Memory
Slide 46 of 77 Daniel Todd Gilbert is Professor of Psychology at Harvard University.
In this experiment two groups of people set down to watch a set of slides, the question group and the now question group. The slides were about two cars approaching a yield sign, one car turns right and then the two cars collide.
58. System One: Memory
Slide 46 of 77 Daniel Todd Gilbert is Professor of Psychology at Harvard University.
In this experiment two groups of people set down to watch a set of slides, the question group and the now question group. The slides were about two cars approaching a yield sign, one car turns right and then the two cars collide.
59. System One: Memory
Slide 47 of 77
•The no question group wasn’t asked any questions.
•The question group was asked the following question:
•Did another car pass by the blue car while it stopped at the Stop Sign?
•And then they were asked to pick which set of slides did they see, the one with the yield sign or the one with the stop sign.
60. System One: Memory
Slide 47 of 77
•90% of the no question group chose the yield sign
•80% of the question group chose the stop sign
61. System One: Memory
Slide 47 of 77
•90% of the no question group chose the yield sign
•80% of the question group chose the stop sign
•The general finding is: our brains compress experiences into key features and fill in details that were not actually stored. And this is the basic idea behind the auto-encoders
63. •An Auto-encoder neural network is an unsupervised learning algorithm that applies back propagation, on a set of unlabeled training examples {푥1, 푥2, 푥4,….} where 푥푖 ∈푅푛 by setting the target values to be equal to the inputs.[6]
•i.e. it uses 푦푖=푥푖
•Original contributions in back propagation was made by Hinton and Hebbian in 1980s and nowadays by Hinton , Salakhutdinov, Bengio, LeCun and Erhan (2006-2010)
Sparse Auto-encoder
Slide 49 of 77
64. •Before we get further into the details of the algorithm, we need to quickly go through neural network.
•To describe neural networks, we will begin by describing the simplest possible neural network. One that comprises a single "neuron." We will use the following diagram to denote a single neuron [5]
Neural Network
Single Neuron [8]
Slide 50 of 77
65. •This "neuron" is a computational unit that takes as input x1,x2,x3 (and a +1 intercept term), and outputs
• ℎ푊,푏푋=푓푊푇푥=푓( 푊푖푥푖+푏)3푖 =1 where 푓:ℜ→ℜ is called the activation function. [5]
Neural Network
Slide 51 of 77
66. •The activation function can be:[8]
1)Sigmoid function : 푓푧= 11+exp (−푧) , output scale from [0,1]
Sigmoid Activation Function
Sigmoid Function [8]
Slide 52 of 77
67. •2) Tanh function: : 푓푧=tanh(푧) 푒푧−푒−푧 푒푧+푒−푧 , output scale from [-1,1]
Tanh Activation Function
Tanh Function [8]
Slide 53 of 77
68. •Neural network parameters are:
•(W,b) = (W(1),b(1),W(2),b(2)), where we write 푊푖푗 (푙) to denote the parameter (or weight) associated with the connection between unit j in layer l, and unit i in layer l+ 1.
•푏푖 (푙) the bias associated with unit i in layer l + 1.
•푎푖 (푙) will denote the activation (meaning output value) of unit i in layer l.
•Given a fixed setting of the parameters W, b, our neural network defines a hypothesis hW,b(x) that outputs a real number.
Neural Network Model
Slide 54 of 77
70. •The auto-encoder tries to learn a function ℎ푤,푏(푥)≈푥 . In other words, it is trying an approximation to the identity function, so as to output 푥^ is similar to 푥
•Placing constraints on the network, such as limiting the number of hidden units, or imposing a sparsity constraint on the hidden units, lead to discover interesting structure in the data, even if the number of hidden units is large.
Auto-encoders and Sparsity
Slide 56 of 77
71. •Assumption :
1.The neurons to be inactive most of the time (a neuron to be "active" (or as "firing") if its output value is close to 1, or "inactive" if its output value is close to 0) and the activation function is sigmoid function.
2.Recall that 푎푗 (2) denotes the activation of hidden unit 푗 in layer 2 in the auto-encoder
3. 푎푗 (2) (x) to denote the activation of this hidden unit when the network is given a specific input 푥
4.Let: 휌 = 1 푚 [푎푗 (2)(푥푖)] 휌 푗=휌 푚푖=1 be the average activation unit 푗 (averaged over the training set).
•Objective:
•We would like to (approximately) enforce the constraint: 휌푗 = 휌 where 휌 is a sparsity parameter, a small value close to zero
Auto-encoders and Sparsity Algorithm
Slide 57 of 77
72. •To achieve this, we will add an extra penalty term to our optimization objective that penalizes : 휌푗 deviating significantly from 휌.
• 휌log 휌 휌푗 푠2 푗=1 +(1- 휌) log1−휌 1−휌푗 , here “푠2” is the number of neurons in the hidden layer, and the index 푗 is the summing over the hidden units in the network.[6]
• It can also be written 퐾퐿(휌 || 휌푗 )푠2 푗=1 where 퐾퐿(휌 || 휌푗 ) = 휌log 휌 휌푗 +(1-휌) log1−휌 1− 휌푗 is the Kullback-Leibler (KL) divergence between a Bernoulli random variable with mean 휌 and a Bernoulli random variable with mean 휌푗 . [6]
•KL-divergence is a standard function for measuring how different two different distributions are.
Autoencoders and Sparsity Algorithm
Slide 58 of 77
73. •Kl penalty function has the following property 퐾퐿(휌 || 휌푗 ) =0 if 휌푗 =휌 and otherwise it increases monotonically as 휌푗 diverges from 휌 .
•For example, if we plotted 퐾퐿(휌 || 휌푗 ) for a range of values 휌푗
•(set 휌=0.2), We will see that the KL-divergence reaches its minimum
•of 0 at 휌푗 = 휌 and approach ∞ as 휌푗 approaches 0 or 1.
•Thus, minimizing this penalty term has the effect of causing 휌푗
• to close to 휌
Auto-encoders and Sparsity Algorithm –cont’d
KL Function
Slide 59 of 77
76. •We implemented a sparse auto-encoder, trained with 8×8 image patches using the L-BFGS optimization algorithm
Auto-encoder Implementation
A random sample of 200 patches from the dataset.
Slide 62 of 77
83. In Progress Work (Future Results)
•Given the fact of small dataset for facial features
•We train the neural network with a random dataset in hope that the average mean would be a nice base start for the tuning phase of the neural network
•We then fine tune with the smaller dataset of facial features
Slide 69 of 77
87. •Twitter:
Data - Now
Slide 72 of 77
7 terabytes of Data / Day
•Facebook:
88. •Twitter:
Data - Now
Slide 72 of 77
7 terabytes of Data / Day
•Facebook:
500 terabytes of Data / Day
89. •NASA announced its square kilometer telescope.
Data – Tomorrow
Slide 73 of 77
90. •NASA announced its square kilometer telescope.
Data – Tomorrow
Slide 73 of 77
•It will generate 700 terabyte of data every second.
91. •NASA announced its square kilometer telescope.
Data – Tomorrow
Slide 73 of 77
•It will generate 700 terabyte of data every second.
•It will generate data of the same size as the internet today in two days.
•Do you know how long it is going to take Google, with all its resources, to just index data generated from this beast in a year? 3 whole months, 90 days !