The document discusses understanding autoencoders through interventions on latent variables. It introduces representation learning and the manifold hypothesis. Good representations are described as extensible, compact, able to extrapolate, robust, and self-aware. Disentanglement aims to maximize independence between latent variables but factors may be related. Interventional consistency focuses on the causal structure of the generative process. Probing a trained VAE using interventions reveals unexpected structure in the learned latent space beyond statistical independence of factors. Response functions and matrices are used to analyze causal links between latent variables and disentangle factors in a semantically meaningful way.
Representation Learning & Generative Modeling with Variational Autoencoder(VA...changedaeoh
표현학습(representation learning)과 생성모델링(generative modeling)에 대한 개요를 설명하고 이를 Auto-Encoding Variational Bayes 논문의 내용과 연결시켜 VAE를 이해한다.
연합동아리 TAVE research seminar 21.05.18 발표자료
발표자: 오창대
Professor Steve Roberts; The Bayesian Crowd: scalable information combinati...Ian Morgan
Professor Steve Roberts, Machine learning research group and Oxford-Man Institute + Alan Turing Institute. Steve gave this talk on the 24th January at the London Bayes Nets meetup.
Rsqrd AI - ML Interpretability: Beyond Feature ImportanceAlessya Visnjic
In this talk, Javier Antorán discusses the importance of uncertainty when it comes to ML interpretability. He offers a new uncertainty-based interpretability technique called CLUE and compares it to existing model interpretability techniques in two usability studies. Javier is a Ph.D. student at the University of Cambridge. His research interests include Bayesian deep learning, uncertainty in machine learning, representation learning, and information theory.
Representation Learning & Generative Modeling with Variational Autoencoder(VA...changedaeoh
표현학습(representation learning)과 생성모델링(generative modeling)에 대한 개요를 설명하고 이를 Auto-Encoding Variational Bayes 논문의 내용과 연결시켜 VAE를 이해한다.
연합동아리 TAVE research seminar 21.05.18 발표자료
발표자: 오창대
Professor Steve Roberts; The Bayesian Crowd: scalable information combinati...Ian Morgan
Professor Steve Roberts, Machine learning research group and Oxford-Man Institute + Alan Turing Institute. Steve gave this talk on the 24th January at the London Bayes Nets meetup.
Rsqrd AI - ML Interpretability: Beyond Feature ImportanceAlessya Visnjic
In this talk, Javier Antorán discusses the importance of uncertainty when it comes to ML interpretability. He offers a new uncertainty-based interpretability technique called CLUE and compares it to existing model interpretability techniques in two usability studies. Javier is a Ph.D. student at the University of Cambridge. His research interests include Bayesian deep learning, uncertainty in machine learning, representation learning, and information theory.
Professor Maria Petrou gave a lecture on "A Classification Framework for Software Component Models" in the Distinguished Lecturer Series - Leon The Mathematician.
More Information available at:
http://dls.csd.auth.gr
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...Chris Rackauckas
How does automatic differentiation work, what happens when you apply it to equation solvers, and how can it go wrong? This talk is all about the details of how scientific machine learning (SciML) works. It goes into detail as to how neural networks are trained in the context of equation solvers, along with the numerical issues that can arise in the differentiation processes.
https://sciml.ai/
An overview on unsupervised deep learning models and strategies to speed up Reinforcement Learning and make it more sample efficient in goal-based robotics tasks.
Data science is an area at the interface of statistics, computer science, and mathematics.
• Statisticians contributed a large inferential framework, important Bayesian perspectives, the bootstrap and CART and random forests, and the concepts of sparsity and parsimony.
• Computer scientists contributed an appetite for big, challenging problems.They also pioneered neural networks, boosting, PAC bounds, and developed programming languages, such as Spark and hadoop, for handling Big Data.
• Mathematicians contributed support vector machines, modern optimization, tensor analysis, and (maybe) topological data analysis.
Errors of Artificial Intelligence, their Correction and Simplicity Revolution...Alexander Gorban
We review and analyse biological, physical, and mathematical problems at the core of the fundamental question: how can high-dimensional brain organise reliable and fast learning in high-dimensional world of data by simple tools?
Two critical applications are reviewed: one-shot correction of errors in artificial intellectual systems and emergence of static and associative memories in ensembles of single neurons. Error correctors should be simple; not damage the existing skills of the system; allow fast non-iterative learning and correction of new mistakes without destroying the previous fixes. All these demands can be satisfied by new tools based on the concentration of measure phenomena and stochastic separation theory.
In several words, the stochastic separation theorems state that for an essentially high-dimensional distributions a random point can be separated from a random set by Fisher's linear discriminant with high probability. The number of points in this set can grow exponentially with dimension. Different versions of stochastic separation theorems use different definitions of `random sets' and `essentially high-dimensional distributions' but the essence of these definitions is simple: sets with very small (vanishing) volume should not have high probability even for large dimension.
The talk is based on the work: A.N. Gorban, V.A. Makarov, I.Y. Tyukin, The unreasonable effectiveness of small neural ensembles in high-dimensional brain. Physics of Life Reviews, 2019, https://doi.org/10.1016/j.plrev.2018.09.005
May 2015 talk to SW Data Meetup by Professor Hendrik Blockeel from KU Leuven & Leiden University.
With increasing amounts of ever more complex forms of digital data becoming available, the methods for analyzing these data have also become more diverse and sophisticated. With this comes an increased risk of incorrect use of these methods, and a greater burden on the user to be knowledgeable about their assumptions. In addition, the user needs to know about a wide variety of methods to be able to apply the most suitable one to a particular problem. This combination of broad and deep knowledge is not sustainable.
The idea behind declarative data analysis is that the burden of choosing the right statistical methodology for answering a research question should no longer lie with the user, but with the system. The user should be able to simply describe the problem, formulate a question, and let the system take it from there. To achieve this, we need to find answers to questions such as: what languages are suitable for formulating these questions, and what execution mechanisms can we develop for them? In this talk, I will discuss recent and ongoing research in this direction. The talk will touch upon query languages for data mining and for statistical inference, declarative modeling for data mining, meta-learning, and constraint-based data mining. What connects these research threads is that they all strive to put intelligence about data analysis into the system, instead of assuming it resides in the user.
Hendrik Blockeel is a professor of computer science at KU Leuven, Belgium, and part-time associate professor at Leiden University, The Netherlands. His research interests lie mostly in machine learning and data mining. He has made a variety of research contributions in these fields, including work on decision tree learning, inductive logic programming, predictive clustering, probabilistic-logical models, inductive databases, constraint-based data mining, and declarative data analysis. He is an action editor for Machine Learning and serves on the editorial board of several other journals. He has chaired or organized multiple conferences, workshops, and summer schools, including ILP, ECMLPKDD, IDA and ACAI, and he has been vice-chair, area chair, or senior PC member for ECAI, IJCAI, ICML, KDD, ICDM. He was a member of the board of the European Coordinating Committee for Artificial Intelligence from 2004 to 2010, and currently serves as publications chair for the ECMLPKDD steering committee.
A measure to evaluate latent variable model fit by sensitivity analysisDaniel Oberski
Latent variable models involve restrictions on the data that can be formulated in terms of "misspecifications": restrictions with a model-based meaning. Examples include zero cross-loadings and local dependencies, as well as “measurement invariance” or “differential item functioning”. If incorrect, misspecifications can potentially disturb the main purpose of the latent variable analysis—seriously so in some cases.
Recently, I proposed to evaluate whether a particular analysis at hand is such a case or not.
To do this, I define a measure based on the likelihood of the restricted model that approximates the change in the parameters of interest if the misspecification were freed, the EPC-interest. The main idea is to examine the EPC-interest and free those misspecifications that are “important” while ignoring those that are not. I have implemented the EPC-interest in the lavaan software for structural equation modeling and the Latent Gold software for latent class analysis.
This approach can resolve several problems and inconsistencies in the current practice of model fit evaluation used in latent variable analysis, something I illustrate using analyses from the “measurement invariance” literature and from item response theory.
invited talk in the ExUM workshop in the UMAP 2022 conference
abstract:
Explainability has become an important topic both in Data Science and AI in general and in recommender systems in particular, as algorithms have become much less inherently explainable. However, explainability has different interpretations and goals in different fields. For example, interpretability and explanainability tools in machine learning are predominantly developed for Data Scientists to understand and scrutinize their models. Current tools are therefore often quite technical and not very ‘user-friendly’. I will illustrate this with our recent work on improving the explainability of model-agnostic tools such as LIME and SHAP. Another stream of research on explainability in the HCI and XAI fields focuses more on users’ needs for explainability, such as contrastive and selective explanations and explanations that fit with the mental models and beliefs of the user. However, how to satisfy those needs is still an open question. Based on recent work in interactive AI and machine learning, I will propose that explainability goes together with interactivity, and will illustrate this with examples from our own work in music genre exploration, that combines visualizations and interactive tools to help users understand and tune our exploration model.
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!Sri Ambati
This meetup took place in Mountain View on January 24th, 2019.
Description:
With the effort and contributions from researchers and practitioners from academia and industry, Machine Learning Interpretation has become a young sub-field of ML. However, the norms around its definition and understanding is still in its infancy and there are numerous different approaches emerging rapidly. However, there seems to be a lack of a consistent explanation framework to evaluate and consistently benchmark different algorithms - evaluating against interpretation, completeness and consistency of the algorithms.
The idea with the gym is to provide a controlled interactive environment for all forms of Machine Learning algorithms, - initially focusing on supervised predictive modeling problems, to allow analysts and data-scientists to explore, debug and generate insightful understanding of the models by
1.Model Validation: Ways to explore and validate black box ML systems enabling model comparison both globally and locally - identifying biases in the training data through interpretation.
2.What-if Analysis: An interactive environment where communication can happen i.e. enable learning through interactions. User having the ability to conduct "What-If" analysis - effect of single or multiple features and their interactions
3.Model Debugging: Ways to analyze the misbehavior of the model by exploring counterfactual examples(adversarial examples and training)
4. Interpretable Models: Ability to build natively interpretable models - with the goal to simplify complex models to enable better understanding.
The central concept with MLI gym is to have an interactive environment where one could explore and simulate variations in the world(a world post a model is operationalized) beyond the defined model metrics point estimates - e.g. ROC-AUC, confusion matrix, RMSE, R2 score and others.
Speaker's Bio:
Pramit is a Lead Data Scientist/ at H2O.ai. His area of interests is building Statistical/Machine Learning models(Bayesian and Frequentist Modeling techniques) to help the business realize their data-driven goals.
Currently, he is exploring "Model Interpretation" as means to efficiently understand the true nature of predictive models to enable model robustness and security. He believes effective Model Inference coupled with Adversarial training could lead to building trustworthy models with known blind spots. He has started an open source project Skater: https://github.com/datascienceinc/Skater to solve the need for Model Inference(The project is still in its early stages of development but check it out, always eager for feedback)
Professor Maria Petrou gave a lecture on "A Classification Framework for Software Component Models" in the Distinguished Lecturer Series - Leon The Mathematician.
More Information available at:
http://dls.csd.auth.gr
Automatic Differentiation and SciML in Reality: What can go wrong, and what t...Chris Rackauckas
How does automatic differentiation work, what happens when you apply it to equation solvers, and how can it go wrong? This talk is all about the details of how scientific machine learning (SciML) works. It goes into detail as to how neural networks are trained in the context of equation solvers, along with the numerical issues that can arise in the differentiation processes.
https://sciml.ai/
An overview on unsupervised deep learning models and strategies to speed up Reinforcement Learning and make it more sample efficient in goal-based robotics tasks.
Data science is an area at the interface of statistics, computer science, and mathematics.
• Statisticians contributed a large inferential framework, important Bayesian perspectives, the bootstrap and CART and random forests, and the concepts of sparsity and parsimony.
• Computer scientists contributed an appetite for big, challenging problems.They also pioneered neural networks, boosting, PAC bounds, and developed programming languages, such as Spark and hadoop, for handling Big Data.
• Mathematicians contributed support vector machines, modern optimization, tensor analysis, and (maybe) topological data analysis.
Errors of Artificial Intelligence, their Correction and Simplicity Revolution...Alexander Gorban
We review and analyse biological, physical, and mathematical problems at the core of the fundamental question: how can high-dimensional brain organise reliable and fast learning in high-dimensional world of data by simple tools?
Two critical applications are reviewed: one-shot correction of errors in artificial intellectual systems and emergence of static and associative memories in ensembles of single neurons. Error correctors should be simple; not damage the existing skills of the system; allow fast non-iterative learning and correction of new mistakes without destroying the previous fixes. All these demands can be satisfied by new tools based on the concentration of measure phenomena and stochastic separation theory.
In several words, the stochastic separation theorems state that for an essentially high-dimensional distributions a random point can be separated from a random set by Fisher's linear discriminant with high probability. The number of points in this set can grow exponentially with dimension. Different versions of stochastic separation theorems use different definitions of `random sets' and `essentially high-dimensional distributions' but the essence of these definitions is simple: sets with very small (vanishing) volume should not have high probability even for large dimension.
The talk is based on the work: A.N. Gorban, V.A. Makarov, I.Y. Tyukin, The unreasonable effectiveness of small neural ensembles in high-dimensional brain. Physics of Life Reviews, 2019, https://doi.org/10.1016/j.plrev.2018.09.005
May 2015 talk to SW Data Meetup by Professor Hendrik Blockeel from KU Leuven & Leiden University.
With increasing amounts of ever more complex forms of digital data becoming available, the methods for analyzing these data have also become more diverse and sophisticated. With this comes an increased risk of incorrect use of these methods, and a greater burden on the user to be knowledgeable about their assumptions. In addition, the user needs to know about a wide variety of methods to be able to apply the most suitable one to a particular problem. This combination of broad and deep knowledge is not sustainable.
The idea behind declarative data analysis is that the burden of choosing the right statistical methodology for answering a research question should no longer lie with the user, but with the system. The user should be able to simply describe the problem, formulate a question, and let the system take it from there. To achieve this, we need to find answers to questions such as: what languages are suitable for formulating these questions, and what execution mechanisms can we develop for them? In this talk, I will discuss recent and ongoing research in this direction. The talk will touch upon query languages for data mining and for statistical inference, declarative modeling for data mining, meta-learning, and constraint-based data mining. What connects these research threads is that they all strive to put intelligence about data analysis into the system, instead of assuming it resides in the user.
Hendrik Blockeel is a professor of computer science at KU Leuven, Belgium, and part-time associate professor at Leiden University, The Netherlands. His research interests lie mostly in machine learning and data mining. He has made a variety of research contributions in these fields, including work on decision tree learning, inductive logic programming, predictive clustering, probabilistic-logical models, inductive databases, constraint-based data mining, and declarative data analysis. He is an action editor for Machine Learning and serves on the editorial board of several other journals. He has chaired or organized multiple conferences, workshops, and summer schools, including ILP, ECMLPKDD, IDA and ACAI, and he has been vice-chair, area chair, or senior PC member for ECAI, IJCAI, ICML, KDD, ICDM. He was a member of the board of the European Coordinating Committee for Artificial Intelligence from 2004 to 2010, and currently serves as publications chair for the ECMLPKDD steering committee.
A measure to evaluate latent variable model fit by sensitivity analysisDaniel Oberski
Latent variable models involve restrictions on the data that can be formulated in terms of "misspecifications": restrictions with a model-based meaning. Examples include zero cross-loadings and local dependencies, as well as “measurement invariance” or “differential item functioning”. If incorrect, misspecifications can potentially disturb the main purpose of the latent variable analysis—seriously so in some cases.
Recently, I proposed to evaluate whether a particular analysis at hand is such a case or not.
To do this, I define a measure based on the likelihood of the restricted model that approximates the change in the parameters of interest if the misspecification were freed, the EPC-interest. The main idea is to examine the EPC-interest and free those misspecifications that are “important” while ignoring those that are not. I have implemented the EPC-interest in the lavaan software for structural equation modeling and the Latent Gold software for latent class analysis.
This approach can resolve several problems and inconsistencies in the current practice of model fit evaluation used in latent variable analysis, something I illustrate using analyses from the “measurement invariance” literature and from item response theory.
invited talk in the ExUM workshop in the UMAP 2022 conference
abstract:
Explainability has become an important topic both in Data Science and AI in general and in recommender systems in particular, as algorithms have become much less inherently explainable. However, explainability has different interpretations and goals in different fields. For example, interpretability and explanainability tools in machine learning are predominantly developed for Data Scientists to understand and scrutinize their models. Current tools are therefore often quite technical and not very ‘user-friendly’. I will illustrate this with our recent work on improving the explainability of model-agnostic tools such as LIME and SHAP. Another stream of research on explainability in the HCI and XAI fields focuses more on users’ needs for explainability, such as contrastive and selective explanations and explanations that fit with the mental models and beliefs of the user. However, how to satisfy those needs is still an open question. Based on recent work in interactive AI and machine learning, I will propose that explainability goes together with interactivity, and will illustrate this with examples from our own work in music genre exploration, that combines visualizations and interactive tools to help users understand and tune our exploration model.
Get hands-on with Explainable AI at Machine Learning Interpretability(MLI) Gym!Sri Ambati
This meetup took place in Mountain View on January 24th, 2019.
Description:
With the effort and contributions from researchers and practitioners from academia and industry, Machine Learning Interpretation has become a young sub-field of ML. However, the norms around its definition and understanding is still in its infancy and there are numerous different approaches emerging rapidly. However, there seems to be a lack of a consistent explanation framework to evaluate and consistently benchmark different algorithms - evaluating against interpretation, completeness and consistency of the algorithms.
The idea with the gym is to provide a controlled interactive environment for all forms of Machine Learning algorithms, - initially focusing on supervised predictive modeling problems, to allow analysts and data-scientists to explore, debug and generate insightful understanding of the models by
1.Model Validation: Ways to explore and validate black box ML systems enabling model comparison both globally and locally - identifying biases in the training data through interpretation.
2.What-if Analysis: An interactive environment where communication can happen i.e. enable learning through interactions. User having the ability to conduct "What-If" analysis - effect of single or multiple features and their interactions
3.Model Debugging: Ways to analyze the misbehavior of the model by exploring counterfactual examples(adversarial examples and training)
4. Interpretable Models: Ability to build natively interpretable models - with the goal to simplify complex models to enable better understanding.
The central concept with MLI gym is to have an interactive environment where one could explore and simulate variations in the world(a world post a model is operationalized) beyond the defined model metrics point estimates - e.g. ROC-AUC, confusion matrix, RMSE, R2 score and others.
Speaker's Bio:
Pramit is a Lead Data Scientist/ at H2O.ai. His area of interests is building Statistical/Machine Learning models(Bayesian and Frequentist Modeling techniques) to help the business realize their data-driven goals.
Currently, he is exploring "Model Interpretation" as means to efficiently understand the true nature of predictive models to enable model robustness and security. He believes effective Model Inference coupled with Adversarial training could lead to building trustworthy models with known blind spots. He has started an open source project Skater: https://github.com/datascienceinc/Skater to solve the need for Model Inference(The project is still in its early stages of development but check it out, always eager for feedback)
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Macroeconomics- Movie Location
This will be used as part of your Personal Professional Portfolio once graded.
Objective:
Prepare a presentation or a paper using research, basic comparative analysis, data organization and application of economic information. You will make an informed assessment of an economic climate outside of the United States to accomplish an entertainment industry objective.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
2024.06.01 Introducing a competency framework for languag learning materials ...Sandy Millin
http://sandymillin.wordpress.com/iateflwebinar2024
Published classroom materials form the basis of syllabuses, drive teacher professional development, and have a potentially huge influence on learners, teachers and education systems. All teachers also create their own materials, whether a few sentences on a blackboard, a highly-structured fully-realised online course, or anything in between. Despite this, the knowledge and skills needed to create effective language learning materials are rarely part of teacher training, and are mostly learnt by trial and error.
Knowledge and skills frameworks, generally called competency frameworks, for ELT teachers, trainers and managers have existed for a few years now. However, until I created one for my MA dissertation, there wasn’t one drawing together what we need to know and do to be able to effectively produce language learning materials.
This webinar will introduce you to my framework, highlighting the key competencies I identified from my research. It will also show how anybody involved in language teaching (any language, not just English!), teacher training, managing schools or developing language learning materials can benefit from using the framework.
Instructions for Submissions thorugh G- Classroom.pptxJheel Barad
This presentation provides a briefing on how to upload submissions and documents in Google Classroom. It was prepared as part of an orientation for new Sainik School in-service teacher trainees. As a training officer, my goal is to ensure that you are comfortable and proficient with this essential tool for managing assignments and fostering student engagement.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
How to Make a Field invisible in Odoo 17Celine George
It is possible to hide or invisible some fields in odoo. Commonly using “invisible” attribute in the field definition to invisible the fields. This slide will show how to make a field invisible in odoo 17.
A Strategic Approach: GenAI in EducationPeter Windle
Artificial Intelligence (AI) technologies such as Generative AI, Image Generators and Large Language Models have had a dramatic impact on teaching, learning and assessment over the past 18 months. The most immediate threat AI posed was to Academic Integrity with Higher Education Institutes (HEIs) focusing their efforts on combating the use of GenAI in assessment. Guidelines were developed for staff and students, policies put in place too. Innovative educators have forged paths in the use of Generative AI for teaching, learning and assessments leading to pockets of transformation springing up across HEIs, often with little or no top-down guidance, support or direction.
This Gasta posits a strategic approach to integrating AI into HEIs to prepare staff, students and the curriculum for an evolving world and workplace. We will highlight the advantages of working with these technologies beyond the realm of teaching, learning and assessment by considering prompt engineering skills, industry impact, curriculum changes, and the need for staff upskilling. In contrast, not engaging strategically with Generative AI poses risks, including falling behind peers, missed opportunities and failing to ensure our graduates remain employable. The rapid evolution of AI technologies necessitates a proactive and strategic approach if we are to remain relevant.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Model Attribute Check Company Auto PropertyCeline George
In Odoo, the multi-company feature allows you to manage multiple companies within a single Odoo database instance. Each company can have its own configurations while still sharing common resources such as products, customers, and suppliers.
Read| The latest issue of The Challenger is here! We are thrilled to announce that our school paper has qualified for the NATIONAL SCHOOLS PRESS CONFERENCE (NSPC) 2024. Thank you for your unwavering support and trust. Dive into the stories that made us stand out!
2. Representation Learning
• The fundamental claim of representation learning is that our
problem can be better solved using a different (smaller) space
than the input (ambient) space → the Manifold Hypothesis
• So, break down our solution into two pieces:
1. Organize input into a more useful form → learn a representation
2. Focusing on what is left → solving the actual problem/s (not shown)
• In other words, if the input lives in , we only need , where
2
3. Understanding the representation
• High-level features of good representations:
❑Extensible – easily integrate expert knowledge
❑Compact – efficient time and space complexity
❑Extrapolate – generalize on a semantic level
❑Robust – not sensitive to unimportant changes
❑Self-aware – estimates uncertainties
• Why bother with story-telling, when the performance is what matters?
• Form connections with past work → educational
• Identify weaknesses and motivate improvements → innovative
3
Key question: On the quest for good representations, how can we make sense of what we have?
The Mythos of Model Interpretability
by Zachary Lipton (2017)
4. Disentanglement (the obvious)
• With the manifold hypothesis, we assumed there are a small
number of underlying factors that give rise to the observation, so
how about the representation just disentangles those factors?
• Simple inductive bias: maximize statistical independence
between latent variables to ensure there’s no overlapping
information
• What if the factors are not statistically independent?
What about non-trivial variable structure?
4
Example from Yoshua Bengio: a fork and knife are not statistically
independent, but can however be separately manipulated.
β-VAE, FVAE, DIP-VAE, TC-VAE, β-TC-VAE, etc.
5. Causality: Genuinely predictive models
• Statistical models identify patterns in
the dataset, but these correlations
may be spurious → non-predictive!
• ICM Principle – although individual
factors may not be independent,
the true process to be is comprised
of independent mechanisms
(→ interventions in an SCM)
• However, without strong assumptions or supervision, the true causal variables
cannot be identified (much less the full mechanisms) → guarantees are unrealistic
5
Towards Causal Representation Learning by Schölkopf et al. (2021) Locatello et al. (2018; arXiv 1811.12359)
6. Guiding Principle: Interventional Consistency
7
noise
interventions
Identifiability problem: impossible to guarantee that the model learns the true causal drivers
The effect of each individual lever may be different, but if they are equivalent in aggregate,
then you can’t distinguish the “true” from the learned generative process.
→ Now let’s focus on the causal structure of our learned generative process?
noise
interventions
7. Setting the Scene
• I give you a trained (beta-)VAE using a
deep CNN (500k params). Nothing special.
• Trained on 3D-Shapes – synthetic process
with relatively small observations and
6 independent DOFs (no supervision).
• The true factors of variation are
independent, and we see disentanglement
…but is that the full story?
Ours:
Disentangled:
8. Curiosity #1:
Prior doesn’t match the aggregate posterior
• Prior (green) doesn’t match the
aggregate posterior (blue)
• Latent variables are not statistically
independent
• But maybe that’s the point – you
can only trust your regularization
objective so much.
Weird
Ms. Statistics
9. Curiosity #2:
Decoder extends beyond the training manifold
• Decoder can still generate sensible
samples beyond the aggregate
posterior → good for generative
modeling
• Decoder doesn’t just invert the
encoder, but is doing more work
“for free”.
Weird
Manifold Man
2D Latent Traversal
VAEs display some interventional consistency out of the box
→How can we use that?
10. Hypothesis: Latent Space vs Latent Manifold
Can we separate the semantic information S (→ necessary to reconstruct
the sample) from the any “exogenous” information U in the latent space
which the decoder ignores anyway?
For each observation, find a latent vector that makes the
subsequent reconstruction as good as possible (without
straying too far from the prior)
For each point in the latent space, place it as close to the
data manifold as possible consistent with the encoder
(for reconstruction)
11. Latent Responses
• We quantify the semantic change in the
sample by measuring the effect of the
intervention in
• This enables quantifying the relationship
between latent variables by observing
how interventions “propagate”
→ how is semantics captured?
13
?
12. Probing the Learned Manifold
• Assuming the reconstructions have sufficiently high fidelity,
we can treat
• Response function:
• Interventional Response:
14
The response function projects
the perturbed point back onto the
latent manifold*
*similar to memorization in Radhakrishnan et. al. (2018; arXiv: 1810.10333)
Ambient Space Latent Space Latent Space
Ambient Space
Latent
Manifold
Data
Manifold
Generative
Manifold
“Response”
Manifold
13. Structure of the Latent Space
Assuming the fidelity of reconstructions is sufficiently high, the response
function filters out noise leaving only the semantic information in the
latent code.
15
14. Latent Response Matrix
• Define as resampling only
• To identify the causal links between the
latent variables, we intervene on one latent
variable at a time and compute the average
resulting effect on all latent variables
• Note that for this model, interventions on
many of the latent variables doesn’t result in
any significant effect → non-informative
17
15. Curiosity #3: Unexpected structure emerges
• Despite the true factors being
statistically independent, the
learned variables are not
• Perhaps the latent variables
contains additional structure
selected (implicitly) by our
inductive biases (e.g. continuity)
Cool
Frau Causality
→ What is this unexpected structure in the learned generative process?
16. Causal Disentanglement
• Conventionally disentanglement is evaluated by quantifying
how predictive each latent variable is for each true factor
• But for a generative model, what matters is how well a latent
variable controls a desired true factor
Conditioned Response Matrix (Causal)
DCI Responsibility Matrix (Statistical)
Eastwood et al. (2018; OpenReview By-7dz-AZ)
20
17. Latent Response Maps
• Starting from a 2D projection of the latent space, we can evaluate the
latent motion all over the latent space to map out the latent manifold
directly.
• Think of the response map as a field showing how far the model will
move in the latent space to reach the manifold
• We can use the divergence of the response map to get a sense
whether the response is converging or diverging at any point in the
latent space
• Lastly, the mean curvature , tells us where the response is
converging to → the latent manifold
23
Example Response Map
Blue:
Orange:
Arrow:
18. Double-helix Toy Example
• Given noisy samples from a double helix (3D ambient
space), our representation is 2D
24
19. Traversing the Helix Manifold
• Now that we can explicitly map out the latent manifold, we can directly
traverse along the maximum curvature regions of the latent space to
avoid leaving the manifold
→ semantic interpolations
Interpolating between two
(orange) samples. Naively we
take the Euclidean shortest path
(red), but using the response
maps, we can find a more
meaningful path (green)
25
Latent Space Ambient Space
20. So… what’s the manifold look like?
Divergence Mean Curvature
Decoded Samples
Note, floor color changes when
crossing the “decision boundaries”
where the latent response has high
divergence.
The high curvature regions (i.e. where
the responses converge to) resemble
10 categories ordered as a circle →
ground truth hue!
22. Another Opportunity for some Interpolations
30
Shortest path (Euclidean)
Best path (using response maps)
23. Conclusions
• Naïve disentanglement fails to capture:
• Non-trivial geometry of the true factors (e.g. periodicity)
• Relationships between true factors (e.g. facial hair vs. sex)
• Latent Responses – in reality, the true factors are out of scope, so let’s use
the causal machinery to understand the learned process in its own right.
• Identify causal links between learned variables directly
• Condition on true factors to evaluate causal disentanglement → fairness of
generative model
• Visualize the learned manifold directly (to reveal learned hidden geometry)
Yup!
Frau Causality
Hmm
Manifold Man
Hmm
Ms. Statistics
For links to identifiability: Reizinger et al. (2022: arXiv:2206.02416) 41
24. 42
2D MNIST
• Everyone who worked on the
project: Stefan Bauer,
Michel Besserve, and of
course Bernhard Schölkopf
• Thanks to: the EI department
and the MPI-IS
• For more details, see our
arXiv paper: 2106.16091
Thank you!