This document discusses generative adversarial networks (GANs) and their applications. It begins with an overview of generative models including variational autoencoders and GANs. GANs use two neural networks, a generator and discriminator, that compete against each other in a game theoretic framework. The generator learns to generate fake samples to fool the discriminator, while the discriminator learns to distinguish real and fake samples. Applications discussed include image-to-image translation using conditional GANs to map images from one domain to another, and text-to-image translation using GANs to generate images from text descriptions.
In these slides, Generative Adversarial Network (GAN) is briefly introduced, and some GAN applications in medical imaging are presented. In the conclusions, some comments are given for persons who are interested in research of medical imaging using GAN.
In these slides, Generative Adversarial Network (GAN) is briefly introduced, and some GAN applications in medical imaging are presented. In the conclusions, some comments are given for persons who are interested in research of medical imaging using GAN.
In this talk, Dmitry shares his approach to feature engineering which he used successfully in various Kaggle competitions. He covers common techniques used to convert your features into numeric representation used by ML algorithms.
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
GANs are the new hottest topic in the ML arena; however, they present a challenge for the researchers and the engineers alike. Their design, and most importantly, the code implementation has been causing headaches to the ML practitioners, especially when moving to production.
Starting from the very basic of what a GAN is, passing trough Tensorflow implementation, using the most cutting-edge APIs available in the framework, and finally, production-ready serving at scale using Google Cloud ML Engine.
Slides for the talk: https://www.pycon.it/conference/talks/deep-diving-into-gans-form-theory-to-production
Github repo: https://github.com/zurutech/gans-from-theory-to-production
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIWithTheBest
This is how Generative Adversarial Networks (GANs) work and benefit the tech and dev industry. Although GANs still have room for improvement, GANs are important generative models that learn how to create realistic samples.
GANS
Ian Goodfellow, OpenAI Research Scientist
[Video recording available at https://www.youtube.com/playlist?list=PLewjn-vrZ7d3x0M4Uu_57oaJPRXkiS221]
Artificial Intelligence is increasingly playing an integral role in determining our day-to-day experiences. Moreover, with proliferation of AI based solutions in areas such as hiring, lending, criminal justice, healthcare, and education, the resulting personal and professional implications of AI are far-reaching. The dominant role played by AI models in these domains has led to a growing concern regarding potential bias in these models, and a demand for model transparency and interpretability. In addition, model explainability is a prerequisite for building trust and adoption of AI systems in high stakes domains requiring reliability and safety such as healthcare and automated transportation, and critical industrial applications with significant economic implications such as predictive maintenance, exploration of natural resources, and climate change modeling.
As a consequence, AI researchers and practitioners have focused their attention on explainable AI to help them better trust and understand models at scale. The challenges for the research community include (i) defining model explainability, (ii) formulating explainability tasks for understanding model behavior and developing solutions for these tasks, and finally (iii) designing measures for evaluating the performance of models in explainability tasks.
In this tutorial, we present an overview of model interpretability and explainability in AI, key regulations / laws, and techniques / tools for providing explainability as part of AI/ML systems. Then, we focus on the application of explainability techniques in industry, wherein we present practical challenges / guidelines for effectively using explainability techniques and lessons learned from deploying explainable models for several web-scale machine learning and data mining applications. We present case studies across different companies, spanning application domains such as search & recommendation systems, hiring, sales, and lending. Finally, based on our experiences in industry, we identify open problems and research directions for the data mining / machine learning community.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
Scaling Multinomial Logistic Regression via Hybrid ParallelismParameswaran Raman
Distributed algorithms in machine learning follow two main paradigms: data parallel, where the data is distributed across multiple workers and model parallel, where the model parameters are partitioned across multiple workers. The main limitation of the first approach is that the model parameters need to be replicated on every machine. This is problematic when the number of parameters is very large, and hence cannot fit in a single machine. The drawback of the latter approach is that the data needs to be replicated on each machine. Such replications limit the scalability of machine learning algorithms, since in several real-world tasks it is observed that the data and model sizes typically grow hand in hand. In this talk, I will present Hybrid-Parallelism, a new paradigm that partitions both, the data as well as the model parameters simultaneously in a completely de-centralized manner. As a result, each worker only needs access to a subset of the data and a subset of the parameters while performing parameter updates. Next, I will present a case-study showing how to apply these ideas to reformulate Multinomial Logistic Regression to achieve Hybrid Parallelism (DSMLR: Doubly-Separable Multinomial Logistic Regression). Finally, I will demonstrate the versatility of DS-MLR under various scenarios in data and model parallelism, through an empirical study consisting of real-world datasets.
In this talk, Dmitry shares his approach to feature engineering which he used successfully in various Kaggle competitions. He covers common techniques used to convert your features into numeric representation used by ML algorithms.
https://telecombcn-dl.github.io/2017-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.
GANs are the new hottest topic in the ML arena; however, they present a challenge for the researchers and the engineers alike. Their design, and most importantly, the code implementation has been causing headaches to the ML practitioners, especially when moving to production.
Starting from the very basic of what a GAN is, passing trough Tensorflow implementation, using the most cutting-edge APIs available in the framework, and finally, production-ready serving at scale using Google Cloud ML Engine.
Slides for the talk: https://www.pycon.it/conference/talks/deep-diving-into-gans-form-theory-to-production
Github repo: https://github.com/zurutech/gans-from-theory-to-production
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIWithTheBest
This is how Generative Adversarial Networks (GANs) work and benefit the tech and dev industry. Although GANs still have room for improvement, GANs are important generative models that learn how to create realistic samples.
GANS
Ian Goodfellow, OpenAI Research Scientist
[Video recording available at https://www.youtube.com/playlist?list=PLewjn-vrZ7d3x0M4Uu_57oaJPRXkiS221]
Artificial Intelligence is increasingly playing an integral role in determining our day-to-day experiences. Moreover, with proliferation of AI based solutions in areas such as hiring, lending, criminal justice, healthcare, and education, the resulting personal and professional implications of AI are far-reaching. The dominant role played by AI models in these domains has led to a growing concern regarding potential bias in these models, and a demand for model transparency and interpretability. In addition, model explainability is a prerequisite for building trust and adoption of AI systems in high stakes domains requiring reliability and safety such as healthcare and automated transportation, and critical industrial applications with significant economic implications such as predictive maintenance, exploration of natural resources, and climate change modeling.
As a consequence, AI researchers and practitioners have focused their attention on explainable AI to help them better trust and understand models at scale. The challenges for the research community include (i) defining model explainability, (ii) formulating explainability tasks for understanding model behavior and developing solutions for these tasks, and finally (iii) designing measures for evaluating the performance of models in explainability tasks.
In this tutorial, we present an overview of model interpretability and explainability in AI, key regulations / laws, and techniques / tools for providing explainability as part of AI/ML systems. Then, we focus on the application of explainability techniques in industry, wherein we present practical challenges / guidelines for effectively using explainability techniques and lessons learned from deploying explainable models for several web-scale machine learning and data mining applications. We present case studies across different companies, spanning application domains such as search & recommendation systems, hiring, sales, and lending. Finally, based on our experiences in industry, we identify open problems and research directions for the data mining / machine learning community.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
Scaling Multinomial Logistic Regression via Hybrid ParallelismParameswaran Raman
Distributed algorithms in machine learning follow two main paradigms: data parallel, where the data is distributed across multiple workers and model parallel, where the model parameters are partitioned across multiple workers. The main limitation of the first approach is that the model parameters need to be replicated on every machine. This is problematic when the number of parameters is very large, and hence cannot fit in a single machine. The drawback of the latter approach is that the data needs to be replicated on each machine. Such replications limit the scalability of machine learning algorithms, since in several real-world tasks it is observed that the data and model sizes typically grow hand in hand. In this talk, I will present Hybrid-Parallelism, a new paradigm that partitions both, the data as well as the model parameters simultaneously in a completely de-centralized manner. As a result, each worker only needs access to a subset of the data and a subset of the parameters while performing parameter updates. Next, I will present a case-study showing how to apply these ideas to reformulate Multinomial Logistic Regression to achieve Hybrid Parallelism (DSMLR: Doubly-Separable Multinomial Logistic Regression). Finally, I will demonstrate the versatility of DS-MLR under various scenarios in data and model parallelism, through an empirical study consisting of real-world datasets.
As optimization (or prescriptive analytics) has grown as a tool for business decision-making, a key factor in its success has been the adoption of model-based optimization. Using this approach, an analyst’s major work is to describe a problem of interest by means of an algebraic model, while the computation of a solution is left to general-purpose, off-the-shelf software. Powerful modeling systems manage the difficulties of translating between the human modeler’s ideas and the computer software’s needs. This tutorial introduces model-based optimization and offers a guide to its effective use.
It's a well-known fact that the best explanation of a simple model is the model itself. But often we use complex models, such as ensemble methods or deep networks, so we cannot use the original model as its own best explanation because it is not easy to understand.
In the context of this topic, we will discuss how methods for interpreting model predictions work and will try to understand practical value of these methods.
Using Graph Algorithms for Advanced Analytics - Part 5 ClassificationTigerGraph
What atmospheric data will help you predict if it's going to rain, snow, or be windy? What position should that new athlete play? How well can you guess a person's demographic background, based on their chat activity? These are all classification problems -- trying to pick the right category or label for an entity, based on observable features. They can also be solved with machine learning.
Graph Gurus Episode 32: Using Graph Algorithms for Advanced Analytics Part 5TigerGraph
Full Webinar: https://info.tigergraph.com/graph-gurus-32
By watching this webinar you will:
-See how similarity algorithms are used to calculate "distances" between entities.
-Learn what data scientists mean when they say Labels and Training.
-Understand the full workflow for the k-Nearest Neighbor machine learning technique, from computing distances to predicting labels for a given value of k, to learning the best value of k.
ABSTRACT: In the field of computer science known as "machine learning," a computer makes predictions about
the tasks it will perform next by examining the data that has been given to it. The computer can access data via
interacting with the environment or by using digitalized training sets. In contrast to static programming
algorithms, which require explicit human guidance, machine learning algorithms may learn from data and
generate predictions on their own. Various supervised and unsupervised strategies, including rule-based
techniques, logic-based techniques, instance-based techniques, and stochastic techniques, have been presented in
order to solve problems. Our paper's main goal is to present a comprehensive comparison of various cutting-edge
supervised machine learning techniques.
Show drafts
volume_up
Empowering the Data Analytics Ecosystem: A Laser Focus on Value
The data analytics ecosystem thrives when every component functions at its peak, unlocking the true potential of data. Here's a laser focus on key areas for an empowered ecosystem:
1. Democratize Access, Not Data:
Granular Access Controls: Provide users with self-service tools tailored to their specific needs, preventing data overload and misuse.
Data Catalogs: Implement robust data catalogs for easy discovery and understanding of available data sources.
2. Foster Collaboration with Clear Roles:
Data Mesh Architecture: Break down data silos by creating a distributed data ownership model with clear ownership and responsibilities.
Collaborative Workspaces: Utilize interactive platforms where data scientists, analysts, and domain experts can work seamlessly together.
3. Leverage Advanced Analytics Strategically:
AI-powered Automation: Automate repetitive tasks like data cleaning and feature engineering, freeing up data talent for higher-level analysis.
Right-Tool Selection: Strategically choose the most effective advanced analytics techniques (e.g., AI, ML) based on specific business problems.
4. Prioritize Data Quality with Automation:
Automated Data Validation: Implement automated data quality checks to identify and rectify errors at the source, minimizing downstream issues.
Data Lineage Tracking: Track the flow of data throughout the ecosystem, ensuring transparency and facilitating root cause analysis for errors.
5. Cultivate a Data-Driven Mindset:
Metrics-Driven Performance Management: Align KPIs and performance metrics with data-driven insights to ensure actionable decision making.
Data Storytelling Workshops: Equip stakeholders with the skills to translate complex data findings into compelling narratives that drive action.
Benefits of a Precise Ecosystem:
Sharpened Focus: Precise access and clear roles ensure everyone works with the most relevant data, maximizing efficiency.
Actionable Insights: Strategic analytics and automated quality checks lead to more reliable and actionable data insights.
Continuous Improvement: Data-driven performance management fosters a culture of learning and continuous improvement.
Sustainable Growth: Empowered by data, organizations can make informed decisions to drive sustainable growth and innovation.
By focusing on these precise actions, organizations can create an empowered data analytics ecosystem that delivers real value by driving data-driven decisions and maximizing the return on their data investment.
4. Supervised vs. Unsupervised Learning
• Supervised Learning
Data : (x, y)
x: data, y: label
Goal: Learn a function f to map x-> y
Tasks: Classification, Regression, Detection, etc
http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture13.pdf
28/04/2018 4
5. Supervised vs. Unsupervised Learning
• Unsupervised Learning
Data : only data x, no label y
Goal: Learn some underlying hidden structures of data
Tasks: clustering, dim reduction, density estimation, etc.
28/04/2018 5
6. Unsupervised Learning
• Taxonomy tree of unsupervised leanring
Source:
https://allenlu2007.wordpress.com/2018/01/10/variational-autoencoder-的原理/
28/04/2018 6
7. Generative models
• Goal:
Given training samples, generate new samples from the
same distribution
Training data ~ 𝑝"#$#(𝑥) Generated samples ~ 𝑝()"*+(𝑥)
In other words, try to learn a model 𝑝()"*+(𝑥) similar to 𝑝"#$#(𝑥)
28/04/2018 7
8. Generative models
• Maximum Likelihood Estimation (MLE):
Given training samples 𝑥,, 𝑥.,…, 𝑥/, how to learn 𝑝()"*+ 𝑥; 𝜃 from
which training samples are likely to be generated
𝜃∗
= 𝑎𝑟𝑔𝑚𝑎𝑥8 9 log
𝑝()"*+(𝑥>; 𝜃)
/
>?,
28/04/2018 8
11. Variational Autoencoder
• (probabilistic) generative model to generate samples from latent
variable.
• Assumption: training data {𝑥,, 𝑥.,…, 𝑥/} is generated from latent
variable 𝑧
Sample 𝑥~𝑝8(𝑥|𝑧)
Sample z~𝑝(𝑧)
Vary
z1
Vary
z2
Example:
Samples
x
are
face
images
Latent
z
is
2d
vector:
Z1:
head
orientation
Z2
:
degree
of
smile
28/04/2018 11
12. Variational Autoencoder
• How to learn the model?
MLE again !
𝜃∗
= 𝑎𝑟𝑔𝑚𝑎𝑥8 9 log 𝑝8(𝑥>)
/
>?,
Where 𝑝8 𝑥 = ∫ 𝑝8 𝑥 𝑧 𝑝 𝑧 𝑑𝑧E
-> intractable to compute
• Solution: Variational Approximation
28/04/2018 12
13. Variational Autoencoder
• Variational approximation
log 𝑝8(𝑥) can be written as the following formulation:
Likelihood
term
to
quantifyhow
good
the
sample
is
reconstructed
from
z.
This
can
be
estimated
by
a
network.
KL
divergence
term
to
estimate
the
difference
between
two
distribution
This
has
good
form
if
both
of
distributions
are
Gaussian-‐>
easy
to
estimate
This
KL
divergence
term
is
intractable
because
p(z|x)
cannot
computed.
But
it
is
aways >=
0
28/04/2018 13
14. Variational Autoencoder
• Variational approximation
log 𝑝8(𝑥) can be written as the following formulation:
Tractable
lower
bound
(ELBO)
28/04/2018 14
15. Variational Autoencoder
• Variational approximation
log 𝑝8(𝑥) can be written as the following formulation:
Tractable
lower
bound
(ELBO) Strategy:
• Maxmizing ELBO instead of intractable logp(x)
• What to be modeled:
1. 𝑝8(𝑥|𝑧) by a network (decoder)
2. 𝑞J 𝑥 𝑥 by another network (encoder)
28/04/2018 15
21. Generative Adversarial Network: Idea
Key
points:
q Belongs
to
“Implicit
density”
group
and
“hot”
method
in
ML
by
Goodfellow
q Motivated
by
game
theory
q Two
players:
1. Generator
tries
to
generate
“fake”
samples
from
its
model
2. Discriminator
tries
to
distinguish
“fake”
and
“real”
samples
28/04/2018 21
22. GAN: Two player game
Model:
q Generator
network:
try
to
fool
the
discriminator
by
generating
“like-‐‑real”
images
qDiscriminator
network:
try
to
distinguish
real
and
fake
samples
28/04/2018 22
23. GAN: Two player game
Objective
fucntion:
Loss
for
real
data
x Loss
for
fake
data
x
How
this
work
-‐‑ D
tries
to
maximize
the
cost
such
that
D(x)
close
to
1
(for
real
x)
and
D(G(z))
close
to
0
(fake)
-‐‑ G
tries
to
minimize
the
cost
such
that
D(G(z))
is
close
to
1
(try
to
make
generated
samples
real-‐‑looking
as
much
as
possible,
to
fool
D)
28/04/2018 23
24. GAN: Two player game
Objective
fucntion:
Loss
for
real
data
x Loss
for
fake
data
x
How
to
train:
alternative
approach
-‐‑ Fix
G,
D
maximize
the
cost
-‐‑ Fix
D,
G
minimize
the
cost
28/04/2018 24
25. GAN: density ratio estimation
Density
estimation
via
density
ratio
estimation:
28/04/2018 25