Generative adversarial networks (GANs) are a class of machine learning frameworks where two neural networks compete against each other. One network generates synthetic data while the other evaluates it as real or fake. GANs have been applied to medical imaging tasks like generating additional patient data, translating between image modalities, enhancing image quality, and segmenting anatomical structures. Recent advances include conditioning GANs on text or labels to control image attributes, unpaired image-to-image translation using cycle consistency, and training a single GAN to handle multiple image domains. GANs show promise for improving diagnostic models by providing more training data and enabling new applications like noise reduction and accelerated acquisition.
In these slides, Generative Adversarial Network (GAN) is briefly introduced, and some GAN applications in medical imaging are presented. In the conclusions, some comments are given for persons who are interested in research of medical imaging using GAN.
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIWithTheBest
This is how Generative Adversarial Networks (GANs) work and benefit the tech and dev industry. Although GANs still have room for improvement, GANs are important generative models that learn how to create realistic samples.
GANS
Ian Goodfellow, OpenAI Research Scientist
Generative Adversarial Networks and Their ApplicationsArtifacia
This is the presentation from our AI Meet Jan 2017 on GANs and its applications.
You can join Artifacia AI Meet Bangalore Group: https://www.meetup.com/Artifacia-AI-Meet/
Generative Adversarial Networks is an advanced topic and requires a prior basic understanding of CNNs. Here is some pre-reading material for you.
- https://arxiv.org/pdf/1406.2661v1.pdf
- https://arxiv.org/pdf/1701.00160v1.pdf
basics of GAN neural network
GAN is a advanced tech in area of neural networks which will help to generate new data . This new data will be developed based over the past experiences and raw data.
In these slides, Generative Adversarial Network (GAN) is briefly introduced, and some GAN applications in medical imaging are presented. In the conclusions, some comments are given for persons who are interested in research of medical imaging using GAN.
Generative Adversarial Networks (GANs) - Ian Goodfellow, OpenAIWithTheBest
This is how Generative Adversarial Networks (GANs) work and benefit the tech and dev industry. Although GANs still have room for improvement, GANs are important generative models that learn how to create realistic samples.
GANS
Ian Goodfellow, OpenAI Research Scientist
Generative Adversarial Networks and Their ApplicationsArtifacia
This is the presentation from our AI Meet Jan 2017 on GANs and its applications.
You can join Artifacia AI Meet Bangalore Group: https://www.meetup.com/Artifacia-AI-Meet/
Generative Adversarial Networks is an advanced topic and requires a prior basic understanding of CNNs. Here is some pre-reading material for you.
- https://arxiv.org/pdf/1406.2661v1.pdf
- https://arxiv.org/pdf/1701.00160v1.pdf
basics of GAN neural network
GAN is a advanced tech in area of neural networks which will help to generate new data . This new data will be developed based over the past experiences and raw data.
GANs are the new hottest topic in the ML arena; however, they present a challenge for the researchers and the engineers alike. Their design, and most importantly, the code implementation has been causing headaches to the ML practitioners, especially when moving to production.
Starting from the very basic of what a GAN is, passing trough Tensorflow implementation, using the most cutting-edge APIs available in the framework, and finally, production-ready serving at scale using Google Cloud ML Engine.
Slides for the talk: https://www.pycon.it/conference/talks/deep-diving-into-gans-form-theory-to-production
Github repo: https://github.com/zurutech/gans-from-theory-to-production
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)Thomas da Silva Paula
A basic introduction to Generative Adversarial Networks, what they are, how they work, and why study them. This presentation shows what is their contribution to Machine Learning field and for which reason they have been considered one of the major breakthroughts in Machine Learning field.
발표자: 최윤제(고려대 석사과정)
최윤제 (Yunjey Choi)는 고려대학교에서 컴퓨터공학을 전공하였으며, 현재는 석사과정으로 Machine Learning을 공부하고 있는 학생이다. 코딩을 좋아하며 이해한 것을 다른 사람들에게 공유하는 것을 좋아한다. 1년 간 TensorFlow를 사용하여 Deep Learning을 공부하였고 현재는 PyTorch를 사용하여 Generative Adversarial Network를 공부하고 있다. TensorFlow로 여러 논문들을 구현, PyTorch Tutorial을 만들어 Github에 공개한 이력을 갖고 있다.
개요:
Generative Adversarial Network(GAN)은 2014년 Ian Goodfellow에 의해 처음으로 제안되었으며, 적대적 학습을 통해 실제 데이터의 분포를 추정하는 생성 모델입니다. 최근 들어 GAN은 가장 인기있는 연구 분야로 떠오르고 있고 하루에도 수 많은 관련 논문들이 쏟아져 나오고 있습니다.
수 없이 쏟아져 나오고 있는 GAN 논문들을 다 읽기가 힘드신가요? 괜찮습니다. 기본적인 GAN만 완벽하게 이해한다면 새로 나오는 논문들도 쉽게 이해할 수 있습니다.
이번 발표를 통해 제가 GAN에 대해 알고 있는 모든 것들을 전달해드리고자 합니다. GAN을 아예 모르시는 분들, GAN에 대한 이론적인 내용이 궁금하셨던 분들, GAN을 어떻게 활용할 수 있을지 궁금하셨던 분들이 발표를 들으면 좋을 것 같습니다.
발표영상: https://youtu.be/odpjk7_tGY0
발표자: 박태성 (UC Berkeley 박사과정)
발표일: 2017.6.
Taesung Park is a Ph.D. student at UC Berkeley in AI and computer vision, advised by Prof. Alexei Efros.
His research interest lies between computer vision and computational photography, such as generating realistic images or enhancing photo qualities. He received B.S. in mathematics and M.S. in computer science from Stanford University.
개요:
Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs.
However, for many tasks, paired training data will not be available.
We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples.
Our goal is to learn a mapping G: X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss.
Because this mapping is highly under-constrained, we couple it with an inverse mapping F: Y → X and introduce a cycle consistency loss to push F(G(X)) ≈ X (and vice versa).
Qualitative results are presented on several tasks where paired training data does not exist, including collection style transfer, object transfiguration, season transfer, photo enhancement, etc.
Quantitative comparisons against several prior methods demonstrate the superiority of our approach.
The detailed overview of the whole family of StyleGANs starting from the ProgressiveGAN to the latest StyleGAN3.
Such a continuous look at the StyleGAN improvements gives an excellent understanding of the research principles and approaches for improving your own models.
Unsupervised learning representation with Deep Convolutional Generative Adversarial Network, Paper by Alec Radford, Luke Metz, and Soumith Chintala
(indico Research, Facebook AI Research).
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Variational Autoencoders For Image GenerationJason Anderson
Meetup: https://www.meetup.com/Cognitive-Computing-Enthusiasts/events/260580395/
Video: https://www.youtube.com/watch?v=fnULFOyNZn8
Blog: http://www.compthree.com/blog/autoencoder/
Code: https://github.com/compthree/variational-autoencoder
An autoencoder is a machine learning algorithm that represents unlabeled high-dimensional data as points in a low-dimensional space. A variational autoencoder (VAE) is an autoencoder that represents unlabeled high-dimensional data as low-dimensional probability distributions. In addition to data compression, the randomness of the VAE algorithm gives it a second powerful feature: the ability to generate new data similar to its training data. For example, a VAE trained on images of faces can generate a compelling image of a new "fake" face. It can also map new features onto input data, such as glasses or a mustache onto the image of a face that initially lacks these features. In this talk, we will survey VAE model designs that use deep learning, and we will implement a basic VAE in TensorFlow. We will also demonstrate the encoding and generative capabilities of VAEs and discuss their industry applications.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...宏毅 李
Generative adversarial network (GAN) is a new idea for training models, in which a generator and a discriminator compete against each other to improve the generation quality. Recently, GAN has shown amazing results in image generation, and a large amount and a wide variety of new ideas, techniques, and applications have been developed based on it. Although there are only few successful cases, GAN has great potential to be applied to text and speech generations to overcome limitations in the conventional methods.
The tutorial includes three parts. The first part provides a thorough review of GAN. We will first introduce GAN to newcomers. Then, we will introduce the approaches that aim to improve the training procedure and the variants of GAN beyond simply generating random objects. The second part of this tutorial will focus on the applications of GAN on speech. Although most techniques related to GAN are developed on image generation today, GAN can also generate speech. However, speech signals are temporal sequences which have very different nature from images. We will describe how to apply GAN on speech signal processing, including text-to-speech synthesis, voice conversion, speech enhancement, and domain adversarial training on speech-related tasks. The third part of this tutorial will focus on the applications of GAN on natural language processing. The major challenge for applying GAN on natural language is its discrete nature (words are usually represented by one-hot encodings), which makes the original GAN fails. We will review a series of approaches dealing with this problem, and finally demonstrate the applications of GAN on chat-bot, abstractive summarization, and text style transformation.
GANs are the new hottest topic in the ML arena; however, they present a challenge for the researchers and the engineers alike. Their design, and most importantly, the code implementation has been causing headaches to the ML practitioners, especially when moving to production.
Starting from the very basic of what a GAN is, passing trough Tensorflow implementation, using the most cutting-edge APIs available in the framework, and finally, production-ready serving at scale using Google Cloud ML Engine.
Slides for the talk: https://www.pycon.it/conference/talks/deep-diving-into-gans-form-theory-to-production
Github repo: https://github.com/zurutech/gans-from-theory-to-production
A (Very) Gentle Introduction to Generative Adversarial Networks (a.k.a GANs)Thomas da Silva Paula
A basic introduction to Generative Adversarial Networks, what they are, how they work, and why study them. This presentation shows what is their contribution to Machine Learning field and for which reason they have been considered one of the major breakthroughts in Machine Learning field.
발표자: 최윤제(고려대 석사과정)
최윤제 (Yunjey Choi)는 고려대학교에서 컴퓨터공학을 전공하였으며, 현재는 석사과정으로 Machine Learning을 공부하고 있는 학생이다. 코딩을 좋아하며 이해한 것을 다른 사람들에게 공유하는 것을 좋아한다. 1년 간 TensorFlow를 사용하여 Deep Learning을 공부하였고 현재는 PyTorch를 사용하여 Generative Adversarial Network를 공부하고 있다. TensorFlow로 여러 논문들을 구현, PyTorch Tutorial을 만들어 Github에 공개한 이력을 갖고 있다.
개요:
Generative Adversarial Network(GAN)은 2014년 Ian Goodfellow에 의해 처음으로 제안되었으며, 적대적 학습을 통해 실제 데이터의 분포를 추정하는 생성 모델입니다. 최근 들어 GAN은 가장 인기있는 연구 분야로 떠오르고 있고 하루에도 수 많은 관련 논문들이 쏟아져 나오고 있습니다.
수 없이 쏟아져 나오고 있는 GAN 논문들을 다 읽기가 힘드신가요? 괜찮습니다. 기본적인 GAN만 완벽하게 이해한다면 새로 나오는 논문들도 쉽게 이해할 수 있습니다.
이번 발표를 통해 제가 GAN에 대해 알고 있는 모든 것들을 전달해드리고자 합니다. GAN을 아예 모르시는 분들, GAN에 대한 이론적인 내용이 궁금하셨던 분들, GAN을 어떻게 활용할 수 있을지 궁금하셨던 분들이 발표를 들으면 좋을 것 같습니다.
발표영상: https://youtu.be/odpjk7_tGY0
발표자: 박태성 (UC Berkeley 박사과정)
발표일: 2017.6.
Taesung Park is a Ph.D. student at UC Berkeley in AI and computer vision, advised by Prof. Alexei Efros.
His research interest lies between computer vision and computational photography, such as generating realistic images or enhancing photo qualities. He received B.S. in mathematics and M.S. in computer science from Stanford University.
개요:
Image-to-image translation is a class of vision and graphics problems where the goal is to learn the mapping between an input image and an output image using a training set of aligned image pairs.
However, for many tasks, paired training data will not be available.
We present an approach for learning to translate an image from a source domain X to a target domain Y in the absence of paired examples.
Our goal is to learn a mapping G: X → Y such that the distribution of images from G(X) is indistinguishable from the distribution Y using an adversarial loss.
Because this mapping is highly under-constrained, we couple it with an inverse mapping F: Y → X and introduce a cycle consistency loss to push F(G(X)) ≈ X (and vice versa).
Qualitative results are presented on several tasks where paired training data does not exist, including collection style transfer, object transfiguration, season transfer, photo enhancement, etc.
Quantitative comparisons against several prior methods demonstrate the superiority of our approach.
The detailed overview of the whole family of StyleGANs starting from the ProgressiveGAN to the latest StyleGAN3.
Such a continuous look at the StyleGAN improvements gives an excellent understanding of the research principles and approaches for improving your own models.
Unsupervised learning representation with Deep Convolutional Generative Adversarial Network, Paper by Alec Radford, Luke Metz, and Soumith Chintala
(indico Research, Facebook AI Research).
https://telecombcn-dl.github.io/2018-dlai/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks or Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles of deep learning from both an algorithmic and computational perspectives.
Variational Autoencoders For Image GenerationJason Anderson
Meetup: https://www.meetup.com/Cognitive-Computing-Enthusiasts/events/260580395/
Video: https://www.youtube.com/watch?v=fnULFOyNZn8
Blog: http://www.compthree.com/blog/autoencoder/
Code: https://github.com/compthree/variational-autoencoder
An autoencoder is a machine learning algorithm that represents unlabeled high-dimensional data as points in a low-dimensional space. A variational autoencoder (VAE) is an autoencoder that represents unlabeled high-dimensional data as low-dimensional probability distributions. In addition to data compression, the randomness of the VAE algorithm gives it a second powerful feature: the ability to generate new data similar to its training data. For example, a VAE trained on images of faces can generate a compelling image of a new "fake" face. It can also map new features onto input data, such as glasses or a mustache onto the image of a face that initially lacks these features. In this talk, we will survey VAE model designs that use deep learning, and we will implement a basic VAE in TensorFlow. We will also demonstrate the encoding and generative capabilities of VAEs and discuss their industry applications.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
http://imatge-upc.github.io/telecombcn-2016-dlcv/
Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of big annotated data and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which had been addressed until now with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or text captioning.
ICASSP 2018 Tutorial: Generative Adversarial Network and its Applications to ...宏毅 李
Generative adversarial network (GAN) is a new idea for training models, in which a generator and a discriminator compete against each other to improve the generation quality. Recently, GAN has shown amazing results in image generation, and a large amount and a wide variety of new ideas, techniques, and applications have been developed based on it. Although there are only few successful cases, GAN has great potential to be applied to text and speech generations to overcome limitations in the conventional methods.
The tutorial includes three parts. The first part provides a thorough review of GAN. We will first introduce GAN to newcomers. Then, we will introduce the approaches that aim to improve the training procedure and the variants of GAN beyond simply generating random objects. The second part of this tutorial will focus on the applications of GAN on speech. Although most techniques related to GAN are developed on image generation today, GAN can also generate speech. However, speech signals are temporal sequences which have very different nature from images. We will describe how to apply GAN on speech signal processing, including text-to-speech synthesis, voice conversion, speech enhancement, and domain adversarial training on speech-related tasks. The third part of this tutorial will focus on the applications of GAN on natural language processing. The major challenge for applying GAN on natural language is its discrete nature (words are usually represented by one-hot encodings), which makes the original GAN fails. We will review a series of approaches dealing with this problem, and finally demonstrate the applications of GAN on chat-bot, abstractive summarization, and text style transformation.
A short presentation on the current state of Generative Adversarial Networks. Some of the materials are borrowed from the ICCV 2017 tutorial on GANs. I have put a reference where applicable at the bottom of the slide.
Generative adversarial network and its applications to speech signal and natu...宏毅 李
Generative adversarial network (GAN) is a new idea for training models, in which a generator and a discriminator compete against each other to improve the generation quality. Recently, GAN has shown amazing results in image generation, and a large amount and a wide variety of new ideas, techniques, and applications have been developed based on it. Although there are only few successful cases, GAN has great potential to be applied to text and speech generations to overcome limitations in the conventional methods.
There are three parts in this tutorial. In the first part, we will give an introduction of generative adversarial network (GAN) and provide a thorough review about this technology. In the second part, we will focus on the applications of GAN to speech signal processing, including speech enhancement, voice conversion, speech synthesis, and the applications of domain adversarial training to speaker recognition and lip reading. In the third part, we will describe the major challenge of sentence generation by GAN and review a series of approaches dealing with the challenge. Meanwhile, we will present algorithms that use GAN to achieve text style transformation, machine translation and abstractive summarization without paired data.
Generative Adversarial Network and its Applications to Speech Processing an...宏毅 李
Generative adversarial network (GAN) is a new idea for training models, in which a generator and a discriminator compete against each other to improve the generation quality. Recently, GAN has shown amazing results in image generation, and a large amount and a wide variety of new ideas, techniques, and applications have been developed based on it. Although there are only few successful cases, GAN has great potential to be applied to text and speech generations to overcome limitations in the conventional methods.
There are three parts in this tutorial. In the first part, we will give an introduction of generative adversarial network (GAN) and provide a thorough review about this technology. In the second part, we will focus on the applications of GAN to speech signal processing, including speech enhancement, voice conversion, speech synthesis, and the applications of domain adversarial training to speaker recognition and lip reading. In the third part, we will describe the major challenge of sentence generation by GAN and review a series of approaches dealing with the challenge. Meanwhile, we will present algorithms that use GAN to achieve text style transformation, machine translation and abstractive summarization without paired data.
We describe a new training methodology for generative adversarial networks. The
key idea is to grow both the generator and discriminator progressively: starting
from a low resolution, we add new layers that model increasingly fine details as
training progresses. This both speeds the training up and greatly stabilizes it, al-
lowing us to produce images of unprecedented quality, e.g., CELEBA images at 1024
We also propose a simple way to increase the variation in generated images, and achieve a record inception score of 8 . 80 in unsupervised CIFAR10.
Additionally, we describe several implementation details that are important for
discouraging unhealthy competition between the generator and discriminator. Fi-
nally, we suggest a new metric for evaluating GAN results, both in terms of image
quality and variation. As an additional contribution, we construct a higher-quality
version of the CELEBA datase
ages, and achieve a record inception score of
8
.
80
in unsupervised CIFAR10.
Additionally, we describe several implementation details that are important for
discouraging unhealthy competition between the generator and discriminator. Fi-
nally, we suggest a new metric for evaluating GAN results, both in terms of image
quality and variation. As an additional contribution, we construct a higher-quality
version of the C
ELEB
A datase
This presentation is for my Seminar Course at the University of Tehran. in this presentation, I will introduce some of the newest and also exciting developments in Generative Adversarial Networks.
Automatic attendance system has two stages: Face recognition and face detection. Though there are many efficient algorithms for frontal face detection and recognition when large pose comes into picture most of the developed algorithm fails to detect. Here in this presentation I used two state-of-the-art results to make an automatic attendance system.
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Adjusting primitives for graph : SHORT REPORT / NOTESSubhajit Sahu
Graph algorithms, like PageRank Compressed Sparse Row (CSR) is an adjacency-list based graph representation that is
Multiply with different modes (map)
1. Performance of sequential execution based vs OpenMP based vector multiply.
2. Comparing various launch configs for CUDA based vector multiply.
Sum with different storage types (reduce)
1. Performance of vector element sum using float vs bfloat16 as the storage type.
Sum with different modes (reduce)
1. Performance of sequential execution based vs OpenMP based vector element sum.
2. Performance of memcpy vs in-place based CUDA based vector element sum.
3. Comparing various launch configs for CUDA based vector element sum (memcpy).
4. Comparing various launch configs for CUDA based vector element sum (in-place).
Sum with in-place strategies of CUDA mode (reduce)
1. Comparing various launch configs for CUDA based vector element sum (in-place).
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
3. ▪ Generative
–Can generate samples
▪ Adversarial
–Trained by competing each other
▪ Networks
–Use neural networks
Definition
Generative Adversarial Networks
4. ▪ Discriminative Models
–Given X, predict Y
–We learn P(Y | X) directly
–We cannot can generate samples
–Examples
• Logistic regression, SVM, CRF, Decision trees
▪ Generative models
–Given X, predict P(X|Y)
–We learn P(X, Y)
–We can generate samples from P(X)
–Examples
• Markov chains, Naïve Bayes, GMM
Discriminative Models vs Generative Models
Generative Models?
Richard Feynmann
5. ▪ Adversarial Training between Discriminator vs Generator
–Discriminator is a classifier that determines whether given images is real data from the
world or fake data generated by generator.
–Proposed by Ian Goodfellow(NIPS 2014)
Discriminator vs Generator
Adversarial Networks?
https://www.oreilly.com/learning/generative-adversarial-networks-for-beginners?imm_mid=0f6436&cmp=em-data-na-na-newsltr_ai_20170918
VS
6. ▪ Alternate the training of discriminator and generator until convergence(may not happen)
Overview
Training GAN
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
7. ▪ Fix the generator, then train discriminator to distinguish samples of real images from
samples generated by the generator
Discriminator Training
Training GAN
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
8. ▪ Fix the discriminator, then train generator from the feedback of discriminator using
samples generated by the generator
Generator Training
Training GAN
https://www.slideshare.net/xavigiro/deep-learning-for-computer-vision-generative-models-and-adversarial-training-upc-2016
9. ▪ Training GAN is a minmax problem where
–The discriminator D tries to maximize its classification accuracy
–The generator G tries to minimize the discriminator’s classification accuracy
–The optimal solution for D
–The optimal solution for G
Mathematical Formulation
Training GAN
Iam Goodfellow et. al, NIPS 2014
Maximized by D Minimized by G
(in practice, we maximize )
11. ▪ With the optimal discriminator, training GAN is equivalent to minimizing Jensen-Shannon
divergence as
What happens during the training of GAN?
Why GAN Works?
http://videolectures.net/site/normal_dl/tag=1129740/deeplearning2017_courville_generative_models_01.pdf
Christian Leidig et. al, CVPR 2017
the only solution is
12. Visualization of samples generated from trained Generator
Generating Samples from GAN
Iam Goodfellow et. al, NIPS 2014
18. ▪ Replace discriminator loss from binary cross entropy to least square loss
Least Square GAN(LS-GAN)
Improving Loss Function of GAN
Xudong Mao et. al, arXiv 2016
They don’t move.
0
1
19. ▪ Is JS divergence good enough to train GAN?
Wasserstein GAN(WGAN)
Improving Loss Function of GAN
Martin Arjovsky et. al, arXiv 2016
Continuous everywhere and differentiable almost everywhere
Vanishing gradient and bad convergence
20. ▪ WGAN objective function
▪ How can we efficiently enforce the Lipschitz
constraint on the critic D ?
–Weight clipping
Wasserstein GAN(WGAN)
Improving Loss Function of GAN
𝐷 𝑥1 − 𝐷 𝑥2 ≤ 𝐾 𝑥1 − 𝑥2 , K=1
Martin Arjovsky et. al, arXiv 2016
21. ▪ Learning curve and sample quality
Wasserstein GAN(WGAN)
Improving Loss Function of GAN
Vanilla GAN
WGAN
Martin Arjovsky et. al, arXiv 2016
22. ▪ Issues with Weight Clipping
–Fail to capture higher moments of the data distribution
–Either exploding or vanishing gradient
Wasserstein GAN with Gradient Penalty(WGAN-GP)
Improving Loss Function of GAN
Weight
Clipping
Gradient
Clipping
Ishaan Gulrajani et. al, NIPS 2017
23. ▪ WGAN with Gradient Penalty
–Penalize the norm of the gradient instead of clipping the weights of critics
Wasserstein GAN with Gradient Penalty(WGAN-GP)
Improving Loss Function of GAN
Ishaan Gulrajani et. al, NIPS 2017
24. ▪ Comparison of various loss functions and their sample quality
Wasserstein GAN with Gradient Penalty(WGAN-GP)
Improving Loss Function of GAN
Ishaan Gulrajani et. al, NIPS 2017
26. ▪ Conditioning the model on additional information for better multi-modal learning
Conditional GAN(CGAN)
Conditional Generation
Mirza et. al, arXiv 2014
https://github.com/hwalsuklee/tensorflow-generative-model-collections
27. ▪ Disentangle individual dimensions in latent vector for capturing key attributes
InfoGAN
Conditional Generation
Xi Chen et al arXiv 2016
https://github.com/hwalsuklee/tensorflow-generative-model-collections
29. ▪ Use both real/fake and label classifiers for discriminator training
Auxiliary Classifier GAN(AC-GAN)
Conditional Generation
Augustus Odena et al arXiv 2016
https://github.com/hwalsuklee/tensorflow-generative-model-collections
31. ▪ “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network”
–Perceptual Loss
GAN for Image Super Resolution
SRGAN
Chrisitan Leidig et al, CVPR 2017
32. ▪ Super Resolution Results
GAN for Image Super Resolution
SRGAN
Chrisitan Leidig et al, CVPR 2017
33. ▪ Super Resolution Results
GAN for Image Super Resolution
SRGAN
Chrisitan Leidig et al, CVPR 2017
34. ▪ “Image-to-Image Translation with Conditional Adversarial Networks”
Conditional GAN for Image Domain Transfer
Image-to-Image Translation
Phillip Isola et al, CVPR 2017
35. ▪ Image Translation Results
Conditional GAN for Image Domain Transfer
Image-to-Image Translation
Phillip Isola et al, CVPR 2017
36. ▪ Image Translation Results
Conditional GAN for Image Domain Transfer
Image-to-Image Translation
Phillip Isola et al, CVPR 2017
37. ▪ “Generative Adversarial Text to Image Synthesis”
–Generator is conditioned on text embedding
–Discriminator uses both visual and textual features by concatenation
Conditional GAN for Text to Image Translation
Text2Image
Scott Reed et al, ICML 2016
38. ▪ Text-to-Image Translation Results
Conditional GAN for Text to Image Translation
Text2Image
Scott Reed et al, ICML 2016
39. ▪ “Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks”
–Generate high-resolution images from text by stacking conditional GANs
Stacked Conditional GAN for Text to Image Translation
StackGAN
Han Zhang et al, ICCV 2017
40. ▪ Text-to-Image Translation Results
Stacked Conditional GAN for Text to Image Translation
StackGAN
Han Zhang et al, ICCV 2017
41. ▪ Text-to-Image Translation Results
Stacked Conditional GAN for Text to Image Translation
StackGAN
Han Zhang et al, ICCV 2017
42. ▪ “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”
Unpaired Image Translation
CycleGAN
Jun-Yan Zhu et al, ICCV 2017
43. ▪ Paired/Unpaired and Generated Image/Reconstruction
Unpaired Image Translation
CycleGAN
Jun-Yan Zhu et al, ICCV 2017
44. ▪ Image Translation Example
Unpaired Image Translation
CycleGAN
Jun-Yan Zhu et al, ICCV 2017
45. ▪ “StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation”
–Instead of training generators for each source-domain pair, we can train one generator handling
multiple domains to utilize all training dataset for all domain pairs and even for different dataset
Multi-domain Image-to-Image Translation with a Single Network
StarGAN
for real
for fake
Yunjey Choi et. al, CVPR 2018
46. ▪ Multi-domain Translation Result
Multi-domain Image-to-Image Translation with a Single Network
StarGAN
Yunjey Choi et. al, CVPR 2018
47. ▪ Use of ‘mask vector’ to jointly train multiple dataset
Multi-domain Image-to-Image Translation with a Single Network
StarGAN
Yunjey Choi et. al, CVPR 2018
48. ▪ Use of ‘mask vector’ to jointly train multiple dataset
Multi-domain Image-to-Image Translation with a Single Network
StarGAN
Yunjey Choi et. al, CVPR 2018
49. ▪ Multi-domain Translation Results
Multi-domain Image-to-Image Translation with a Single Network
StarGAN
Yunjey Choi et. al, CVPR 2018
50. ▪ “Learning from Simulated and Unsupervised Images through Adversarial Training”
Refining Simulated Image for Data Augmentation
SimGAN
Ashish Shrivastava et. al, CVPR 2017
Self-regularization loss
51. ▪ “Learning from Simulated and Unsupervised Images through Adversarial Training”
Refining Simulated Image for Data Augmentation
SimGAN
Ashish Shrivastava et. al, CVPR 2017
Visual Turing Test d=7 degree
53. –In many cases, data augmentation techniques used in natural images does not
semantically make sense in medical image
(flips, rotations, scale shifts, color shifts)
–Physically-plausible deformations or morphological transform can be used in limited
cases.
–More augmentation choices for texture classification problems.
Data Augmentation for Effective Training Set Expansion
Medical Data Generation
Source : H. R. Roth et. al., MICCAI, 2015
54. Generation of synthetic dataset for dataset expansion
Medical Data Generation
M.J.M. Chuquicusma, ISBI 2018
▪ “How To Fool Radiologists with Generative Adversarial Networks? A Visual Turing Test For
Lung Cancer Diagnosis”
55. Generation of synthetic dataset for dataset expansion
Medical Data Generation
M.J.M. Chuquicusma, ISBI 2018
▪ Visual Turing Test
56. ▪ “Synthetic Medical Images from Dual Generative Adversarial Networks”
Generation of Fundus Image using Dual GANs
Medical Data Generation
John T. Guibas et. al, NIPS Workshop 2017
57. ▪ “Synthetic Medical Images from Dual Generative Adversarial Networks”
Generation of Fundus Image using Dual GANs
Medical Data Generation
John T. Guibas et. al, NIPS Workshop 2017
58. ▪ “Synthetic Data Augmentation using GAN for Improved Liver Lesion Classification”
Liver Lesion Generation for Data Augmentation
Medical Data Generation
Maayan Frid-Ada et. al, ISBI 2018
Real Synthetic
59. ▪ “Generalization of Deep Neural Networks for Chest Pathology Classification in X-ray using
Generative Adversarial Networks”
Chest X-ray Generation for Data Augmentation
Medical Data Generation
Hojjat Salehinejad et. al, ICASSP 2018
60. ▪ Generated Samples and Performance Improvement by Synthetic Augmentation
Chest X-ray Generation for Data Augmentation
Medical Data Generation
Hojjat Salehinejad et. al, ICASSP 2018
61. ▪ “Visual Feature Attribution using Wasserstein GANs “
VA-GAN : Understanding Visual Feature Attribution for Alzheimer Disease
Visual Feature Attribution
Christian F. Baumgatrner et. al, arXiv 2018
Overall Objective
GAN Loss Term
RegularizationTerm
62. ▪ “Visual Feature Attribution using Wasserstein GANs “
VA-GAN : Understanding Visual Feature Attribution for Alzheimer Disease
Visual Feature Attribution
Christian F. Baumgatrner et. al, arXiv 2018
63. Enhance the quality of low-dose CT to normal-dose CT
Synthesis or Enhancement of Medical Image
Dong Nie et. al, MICCAI 2017
▪ “Medical Image Synthesis using Context-aware Generative Adversarial Networks”
64. Unpaired image translation from MR to CT using CycleGAN
Synthesis or Enhancement of Medical Image
Jelmer M. Wolterink et. al, MICCAI 2017 Workshop
Setting Mean Absolute Error
Paired Voxel-wise Loss 89.4 ± 6.8 HU
Unpaired Cycle Consistency Loss 73.7 ± 2.3 HU
▪ “Deep MR to CT Synthesis using Unpaired Data”
65. Enhance the quality of low-dose CT to normal-dose CT
Synthesis or Enhancement of Medical Image
J. M. Wolterink et. al, IEEE Trans. Medical Imaging
▪ “Generative Adversarial Networks for Noise Reduction in Low-Dose CT”
66. Deep learning for undersampled MRI reconstruction
Synthesis or Enhancement of Medical Image
Tran Min Quan et. al, arXiv 2018
▪ “Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss”
–Problem of compressed sensing MRI reconstruction
–CS-MRI using GAN objective
67. Deep learning for undersampled MRI reconstruction
Synthesis or Enhancement of Medical Image
Tran Min Quan et. al, arXiv 2018
▪ “Compressed Sensing MRI Reconstruction using a Generative Adversarial Network with a Cyclic Loss”
68. Deep learning for undersampled MRI reconstruction
Synthesis or Enhancement of Medical Image
Tran Min Quan et. al, arXiv 2018
▪ Experimental Results for Understampled MRI Reconstruction using GAN
69. Generation of segmentation mask undistinguishable from physician’s mask
Physician Friendly Loss for Segmentation
Source : P. Costa(2017), VUNO(2017)
▪ “Retinal Vessel Segmentation in Fundoscopic Images with Generative Adversarial Networks”
70. ▪ “Adversarial Networks for the Detection of Aggressive Prostate Cancer”
Generation of segmentation mask undistinguishable from physician’s mask
Physician Friendly Loss for Segmentation
Source : S. Kohl et. al(2017)
71. ▪ “SegAN : Adversarial Network with Multi-scale L1 Loss for Medical Image Segmentation”
SegAN : Adversarial Network with Multi-scale Loss
Physician Friendly Loss for Segmentation
Yuan Xue et. al, arXiv 2017
72. ▪ “Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-
Consistency Generative Adversarial Network”
Segmentation of Multimodal Images using Image-to-Image Translation
Multimodal Image Segmentation
Zizhao Zhang et. al, CVPR 2018
73. ▪ “Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-
Consistency Generative Adversarial Network”
Segmentation of Multimodal Images using Image-to-Image Translation
Multimodal Image Segmentation
Zizhao Zhang et. al, CVPR 2018
75. ▪ Image Generation
– Generation of rare cases
– Understanding latent structure of lesions
– Improving the performance of diagnostic models
▪ Image Synthesis and Translation
– Noise reduction, modality translation
– Accelerating image acquisition time
– Improving diagnostic performance
▪ Lesion Detection and Segmentation
– More physician friendly training
– Better performance for lesions or organs with complex structure
▪ Future of GANs in Medical Imaging
– More GANs to come in medical imaging with clinical and commercial values
More GANs in Medical Imaging
Future of GANs