How might machine learning help advance solar PV research?
Final_Talk_Tool_Team
1. EMETTEUR - NOM DE LA PRESENTATION 00 MOIS 2011-
New Tools Team
Mehdi Lamee
Olorato Mosiane
Corinne Vassallo
Chedy Raissi
Mentor: Michael W. Busch
2. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Team New Tools
Mehdi Lamee
Phd candidate
University of Minnesota
Olorato Mosiane
Msc
North West University,
South Africa
Corinne Vassallo
Phd Candiate
University of Texas, Austin
Chedy Raïssi
Research Scientist
INRIA, France
3. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Motivation
• Asteroid Grand Challenge
-Find all asteroid threats to human populations and know what to do about them
Detect
Track
Characterize
Mitigate
Communicate
• Global effort
-Innovative solutions
-Accelerate the completion of the survey of main belt and potentially hazardous
asteroids
4. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Toutatis Itokawa Golevka
Motivation: Asteroid shape
modeling
• Planetary scientists need precise shape models and extra parameters to
compute precise orbits and plan safe robotic landings
Credits: Ian Webster,
http://www.asterank.com
5. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Existing Modeling Process
• Radar Imaging is a powerful technique to obtain information about solar system
bodies
- SHAPE modeling software uses radar and optical light curve observations to
produce 3D model of asteroids (Inverse problem).
SHAPE
inversion
software
1992UY4, Nick Duong.
?
6. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
SHAPE Software: An Overview
Generate
3D model
from radar
images
Radar
images
Project into
simulated radar
images
Compute
residuals
Vary Param
eters
7. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Motivation
• Existing Modeling Process: 1992 UY4
-processed by Nick Duong (REU Student @SETI)
Form
at
observation
files
1
day
Fitbasicm
odel
1
day
M
anualgrid
search
3
days
Fitspherical
harm
onicsm
odel
3
days
Fitting
vertices
m
odels
3
weeks
4 weeks!
8. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Context
• We wish to improve the speed and automation of asteroid shape modeling
-Two of the main time intensive steps:
1) The human-controlled manual grid search of spin axis orientations
2) Reliable and fast 3D shape generation (pre-process for SHAPE)
How to improve
the speed and
automation?
Spin axis
search
Fast 3D
shape
generation
Deep
Generative
models
Bayesian
Optimization
9. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Motivation
• Existing Modeling Process: 1992 UY4
-processed by Nick Duong (REU Student @SETI)
Form
at
observation
files
1
day
Fitbasicm
odel
1
day
M
anualgrid
search
3
days
Fitspherical
harm
onicsm
odel
3
days
Fitting
vertices
m
odels
3
weeks
{
Bayesian
Optimization
A few hours
{
Deep
Generative
Models
Inference: few seconds
1.5 week!
Refine
com
plex
vertexm
odel
1
week
10. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Context
How to improve
the speed and
automation?
Spin axis
search
Fast 3D
shape
generation
Deep
Generative
models
Bayesian
Optimization
11. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Bayesian Optimization: What are we
trying to find?
β
λ
Latitude
Longitude
12. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Bayesian Optimization
• Automate and reduce search area
• Global optimization rather than local
• Deals with unknown non-convex functions with no derivatives
13. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Bayesian Optimization
1. Start with some assumptions (prior) for the distribution of model parameters
2. Use the observations to predict a new distribution for the model parameters
(posterior)
3. Decide where to sample next in the parameter space to have the maximum
improvement in optimization
Acquisition function
4. Posterior becomes the new prior for the next iteration
14. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Bayesian Optimization
https://github.com/fmfn/BayesianOptimization
15. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Bayesian Optimization
16. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Results
• Teams’ Results: 2008 EV5
-Runtime: ~24 hours
- Best parameters
Longitude: 255º
Latitude: -75º
χ2: ~1.641
-Michael Busch parameters:
~3 days
Longitude: 270º
Latitude: -84º
χ2: ~1.678
17. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Results
• Teams’ Results: 1992 UY4
-Runtime: ~6 hours
- Best parameters
Longitude: 0º
Latitude: 180º
χ2: ~1.43
-Nick Duong’s parameters:
~3 days
Longitude: 15º
Latitude: 170º
χ2: ~1.60
1992UY4
18. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Results
• Teams’ Results: 2000 RS11
-Runtime: ~4 hours
-Best parameters:
Longitude: 270º
Latitude: 45º
χ2: ~0.74463
-Kaley Brauer parameters:
~3 days
Longitude: 245º
Latitude: 60º
χ2: ~0.745825
19. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Context
How to improve
the speed and
automation?
Spin axis
search
Fast 3D
shape
generation
Deep
Generative
models
Bayesian
Optimization
20. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• What is a deep learning model?
-A directed acyclic graph
Multiple processing units (nodes of the graph) with linear / non-linear transformations
-Model high-level abstractions of the data through processing layers
cs231n, Andrej Karpathy
• How to force the model to discover and efficiently internalize the essence
of the data in order to generate it?
-Use generative models
It works (somehow) for 2D images.
Unsupervised Representation Learning
with Deep Convolutional Generative
Adversarial Networks
[Radford et al, 2015]
Not real images!
21. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Reliable and fast 3D shape
generation
• Our idea:
-Develop deep generative models to output 3D asteroid shapes
-Recover these structures from radar images via probabilistic inference
-Recent work from Google DeepMind
Unsupervised Learning of 3D Structure from Images, [Rezende et al. 2016.]
-Using Voxels representation, Variational Auto-Encoder
-3D shape as output
22. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Reliable and fast 3D shape generation
• BUT:
-Deep models (deep neural networks)
Need large set of data to represent the vast amount of asteroid shapes and orbits
-Our dataset is limited to asteroids that have been observed (light curves and
radar imaging)
Clearly not enough (less than 1650 models LC + radar!)
• First step in our work
-Create synthetic data by matching previously modeled asteroids with orbits
from different asteroids
23. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Dataset Generation: Process
• Acquire asteroid shapes:
-1620 in total from JPL echo database (radar) and DAMIT database (light curve)
-Transform to voxels
• Pull ephemeris from JPL’s Horizons: viewed only from Arecibo Observatory
More than 60 different orbits
• Create fake observation files based on the orbits:
-1620 different shapes available
-Use different rotation angles
• Generation of synthetic radar images
-Noise addition
• Total synthetic radar images: ~546000
24. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• What is the general idea?
• We decided to train a Variational Auto-encoder
Training as an autoencoder
pθ (x z)pθ (z x)
Training use maximum likelihood
of p(x) given the training data
OpenAI blog
Latent variables
25. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• Training use maximum likelihood of p(x) given the training data
-Problem: intractable posterior distribution p_θ (z | x)
-Solution: sampling (MCMC) or approximate p(z|x) with q(z|x)
Training as an autoencoder
pθ (x z)pθ (z x)
Training use maximum likelihood
of p(x) given the training data
Problem:
Cannot be calculated:
Solution:
• MCMC (too costly)
• Approximate p(z|x) with q(z|x)
pθ (z x)
The complete auto-encoder
qϕ (x z) pθ (x z)
Learning the parameters φ and θ via backpropagation
Determining the loss function
The complete auto-encoder
qϕ (x z) pθ (x z)
Learning the parameters φ and θ via backpropagation
Determining the loss function
The complete auto-encoder
qϕ (x z) pθ (x z)
Learning the parameters φ and θ via backpropagation
Determining the loss function
{
Encoder {Decoder
26. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• Examples for 2D images
Variational auto encoders (idea of low dim manifold)
Kingma and Welling. “Auto-Encoding Variational Bayes, International Conference on Learning Representations.” ICLR, 2014. arXiv:1312.6114
Examples of successful unfolding (2D àR28x28, R20x26)
Frey Face dataset
2000 pictures of Brendan
Frey (20x26)
27. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• Asteroids generation
-Using 3D Convolutional neural networks
Roundness
Elongation
Condition the generation with
Radar
image
Get plausible 3D
shapes for asteroids!
28. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
The complete auto-encoder
qϕ (x z) pθ (x z)
Learning the parameters φ and θ via backpropagation
Determining the loss function
The complete auto-encoder
qϕ (x z) pθ (x z)
Learning the parameters φ and θ via backpropagation
Determining the loss function
The complete auto-encoder
qϕ (x z) pθ (x z)
Learning the parameters φ and θ via backpropagation
Determining the loss function
2D Convolutional NN + GRU
3D Convolutional VAE
At training timeAt inference time
29. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
3D Deep Generative Models
• Using Keras framework with Theano backend
-2 Nvidia TitanX used for training
• Preliminary results:
Input:
Output:
Real shape:
Object 2002 CE26 alphaObject 1992 SK
30. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Next Steps
• Bayesian Optimisation
-New metrics instead of Chi Square
Having a minimum Chi square does not guarantee a physically accurate shape
Work with Naidu Shantanu (NASA JPL) and Michael Busch (SETI)
-Investigate bayesian optimisation with spherical harmonics model
-Develop a full web user interface
Simply upload the radar images and metadata
• Deep Generative Networks
-Try with different generative models
DCGAN (Deep Convolutional Generative Adversarial Networks) [Radford et al, 2015]
PixelRNN (Pixel Recurrent Neural Networks) [Oord et al, 2016]
-Add more parameters in the input (ephemeris, baud length, bandwidth)
31. 00 MOIS 2011EMETTEUR - NOM DE LA PRESENTATION -
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
Next Steps
• Project sustainability
-Push the code and documentation as an open source project over
Scientific packaging
-Open Data:
Share the generated data (synthetic radar images and shapes)
Increase visibility of the project through a challenge?
Successful data science challenge: https://www.kaggle.com/c/higgs-boson)
32. EMETTEUR - NOM DE LA PRESENTATION 00 MOIS 2011-
CONTACT INFO
D E F E N S E
ACCELERATOR
PLANETARY
D E V E L O P M E N T
FRONTIER
LAB
THANK YOU
