The document discusses using machine learning techniques to classify astronomical objects from large surveys. It notes that surveys are producing huge amounts of data that conventional methods cannot fully process. Machine learning can be used to help classify objects and sort candidates. Specifically, the document discusses using machine learning on photometric data from the Sloan Digital Sky Survey (SDSS) to identify low-redshift quasars. It notes challenges including the large size and dimensionality of the data, and proposes using a boosted ensemble method to learn weights for different regions of feature space rather than trying to estimate probabilities. This would help classify objects from the SDSS into categories like quasars, stars or galaxies.

2.
Ninan Sajeeth PhilipNinan Sajeeth Philip
St. Thomas College, Kozhencherry

3.
Ninan Sajeeth PhilipNinan Sajeeth Philip
St. Thomas College, Kozhencherry

4.
I am visiting Sudhanshu Barway as part of the
joint South­Africa India bilateral project with
SAAO for developing Virtual Observatory
tools for SALT.

5.
Machine Learning Challenges in
Astronomy
Ninan Sajeeth Philip
nspp@iucaa.ernet.in
http://www.iucaa.ernet.in/~nspp

6.
Machine Learning
● Objective : Mimic human ability to learn and make
decisions.
● Need : Surveys are producing huge data and
conventional methods are insufficient to process
them.
Eg: SDSS survey could spectroscopically confirm the nature of
only less than 1% of its photometric detections.
● Limitation : The goodness of the outputs depend
on the discriminative power of the inputs.
● Advantage : First level sorting of candidates.

7.
Machine Learning
● Objective : Mimic human ability to learn and make
decisions.
● Need : Surveys are producing huge data and
conventional methods are insufficient to process
them.
Eg: SDSS survey could spectroscopically confirm the nature of
only less than 1% of its photometric detections.
● Limitation : The goodness of the outputs depend
on the discriminative power of the inputs.
● Advantage : First level sorting of candidates.

8.
Machine Learning
● Objective : Mimic human ability to learn and make
decisions.
● Need : Surveys are producing huge data and
conventional methods are insufficient to process
them.
Eg: SDSS survey could spectroscopically confirm the nature of
only less than 1% of its photometric detections.
● Limitation : The goodness of the outputs depend
on the discriminative power of the inputs.
● Advantage : First level sorting of candidates.

9.
Machine Learning
● Objective : Mimic human ability to learn and make
decisions.
● Need : Surveys are producing huge data and
conventional methods are insufficient to process
them.
Eg: SDSS survey could spectroscopically confirm the nature of
only less than 1% of its photometric detections.
● Limitation : The goodness of the outputs depend
on the discriminative power of the inputs.
● Advantage : First level sorting of candidates.

10.
Machine Learning
● Objective : Mimic human ability to learn and make
decisions.
● Need : Surveys are producing huge data and
conventional methods are insufficient to process
them.
Eg: SDSS survey could spectroscopically confirm the nature of
only less than 1% of its photometric detections.
● Limitation : The goodness of the outputs depend
on the discriminative power of the inputs.
● Advantage : First level sorting of candidates.

11.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

12.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

13.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

14.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

15.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

16.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

17.
Large Data Issues
● Multidimensional data – requires more memory
● Diversity in the data – rare to rich populations
● Overlapping features – observational limitations
● Missing values – observational limitations
● Uncertainties – inherent and observational
● Processing Power – silicon limitations
● Storage and retrieval – bandwidth limitations

18.
Different Machine Learning Methods
● All methods assume that the different types
(classes) are separable in the feature space.
Light profile of galaxies are different from that of stars

19.
A Real Example
Composed of about a
million points showing
clustering of Quasars
(blue and red), main
sequence stars (green),
late type stars (yellow)
and unresolved galaxies
(pink) in a colour ­colour
plot of SDSS colours.
SDSS
colour­colour plot

20.
A Real Example
Composed of about a
million points showing
clustering of Quasars
(blue and red), main
sequence stars (green),
late type stars (yellow)
and unresolved galaxies
(pink) in a colour ­colour
plot of SDSS colours.
Blue are low redshift
Quasars and our goal is
to identify them and
verify whether the actual
number count match
with the estimated
values.

21.
A Real Example Blue are low redshift
Quasars and our goal is
to identify them and
verify whether the actual
number count match
with the estimated
values.
The region in the box
has about 150,000
confirmed observations
and about 6 million
unconfirmed cases.

22.
A Real Example Blue are low redshift
Quasars and our goal is
to identify them and
verify whether the actual
number count match
with the estimated
values.
The region in the box
has about 150,000
confirmed observations
and about 6 million
unconfirmed cases.
All objects have known
colours – only their
classification is
unknown.

23.
Bayesian Model

24.
Feature Space
● SDSS provides 5 magnitudes for each object in
bands u, g, r, i and z that can be used to
construct a ten dimensional colour space.
● A subset of the 150,000 objects with confirmed
spectroscopic classification can be used to
estimate the likelihood.
● The classifier can be tested on remaining data
to verify the accuracy of the model.

25.
Feature Space
● SDSS provides 5 magnitudes for each object in
bands u, g, r, i and z that can be used to
construct a ten dimensional colour space.
● A subset of the 150,000 objects with confirmed
spectroscopic classification can be used to
estimate the likelihood.
● The classifier can be tested on remaining data
to verify the accuracy of the model.
The distribution is not smooth

26.
Feature Space
● SDSS provides 5 magnitudes for each object in
bands u, g, r, i and z that can be used to construct a
ten dimensional colour space.
● A subset of the 150,000 objects with confirmed
spectroscopic classification can be used to estimate
the likelihood.
The colour space need to be binned to approximate
the distribution.
● The classifier can be tested on remaining data to
verify the accuracy of the model.
The distribution is not smooth

27.
Feature Space
● SDSS provides 5 magnitudes for each object in bands u,
g, r, i and z that can be used to construct a ten
dimensional colour space.
● A subset of the 150,000 objects with confirmed
spectroscopic classification can be used to estimate the
likelihood.
The colour space need to be binned to approximate the
distribution. Computing conditional likelihood of the binned
high dimensional feature space is nearly impossible.
● The classifier can be tested on remaining data to verify the
accuracy of the model.
The distribution is not smooth

28.
Two issues with Bayesian Formalism
● How would you guess the True value of the
Prior for each bin?
● Conditional dependency of the input feature
space – likelihood is conditionally dependent on
input values ­ Naive Bayesian models fail on
even simple XOR problems.

29.
Bayesian Methods
● Ensemble methods: Multiple models, same data : many
weak learners combined to form a strong learning model
● Bagging: each model in ensemble vote for the probable
candidate
● Boosting: Emphasise the failing models with weights
● Bayesian Model Averaging (BMA): Sampling
Hypothesis from Hypothesis Space
● Bayesian Model Combination (BMC): Seek
combination of models closest to a distribution.

30.
Bayesian Methods
● Ensemble methods: Multiple models, same data
● Bagging: each model in ensemble vote for the
probable candidate
● Boosting: Emphasise the failing models with
weights
● Bayesian Model Averaging (BMA): Sampling
Hypothesis from Hypothesis Space
● Bayesian Model Combination (BMC): Seek
combination of models closest to a distribution.

31.
Bayesian Methods
● Ensemble methods: Multiple models, same data
● Bagging: each model in ensemble vote for the
probable candidate
● Boosting: Emphasise the failing models with
weights
● Bayesian Model Averaging (BMA): Sampling
Hypothesis from Hypothesis Space
● Bayesian Model Combination (BMC): Seek
combination of models closest to a distribution.

32.
Bayesian Methods
● Ensemble methods: Multiple models, same data
● Bagging: each model in ensemble vote for the
probable candidate
● Boosting: Emphasise the failing models with
weights
● Bayesian Model Averaging (BMA): Sampling
Hypothesis from Hypothesis Space
● Bayesian Model Combination (BMC): Seek
combination of models closest to a distribution.

33.
Bayesian Methods
● Ensemble methods: Multiple models, same data
● Bagging: each model in ensemble vote for the
probable candidate
● Boosting: Emphasise the failing models with
weights
● Bayesian Model Averaging (BMA): Sampling
Hypothesis from Hypothesis Space
● Bayesian Model Combination (BMC): Seek a
combination of models closest to a distribution.

34.
Our Solution
● Estimating both Prior and Likelihood from data.
● Boosting: Emphasise the failing models with
weights
Can Prior be replaced with weights for each
range (bin) of input feature values within the
same model?

35.
Replacing Prior with weights
● Partition the input feature space into M bins – the bins the bins
can be centred around clusters or simple uniform binning.can be centred around clusters or simple uniform binning.
●
Assign uniform small prior/weight to all the bins.Assign uniform small prior/weight to all the bins.
●
Compute Bayesian Posterior Probability based on Compute Bayesian Posterior Probability based on
input features in the training data and identify failed input features in the training data and identify failed
instances.instances.
●
Update weights associated with the input feature Update weights associated with the input feature
bins corresponding to failed cases by A bins corresponding to failed cases by A x x (1­P/P*) (1­P/P*)
where A is a learning constant and P and P* are BP where A is a learning constant and P and P* are BP
for failed and target outcomes respectively.for failed and target outcomes respectively.

36.
Replacing Prior with weights
● Partition the input feature space into M bins – the bins the bins
can be centred around clusters or simple uniform binning.can be centred around clusters or simple uniform binning.
●
Assign Assign uniformuniform small prior/weight to all the bins. small prior/weight to all the bins.
●
Compute Bayesian Posterior Probability based on Compute Bayesian Posterior Probability based on
input features in the training data and identify failed input features in the training data and identify failed
instances.instances.
●
Update weights associated with the input feature Update weights associated with the input feature
bins corresponding to failed cases by A bins corresponding to failed cases by A x x (1­P/P*) (1­P/P*)
where A is a learning constant and P and P* are BP where A is a learning constant and P and P* are BP
for failed and target outcomes respectively.for failed and target outcomes respectively.

37.
Replacing Prior with weights
● Partition the input feature space into M bins – the bins the bins
can be centred around clusters or simple uniform binning.can be centred around clusters or simple uniform binning.
●
Assign Assign uniformuniform small prior/weight to all the bins. small prior/weight to all the bins.
●
Compute Bayesian Posterior Probability based on Compute Bayesian Posterior Probability based on
input features in the training data and input features in the training data and identify failed identify failed
instances.instances.
●
Update weights associated with the input feature Update weights associated with the input feature
bins corresponding to failed cases by A bins corresponding to failed cases by A x x (1­P/P*) (1­P/P*)
where A is a learning constant and P and P* are BP where A is a learning constant and P and P* are BP
for failed and target outcomes respectively.for failed and target outcomes respectively.

38.
Replacing Prior with weights
● Partition the input feature space into M bins – the bins the bins
can be centred around clusters or simple uniform binning.can be centred around clusters or simple uniform binning.
●
Assign Assign uniformuniform small prior/weight to all the bins. small prior/weight to all the bins.
●
Compute Bayesian Posterior Probability based on Compute Bayesian Posterior Probability based on
input features in the training data and input features in the training data and identify failed identify failed
instances.instances.
●
Update weights associated with the input feature Update weights associated with the input feature
bins corresponding to failed cases by bins corresponding to failed cases by A A x x (1­P/P*)(1­P/P*)
where where AA is a learning constant and is a learning constant and PP and and P*P* are BP are BP
for failed and target outcomes respectively.for failed and target outcomes respectively.

39.
Replacing Prior with weights
● Partition the input feature space into M bins – the bins the bins
can be centred around clusters or simple uniform binning.can be centred around clusters or simple uniform binning.
●
Assign Assign uniformuniform small prior/weight to all the bins. small prior/weight to all the bins.
●
Compute Bayesian Posterior Probability based on Compute Bayesian Posterior Probability based on
input features in the training data and input features in the training data and identify failed identify failed
instances.instances.
●
Update weights associated with the input feature Update weights associated with the input feature
bins corresponding to failed cases by bins corresponding to failed cases by A A x x (1­P/P*)(1­P/P*)
where A is a learning constant and P and P* are BP where A is a learning constant and P and P* are BP
for failed and target outcomes respectively. for failed and target outcomes respectively. Since the Since the
update is based on probability, outliers do not cause an issue.update is based on probability, outliers do not cause an issue.

40.
Likelihood estimation of Binned Space
● Likelihood estimation becomes an issue because
we want to know the conditional likelihood of the
binned feature space. There may not be sufficient
samples in each bin to estimate likelihood when
conditional dependence constrains are imposed on
them.
● We adopted an imposed conditional independence
formula that approximate the likelihood for a
conditionally dependent event as the product of the
likelihood for pairs of input features.

41.
Likelihood estimation of Binned Space
● Likelihood estimation becomes an issue because
we want to know the conditional likelihood of the
binned feature space. There may not be sufficient
samples to estimate likelihood when constrains on
conditional dependence is imposed on them.
● We adopted an imposed conditional independence
formula that approximate the likelihood for a
conditionally dependent event as the product of the
likelihood for pairs of input features.

42.
Imposed Conditional Independence
The likelihood for a conditionally dependent event A can
be approximated as the product of the likelihood of paired
input features.
● L(A|b,c,d,e,f) ~
M*L(A|b,c)* L(A|b,d)* L(A|b,e)* L(A|b,f)* L(A|c,d) *L(A|c,e)*
L(A|c,f)* L(A|d,e)* L(A|d,f)* L(A|e,f)
● Works better than Naive Bayes – no issue with XOR gate

43.
Imposed Conditional Independence
The likelihood for a conditionally dependent event A can
be approximated as the product of the likelihood of its
paired inputs.
● L(A|b,c,d,e,f) ~
M*L(A|b,c)* L(A|b,d)* L(A|b,e)* L(A|b,f)* L(A|c,d) *L(A|c,e)*
L(A|c,f)* L(A|d,e)* L(A|d,f)* L(A|e,f)
● Works better than Naive Bayes – no issue with XOR gate

44.
Classification of the 6 million Objects
Blue are Quasars, Yellow are unresolved Galaxies and Green are
main sequence Stars

45.
Verification of Predictions

46.
Comparison with expected number counts

47.
Further Information

48.
The Predicted Catalogue

49.
A more complex situation
● What if all input features are not known?
Straightforward solution : Compute the inverse
probability for the missing feature just as you
handle missing values.
Not so easy situation: What if we do not have a
training data with all features for computing inverse
probability?

50.
A more complex situation
● What if all input features are not known?
Straightforward solution : Compute the inverse
probability for the missing feature just as you
handle missing values.
Not so easy situation: What if we do not have a
training data with all features for computing inverse
probability?

51.
A more complex situation
● What if all input features are not known?
Straightforward solution : Compute the inverse
probability for the missing feature just as you
handle missing values.
Not so easy situation: What if we do not have a
training data with all features for computing inverse
probability?

52.
A Challenging Problem

53.
A Challenging Problem
● Generate alerts on optical transient detections
● Minimize false alarms
● Customize alarms to user demands
● Send the alarms immediately – given minimal or
sometime very little information about it.
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier detections

54.
A Challenging Problem
● Generate alerts on optical transient detections
● Minimize false alarms
● Customize alarms to user demands
● Send the alarms immediately – given minimal or
sometime very little information about it.
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier detections

55.
A Challenging Problem
● Generate alerts on optical transient detections
● Minimize false alarms
● Customize alerts to user demands
● Send the alarms immediately – given minimal
or sometime very little information about it.
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier
detections

56.
A Challenging Problem
● Generate alerts on optical transient detections
● Minimize false alarms
● Customize alarms to user demands
● Send the alerts immediately – given minimal or
sometime very little information about it.
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier
detections

57.
A Challenging Problem
● Generate alerts on optical transient detections
● Minimize false alarms
● Customize alarms to user demands
● Send the alerts immediately – given minimal or
sometime very little information about it.
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier
detections

58.
Missing Values
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier detections
Possible only if the object is within the foot print
of a survey
Each survey may use a different unit for their
catalogues – need to be considered separately

59.
Missing Values
Example : Nearest distance to a galaxy or star
: Distance to nearest known radio object
: Distance to nearest known x­ray detections
: Magnitudes in archives and in earlier detections
Possible only if the object is within the foot print
of a survey
Each survey may use a different unit for their
catalogues – need to be considered separately

60.
Missing Data Values
The training data itself has missing data values.
Note: The accuracy of the actual observation is not beyond one or two decimal
places. The double precision is used here only to reduce round off error while
rescaling the data during the processing.

61.
Missing Values
No way to compute inverse probability­ makes
No way to compute inverse probability­ makes
it impossible for standard machine learning
it impossible for standard machine learning
algorithms to learn and predict the outcome
algorithms to learn and predict the outcome

62.
Our Approach
The likelihood for a conditionally dependent event A
can be approximated as the product of the likelihood
of its paired inputs.
● L(A|b,c,d,e,f) ~
M*L(A|b,c)* L(A|b,d)* L(A|b,e)* L(A|b,f)* L(A|c,d)
*L(A|c,e)* L(A|c,f)* L(A|d,e)* L(A|d,f)* L(A|e,f)

63.
Our Approach
The likelihood for a conditionally dependent event A
can be approximated as the product of the likelihood
of its paired inputs.
● L(A|b,c,d,e,f) ~
M*L(A|b,c)* L(A|b,d)* L(A|b,e)* L(A|b,f)* L(A|c,d)
*L(A|c,e)* L(A|c,f)* L(A|d,e)* L(A|d,f)* L(A|e,f)

64.
Our Approach
The likelihood for a conditionally dependent event A can
be approximated as the product of the likelihood of its
paired inputs.
● L(A|b,c,d,e,f) ~
M*L(A|b,c)* L(A|b,d)* L(A|b,e)* L(A|b,f)* L(A|c,d) *L(A|c,e)*
L(A|c,f)* L(A|d,e)* L(A|d,f)* L(A|e,f)
● Estimate approximate Likelihood based on whatever
information available and use it for training and testing.

65.
Approximate Likelihood
● Since we do not have the luxury to decide the
input features, we go for a greedy collection of
what all information – input data – that can be
collected to compute the approximate likelihood.
● It is expected (assumed) that the redundancy in
the available information will help us to
approximate the likelihood to some reasonable
accuracy.

66.
Approximate Likelihood
● Since we do not have the luxury to decide the
input features, we go for a greedy collection of
what all information – input data – that can be
collected to compute the approximate likelihood.
● It is expected (assumed) that the redundancy in
the available information will help us to
approximate the likelihood to some reasonable
accuracy.

67.
Approximate Likelihood
● Since we do not have the luxury to decide the
input features, we go for a greedy collection of
what all information – input data – that can be
collected to compute the approximate likelihood.
● It is expected (assumed) that the redundancy in
the available information will help us to
approximate the likelihood to some reasonable
accuracy.
More like a forensic investigationMore like a forensic investigation

68.
What about Prior?
The likelihood for a conditionally dependent
event A is approximated to the product of the
likelihood of paired input features.
The prior is to be determined from the data
Uses a gradient descent algorithm to determine
the prior from the data

69.
What about Prior?
The likelihood for a conditionally dependent event
A can is approximated to the product of the
likelihood of paired input features.
The prior is to be determined from the data with
missing data
We use a gradient descent algorithm to determine
the prior from the data

70.
What about Prior?
The likelihood for a conditionally dependent event A is
approximated as the product of the likelihood of paired
input features.
The prior is to be determined from the data with missing
data
We use a gradient descent algorithm to determine the
prior – weights ­ from the data, similar to boosting.

71.
Dynamic Learning
● With lot of missing values in the observations,
each input data has partial information about
the features associated to an outcome.
● Learn as we go... use Bayesian update rule to
update the belief in each input feature and its
consequences.

72.
Dynamic Learning
● With lot of missing values in the observations,
each input data has partial information about
the features associated to an outcome.
● Learn as we go... use Bayesian update rule to
update the belief in each input feature and its
effect on the outcome.

73.
Dynamic Addition of Features
● We want to use all available information about
the detections as and when they become
available.
● Since likelihood is computed as the product, it is
feasible to update it with new evidences as and
when they become available.

74.
Dynamic Addition of Features
● We want to use all available information about
the detections
● Since likelihood is computed as the product, it is
feasible to update it with new evidences as and
when they become available.

75.
Dreaming Computers
● We now have many input features but not so
many examples to learn from. This can lead to
over­fitting the data and Memorising rather than
generalising the situation.
● Dreams are synthetic inputs our brain uses to
teach us how to react to plausible situations.
Can we create dreams for computers?

76.
Dreaming Computers
● We now have many input features but not so
many examples to learn from. This can lead to
over­fitting the data and Memorising rather than
generalising the situation.
● Dreams are synthetic inputs our brain uses to
teach us how to react to plausible situations.
Can we create dreams for computers?

77.
Information from Error Bars
● Can error bars give additional information?
● Error bars tell us that the nature of the object
remains same even if the measurement value is
varied within the range of the error bar – can be
used to generate new data

78.
Information from Error Bars
● Can error bars give additional information?
● Error bars tell us that the nature of the object
remains same even if the measurement value is
varied within the range of the error bar – can be
used to generate new data

79.
DBNN
● The algorithm described so far is the core
design concept of the Difference Boosting
Neural Network or DBNN algorithm.
● It is GNU public and the source code can be
downloaded from
http://www.iucaa.ernet.in/~nspp/dbnn.html

80.
DBNN Annotator
A collaborative project with Ashish Mahabal A collaborative project with Ashish Mahabal
(Caltech), IUCAA, Pune and the CRTS Team (Caltech), IUCAA, Pune and the CRTS Team
with funding from IUSSTF and ISRO.with funding from IUSSTF and ISRO.

81.
CRTS Predictions
1, "Cataclysmic Variable"
2 "Supernova"
3 "other"
5 "Blazar Outburst"
6 "AGN Variability"
7 "UVCeti Variable"
8 "Asteroid"
9 "Variable"
10 "Mira Variable"
11 "High Proper Motion Star"
12 "Comet"

82.
CRTS Predictions
[1] [2] [3] [5] [6] [7] [8] [9] [10] [11] [12] [16] Total
[1] 273 3 4 1 1 0 1 3 3 3 0 1 293
[2] 4 402 3 0 4 1 3 2 0 1 1 0 421
[3] 0 0 34 0 0 0 0 0 0 0 0 0 34
[5] 0 0 0 60 0 0 0 0 1 0 0 0 61
[6] 0 0 0 0 126 0 0 0 0 0 0 0 126
[7] 0 0 0 0 0 32 0 0 0 0 0 0 32
[8] 0 0 0 0 0 0 6 0 0 0 0 0 6
[9] 0 0 0 0 0 0 0 18 0 0 0 0 18
[10] 0 0 0 0 0 0 0 0 12 0 0 0 12
[11] 0 0 0 0 0 0 0 0 0 43 0 0 43
[12] 0 0 0 0 0 0 0 0 0 0 5 0 5
[16] 0 0 0 0 0 0 0 0 0 0 0 1 1
_________________________________________________________
Total 277 405 41 61 131 33 10 23 16 47 6 2 1052
1,"Cataclysmic Variable"
2,"Supernova"
3,"other"
5,"Blazar Outburst"
6,"AGN Variability"
7,"UVCeti Variable"
8,"Asteroid"
9,"Variable"
10,"Mira Variable"
11,"High Proper Motion Star"
12,"Comet"

83.
CRTS Predictions
[1] [2] [3] [5] [6] [7] [8] [9] [10] [11] [12] [16] Total
[1] 273 3 4 1 1 0 1 3 3 3 0 1 293
[2] 4 402 3 0 4 1 3 2 0 1 1 0 421
[3] 0 0 34 0 0 0 0 0 0 0 0 0 34
[5] 0 0 0 60 0 0 0 0 1 0 0 0 61
[6] 0 0 0 0 126 0 0 0 0 0 0 0 126
[7] 0 0 0 0 0 32 0 0 0 0 0 0 32
[8] 0 0 0 0 0 0 6 0 0 0 0 0 6
[9] 0 0 0 0 0 0 0 18 0 0 0 0 18
[10] 0 0 0 0 0 0 0 0 12 0 0 0 12
[11] 0 0 0 0 0 0 0 0 0 43 0 0 43
[12] 0 0 0 0 0 0 0 0 0 0 5 0 5
[16] 0 0 0 0 0 0 0 0 0 0 0 1 1
_________________________________________________________
Total 277 405 41 61 131 33 10 23 16 47 6 2 1052
1,"Cataclysmic Variable"
2,"Supernova"
3,"other"
5,"Blazar Outburst"
6,"AGN Variability"
7,"UVCeti Variable"
8,"Asteroid"
9,"Variable"
10,"Mira Variable"
11,"High Proper Motion Star"
12,"Comet"
Recall 273/277 =98.5%→
False Alarms (293­273)/293 = 7%→

84.
Parallel DBNN
● Since DBNN split likelihood as the product of
individual pairs, computation of likelihood may
be independently carried out by a different node
in a HPC system.
● Broadcast likelihoods to nodes
● Compute
● Gather Bayesian belief for each outcome

85.
Parallel DBNN
● Since DBNN split likelihood as the product of
individual pairs, computation of likelihood may
be independently carried out by a different node
in a HPC system.
● Broadcast likelihoods to nodes
● Compute
● Gather Bayesian belief for each outcome

86.
Parallel DBNN
● Since DBNN split likelihood as the product of
individual pairs, computation of likelihood may
be independently carried out by a different node
in a HPC system.
● Broadcast likelihoods to nodes
● Compute
● Gather Bayesian belief for each outcome

87.
Parallel DBNN
● Since DBNN split likelihood as the product of
individual pairs, computation of likelihood may
be independently carried out by a different node
in a HPC system.
● Broadcast likelihoods to nodes
● Compute
● Gather Bayesian belief for each outcome

88.
Parallel DBNN Code
Ajay Vibhute
Photometric Redshift Estimation
8 million Point sources from SDSS
Redshift with a step size of 0.05
Ranging from 0 to 7
Four compute nodes, 16 Gb RAM
Training time reduced from 3 days
to 11 hours

89.
Parallel DBNN Code
Ajay Vibhute
Photometric Redshift Estimation
8 million Point sources from SDSS
Redshift with a step size of 0.05
Ranging from 0 to 7
Four compute nodes, 16 Gb RAM
Training time reduced from 3 days
to 11 hours

90.
Parallel DBNN Code
Ajay Vibhute
Photometric Redshift Estimation
8 million Point sources from SDSS
Redshift with a step size of 0.05
Ranging from 0 to 7
Four compute nodes, 16 Gb RAM
Training time reduced from 3 days
to 11 hours

91.
Parallel DBNN Code
Ajay Vibhute
Photometric Redshift Estimation
8 million Point sources from SDSS
Redshift with a step size of 0.05
Ranging from 0 to 7
Four compute nodes, 16 Gb RAM
Training time reduced from 3 days
to 11 hours

92.
Parallel DBNN Results
Photometric redshift estimation of unresolved SDSS detections
compared with spectroscopically confirmed samples.
Unpublished : under preparation

93.
Parallel DBNN Results

94.
Work in Progress
● Use of features extracted from light curves can
improve the classification
● Not all interesting objects – discoveries – may
have light curves
● A VO ­ Machine learning Tool kit is under
development
● It will provide a VO compatible platform for
astronomical data mining.

95.
Work in Progress
● Use of features extracted from light curves can
improve the classification
Eclipsing Binary
Planetary System
Red Giant

96.
Work in Progress
● Use of features extracted from light curves can
improve the classification
Correlation analysis of 58 features
extracted from CRTS light curves.
Arun Kumar

97.
Work in Progress
● Use of features extracted from light curves can
improve the classification
● Not all interesting objects – discoveries – may
have light curves
● A VO ­ Machine learning Tool kit is under
development
● It will provide a VO compatible platform for
astronomical data mining.

98.
Work in Progress
● Use of features extracted from light curves can
improve the classification
Spectroscopic Pipeline for
the Double Spectrograph
at Palomar Observatory
Sheelu Abraham

99.
Work in Progress
● Use of features extracted from light curves can
improve the classification
● Not all interesting objects – discoveries – may
have light curves
● A VO ­ Machine learning Tool kit is under
development
● It will provide a VO compatible platform for
astronomical data mining.

100.
Photometric Databases and Data
Analysis Techniques
Indo­ US Joint Centers
Jan 20­24th
2014
1. CLASS ACT ­ IUCAA, Caltech, St. Thomas
College
2. Variable Stars ­ Univ. Delhi, SUNY
Oswego, Univ. of Florida, Gainesville, Texas
A&M Univ., IUCAAA