Relabeling and summarizing posterior distributions                            Proposed approach                           ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                         Proposed approach              ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                               Proposed approach                        ...
Relabeling and summarizing posterior distributions                                Proposed approach                       ...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach            Label-switc...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                 Proposed approach        Label-switchin...
Relabeling and summarizing posterior distributions                                 Proposed approach        Label-switchin...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                               Proposed approach        Label-switching ...
Relabeling and summarizing posterior distributions                               Proposed approach        Label-switching ...
Relabeling and summarizing posterior distributions                               Proposed approach        Label-switching ...
Relabeling and summarizing posterior distributions                                Proposed approach         Label-switchin...
Relabeling and summarizing posterior distributions                               Proposed approach        Label-switching ...
Relabeling and summarizing posterior distributions                               Proposed approach        Label-switching ...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Relabeling and summarizing posterior distributions                                Proposed approach        Label-switching...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                           An original v...
Relabeling and summarizing posterior distributions                                                           An original v...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                              An origina...
Relabeling and summarizing posterior distributions                                                              An origina...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                             An original...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                           An original v...
Relabeling and summarizing posterior distributions                                                           An original v...
Relabeling and summarizing posterior distributions                                                        An original vari...
Relabeling and summarizing posterior distributions                                                        An original vari...
Relabeling and summarizing posterior distributions                                                        An original vari...
Relabeling and summarizing posterior distributions                                                        An original vari...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Relabeling and summarizing posterior distributions                                                         An original var...
Outline  1   Relabeling and summarizing posterior distributions        Label-switching issue        Variable-dimensional s...
Relabeling and summarizing posterior distributions                                Proposed approach             Detection ...
Relabeling and summarizing posterior distributions                                Proposed approach        Detection and e...
Relabeling and summarizing posterior distributions                                Proposed approach             Detection ...
Relabeling and summarizing posterior distributions                                Proposed approach            Detection a...
Relabeling and summarizing posterior distributions                                       Proposed approach            Dete...
Relabeling and summarizing posterior distributions                                       Proposed approach            Dete...
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense
Upcoming SlideShare
Loading in …5
×

Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense

403
-1

Published on

These are the slides I have used during the defense of my thesis (see https://sites.google.com/site/alireza4702/publications/phd-thesis for more information).

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
403
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Signal decompositions using trans-dimensional Bayesian methods: Alireza Roodaki's Ph.D. defense

  1. 1. Relabeling and summarizing posterior distributions Proposed approach Results Conclusion Signal decompositions using trans-dimensional Bayesian methods Alireza Roodaki Ph.D. Thesis Defense Department of Signal Processing and Electronic Systems 2012, May 14th Advisors: Julien Bect and Gilles Fleury Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  2. 2. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project Ultra high energy particles coming from space (E ∼ 1019 eV) How and where? What is their composition (Proton, Iron)?Figure: A conceptual shower(http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  3. 3. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project Ultra high energy particles coming from space (E ∼ 1019 eV) How and where? What is their composition (Proton, Iron)?Figure: A conceptual shower(http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  4. 4. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project Ultra high energy particles coming from space (E ∼ 1019 eV) How and where? What is their composition (Proton, Iron)?Figure: A conceptual shower(http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 1/ 52
  5. 5. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project (Contd.) muons are generated when particles cross the atmosphere the number k of muons and their arrival times t µ are indicators of the originFigure: A conceptual shower and and composition of particledetectors (water tanks)(http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
  6. 6. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project (Contd.) muons are generated when particles cross the atmosphere the number k of muons and their arrival times t µ are indicators of the originFigure: A conceptual shower and and composition of particledetectors (water tanks)(http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
  7. 7. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project (Contd.) muons are generated when particles cross the atmosphere the number k of muons and their arrival times t µ are indicators of the originFigure: A conceptual shower and and composition of particledetectors (water tanks)(http://auger.org). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 2/ 52
  8. 8. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project (Contd.) Figure: Water tank detector. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 3/ 52
  9. 9. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project (Contd.) #PE 20 10 0 1.5 intensity 1 0.5 0 100 200 300 400 500 600 t[ns] Figure: Observed signal (n) Prof. Balázs Kégl from LAL, University of Paris 11. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 4/ 52
  10. 10. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 1: Detection and estimation of muons in theAuger project (Contd.) #PE 20 10 0 1.5 intensity 1 0.5 0 100 200 300 400 500 600 t[ns] Figure: Observed signal (n) Prof. Balázs Kégl from LAL, University of Paris 11. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 4/ 52
  11. 11. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 2: Spectral analysis 10 0 −10 Applications 0 10 20 30 40 50 60 time RADAR / SONAR 150 100 Array signal processingPower 50 Vibration analysis 0 ... 0 0.5 1 1.5 2 2.5 3 radial frequencyFigure: Observed signal (top) and itsperiodogram (bottom). Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 5/ 52
  12. 12. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 2: Spectral analysis (Cont.) Detection and estimation of sinusoids in white noise model the observed signal y by sinusoidal components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
  13. 13. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 2: Spectral analysis (Cont.) Detection and estimation of sinusoids in white noise model the observed signal y by sinusoidal components observed signal k Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i]. j=1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
  14. 14. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 2: Spectral analysis (Cont.) Detection and estimation of sinusoids in white noise model the observed signal y by sinusoidal components observed signal k Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i]. j=1 Joint model selection and parameter estimation problem Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 6/ 52
  15. 15. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionTrans-dimensional problems Def. The problems in which the number of things that we don′ t know is one of the things that we don′ t know [Green, 2003.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
  16. 16. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionTrans-dimensional problems Def. The problems in which the number of things that we don′ t know is one of the things that we don′ t know [Green, 2003.] space X = k ∈K {k } × Θk with points x = (k , θk ) « k ∈ K denotes number of components « θk ∈ Θk is a vector of component-specific parameters Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
  17. 17. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionTrans-dimensional problems Def. The problems in which the number of things that we don′ t know is one of the things that we don′ t know [Green, 2003.] space X = k ∈K {k } × Θk with points x = (k , θk ) « k ∈ K denotes number of components « θk ∈ Θk is a vector of component-specific parameters Applications: « Spectral Analysis (Array signal processing) « (Gaussian) Mixture modeling & Clustering « Object detection and recognition Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 7/ 52
  18. 18. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionBayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  19. 19. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionBayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  20. 20. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionBayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  21. 21. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionBayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution x = (k , θk ) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  22. 22. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionBayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution x = (k , θk ) « both detection and estimation problems Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  23. 23. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionBayesian inference Likelihood p(y | x) p(x) p(x | y) = X p(y | x ′ ) p(x ′ )dx ′ Prior distribution Posterior distribution x = (k , θk ) « both detection and estimation problems high-dimensional / intractable integrals Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 8/ 52
  24. 24. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionMarkov Chain Monte Carlo (MCMC) methods generate samples from the posterior distribution of interest (target distribution), say, π. construct a Markov chain (x (1) , . . . , x (M) ) that under some conditions converges to π. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
  25. 25. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionMarkov Chain Monte Carlo (MCMC) methods generate samples from the posterior distribution of interest (target distribution), say, π. construct a Markov chain (x (1) , . . . , x (M) ) that under some conditions converges to π. Famous algorithms: « Metropolis-Hastings (MH) sampler [Metropolis, et al. 1953, Hastings, 1970.] « Gibbs sampler [Geman and Geman, 1984.] « RJ-MCMC sampler [Green, 1995.] [Robert and Casella, 2004.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
  26. 26. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionMarkov Chain Monte Carlo (MCMC) methods generate samples from the posterior distribution of interest (target distribution), say, π. construct a Markov chain (x (1) , . . . , x (M) ) that under some conditions converges to π. Famous algorithms: « Metropolis-Hastings (MH) sampler [Metropolis, et al. 1953, Hastings, 1970.] « Gibbs sampler [Geman and Geman, 1984.] « RJ-MCMC sampler [Green, 1995.] [Robert and Casella, 2004.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 9/ 52
  27. 27. Relabeling and summarizing posterior distributions Proposed approach Results ConclusionExample 2: spectral analysis (Cont.) RJ-MCMC sampler ⇒ variable dimensional samples 0.8 0.6 ωk 0.4 4 3 k 2 160 170 180 190 200 Iteration number Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 10/ 52
  28. 28. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization
  29. 29. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies
  30. 30. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project
  31. 31. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  32. 32. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  33. 33. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  34. 34. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  35. 35. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  36. 36. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  37. 37. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Summarization human readable summaries Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  38. 38. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Summarization human readable summaries interpretable figures (e.g., histograms) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  39. 39. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionsummarizing posterior distributions Posterior = all the information « It is a complex mathematical object (not easy to manipulate) (RJ)-MCMC sampler « What to do with the generated samples? Summarization human readable summaries interpretable figures (e.g., histograms) statistical measures (e.g., mean and variance) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 12/ 52
  40. 40. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionFixed-dimensional problems uni-modal uni-variate case 0.6 p 0.4 0.2 0 0 2 4 Samples report location (mean and median) and dispersion (variance and confidence intervals) parameters µ σ2 2.0 0.25 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 13/ 52
  41. 41. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionLabel-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  42. 42. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionLabel-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components the posterior distribution is invariant under permutation of components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  43. 43. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionLabel-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components the posterior distribution is invariant under permutation of components Marginal posteriors ofComp. #1 10 component-specific 0 parameters are identical!Comp. #2 10 0Comp. #3 10 0 0.5 0.75 1 ω Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  44. 44. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionLabel-switching issue Additive mixture: lack of identifiability the likelihood is invariant under relabeling of components the posterior distribution is invariant under permutation of components Marginal posteriors ofComp. #1 10 component-specific 0 parameters are identical!Comp. #2 10 How to summarize the 0 posterior information?Comp. #3 10 0 0.5 0.75 1 ω Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 14/ 52
  45. 45. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionstrategies to deal with label-switching imposing artificial “identifiability constraints” Exp: sorting the components [Richardson and Green, 1997.] Comp. #1 10 0 Comp. #2 10 0 Comp. #3 10 0 0.5 0.75 1 Figure: components are sorted based on ω. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 15/ 52
  46. 46. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization Conclusionstrategies to deal with label-switching imposing artificial “identifiability constraints” Exp: sorting the components [Richardson and Green, 1997.] Comp. #1 10 0 Comp. #2 10 0 Comp. #3 10 0 0.5 0.75 1 Figure: components are sorted based on ω. relabeling algorithms [Celeux, et al. 1998, Stephens, 2000, Jasra, et al, 2005, Sperrin et al, 2010, Yao, 2011.]. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 15/ 52
  47. 47. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionExample 2: spectral analysis (Cont.) Variable-dimensional posterior distribution 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: Posteriors of k and sorted radial frequencies, ωk , given k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 16/ 52
  48. 48. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionClassical Bayesian approaches Bayesian Model Selection (BMS) One model is selected (estimated) by looking at the MAP, ˆ i.e. k = argmax p(k |y). Component-specific parameters are summarized given ˆ k = k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 17/ 52
  49. 49. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionBayesian Model Selection (BMS) 3 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: The model with k = 2 is selected. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 18/ 52
  50. 50. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionClassical Bayesian approaches Bayesian Model Selection (BMS) One model is selected (estimated) by looking at the MAP, ˆ i.e. k = argmax p(k |y). Component-specific parameters are summarized given ˆ k = k. Bayesian Model Averaging (BMA) Use the information from all possible models: p(∆|y) = k p(∆|k , y)p(k |y) However, ∆ cannot be ωk as its size changes from model to model. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 19/ 52
  51. 51. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionBayesian Model Averaging (BMA) Binned data representation (∆ = N(Bj )): kmax E(N(Bj ) | y) = E(N(Bj ) | k , y) · p(k | y) k =1 1 expected nbr comp where .75 j = 1, . . . , Nbin .5 and E(N(Bj )) is the .25 expected number of components in bin 0 0 0.5 1 1.5 2 2.5 3 Bj . ω Figure: Expected number of components using BMA. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 20/ 52
  52. 52. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionAre BMS and BMA approaches satisfactory? Bayesian Model Selection (BMS) « selects a model ⇒ component-specific parameters « losing information from the discarded models « ignoring the uncertainties about the presence of components. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 21/ 52
  53. 53. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionAre BMS and BMA approaches satisfactory? (Cont.) Bayesian Model Averaging (BMA) « appropriate for signal reconstruction and prediction « does not provide information about component-specific parameters Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 22/ 52
  54. 54. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionA novel approach is needed! A novel approach is needed! Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 23/ 52
  55. 55. Relabeling and summarizing posterior distributions Proposed approach Label-switching issue Results Variable-dimensional summarization ConclusionA novel approach is needed! A novel approach is needed! Properties of an “ideal” approach information from all (plausible) models « interpretable summaries for component-specific parameters uncertainties about the presence of components Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 23/ 52
  56. 56. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  57. 57. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionBig picture: relabeling and summarizing posteriordistributions True posterior f = p(· | y) Approximate posterior qη 2 2 1 1 0 0 0 1 2 3 0 1 2 3 Parametric family {qη , η ∈ N} Measure of “distance” [Stephens, 2000.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 24/ 52
  58. 58. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionBig picture: relabeling and summarizing posteriordistributions True posterior f = p(· | y) Samples Approximate posterior qη 2 2 2 1 1 1 0 0 0 0 1 2 3 0 1 2 3 0 1 2 3 Parametric family {qη , η ∈ N} Measure of “distance” Samples [Stephens, 2000.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 24/ 52
  59. 59. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionFixed-dimensional problems uni-modal uni-variate case 0.6 p 0.4 0.2 0 0 2 4 Samples report location (mean and median) and dispersion (variance and confidence intervals) parameters µ σ2 2.0 0.25 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 25/ 52
  60. 60. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionvariable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  61. 61. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionvariable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk 2 Must be permutation invariance Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  62. 62. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionvariable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk 2 Must be permutation invariance 3 Must be “simple” (small number of parameters) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  63. 63. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionvariable-dimensional approximate posterior An original (variable-dimensional) parametric model qη Four main requirements: 1 Must be defined on the same space X = k ∈K {k } × Θk 2 Must be permutation invariance 3 Must be “simple” (small number of parameters) 4 Must be able to capture the main features of the posterior distributions typically met in practice. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 26/ 52
  64. 64. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view ··· 1 2 L x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  65. 65. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view ··· 1 2 L u1 u2 uL ul ∈ Θ x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  66. 66. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  67. 67. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  68. 68. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} random arrangement x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  69. 69. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view π1 π2 πL ··· 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} ξl ∼ B(πl ) random arrangement x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  70. 70. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view π1 π2 πL µ1 Σ1 µ2 Σ2 ··· µL ΣL 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} ξl ∼ B(πl ) u l ∼ N (µl , Σl ) random arrangement x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  71. 71. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view π1 π2 πL µ1 Σ1 µ2 Σ2 ··· µL ΣL 1 2 L ξ1 u1 ξ2 u2 ξL uL ul ∈ Θ ξl ∈ {0, 1} {u l | ξl = 1} ξl ∼ B(πl ) u l ∼ N (µl , Σl ) random arrangement ηl = {πl , µl , Σl } x = (k , θk ) ∈ X = k {k } × Θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 27/ 52
  72. 72. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  73. 73. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  74. 74. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: 18 L = 3 12 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) s2 = (0.05, 0.02, 0.1) 6 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  75. 75. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 1, 1) ⇒ k = 2 & θk = (0.52, 1.05) 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  76. 76. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 1, 0) ⇒ k = 1 & θk = 0.49 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  77. 77. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (1, 0, 1) ⇒ k = 2 & θk = (0.27, 1.03) 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  78. 78. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 0, 1) ⇒ k = 1 & θk = 1.05 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  79. 79. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (1, 1, 1) ⇒ k = 3 & θk = (0.27, 0.53, 1.15) 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  80. 80. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionA generative model point of view 1 for l = 1, . . . , L generate binary number ξl ∼ B (πl ) end L 2 set k = l=1 ξl ; 3 for each l such that ξl = 1 generate random sample u l ∼ N (µl , Σl ) end 4 Random arrangement ⇒ θk Example: L = 3 18 π = (0.4, 0.9, 0.7) µ = (0.2, 0.5, 1) 12 s2 = (0.05, 0.02, 0.1) 6 ξ = (0, 0, 0) ⇒ k = 0 & θk = () 0.4 0.8 1.2 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 28/ 52
  81. 81. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionBig picture: relabeling and summarizing posteriordistributions True posterior f = p(· | y) Samples Approximate posterior qη 2 2 2 1 1 1 0 0 0 0 1 2 3 0 1 2 3 0 1 2 3 Parametric family {qη , η ∈ N} Measure of “distance” Samples [Stephens, 2000.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 29/ 52
  82. 82. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionfitting the parametric model qη to the posterior f minimizing the Kullback-Leibler divergence f (x) J (η) DKL (f (x) qη (x)) = f (x) log dx qη (x) Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
  83. 83. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionfitting the parametric model qη to the posterior f minimizing the Kullback-Leibler divergence f (x) J (η) DKL (f (x) qη (x)) = f (x) log dx qη (x) A key point: samples x (i) , i = 1, 2, · · · , M, are generated from f , so M 1 J (η) ≃ − log qη (x (i) ) + Const. M i=1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
  84. 84. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies Conclusionfitting the parametric model qη to the posterior f minimizing the Kullback-Leibler divergence f (x) J (η) DKL (f (x) qη (x)) = f (x) log dx qη (x) A key point: samples x (i) , i = 1, 2, · · · , M, are generated from f , so M 1 J (η) ≃ − log qη (x (i) ) + Const. M i=1 M Objective: η = argmaxη ˆ i=1 log qη (x (i) ) . Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 30/ 52
  85. 85. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionExpectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  86. 86. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionExpectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation « Define z = (z1 , . . . , zk ) as an allocation vector for x « zj = l ⇒ x j comes from component l Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  87. 87. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionExpectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation « Define z = (z1 , . . . , zk ) as an allocation vector for x « zj = l ⇒ x j comes from component l idea: use EM to maximize the likelihood. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  88. 88. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionExpectation Maximization (EM) latent (hidden) variable: « Binary indicator vector ξ « Random permutation « Define z = (z1 , . . . , zk ) as an allocation vector for x « zj = l ⇒ x j comes from component l idea: use EM to maximize the likelihood. The E-step is computationally expensive! For example, assuming L = 15 and k (i) = 10, then, it contains 1.1 × 1010 terms. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 31/ 52
  89. 89. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionStochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  90. 90. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionStochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  91. 91. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionStochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  92. 92. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionStochastic EM (SEM) at iteration (r + 1) Stochastic (S)-step: for i = 1, . . . , M generate z(i) from p( · | x (i) , η (r ) ) end ˆ M M-step: η (r +1) = argmaxη ˆ i=1 log(p(x (i) , z(i) | η)) [Broniatowski, et al. 1983. Celeux and Diebolt, 1986. Celeux and Diebolt, 1993.] S-step To draw z(i) ∼ p( · | x (i) , η (r ) ) we developed an I-MH sampler. ˆ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 32/ 52
  93. 93. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionExample 2: spectral analysis (Cont.) Variable-dimensional posterior distribution 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: Posteriors of k and sorted radial frequencies, ωk , given k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 33/ 52
  94. 94. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionRobustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  95. 95. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionRobustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  96. 96. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionRobustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  97. 97. Relabeling and summarizing posterior distributions An original variable-dimensional parametric model Proposed approach Estimating the model parameters (SEM-type algorithms) Results Robustifying strategies ConclusionRobustifying solutions Add a Poisson point process component To capture the “outliers” λ is the mean parameter points are uniformly distributed on Θ Other possibilities Robust estimates in the M-step. « Median instead of mean « interquartile range instead of variance Using another divergence measure. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 34/ 52
  98. 98. Outline 1 Relabeling and summarizing posterior distributions Label-switching issue Variable-dimensional summarization 2 Proposed approach An original variable-dimensional parametric model Estimating the model parameters (SEM-type algorithms) Robustifying strategies 3 Results Detection and estimation of sinusoids in white noise Detection and estimation of muons in the Auger project 4 Conclusion
  99. 99. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project ConclusionExample 2: Spectral analysis 10 0 −10 0 10 20 30 40 50 60 time 150 100 Power 50 0 0 0.5 1 1.5 2 2.5 3 radial frequency Figure: Observed signal (top) and its periodogram (bottom). k Mk : y [i] = aj cos[ωj i] + bj sin[ωj i] + n[i]. j=1 [Andrieu and Doucet, 1999.] Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 35/ 52
  100. 100. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project ConclusionExample 2: spectral analysis (Cont.) Variable-dimensional posterior distribution 4 3 k 2 0 0.3 0.6 0.5 0.75 1 p(k |y) ω Figure: Posteriors of k and sorted radial frequencies, ωk , given k. Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 36/ 52
  101. 101. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project ConclusionS-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 z1 z2 z1 z2 z3 z4 z1 z2 z3M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.1 Iter = 0 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  102. 102. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project ConclusionS-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 4 2M-step norm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.1 Iter = 0 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  103. 103. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project ConclusionS-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 4 2M-stepnorm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.167 Iter = 1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  104. 104. Relabeling and summarizing posterior distributions Proposed approach Detection and estimation of sinusoids in white noise Results Detection and estimation of muons in the Auger project ConclusionS-step: randomized allocation procedure 0.72 0.62 0.73 0.83 0.67 0.64 0.64 0.74 0.72 3 1 4 2 3 1 1 3 2M-stepnorm. density 1 0.5 0 0.5 0.75 1 ω ˆ λ = 0.167 Iter = 1 Alireza Roodaki (SUPELEC) Signal decompositions using trans-dimensional . . . 37/ 52
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×