Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Random Walker for Content-Based Image Retrieval
          with Relevance Feedback




                                  Ma...
Image Retrieval - La ricerca di immagini
• Perch` cercare immagini?
       e
  ◦ Le immagini sono informazioni
  ◦ Catalog...
CBIR - La ricerca basata sul contenuto

• Content-Based VS Text-Based
• Concetto di “informazione visuale”
  ◦ visuale ` l...
Relevance Feedback
• Tecnica nata negli anni ’60 nell’ambito del document retrieval e
  ripresa con interesse dalla comuni...
Analisi del lavoro di tesi svolto


• Algoritmo di relevance feedback per l’image retrieval, in particolare
  CBIR.
• Appr...
Formulazione basata sulla teoria dei grafi 1/2


• Input del sistema: grafo pesato G = (V , E , w ), insieme di vertici
   ...
Formulazione basata sulla teoria dei grafi 2/2

• Riscrivendo in forma matriciale:
                  x(r ) = arg min    x L...
Random walk su grafi


• I grafi come struttura per rappresentare i dati di un problema.
• Un random walk su un grafo indire...
Random walker per l’image segmentation
• Algoritmo per l’image segmentation interattivo introdotto da Leo
  Grady (2006).
...
Il problema di Dirichlet
• Le probabilit` del random walker possono essere ottenute
               a
  risolvendo il probl...
Analogia tra seeded segmentation e circuiti elettrici




11 of 20
Random walker per l’image retrieval


• I nuovi nodi del grafo sono le immagini.
• Gli unici due seed necessari sono le ca...
Test sperimentali - Datasets
• Wang Dataset - 1000 immagini - 10 categorie


• Oliva Dataset - 2688 immagini - 8 categorie...
Test sperimentali - Features, algoritmi e impostazioni

• Features utilizzate:
  ◦ Color Histogram 32-D
  ◦ Color Histogra...
Test sperimentali - Precisione della ricerca

                 100                                                        ...
Test sperimentali - Scelta delle feature
                 100                                                             ...
Test sperimentali - Perfomance sui tempi

                             10                                                 ...
Test sperimentali - Implementazione sparsa
                    100                                                        ...
Conclusioni e sviluppi futuri
• Proposto un nuovo algoritmo per il CBIR con relevance feedback.
• Bont` dei risultati sper...
Bibliografia

     R. Datta, D. Joshi, J. Li, J.Z. Wang.
     Image retrieval: Ideas, influences, and trends of the new age....
Upcoming SlideShare
Loading in …5
×

Tesi specialistica Informatica

5,761 views

Published on

Presentazione della Tesi di Laurea Specialistica in Informatica tenutasi il 16 aprile 2010.

Published in: Technology
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Follow the link, new dating source: ❶❶❶ http://bit.ly/2F7hN3u ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating direct: ♥♥♥ http://bit.ly/2F7hN3u ♥♥♥
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Tesi specialistica Informatica

  1. 1. Random Walker for Content-Based Image Retrieval with Relevance Feedback Massimo Rabbi Universit` Ca’ Foscari di Venezia a Dipartimento di Informatica Laurea Specialistica in Informatica 16 aprile 2010
  2. 2. Image Retrieval - La ricerca di immagini • Perch` cercare immagini? e ◦ Le immagini sono informazioni ◦ Cataloghi pubblici e privati di foto/immagini • Come cercare immagini? ◦ Contenuto (Content-Based Image Retrieval) ◦ Descrizione (Text-Based Image Retrieval) ◦ Sistemi misti • Possibili applicazioni ◦ Sistemi di ricerca web ◦ Sicurezza e biometria ◦ Diagnostica medicale ◦ GIS ◦ Giornalismo e pubblicit` a ◦ Beni culturali 2 of 20
  3. 3. CBIR - La ricerca basata sul contenuto • Content-Based VS Text-Based • Concetto di “informazione visuale” ◦ visuale ` la query e ◦ visuale ` il ragionamento che guida la similitudine e ◦ visuali sono i criteri utilizzati per l’indicizzazione • Elementi percettivi per “descrivere” l’immagine ◦ features basate su colore, forma, texture, etc. ◦ features complesse, vedi SIFT o GIST • Problemi e ambiti di ricerca aperti: ◦ la questione “semantic gap” ◦ domini delle immagini: narrow VS broad ◦ features e misure di similarit` a ◦ interazione dell’utente 3 of 20
  4. 4. Relevance Feedback • Tecnica nata negli anni ’60 nell’ambito del document retrieval e ripresa con interesse dalla comunit` CBIR verso inizio-met` anni a a ’90. • Metodo fondamentale per attaccare il problema del “semantic gap”. • Concetto chiave di “user-in-the-loop”. 4 of 20
  5. 5. Analisi del lavoro di tesi svolto • Algoritmo di relevance feedback per l’image retrieval, in particolare CBIR. • Approccio graph-based: ◦ i nodi rappresentano le immagini; ◦ i pesi sugli archi rappresentano la similarit`. a • Concetto di base: random walker. • Analogie con algoritmo gi` esistente nel campo dell’image a segmentation. • Legami tra la teoria dei random walk su grafi, la teoria del potenziale nel discreto e i circuiti elettrici. 5 of 20
  6. 6. Formulazione basata sulla teoria dei grafi 1/2 • Input del sistema: grafo pesato G = (V , E , w ), insieme di vertici (r ) VL ⊆ V , funzione Ψ : V → {0, 1}. • Output del sistema: ranking vector x(r ) . • Problema di ottimizzazione: x(r ) = arg min (xi − xj )2 wij x (i,j)∈E • con i vincoli: (r ) (a) 0 ≤ xi ≤ 1, for all i ∈ V (r ) (r ) (b) xi = Ψ(i), for all i ∈ VL 6 of 20
  7. 7. Formulazione basata sulla teoria dei grafi 2/2 • Riscrivendo in forma matriciale: x(r ) = arg min x Lx , x (r ) con xi = Ψ(i) per tutti i ∈ VL • e pi` precisamente come: u LUU LUM xU x(r ) = arg min [xU xM ] LMU LMM xM xU • Derivando rispetto a xU : LUU xU = −LUM xM • Risoluzione di un sistema di equazioni lineari nelle variabili xU . 7 of 20
  8. 8. Random walk su grafi • I grafi come struttura per rappresentare i dati di un problema. • Un random walk su un grafo indiretto corrisponde ad una catena di Markov time-reversible. • Il random walk soddisfa infatti la cosiddetta Propriet` di Markov : a P (Xt+1 |X0 , X1 , ..., Xt ) = P (Xt+1 |Xt ) . • Dato un grafo indiretto G = (V , E , w ), i pesi sugli archi indicano la probabilit` di spostarsi verso il nodo vicino. a • Esempio di random walker: la navigazione di un utente nel web. 8 of 20
  9. 9. Random walker per l’image segmentation • Algoritmo per l’image segmentation interattivo introdotto da Leo Grady (2006). • L’immagine ` un grafo di pixel. e • Fondamenta nella teoria dei random walk su grafi. • Input: un numero di pixel predefiniti dall’utente, chiamati seed. • Output: immagine segmentata 9 of 20
  10. 10. Il problema di Dirichlet • Le probabilit` del random walker possono essere ottenute a risolvendo il problema di Dirichlet. • La sua formulazione combinatoria ` la seguente: e 1 1 D[x] = xT Lx = wij (xi − xj )2 2 2 eij ∈E • Riscrivendo otteniamo: 1 T LM B xM D[xU ] = x xT 2 M U BT LU xU • Derivando rispetto a xU : LU xU = −B T xM 10 of 20
  11. 11. Analogia tra seeded segmentation e circuiti elettrici 11 of 20
  12. 12. Random walker per l’image retrieval • I nuovi nodi del grafo sono le immagini. • Gli unici due seed necessari sono le categorie “rilevante” e “non rilevante”. • I pesi sugli archi rappresentano la similarit` tra coppie di immagini: a ||Ii − Ij || wij = exp − σ • I round di feedback per “popolare” l’insieme di seed. 12 of 20
  13. 13. Test sperimentali - Datasets • Wang Dataset - 1000 immagini - 10 categorie • Oliva Dataset - 2688 immagini - 8 categorie • Custom-Caltech Dataset - 4920 immagini - 43 categorie 13 of 20
  14. 14. Test sperimentali - Features, algoritmi e impostazioni • Features utilizzate: ◦ Color Histogram 32-D ◦ Color Histogram Layout 32-D ◦ Color Moments 9-D ◦ Gray Level Co-Occurrence Matrix 20-D ◦ Global Scene (GIST) 60-D • Algoritmi usati per i confronti: ◦ Feature Re-Weighting ◦ Relevance Score ◦ Relevance Score Stabilized ◦ Multiple Random Walk • 8 round di feedback. • Dimensioni dello scope: 20,30,40. 14 of 20
  15. 15. Test sperimentali - Precisione della ricerca 100 100 100 RS 90 90 90 RS-S RW 80 80 80 MRW FR Precision (%) Precision (%) Precision (%) 70 70 70 60 60 60 50 50 50 40 40 40 RS RS 30 RS-S 30 RS-S 30 RW RW 20 MRW 20 MRW 20 FR FR 10 10 10 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Feedback Rounds Feedback Rounds Feedback Rounds (a) Wang Dataset (b) Oliva Dataset (c) Custom Caltech Dataset • Feature usata: Color Histogram. • Random query: 500 query per (a) e (b), 100 query (c). • Dimensione scope: 20 elementi. 15 of 20
  16. 16. Test sperimentali - Scelta delle feature 100 100 100 90 90 90 80 80 80 Precision (%) Precision (%) Precision (%) 70 70 70 60 60 60 50 50 50 40 40 40 RS RS RS 30 RS-S 30 RS-S 30 RS-S RW RW RW 20 MRW 20 MRW 20 MRW FR FR FR 10 10 10 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Feedback Rounds Feedback Rounds Feedback Rounds (a) Color Histogram (b) Color Histogram Layout (c) Color Moments 100 100 90 90 80 80 Precision (%) Precision (%) 70 70 60 60 50 50 40 40 RS RS 30 RS-S 30 RS-S RW RW 20 MRW 20 MRW FR FR 10 10 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Feedback Rounds Feedback Rounds (d) GLCM (e) GIST 16 of 20
  17. 17. Test sperimentali - Perfomance sui tempi 10 1000 100 RS RS-S RW 1 10 100 MRW Average time per round Average time per round Average time per round FR 0.1 1 10 0.1 0.01 1 0.01 0.001 0.1 0.001 RS RS 0.0001 RS-S RS-S 0.01 RW 0.0001 RW MRW MRW FR FR 1e-05 1e-05 0.001 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Feedback Rounds Feedback Rounds Feedback Rounds (a) Wang Dataset (b) Oliva Dataset (c) Custom Caltech Dataset • Feature usata: Color Histogram. • Random query: 500 query per (a) e (b), 100 query (c). • Dimensione scope: 20 elementi. 17 of 20
  18. 18. Test sperimentali - Implementazione sparsa 100 1 90 Average time per round 80 Precision (%) 70 0.1 60 50 40 0.01 30 20 Original G Original G Sparse G Sparse G 10 0.001 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 Feedback Rounds Feedback Rounds (a) Precisione (b) Tempi per round • Feature scelta: Color Moments. • Query random: 500 query random su dataset Oliva. • Dimensione scope: 20 elementi. • Versione sparsa: regola di approssimazione k-nn con k=20. 18 of 20
  19. 19. Conclusioni e sviluppi futuri • Proposto un nuovo algoritmo per il CBIR con relevance feedback. • Bont` dei risultati sperimentali ottenuti. a • Metodo semplice da implementare e parameter-free. • Possibilit` di utilizzo in versione sparsa in dataset pi` grandi. a u • 11th European Conference on Computer Vision (ECCV 2010) - S.Rota Bul`, M.Rabbi, M.Pelillo - Paper Submitted. o • Future works: ◦ Modello di feedback: es. Ψ = {0, 0.5, 1} ◦ Query multi-immagine. ◦ Estendere la parte sperimentale: implementazione sparsa, datasets e features. ◦ Implementazione di un sistema completo di image retrieval. 19 of 20
  20. 20. Bibliografia R. Datta, D. Joshi, J. Li, J.Z. Wang. Image retrieval: Ideas, influences, and trends of the new age. ACM Computing Surveys 40, 1-60, 2008. M.S. Lew, N. Sebe, C. Djerba, R. Jain. Content-based multimedia information retrieval: State-of-the-art and challenges. ACM Transactions on Multimedia Computing, Communications and Applications 2, 1-19, 2006. L. Grady. Random Walks for Image Segmentation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1768-1783, 2006. L. Lov´sz. a Random Walks on Graphs: A survery. Combinatorics, Paul Erd¨s is Eighty (Vol. 2), 1-46, 1993. o 20 of 20

×