SlideShare a Scribd company logo

LIvRE: A Video Extension to the LIRE Content-Based Image Retrieval System

https://imatge.upc.edu/web/publications/livre-video-extension-lire-content-based-image-retrieval-system This project explores the expansion of Lucene Image Retrieval Engine (LIRE), an open-source Content-Based Image Retrieval (CBIR) system, for video retrieval on large scale video datasets. The fast growth of the need to store huge amounts of video in servers requires efficient, scalable search and indexing engines capable to assist users in their management and retrieval. In our tool, queries are formulated by visual examples allowing users to find the videos and the moment of time when the query image is matched with. The video dataset used on this scenario comprise over 1,000 hours of different news broadcast channels. This thesis presents an extension and adaptation of Lire and its plugin for Solr, an open-source enterprise search platform from the Apache Lucene project, for video retrieval based on visual features, as well as a web-interface for users from different devices.

1 of 23
Download to read offline
LIVRE: A VIDEO EXTENSION TO THE
LIRE CONTENT-BASED IMAGE
RETRIEVAL SYSTEM
Degree’s Final Project Dissertation
Telecommunications Engineering
Gabriel de Oliveira
Supervisors:
Assoc. Prof. Mathias Lux
Assoc. Prof. Xavier Giró
Outline of the Thesis
1. Introduction
i. Motivation
ii. Overview and previous work
2. Proposed solution: The LIvRE system
i. Parsing
ii. Indexing
iii.Retrieval
3. Validation
i. Dataset
- Stanford I2V Newscasts dataset
ii. Experiments
- Quantitative evaluation
- Qualitative evaluation - The thinking-aloud test.
4. Conclusions and Further Work
March – October 2015
Slide 2
Motivation
Goal: To develop an all-in-one open source system for CBVR.
• Server side requeriments:
• Fast
• Scalable
• Flexible
• Automated
• User interface requeriments:
• Fast
• OS and device independent
• Mobile
Slide 3
Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
Overview and previous work
Slide 6
• Open source CBIR Library in Java
• Apache Lucene core
• Solr plugin
• Supports parsing, indexing and retrieval
• Global and local descriptors
• Web-based interface
[1] Mathias Lux. LIRE: Open source image retrieval in java. In Proceedings of the 21st ACM international conference on Multimedia, pages 843{846.ACM, 2013.
Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
Developed solution: The LIvRE CBVR system
Slide 7
CBVR system - concept and requirements.
database
Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
Developed solution: The LIvRE CBVR system
Slide 7
User sideServer side
LIvRE CBVR system architecture.
Introduction · Overview · LIvRE CBVR system · Validation · Conclusions

Recommended

Content based image retrieval with LIRe
Content based image retrieval with LIReContent based image retrieval with LIRe
Content based image retrieval with LIRedermotte
 
Evolving Communication Mechanisms of the OSGi Framework - Xuejun Chen, BMW Re...
Evolving Communication Mechanisms of the OSGi Framework - Xuejun Chen, BMW Re...Evolving Communication Mechanisms of the OSGi Framework - Xuejun Chen, BMW Re...
Evolving Communication Mechanisms of the OSGi Framework - Xuejun Chen, BMW Re...mfrancis
 
OSGi Community Event 2010 - Experiences with OSGi in Industrial Applications
OSGi Community Event 2010 - Experiences with OSGi in Industrial ApplicationsOSGi Community Event 2010 - Experiences with OSGi in Industrial Applications
OSGi Community Event 2010 - Experiences with OSGi in Industrial Applicationsmfrancis
 
Adaptive Network Middleware CSC (Communication Service Concierge) - R Kawamura
Adaptive Network Middleware CSC (Communication Service Concierge)  - R KawamuraAdaptive Network Middleware CSC (Communication Service Concierge)  - R Kawamura
Adaptive Network Middleware CSC (Communication Service Concierge) - R Kawamuramfrancis
 
LIRE presentation at the ACM Multimedia Open Source Software Competition 2013
LIRE presentation at the ACM Multimedia Open Source Software Competition 2013LIRE presentation at the ACM Multimedia Open Source Software Competition 2013
LIRE presentation at the ACM Multimedia Open Source Software Competition 2013dermotte
 

More Related Content

Viewers also liked

Image enhancement
Image enhancementImage enhancement
Image enhancementMANISH T I
 
Block Matching Project
Block Matching ProjectBlock Matching Project
Block Matching Projectdswazalwar
 
Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...
Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...
Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...tmra
 
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...Mateus S. H. Cruz
 
Semantic Search Over The Web
Semantic Search Over The WebSemantic Search Over The Web
Semantic Search Over The Webalierkan
 
Application Architecture Summit - Monitoring the Dynamic Cloud
Application Architecture Summit - Monitoring the Dynamic Cloud Application Architecture Summit - Monitoring the Dynamic Cloud
Application Architecture Summit - Monitoring the Dynamic Cloud New Relic
 
Fuzzy Logic in the Real World
Fuzzy Logic in the Real WorldFuzzy Logic in the Real World
Fuzzy Logic in the Real WorldBCSLeicester
 
Privacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the Cloud
Privacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the CloudPrivacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the Cloud
Privacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the CloudMateus S. H. Cruz
 
Advances in Image Search and Retrieval
Advances in Image Search and RetrievalAdvances in Image Search and Retrieval
Advances in Image Search and RetrievalOge Marques
 
LinkedIn SlideShare: Knowledge, Well-Presented
LinkedIn SlideShare: Knowledge, Well-PresentedLinkedIn SlideShare: Knowledge, Well-Presented
LinkedIn SlideShare: Knowledge, Well-PresentedSlideShare
 

Viewers also liked (19)

Relevance feedback for image retrieval with EEG signals
Relevance feedback for image retrieval with EEG signalsRelevance feedback for image retrieval with EEG signals
Relevance feedback for image retrieval with EEG signals
 
Part-based Object Retrieval with Binary Partition Trees
Part-based Object Retrieval with Binary Partition TreesPart-based Object Retrieval with Binary Partition Trees
Part-based Object Retrieval with Binary Partition Trees
 
Image enhancement
Image enhancementImage enhancement
Image enhancement
 
Tools for Image Retrieval in Large Multimedia Databases
Tools for Image Retrieval in Large Multimedia DatabasesTools for Image Retrieval in Large Multimedia Databases
Tools for Image Retrieval in Large Multimedia Databases
 
Europeana Tech 2011
Europeana Tech 2011Europeana Tech 2011
Europeana Tech 2011
 
CBIR with RF
CBIR with RFCBIR with RF
CBIR with RF
 
Block Matching Project
Block Matching ProjectBlock Matching Project
Block Matching Project
 
Convolutional Features for Instance Search
Convolutional Features for Instance SearchConvolutional Features for Instance Search
Convolutional Features for Instance Search
 
Region-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object RetrievalRegion-oriented Convolutional Networks for Object Retrieval
Region-oriented Convolutional Networks for Object Retrieval
 
Multi-label Remote Sensing Image Retrieval based on Deep Features
Multi-label Remote Sensing Image Retrieval based on Deep FeaturesMulti-label Remote Sensing Image Retrieval based on Deep Features
Multi-label Remote Sensing Image Retrieval based on Deep Features
 
Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...
Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...
Quality, Relevance and Importance in Information Retrieval with Fuzzy Semanti...
 
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
Inverted Index Based Multi-Keyword Public-key Searchable Encryption with Stro...
 
Semantic Search Over The Web
Semantic Search Over The WebSemantic Search Over The Web
Semantic Search Over The Web
 
Application Architecture Summit - Monitoring the Dynamic Cloud
Application Architecture Summit - Monitoring the Dynamic Cloud Application Architecture Summit - Monitoring the Dynamic Cloud
Application Architecture Summit - Monitoring the Dynamic Cloud
 
Fuzzy Logic in the Real World
Fuzzy Logic in the Real WorldFuzzy Logic in the Real World
Fuzzy Logic in the Real World
 
Fuzzy logic
Fuzzy logicFuzzy logic
Fuzzy logic
 
Privacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the Cloud
Privacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the CloudPrivacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the Cloud
Privacy-Preserving Multi-Keyword Fuzzy Search over Encrypted Data in the Cloud
 
Advances in Image Search and Retrieval
Advances in Image Search and RetrievalAdvances in Image Search and Retrieval
Advances in Image Search and Retrieval
 
LinkedIn SlideShare: Knowledge, Well-Presented
LinkedIn SlideShare: Knowledge, Well-PresentedLinkedIn SlideShare: Knowledge, Well-Presented
LinkedIn SlideShare: Knowledge, Well-Presented
 

Similar to LIvRE: A Video Extension to the LIRE Content-Based Image Retrieval System

No Compromise - Better, Stronger, Faster Java in the Cloud
No Compromise - Better, Stronger, Faster Java in the CloudNo Compromise - Better, Stronger, Faster Java in the Cloud
No Compromise - Better, Stronger, Faster Java in the CloudAll Things Open
 
Laying the Foundation for Ionic Platform Insights on Spark
Laying the Foundation for Ionic Platform Insights on SparkLaying the Foundation for Ionic Platform Insights on Spark
Laying the Foundation for Ionic Platform Insights on SparkIonic Security
 
Netw200 final joshua_brown
Netw200 final joshua_brownNetw200 final joshua_brown
Netw200 final joshua_brownJoshuaBrown233
 
[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇
[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇
[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇台灣資料科學年會
 
CloudHealth: A Model-Driven Approach to Watch the Health of Cloud Services
CloudHealth: A Model-Driven Approach to Watch the Health of Cloud ServicesCloudHealth: A Model-Driven Approach to Watch the Health of Cloud Services
CloudHealth: A Model-Driven Approach to Watch the Health of Cloud ServicesAnas Shatnawi
 
Cross Cloud CI Intro
Cross Cloud CI IntroCross Cloud CI Intro
Cross Cloud CI IntroW Watson
 
[第45会コンピュータービジョン勉強会@関東] ChainerCV
[第45会コンピュータービジョン勉強会@関東] ChainerCV[第45会コンピュータービジョン勉強会@関東] ChainerCV
[第45会コンピュータービジョン勉強会@関東] ChainerCVYusuke Niitani
 
Securing deployment pipeline
Securing deployment pipelineSecuring deployment pipeline
Securing deployment pipelineLen Bass
 
Test parallelization using Jenkins
Test parallelization using JenkinsTest parallelization using Jenkins
Test parallelization using JenkinsRogue Wave Software
 
Azure meetup cloud native concepts - may 28th 2018
Azure meetup   cloud native concepts - may 28th 2018Azure meetup   cloud native concepts - may 28th 2018
Azure meetup cloud native concepts - may 28th 2018Jim Bugwadia
 
Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10MagaliDavidCruz
 
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREUVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREVLSICS Design
 
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREUVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREVLSICS Design
 
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREUVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREVLSICS Design
 
GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"
GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"
GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"Daniel Bryant
 
VLSI lab manual Part A, VTU 7the sem KIT-tiptur
VLSI lab manual Part A, VTU 7the sem KIT-tipturVLSI lab manual Part A, VTU 7the sem KIT-tiptur
VLSI lab manual Part A, VTU 7the sem KIT-tipturPramod Kumar S
 
Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...
Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...
Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...KTN
 
Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)
Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)
Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)Saimunur Rahman
 
JCON_15FactorWorkshop.pptx
JCON_15FactorWorkshop.pptxJCON_15FactorWorkshop.pptx
JCON_15FactorWorkshop.pptxGrace Jansen
 

Similar to LIvRE: A Video Extension to the LIRE Content-Based Image Retrieval System (20)

No Compromise - Better, Stronger, Faster Java in the Cloud
No Compromise - Better, Stronger, Faster Java in the CloudNo Compromise - Better, Stronger, Faster Java in the Cloud
No Compromise - Better, Stronger, Faster Java in the Cloud
 
Laying the Foundation for Ionic Platform Insights on Spark
Laying the Foundation for Ionic Platform Insights on SparkLaying the Foundation for Ionic Platform Insights on Spark
Laying the Foundation for Ionic Platform Insights on Spark
 
Netw200 final joshua_brown
Netw200 final joshua_brownNetw200 final joshua_brown
Netw200 final joshua_brown
 
[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇
[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇
[2018 台灣人工智慧學校校友年會] 視訊畫面生成 / 林彥宇
 
CloudHealth: A Model-Driven Approach to Watch the Health of Cloud Services
CloudHealth: A Model-Driven Approach to Watch the Health of Cloud ServicesCloudHealth: A Model-Driven Approach to Watch the Health of Cloud Services
CloudHealth: A Model-Driven Approach to Watch the Health of Cloud Services
 
Cross Cloud CI Intro
Cross Cloud CI IntroCross Cloud CI Intro
Cross Cloud CI Intro
 
[第45会コンピュータービジョン勉強会@関東] ChainerCV
[第45会コンピュータービジョン勉強会@関東] ChainerCV[第45会コンピュータービジョン勉強会@関東] ChainerCV
[第45会コンピュータービジョン勉強会@関東] ChainerCV
 
Securing deployment pipeline
Securing deployment pipelineSecuring deployment pipeline
Securing deployment pipeline
 
Test parallelization using Jenkins
Test parallelization using JenkinsTest parallelization using Jenkins
Test parallelization using Jenkins
 
Azure meetup cloud native concepts - may 28th 2018
Azure meetup   cloud native concepts - may 28th 2018Azure meetup   cloud native concepts - may 28th 2018
Azure meetup cloud native concepts - may 28th 2018
 
Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10Meetup Openshift Geneva 03/10
Meetup Openshift Geneva 03/10
 
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREUVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
 
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREUVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
 
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER COREUVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
UVM BASED REUSABLE VERIFICATION IP FOR WISHBONE COMPLIANT SPI MASTER CORE
 
GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"
GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"
GOTOpia 2020: "The Past, Present, and Future of Cloud Native API Gateways"
 
VLSI lab manual Part A, VTU 7the sem KIT-tiptur
VLSI lab manual Part A, VTU 7the sem KIT-tipturVLSI lab manual Part A, VTU 7the sem KIT-tiptur
VLSI lab manual Part A, VTU 7the sem KIT-tiptur
 
01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf
 
Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...
Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...
Implementing AI: Running AI at the Edge: ClickCV – Providing high-performance...
 
Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)
Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)
Reading group - Week 2 - Trajectory Pooled Deep-Convolutional Descriptors (TDD)
 
JCON_15FactorWorkshop.pptx
JCON_15FactorWorkshop.pptxJCON_15FactorWorkshop.pptx
JCON_15FactorWorkshop.pptx
 

More from Universitat Politècnica de Catalunya

The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...Universitat Politècnica de Catalunya
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoUniversitat Politècnica de Catalunya
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Universitat Politècnica de Catalunya
 
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosGeneration of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosUniversitat Politècnica de Catalunya
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Universitat Politècnica de Catalunya
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Universitat Politècnica de Catalunya
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Universitat Politècnica de Catalunya
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Universitat Politècnica de Catalunya
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Universitat Politècnica de Catalunya
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Universitat Politècnica de Catalunya
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Universitat Politècnica de Catalunya
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Universitat Politècnica de Catalunya
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Universitat Politècnica de Catalunya
 

More from Universitat Politècnica de Catalunya (20)

Deep Generative Learning for All
Deep Generative Learning for AllDeep Generative Learning for All
Deep Generative Learning for All
 
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
The Transformer in Vision | Xavier Giro | Master in Computer Vision Barcelona...
 
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-NietoTowards Sign Language Translation & Production | Xavier Giro-i-Nieto
Towards Sign Language Translation & Production | Xavier Giro-i-Nieto
 
The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021The Transformer - Xavier Giró - UPC Barcelona 2021
The Transformer - Xavier Giró - UPC Barcelona 2021
 
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
Learning Representations for Sign Language Videos - Xavier Giro - NIST TRECVI...
 
Open challenges in sign language translation and production
Open challenges in sign language translation and productionOpen challenges in sign language translation and production
Open challenges in sign language translation and production
 
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in VideosGeneration of Synthetic Referring Expressions for Object Segmentation in Videos
Generation of Synthetic Referring Expressions for Object Segmentation in Videos
 
Discovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in MinecraftDiscovery and Learning of Navigation Goals from Pixels in Minecraft
Discovery and Learning of Navigation Goals from Pixels in Minecraft
 
Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...Learn2Sign : Sign language recognition and translation using human keypoint e...
Learn2Sign : Sign language recognition and translation using human keypoint e...
 
Intepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural NetworksIntepretability / Explainable AI for Deep Neural Networks
Intepretability / Explainable AI for Deep Neural Networks
 
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
Convolutional Neural Networks - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
Self-Supervised Audio-Visual Learning - Xavier Giro - UPC TelecomBCN Barcelon...
 
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
Attention for Deep Learning - Xavier Giro - UPC TelecomBCN Barcelona 2020
 
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
Generative Adversarial Networks GAN - Xavier Giro - UPC TelecomBCN Barcelona ...
 
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
Q-Learning with a Neural Network - Xavier Giró - UPC Barcelona 2020
 
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
Language and Vision with Deep Learning - Xavier Giró - ACM ICMR 2020 (Tutorial)
 
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
Image Segmentation with Deep Learning - Xavier Giro & Carles Ventura - ISSonD...
 
Curriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object SegmentationCurriculum Learning for Recurrent Video Object Segmentation
Curriculum Learning for Recurrent Video Object Segmentation
 
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
Deep Self-supervised Learning for All - Xavier Giro - X-Europe 2020
 
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
Deep Learning Representations for All - Xavier Giro-i-Nieto - IRI Barcelona 2020
 

Recently uploaded

Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docxLeveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docxVotarikari Shravan
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17Ana-Maria Mihalceanu
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, GoogleISPMAIndia
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxInfosec
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...Neo4j
 
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...Product School
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERNRonnelBaroc
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...htrindia
 
Campotel: Telecommunications Infra and Network Builder - Company Profile
Campotel: Telecommunications Infra and Network Builder - Company ProfileCampotel: Telecommunications Infra and Network Builder - Company Profile
Campotel: Telecommunications Infra and Network Builder - Company ProfileCampotelPhilippines
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...MarcovanHurne2
 
AI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvementAI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvementMimmo Squillace
 
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro KozhevinFwdays
 
LF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIELF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIEDanBrown980551
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxNeo4j
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stackSummit
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolProduct School
 
Imaging and Design for the Online Environment Part 1.pptx
Imaging and Design for the Online Environment Part 1.pptxImaging and Design for the Online Environment Part 1.pptx
Imaging and Design for the Online Environment Part 1.pptxPower Point
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...ISPMAIndia
 

Recently uploaded (20)

Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docxLeveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
Leveraging SLF4j for Effective Logging in IBM App Connect Enterprise.docx
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
 
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
"The Transformative Power of AI and Open Challenges" by Dr. Manish Gupta, Google
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptx
 
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
ASTRAZENECA. Knowledge Graphs Powering a Fast-moving Global Life Sciences Org...
 
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
 
Campotel: Telecommunications Infra and Network Builder - Company Profile
Campotel: Telecommunications Infra and Network Builder - Company ProfileCampotel: Telecommunications Infra and Network Builder - Company Profile
Campotel: Telecommunications Infra and Network Builder - Company Profile
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
 
AI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvementAI Act & Standardization: UNINFO involvement
AI Act & Standardization: UNINFO involvement
 
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
 
LF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIELF Energy Webinar: Introduction to TROLIE
LF Energy Webinar: Introduction to TROLIE
 
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stack
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product School
 
Imaging and Design for the Online Environment Part 1.pptx
Imaging and Design for the Online Environment Part 1.pptxImaging and Design for the Online Environment Part 1.pptx
Imaging and Design for the Online Environment Part 1.pptx
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
 

LIvRE: A Video Extension to the LIRE Content-Based Image Retrieval System

  • 1. LIVRE: A VIDEO EXTENSION TO THE LIRE CONTENT-BASED IMAGE RETRIEVAL SYSTEM Degree’s Final Project Dissertation Telecommunications Engineering Gabriel de Oliveira Supervisors: Assoc. Prof. Mathias Lux Assoc. Prof. Xavier Giró
  • 2. Outline of the Thesis 1. Introduction i. Motivation ii. Overview and previous work 2. Proposed solution: The LIvRE system i. Parsing ii. Indexing iii.Retrieval 3. Validation i. Dataset - Stanford I2V Newscasts dataset ii. Experiments - Quantitative evaluation - Qualitative evaluation - The thinking-aloud test. 4. Conclusions and Further Work March – October 2015 Slide 2
  • 3. Motivation Goal: To develop an all-in-one open source system for CBVR. • Server side requeriments: • Fast • Scalable • Flexible • Automated • User interface requeriments: • Fast • OS and device independent • Mobile Slide 3 Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 4. Overview and previous work Slide 6 • Open source CBIR Library in Java • Apache Lucene core • Solr plugin • Supports parsing, indexing and retrieval • Global and local descriptors • Web-based interface [1] Mathias Lux. LIRE: Open source image retrieval in java. In Proceedings of the 21st ACM international conference on Multimedia, pages 843{846.ACM, 2013. Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 5. Developed solution: The LIvRE CBVR system Slide 7 CBVR system - concept and requirements. database Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 6. Developed solution: The LIvRE CBVR system Slide 7 User sideServer side LIvRE CBVR system architecture. Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 7. Block 1: Parsing System Architecture - Parsing Slide 8 1. Find videos in any given folder structure. 2. Extract keyframes from those videos. 3. Parse extracted keyframes with selected image descriptors. - Color Layout, Edge Histogram, JCD and PHOG 4. Generate XML Documents with the Feature Vectors. Tools are provided to: Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 8. Block 2: Indexing Fig. System Architecture Slide 8 1. Find XML Documents containing the Feature Vectors (generated from Parsing Block). 2. Upload XML documents to Solr. 3. Commit changes in Solr core. Tools are provided to: Fig. System Architecture - Indexing Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 9. Block 3: Retrieval System Architecture - Retrieval Slide 8 1. Image search field. 2. Settings. User web-based interface input: Web-based user interface input as displayed on small screen devices. Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 10. Block 3: Retrieval System Architecture - Retrieval Slide 8 1. Image search field. 2. Settings. User web-based interface input: Web-based user interface input as displayed on small screen devices. Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 11. Block 3: Retrieval Slide 8 1. Candidate videos displayed using HTML5. 2. Thumbnails with other similar frames. 3. Time refinement. 4. Video information. User web-based results presentation: System Architecture - Retrieval Retrieval results presentation for small screen devices Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 12. Block 3: Retrieval Slide 8 1. Candidate videos displayed using HTML5. 2. Thumbnails with other similar frames. 3. Time refinement and ranking. 4. Video information. User web-based results presentation: Fig. System Architecture - Retrieval Fig. Retrieval results presentation for small screen devices Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 13. LIvRE CBVR system demo Introduction · Overview · LIvRE CBVR system · Dataset · Experiments · Conclusions Slide 9
  • 14. Validation Stanford I2V Dataset  Freely available data set.  Large (~1TB Video) • 23,443 video clips • Average video duration: 2,65min. • Keyframes @1fps: 3,808,760 • Video hours: 1,035h  Ground-truth • 78 queries Some query images and video frames from the Stanford I2V dataset. Slide 14 Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 15. Validation Experiments LIvRE CBVR system tested with 2 different evaluation methods: Slide 16 1 2 Quantitative evaluation Qualitative evaluation (Thinking-aloud Test) Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 16. Quantitative study: • Use ground-truth provided with the dataset for: • Scene Retrieval evaluation (finding the right video). • Time Refinement evaluation (finding the right moment of time at the right video). Qualitative study: • Web-based user interface. • Thinking-aloud Test (offline). • Participants are expert and non expert users. • 4 Non-expert users. • 2 Expert users. Slide 17 1 2 Validation Experiments Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 17. Quantitative study Slide 18 LIvRE Quantitative study evaluation process from provided ground-truth Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 18. Quantitative study Slide 18 1st Stage: Scene Retrieval Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 19. Quantitative study Slide 18 2nd Stage: Temporal Refinement Temporal Refinement results for 100k candidates Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 20. Qualitative study Thinking-aloud Test • Volunteer participants perform specific tasks with the web-based user interface. • LIvRE CBVR system is running locally (offline) on the machine. • Participants show their thoughts in loud-voice. • Sessions are recorded and evaluated. Slide 18 Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 21. Qualitative study Thinking-aloud Test Slide 19 Sample input query frames Screenshots from Thinking-aloud test 1 Introduction · Overview · LIvRE CBVR system · Validation · Conclusions Timing results for 50K candidates (in miliseconds)
  • 22. Conclusions and Future Work - New CBVR System, LIvRE, was developed as an extension of LIRE. - LIvRE is now a branch of the LIRE Solr Project. Slide 28 Future work: • Local image descriptors. • Integration of sound descriptors. • Simplified set-up and deployment. • Demo paper at ICMR 2016. • Add Video annotation tool. • Integration with computer vision / deep learning projects. Introduction · Overview · LIvRE CBVR system · Validation · Conclusions
  • 23. Thank you for your attention Do you have any question? 8 October 2015 LIvRE: A Video Extension to the LIRE Content-Based Image Retrieval System. Gabriel de Oliveira

Editor's Notes

  1. Intro to what is a CBVR system.
  2. http://es.slideshare.net/dermotte/lire-27544341?related=2 On this slide: Explain LIRE, the Lucene core, the LireSolr plugin, the descriptors flexibility, the web based interface for image retrieval
  3. In this slide: Explain the concepts and requirements for a CBVR system
  4. 3 Blocks: Parsing, Indexing and Retrieval (Querying) Server side: From the video dataset to the search engine. User side: Interface
  5. Given a video dataset, the first block, Parsing, performs the first step by taking this video dataset as input and outputs documents containing all the image features from the keyframes of each one of the videos.
  6. Given a running and set-up deployment of the Apache Solr search engine with the LireSolr plugin installed and configured, as well and the XML documents obtained during the parsing stage containing the image features of the keyframes, the user is given a tool to perform the following actions automatically:
  7. Given a previously indexed video dataset on a deployment of the Apache Solr search engine, with the LireSolr plugin installed and congured, the user must be given a web-based interface to perform the following actions:
  8. Given a previously indexed video dataset on a deployment of the Apache Solr search engine, with the LireSolr plugin installed and congured, the user must be given a web-based interface to perform the following actions:
  9. In addition, the web-based interface should independent from the device, OS, and web browser. It should as well be scalable and modular to be usable from any screen size.
  10. In addition, the web-based interface should independent from the device, OS, and web browser. It should as well be scalable and modular to be usable from any screen size.
  11. http://localhost:8983/solr/livre_evaluation.html http://localhost:8983/solr/livre5.html http://www.filmsnmovies.com/media/thumbs/1357673458.jpg file:///C:\Users\Gabriel\Desktop\schwarzenegger.jpg
  12. k is the rank in the sequence of retrieved documents, n is the number of retrieved documents, P(k) is the precision at cut-o k in the list, and r(k) is the change in recall from items k-1 to k. Although Average Precision assesses the quality of the returned ranked list of results and is useful in applications where a list of potential results is shown to the user, we also measure Precision at 1 (p@1), since it is important in cases where the best result is directly returned to the user (for example, in the case where the system would start playing the best clip match without further interaction with the user).
  13. Jaccard index is computed by the ratio between the intersection of the retrieved and ground truth sequences and their union.
  14. http://localhost:8983/solr/livre_evaluation.html http://localhost:8983/solr/livre5.html 10.183.94.148:8983/solr/livre55.html http://www.filmsnmovies.com/media/thumbs/1357673458.jpg file:///C:\Users\Gabriel\Desktop\schwarzenegger.jpg