1.
Video Object
Linguistic GroundingWorkshop on
Multimodal
Understanding &
Learning for
Embodied
Applications
(MULEA)
Nice, France
25 October 2019
Carles
Ventura
Alba M.
Herrera
Xavier
Giro-i-Nieto

2.
2
HCI with Natural Language

3.
#RVOS Carles Ventura, Miriam Bellver, Andreu Girbau, Amaia Salvador, Ferran Marques and Xavier Giro-i-Nieto. “RVOS:
End-to-End Recurrent Network for Video Object Segmentation”, CVPR 2019.

4.
4
From Masks to Referring Expressions
Model
time
Model
One-shot RVOS [*]
Model
time
Model
Referring expression
“the woman”
[*] #RVOS Carles Ventura, Miriam Bellver, Andreu Girbau, Amaia Salvador, Ferran Marques and Xavier Giro-i-Nieto. “RVOS:
End-to-End Recurrent Network for Video Object Segmentation”, CVPR 2019.

5.
5
Related Work
Khoreva, A., Rohrbach, A., & Schiele, B. Video object segmentation with language referring expressions. ACCV 2018.

6.
6
Image Segmentation with Refers
#MAttNet Yu, L., Lin, Z., Shen, X., Yang, J., Lu, X., Bansal, M., & Berg, T. L. . Mattnet: Modular attention network for referring
expression comprehension. CVPR 2018

7.
7
Proposed Approach: MAttNet + RVOS

8.
8
Qualitative Results
FRAME 0 FRAME 2 FRAME 4
REFERRING EXPRESSIONS:
"a brown deer on the left"
"a brown deer on the right with branched horns"

9.
9
Quantitative Results

10.
Merci beaucoup
xavier.giro@upc.edu
@DocXavi
Xavier
Giro-i-Nieto