SlideShare a Scribd company logo
Embedded Media Markers: Marks on Paper that Signify Associated Media Qiong Liu, Chunyuan Liao, Lynn Wilcox, Anthony Dunnigan, Bee Liew FX Palo Alto Laboratory, Palo Alto, CA, USA
Motivation & Vision Multimedia and digital content displayed on cell phone User takes a picture of a document patch Motivation: Combine advantages of paper and cell phones Vision: Use Embedded Media Markers (EMMs) to link multimedia to paper patch  EMMs are meaningful marks on paper Use visual features inside EMM to identify document patch  Slide 2
What is EMM Embedded Media Markers (EMM) are optical-filter-like overlaid marks printed on paper documents.  Printed marks (EMMs) that improve usability and recognition accuracy Computationally efficient features (FIT) that identify document patches Slide 3 Media type (video) EMM-signified document location EMM-signified document patch
EMM Patch Identification Patch identification is based on image local feature (FIT)  Similar to SIFT features. More efficient. Robust to scaling, rotation and partial occlusion FIT features work on a broad range of document types Western/Asian text, images, photos  Slide 4
EMM Operation Flowchart Compute feature points Take a snapshot of EMM Match feature points against the document database Find the matched document patch Retrieve the linked video Play the video on the phone
Document Identification Methods Barcode QR-code Data Glyphs Anoto Pattern BWC EMM Marker-based Technologies Recognizable Visual Markers (e.g. Barcode, QR-Code) Nearly Invisible Fiduciary Markers (e.g. DataGlyphs, Anoto) Content-based Technologies Text-words-based Features (e.g. BWC) Image-based Features (e.g. SIFT, FIT) EMM is a hybrid approach Provide users multimedia cues and interaction guidance Save computation and 	improve performance Minimize the interference with 	document content SIFT
EMM Design Guidelines Major Entities User Friendly – guide users for media finding and query. Document Friendly –minimize changes to document content and format. Machine Friendly – save computation resources for document identification process.
EMM Components’ Functions feature-boundary-circle  media-type-icon Anchor point Top View of a 3-level EMM Model Feature boundary mark – framing feature capture Media type mark Anchor point – associate media with an accurate location. Graphical effects – attract attention and help separate content and marker.
Semi-automatically arrange EMM based on feature distribution Criterion 1 – Minimize author’s effort. Author only selects an EMM anchor point. Criterion 2 – Minimize machine resources used for patch query. Find a small feature boundary, and index the small number of keypoints inside the feature boundary. Criterion 3 – Minimize the EMM interference to document. Minimize the number of keypoints overlaid by an EMM. EMM Authoring Tool
Parameters for EMM Placement Anchor point location (m,n) – provided by a user Feature-boundary-circle center (X,Y) Feature-boundary-circle radius R Media-type-circle center (x,y)
Parameter Optimization EMM boundary optimization Find the center (X,Y) and the minimum radius R such that The number of keypoints in the circle exceeds the threshold to ensure patch identification accuracy The user selected anchor point is contained in the boundary Media type icon placement Select the media-type-icon center (x,y) so that the icon covers the minimum number of keypoints
Parameter Optimization Speedup Counting the number of keypoints in a circle is time consuming  O(2N). N is the number of keypoints in a page. Speedup Inscribed square to estimate the number of keypoint in a circle Use the cumulative feature-point histogram IABCD =IC-IB-ID+IA Optimization of media type circle is similar A B IP P C D B A Boundary Circle C D
Video Slide 13 http://www.youtube.com/watch?v=oub-2b6wQIY
2188 EMMs in the database At randomly selected 2188 document locations of2188 pages 110 EMM Overlaid Page Printouts 90 1-up + 20 4-up on letter-size paper 110 Queries Either in an office or open patio w/natural lighting conditions EMM Evaluation in Real Life
The Missed One Include many less distinctive keypoints Correctly identified after position adjustment Slide 15
Related Work University of Oldenberg “Bookmarkr” Link photobook picture to digital photo. Use SIFT features ACM Multimedia 08 Ricoh “HotPaper” Link personal media to text. Use BWC (Brick Wall Coding) Only works on Western text ACM Multimedia 08
Document ID Technology Comparison
Technology Comparison - cont * SP-code requires high-res printer and scanner
Summary EMMs are nearly transparent iconic marks printed on paper document to link various multimedia to specific document locations EMMs are robust to scaling, rotation and partial occlusion, and applicable to a broad range of document types Unlike barcodes, EMMs do not change document layouts and are less visually obtrusive Unlike other feature-based approaches, EMMs explicitly signify the media-augmented document regions and guide user interaction EMM can improve the efficiency of computation
Thank You

More Related Content

Similar to EMM - Embedded Media Markers

Empowerment Technologies Lecture 7 (Philippines SHS)
Empowerment Technologies Lecture 7 (Philippines SHS)Empowerment Technologies Lecture 7 (Philippines SHS)
Empowerment Technologies Lecture 7 (Philippines SHS)
John Bosco Javellana, MAEd.
 
What is IETM and What is Not IETM.pdf cnp
What is IETM and What is Not IETM.pdf cnpWhat is IETM and What is Not IETM.pdf cnp
What is IETM and What is Not IETM.pdf cnp
s1000dcodeandpixels
 
Integra Micro Software Services (P) Ltd. - Imaging Expertise
Integra Micro Software Services (P) Ltd. - Imaging ExpertiseIntegra Micro Software Services (P) Ltd. - Imaging Expertise
Integra Micro Software Services (P) Ltd. - Imaging Expertise
sreesinbox
 
Optical Recognition of Handwritten Text
Optical Recognition of Handwritten TextOptical Recognition of Handwritten Text
Optical Recognition of Handwritten Text
IRJET Journal
 
Computer
ComputerComputer
Computer
kika pusa
 
Developing Tools for “What if…” Testing of Large-scale Software Systems
Developing Tools for “What if…” Testing of Large-scale Software SystemsDeveloping Tools for “What if…” Testing of Large-scale Software Systems
Developing Tools for “What if…” Testing of Large-scale Software Systems
James Hill
 
IRJET-Raspberry Pi Based Reader for Blind People
IRJET-Raspberry Pi Based Reader for Blind PeopleIRJET-Raspberry Pi Based Reader for Blind People
IRJET-Raspberry Pi Based Reader for Blind People
IRJET Journal
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 Documentation
Salocin Dot TEN
 
IRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCR
IRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCRIRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCR
IRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCR
IRJET Journal
 
Training Agenda
Training AgendaTraining Agenda
Training Agenda
Roberto Arias García
 
SW_Cost_Estimation.ppt
SW_Cost_Estimation.pptSW_Cost_Estimation.ppt
SW_Cost_Estimation.ppt
PerumalPitchandi
 
IRJET-Feature Extraction from Video Data for Indexing and Retrieval
IRJET-Feature Extraction from Video Data for Indexing and Retrieval IRJET-Feature Extraction from Video Data for Indexing and Retrieval
IRJET-Feature Extraction from Video Data for Indexing and Retrieval
IRJET Journal
 
How to prepare a perfect video abstract for your research paper – Pubrica.pptx
How to prepare a perfect video abstract for your research paper – Pubrica.pptxHow to prepare a perfect video abstract for your research paper – Pubrica.pptx
How to prepare a perfect video abstract for your research paper – Pubrica.pptx
Pubrica
 
Video Content Identification using Video Signature: Survey
Video Content Identification using Video Signature: SurveyVideo Content Identification using Video Signature: Survey
Video Content Identification using Video Signature: Survey
IRJET Journal
 
IRJET - Information Hiding in H.264/AVC using Digital Watermarking
IRJET -  	  Information Hiding in H.264/AVC using Digital WatermarkingIRJET -  	  Information Hiding in H.264/AVC using Digital Watermarking
IRJET - Information Hiding in H.264/AVC using Digital Watermarking
IRJET Journal
 
Effective Compression of Digital Video
Effective Compression of Digital VideoEffective Compression of Digital Video
Effective Compression of Digital Video
IRJET Journal
 
Prior AI consulting use cases
Prior AI consulting use casesPrior AI consulting use cases
Prior AI consulting use cases
Harendra Singh
 
Aps11 design interface
Aps11 design interfaceAps11 design interface
Aps11 design interface
Arif Rahman
 
Design Steps for any MultiMedia Applications
Design Steps for any MultiMedia ApplicationsDesign Steps for any MultiMedia Applications
Design Steps for any MultiMedia Applications
techbirbal
 
Overview of Selected Current MPEG Activities
Overview of Selected Current MPEG ActivitiesOverview of Selected Current MPEG Activities
Overview of Selected Current MPEG Activities
Alpen-Adria-Universität
 

Similar to EMM - Embedded Media Markers (20)

Empowerment Technologies Lecture 7 (Philippines SHS)
Empowerment Technologies Lecture 7 (Philippines SHS)Empowerment Technologies Lecture 7 (Philippines SHS)
Empowerment Technologies Lecture 7 (Philippines SHS)
 
What is IETM and What is Not IETM.pdf cnp
What is IETM and What is Not IETM.pdf cnpWhat is IETM and What is Not IETM.pdf cnp
What is IETM and What is Not IETM.pdf cnp
 
Integra Micro Software Services (P) Ltd. - Imaging Expertise
Integra Micro Software Services (P) Ltd. - Imaging ExpertiseIntegra Micro Software Services (P) Ltd. - Imaging Expertise
Integra Micro Software Services (P) Ltd. - Imaging Expertise
 
Optical Recognition of Handwritten Text
Optical Recognition of Handwritten TextOptical Recognition of Handwritten Text
Optical Recognition of Handwritten Text
 
Computer
ComputerComputer
Computer
 
Developing Tools for “What if…” Testing of Large-scale Software Systems
Developing Tools for “What if…” Testing of Large-scale Software SystemsDeveloping Tools for “What if…” Testing of Large-scale Software Systems
Developing Tools for “What if…” Testing of Large-scale Software Systems
 
IRJET-Raspberry Pi Based Reader for Blind People
IRJET-Raspberry Pi Based Reader for Blind PeopleIRJET-Raspberry Pi Based Reader for Blind People
IRJET-Raspberry Pi Based Reader for Blind People
 
CSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 DocumentationCSE 5930 Assignment 2 Documentation
CSE 5930 Assignment 2 Documentation
 
IRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCR
IRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCRIRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCR
IRJET- Proposed Approach for Layout & Handwritten Character Recognization in OCR
 
Training Agenda
Training AgendaTraining Agenda
Training Agenda
 
SW_Cost_Estimation.ppt
SW_Cost_Estimation.pptSW_Cost_Estimation.ppt
SW_Cost_Estimation.ppt
 
IRJET-Feature Extraction from Video Data for Indexing and Retrieval
IRJET-Feature Extraction from Video Data for Indexing and Retrieval IRJET-Feature Extraction from Video Data for Indexing and Retrieval
IRJET-Feature Extraction from Video Data for Indexing and Retrieval
 
How to prepare a perfect video abstract for your research paper – Pubrica.pptx
How to prepare a perfect video abstract for your research paper – Pubrica.pptxHow to prepare a perfect video abstract for your research paper – Pubrica.pptx
How to prepare a perfect video abstract for your research paper – Pubrica.pptx
 
Video Content Identification using Video Signature: Survey
Video Content Identification using Video Signature: SurveyVideo Content Identification using Video Signature: Survey
Video Content Identification using Video Signature: Survey
 
IRJET - Information Hiding in H.264/AVC using Digital Watermarking
IRJET -  	  Information Hiding in H.264/AVC using Digital WatermarkingIRJET -  	  Information Hiding in H.264/AVC using Digital Watermarking
IRJET - Information Hiding in H.264/AVC using Digital Watermarking
 
Effective Compression of Digital Video
Effective Compression of Digital VideoEffective Compression of Digital Video
Effective Compression of Digital Video
 
Prior AI consulting use cases
Prior AI consulting use casesPrior AI consulting use cases
Prior AI consulting use cases
 
Aps11 design interface
Aps11 design interfaceAps11 design interface
Aps11 design interface
 
Design Steps for any MultiMedia Applications
Design Steps for any MultiMedia ApplicationsDesign Steps for any MultiMedia Applications
Design Steps for any MultiMedia Applications
 
Overview of Selected Current MPEG Activities
Overview of Selected Current MPEG ActivitiesOverview of Selected Current MPEG Activities
Overview of Selected Current MPEG Activities
 

Recently uploaded

National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
Matthew Sinclair
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 

Recently uploaded (20)

National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 202420240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 

EMM - Embedded Media Markers

  • 1. Embedded Media Markers: Marks on Paper that Signify Associated Media Qiong Liu, Chunyuan Liao, Lynn Wilcox, Anthony Dunnigan, Bee Liew FX Palo Alto Laboratory, Palo Alto, CA, USA
  • 2. Motivation & Vision Multimedia and digital content displayed on cell phone User takes a picture of a document patch Motivation: Combine advantages of paper and cell phones Vision: Use Embedded Media Markers (EMMs) to link multimedia to paper patch EMMs are meaningful marks on paper Use visual features inside EMM to identify document patch Slide 2
  • 3. What is EMM Embedded Media Markers (EMM) are optical-filter-like overlaid marks printed on paper documents. Printed marks (EMMs) that improve usability and recognition accuracy Computationally efficient features (FIT) that identify document patches Slide 3 Media type (video) EMM-signified document location EMM-signified document patch
  • 4. EMM Patch Identification Patch identification is based on image local feature (FIT) Similar to SIFT features. More efficient. Robust to scaling, rotation and partial occlusion FIT features work on a broad range of document types Western/Asian text, images, photos Slide 4
  • 5. EMM Operation Flowchart Compute feature points Take a snapshot of EMM Match feature points against the document database Find the matched document patch Retrieve the linked video Play the video on the phone
  • 6. Document Identification Methods Barcode QR-code Data Glyphs Anoto Pattern BWC EMM Marker-based Technologies Recognizable Visual Markers (e.g. Barcode, QR-Code) Nearly Invisible Fiduciary Markers (e.g. DataGlyphs, Anoto) Content-based Technologies Text-words-based Features (e.g. BWC) Image-based Features (e.g. SIFT, FIT) EMM is a hybrid approach Provide users multimedia cues and interaction guidance Save computation and improve performance Minimize the interference with document content SIFT
  • 7. EMM Design Guidelines Major Entities User Friendly – guide users for media finding and query. Document Friendly –minimize changes to document content and format. Machine Friendly – save computation resources for document identification process.
  • 8. EMM Components’ Functions feature-boundary-circle media-type-icon Anchor point Top View of a 3-level EMM Model Feature boundary mark – framing feature capture Media type mark Anchor point – associate media with an accurate location. Graphical effects – attract attention and help separate content and marker.
  • 9. Semi-automatically arrange EMM based on feature distribution Criterion 1 – Minimize author’s effort. Author only selects an EMM anchor point. Criterion 2 – Minimize machine resources used for patch query. Find a small feature boundary, and index the small number of keypoints inside the feature boundary. Criterion 3 – Minimize the EMM interference to document. Minimize the number of keypoints overlaid by an EMM. EMM Authoring Tool
  • 10. Parameters for EMM Placement Anchor point location (m,n) – provided by a user Feature-boundary-circle center (X,Y) Feature-boundary-circle radius R Media-type-circle center (x,y)
  • 11. Parameter Optimization EMM boundary optimization Find the center (X,Y) and the minimum radius R such that The number of keypoints in the circle exceeds the threshold to ensure patch identification accuracy The user selected anchor point is contained in the boundary Media type icon placement Select the media-type-icon center (x,y) so that the icon covers the minimum number of keypoints
  • 12. Parameter Optimization Speedup Counting the number of keypoints in a circle is time consuming O(2N). N is the number of keypoints in a page. Speedup Inscribed square to estimate the number of keypoint in a circle Use the cumulative feature-point histogram IABCD =IC-IB-ID+IA Optimization of media type circle is similar A B IP P C D B A Boundary Circle C D
  • 13. Video Slide 13 http://www.youtube.com/watch?v=oub-2b6wQIY
  • 14. 2188 EMMs in the database At randomly selected 2188 document locations of2188 pages 110 EMM Overlaid Page Printouts 90 1-up + 20 4-up on letter-size paper 110 Queries Either in an office or open patio w/natural lighting conditions EMM Evaluation in Real Life
  • 15. The Missed One Include many less distinctive keypoints Correctly identified after position adjustment Slide 15
  • 16. Related Work University of Oldenberg “Bookmarkr” Link photobook picture to digital photo. Use SIFT features ACM Multimedia 08 Ricoh “HotPaper” Link personal media to text. Use BWC (Brick Wall Coding) Only works on Western text ACM Multimedia 08
  • 18. Technology Comparison - cont * SP-code requires high-res printer and scanner
  • 19. Summary EMMs are nearly transparent iconic marks printed on paper document to link various multimedia to specific document locations EMMs are robust to scaling, rotation and partial occlusion, and applicable to a broad range of document types Unlike barcodes, EMMs do not change document layouts and are less visually obtrusive Unlike other feature-based approaches, EMMs explicitly signify the media-augmented document regions and guide user interaction EMM can improve the efficiency of computation