MPEG Technologies and roadmap for Augmented RealityMarius Preda PhD
This is a presentation I did during the 5th ATStandards meeting in Austin, 2012. It contains the MPEG Vision on AR as well as a very short overview of MPEG technologies related to AR
Some ideas of how to bring the television closer to the web advancements, while preserving its own mission. Additionally, a set of MPEG tools covering aspects such as visual search, multimedia linking and multi-sensory experiences are also introduced.
We employ the SURF detector to define salient image patches of blob-like textures and use the MPEG-7 SCD, the MPEG-7 CLD, the MPEG-7 EHD and the CEDD descriptors to produce local features’ vectors named SIMPLE-SC, SIMPLE-CL, SIMPLE-EH and SIMPLE-CEDD or “LoCATe” (Local Color And Texture descriptor) respectively. In order to test the new descriptors in the most straightforward fashion, we use the Bag-Of-Visual-Words framework for indexing and retrieval. The experimental results conducted on two different benchmark databases, with varying codebook sizes revealed an astonishing boost in the retrieval performance of the proposed descriptors compared both to their own performance (in their original form) and to other state-of-the-art methods of local and global descriptors. From this page, one can download the open source implementation of the SIMPLE descriptors (C#, Java and MATLAB).
Prototype of a Wireless PC2TV solution. Extending your PC/laptop screen to a digital television or a projector at your home, office or an exhibition center.
MPEG Technologies and roadmap for Augmented RealityMarius Preda PhD
This is a presentation I did during the 5th ATStandards meeting in Austin, 2012. It contains the MPEG Vision on AR as well as a very short overview of MPEG technologies related to AR
Some ideas of how to bring the television closer to the web advancements, while preserving its own mission. Additionally, a set of MPEG tools covering aspects such as visual search, multimedia linking and multi-sensory experiences are also introduced.
We employ the SURF detector to define salient image patches of blob-like textures and use the MPEG-7 SCD, the MPEG-7 CLD, the MPEG-7 EHD and the CEDD descriptors to produce local features’ vectors named SIMPLE-SC, SIMPLE-CL, SIMPLE-EH and SIMPLE-CEDD or “LoCATe” (Local Color And Texture descriptor) respectively. In order to test the new descriptors in the most straightforward fashion, we use the Bag-Of-Visual-Words framework for indexing and retrieval. The experimental results conducted on two different benchmark databases, with varying codebook sizes revealed an astonishing boost in the retrieval performance of the proposed descriptors compared both to their own performance (in their original form) and to other state-of-the-art methods of local and global descriptors. From this page, one can download the open source implementation of the SIMPLE descriptors (C#, Java and MATLAB).
Prototype of a Wireless PC2TV solution. Extending your PC/laptop screen to a digital television or a projector at your home, office or an exhibition center.
OBJECT-MEDIA: FROM PERSONALISATION TO A SEAMLESS TV/VR CONVERGENCEJerry Foss
This presentation proposes in-video personalisation using object-based media for the insertion of externally sourced video placements into broadcast content (real-time or on-demand). Placement content is selected in accordance with viewer profiles, and may be produced independently of the source video.
Using this this technology user-object interaction is also possible. Use cases, include personalised product placement, training, education and accessibility for hearing and visually impaired viewers. The technology specifically fits existing distribution platforms (cable, satellite, DTTV, IPTV/OTT) with minimal infrastructure upgrade.
This presentation is meant to discuss the basics of video compression like DCT, Color space conversion, Motion Compensation etc. It also discusses the standards like H.264, MPEG2, MPEG4 etc.
I made this tutorial at Web3D 2012 conference. It provides MPEG position to AR, technologies currently used, as well as explanations on how to set up AR applications.
What's new in MPEG? A brief update about the results of its 131st MPEG meeting featuring:
- Welcome and Introduction: Jörn Ostermann, Acting Convenor of WG11 (MPEG)
- Versatile Video Coding (VVC): Jens-Rainer Ohm and Gary Sullivan, JVET Chairs
- MPEG 3D Audio: Schuyler Quackenbusch, MPEG Audio Chair
- Video-based Point Cloud Compression (V-PCC): Marius, Preda, MPEG 3DG Chair
- MPEG Immersive Video (MIV): Bart Kroon, MPEG Video BoG Chair
- Carriage of Versatile Video Coding (VVC) and Enhanced Video Coding (EVC): Young-Kwon Lim, MPEG Systems Chair
- MPEG Roadmap: Jörn Ostermann, Acting Convenor of WG11 (MPEG)
MPEG Web site: https://mpeg-standards.com/meetings/mpeg-131/
Interactive Content Authoring for A153 ATSC Mobile Digital Television Employi...Brad Fortner
Presentation to SMPTE Toronto on the pioneering work undertaken by the ATSC M/H content group formed at Ryerson University developing ATSC Mobile Data Content. Presentation occurred on April 10, 2012.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Monitoring Java Application Security with JDK Tools and JFR Events
MPEG-4 Developments
1. Overview of MPEG-4
Developments
Olivier Avaro
Chairman MPEG Systems
Deutsche Telekom - Berkom
Davic European Information Day
Geneva, 12 January 1999
Berkom
===!" §
2. Goals of the presentation
• Give a report of the status of MPEG-4 developments.
• Give a report of the status of the AIC Initiative.
• Some special highlights :
– Video and Systems tools,
– Carriage of MPEG-4 content over MPEG-2 and IP.
Berkom
===!" §
3. Messages
• MPEG-4 is not meant to replace existing architectures (ex :
MPEG-2 or IP based architecture).
• MPEG-4 augments existing architecture’s strong points with
new concepts (ex : Interactive Scene Description, new content
types, Java APIs).
• MPEG-4 extends existing architecture’s strong points to new
environments (ex : delivery independent representation of the
content).
Berkom
===!" §
4. Overview
• Introduction to MPEG-4 :
– History, Requirements and Architecture,
– MPEG-4 Video Version 1 tools,
– MPEG-4 Systems Version 1 tools.
• Carriage of MPEG-4 over MPEG-2 and IP.
• Introduction to the AIC Initiative :
– Objectives and architecture,
– Applications,
– Work plan.
• Annexes
– MPEG-4 Object Content Information (OCI)
– MPEG-4 Intellectual Property Management and Protection (IPMP)
Berkom
===!" §
5. A bit of history on MPEG-4
• Participants MPEG: Consumer Electronics, Computer
Industry, Telecom Operators, Academia.
• 1993 : started as ‘Very Low Bitrate Audio-visual Coding’.
• 1994: goal changed to ‘Coding of Audio-visual Objects ’.
• Now: Version 1 is ready; released for IS ballot at the end of
the month. Version 2 released for CD ballot in March.
‘TV/film’
AV-
data
From the MPEG-4
Call for Proposals
(drafted 1994) interactivity
Wireless
‘Computer’
‘Telecom’
Berkom
===!" §
6. Requirements
• Provide common technology for many types of services :
interactive, broadcast, conversational.
• Increase client-side and client-server interactivity.
• Integrate natural and synthetic content.
• Cover a wide range of access conditions.
• Provide serious solutions for the management and
protection of intellectual property.
Berkom
===!" §
7. MPEG-4 Principles
• Audio-visual scenes made of audio-visual objects composed
together according to a scene description :
– allows interaction with elements within the audio-visual scene,
– coding scheme can differ for individual objects,
– allows easy re-use of audio-visual content.
• Audio-visual objects can be of different nature :
– audio (single or multi-channel) or video (arbitrary shape or rectangular),
– natural (natural audio or video) or synthetic (text & graphics, animated faces,
synthetic music),
– 2D (Web like pages) or 3D (spatialized sound, 3D virtual world),
– streamed (video movie) or downloaded (audio jingle).
Berkom
===!" §
8. MPEG-4 Principles
• Scene Description provides :
– the spatial/temporal relationship between the audiovisual objects (2D, 3D,
mixed 2D and 3D scene description),
– the behavior and interactivity of the audio-visual objects and scenes,
– protocols to modify and animate the scene in time,
– a binary encoding for the scene.
• These principles are independent of the bit rate.
Berkom
===!" §
9. Architecture
Decoding
TransMux FlexMux Composition and
N Rendering
Primitive
e
t
AV Objects ...
w ... ...
o
r Ex: MPEG-2 Elementary
Transport Streams Scene Description Audiovisual Interactive Display and User
k Information Scene Interaction
Object Descriptor
Berkom
DAI
===!" §
10. Example of an MPEG-4
Audiovisual Scene (1)
2D Audio-visual scene 2D Audio-visual scene
Audio and Video + Scrolling Text Audio and Video + Still Images
and Still Images
Berkom
===!" §
11. Example of an MPEG-4
Audiovisual Scene (2)
3D Audio-visual scene
3D World + arbitrary shaped
video + still images + 3D
Objects
Berkom
===!" §
12. MPEG-4 Visual Version 1 Tools
• Natural video from 10 kbit/s - 10 Mbit/s
– Rectangular or arbitrary shape objects,
– Scalable,
– Interlaced and Progressive scan supported,
– ‘Sprites’ (e.g. backgrounds): send once, warp after.
• Computer-generated visual information
– facial animation,
– animated meshes with moving texture,
– animated text and graphics.
• All these objects can be synchronised together.
Berkom
===!" §
13. Example of MPEG-4 Video Objects
Arbitrary shape Animated Face
Rectangular shape video object
video object
Berkom
===!" §
14. Visual Version 2 Tools
• Body animation : to augment the face animation tool.
• Compression of 3D meshes with texture :
– games, higher quality virtual environments.
• Higher Quality Video
– currently under discussion.
• Ready end ‘99
• Note that V.2 is a compatible extension of V.1
Berkom
===!" §
15. MPEG-4 Systems Version 1Tools
• Decoder Model
– Predictable behavior of a decoder and decoded content,
– Management of decoder resources (buffer and memory),
– Tight synchronization of Audio, Video and synthetic elements.
• Object Descriptor Protocol
– Identification (location and stream type) of Elementary Streams
(content in the same scene can come from different sources),
– Association of Elementary Streams (alternative content, scalable
content, synchronization hierarchy),
– Association of the scene description and the streams,
– Object Content Information,
– Intellectual property protection and management.
Berkom
===!" §
16. MPEG-4 Systems Version 1Tools
• BIFS Scene Description
– VRML concepts : set of nodes to represent the primitive scene objects to
be composed, the scene graph constructs, the behavior and interactivity
+ Integration of streams,
+ 2D capabilities,
+ Integration of 2D and 3D,
+ Advanced Audio Features,
+ Timing model,
+ BIFS - Update protocol to modify the scene in time,
+ BIFS - Anim protocol to animate the scene in time,
+ Compression efficiency (BIFS is Binary Format for Scene).
Berkom
===!" §
17. Example of MPEG-4 Scenes (1)
2D Audio-visual scene
Audio and Video + Still Images
2D Audio-visual scene
Animated Text+ Video + Still Images
Berkom
===!" §
18. Example of MPEG-4 Scenes (2)
2D and 3D Graphics Primitive
Scene with Face Object
Web-like MPEG-4 Scene
Multi audio and video
MPEG-4 Scene with text
Berkom
===!" §
19. MPEG-4 Systems Version 2 Tools
• File format
– allows interchange and streaming of MPEG-4 content.
• Advanced BIFS
– New BIFS functionality : environmental spatialization, integration of
HTML and BIFS, chroma keying, application signaling,
– Advanced BIFS coding : PROTOs and ExternProtos, Integration of
Mesh, coding of MField.
• MPEG-J
– The MPEG-4 ‘Application Execution Engine’
– MPEG-4 specific API for interaction with the scene (complicated
content behavior), with the network / terminal / UI resources.
Berkom
===!" §
20. MPEG-J Architecture
MPEG-J Run Time Environment
Java MPEG-let
Terminal
Java Virtual Machine
DMIF Application Decoders Compositor
Interface
Berkom
===!" §
21. Profiles and Levels
• Most applications only need a part of the MPEG-4 tool set.
• Profiles define subsets useful for a large class of
applications/services.
• Types of Profiles:
– Scene description (e.g. behaviour),
– Object Descriptor (mainly timing models),
– Audio (natural and synthetic): types of objects,
– Visual (natural and synthetic): types of objects,
– Graphics.
• Levels limit the number of objects and complexity
• Profiles/Levels can be easily added when need arises
Berkom
===!" §
22. Profiles and Levels
MPEG-4 device
Scene
Description
Profiles
Main
speech simple 2D 3D
Audio
Audio
Profiles
core Core
hybrid 3D
FA 2D Object Descriptor
Profile
Visual Graphics
Profiles Profiles
Media Profiles
Berkom
===!" §
23. Carriage of MPEG-4 Content
• Transport of MPEG-4 content : left to other bodies
– Choose IP, MPEG-2 TS, ATM AAL5, …
• DMIF abstracts Application from delivery of data and provide
a uniform walkthrough to access content
– e.g. Broadcast, Internet protocol stack
• Create content once, play on any network configuration
Berkom
===!" §
24. Delivery Framework
SL-Packetized Streams Media
DAI
FlexMux FlexMux
FlexMux Streams
Protection sL Protection sL Delivery
(RTP) (PES) AAL2 H223 ….
UDP MPEG2 ATM PSTN
Mux sub Layer
IP TS
TransMux Streams
Berkom
===!" §
25. MPEG-4 over MPEG-2 and IP
• Technical issues to be addressed :
– Transport/Multiplexing/Packetization (ex : FlexMux, PES only or IETF
RTP multiplex),
– Synchronization (ex : Sync Layer, RTP sync, MPEG-2 sync),
– Identification and association of streams (ex : Object Descriptor, SDP).
• Solved by making tradeoff between :
– How far the integration should be done (ex : using MPEG-2 content or
RTP streams in MPEG-4 scenes, using the MPEG-4 content types in
MPEG-2 or RTP applications),
– How much the duplication of functionality can be reduced.
Berkom
===!" §
26. MPEG-4 over MPEG-2 and IP
• Examples
– Blind transmission of MPEG-4 Programs
• MPEG-4 content separate from others MPEG-2/RTP streams
– MPEG-4 Content (ES) in an MPEG-2 Systems /RTP application
• adding MPEG-4 content types to MPEG-2/RTP
– MPEG-4 ES + Systems in MPEG-2/RTP environment
• Adding MPEG-4 Interactive programming to MPEG-2/RTP
• MPEG-2/RTP ‘Objects’ may be used in the MPEG-4 application
• On-going activity within MPEG and IETF that should lead for
end 99 to :
– addenda to MPEG-2 and MPEG-4,
– IETF RFC.
Berkom
===!" §
27. AIC-I Charter
• Goal: Harmonize standards efforts and enable
interactive 2D, 3D and streaming content for commerce
and entertainment on set top devices.
• The Advanced Interactive Content Initiative (AIC Initiative)
applied to be an industry consortium operating under the
Industry Technical Agreement (ITA) of the International
Electro technical Commission (IEC).
• The purpose of the AIC Initiative is the development of
interactive content specifications for user devices to offer a
range of basic through advanced interactive applications using
3D as well as 2D content, in stored and streamed form.
Berkom
===!" §
28. AIC-I Applications
• Advanced Electronic Program Guide.
• Buy-me-button.
• Data Ticker.
• Program Enhancement.
• Interactive Home Shopping.
• Interactive Commercial.
• Interactive Entertainment.
• Demographic Programming.
Berkom
===!" §
29. AIC-I Principles
• Delivery in various transport environments
– including MPEG-2 and IP environments,
– and combinations of broadcast/interactive delivery.
• Presentation engine :
– One specification for integration of MPEG-4, ‘X-HTML’, VRML
content.
• Application execution engine
– Not the priority right now.
Berkom
===!" §
30. AIC-I Workplan
• 20th December: Delivery of Version 1.0.
• 8th-9th February: Delivery of Version 1.1.
• 21st-22nd of March: Field trial #1. Delivery of Version 1.2.
• May/June: Delivery of Version 1.3.
• 17th-18th of July: Delivery of Version 2.0.
• + Various demonstrations at NAB, IBC, CES ...
Berkom
===!" §
31. Conclusions
• First MPEG-4 applications will appear probably on the Internet
based on subsets of MPEG-4 integrated in the IP environment.
• In parallel, MPEG-4 can enhance MPEG-2 digital broadcast
with attractive and interactive content.
• Ongoing AIC-I activity, jointly with W3C, W3D, MPEG-4
will provide specification for advanced interactive content on
MPEG-2 and IP environments in an integrated way.
Berkom
===!" §
33. Credits
• The MPEG-4 Systems Sub-group.
• The European project MoMuSys.
• Material for the presentation
Text and pictures from slides courtesy of KPN Research
• Tape demonstration - shows possibilities
pictures courtesy of France Telecom CNET, Deutsche
Telekom, Instituto Superior Técnico
Berkom
===!" §
35. OCI - Object Content Information
• Waiting for MPEG-7 would be too long for some application,
MPEG-4 designed small, synchronised events containing
information descriptors.
• Carried in separate ES associated with an Object or in the ES
Descriptor of the Object.
• Types:
– content classification (entity + table number + info - open system! )
– rating, (entity + criteria + info)
– language (ISO 639)
– textual descriptors (language code + free text)
– keywords (ditto)
– creation of the content (who, date - no owner! )
Berkom
===!" §
36. IPMP - Intellectual Property
Management and Protection
• ‘IPMP’ is prerequisite for ‘publishing’ serious, valuable
content in digital form
– in any open environment - which nowadays even includes the CD!
• IPMP support integrated deeply into the MPEG-4 system
• Yet only the minimum is specified
– It makes no sense standardizing protection schemes, cryptography ,
etc.
– Only hooks are needed
• These hooks are already included in Version 1.
Berkom
===!" §