EPICentre's Multi-Modal HEVS for Immersive Visualisation
•Download as PPTX, PDF•
0 likes•91 views
The High-End Visualisation System (HEVS) is a multi-modal API developed by the Expanded Perception and Interaction Centre (EPICentre) to simplify deployment of visualizations across different platforms. HEVS supports desktop, VR/AR, domes, touch displays, and clustered environments. It handles configuration, input handling, and synchronization to allow projects to run on any platform without recompilation. Additional libraries provide asset sharing, collaborative sessions, and remote administration of visualization sites. HEVS aims to accelerate scientific discovery through unified access to advanced visualization resources.
Recommended
Recommended
More Related Content
Similar to EPICentre's Multi-Modal HEVS for Immersive Visualisation
Similar to EPICentre's Multi-Modal HEVS for Immersive Visualisation (20)
More from Tomasz Bednarz
More from Tomasz Bednarz (14)
Recently uploaded
Recently uploaded (20)
EPICentre's Multi-Modal HEVS for Immersive Visualisation
- 1. http://epicentre.matters.today Expanded Perception and Interaction Centre Multi-Modal High-End Visualisation System Tomasz Bednarz Director, EPICentre UNSW | Team Leader, CSIRO’s Data61 Luc Betbeder-Matibet Director Infrastructure, EPICentre UNSW | Visiting Scientist, CSIRO’s Data61
- 2. UNSW Paddington Campus EPICentre Block C a heritage building at UNSW Art & Design in Paddington Sydney
- 3. a heritage building at UNSW Art & Design in Paddington Sydney UNSW Shared Facility Multi-modal Multi-disciplinary Visual Analytics, Visualisation + Simulation + AI, Media Arts, Virtual Reality and Augmented Reality, Human in the Loop, HCI Research, Digital and Interaction Design, High Performance Visualisation. Expanded Perception and Interaction Centre
- 4. Our Mission To solve complex problems using visualisation, CG and modern computational platforms. To deliver interdisciplinary tools acting as enablers accelerating scientific discovery. Leading community of practice connecting visualisation, AI, modelling, and simulation.
- 5. http://epicentre.matters.today/hevs/ Best URL for EPICentre and HEVS
- 6. EPICYLINDER DOME LAB XR LAB COMPUTER GRAPHICS LAB DIGITAL TWINS LAB EPICENTRE LABS
- 7. ASKAP, CSIRO – Digital Twin • Digital Twin of CSIRO's Australian Square Kilometer Array Pathfinder antennas with realistic Western Australia landscape and astronomically accurate positions of the sun and the moon • Ability to replay past observations and plan future observations • Designed cross-platform for PC, VR, Mixed Reality and clustered display devices • Initially created for permanent interactive display at the CSIRO Parkes Radio Telescope Visitor's Centre • Planned extension for remote monitoring and remote control of antennas array
- 8. GENOMICS VIEWER Genomics Viewer is an innovative fully interactive and ultra-high resolution navigation tool has been developed to browse and analyse gene expression levels from human cancer cells, acting as a visual microscope on data.
- 9. Pipeline architecture for visualizing very large-scale graphs in an immersive environment. MASSIVE NETWORKS HEVS – UNSW EPICentre | Tomasz Bednarz
- 10. What is HEVS? • High-End Visualisation System (HEVS) • A multi-modal API for multiple use-cases, such as: – Immersive and collaborative data visualisation, – Display walls, CAVE-like projection- based installations, – Easily port applications between platforms, – Integrated interaction devices, – Supported systems: Windows/Linux/OSX/Android/iOS http://epicentre.matters.today/hevs HEVS – UNSW EPICentre | Tomasz Bednarz
- 11. What is HEVS? • Multi-modal platforms can include • Desktop/CAVE/Projection/Dome/XR • Single-node or Clustered • Multi-user/interconnected platforms for collaborative experiences • Various methods of interaction – Controllers/Touch/Gestures/Voice • Current iterations • Unity3D 2019 - 2021 • OpenGL/Vulkan Low-level APIs • Potential future Python wrapper http://epicentre.matters.today/hevs HEVS – UNSW EPICentre | Tomasz Bednarz
- 12. High-End Visualisation System (HEVS) • Aims to simplify deployment to various visualisation platforms without requiring per-project setup by the project author • Including unified input and interaction methods • APP layer adds additional functionality • Record and annotate visualisations for story development and demo playback • Cloud-based project and asset library • Graphing and Plot libraries • Collaborative cross-platform applications – Multiple platforms can connect and share state and communication Develop Visualisation Deploy XR Dome Cluster Network Collaborative Interactions between Platforms Desktop HEVS – UNSW EPICentre | Tomasz Bednarz
- 13. High-End Visualisation System (HEVS) • Highly configurable and extendable run-time platform definitions • Configuration is read on-load to reconfigure the application’s environment based on selected target platform • A single compiled executable can run as a cluster in a CAVE, on a dome, in VR, or on a large touch-wall, all without having to rebuild or change the way the scene was built – Input can also be reconfigured, including tracked sources • “Build-once run-everywhere*” HEVS – UNSW EPICentre | Tomasz Bednarz
- 14. Multi-modal display Support • HEVS supports various display types: • Standard displays – Desktop / tablet / flat projection • Immersive displays – CAVE-like where the display is based off a user’s perspective • Projection displays – Warp & blend capabilities – Curved surfaces – Hemispherical domes • Head-mounted Displays (HMD) – VR, MR and AR supported HEVS – UNSW EPICentre | Tomasz Bednarz
- 15. Immersive Displays • Single and multi-user immersive displays • CAVE-like displays that surround the user/s • Mono and Stereo support • Head tracking • Multi-user gaze tracking HEVS – UNSW EPICentre | Tomasz Bednarz
- 16. Projection Displays • Multi-projector configurations • Custom warp & blend tools • Immersaview and VIOSO warp & blend integrations • Project onto: • Curved cylindrical displays • Domes • Installation environments • Hemispherical Domes • Single fisheye projector or multiple overlapping projectors HEVS – UNSW EPICentre | Tomasz Bednarz
- 17. Touch Displays • Support for touch-wall displays • Touch interactions can mimic mouse and VR pointer interactions • Seamless portability between desktop/VR/CAVE and touch walls • Touch can come from networked sources – Tablet / phone HEVS – UNSW EPICentre | Tomasz Bednarz
- 18. Stereoscopic Rendering Support • Multiple implementations of stereoscopic 3D are available where required • Quad-buffered* – Requires compatible Nvidia Quadro hardware • Side-by-side • Top/bottom • Anaglyphs • Can change at run-time between mono and stereo Responsible Technology through Visual Analytics and Computational Angles | Tomasz Bednarz 18 |
- 19. Clustering • Platforms can run as a cluster of Client nodes connected to a single Master node • A cluster supports: • Software or hardware* frame sync – Hardware frame synchronisation currently requires Nvidia Quadro GPU and OpenGL support • Time & Input synced • Object and variable synchronisation • Remote Procedure Calls (RPC) HEVS – UNSW EPICentre | Tomasz Bednarz
- 20. Interaction Layer • Multiple interaction methods supported • Keyboard, Mouse, Controllers • Tracked objects – Tracking data can come from XR trackers, motion capture systems and alternate sources such as OSC and VRPN • Touch • Voice Commands • OpenSoundControl (OSC) and Virtual Reality Peripheral Network (VRPN) allows input to come from networked sources • Sources are changeable within the configuration file, rather than within the compiled project • New interaction devices can be added and extended HEVS – UNSW EPICentre | Tomasz Bednarz
- 21. APP Layer • Additional features that extend HEVS • HEVS Portal launcher and daemon • Demo Tool to record, annotate and playback sessions • Collaborative module & Web-based Session Manager • Plot and data visualisation components • Compute integration • Web-based HEVS Library for sharing Assets and Projects via the cloud • Web-based HEVS Site Manager HEVS – UNSW EPICentre | Tomasz Bednarz
- 22. HEVS Library • HEVS Library also provides a full REST API for interfacing with it through code • This way users can browse the Library, and then download and display assets all within a HEVS application • Applications can use the metadata fields for communication HEVS – UNSW EPICentre | Tomasz Bednarz
- 23. HEVS Site Manager • A web-based service for managing HEVS Sites • HEVS Sites can choose to publish the current state of each of their Installations, so anyone with permission can see what applications are currently running • In conjunction with the HEVS Portal daemon, this can also be used to manage remote sites (start and stop applications, push new applications, distribute data etc) HEVS – UNSW EPICentre | Tomasz Bednarz
- 24. HEVS Session Manager • Acts as a “Game Lobby” for collaborative projects • Applications can announce on the Session Manager that they are hosting a session and are ready for others to join • Applications can also search for other sessions that are waiting HEVS – UNSW EPICentre | Tomasz Bednarz
- 25. HEVS Library - Use Cases Use cases include • Someone creates a dataset and wants to be able to view it in multiple display environments Create a new asset, leave it as private, view it inside HEVS application using Asset ID • Someone creates a dataset or application and wants to share it with others Create a new asset, set it to public, share the Asset ID and/or URL • Using a mobile device to interact with data in a CAVE-like system Mobile device and CAVE both download and view the same asset. When changes are made on the mobile device, these changes are set on the asset metadata fields through HTTP PATCH requests. These changes are then detected and applied on the CAVE. HEVS – UNSW EPICentre | Tomasz Bednarz
- 26. Expanded Perception and Interaction Centre http://epicentre.matters.today/hevs Thanks eResearch 2021 Contact us with any questions t.bednarz@unsw.edu.au
Editor's Notes
- Hi everyone. I’m Luc Betbeder, I’m the Director of our Research Technology Services Team at UNSW in Sydney. Or in our conference language I’m the eResearch Director. But for this talk I have an even more cool role which is Director of Infrastructure for our EPICentre. And today I get to give you an intro about EPICentre and set up the presentation for Tomasz who will cover the main aspects of HEVS our High End Visualisation System.
- For those that haven’t visited EPICentre we extend to you a warm invitation to join us in person as soon as local restrictions permit to our Paddington Campus in Sydney. EPICentre is located in that white heritage school building at the back of the courtyard.
- EPICentre is a shared UNSW research centre with significant investments in different visualisation and interaction modalities. EPICentre's major research facilities are accessible to researchers based at UNSW, other universities and to our government and industry partners. The work we do means that we cut across a number of disiplines And over the last few years we have worked with researchers in Medicine, Science and Engineering as well as Art&Design and external agencies.
- Our missions at EPICentre are To solve complex problems using visualisation, CG and modern computational platforms. And To deliver interdisciplinary tools acting as enablers accelerating scientific discovery. And we do this by Leading community of practice connecting visualisation, AI, modelling, and simulation
- And we do this with some of the coolest people on the planet and by working on awesome projects. This is a shout out to the EPICentre team I miss you all and hope to see you in person soon.
- EPICentre is home to some of the best Visualisation infrastructure nationally and globally. Our EPICylinder is still one of the highest resolution screens in the world able to render 120 millon pixels in 3D and we are pleased to announce that we will be upgrading our cluster infrastructure in early 2022. We have just ordered new systems and will be running the EPICylinder and DomeLab with NVIDIA RTX A6000 cards which is a big step up from our M6000 cards which have served us well for the last 5 years. As well as the Cylinder and Dome Systems we also work with Large multi-touch interactive walls, HMD systems and a wider variety of interfaces to interact with data in 2D and 3D space.
- Over the last few years we have been especially active in Digital Twins projects including work to build a digital twin of ASKAP the Australian Square Kilometer Array Pathfinder antennas with realistic Western Australia landscape and astronomically accurate positions of the sun and the moon and an Ability to replay past observations and plan future observations
- EPICylinder has been used on several groundbreaking projects including this interactive ultra-high resolution navigation viewer for browsing and analyzing gene expression levels from human cancer cells. Our viewer also allows the user to view the level of RNA expression at a particular coordinate inside the DNA sequence, and also the locations of Exons/Introns/UTRs when zoomed on a gene. The basic interactions are usually carried out using a game controller that is being tracked in the space by our tracking system (VICON). The application also can use Microsoft HoloLens, with voice/gesture recognition substantially speeding up browsing time. The user can select a chromosome of interest, zoom in or out on any region of interest within a chromosome, zoom in on a spe- cific gene, save/load a bookmark for the selected region of interest, highlight significant expression levels by changing scale between linear and logarithmic, and also changing heat map colour.
- And this work which demonstrates a pipeline architecture for visualizing very large-scale graphs in an immersive environment. The contribution here is in utilizing GPUs for real-time cluster-based interactive rendering and intermediate graph representation using Khronos Group’s GLTF file format, with new interaction design from the EPICentre team. I’m going to pass you onto Tomasz now to tell you about HEVS.
- For those that haven’t visited EPICentre we extend you a warm invitation to join us in person as soon as local restrictions permit at our Paddington Campus in Sydney.