3DUI 2015 Keynote talk given by Mark Billinghurst on March 24th 2015, as part of the 3DUI 2015 conference. The talk is a survery of Augmented Reality and Empathic Computing
Lecture given by Mark Billinghurst on May 21st 2015 as part of the Gibbon's Lecture series on Human Computer Interaction. See https://www.cs.auckland.ac.nz/gibbons_lectures/
On how computer interfaces have evolved from interaction to systems that understand users
COSC 426 Lecture 1: Introduction to Augmented RealityMark Billinghurst
This is the first lecture of the COSC 426 graduate course on Augmented Reality taught at the University of Canterbury. It was taught by Mark Billinghurst on July 17th 2014. It covers a basic introduction to Augmented Reality.
AR101 Lecture - Introduction to Augmented Reality. Lecture providing an introduction to AR, the history of AR and some example applications. Presented by Mark Billinghurst at the AR101 summer school at the ISMAR 2016 conference, September 18th 2016.
Lecture 1 of the COMP 4010 course on AR and VR. This lecture provides an introduction to AR/VR/MR/XR. The lecture was taught at the University of South Australia by Mark Billinghurst on July 21st 2021.
Lecture given by Mark Billinghurst on May 21st 2015 as part of the Gibbon's Lecture series on Human Computer Interaction. See https://www.cs.auckland.ac.nz/gibbons_lectures/
On how computer interfaces have evolved from interaction to systems that understand users
COSC 426 Lecture 1: Introduction to Augmented RealityMark Billinghurst
This is the first lecture of the COSC 426 graduate course on Augmented Reality taught at the University of Canterbury. It was taught by Mark Billinghurst on July 17th 2014. It covers a basic introduction to Augmented Reality.
AR101 Lecture - Introduction to Augmented Reality. Lecture providing an introduction to AR, the history of AR and some example applications. Presented by Mark Billinghurst at the AR101 summer school at the ISMAR 2016 conference, September 18th 2016.
Lecture 1 of the COMP 4010 course on AR and VR. This lecture provides an introduction to AR/VR/MR/XR. The lecture was taught at the University of South Australia by Mark Billinghurst on July 21st 2021.
A presentation given by Mark Billinghurst on April 21st 2015 at the CHI 2015 conference. This talk presents highlights from the journal paper:
M. Billinghurst, A. Clark, and G. Lee. A Survey
of Augmented Reality, Foundations and
Trends in Human-Computer Interaction.
Vol. 8, No. 1 (2015) 1–202, 2015
Available at :http://www.nowpublishers.com/article/Details/HCI-049
This is my presentation for augmented reality and its applications in education. It took place in the Technical University of Crete,Greece for 3 visiting high schools from Germany,Italy and Spain.
Presentation on trends and future research directions in Augmented Reality. Given by Mark Billinghurst at the Smart Cloud 2015 conference on September 16th, 2015, in Seoul, Korea.
2013 426 Lecture 1: Introduction to Augmented RealityMark Billinghurst
Lecture 1 of the 2013 COSC 426 graduate course on Augmented Reality taught by Mark Billinghurst at the HIT Lab NZ. This lecture provides an Introduction to Augmented Reality
COMP 4010 Lecture 9 providing an overview of Augmented Reality Technology. Taught by Mark Billinghurst on October 8th 2019 at the University of South Australia.
This presentation is to understand what is Augmented Reality, its use and future. It also contains some slides to show how iOS developer create app with facility of AR using ARKit framework introduced in iOS 11.
The second lecture for the course COMP 4010. This lecture was about the concept of Presence in Virtual Reality and was taught by Bruce Thomas on August 2nd 2016.
COMP 4010 Lecture12 - Research Directions in AR and VRMark Billinghurst
COMP 4010 lecture on research directions in AR and VR, taught by Mark Billinghurst on November 2nd 2017 at the University of South Australia. This is the final lecture in the 2017 COMP 4010 course on AR and VR
Lecture 2 of the COMP 4010 class on AR/VR. This lecture is about the human perception system. This lecture was given on August 3rd 2021 by Mark Billinghurst from the University of South Australia.
A lecture on Mobile Augmented Reality. A lecture given by Mark Billinghurst at the University of Canterbury on Friday September 13th 2013. This is part of the COSC 426 graduate course on Augmented Reality.
Lecture 1 of the COMP 4010 course on Augmented and Virtual Reality. Taught by Mark Billinghurst, Bruce Thomas and Gun Lee from the University of South Australia. This lecture provides an introduction to Virtual Reality. Taught on July 24th 2018.
Slides put together for a workshop on AR in Education for the ULearn 2016 conference. Gives a good overview of how to use the EnvisageAR software for AR. Presentation created by Mark Billinghurst, October 2016.
Lecture about Augmented Reality displays given by Mark Billinghurst on October 11th 2016 as part of the COMP 4010 class on Virtual Reality at the University of South Australia
CHI 2015: The Glass Class - Designing Wearable InterfacesMark Billinghurst
Course given by Mark Billinghurst, Thad Starner and Claire Lee on how to design wearable interfaces for head mounted and wrist worn computers. The course was a three hour class taught on April 22nd as part of the CHI 2015 conference in Korea.
Natural Interaction for Augmented Reality ApplicationsMark Billinghurst
Keynote talk giving by Mark Billinghurst from the HIT Lab NZ at the IVCNZ 2013 conference, November 28th 2013. The talk focuses on Natural Interaction with Augmented Reality applications using speech and gesture and demonstrates some of the projects in this area developed by the HIT Lab NZ.
A presentation given by Mark Billinghurst on April 21st 2015 at the CHI 2015 conference. This talk presents highlights from the journal paper:
M. Billinghurst, A. Clark, and G. Lee. A Survey
of Augmented Reality, Foundations and
Trends in Human-Computer Interaction.
Vol. 8, No. 1 (2015) 1–202, 2015
Available at :http://www.nowpublishers.com/article/Details/HCI-049
This is my presentation for augmented reality and its applications in education. It took place in the Technical University of Crete,Greece for 3 visiting high schools from Germany,Italy and Spain.
Presentation on trends and future research directions in Augmented Reality. Given by Mark Billinghurst at the Smart Cloud 2015 conference on September 16th, 2015, in Seoul, Korea.
2013 426 Lecture 1: Introduction to Augmented RealityMark Billinghurst
Lecture 1 of the 2013 COSC 426 graduate course on Augmented Reality taught by Mark Billinghurst at the HIT Lab NZ. This lecture provides an Introduction to Augmented Reality
COMP 4010 Lecture 9 providing an overview of Augmented Reality Technology. Taught by Mark Billinghurst on October 8th 2019 at the University of South Australia.
This presentation is to understand what is Augmented Reality, its use and future. It also contains some slides to show how iOS developer create app with facility of AR using ARKit framework introduced in iOS 11.
The second lecture for the course COMP 4010. This lecture was about the concept of Presence in Virtual Reality and was taught by Bruce Thomas on August 2nd 2016.
COMP 4010 Lecture12 - Research Directions in AR and VRMark Billinghurst
COMP 4010 lecture on research directions in AR and VR, taught by Mark Billinghurst on November 2nd 2017 at the University of South Australia. This is the final lecture in the 2017 COMP 4010 course on AR and VR
Lecture 2 of the COMP 4010 class on AR/VR. This lecture is about the human perception system. This lecture was given on August 3rd 2021 by Mark Billinghurst from the University of South Australia.
A lecture on Mobile Augmented Reality. A lecture given by Mark Billinghurst at the University of Canterbury on Friday September 13th 2013. This is part of the COSC 426 graduate course on Augmented Reality.
Lecture 1 of the COMP 4010 course on Augmented and Virtual Reality. Taught by Mark Billinghurst, Bruce Thomas and Gun Lee from the University of South Australia. This lecture provides an introduction to Virtual Reality. Taught on July 24th 2018.
Slides put together for a workshop on AR in Education for the ULearn 2016 conference. Gives a good overview of how to use the EnvisageAR software for AR. Presentation created by Mark Billinghurst, October 2016.
Lecture about Augmented Reality displays given by Mark Billinghurst on October 11th 2016 as part of the COMP 4010 class on Virtual Reality at the University of South Australia
CHI 2015: The Glass Class - Designing Wearable InterfacesMark Billinghurst
Course given by Mark Billinghurst, Thad Starner and Claire Lee on how to design wearable interfaces for head mounted and wrist worn computers. The course was a three hour class taught on April 22nd as part of the CHI 2015 conference in Korea.
Natural Interaction for Augmented Reality ApplicationsMark Billinghurst
Keynote talk giving by Mark Billinghurst from the HIT Lab NZ at the IVCNZ 2013 conference, November 28th 2013. The talk focuses on Natural Interaction with Augmented Reality applications using speech and gesture and demonstrates some of the projects in this area developed by the HIT Lab NZ.
Tish Shute, Director AR/VR, Corporate Technology Strategy, Huawei
A talk from Inspire Track at AWE USA 2017 - the largest conference for AR+VR in Santa Clara, California May 31- June 2, 2017.
XR is intelligent and interactive connecting real humans and physical objects with digital agents and entities. VR/AR will evolve into XR to become the future interface for Cloud Computing, IoT, Big Data, Prediction, Self-driving cars, Personalized Medicine, Robots, Drones, Cryptocurrency, Smart Cities, and AI. Social VR and AR will connect people in new and powerful ways but XR will connect the intelligence of people to the intelligence of machines in a space shared and understood by both. This talk will look at this new and intimate relationship between humans and intelligent agents.
http://AugmentedWorldExpo.com
This talk looks at the future of AR/VR and the new and intimate relationship between humans and intelligent agents that XR - the next generation of AR/VR, makes possible. XR is intelligent and interactive connecting real humans and physical objects with digital agents and entities. VR/AR will evolve into XR to become the future interface for Cloud Computing, IoT, Big Data, Prediction, Self-driving cars, Personalized Medicine, Robots, Drones, Cryptocurrency, Smart Cities, and AI. Social VR and AR will connect people in new and powerful ways but XR will connect the intelligence of people to the intelligence of machines in a space shared and understood by both.
keynote speech by Mark Billinghurst at the Workshop on Transitional Interfaces in Mixed and Cross-Reality, at the ACM ISS 2021 Conference. Given on November 14th 2021
Introduction to Virtual Reality (VR) or Augmented Reality: VR is the illusion of a three-dimensional, interactive, computer-generated reality. It can be used in the field of medicine, architecture as well as education. VR can
influence human behavior, interpersonal communication, and cognition (i.e., virtual genetics).
Fifty Shades of Augmented Reality: Creating Connection Using ARMark Billinghurst
Keynote speech by Mark Billinghurst at the Laval Virtual 2017 conference on March 24th 2017. The presentation talks about how Augmented Reality can be used to enhance remote collaboration.
Designing the future of Augmented RealityCarina Ngai
Presented on March 4th, 2016 at Interaction16 in Helsinki, Finland.
Until now, augmented reality has so far been mostly a sci-fi vision that overlays visual information to what we see in the physical world. It’s widely perceived as a “cool and interesting feature” for brands and advertising, but doesn’t have much practicality yet. To harness the real power of AR, which includes geolocation, image recognition, we believe that a more utilitarian visual search would be next.
To design for such possibilities, we begin to question even the fundamental basis of AR. For example, what would AR become beyond a rich visual layer? Will this change people’s motivation and behavior to use AR? How can we redefine AR to be a tool to give augmented information on objects? And how we can speculate its usage in the future?
AUGMENTED REALITY (AR) IN DAILY LIFE: EXPANDING BEYOND GAMINGLiveplex
The global fascination with AR was arguably ignited by the gaming industry, with titles like Pokémon GO captivating millions and showcasing the potential of immersive technology. However, the true power of AR lies not in its ability to entertain but in its capacity to enhance, transform, and simplify everyday tasks and experiences. Through the seamless integration of digital information with the physical environment, AR has emerged as a versatile tool, enriching user interactions across various domains.
Reality 2.0: The Evolution of InteractionKim Mats Mats
This poster, presented at the Information Architecture Summit in Vancouver, March 2017 with Lisa Nguyen, presents an interaction model for mixed reality, including gesture, gaze, sound, spatial mapping, and emotion.
When personal computing software and the Web were young, user interfaces and navigation controls borrowed heavily from physical world metaphors, from the trash can icon to Web "pages". Since then, we have come to rely on native digital UI frameworks and pattern libraries. Now we face new design challenges for mixed reality, which allows us to collaborate over digital holograms in physical space. Early mixed reality applications rely on familiar Web controls, porting the digital affordances we are familiar with - buttons, menus, and two-dimensional displays of data - into our physical environment. These familiar controls allow us to easily transition from 2D interactions to 3D. But will this approach be successful or will it prove too limiting? Recent research in embodied cognition hypothesizes that knowledge is created through how our bodies interact with objects in the world. To fully embrace the knowledge creation and sharing opportunities that mixed reality offers, we designers need to expand our vocabulary of interactions to engage with digital objects and data in the real world. This poster presents a conceptual framework that will continue to evolve as augmented reality and mixed reality experiences become part of our daily lives.
"Click to Continue" by Sam Otis, from Content+Design Meetup, Oct. 4, 2017Blend Interactive
Graphical interfaces help make powerful technology intuitive and accessible. They give us super powers. Join Sam Otis, Lead Designer at Blend Interactive, as the Sioux Falls Content + Design Group joins up with Sioux Falls Design Week for a fun look at how GUIs (Graphical User Interfaces) have developed, what makes an interface good today, and what challenges the future holds.
Similar to The Reality of Augmented Reality: Are we there yet? (20)
Keynote talk by Mark Billinghurst at the 9th XR-Metaverse conference in Busan, South Korea. The talk was given on May 20th, 2024. It talks about progress on achieving the Metaverse vision laid out in Neil Stephenson's book, Snowcrash.
These are slides from the Defence Industry event orgranized by the Australian Research Centre for Interactive and Virtual Environments (IVE). This was held on April 18th 2024, and showcased IVE research capabilities to the South Australian Defence industry.
This is a guest lecture given by Mark Billinghurst at the University of Sydney on March 27th 2024. It discusses some future research directions for Augmented Reality.
Presentation given by Mark Billinghurst at the 2024 XR Spring Summer School on March 7 2024. This lecture talks about different evaluation methods that can be used for Social XR/AR/VR experiences.
Empathic Computing: Delivering the Potential of the MetaverseMark Billinghurst
Invited guest lecture by Mark Billingurust given at the MIT Media Laboratory on November 21st 2023. This was given as part of Professor Hiroshi Ishii's class on Tangible Media
Talk to Me: Using Virtual Avatars to Improve Remote CollaborationMark Billinghurst
A talk given by Mark Billinging in the CLIPE workshop in Tubingen, Germant on April 27th 2023. This talk describes how virtual avatars can be used to support remote collaboration.
Empathic Computing: Designing for the Broader MetaverseMark Billinghurst
Keynote talk given by Mark Billinghurst at the CHI 2023 Workshop on Towards and Inclusive and Accessible Metaverse. The talk was given on April 23rd 2023.
Lecture 6 of the COMP 4010 course on AR/VR. This lecture is about designing AR systems. This was taught by Mark Billinghurst at the University of South Australia on September 1st 2022.
Keynote speech given by Mark Billinghurst at the ISS 2022 conference. Presented on November 22nd, 2022. This keynote outlines some research opportunities in the Metaverse.
Lecture 5 in the 2022 COMP 4010 lecture series. This lecture is about AR prototyping tools and techniques. The lecture was given by Mark Billinghurst from University of South Australia in 2022.
Lecture 4 in the 2022 COMP 4010 lecture series on AR/VR. This lecture is about AR Interaction techniques. This was taught by Mark Billinghurst at the University of South Australia in 2022.
Lecture 3 in the 2022 COMP 4010 lecture series on AR/VR. This lecture provides an introduction for AR Technology. This was taught by Mark Billinghurst at the University of South Australia in 2022.
Lecture 2 in the 2022 COMP 4010 Lecture series on AR/VR and XR. This lecture is about human perception for AR/VR/XR experiences. This was taught by Mark Billinghurst at the University of South Australia in 2022.
Lecture 1 for the 2022 COMP 4010 course on AR and VR. This course was taught by Mark Billinghurst at the University of South Australia in 2022. This lecture provides an introduction to AR, VR and XR.
Empathic Computing and Collaborative Immersive AnalyticsMark Billinghurst
Short talk by Mark Billinghurst on Empathic Computing and Collaborative Immersive Analytics, presented on July 28th 2022 at the Siggraph 2022 conference.
Lecture given by Mark Billinghurst on June 18th 2022 about how the Metaverse can be used for corporate training. In particular how combining AR, VR and other Metaverse elements can be used to provide new types of learning experiences.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Epistemic Interaction - tuning interfaces to provide information for AI support
The Reality of Augmented Reality: Are we there yet?
1. The Reality of Augmented Reality:
Are we there yet?
Mark Billinghurst
mark.billinghurst@hitlabnz.org
The HIT Lab NZ, University of Canterbury
March 24th 2015
10. Lesson’s Learned
! Have a clear driving vision
! Enjoy the journey
! Travel with others
! Have a well equipped vehicle
! The end always seems nearer than it really is
12. The Ultimate Display
The ultimate display would .. be a room within which
the computer can control the existence of matter. A
chair displayed in such a room would be good enough
to sit in. Handcuffs .. would be confining, and a bullet ..
would be fatal.
With appropriate programming such a display could
literally be the Wonderland into which Alice walked.
Ivan E. Sutherland
Sutherland, I.E. The ultimate display. Information Processing 1965, Proc. IFIP
Congress 65, 506-508.
13. Realizing the Vision
! 3D computer graphics
! Physical input devices
! Speech interaction
! Eye-gaze input
! Kinesthetic force feedback
Sutherland, I. E. (1968, December). A head-mounted three dimensional display. In Proceedings
of the December 9-11, 1968, fall joint computer conference, part I (pp. 757-764). ACM.
16. The Super Cockpit (1980’s)
! Furness - USAF
Furness, T. A. (1986, September). The super cockpit and its human factors challenges. In Proceedings of the Human
Factors and Ergonomics Society Annual Meeting (Vol. 30, No. 1, pp. 48-52). SAGE Publications.
23. To Make the Vision Real..
! Automatically detecting real environment
! Environmental awareness, Physically based interaction
! Gesture interaction
! Free-hand interaction
! Multimodal input
! Speech and gesture interaction
! Intelligent interfaces
! Implicit rather than Explicit interaction
24. Environmental Awareness
! AR MicroMachines
! AR experience with environment awareness and
physically-based interaction
! Based on MS Kinect RGB-D sensor
! Augmented environment supports
! occlusion, shadows
! physically-based interaction between real and
virtual objects
Clark, A., & Piumsomboon, T. (2011). A realistic augmented reality racing game using a
depth-sensing camera. In Proceedings of the 10th International Conference on Virtual
Reality Continuum and Its Applications in Industry (pp. 499-502). ACM.
25. Physics Simulation
! Create virtual mesh over real world
! Update at 10 fps – can move real objects
! Use by physics engine for collision detection (virtual/real)
! Use by OpenScenegraph for occlusion and shadows
27. Natural Hand Interaction
! Using bare hands to interact with AR content
! MS Kinect depth sensing
! Real time hand tracking
! Physics based simulation model
28. Skeleton Based Interaction
! 3 Gear Systems
! Kinect/Primesense Sensor
! Two hand tracking
! http://www.threegear.com
30. Skeleton Interaction + AR
! HMD AR View
! Viewpoint tracking
! Two hand input
! Skeleton interaction, occlusion
Piumsomboon, T., Altimira, D., Kim, H., Clark, A., Lee, G., & Billinghurst, M. (2014, September). Grasp-Shell
vs gesture-speech: A comparison of direct and indirect natural interaction techniques in augmented reality.
In Mixed and Augmented Reality (ISMAR), 2014 IEEE International Symposium on (pp. 73-82). IEEE.
31. Multimodal Interaction
! Combined speech and gesture input
! Gesture and Speech complimentary
! Speech: modal commands, quantities
! Gesture: selection, motion, qualities
! Previous work found multimodal interfaces
intuitive for 2D/3D graphics interaction
! However, few multimodal AR interfaces
32. Free Hand Multimodal Input
! Use free hand to interact with AR content
! Recognize simple gestures
! Open hand, closed hand, pointing
Point Move Pick/Drop
Lee, M., Billinghurst, M., Baek, W., Green, R., & Woo, W. (2013). A usability study of multimodal
input in an augmented reality environment. Virtual Reality, 17(4), 293-305.
35. Results - Performance
! Average performance time
! Gesture: 15.44s
! Speech: 12.38s
! Multimodal: 11.78s
! Significant difference across conditions (p < 0.01)
! Difference between gesture and speech/MMI
36. Subjective Results (Likert 1-7)
! User subjective survey
! Gesture significantly worse, MMI and Speech same
! MMI perceived as most efficient
! Preference
! 70% MMI, 25% speech only, 5% gesture only
Gesture Speech MMI
Naturalness 4.60 5.60 5.80
Ease of Use 4.00 5.90 6.00
Efficiency 4.45 5.15 6.05
Physical Effort 4.75 3.15 3.85
37. Lessons Learned
! Multimodal interaction significantly better than
gesture alone in AR interfaces for 3D tasks
! Shorter task time, more efficient
! Multimodal input was more natural, easier,
and more effective that gesture/speech only
! Simultaneous input rarely used
! More studies need to be conducted
! What gesture/speech patterns? Richer input
38. Intelligent Interfaces
! AR interface + intelligent tutoring system
! ASPIRE constraint based system (from UC)
! Constraints
- relevance cond., satisfaction cond., feedback
Westerfield, G., Mitrovic, A., & Billinghurst, M. (2013). Intelligent Augmented Reality Training for
Assembly Tasks. In Artificial Intelligence in Education (pp. 542-551). Springer Berlin Heidelberg.
42. ! Digital Imperceptible from the Real World
! Multi-sensory Display
! Natural unencumbered input
! Seamless blend between real and virtual
! Huge technical and non-technical challenges
The Ultimate AR Display
43. AR Research Trends
! Zhou 10 year ISMAR survey (1998-2008)
! 276 papers reviewed
! Most researched topics
! (1) Tracking techniques (20%)
! (2) Interaction techniques (15%)
! (3) Calibration and registration (14%)
! (4) AR applications (14%)
! (5) Display techniques (12%)
Feng Zhou, Henry B.L. Duh, Mark Billinghurst. Trends in Augmented Reality Tracking, Interaction and Display:
A Review of Ten Years of ISMAR. In proceedings of ISMAR 2008, Cambridge, UK, 15-18th of Sept. 2008.
58. Vision
“Using technology to create shared
emotional experiences between users
and so create a deeper sense of
empathy and understanding”
59.
60. Empathic Computing
1. Computing systems that can
understand your feelings and emotions
2. Computing systems that help you
better understand the feelings of others
62. Appliances That Make You Happy
! Jun Rekimoto – Univ. Tokyo
! Smile detection + smart devices
63. Can we develop interfaces
that allow us to be more
empathetic to others?
64. Movies are like a machine
that generates Empathy
Roger Ebert
65. Empathy Computing Requirements
! Basic Requirements
! Making the technology transparent
! Empathy Definition
! Seeing with the eyes of another
! Hearing with the ears of another
! Feeling with the heart of another
66. Using AR for Empathy
! Augmented Reality can:
! Remove technology barriers
! Enhance communication
! Change perspective
! Share experiences
! Enhance interaction in real world
68. Current Collaboration on Wearables
! First person remote conferencing/hangouts
! Limitations
! Single POV, no spatial cues, no annotations, etc
69. Sharing Space: Social Panoramas
! Capture and share social spaces in real time
! Enable remote people to feel like they’re with you
70. Technology
! Google Glass
! Image capture, viewpoint sharing
! Remote device (desktop, tablet)
! Immersive viewing, live annotation
71. Key Research Questions
! Where is my partner looking?
! Enhanced radar display, Context compass
! How can we interact together?
! Shared pointers, Shared drawing
74. Wearable Interface
! Google Glass + e-Health + Spydroid + SSI
! Measure GSR, pulse oxygen, ECG, pitch
! Share video and audio remotely
! Represent emotions back to Glass user (4 states)
! !
75. Desktop Interface
! Live video, real time emotion data
! See what sender sees, emotion representation
!
76. Emotion Representation
! How can we show what you’re feeling?
! Tested - Raw data, Visual tinting, Emotion labeling
!
! !
Excited Happy
77. Early Results
! Colour overlay + video stream preferred
! Easier to understand
! Disagreement over best colour coding
! Different emotional response with diff. stimuli
! Scary movie best
! Remote users felt connected to local user
! Understanding of emotions
79. Analysis
Table 2. Correlation between different streaming scenarios
obtained from the feedback of the local and remote users
Experiment
3
(Creating
a
shared
experience
while
watching
an
audio
Visual
Watching
an
audio
visual)
Happy
Sad
Neutral
Excited
Case1
(Only
video
stream)
0.77
0.84
0.85
0.85
Case
2
(Case
1
+
Color
Overlay)
0.92
0.91
0.85
0.98
Case
3
(Case
2
+
Graphical
Analysis)
0.86
0.75
0.85
0.94
80. Capturing Space: Real World Capture
! Hands free AR
! Portable scene capture (color + depth)
! Projector/Kinect combo, Remote controlled pan/tilt
! Remote expert annotation interface
86. Scaling Up
! Seeing actions of millions of users in the world
! Augmentation on city/country level
87. AR + Smart Sensors + Social Networks
! Track population at city scale (mobile networks)
! Match population data to external sensor data
! medical, environmental, etc
! Mine data to improve social services
88.
89.
90. Research Challenges
! How to convey emotion?
! How to measure empathy?
! Interface/interaction models?
! How to communicate emotion?
! Scaling up to city/country scale?
92. Take Home Messages
! Have a Vision
! Find people to travel with
! Equip the vehicle
! Enjoy the journey
Most Important: We’re nowhere near the end
of interesting research in Augmented Reality
93. More Information
• Mark Billinghurst
– mark.billinghurst@hitlabnz.org
– @marknb00
• Website
– www.hitlabnz.org