This document describes an augmented reality telehealth ecosystem called XRCare that aims to improve patient education and remote consultations. It consists of several applications that allow doctors and patients to collaboratively view and annotate interactive 3D models. Patients can use a chatbot to get answers or monitor vitals using their smartphone. Doctors can conduct remote consultations using HoloLens to share their view with patients. The ecosystem is built with Unity and uses Azure services for storage, AI and hosting. It aims to make remote learning and consultations more effective compared to traditional video.

1. Copyright © 2020 HCL Technologies Limited | www.hcltech.com
XRCare
Patient Education ecosystem of applications using
the utopian might of Augmented Reality
Diagnosis, Treatment & Therapeutic Management
A foray into possibilities by DigitalExperienceApps

2. 2 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
Problem Statement
COVID19 has accelerated the demand of medical equipment among
consumers for home assessments & triage. Imparting proper
education on uses, operation and functionality of these medical
devices remains a challenge and further increases pressure on
healthcare professionals for Consultation and Patient Education.
It is imperative people stay indoors to Prevent & Contain further
spread with access to accurate and credible information.
Diagnosis, Treatment & Therapeutic
Management

3. 3 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
Augmented Reality telehealth ecosystem of applications for:
o Imparting Effective & Credible knowledge on using medical devices like an
Oximeter at ‘do-it-at-your-own-pace’ sessions.
o Establish remote shared presence in Augmented & Mixed Reality realms
for contextualizing information shared between doctors & patients.
o Capture health vitals using a smartphone for remote triaging.
Solution Proposition
Solution Tenets
o With rise in telehealth, it often gets difficult for a doctor to elucidate
medical nuances and human anatomy to patients without access to 3D
printed models. XRCare bridges the gap by providing interactable to-scale
3D models in a collaborative environment in the safety & comfort of the
patient’s home.
o Developed on Unity with Azure’s backend services, the ecosystem of
applications is multi-tenanted running on iOS, Android & Hololens.
o Patients receive unhindered access to credible information from a bot.
Benefits Yielded

4. 4 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
Solution Snapshot
NLP & Chatbot
Users may converse with a chatbot to seek answers to
queries. Answers to which may be supported by animated
user flows on the 3D model.
Patient Education
A shared Augmented Reality Session assists doctors to
contextualize conversation for effective information delivery
E-Consultation
Based on the diagnosis, the doctor augments a 3D
model on a Hololens and proceeds to explain the
anatomy using annotations on a shared augmented
reality session and suggests a line of treatment.
1
Self Learning
An Augmented Reality Android Application lets users augment
realistic and to-scale models to effectively describe its components,
functionality & uses.
2
3
Patient Education E-Consultation
1
2
Triaging Vitals
Vitals like heart rate can be measured by smart
phones and shared with health care professionals

5. 5 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
Why Augmented Reality?
Source: pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf
4X more focused
than learning through video based content
3.75X more emotionally
Connected to content
275% more confident
to apply learning
4X faster training of content

6. 6 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
High Level Technical Architecture
Azure Blob Storage
Storage of 3D models
Azure Cosmos
Storage of 3D models Metadata
Azure Cognitive Services
Speech Recognition and Synthesizer
Azure Bot Services
Trained through qna Maker
Azure Systems
Rapid Data Sharing
Native iOS Application
Measuring Heart Rate
Unity Hololens Application
Visualizing Human Anatomy
Unity Android Application
Patient and Doctor Shared AR experience
Animations
Cross Platform Low-Poly 3D Models
ARKit
Technology Enablers

7. 7 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
XRCare Ecosystem Feature List
Device Agnostic Augmented Reality
applications built with Unity for Android &
Hololens
Remote shared Augmented Reality
experiences using Photon Match Making
Service
Model cloud hosting & dynamic ingestion
with Azure Blob Storage
Hosting of Model metadata on Azure Cosmos
DB for scalability
NLP with Azure Cognitive Speech Services for
Speech recognition and synthesis
Standalone iOS application for monitoring
patient’s heart rate through smartphones
Chat bot integration with Azure Cognitive Bot
Services for responding to patient queries
Chatbot triggered animations with remote
configuration on Bot metadata

8. 8 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
o The current Unity Doctor & Patient mobile applications run on Android.
We are in the process of extending support to iOS
o E-Consultation is driven on Hololens for the doctor. A shared Augmented
Reality experience between the Doctor using Hololens & a patient using
an Android/iOS application
o E-consultations and shared Augmented Reality experiences should be
accompanied with an Audio + Video call and chat for real-life application
o User vitals tracking application on iOS is a standalone application. Vitals
should be stored on the cloud for ingestion on the doctor’s application
Futuristic Possibilities
We fell short on a few features…

9. 9 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
View the Video Here!

10. 10 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
Meet The Team!
Raihanaparveen Adhimohamed
Native iOS Ninja
Rajeev Katyal
Creative Guru
Santhosh Kumar Viswanathan
Creative Genie & Unity Prophet
Nandhakumar Ramakrishnan
Native iOS Wiz
Vinayahamoorthy Viswanathan
C# and Azure Wiz
Daksh Srivastava
C#, Unity & Azure Wiz

11. 11 | Technology for the next decade, today Copyright © 2020 HCL Technologies Limited | www.hcltech.com
Git Repository Access
Git Repository URL
https://repo.hclets.com/Health_Hackathon/Hackathon
Credentials
Username : Health-Hackathon
Password : HclDEA@123As+ck5
Please reach out for assistance:
deapmg@hcl.com, daksh_s@hcl.com, SanthoshKumarV@hcl.com
Steps
o Visit the link above
o Select Standard Login
o Enter the credentials above

12. $9.9 BILLION | 150,000+ IDEAPRENEURS | 46 COUNTRIES