20240515 SJSU White Paper Revised AI_Digital_Twins.pdf

AI Digital Twins of Humans: Redefining
Human Presence in the Digital Era
Bradford Lee
Nikhil Bhatia
Vincent Li
Viola Shao
Victor Lin
Vy Nguyen
May 15, 2024
Cite this paper as:
Lee B, Bhatia N, Li V, Shao V, Lin V, Nguyen V (2024) AI Digital Twins of Humans:
Redefining Human Presence in The Digital Era. ISSIP-SJSU AI Collaboration. 2024-05-15
1

Table of Contents
Introduction..................................................................................................................................2
Impact...................................................................................................................................... 3
Applications........................................................................................................................4
Experimentation........................................................................................................... 7
Conclusion............................................................................................................. 9
References.....................................................................................................10
Introduction
Imagine a digital version of you that looks, sounds, thinks, and reacts exactly as you
would in real life. This is the perfected version of the concept of Human Digital Twins (HDTs).
They are digital entities powered by a person’s data to create an interactive, virtual version of
themselves. Through the integration of existing technologies such as LLMs, AI-powered video
clones, and voice replicas, HDTs may transform digital communication and personal
representation in the future.
The term “digital twin” first appeared in the early 2000s within engineering circles to
refer to digital models of physical assets like buildings or engines. The idea was first formally
expounded on in 2003 when Professor Grieves at the University of Michigan coined the general
concept of a “virtual, digital equivalent to a physical product” used to simulate complex systems
in real-time (Grieves, 2015). These virtual models would allow engineers to forecast
maintenance requirements and to optimize performance. But long before that, NASA had already
applied this concept in the 1960s during the Apollo mission when a spacecraft experienced a
catastrophic failure over 200,000 miles (approximately 330,000 km) away from Earth. NASA
technicians used a digital model of the spacecraft to model events leading up to the rupture in
order to understand and diagnose the problem (Allen, 2021). This introduced the idea of creating
digital representations of not just spacecrafts but entire systems, including factories, cities and
now humans.
Societal viewpoints on this technology are complex, often rooted in science fiction and
generally overlap with society’s stance on AI as a whole. Films like Blade Runner and
Prometheus explored the ethical concerns presented by AI that is indistinguishable from humans.
These anxieties extend to the real world with the increased risk of privacy violations, identity
theft, manipulation, and the blurring line between reality and simulation this technology poses.
Yet, the number of users of AI globally continues to surge while the market for AI amounted to
nearly 200 billion U.S. dollars in 2023, and is expected to grow to well over 1 trillion U.S.
2

dollars by the decade's end (Thormundsson, 2024). The overarching theme, whether it be in pop
culture or everyday life, is that people recognize that AI augments society immensely, while still
being a cause for concern. This makes it clear that despite long-held anxieties about AI, mass
adoption of this technology by businesses and individuals alike will only become more of a
reality as time progresses.
Although the technology is still in its infancy, HDTs mark an important moment in AI
technology that aims to improve human life. Going forward, it is imperative that humans
carefully navigate ethical and societal standards. As HDTs advance, questions regarding privacy,
consent, identity, and manipulation of digital personas will undoubtedly arise. Therefore, a robust
framework with developer close observation must be established to ensure responsible
development and deployment of this technology.
Impact
For the ISSIP community and industry as a whole, human digital twins offer exciting
possibilities. Anyone, for instance, could leverage their HDT for brainstorming sessions or
creative applications; having a person-like digital assistant to converse with would help facilitate
the back and forth bouncing of ideas and creativity. In the healthcare domain, digital twins could
provide invaluable companionship and support to patients, particularly those facing chronic
illnesses or limited mobility. Additionally, their application in therapy and cognitive training
offers significant potential. Industries like UI/UX design can benefit from personalized user
interfaces tailored to individual preferences gleaned from a user's digital twin, resulting in a more
intuitive and user-friendly experience. The educational sector can be revolutionized by allowing
teachers to have their digital twins attend classes in their absence, mitigating the impact of
missed sessions.
Beyond specific applications, the economic benefits of HDTs are undeniable. Increased
productivity through remote collaboration with digital twins can empower diverse industries. The
creation and maintenance of HDTs opens doors for new job opportunities in the tech and legal
services industries as more developers will be needed to create and maintain software systems
that build and support HDTs, and more legal experts to navigate the legal landscape born out of
the use of HDTs. Furthermore, personalized training with HDTs can help integrate less
experienced workers, creating a stronger workforce. The preservation of knowledge and legacy
through a person's digital twin allows for the transmission of experiences and wisdom even after
they are gone. Enhanced communication for individuals with disabilities offers a pathway to
greater social inclusion which could also contribute to a stronger workforce.
3

However, the benefits of HDTs also come with potential ethical concerns. The very
existence of such sophisticated digital replicas compels us to confront profound existential
questions about the nature of human identity. Likewise, uploading one’s personality raises
critical questions about data privacy and security. Determining ownership of this data and
establishing safeguards against misuse are incredibly important. Identity theft is a serious
concern, as malicious actors could create fraudulent digital twins for nefarious purposes. AI is
already being exploited by scammers to impersonate victims’ loved ones in order to solicit
money from their relatives. Additionally, overreliance on digital companions for emotional
support could lead to further social isolation and hinder the development of real-world
relationships. Unequal access to this technology could exacerbate existing economic disparities
as there remain concerns over how such technology could affect older, less tech-savvy segments
of the workforce. The automated nature of HDTs will also inevitably replace humans in some
capacities, leading to job displacement.
Therefore, developing robust guidelines and regulations that address data ownership,
privacy, and potential biases within the underlying AI algorithms is critical. Open discourse
fostering collaboration between technologists, ethicists, and the public is essential to ensure the
responsible development and deployment of this transformative technology. Widespread
adoption is crucial, but ensuring equitable access is paramount to guarantee all demographics and
socioeconomic groups benefit from this new technology.
Applications
1. Conferences
Human digital twins pave the way for new forms of remote collaboration. In a conference
setting, speakers can have their digital twins fill their place if they cannot physically or remotely
attend. HDTs could be able to present slides, answer questions in real-time, and even engage in
conversations. This has potential to make almost any meeting accessible for everyone. Attendees
with disabilities would be able to attend meetings no matter how geographically distant.
Attendees who have physical mobility limitations would have no problem attending meetings
from the comfort of their homes, and attendees who have speech impediments would be able to
comfortably network and converse with other attendees.
2. Mentorship
With ISSIP having a mentorship program, it’s important to note that AI digital twins of
humans have the potential to streamline communication and reduce overhead while preserving
quality. The proliferation of remote work has made distance irrelevant, but mentors can also
4

reach out to mentees all around the world with their digital twins. These digital twins can provide
24/7 guidance and support, offering real-time feedback and answering questions outside of
pre-planned mentoring sessions. Furthermore, these digital twins can be consistent with a
mentor's established values and teaching style, so they could adequately fill the role in case the
mentor themselves is unavailable due to scheduling conflicts.
Busy professionals can also leverage their digital twins to mentor a broader range of
individuals, optimizing the use of a mentor’s expertise and time. Their knowledge can also be
preserved through their digital twins to ensure that their guidance can benefit future mentees,
even after retirement.
3. Enhanced Student Development
Beyond simply replacing absent professors, HDTs can be extremely useful resources for
students' growth. Consider a scenario where a student of engineering practices presents their
most recent project to a digital twin that serves as a professor. The student receives instant
feedback on the communication style and technical correctness. In a similar vein, a business
student may utilize a digital twin that was set up to replicate the precise interview questions and
styles that their target firms would use to practice employment interviews. Students could refine
their communication and presenting abilities in a secure, controlled setting in this customized
practice setting, which would increase their confidence and interview readiness.
4. Healthcare & Life Sciences
In healthcare, HDTs can revolutionize patient care by offering companionship and
support, particularly for those facing chronic illnesses or limited mobility. These virtual
companions can also be instrumental in therapy sessions, medication trials (conducted on virtual
models), and the development of personalized treatment plans. In fact, digital twins are now
being used to model patients’ bodies based on the patients’ unique medical information (MRI,
CT scan, etc.) and other health-related data such as heart rate, body mass index, etc. (Dassault
Systèmes, 2023). These types of models offer a holistic patient health overview and have the
potential to streamline and refine treatment. For example, by simulating the effects of various
treatments on a patient's virtual twin, doctors can gain valuable predictive insights, allowing
them to make more informed decisions. Furthermore, data from smart wearable devices can be
used to update a patient’s HDT continuously. That real-time data could be automatically
analyzed, compared to historical data and even health-related data from a patient’s relatives,
allowing for improved monitoring and early detection of health issues (Shengli, 2021).
In the same vein, companies are currently experimenting with the idea of using HDTs to
enhance employee wellness and safety in industrial environments (Davila-Gonzalez, 2024).
5

Conceptually, HDT profiles of employees would be built through the real-time collection of
health data from wearable devices. Their mental, emotional and physical states would be gauged
from that data through the use of analytic AI models. This could allow employers to proactively
address work-related stress and burnout among employees through organizational mechanisms,
in turn preventing accidents in the workplace.
5. Technology
HDTs can act as virtual assistants, providing assistance to software developers or other
types of engineers in projects. They can also act as virtual users, testing and refining software
functionalities without actual human testers. Professionals can also have their own digital twins
assist colleagues remotely in troubleshooting issues or providing real-time advice. Developers
can fine-tune the user experience by using HDTs to replicate user interactions with chatbots,
thereby helping them create even more effective virtual assistants (Hansen, 2023).
6. Media
Digital twins can be used in media applications, recommending articles or videos based
on an individual’s interests and past viewing habits. Such systems have already been
implemented in E-Commerce, where HDTs of customers are used to track users’ shopping
behaviors in order to provide better consumer recommendations (Lauer-Schmaltz, 2024). Digital
twins of humans also have immense potential virtual reality (VR) gaming when it comes to
storytelling and immersion. Imagine directly interacting with your digital twin in a virtual world.
Or, if you’re a music lover, imagine attending a concert where a digital replica of a famous
musician performs alongside a live band. This technology is already being experimented with for
use in Hollywood movies, advertisements, and concert venues. In fact, in 2022 Bruce Willis’s
digital twin was used in an advertisement for Russian telecoms company MegaFon (Sharf,
2022). More recently, the Ukrainian Ministry of Foreign Affairs has made headlines for its use of
an HDT going by the name of Victoria Shi, modeled after a Ukrainian singer Rosalie Nombre, as
a spokesperson to “read its statements, which will still be written by humans” (The Guardian,
2024). This would mark the first time HDTs have been used in this capacity, albeit with restricted
responsibilities. In the future, HDTs like Victoria Shi may do more than just vocalize human
input, but possibly have the entire role of a spokesperson delegated to them.
Sector Use Case Benefit
Conferences
AI Twins of Speakers Present & Answer
Questions
Increased accessibility for attendees with
disabilities, mobility limitations, or speech
impediments
6

ISSIP
Mentorship
Mentor Twins Provide 24/7 Guidance,
preserve knowledge
Streamlined communication, reduced overhead,
broader reach for mentors; expertise persists even
after mentor’s retirement
Student
Development
Practice Presentations with Professor
Twin; practice interviews with
company-specific twin
Personalized feedback on communication and
technical accuracy; improved interview skills and
confidence through targeted practice
Healthcare &
Life Sciences
Personalized care and improved
monitoring
Digital companions for patients with limited mobility
or chronic illness; Virtual models for medication
trials and treatment plans; Real-time health data
analysis for early issue detection; Employee
wellness monitoring in industrial settings
Technology
Virtual Assistants for Engineers; virtual
users for software testing
Improved project efficiency, remote troubleshooting
assistance; enhanced software quality without
human testers
Media
Personalized Media Recommendations;
Immersive VR experiences with Digital
Twins
Content tailored to individual preferences;
Enhanced storytelling and audience engagement
Experimentation
HeyGen is capable of generating AI avatars of people. There are multiple options, but the
one most relevant to ISSIP and digital twins are “Instant Avatars.” To create one, upload a video
following HeyGen guidelines such as 2-5 minute duration (HeyGen will not accept it if it is not),
pause between each sentence, and record in a well-lit and quiet environment. Then the user will
be asked to record a consent statement before it takes a few minutes to generate an avatar of the
person. After that, videos can be made with the avatar and scripts can dictate what the avatar
says with the same voice from the uploaded sample, or choose from a variety of voices that
HeyGen provides. Submitting will create the video at a cost of credits that will need to be
purchased. The cost of instant avatar generation is 1 credit per minute of content.
7

The limitations of the project would be security and confidentiality because creation of
the digital AI twins that can respond with the likeness of the actual person would need a lot of
personal data such as their research papers, personal information, and videos of themselves.
HeyGen may not be advanced enough to meet ISSIP’s wants as a standalone digital twin who
responds in real time. It is less like a real-time assistant and more of a one-time creation. In order
for the twin to say anything or be animated, it needs to be fed a script and generated which will
take a few minutes to a few hours depending how long the duration of the video is. Another
limitation is the cost. Depending on how long the talks are and how often avatars would need to
be created, the price could add up and be costly. $29 a month for the Creator plan comes with 15
credits. That means that every minute of content costs roughly $2.
8

While HDTs are a groundbreaking technology, there are also potential harms of the
creation and usage of them. Some of the harms include maliciously impersonating others without
their consent. They can also be used to deceive audiences using the likeness of a friend, family
member, or celebrity. These cases could potentially harm both the imitated individual’s
reputation and the general public who was misled. As this technology gets better and more
realistic, it will become more difficult to determine if the video is real or fake, a human or AI.
Conclusion
The technology behind human digital twins (HDTs), while still in its early stages, holds
immense potential for the ISSIP community and beyond. It offers exciting possibilities for
remote collaboration, personalized learning, and knowledge preservation. This technology offers
9
Tools Description Project Uses Model Pricing
HeyGen
Platform to generate AI avatars
by uploading a video of a person
speaking. Ability to make the
avatar speak user inputted script.
Costs credits to generate a video.
Created AI avatars to understand
the technology of digital AI twins
and to explore the limitations,
capabilities of the current
technology.
Pricing is monthly or
annually. Free is
available but not
recommended. Without
credits, HeyGen will
not generate the final
video avatar.
Claude
AI assistant created by Anthropic.
Trained on a vast amount of data
and capable of assisting in
writing, analysis, math, coding,
and more.
Used to analyze a blog on AI
digital twins written by the project
sponsor to create a script for the
digital twin. Also used to generate
potential questions and answers
based on the script.
Free model is
available, with the
option to upgrade to
Pro for $20 per month.
Both have a daily
message limit.

whole new learning experiences, and invaluable expertise across industries, from healthcare to
education and media.
However, the ethical implications regarding data privacy, security, and potential misuse
require careful consideration. Developing robust regulations and fostering open discourse among
technologists, ethicists, and the public is crucial to ensure the responsible development and
deployment of HDTs. Only through collective diligence and transparent collaboration between
AI developers and regulators, the development and implementation of HDTs are both responsible
and ethical, paving the way for a future where innovation serves humanity’s best interests.
References
Allen, B Danette. “Digital Twins and Living Models at NASA - NASA Technical Reports
Server (NTRS).” NASA, NASA, 1 Nov. 2021, ntrs.nasa.gov/citations/20210023699.
Grieves, Michael. (2015). Digital Twin: Manufacturing Excellence through Virtual Factory
Replication.
“Virtual Twin Experiences in Life Sciences & Healthcare.” Dassault Systèmes, 31 Aug.
2023, www.3ds.com/virtual-twin/life-sciences-healthcare.
Davila-Gonzalez, Saul, and Sergio Martin. “Human Digital Twin in Industry 5.0: A
Holistic Approach to Worker Safety and Well-Being through Advanced AI and Emotional
Analytics.” PubMed Central, U.S. National Library of Medicine, 19 Jan. 2024,
www.ncbi.nlm.nih.gov/pmc/articles/PMC10818408/.
Thormundsson, Bergur. “Topic: Artificial Intelligence (AI) Worldwide.” Statista,
www.statista.com/topics/3104/artificial-intelligence-ai-worldwide/#topicOverview.
Accessed 26 Apr. 2024.
Hansen, Mitch. “The Future of Innovation: Immersive Digital Twins.” Kinetic Vision, 26
Sept. 2023, kinetic-vision.com/the-future-of-innovation-immersive-digital-twins/.
Lauer-Schmaltz, Martin Wolfgang, et al. “Towards the Human Digital Twin: Definition and
Design – A Survey.” arXiv, Cornell University, 3 Feb. 2024,
arxiv.org/html/2402.07922v1#bib.bib21.
Sharf, Zack. “Deepfake Studio Used 34,000 Bruce Willis Images to Create the Actor’s
‘Digital Twin,’ but It Doesn’t Own the Rights to His Image.” Variety, Penske Media
10

Corporation, 3 Oct. 2022,
variety.com/2022/film/news/bruce-willis-sells-rights-deepfake-digital-twin-1235388442/.
Shengli, Wei. “Is Human Digital Twin Possible?” Computer Methods and Programs in
Biomedicine Update, Elsevier, 21 May 2021,
www.sciencedirect.com/science/article/pii/S2666990021000136.
“Ukraine Unveils AI-Generated Foreign Ministry Spokesperson.” The Guardian, Guardian
News and Media, 3 May 2024,
www.theguardian.com/technology/article/2024/may/03/ukraine-ai-foreign-ministry-spokes
person.
Google Gemini. (2024). Google Inc. Generative AI Tool for Text Refinement. Available at:
https://www.google.com/gemini. Accessed May 13, 2024.
11

20240515 SJSU White Paper Revised AI_Digital_Twins.pdf

Recommended

Recommended

More Related Content

Similar to 20240515 SJSU White Paper Revised AI_Digital_Twins.pdf

Similar to 20240515 SJSU White Paper Revised AI_Digital_Twins.pdf (20)

More from International Society of Service Innovation Professionals

More from International Society of Service Innovation Professionals (20)

Recently uploaded

Recently uploaded (20)

20240515 SJSU White Paper Revised AI_Digital_Twins.pdf