SlideShare a Scribd company logo
@frankcarey
Sr. Machine Learning Engineer
Creating Human VR
Characters With A.I.
2
My VR Game Idea
3
My Terrible VR Game Idea
4
Talk to the DEAD!
From https://i.ytimg.com/vi/0C6e9i0u0Bw/maxresdefault.jpg
What do we need?
• 3D Models of a character’s face
• Dialog
• Generate Speech from Dialog
• Animate the Face-model to match the
dialog.
5
Making VR is Expensive
"With a few exceptions..
all the art in a video game is hand-crafted.
As characters, items, levels and visual effects have become more
intricate and detailed, developers have had little choice but to throw
more and more artists at the problem.”
-- The Economist
6From https://www.economist.com/blogs/economist-explains/2014/09/economist-
explains-15
Generative Models
7
Generating Speech
8
Parametric
Concatenative
DeepMind
WaveNet
From https://deepmind.com/blog/wavenet-generative-model-raw-audio/ and https://lyrebird.ai/
Lyrebird -
Candidates
Generating Things
9
Generating Cats – From Noise
10From
https://ajolicoeur.wordpress.com/cats/
Generating Cats
11From https://affinelayer.com/pixsrv/index.html
Generating Buildings
12From https://affinelayer.com/pixsrv/index.html
Generating Faces - Humans
13From https://twitter.com/dribnet/status/852770111286943745
Capturing Faces
14
Photo -> 3D Model
15From http://webneel.com/35-amazing-celebrity-3d-models-and-cg-character-
designs
Capturing Faces
16From https://arxiv.org/pdf/1612.00523.pdf
Capturing Faces
17From
https://www.disneyresearch.com/publication/realtimeperformancecapture/
Animating Faces
18
Animation - Face Actors
19
From http://www.graphics.stanford.edu/~niessner/thies2016face.html
Animation - Audio Only
20From https://www.theverge.com/2017/7/12/15957844/ai-fake-video-audio-speech-
obama
Animating Movement
21
Animating Movement
22From https://deepmind.com/blog/wavenet-generative-model-raw-audio/
Putting it all together..
23
Additional Resources
• Pix2Pix Blog post - https://affinelayer.com/pix2pix/
• BotsAndBrains.com – NYC Meetup and ML Blog
24
Thank You
BotsAndBrains.com
Next Meetup is
Wednesday.. July
26th @Vimeo HQ
25
Follow me:
@frankcarey
Follow us:
@vimeo
We’re Hiring!
Vimeo.com/Jobs

More Related Content

Similar to Nyai deck-vimeo-july-2017

The Immersive Web
The Immersive WebThe Immersive Web
The Immersive Web
Tony Parisi
 
Skills You Need to Be a Video Game Developer
Skills You Need to Be a Video Game DeveloperSkills You Need to Be a Video Game Developer
Skills You Need to Be a Video Game Developer
MSBCollege
 
Being a game developer with the skills you have
Being a game developer with the skills you haveBeing a game developer with the skills you have
Being a game developer with the skills you have
Johan Lindfors
 
Coded Design and the Everlearning Designer
Coded Design and the Everlearning DesignerCoded Design and the Everlearning Designer
Coded Design and the Everlearning Designer
Karsten Vestergaard
 
FETC 2015 Advanced Game Design Presentation - Workshop
FETC 2015 Advanced Game Design Presentation - WorkshopFETC 2015 Advanced Game Design Presentation - Workshop
FETC 2015 Advanced Game Design Presentation - Workshop
Mike Ploor
 
Viz Populi
Viz PopuliViz Populi
Viz Populi
Tony Parisi
 
Tony Parisi (VP Platform, Wevr) The Immersive Web
Tony Parisi (VP Platform, Wevr) The Immersive WebTony Parisi (VP Platform, Wevr) The Immersive Web
Tony Parisi (VP Platform, Wevr) The Immersive Web
AugmentedWorldExpo
 
Tech jobs beyond programming - Game design
Tech jobs beyond programming - Game designTech jobs beyond programming - Game design
Tech jobs beyond programming - Game design
Vasu Jain
 
Introduction to game development
Introduction to game developmentIntroduction to game development
Introduction to game development
Gaetano Bonofiglio
 
20 Things Successful Game Developers Do Beyond Making Games
20 Things Successful Game Developers Do Beyond Making Games20 Things Successful Game Developers Do Beyond Making Games
20 Things Successful Game Developers Do Beyond Making Games
Vlad Micu
 
Neev Hackathon 2013 - Super Feet
Neev Hackathon 2013 - Super FeetNeev Hackathon 2013 - Super Feet
Neev Hackathon 2013 - Super Feet
Neev Technologies
 
Goto berlin 10-2014
Goto berlin 10-2014Goto berlin 10-2014
Goto berlin 10-2014
Courtney Hemphill
 
Mobile Dev For Web Devs
Mobile Dev For Web DevsMobile Dev For Web Devs
Mobile Dev For Web Devs
Justin James
 
From Web to Game Development
From Web to Game DevelopmentFrom Web to Game Development
From Web to Game Development
Pietro Polsinelli
 
Metaverse Game Development How do entrepreneurs instantly launch a metaverse...
Metaverse Game Development  How do entrepreneurs instantly launch a metaverse...Metaverse Game Development  How do entrepreneurs instantly launch a metaverse...
Metaverse Game Development How do entrepreneurs instantly launch a metaverse...
Andrew Miller
 
Andy,C.V.
Andy,C.V.Andy,C.V.
Andy,C.V.
andy685
 
Oct 2012 e newsletter
Oct 2012 e newsletterOct 2012 e newsletter
Oct 2012 e newsletter
David Wortley
 
Magic Leap
Magic LeapMagic Leap
Magic Leap
Magic LeapMagic Leap
Magic Leap
Team2080
 
Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3
Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3
Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3
Doris Chen
 

Similar to Nyai deck-vimeo-july-2017 (20)

The Immersive Web
The Immersive WebThe Immersive Web
The Immersive Web
 
Skills You Need to Be a Video Game Developer
Skills You Need to Be a Video Game DeveloperSkills You Need to Be a Video Game Developer
Skills You Need to Be a Video Game Developer
 
Being a game developer with the skills you have
Being a game developer with the skills you haveBeing a game developer with the skills you have
Being a game developer with the skills you have
 
Coded Design and the Everlearning Designer
Coded Design and the Everlearning DesignerCoded Design and the Everlearning Designer
Coded Design and the Everlearning Designer
 
FETC 2015 Advanced Game Design Presentation - Workshop
FETC 2015 Advanced Game Design Presentation - WorkshopFETC 2015 Advanced Game Design Presentation - Workshop
FETC 2015 Advanced Game Design Presentation - Workshop
 
Viz Populi
Viz PopuliViz Populi
Viz Populi
 
Tony Parisi (VP Platform, Wevr) The Immersive Web
Tony Parisi (VP Platform, Wevr) The Immersive WebTony Parisi (VP Platform, Wevr) The Immersive Web
Tony Parisi (VP Platform, Wevr) The Immersive Web
 
Tech jobs beyond programming - Game design
Tech jobs beyond programming - Game designTech jobs beyond programming - Game design
Tech jobs beyond programming - Game design
 
Introduction to game development
Introduction to game developmentIntroduction to game development
Introduction to game development
 
20 Things Successful Game Developers Do Beyond Making Games
20 Things Successful Game Developers Do Beyond Making Games20 Things Successful Game Developers Do Beyond Making Games
20 Things Successful Game Developers Do Beyond Making Games
 
Neev Hackathon 2013 - Super Feet
Neev Hackathon 2013 - Super FeetNeev Hackathon 2013 - Super Feet
Neev Hackathon 2013 - Super Feet
 
Goto berlin 10-2014
Goto berlin 10-2014Goto berlin 10-2014
Goto berlin 10-2014
 
Mobile Dev For Web Devs
Mobile Dev For Web DevsMobile Dev For Web Devs
Mobile Dev For Web Devs
 
From Web to Game Development
From Web to Game DevelopmentFrom Web to Game Development
From Web to Game Development
 
Metaverse Game Development How do entrepreneurs instantly launch a metaverse...
Metaverse Game Development  How do entrepreneurs instantly launch a metaverse...Metaverse Game Development  How do entrepreneurs instantly launch a metaverse...
Metaverse Game Development How do entrepreneurs instantly launch a metaverse...
 
Andy,C.V.
Andy,C.V.Andy,C.V.
Andy,C.V.
 
Oct 2012 e newsletter
Oct 2012 e newsletterOct 2012 e newsletter
Oct 2012 e newsletter
 
Magic Leap
Magic LeapMagic Leap
Magic Leap
 
Magic Leap
Magic LeapMagic Leap
Magic Leap
 
Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3
Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3
Building Beautiful and Interactive Metro apps with JavaScript, HTML5 & CSS3
 

Recently uploaded

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
Neo4j
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Adtran
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Neo4j
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Neo4j
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
Quotidiano Piemontese
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 

Recently uploaded (20)

GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024GraphSummit Singapore | The Art of the  Possible with Graph - Q2 2024
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024
 
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 daysPushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
 
National Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practicesNational Security Agency - NSA mobile device best practices
National Security Agency - NSA mobile device best practices
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 

Nyai deck-vimeo-july-2017

Editor's Notes

  1. Logo Slide