1
Integrating Artificial Intelligence and
Virtual Reality – A Feasibility Study
Dr. R.S.Kamath
Dept. of Computer Studies,
CSIBER, Kolhapur
2
Contents

Introduction

Recent Technological Advancements

Intelligent Virtual Environment

IVE Suite Architecture

Natural Language Processor

NLP to VR

Virtual World

Significance and Conclusions
3
Introduction

Intelligent Virtual Environment, convergence of
AI and VR

Human Computer Interaction

User’s ability to communicate in natural language
in the VR paradigm

Natural language text driven system to visually
simulate the modeled operation in 3D virtual
space
4
Recent Technological Advancements
Integration of Virtual Reality with branches of Artificial Intelligence.
5
Recent Technological Advancements

Integrating Natural Language Processing with VR

VR and Robotics
N. Miner, Sharon A., “An Interactive Virtual Reality Simulation System
for Robot Control and Operator Training”

Combination of VR and Fuzzy Logic
Kutuva, Narender Reddy, Yingcai , Hariharan “A Novel and Fast Virtual
Surgical System using Fuzzy Logic”

Blending Machine Learning with VR
Retrieved from http://neurocomputation.wordpress.com/current-and-
recent-projects/

Expert System with VR
R.L Grimsdale, S. M. Lambourn, Generation of Virtual Reality
Environments using Expert Systems.
6
Intelligent Virtual Environment

Culmination of converging technologies, NLP and VR

Natural Language input

Parses the text inputted by the user and obtains the
desired display

VR achieves a feeling of immersion in the simulated
world

Expanding benefits of NLP and VR by the
construction of IVE suite.
7
IVE Suite Architecture

User Input in natural language
Example Scenario,
“Show me a bronze textured sphere in wireframe
stereo rendering from an isometric view."

Text Analyzer
Various levels of NLP analysis

Rule Base Construction
Rules denotes the actions the Renderer
8
IVE Suite Architecture
9
IVE Suite Architecture
Attribute Values
Object Sphere
Material Bronze
Outlook Wireframe
View Isometric
Tools Texture
VR 3D Vision
Attribute Values
Material Gold, Silver, Chrome, Emerald, Perl, Copper, Brass,
Bronze
Background Color, Image
Outlook Wireframe, Vertices, Line
View Front, Back, Top, Bottom, Left, Right, Isometric
Tools Texture, Fog, Walkthrough
VR 3D Vision

IVE Library Design
Container for set of CAD models maintained in
the form of ASCII formats

NLP to VR Interface
forwarding Knowledge to the creation of final
“Virtual World”

Renderer
Display of Model in Virtual Envt.
10
IVE Suite Architecture

Interactivity found in NLP different from that
of VR

Information retrieval from the inputted text

Syntactically correct input but meaning less

Ambiguity of words
11
Challenging Issues
12
Conclusions

Combination of AI and VR modeling a brand new area
of research unfolds.

Feasibility of emerging area that converge AI and VR

Enables the user to interact with VR system in a natural
and intuitive way.

Applicable to various fields - manufacturing industries,
architectural designs, education, training etc.,
References

Stephanie S. Everett, Kenneth Wauchope, Naval
Research Laboratory, Washington, DC “A spoken
language interface to a virtual reality system (Video)”,
ANLC '97

Muller, J. ; Krapichler, Christian ; Lam Son Nguyen ;
Englmeier, K.-H. ; Lang, M., “Speech interaction in
virtual reality” Proceedings of the 1998 IEEE
International Conference on Acoustics, Speech and Signal
Processing,

Ozawa, T, “An integral image and text processing system
for automatic generation of 3D sign-language
animations”, Proceedings IEEE International Symposium
on Circuits and Systems, 2001.
References

Tamer M Wasfy, Ahmed K. Noor, USA, “Rule based
Natural Language Interface for virtual Environments”,
Advances in Engineering Software, Elsevier, 2002, 155-
168

Inkpen, D.; Kipp, D., “A prototype natural language
interface for animation systems”, The 3rd IEEE
International Workshop on Haptic, Audio and Visual
Environments and Their Applications, 2004

Hong-Wei Yang et.al.,“Machine learning-based
intelligent recommendation in virtual mall”, IEEE
Proceedings of International Conference on Machine
Learning and Cybernetics 2004
References

Ruth Aylett, Michael Luck Applying Artificial
Intelligence to Virtual Reality: Intelligent Virtual
Environments

Nadine E. Miner, Sharon A. Stansfield, An Interactive
Virtual Reality Simulation System for Robot Control and
Operator Training

Retrieved from
http://neurocomputation.wordpress.com/current-and-
recent-projects/

Shanthanand Kutuva, Narender P. Reddy, Yingcai Xiao,
S. I. Hariharan A Novel and Fast Virtual Surgical System
using Fuzzy Logic

Milos Cernak1, Adrian Sannier, Iowa State
University “Command Speech Interface to Virtual
Reality Applications”
16
Thank You
17
Natural Language Processing

Subfield of Artificial Intelligence and Linguistic

Computers understand statements written in natural
language

Natural Language are those spoken by people

Natural Language Understanding and Generation

Phases of Linguistic analysis
18
Virtual Reality

It is a creation of virtual 3D world in which one can feel
and sense the world as if it is real

Natural extension to 3D computer graphics with advanced
input and output devices

It is a technology that allows user to enter and interact in a
world that is generated by computer

VR has its impact on various fields
19
Virtual Reality

Op13

  • 1.
    1 Integrating Artificial Intelligenceand Virtual Reality – A Feasibility Study Dr. R.S.Kamath Dept. of Computer Studies, CSIBER, Kolhapur
  • 2.
    2 Contents  Introduction  Recent Technological Advancements  IntelligentVirtual Environment  IVE Suite Architecture  Natural Language Processor  NLP to VR  Virtual World  Significance and Conclusions
  • 3.
    3 Introduction  Intelligent Virtual Environment,convergence of AI and VR  Human Computer Interaction  User’s ability to communicate in natural language in the VR paradigm  Natural language text driven system to visually simulate the modeled operation in 3D virtual space
  • 4.
    4 Recent Technological Advancements Integrationof Virtual Reality with branches of Artificial Intelligence.
  • 5.
    5 Recent Technological Advancements  IntegratingNatural Language Processing with VR  VR and Robotics N. Miner, Sharon A., “An Interactive Virtual Reality Simulation System for Robot Control and Operator Training”  Combination of VR and Fuzzy Logic Kutuva, Narender Reddy, Yingcai , Hariharan “A Novel and Fast Virtual Surgical System using Fuzzy Logic”  Blending Machine Learning with VR Retrieved from http://neurocomputation.wordpress.com/current-and- recent-projects/  Expert System with VR R.L Grimsdale, S. M. Lambourn, Generation of Virtual Reality Environments using Expert Systems.
  • 6.
    6 Intelligent Virtual Environment  Culminationof converging technologies, NLP and VR  Natural Language input  Parses the text inputted by the user and obtains the desired display  VR achieves a feeling of immersion in the simulated world  Expanding benefits of NLP and VR by the construction of IVE suite.
  • 7.
  • 8.
     User Input innatural language Example Scenario, “Show me a bronze textured sphere in wireframe stereo rendering from an isometric view."  Text Analyzer Various levels of NLP analysis  Rule Base Construction Rules denotes the actions the Renderer 8 IVE Suite Architecture
  • 9.
    9 IVE Suite Architecture AttributeValues Object Sphere Material Bronze Outlook Wireframe View Isometric Tools Texture VR 3D Vision Attribute Values Material Gold, Silver, Chrome, Emerald, Perl, Copper, Brass, Bronze Background Color, Image Outlook Wireframe, Vertices, Line View Front, Back, Top, Bottom, Left, Right, Isometric Tools Texture, Fog, Walkthrough VR 3D Vision
  • 10.
     IVE Library Design Containerfor set of CAD models maintained in the form of ASCII formats  NLP to VR Interface forwarding Knowledge to the creation of final “Virtual World”  Renderer Display of Model in Virtual Envt. 10 IVE Suite Architecture
  • 11.
     Interactivity found inNLP different from that of VR  Information retrieval from the inputted text  Syntactically correct input but meaning less  Ambiguity of words 11 Challenging Issues
  • 12.
    12 Conclusions  Combination of AIand VR modeling a brand new area of research unfolds.  Feasibility of emerging area that converge AI and VR  Enables the user to interact with VR system in a natural and intuitive way.  Applicable to various fields - manufacturing industries, architectural designs, education, training etc.,
  • 13.
    References  Stephanie S. Everett,Kenneth Wauchope, Naval Research Laboratory, Washington, DC “A spoken language interface to a virtual reality system (Video)”, ANLC '97  Muller, J. ; Krapichler, Christian ; Lam Son Nguyen ; Englmeier, K.-H. ; Lang, M., “Speech interaction in virtual reality” Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing,  Ozawa, T, “An integral image and text processing system for automatic generation of 3D sign-language animations”, Proceedings IEEE International Symposium on Circuits and Systems, 2001.
  • 14.
    References  Tamer M Wasfy,Ahmed K. Noor, USA, “Rule based Natural Language Interface for virtual Environments”, Advances in Engineering Software, Elsevier, 2002, 155- 168  Inkpen, D.; Kipp, D., “A prototype natural language interface for animation systems”, The 3rd IEEE International Workshop on Haptic, Audio and Visual Environments and Their Applications, 2004  Hong-Wei Yang et.al.,“Machine learning-based intelligent recommendation in virtual mall”, IEEE Proceedings of International Conference on Machine Learning and Cybernetics 2004
  • 15.
    References  Ruth Aylett, MichaelLuck Applying Artificial Intelligence to Virtual Reality: Intelligent Virtual Environments  Nadine E. Miner, Sharon A. Stansfield, An Interactive Virtual Reality Simulation System for Robot Control and Operator Training  Retrieved from http://neurocomputation.wordpress.com/current-and- recent-projects/  Shanthanand Kutuva, Narender P. Reddy, Yingcai Xiao, S. I. Hariharan A Novel and Fast Virtual Surgical System using Fuzzy Logic  Milos Cernak1, Adrian Sannier, Iowa State University “Command Speech Interface to Virtual Reality Applications”
  • 16.
  • 17.
    17 Natural Language Processing  Subfieldof Artificial Intelligence and Linguistic  Computers understand statements written in natural language  Natural Language are those spoken by people  Natural Language Understanding and Generation  Phases of Linguistic analysis
  • 18.
    18 Virtual Reality  It isa creation of virtual 3D world in which one can feel and sense the world as if it is real  Natural extension to 3D computer graphics with advanced input and output devices  It is a technology that allows user to enter and interact in a world that is generated by computer  VR has its impact on various fields
  • 19.