Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
CASA, Geneva, 2016 Slide 1University of Crete & Foundation for Research and Technology - Hellas
Gamification	with	virtual	...
CASA, Geneva, 2016 Slide 2University of Crete & Foundation for Research and Technology - Hellas
From Augmenta
http://whati...
CASA, Geneva, 2016 Slide 3University of Crete & Foundation for Research and Technology - Hellas
To Magic Leap
http://www.m...
CASA, Geneva, 2016 Slide 4University of Crete & Foundation for Research and Technology - Hellas
From J.A.R.V.I.S.
http://m...
CASA, Geneva, 2016 Slide 5University of Crete & Foundation for Research and Technology - Hellas
To Hololens
https://www.mi...
CASA, Geneva, 2016 Slide 6University of Crete & Foundation for Research and Technology - Hellas
& Asobo
http://www.asobost...
CASA, Geneva, 2016 Slide 7University of Crete & Foundation for Research and Technology - Hellas
Overview
• A journey in Mi...
CASA, Geneva, 2016 Slide 8University of Crete & Foundation for Research and Technology - Hellas
How many Realities? Mixed ...
CASA, Geneva, 2016 Slide 9University of Crete & Foundation for Research and Technology - Hellas
Gamification, Serious Game...
CASA, Geneva, 2016 Slide 10University of Crete & Foundation for Research and Technology - Hellas
Presence
• The concept of...
CASA, Geneva, 2016 Slide 11University of Crete & Foundation for Research and Technology - Hellas
The VHD++ framework
§M. P...
CASA, Geneva, 2016 Slide 12University of Crete & Foundation for Research and Technology - Hellas
MOBILE AUGMENTED
REALITY ...
CASA, Geneva, 2016 Slide 13University of Crete & Foundation for Research and Technology - Hellas
The importance of AR Trac...
CASA, Geneva, 2016 Slide 14University of Crete & Foundation for Research and Technology - Hellas
MOBILE AR CULTURAL HERITA...
CASA, Geneva, 2016 Slide 15University of Crete & Foundation for Research and Technology - Hellas
MOBILE AR CULTURAL HERITA...
CASA, Geneva, 2016 Slide 16University of Crete & Foundation for Research and Technology - Hellas
Dialogue-based interactio...
CASA, Geneva, 2016 Slide 17University of Crete & Foundation for Research and Technology - Hellas
Self Adaptive Animation i...
CASA, Geneva, 2016 Slide 18University of Crete & Foundation for Research and Technology - Hellas
Our current MR Research Q...
CASA, Geneva, 2016 Slide 19University of Crete & Foundation for Research and Technology - Hellas
Back to the drawing board…
CASA, Geneva, 2016 Slide 20University of Crete & Foundation for Research and Technology - Hellas
glGA
Utilities
glGARigMes...
CASA, Geneva, 2016 Slide 21University of Crete & Foundation for Research and Technology - Hellas
Algebras for Computer Gra...
CASA, Geneva, 2016 Slide 22University of Crete & Foundation for Research and Technology - Hellas
Some history
William Hami...
CASA, Geneva, 2016 Slide 23University of Crete & Foundation for Research and Technology - Hellas
Quaternions
Generalises c...
CASA, Geneva, 2016 Slide 24University of Crete & Foundation for Research and Technology - Hellas
Grassmann algebra
German ...
CASA, Geneva, 2016 Slide 25University of Crete & Foundation for Research and Technology - Hellas
Properties of the outer p...
CASA, Geneva, 2016 Slide 26University of Crete & Foundation for Research and Technology - Hellas
Geometric algebra
W.K. Cl...
CASA, Geneva, 2016 Slide 27University of Crete & Foundation for Research and Technology - Hellas
Two dimensions
2D suffici...
CASA, Geneva, 2016 Slide 28University of Crete & Foundation for Research and Technology - Hellas
The bivector
The unit biv...
CASA, Geneva, 2016 Slide 29University of Crete & Foundation for Research and Technology - Hellas
From dual-quaternions to ...
CASA, Geneva, 2016 Slide 30University of Crete & Foundation for Research and Technology - Hellas
Contribution: Main Algori...
CASA, Geneva, 2016 Slide 31University of Crete & Foundation for Research and Technology - Hellas
• Input: quaternion orien...
CASA, Geneva, 2016 Slide 32University of Crete & Foundation for Research and Technology - Hellas
GA for character animatio...
CASA, Geneva, 2016 Slide 33University of Crete & Foundation for Research and Technology - Hellas
AR Presence and Gamificat...
CASA, Geneva, 2016 Slide 34University of Crete & Foundation for Research and Technology - Hellas
The Smartbody character a...
CASA, Geneva, 2016 Slide 35University of Crete & Foundation for Research and Technology - Hellas
A fast and robust pipelin...
CASA, Geneva, 2016 Slide 36University of Crete & Foundation for Research and Technology - Hellas
Life-sized crowd simulati...
CASA, Geneva, 2016 Slide 37University of Crete & Foundation for Research and Technology - Hellas
A Conformal Geometric Alg...
CASA, Geneva, 2016 Slide 38University of Crete & Foundation for Research and Technology - Hellas
An inclusive Conformal Ge...
CASA, Geneva, 2016 Slide 39University of Crete & Foundation for Research and Technology - Hellas
An inclusive Conformal Ge...
CASA, Geneva, 2016 Slide 40University of Crete & Foundation for Research and Technology - Hellas
Mixed Reality Serious Gam...
CASA, Geneva, 2016 Slide 41University of Crete & Foundation for Research and Technology - Hellas
Summary &
Future Work
• O...
CASA, Geneva, 2016 Slide 42University of Crete & Foundation for Research and Technology - Hellas
Acknowledgements
Funded b...
CASA, Geneva, 2016 Slide 43University of Crete & Foundation for Research and Technology - Hellas
and one last thing…
CASA, Geneva, 2016 Slide 44University of Crete & Foundation for Research and Technology - Hellas
http://www.ics.forth.gr/C...
CASA, Geneva, 2016 Slide 45University of Crete & Foundation for Research and Technology - Hellas
Sponsored by
Thank	you	fo...
Upcoming SlideShare
Loading in …5
×

Gamification with virtual characters at the borders of mixed realities and algebras

168 views

Published on

“Gamification” as the use of game design elements in non-game contexts, has attracted significant attention in various application fields of mixed reality, such as cultural heritage and education. At the same time, the rediscovery of geometric algebra via modern GPU-enhanced generators empowers computer graphics scientists with powerful new tools. These allow the replacement of different types of algebras, and their in-between conversions between CPU and GPU, such as linear algebra matrices, quaternions, dual-quaternions and Euler angles with a single geometric algebra mathematical representation. Such a representation enables enhanced gamified character simulation in different realities in the mixed reality spectrum, such as VR and AR. In this talk I will be presenting our latest results in these areas.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Gamification with virtual characters at the borders of mixed realities and algebras

  1. 1. CASA, Geneva, 2016 Slide 1University of Crete & Foundation for Research and Technology - Hellas Gamification with virtual characters at the borders of mixed realities and algebras Prof. George Papagiannakis University of Crete & Foundation for Reserach and Technology Hellas Heraklion, Crete, Greece
  2. 2. CASA, Geneva, 2016 Slide 2University of Crete & Foundation for Research and Technology - Hellas From Augmenta http://whatisaugmenta.com
  3. 3. CASA, Geneva, 2016 Slide 3University of Crete & Foundation for Research and Technology - Hellas To Magic Leap http://www.magicleap.com/#/home
  4. 4. CASA, Geneva, 2016 Slide 4University of Crete & Foundation for Research and Technology - Hellas From J.A.R.V.I.S. http://marvel-movies.wikia.com/wiki/J.A.R.V.I.S.
  5. 5. CASA, Geneva, 2016 Slide 5University of Crete & Foundation for Research and Technology - Hellas To Hololens https://www.microsoft.com/microsoft-hololens/en-us
  6. 6. CASA, Geneva, 2016 Slide 6University of Crete & Foundation for Research and Technology - Hellas & Asobo http://www.asobostudio.com
  7. 7. CASA, Geneva, 2016 Slide 7University of Crete & Foundation for Research and Technology - Hellas Overview • A journey in Mixed Reality (MR) and Algebras • How many Realities? • Gamification • Presence? • Different algebras for character simulation • Dual quaternions • Geometric algebra? • Current motivation and MR research questions • MR character enabling technologies • Conclusions and future work
  8. 8. CASA, Geneva, 2016 Slide 8University of Crete & Foundation for Research and Technology - Hellas How many Realities? Mixed Reality = AR + VR+ AV+ CR+ IR+ HR+ SR… MR Reality-Virtuality Continuum Extent of World Knowledge World Unmodeled World Fully Modeled Real Environment Virtual Environment AR VR AR-VR: Papagiannakis, G., Schertenleib, S., O'Kennedy, B., Poizat, M., Magnenat-Thalmann, N., Stoddart, A., Thalmann, D., 2005. Mixing Virtual and Real scenes in the site of ancient Pompeii. Computer Animation and Virtual Worlds, John Wiley and Sons Ltd 16, 1, 11–24. Cross-Reality (CR): Davies, C.J., Miller, A., and Allison, C. 2012. Virtual Time Windows: Applying cross reality to cultural heritage. Proceedings of the Postgraduate Conference on the Convergence of Networking and Telecomunications, ISBN: 978-1-902560-26-7. Hybrid-Reality (HR): Reda, K., Febretti, A., Knoll, A., et al. 2013. Visualizing Large, Heterogeneous Data in Hybrid-Reality Environments. Computer Graphics and Applications, IEEE 33, 4, 38–48. Indirect Reality (IR): Wither, J., Tsai, Y.-T., and Azuma, R. 2011. Indirect augmented reality. Computers & Graphics 35, 4, 810–822. Substitutional Reality (SR): K. Suzuki, S. Wakisaka, and N. Fujii, “Substitutional Reality System: A Novel Experimental Platform for Experiencing Alternative Reality,” Sci. Rep., vol. 2, pp. 1–9, Jun. 2012. CR-IR HRAV The Mixed Reality (MR) Continuum [Milgram99] [Azuma01] SR
  9. 9. CASA, Geneva, 2016 Slide 9University of Crete & Foundation for Research and Technology - Hellas Gamification, Serious Games • “gamification”: as the use of game design elements in non-game contexts [Deterding et al 11] • Use of game dynamics (e.g. plot), game mechanics (e.g. rules) and game components (e.g. points, avatars) in order to engage more people • “Serious games”: computer games that are not limited to the aim of providing entertainment [Macedonia 2000], [Macedonia 2002] • that allow for collaborative use of 3D spaces that are used for learning and educational purposes in a number of application domains [Macedonia 2002] S. Deterding, D. Dixon, R. Khaled, and L. Nacke,“From game design elements to gamefulness,” presented at the the 15th International Academic MindTrek Conference,NewYork,NewYork,USA,2011, p. 9. Macedonia M (2000) Using technology and innovation to simulate daily life.IEEE Computer 33(4):110–112 Macedonia M (2002) Games soldiers play.IEEE Spectrum 39(3):32–37
  10. 10. CASA, Geneva, 2016 Slide 10University of Crete & Foundation for Research and Technology - Hellas Presence • The concept of ‘presence’ refers to the phenomenon of behaving and feeling as if we are in the virtual world: ‘being there’ created by computer displays [Sanchez-Vives05] • Another approach is that the sense of ‘being there’ or ‘being someone’ in a VE is grounded on the ability ‘to do there’ in VR [Sanchez-Vives05] , AR [Torre00][Egges07]and MR [Wagner09] • …and many more definitions… -Torre R, Fua P,Balcisoy S, Ponder M,Thalmann D. Interaction between real and virtual humans: Playing checkers. Proc. Eurographics Workshop OnVirtual Environments (2000) - Sanchez-Vives M., and Slater, M.,“From presence to consciousness through virtual reality”. Nature Reviews Neuroscience (2005) vol. 6 (4) pp.pp.332- 339 - Egges,A., Papagiannakis,G.,Magnenat-Thalmann, N.,“Presence and Interaction in Mixed Realities”,The Visual Computer, Springer-Verlag,Volume 23, Number 5, May 2007 -Wagner I, Broll W, Jacucci G, Kuutii K, McCall R, Morrison A, Schmalstieg D,Terrin J. On the Role of Presence in Mixed Reality. PRESENCE:Teleoperators and Virtual Environments (2009) vol.18 (4) pp.249-276
  11. 11. CASA, Geneva, 2016 Slide 11University of Crete & Foundation for Research and Technology - Hellas The VHD++ framework §M. Ponder, G. Papagiannakis, T. Molet, N. Magnenat-Thalmann, D. Thalmann. VHD++ Development Framework: Towards Extendible, Component Based VR/AR Simulation Engine Featuring Advanced Virtual Character Technologies. IEEE Computer Society Press, Proceedings of Computer Graphics International (CGI), pp. 96-104. 2003 components •Animation blending • Idle motion • skeleton animation • skin deformation • cloth simulation • face simulation • speech simulation • AR image blending • PRT Illumination • hair animatipn • Crowd simulation • Python scripting • motion capture • VR devices • 3D sound • stereo rendering
  12. 12. CASA, Geneva, 2016 Slide 12University of Crete & Foundation for Research and Technology - Hellas MOBILE AUGMENTED REALITY SYSTEM Real image for tracking 3D Rendering VR Runtime Engine Skeleton Animation Skin Deformation VRML97 Loader XML Loader Real-time, Marker-less Camera Tracking 3D Static Models Virtual Humans Occluder Models VR Content Repository Binocular HMD Augmented Scene FireWire WebCam Local connectivity Camera Matrix & Background image Cloth Simulation Face & Speech Simulation Sound Rendering Python Script Engine sounds Facial expressions 3D clothes Python Scripts Animations User Mobile Workstation Scene Tracking features Papagiannakis, G., Schertenleib,S., O’Kennedy, B., Arevalo-Poizat, M., Magnenat-Thalmann, N., Stoddart,A.,Thalmann, D.,“Mixing Virtual and Real scenes in the site of ancient Pompeii”, Computer Animation and VirtualWorlds, p 11-24,Volume 16, Issue 1, John Wiley and SonsLtd, February 2005.
  13. 13. CASA, Geneva, 2016 Slide 13University of Crete & Foundation for Research and Technology - Hellas The importance of AR Tracking • Tracking is the basic enabling technology for Augmented Reality • Without accurate tracking you can’t generate the merged real-virtual environment • Tracking is significantly more difficult in AR than in Virtual Environments generate character VR track camera G. Papagiannakis and N. Magnenat-Thalmann,“Mobile Augmented Heritage:Enabling Human Life in ancient Pompeii,” The International Journal of Architectural Computing, Multi-Science Publishing, vol.5, no. 2, pp. 395–415, 2007.
  14. 14. CASA, Geneva, 2016 Slide 14University of Crete & Foundation for Research and Technology - Hellas MOBILE AR CULTURAL HERITAGE GUIDE: LIFEPLUS EU PROJECT Papagiannakis,G., Magnenat-Thalmann,N.,“Mobile Augmented Heritage: Enabling Human Life in ancient Pompeii”,International Journal of Architectural Computing, Multi-Science Publishing,July 2007,issue 02, volume 05, pp.395-415, 2007. G. Papagiannakis, S.Schertenleib,B. O'Kennedy, M.Arevalo-Poizat,N. Magnenat-Thalmann,A. Stoddart, and D.Thalmann,“MixingVirtual and Real scenes in the site of ancient Pompeii,” Computer Animation and Virtual Worlds, John Wiley and Sons Ltd,vol. 16, no.1, pp. 11–24, Feb. 2005.
  15. 15. CASA, Geneva, 2016 Slide 15University of Crete & Foundation for Research and Technology - Hellas MOBILE AR CULTURAL HERITAGE GUIDE: LIFEPLUS EU PROJECT G. Papagiannakis, S.Schertenleib,B. O'Kennedy, M.Arevalo-Poizat,N. Magnenat-Thalmann,A. Stoddart, and D.Thalmann,“MixingVirtual and Real scenes in the site of ancient Pompeii,” Computer Animation and Virtual Worlds, John Wiley and Sons Ltd,vol. 16, no.1, pp. 11–24, Feb. 2005.
  16. 16. CASA, Geneva, 2016 Slide 16University of Crete & Foundation for Research and Technology - Hellas Dialogue-based interaction and Presence in AR A. Egges, G. Papagiannakis, and N. Magnenat-Thalmann, “Presence and interaction in mixed reality environments,” The Visual Computer, Springer, vol. 23, no. 5, pp. 317–333, May 2007.
  17. 17. CASA, Geneva, 2016 Slide 17University of Crete & Foundation for Research and Technology - Hellas Self Adaptive Animation in AR • Dual Quaternions 1,2: • Rigid transformation • Blending • Linear Blending(DLB) • Iterative Blending(DIB) (1+ ε 2 (t0i + t1 j + t2k)).q0 1. Clifford,W.: Mathematical Papers.Macmillan (1882). 2. Kavan, L., Collins,S., Zara, J., O’Sullivan,C.: Skinning with dual quaternions.In: 2007 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pp. 39–46.ACM Press (2007). Chaudhuri, P., Papagiannakis, G., Magnenat-Thalmann, N., "Self adaptive animation based on user perspective", The Visual Computer, Springer-Verlag, 24(7-9), pp. 525- 533, July 2008, presented also in CGI08
  18. 18. CASA, Geneva, 2016 Slide 18University of Crete & Foundation for Research and Technology - Hellas Our current MR Research Questions • Does a single mathematical geometric framework exist, to combine a virtual human's skinning and animation in MR? • How can we enhance presence with gamification? • What about modern game engines and 3D platforms?
  19. 19. CASA, Geneva, 2016 Slide 19University of Crete & Foundation for Research and Technology - Hellas Back to the drawing board…
  20. 20. CASA, Geneva, 2016 Slide 20University of Crete & Foundation for Research and Technology - Hellas glGA Utilities glGARigMesh PlatformWrapper glGAMesh glGAHelper Application Init Display Windows Mac Linux IOS Platforms ImageMagick AntTweakBar GLFW GLEW Assimp GLM External Libraries Textures 3D ModelsShader Programs Resources SmartBody “Keep it simple” character rendering framework G. Papagiannakis, P. Papanikolaou,E. Greassidou,and P. E.Trahanias,“glGA: an OpenGL Geometric Application Framework for a Modern, Shader-based Computer Graphics Curriculum.,” Eurographics’14 Education Track,2014.
  21. 21. CASA, Geneva, 2016 Slide 21University of Crete & Foundation for Research and Technology - Hellas Algebras for Computer Graphics (CG) • Limitations of CG linear algebra representations • Points, lines, areas, vector products • E.g. cross product not defined in 3D • CG scientists apply several fixes: • Separate data structures (e.g. quaternions, dual-quaternions) • Different combinational programming procedures (line-sphere-triangle-plane intersections) • Is there an alternative?
  22. 22. CASA, Geneva, 2016 Slide 22University of Crete & Foundation for Research and Technology - Hellas Some history William Hamilton Hermann Grassmann William Clifford Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  23. 23. CASA, Geneva, 2016 Slide 23University of Crete & Foundation for Research and Technology - Hellas Quaternions Generalises complex numbers, introduced the cross product and some notation still in use today. Confusion over status of vectors, but quaternions are very powerful for describing rotations. Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  24. 24. CASA, Geneva, 2016 Slide 24University of Crete & Foundation for Research and Technology - Hellas Grassmann algebra German schoolteacher (1809- 1877) who struggled for recognition in his own lifetime. Published the Lineale Ausdehnungslehre in 1844. Introduced a new, outer product that is antisymmetric. Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  25. 25. CASA, Geneva, 2016 Slide 25University of Crete & Foundation for Research and Technology - Hellas Properties of the outer product The result of the outer product is a new object: a BIVECTOR Bivectors form a linear space We can visualise bivector addition in 3D a b b+c c a Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  26. 26. CASA, Geneva, 2016 Slide 26University of Crete & Foundation for Research and Technology - Hellas Geometric algebra W.K. Clifford (1845 – 1879) introduced the geometric product of two vectors. The product of two vectors is the sum of a scalar and a bivector. Think of the sum as like the real and imaginary parts of a complex number. Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  27. 27. CASA, Geneva, 2016 Slide 27University of Crete & Foundation for Research and Technology - Hellas Two dimensions 2D sufficient to understand basic results. Construct an orthonormal basis. Parallel vectors commute. Orthogonal vectors anticommute. Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  28. 28. CASA, Geneva, 2016 Slide 28University of Crete & Foundation for Research and Technology - Hellas The bivector The unit bivector has negative square. Follows purely form the axioms of geometric algebra. We have not said anything about complex numbers, or solving polynomial equations. We have invented complex numbers! Chris Doran, Introduction to GA, http://geometry.mrao.cam.ac.uk/home/introduction-to-ga/
  29. 29. CASA, Geneva, 2016 Slide 29University of Crete & Foundation for Research and Technology - Hellas From dual-quaternions to GA • In [Kavan et al. 2008] dual quaternions were employed for geometrical skinning (but not for animation). • In [Chaudhuri et al. 2008] we employed them for animation blending • Dual quaternions: • where qε and q0 are real quaternions and ε is the dual number where ε2 = 0 . This looks just like Clifford’s biquaternionq + ωr where ω2 = 0 [Clifford1873]. • In our recent geometric algebra (GA) [Papagiannakis 2013] framework we replaced quaternions with GA: • Character animationblending • Drop-in, efficient replacement of existingquaternionspherical interpolationmethods • Can this GA framework be employed in modern ARM architectures for mobile AR? ˆq = q1,q2,q3,q4( )+ε q01,q02,q03,q04( )= q0 +εqε [Clifford 1873] CLIFFORD, W.K. 1873. A preliminary sketch of biquaternions. Proc. London Math. Soc. 4:381–395 [Kavan 2008] KAVAN, L., COLLINS, S., ZARA, J., AND CO'SULLIVAN. 2008. Geometric skinning with approximate dual quaternion blending. ACM Transactions on Graphics. [Chaudhuri 2008] CHAUDHURI, P., PAPAGIANNAKIS, G., AND MAGNENAT-THALMANN, N. 2008. Self Adaptive Animation based on User Perspective. The Visual Computer, Springer-Verlag 24, 7-9, 525–533. [Papagiannakis 2013] PAPAGIANNAKIS, G. 2013. Geometric algebra rotors for skinned character animation blending. Technical Brief, ACM SIGGRAPH ASIA 2013, Hong Kong, November 2013, 1–6.
  30. 30. CASA, Geneva, 2016 Slide 30University of Crete & Foundation for Research and Technology - Hellas Contribution: Main Algorithm for GA animation interpolation • employment of Euclidean GA rotors • as fast, drop-in replacements for quaternion algebra, during animation orientation interpolation • We provide two different approaches to express quaternions as GA rotors • fast interpolate between them using the rotor exponential formula: RARn = RB ⇒ R = e −I3a φ (2n)
  31. 31. CASA, Geneva, 2016 Slide 31University of Crete & Foundation for Research and Technology - Hellas • Input: quaternion orientation of existing animation & skinning frameworks • Output: express with different ways input and output quaternions as GA rotors and members of the algebra: i = e1I3= e2 ^ e3 j = e2I3= −e1^ e3 k = e3I3= e1^ e2 ijk = −1 q =1+i+ j + k R = ba = b⋅a + b ^ a = cosφ − I sinφ ⇒ R = e−Iφ ⇒ R = e −I3u φ 2 Contribution: Main Algorithm for GA animation interpolation II Papagiannakis,G. 2013. Geometric algebra rotors for skinned character animation blending. Technical Brief, ACM SIGGRAPH ASIA 2013, Hong Kong, November 2013
  32. 32. CASA, Geneva, 2016 Slide 32University of Crete & Foundation for Research and Technology - Hellas GA for character animation on mobile ARM • Use of GA for encoding orientations, for mobile AR devices (ARM architecture) • The GA rotors e(φ I*α/2n) can convert any axis α (not only at the origin) into a rotational operator. • Quaternions can be used only on other quaternions. • GA rotors are universal operators capable of rotating other subspaces: lines, planes, and volumes as first class operators. G. Papagiannakis, G. Elissavet,P. E.Trahanias,and M.Tsioumas,“A Geometric Algebra Animation Method for Mobile Augmented Reality Simulations in Digital Heritage Sites.,” EuroMed, vol.8740, no.25, pp. 258–267, 2014.
  33. 33. CASA, Geneva, 2016 Slide 33University of Crete & Foundation for Research and Technology - Hellas AR Presence and Gamification How to augment indoors and outdoors Cultural Heritage sites with real time Virtual characters Propose a complete methodology for robust authoring of AR virtual characters using only modern mobile devices Propose an alternative method for smooth rotation of AR scene: Geometric Algebra (GA) rotors
  34. 34. CASA, Geneva, 2016 Slide 34University of Crete & Foundation for Research and Technology - Hellas The Smartbody character animation system Animation system with a wide range of capabilities Harness its advanced character behavioral capabilities. SHAPIRO, A. 2011. Building a character animation system, MIG’ 11,Proceedings of the 4th International Conference on Motion in Games Locomotion Object manipulation Gazing Speech synthesis and Lip Sync Blending Steering Eye movementsGestures Head movement
  35. 35. CASA, Geneva, 2016 Slide 35University of Crete & Foundation for Research and Technology - Hellas A fast and robust pipeline for populating mobile AR scenes with gamified virtual characters [Papaefthymiou et al. 2015a] Papaefthymiou, M., Feng, A., Shapiro, A., and Papagiannakis, G. (2015) 'A fast and robust pipeline for populating mobile AR scenes with gamified virtual characters'. In SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications, SA ’15, pages 22:1–22:8, New York, NY, USA.
  36. 36. CASA, Geneva, 2016 Slide 36University of Crete & Foundation for Research and Technology - Hellas Life-sized crowd simulation in mobile AR • Creation of AR environments with real life-sized crowd and group simulation, on current mobile with cardboard style mobile VR displays [Zikas et al. 2016a] • Virtual characters can support many real life behaviors like locomotion and performing gestures • Extend [Papaefthymiou et al. 2015a] to to include group and crowd simulation in mobile AR. [Zikas et al. 2016a] Zikas Paul, Papaefthymiou Margarita, Mpaxlitzanakis Vasilis, and Papagiannakis George. 2016. Life-sized Group and Crowd simulation in Mobile AR. In Proceedings of the 29th International Conference on Computer Animation and Social Agents (CASA '16). ACM, New York, NY, USA, 79-82.
  37. 37. CASA, Geneva, 2016 Slide 37University of Crete & Foundation for Research and Technology - Hellas A Conformal Geometric Algebra code generator comparison for Virtual Character Simulation in MR • Efficient method for robust authoring (rotation) of Augmented reality scene using Euclidean Geometric Algebra rotors [Papaefthymiou et al. 2016a, Papaefthymiou et al.2015b] • Propose two fast animation blending methods using GA and CGA and compare them with: • Quaternions • Dual Quaternions • Compare efficiency of GA code generators: • Gaigen library • Versor library • Gaalop C++ code generator [Papaefthymiouet al.2015a] Papaefthymiou, M., Papagiannakis, G., Aristidou, A., Ioannides, M., “A conformal geometric algebra framework for mixed reality and mobile display”, Applied Geometric Algebra in Computer Science and Engineering 2015, AGACSE2015, Barcelona, July 2015 [Papaefthymiouet al. 2016b] Papaefthymiou M., Hildenbrand D. and Papagiannakis G., “A Conformal Geometric Algebra code generator comparison for Virtual Character Simulation in Mixed Reality”, Advances in Applied Clifford Algebras Journal (accepted), also presented in GACSE workshop, CGI 2016, Heraklion, Greece, June 2016
  38. 38. CASA, Geneva, 2016 Slide 38University of Crete & Foundation for Research and Technology - Hellas An inclusive Conformal Geometric Algebra GPU animation interpolation and deformation algorithm • An inclusive, single mathematical framework [Papaefthymiou et al. 2016c] replacing previous linear algebra, matrices, euler angles, single and dual- quaternions used for: • character animationinterpolationand • skinning (rotation,translationanddilation) • GA Motor to represent (translationand rotation) • GA Dilator to represent uniform scaling • Final transformation is computed by the geometric product of Motor and Dilator [Papaefthymiou et al. 2016c] Papaefthymiou M., Hildenbrand D. and Papagiannakis G., “An inclusive Conformal Geometric Algebra GPU animation interpolation and deformation algorithm”, The Visual computer Journal (accepted), Springer-Nature, also presented in CGI 2016, Heraklion, Greece, June 2016
  39. 39. CASA, Geneva, 2016 Slide 39University of Crete & Foundation for Research and Technology - Hellas An inclusive Conformal Geometric Algebra GPU animation interpolation and deformation algorithm [Papaefthymiou et al. 2016c] Papaefthymiou M., Hildenbrand D. and Papagiannakis G., “An inclusive Conformal Geometric Algebra GPU animation interpolation and deformation algorithm”, The Visual computer Journal (accepted), Springer-Nature, also presented in CGI 2016, Heraklion, Greece, June 2016
  40. 40. CASA, Geneva, 2016 Slide 40University of Crete & Foundation for Research and Technology - Hellas Mixed Reality Serious Games (MRSGs) for smart education • Creation of two MRSGs [Zikas et al. 2016b] with similar educational content for both VR and AR technologies in order to compare: • gamification elements • learning experience • gameplay • Dynamics: are dependent on the used AR or VR display technology (META AR glasses or Google cardboard) • Mechanics and Components: can absorb minimal display changes and not affect the game’s core. [Zikas et al. 2016b] Paul Zikas, Vasileios Bachlitzanakis, Margarita Papaefthymiou, Steve Kateros, Stylianos Georgiou, Nikos Lydatakis, George Papagiannakis, “Mixed Reality Serious Games for smart education” European Conference on Games Based Learning, , Paisley, Scotland, October 2016 (submitted)
  41. 41. CASA, Geneva, 2016 Slide 41University of Crete & Foundation for Research and Technology - Hellas Summary & Future Work • Our work is formulated as a human-centric, technology-fuelled mobile, mixed reality R&D. Its main idea is summarized as: • Mixed Reality Character Enabling Technologies: • Employing Gamification research, • for immersive, integrated simulation employing geometric algebra • In the future we aim to employ GA for • real-time camera tracking • area light spherical harmonic rotation and rendering
  42. 42. CASA, Geneva, 2016 Slide 42University of Crete & Foundation for Research and Technology - Hellas Acknowledgements Funded by: • Marie Curie Initial TrainingNetwork, FP7-PEOPLE-2013-ITN, under grant agreement n° 608013 • Initial Training Network for Digital Cultural Heritage: Projecting our Past to the Future: ITN-DCH • Marie-Curie Intra-EuropeanFellowshipFP7-PEOPLE-2010- IEF, under grant agreement n° 274669 • “hifi-PRINTER”: High fidelity Presence and Interaction: convergence of computer graphics, vision and robotics for improving human-robot and human-computer interaction • nVIDIA Academic Partnershiph/w grant 2012, 2016
  43. 43. CASA, Geneva, 2016 Slide 43University of Crete & Foundation for Research and Technology - Hellas and one last thing…
  44. 44. CASA, Geneva, 2016 Slide 44University of Crete & Foundation for Research and Technology - Hellas http://www.ics.forth.gr/CGI2016
  45. 45. CASA, Geneva, 2016 Slide 45University of Crete & Foundation for Research and Technology - Hellas Sponsored by Thank you for your attention! Special thanks to: Prof. Nadia Magnenat-Thalmann Prof. Daniel Thalmann CASA’16 conference committee M. Papaefthymiou, S. Kateros, S. Georgiou, N. Lydatakis, P. Zikas, V. Mpahlitzanakis FORTH & University of Crete MIRALab

×