426 lecture6a osgART Development


Published on

COSC 426 Lecture 6a on using osgART

Published in: Technology, Art & Photos
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

426 lecture6a osgART Development

  1. 1. COSC 426: Augmented Reality Mark Billinghurst mark.billinghurst@hitlabnz.org August 14th 2012 Lecture 6: osgART
  2. 2. osgARTDeveloping Augmented Reality Applications with osgART
  3. 3. What is a Scene Graph?  Tree-like structure for organising a virtual world   e.g. VRML  Hierarchy of nodes that define:   Groups (and Switches, Sequences etc…)   Transformations   Projections   Geometry   …  And states and attributes that define:   Materials and textures   Lighting and blending   …
  4. 4. Scene Graph Example
  5. 5. Benefits of a Scene Graph  Performance   Structuring data facilitates optimization -  Culling, state management, etc…  Abstraction   Underlying graphics pipeline is hidden   Low-level programming (“how do I display this?”) replaced with high-level concepts Image: sgi (“what do I want to display?”)
  6. 6. About Open Scene Graph  http://www.openscenegraph.org/  Open-source scene graph implementation  Based on OpenGL  Object-oriented C++ following design pattern principles  Used for simulation, games, research, and industrial projects  Active development community   Maintained by Robert Osfield   ~2000 mailing list subscribers   Documentation project: www.osgbooks.com  Uses the OSG Public License (similar to LGPL)
  7. 7. About Open Scene Graph (2)Pirates of the XXI Century Flightgear SCANeR 3DVRII Research Institute EOR VRlab Umeå University
  8. 8. Open Scene Graph Features  Plugins for loading and saving   3D: 3D Studio (.3ds), OpenFlight (.flt), Wavefront (.obj)…   2D: .png, .jpg, .bmp, QuickTime movies  NodeKits to extend functionality   e.g. osgShadow  Cross-platform support for:   Window management (osgViewer)   Threading (OpenThreads)
  9. 9. Open Scene Graph Architecture Inter-operability with other environments, e.g. PythonPlugins read and NodeKits extendwrite 2D image and core functionality,3D model files Scene graph and exposing higher-level rendering node types functionality
  10. 10. Some Open Scene Graph Demos osgviewer osgmotionblur osgparticle osgreflect osgdistortion osgfxbrowser  You may want to get the OSG data package:   Via SVN: http://www.openscenegraph.org/svn/osg/OpenSceneGraph-Data/trunk
  11. 11. Learning OSG  Check out the Quick Start Guide   Free PDF download at http://osgbooks.com/, Physical copy $13US  Join the mailing list: http://www.openscenegraph.org/projects/osg/wiki/ MailingLists  Browse the website: http://www.openscenegraph.org/projects/osg  Use the forum: http://forum.openscenegraph.org  Study the examples  Read the source? 
  12. 12. osgART
  13. 13. What is osgART?  osgART adds AR to Open Scene Graph  Further developed and enhanced by:   Julian Looser   Hartmut Seichter   Raphael Grasset  Current version 2.0, Open Source   http://www.osgart.org
  14. 14. osgART Approach: Basic Scene Graph Root [ ] 0.988 -0.031 -0.145 0 -0.048 0.857 -0.512 0 Transform 0.141 0.513 0.846 10.939 29.859 -226.733 0 1   To add Video see-through AR: 3D Object   Integrate live video   Apply correct projection matrix   Update tracked transformations in realtime
  15. 15. osgART Approach: AR Scene Graph Root Transform 3D Object
  16. 16. osgART Approach: AR Scene Graph Root Virtual Camera Video Transform Layer Video 3D Object Geode
  17. 17. osgART Approach: AR Scene Graph Root Projection matrix from tracker calibration Orthographic Transformation matrix projection Virtual updated from marker Camera tracking in realtime Video Transform LayerFull-screen quadwith live textureupdated fromVideo source Video 3D Object Geode
  18. 18. osgART Approach: AR Scene Graph Root Projection matrix from tracker calibration Orthographic Transformation matrix projection Virtual updated from marker Camera tracking in realtime Video Transform LayerFull-screen quadwith live textureupdated fromVideo source Video 3D Object Geode
  19. 19. osgART Architecture  Like any video see-through AR library, osgART requires video input and tracking capabilities. Video Source e.g. DirectShow AR Library Application Tracking Module (libAR.lib)
  20. 20. osgART Architecture  osgART uses a plugin architecture so that video sources and tracking technologies can be plugged in as necessary OpenCVVideo - VidCapture - CMU1394 - PointGrey SDK - VidereDesign - Video Plugin VideoWrapper - VideoInput - VideoSource - Application DSVL - osgART Intranel RTSP - Tracker Plugin ARToolKit4 - ARToolkitPlus - MXRToolKit - ARLib - bazAR (work in progress) - ARTag (work in progress) -
  21. 21. Basic osgART Tutorial  Develop a working osgART application from scratch.  Use ARToolKit 2.72 library for tracking and video capture
  22. 22. osgART Tutorial 1: Basic OSG Viewer  Start with the standard Open Scene Graph Viewer  We will modify this to do AR!
  23. 23. osgART Tutorial 1: Basic OSG Viewer  The basic osgViewer…#include <osgViewer/Viewer>#include <osgViewer/ViewerEventHandlers>int main(int argc, char* argv[]) { // Create a viewer osgViewer::Viewer viewer; // Create a root node osg::ref_ptr<osg::Group> root = new osg::Group; // Attach root node to the viewer viewer.setSceneData(root.get()); // Add relevant event handlers to the viewer viewer.addEventHandler(new osgViewer::StatsHandler); viewer.addEventHandler(new osgViewer::WindowSizeHandler); viewer.addEventHandler(new osgViewer::ThreadingHandler); viewer.addEventHandler(new osgViewer::HelpHandler); // Run the viewer and exit the program when the viewer is closed return viewer.run();}
  24. 24. osgART Tutorial 2: Adding Video  Add a video plugin   Load, configure, start video capture…  Add a video background   Create, link to video, add to scene-graph
  25. 25. osgART Tutorial 2: Adding Video  New code to load and configure a Video Plugin:// Preload the video and trackerint _video_id = osgART::PluginManager::getInstance()->load("osgart_video_artoolkit2");// Load a video plugin.osg::ref_ptr<osgART::Video> video = dynamic_cast<osgART::Video*>(osgART::PluginManager::getInstance()->get(_video_id));// Check if an instance of the video stream could be createdif (!video.valid()) { // Without video an AR application can not work. Quit if none found. osg::notify(osg::FATAL) << "Could not initialize video plugin!" <<std::endl; exit(-1);}// Open the video. This will not yet start the video stream but will// get information about the format of the video which is essential// for the connected tracker.video->open();
  26. 26. osgART Tutorial 2: Adding Video   New code to add a live video backgroundosg::Group* createImageBackground(osg::Image* video) { osgART::VideoLayer* _layer = new osgART::VideoLayer(); _layer->setSize(*video); osgART::VideoGeode* _geode = new osgART::VideoGeode(osgART::VideoGeode::USE_TEXTURE_2D, video); addTexturedQuad(*_geode, video->s(), video->t()); _layer->addChild(_geode); return _layer;}   In the main function… osg::ref_ptr<osg::Group> videoBackground = createImageBackground(video.get()); videoBackground->getOrCreateStateSet()->setRenderBinDetails(0, "RenderBin"); root->addChild(videoBackground.get()); video->start();
  27. 27. osgART Tutorial 3: Tracking  Add a Tracker plugin   Load, configure, link to video  Add a Marker to track   Load, activate  Tracked node   Create, link with marker via tracking callbacks  Print out the tracking data
  28. 28. osgART Tutorial 3: Tracking  Load a tracking plugin and associate it with the video pluginint _tracker_id = osgART::PluginManager::getInstance()->load("osgart_tracker_artoolkit2");osg::ref_ptr<osgART::Tracker> tracker = dynamic_cast<osgART::Tracker*>(osgART::PluginManager::getInstance()->get(_tracker_id));if (!tracker.valid()) { // Without tracker an AR application can not work. Quit if none found. osg::notify(osg::FATAL) << "Could not initialize tracker plugin!" << std::endl; exit(-1);}// get the tracker calibration objectosg::ref_ptr<osgART::Calibration> calibration = tracker->getOrCreateCalibration();// load a calibration fileif (!calibration->load("data/camera_para.dat")){ // the calibration file was non-existing or couldnt be loaded osg::notify(osg::FATAL) << "Non existing or incompatible calibration file" << std::endl; exit(-1);}// set the image source for the trackertracker->setImage(video.get());osgART::TrackerCallback::addOrSet(root.get(), tracker.get());// create the virtual camera and add it to the sceneosg::ref_ptr<osg::Camera> cam = calibration->createCamera();root->addChild(cam.get());
  29. 29. osgART Tutorial 3: Tracking  Load a marker and activate it  Associate it with a transformation node (via event callbacks)  Add the transformation node to the virtual camera nodeosg::ref_ptr<osgART::Marker> marker = tracker->addMarker("single;data/patt.hiro;80;0;0");if (!marker.valid()){ // Without marker an AR application can not work. Quit if none found. osg::notify(osg::FATAL) << "Could not add marker!" << std::endl; exit(-1);}marker->setActive(true);osg::ref_ptr<osg::MatrixTransform> arTransform = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransform.get(), marker.get());cam->addChild(arTransform.get());  Add a debug callback to print out information about the tracked markerosgART::addEventCallback(arTransform.get(), new osgART::MarkerDebugCallback(marker.get()));
  30. 30. osgART Tutorial 3: Tracking  Tracking information is output to console
  31. 31. osgART Tutorial 4: Adding Content  Now put the tracking data to use!  Add content to the tracked transform  Basic cube code arTransform->addChild(osgART::testCube()); arTransform->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");
  32. 32. osgART Tutorial 5: Adding 3D Model  Open Scene Graph can load some 3D formats directly:   e.g. Wavefront (.obj), OpenFlight (.flt), 3D Studio (.3ds), COLLADA   Others need to be converted  Support for some formats is much better than others   e.g. OpenFlight good, 3ds hit and miss.  Recommend native .osg and .ive formats   .osg – ASCII representation of scene graph   .ive – Binary OSG file. Can contain hold textures.  osgExp : Exporter for 3DS Max is a good choice   http://sourceforge.net/projects/osgmaxexp  Otherwise .3ds files from TurboSquid can work
  33. 33. osgART Tutorial 5: Adding 3D Model  Replace the simple cube with a 3D model  Models are loaded using the osgDB::readNodeFile() functionstd::string filename = "media/hollow_cube.osg";arTransform->addChild(osgDB::readNodeFile(filename)); Export to .osg 3D Studio Max osgART  Note: Scale is important. Units are in mm.
  34. 34. osgART Tutorial 6: Multiple Markers  Repeat the process so far to track more than one marker simultaneously
  35. 35. osgART Tutorial 6: Multiple Markers  Repeat the process so far to track more than one marker  Load and activate two markersosg::ref_ptr<osgART::Marker> markerA = tracker->addMarker("single;data/patt.hiro;80;0;0");markerA->setActive(true);osg::ref_ptr<osgART::Marker> markerB = tracker->addMarker("single;data/patt.kanji;80;0;0");markerB->setActive(true);  Create two transformations, attach callbacks, and add modelsosg::ref_ptr<osg::MatrixTransform> arTransformA = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformA.get(), markerA.get());arTransformA->addChild(osgDB::readNodeFile("media/hitl_logo.osg"));arTransformA->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformA.get());osg::ref_ptr<osg::MatrixTransform> arTransformB = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformB.get(), markerB.get());arTransformB->addChild(osgDB::readNodeFile("media/gist_logo.osg"));arTransformB->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformB.get());
  36. 36. osgART Tutorial 6: Multiple Markers
  37. 37. Basic osgART Tutorial: SummaryStandard OSG Viewer Addition of Video Addition of TrackingAddition of basic 3D Addition of 3D Model Multiple Markers graphics
  38. 38. FLARManagerhttp://transmote.com/flar
  39. 39. FLARManager:  Makes building FLARToolkit apps easier  Is open-source, with a free and commercial license  Is designed to allow exploration of both augmented reality and alternative controllers  Decouples FLARToolkit from Papervision3D  Configuration without recompilation, via xml config
  40. 40. FLARManager: features  Gives more control over application environment  Provides multiple input options  Robust multiple marker management  Supports multiple 3D frameworks  Offers features for optimization  Allows for customization
  41. 41. Resources
  42. 42. Websites  Software Download   http://artoolkit.sourceforge.net/  ARToolKit Documentation   http://www.hitl.washington.edu/artoolkit/  ARToolKit Forum   https://www.artoolworks.com/community/forum/  ARToolworks Inc   http://www.artoolworks.com/
  43. 43.   ARToolKit Plus   http://studierstube.icg.tu-graz.ac.at/handheld_ar/ artoolkitplus.php  osgART   http://www.osgart.org/  FLARToolKit   http://www.libspark.org/wiki/saqoosha/FLARToolKit/  FLARManager   http://words.transmote.com/wp/flarmanager/
  44. 44. Books  Interactive Environments with Open-Source Software: 3D Walkthroughs and Augmented Reality for Architects with Blender 2.43, DART 3.0 and ARToolKit 2.72 by Wolfgang Höhl  A Hitchhikers Guide to Virtual Reality by Karen McMenemy and Stuart Ferguson
  45. 45. More Information•  Mark Billinghurst –  mark.billinghurst@hitlabnz.org •  Websites –  www.hitlabnz.org