Successfully reported this slideshow.

COSC 426 Lect. 3 -AR Developer Tools

2,884 views

Published on

The third lecture in the COSC 426 graduate class on Augmented Reality taught by Mark Billinghurst at the HIT Lab NZ. This lecture is on AR Developer tools.

Published in: Technology, Education
  • Be the first to comment

COSC 426 Lect. 3 -AR Developer Tools

  1. 1. Lecture 3: AR DL 3 Developer Tools l T l Mark Billinghurst mark.billinghurst@hitlabnz.org July 2011 COSC 426: Augmented Reality
  2. 2. Building Compelling AR Experiences experiences applications tools components Display, Tracking Sony CSL © 2004
  3. 3. Low Level AR LibrariesARToolKit Marker based tracking gFLARToolKit Flash Fl h version of ART lK f ARToolKitSSTT Simple Spatial Template TrackingOpiraO ira Robust Natural Feature Tracking
  4. 4. What ARToolKit? Wh is ART lK ?Marker Tracking Library for AR applications Open Source, Multi platform (Linux Windows MacOS) Source Multi-platform (Linux, Windows,Overlays 3D virtual objects on real markers Uses single tracking marker Determines camera pose information (6 DOF)ARToolKit Websitehttp://www.hitl.washington.edu/artoolkit/http://artoolkit.sourceforge.net/
  5. 5. ARToolKit S f ART lK SoftwareARToolKit version: 2.65 or later 2 65Currently two license models Open Source (GPL): ARToolKit 2 72 2.72 Commercial (ARToolWorks): ARToolKit 4.0OS: LiOS Linux, Windows, MacOS X, iPh Wi d M OS X iPhone/Android /A d idProgramming language: CRelated fR l d software ARToolKit Professional: Commercial version ARToolKitPlus: Advanced version NyARToolkit: Java and C# version FLARToolKit: Flash version
  6. 6. ARToolKit Family ART lKit F il ARToolKit NFTARToolKit ARToolKit Plus JARToolKit (Java) ARToolKit (Symbian) FLARToolKit (Flash) NyToolKit - Java, C#, - Android WM Android, FLARManager ( (Flash) )
  7. 7. ARToolKit ART lK contentsLibraries libAR – tracking b t ac g libARvideo – video capturing libARgsub – image/graphics drawing libARmulti – multi-marker trackingUtilities Camera calibration Marker training
  8. 8. ARToolKit Structure ARvideo.lib DirectShowThree key libraries: AR32.lib AR32 lib – ARToolKit image processing functions ARgsub32.lib – ARToolKit graphics functions ARvideo.lib ARvideo lib – DirectShow video capture class
  9. 9. Additional SoftwareTo build an AR application you may needadditional softwareHigh level rendering library Open VRML, Open Inventor, osgART, etcAudio Library Fmod, etcPeripheral support
  10. 10. What does ARToolKit Calculate?Position of makers in the camera coordinatesPose of markers in the camera coordinatesOutput format 3x4 matrix format to represent the transformation matrix from the marker coordinates to the camera coordinates
  11. 11. Coordinate SystemsC d S
  12. 12. Tracking Range with Pattern SizeRule of thumb – range = 10 x pattern width
  13. 13. Tracking Error with Range
  14. 14. Tracking Error with Angle
  15. 15. AR Application Development
  16. 16. An ARToolKit Application Initialization I iti li ti Load camera and pattern parameters Main Loop M i L Step1. Image capture and display Step2. Marker detection Step3. Marker identification Step4. Getting pose information Step5. Object Interactions/Simulation Step6. Display virtual objects End Application Camera shut down
  17. 17. Image capture: libARvideoReturn the pointer for captured image ARUint8 *arVideoGetImage( void );Pixel format and byte size are defined in config h config.h #define AR_PIX_FORMAT_BGR #define AR_PIX_SIZE 3
  18. 18. Graphics handling: libARgsubSet up and clean up the graphics windowvoid argInit( ARParam *cparam double zoom *cparam, zoom, int fullFlag, int xwin, int ywin, int hmd_flag );void argCleanup( void );cparam: camera parameterzoom: zoom ratiofullFlag: 0: normal, 1: full screen modeXwin, ywin: create small window for debughmd_flag: 0: normal, 1: optical see-through mode
  19. 19. Graphics handling: libARgsubGo into the iterative cyclevoid argMainLoop( void (*mouseFunc)(int btn int state int x int y) btn,int state,int x,int y), void (*keyFunc)(unsigned char key, int x, int y), void (*mainFunc)(void));Swap buffers pvoid argSwapBuffers( void );
  20. 20. Graphics handling: libARgsubSet the window for 2D drawingvoid argDrawMode2D( void ) id D M d 2D( id );Set the window for 3D drawing gvoid argDrawMode3D( void );void argDraw3dCamera( int xwin, int ywin );Display imagevoid argDispImage( ARUi t8 *i id Di I ( ARUint8 *image, int xwin, int ywin );
  21. 21. Sample ARToolKit ApplicationsEx. 1: Simple video displayEx. 2: Detecting a markerEx. 3: Using patternEx. 4: GettingE 4 G tti a 3D i f information tiEx.Ex 5: Virtual object overlay
  22. 22. Ex 1: Simple Video DisplayProgram : sample1.cKey points Loop structure Video image handling Camera parameter handling Window setup Mouse and keyboard handling
  23. 23. Sample1.c Main Functionmain(){ init(); argMainLoop( mouseEvent, keyEvent, a oop); mainLoop);}
  24. 24. Sample1.c - mainLoop Function if( dataPtr = (ARUint8 *) arVideoGetImage()) == NULL ) { arUtilSleep(2); return; } argDrawMode2D(); D M d 2D() argDispImage(dataPtr, 0, 0 ); arVideoCapNext(); argSwapBuffers();
  25. 25. Sample1.c – video initializationConfigure th video inputC fi the id i tvconf = <video configuration string>Start video capturearVideoCapStart();In init(), open the videoarVideoOpen( vconf );arVideoInqSize(&xsize, &ysize);When finished, close the video path arVideoCapStop(); arVideoClose();
  26. 26. Changing Image Size For input capture vconf = “videoWidth=320,videoHeight=240"; Note – the camera must support this image size For display argInit( &cparam 1 5 0 0 0 0 ); &cparam, 1.5, 0, 0, 0,TheTh second parameter means zoom ratio for display d f d l image size related to input image.
  27. 27. Ex. 2: Detecting a MarkerProgram : sample2.cP l 2Key points yp Threshold value Important external variables arDebug – keep thresholded image arImage – pointer for thresholded image arImageProcMode – use 50% image for image processing - AR_IMAGE_PROC_IN_FULL - AR IMAGE PROC IN HALF AR_IMAGE_PROC_IN_HALF
  28. 28. Sample2.c – marker detection/* detect the markers in the video frame */if(arDetectMarker(dataPtr, thif( D t tM k (d t Pt thresh, h &marker_info, &marker_num) < 0 ) { cleanup(); l () exit(0);}for( i = 0; i < marker_num; i++ ) { argDrawSquare(marker_info[i].vertex,0,0);}
  29. 29. Sample2.c – marker_info structure typedef struct { int area; int id; int dir; double cf; double pos[2]; double line[4][3]; double vertex[4][2]; } ARMarkerInfo;
  30. 30. Ex. 3: Using a PatternProgram : sample3.cKey points Pattern files loading Structure of marker information - Region features - Pattern Id, direction - Certainty factor Marker identification
  31. 31. Making a pattern templateUse f iliU of utility program: mk_patt.exeShow the patternPut the corner of red linesegments on the left-top left topvertex of the markerPattern stored as atemplate in a file1:2:1 ratio determines thepattern region used tt i d
  32. 32. Sample3.c – Pattern File Loadingint patt_id;char *patt name = “Data/kanjiPatt” patt_name Data/kanjiPatt//* load pattern file *//if(patt_id=arLoadPatt (patt_name) < 0){ printf ("Pattern file load error !! n"); exit(0);}
  33. 33. Checking for known patterns/* check for known patterns */k = -1;for( i = 0; i < marker num; i++ ) { marker_num; if( marker_info[i].id == patt_id) { /* youve found a pattern */ printf("Found pattern: %d n",patt_id); if( k == -1 ) k = i; else /* make sure you have the best pattern (highest confidence factor) */ if( marker_info[k].cf < marker_info[i].cf ) k i f [k] f k i f [i] f k = i; }}
  34. 34. Ex. 4 – Getting 3D informationProgram : sample4.cKey points Definition of a real marker Transformation matrix - Rotation component -TTranslation component l ti t
  35. 35. Sample4.c – Transformation matrixdouble marker_center[2] = {0.0, 0.0};double marker_width = 80.0;double marker_trans[3][4];arGetTransMat(&marker_info[i],arGetTransMat(&marker info[i] marker_center, marker_width, marker_trans); marker trans);
  36. 36. Finding the Camera PositionThis function sets transformation matrix from marker to camera into marker_trans[3][4]. _ [ ][ ] arGetTransMat(&marker_info[k], marker_center, marker_width, marker_trans);You can see the position information in the values of marker_trans[3][4]. marker trans[3][4] Xpos = marker_trans[0][3]; Ypos = marker_trans[1][3]; Zpos = marker_trans[2][3];
  37. 37. ARToolKit Coordinate Frame
  38. 38. Ex. 5- Virtual Object DisplayProgram : sample5.cKey points OpenGL parameter setting Setup of projection matrix Setup of modelview matrix
  39. 39. Appending your own OpenGL codeSet the camera parameters to OpenGL Projection matrix. g argDrawMode3D(); argDraw3dCamera( 0, 0 );Set the transformation matrix from the marker to the camera to the OpenGL ModelView matrix. argConvGlpara(marker_trans, gl_para); glMatrixMode(GL_MODELVIEW); glLoadMatrixd( gl_para );After calling these functions, your OpenGL objects aredrawn in the real marker coordinates.
  40. 40. 3D CG Model RenderingARToolKit does not have a function to handle3D CG models models.3rd party CG rendering software should beemployed. OpenVRML p OpenSceneGraph etc
  41. 41. Loading Multiple PatternsSample File: LoadMulti.c Uses object c to load object.cObject Structuretypedef struct {t d f t t char name[256]; int id; int visible; double marker_coord[4][2]; double trans[3][4]; double marker_width; double marker_center[2];} ObjectData_T;
  42. 42. Finding Multiple Transforms Create object listObjectData_TObjectData T *object; object; Read in objects - in init( )read_ObjData( char *name, int *objectnum ); Find Transform – in mainLoop( )for( i = 0; i < objectnum; i++ ) { ..Check patterns Ch k tt ..Find transforms for each marker }
  43. 43. Drawing Multiple Objects Send the object list to the draw functiondraw( object, objectnum ); Draw each object individually j yfor( i = 0; i < objectnum; i++ ) { if( object[i].visible == 0 ) continue; ( j [ ] ; argConvGlpara(object[i].trans, gl_para); draw_object( draw object( object[i].id, gl para); gl_para);}
  44. 44. Limitations of ARToolKit Partial occlusions cause tracking failure Affected by lighting and shadows Tracking range depends on marker size Performance depends on number of markers P f d d b f k cf artTag, ARToolKitPlus Pose accuracy depends on distance to marker Pose accuracy depends on angle to marker y p g
  45. 45. ARToolKit in the WorldHundreds fH d d of projects jLarge research community g y
  46. 46. FLARToolKitFlash AS3 Version of the ARToolKit(was ported from NyARToolkit the Java Version of the ARToolkit)enables augmented reality on the Browseruses Papervision3D for as 3D Engineavailable at http://saqoosha.net/dual license, GPL and commercial license
  47. 47. AR Application Components Adobe Flash Papervision 3D FLARToolkit
  48. 48. Source Packages„Original“ FLART lki (Libspark, Saqoosha) O i i l“ FLARToolkit(http://www.libspark.org/svn/as3/FLARToolKit/trunk/ )Start-up-guides Saqoosha (http://saqoosha.net/en/flartoolkit/start-up-guide/ ) Miko Haapoja (http://www.mikkoh.com/blog/?p=182 )„Frameworks Frameworks“ Squidder MultipleMarker – Example (http://www.squidder.com/2009/03/06/flar-how-to-multiple-instances-of-multiple- markers/ ) k / FLARManager (http://words.transmote.com/wp/flarmanager/ )
  49. 49. Papervision 3Dhttp://www.papervision3d.org/Flash based 3D EngineFlash-based 3D-EngineSupports import of 3D Models texturing g animation scene graphalternatives: Away3d, Sandy,…
  50. 50. Papervision ExamplesRed Bull|Flugtage Flight Labhttp://www.redbull.com/flightlab/#/Game/TEMP_0Barcinski JeanJean | 3D Portfoliohttp://www.barcinski-jeanjean.com/more cool papervision websiteshttp://blog.papervision3d.org/2009/04/24/2009-webby-awards-nominees/
  51. 51. AR Tools
  52. 52. Building Compelling AR ExperiencesB ildi C lli E i experiences applications tools Authoring components Tracking, Display Sony CSL © 2004
  53. 53. AR AuthoringSoftware Lib iS ft Libraries osgART, Studierstube, MXRToolKit gPlug-ins to existing software DART (Macromedia Director), mARx (M di Di ) ARStand Alone AMIRE, ComposAR, etcNext Generation iaTAR (Tangible AR)
  54. 54. mARx Plug-in3D Studio Max Plug-inCan model and view AR content at the same time
  55. 55. BuildARhttp://www.buildar.co.nz/ ppStand alone applicationVisual interface for AR model viewing applicationEnables non-programmers to build AR scenes
  56. 56. ImageTclARAdds AR components to ImageTcl http://metlab.cse.msu.edu/imagetclar/ Modular Library (Scripting, Tcl) Supports several tracking systems (vision, magnetic, inertial) Easy to learn but little support, small community
  57. 57. DARTDesigners AR Toolkit http://www.cc.gatech.edu/dart/ http://www cc gatech edu/dart/ AR plug-in for Macromedia Director Developed f d i D l d for designers Visual programming Scripting interface
  58. 58. StudierstubeComplete authoring tool http://studierstube.icg.tu-graz.ac.at/ Framework (Low Level Programming, C++) Modularity, Extensibility, Scalability, Heterogeneity Support for wide range of trackers, displays, input
  59. 59. Metaio UnifEye SDKComplete commercial authoring platform http://www.metaio.com/products/ p p Offers viewer and editor tools Visual interface and low level SDK Delivery on desktop or mobile platforms
  60. 60. OSGART Programming LibraryIntegration of ARToolKit with a High-LevelRendering Engine (OpenSceneGraph) g g ( p p )OSGART= OpenSceneGraph + ARToolKitSupporting Geometric + Photometric Registration pp g g
  61. 61. osgART:FeaturesC++ (but also Python Lua, etc). Python, Lua etc)Multiple Video Input supports: Direct (Firewire/USB Camera), Files, Network by ARvideo, PtGrey, CVCam, VideoWrapper, etc.Benefits of Open Scene Graph Rendering Engine Plug-ins etc Engine, Plug ins,
  62. 62. More LibrariesJARToolKitMRToolKit, MXRToolKit ARLibMRToolKit MXRToolKit, ARLib, OpenVIDIADWARF, Goblin XNAAMIRED’FusionD’F i
  63. 63. Advanced Authoring: iaTAR (Lee 2004) Immersive AR A th i I i Authoring Using real objects to create AR applications
  64. 64. osgARTDeveloping Augmented Reality Applications with osgART
  65. 65. What is a Scene Graph?Tree-like structure for organising a virtual world e.g. VRMLHierarchy of nodes that define:Hi h f d th t d fi Groups (and Switches, Sequences etc…) Transformations Projections Geometry …And states and attributes that define: Materials and textures Lighting and blending …
  66. 66. Scene Graph Example
  67. 67. Benefits of a Scene GraphPerformanceP f Structuring data facilitates optimization - Culling, state management, etc…Abstraction Underlying graphics pipeline is hidden Low-level programming (“how do I Image: sgi display this?”) replaced with high- level l l concepts (“what do I want to t (“ h t d tt display?”)
  68. 68. About Open Scene Graphhttp://www.openscenegraph.org/Open-source scene graph implementationBased on OpenGLObject-oriented C++ following design pattern principlesUsed for simulation, games, research, and industrial projectsActive development community Maintained by Robert Osfield ~2000 mailing list subscribers Documentation project: www osgbooks com www.osgbooks.comUses the OSG Public License (similar to LGPL)
  69. 69. About Open Scene Graph (2)Pirates of the XXI Century Flightgear SCANeR 3DVRII Research Institute EOR VRlab Umeå University
  70. 70. Open Scene Graph FeaturesPlugins for loading and saving 3D: 3D Studio (.3ds), OpenFlight (.flt), Wavefront (.obj)… 2D: .png, .jpg, .bmp, QuickTime moviesNodeKits to extend functionality y e.g. osgShadowCross platformCross-platform support for: Window management (osgViewer) Threading (OpenThreads)
  71. 71. Open Scene Graph Architecture Inter-operability with other environments, e.g. PythonPlugins read and NodeKits extendwrite 2D image and core functionality,3D model files Scene graph and exposing higher-level rendering node types functionality
  72. 72. Some Open Scene Graph Demos osgviewer osgmotionblur osgparticle osgreflect osgdistortion osgfxbrowserYou may want to get the OSG data package: Via SVN: http://www.openscenegraph.org/svn/osg/OpenSceneGraph-Data/trunk
  73. 73. Learning OSGCheck out the Quick Start Guide Free PDF download at http://osgbooks.com/, Physical copy $13USJoin hJ i the mailing li ili list:http://www.openscenegraph.org/projects/osg/wiki/MailingListsBrowse the website: http://www.openscenegraph.org/projects/osgUse the forum: http://forum.openscenegraph.org p p g p gStudy the examplesRead the source?
  74. 74. osgART ART
  75. 75. What is osgART?osgART adds AR to Open Scene GraphFurther developed and enhanced by: Julian Looser Hartmut Seichter H S h Raphael GrassetCurrent version 2.0, Open Source http://www.osgart.org http://www osgart org
  76. 76. osgART Approach: Basic Scene Graph Root Transform T f [0.988 -0.031 -0.048 0.857 0.141 0.513 -0.145 -0.512 0.846 10.939 29.859 -226.733 ] 0 0 0 1 To add Video see-through AR: 3D Object Integrate live video g Apply correct projection matrix Update tracked transformations in realtime
  77. 77. osgART Approach: AR Scene Graph Root Transform 3D Object
  78. 78. osgART Approach: AR Scene Graph Root Virtual Camera Video Transform Layer Video 3D Obj Object Geode
  79. 79. osgART Approach: AR Scene Graph Root Projection matrix from tracker calibration Orthographic Transformation matrix projection Virtual updated from marker Camera tracking in realtime Video Transform LayerFull-screen quadwith live textureupdated fromVideo source Video 3D Obj Object Geode
  80. 80. osgART Approach: AR Scene Graph Root Projection matrix from tracker calibration Orthographic Transformation matrix projection Virtual updated from marker Camera tracking in realtime Video Transform LayerFull-screen quadwith live textureupdated fromVideo source Video 3D Obj Object Geode
  81. 81. osgART ArchitectureLike any video see-through AR library, osgART requires videoinput and tracking capabilities. Video Source e.g. e g DirectShow AR A Library Applica ation Tracking Module (libAR.lib)
  82. 82. osgART ArchitectureosgART uses a plugin architecture so that video sources and trackingtechnologies can be plugged in as necessary OpenCVVideo - VidCapture - CMU1394 - PointGrey SDK - VidereDesign - Video Plugin V VideoWrapper - Vd W VideoInput - VideoSource - App DSVL - os Intranel RTSP - sgART plication Tracke Plugin ARToolKit4 - ARToolkitPlus - er n MXRToolKit - ARLib - bazAR (work in progress) - ARTag (work in progress) -
  83. 83. Basic osgART TutorialDevelopD l a working osgART application from scratch. ki ART li ti f t hUse ARToolKit 2.72 library for tracking and videocapture
  84. 84. osgART Tutorial 1: Basic OSG Viewer Start with the standard Open Scene Graph Viewer We will modify this to do AR! y
  85. 85. osgART Tutorial 1: Basic OSG Viewer The basic osgViewer…#include <osgViewer/Viewer>#include <osgViewer/ViewerEventHandlers>int main(int argc, char* argv[]) { // Create a viewer osgViewer::Viewer viewer; // Create a root node osg::ref_ptr<osg::Group> root = new osg::Group; // Attach root node to the viewer viewer.setSceneData(root.get()); g // Add relevant event handlers to the viewer viewer.addEventHandler(new osgViewer::StatsHandler); viewer.addEventHandler(new osgViewer::WindowSizeHandler); viewer.addEventHandler(new osgViewer::ThreadingHandler); viewer.addEventHandler(new viewer addEventHandler(new osgViewer::HelpHandler); // Run the viewer and exit the program when the viewer is closed return viewer.run();}
  86. 86. osgART Tutorial 2: Adding VideoAdd a video plugin Load, configure, start video capture…Add a video background Create, link to video, add to scene-graph g p
  87. 87. osgART Tutorial 2: Adding Video New code to load and configure a Video Plugin:// Preload the video and trackerint _video_id = osgART::PluginManager::getInstance()->load("osgart_video_artoolkit2");// Load a video plugin.osg::ref_ptr<osgART::Video> video = dynamic_cast<osgART::Video*>(osgART::PluginManager::getInstance()->get(_video_id));// Check if an instance of the video stream could be createdif (!video.valid()) { // Without video an AR application can not work. Quit if none found. osg::notify(osg::FATAL) << "Could not initialize video plugin!" << std::endl; exit(-1);}// Open the video. This will not yet start the video stream but will// get information about the format of the video which is essential// for the connected tracker.video->open();
  88. 88. osgART Tutorial 2: Adding Video New code to add a live video backgroundosg::Group* createImageBackground(osg::Image* video) { osgART::VideoLayer* _layer = new osgART::VideoLayer(); _layer->setSize(*video); osgART::VideoGeode* _geode = new osgART::VideoGeode(osgART::VideoGeode::USE_TEXTURE_2D, video); addTexturedQuad(*_geode, video->s(), video->t()); _layer->addChild(_geode); return _layer;} In the main function…osg::ref_ptr<osg::Group> videoBackground = createImageBackground(video.get());videoBackground->getOrCreateStateSet()->setRenderBinDetails(0, "RenderBin");root->addChild(videoBackground.get());video->start();
  89. 89. osgART Tutorial 3: TrackingAdd a Tracker plugin T k l Load, configure, link to videoAdd a Marker to track Load, Load activateTracked node Create, li k with marker via tracking callbacks C link i h k i ki llb kPrint out the tracking data
  90. 90. osgART Tutorial 3: Tracking Load a t ac g plugin and associate it with the video plugin oa tracking p ug a assoc ate t w t t e v eo p ugint _tracker_id = osgART::PluginManager::getInstance()->load("osgart_tracker_artoolkit2");osg::ref_ptr<osgART::Tracker> tracker = dynamic_cast<osgART::Tracker*>(osgART::PluginManager::getInstance()->get(_tracker_id));if (!tracker.valid()) { // Without tracker an AR application can not work. Quit if none found. osg::notify(osg::FATAL) << "Could not initialize tracker plugin!" << std::endl; exit(-1);}// get the tracker calibration objectosg::ref_ptr<osgART::Calibration> calibration = tracker->getOrCreateCalibration();// load a calibration fileif (!calibration->load("data/camera para dat")) (!calibration->load( data/camera_para.dat )){ // the calibration file was non-existing or couldnt be loaded osg::notify(osg::FATAL) << "Non existing or incompatible calibration file" << std::endl; exit(-1);}// set the image source for the trackertracker->setImage(video.get());osgART::TrackerCallback::addOrSet(root.get(), tracker.get());// create the virtual camera and add it to the sceneosg::ref_ptr<osg::Camera> cam = calibration->createCamera();root->addChild(cam.get());
  91. 91. osgART Tutorial 3: Tracking Load a marker and activate it Associate it with a transformation node (via event callbacks) Add the transformation node to the virtual camera nodeosg::ref_ptr<osgART::Marker> marker = tracker->addMarker("single;data/patt.hiro;80;0;0");if (!marker.valid()){ // Without marker an AR application can not work. Quit if none found. osg::notify(osg::FATAL) << "Could not add marker!" << std::endl; exit(-1);}marker->setActive(true);osg::ref_ptr<osg::MatrixTransform> arTransform = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransform.get(), marker.get());cam->addChild(arTransform.get()); Add a debug callback to print out information about the tracked markerosgART::addEventCallback(arTransform.get(), new osgART::MarkerDebugCallback(marker.get()));
  92. 92. osgART Tutorial 3: TrackingTracking information is goutput to console
  93. 93. osgART Tutorial 4: Adding ContentNow put the trackin data t use!N t tracking to se!Add content to the tracked transformBasic cube code arTransform->addChild(osgART::testCube()); arTransform->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");
  94. 94. osgART Tutorial 5: Adding 3D ModelOpen Scene Graph can load some 3D formats directly:O S G h l d f di l e.g. Wavefront (.obj), OpenFlight (.flt), 3D Studio (.3ds), COLLADA Others need to be convertedSupport for some formats is much better than others e.g. OpenFlight good, 3ds hit and miss.Recommend native .osg and .ive formats .osg – ASCII representation of scene graph .ive – Binary OSG file. Can contain hold textures ive file textures.osgExp : Exporter for 3DS Max is a good choice http://sourceforge.net/projects/osgmaxexpOtherwise .3ds files from TurboSquid can work
  95. 95. osgART Tutorial 5: Adding 3D Model Replace the simple cube with a 3D model Models are loaded using the osgDB::readNodeFile() functionstd::string filename = "media/hollow_cube.osg";arTransform >addChild(osgDB::readNodeFile(filename));arTransform->addChild(osgDB::readNodeFile(filename)); Export to .osg 3D Studio M St di Max osgART Note: Scale is important. Units are in mm. important mm
  96. 96. osgART Tutorial 6: Multiple MarkersRepeat the process so far to track more thanone marker simultaneouslyy
  97. 97. osgART Tutorial 6: Multiple Markers Repeat the process so far to track more than one marker R h f k h k Load and activate two markersosg::ref_ptr<osgART::Marker> markerA = tracker->addMarker("single;data/patt.hiro;80;0;0");markerA->setActive(true);osg::ref_ptr<osgART::Marker> markerB = tracker->addMarker("single;data/patt.kanji;80;0;0"); _markerB->setActive(true); Create two transformations, attach callbacks, and add modelsosg::ref_ptr<osg::MatrixTransform> arTransformA = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformA.get(), markerA.get());arTransformA->addChild(osgDB::readNodeFile("media/hitl_logo.osg"));arTransformA->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformA.get());osg::ref_ptr<osg::MatrixTransform> arTransformB = new osg::MatrixTransform();osgART::attachDefaultEventCallbacks(arTransformB.get(), markerB.get());arTransformB->addChild(osgDB::readNodeFile("media/gist_logo.osg"));arTransformB->getOrCreateStateSet()->setRenderBinDetails(100, "RenderBin");cam->addChild(arTransformB.get());
  98. 98. osgART Tutorial 6: Multiple Markers
  99. 99. Basic osgART Tutorial: SummaryStandard OSG Viewer Addition of Video Addition of TrackingAddition of basic 3D Addition of 3D Model Multiple Markers g p graphics
  100. 100. FLARManager FLARMhttp://transmote.com/flarhttp://transmote com/flar
  101. 101. FLARManager:Makes building FLARToolkit apps easierIs open-source, with a free and commercial license open source,Is designed to allow exploration of both augmentedreality and alternative controllersWas initiated by Eric Socolofsky, developed withcontributions from FLARToolkit community ib i f FLART lki iDecouples FLARToolkit from Papervision3DConfiguration without recompilation, via xml config
  102. 102. FLARManager: featuresGives more control over application environmentProvides multiple input optionsRobust multiple marker managementSupports multiple 3D frameworksOffers features f optimizationOff f for i i iAllows for customization
  103. 103. Resources
  104. 104. WebsitesSoftware Download http://artoolkit.sourceforge.net/ http://artoolkit sourceforge net/ARToolKit Documentation http://www.hitl.washington.edu/artoolkit/ARToolKit Forum http://www.hitlabnz.org/wiki/ForumARToolworks Inc http://www.artoolworks.com/ p
  105. 105. ARToolKit Plus http://studierstube.icg.tu- graz.ac.at/handheld_ar/artoolkitplus.php t/h dh ld / t lkit l hosgART http://www.osgart.org/FLARToolKit http://www.libspark.org/wiki/saqoosha/FLARToolKit/FLARManager http://words.transmote.com/wp/flarmanager/ http://words transmote com/wp/flarmanager/
  106. 106. BooksInteractive EI Environments with Open-Source hO SSoftware: 3D Walkthroughs and AugmentedReality for Architects with Bl d 2 43 DARTR li f A hi i h Blender 2.43,3.0 and ARToolKit 2.72 by Wolfgang HöhlA Hitchhikers Guide to Virtual Reality by Karen y yMcMenemy and Stuart Ferguson
  107. 107. More Information• M k Billi h t Mark Billinghurst – mark.billinghurst@hitlabnz.org• Websites – www.hitlabnz.org hi l b

×