Augmented Reality With FlarToolkit and Papervision3D


Published on

Augmented Reality With FlarToolkit and Papervision3D

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Augmented Reality With FlarToolkit and Papervision3D

  1. 1. Augmented Reality with Flex, FlarToolkit and Papervision3D Roman Protsyk 20.12.2010
  2. 2. Agenda <ul><li>Augmented reality definition and examples </li></ul><ul><li>Augmented Reality in web browser with FlarToolkit </li></ul><ul><li>Augmented reality demo creation using Flex, FlarToolkit and Papervision3D demo </li></ul>
  3. 3. What is Augmented Reality? <ul><li>Augmented Reality (AR) </li></ul><ul><li>term referring to all the projects to complement the reality of any virtual elements </li></ul>The term was proposed by Tom Caudell, a researcher of Boeing Corporation,in 1990 . Ronald Azuma in 1997 defined the augmented reality as a system that 1. Combines the virtual and real. 2. Interact in real-time 3. Works in 3D.
  4. 4. Nintendo – Wii Augmented Reality Examples : Game Consoles Sony PlayStation 3 – Controller + Eye Cam Sony PlayStation 3 – Eye Pet XBox – Project Natal
  5. 5. 1. Mobilizy – Wikitude 2. Nokia – Concept with Image Comparison 3. Whiteapp – AR iPhone apps Augmented Reality examples: Location Based Mobile Apps
  6. 6. Augmented Reality examples <ul><li>There are AR applications outside of console games, including military and consumer products, too. Night-vision goggles and targeting-assistance technology help marksmen in battle, and children's stories come to life with AR-enhanced books. The uses are vast. </li></ul>
  7. 7. Augmented reality in web browser <ul><li>The world’s first Flash based augmented reality library ported from NyARToolkit (Java ported version of ARToolKit ) by Tomohiko Koyama aka Saqoosha . </li></ul><ul><li>First version was released in May 2008. </li></ul><ul><li>Porting was completed in about 1 week. </li></ul><ul><li>FLAR is a part of Spark project - Flash/ActionScript open source community . </li></ul>
  8. 8. FlarToolkit example <ul><li> </li></ul>
  9. 9. FlarToolkit example <ul><li> </li></ul>
  10. 10. Basic usage <ul><li>FLARToolKit doesn’t render the 3D objects. </li></ul><ul><li>Calculates only the orientation of the marker. </li></ul><ul><li>Use other 3D engines to render 3D objects. </li></ul><ul><li>- Papervision3D </li></ul><ul><li>- Away3D (both 2.x and 3.x) </li></ul><ul><li>- Sandy </li></ul><ul><li>- Alternativa3D </li></ul><ul><li>Start-up guide for FLARToolKit. </li></ul><ul><li> </li></ul>
  11. 11. Demo <ul><li>Let’s create demo application and look inside FLARToolkit in parallel </li></ul>
  12. 12. Creation of sample application <ul><li>Create marker image “pattern16.pat”: </li></ul><ul><li> </li></ul><ul><li>Open Flex Builder </li></ul><ul><li>Create sample ActionScript “FlarDemo” project </li></ul><ul><li>Download FlarToolkit: </li></ul><ul><li>Download Papervision3d: </li></ul>
  13. 13. Inside FLARToolKit Marker pattern specification: Needs to be square. Only 50% of center area is used in matching process. Marker pattern is 16 x 16 dots bitmap by default. Size of the pattern can be lager, but it requires more recognition time.
  14. 14. Creation of sample application <ul><li>Get “camera_para.dat” file which contains default properties for camera from: </li></ul><ul><li>Camera calibration - is the problem of obtaining internal and external camera parameters from the available photographs or video filmed with it </li></ul>
  15. 15. Inside FLARToolKit <ul><li>Capture the image from webcam. (Step 1 of 7) </li></ul><ul><ul><li>Simply use Camera and Video classes. </li></ul></ul><ul><ul><li>then draw video instance onto the BitmapData. </li></ul></ul>
  16. 16. Inside FLARToolKit <ul><li>Binarize input image. (Step 2 of 7) </li></ul>
  17. 17. Inside FLARToolKit <ul><li>Labeling. (Step 3 of 7) </li></ul>
  18. 18. Inside FLARToolKit <ul><li>Find squares. (Step 4 of 7) </li></ul>
  19. 19. Inside FLARToolKit <ul><li>Matching with patterns. </li></ul><ul><li>(Step 5 of 7) </li></ul><ul><li>Extract the images from detected rectangles using homography transform. </li></ul>
  20. 20. Inside FLARToolKit <ul><li>Render the 3D objects. (Step 6 of 7) </li></ul><ul><li>- FLARToolKit includes support classes which converts FLAR’s transform matrix to each 3D engine internal matrix classes. </li></ul>
  21. 21. Inside FLARToolKit <ul><li>Calculate transform matrix. (Step 7 of 7) </li></ul><ul><li>- It’s calculated by square’s vertices of the detected marker. </li></ul>
  22. 22. Inside FLARToolkit
  23. 23. Demo application <ul><li>public function FlarDemo() </li></ul><ul><li>{ </li></ul><ul><li>setupFlar(); </li></ul><ul><li>setupCamera(); </li></ul><ul><li>setupBitmapData(); </li></ul><ul><li>setupPV3D(); </li></ul><ul><li>addEventListener(Event.ENTER_FRAME, loop); </li></ul><ul><li>} </li></ul><ul><li>private function setupFlar():void </li></ul><ul><li>{ </li></ul><ul><li>fparams= new FLARParam(); </li></ul><ul><li>fparams.loadARParam( new params() as ByteArray); </li></ul><ul><li>mpattern= new FLARCode(16, 16); </li></ul><ul><li>mpattern.loadARPatt( new pattern()); </li></ul><ul><li>} </li></ul><ul><li>private function setupCamera():void </li></ul><ul><li>{ </li></ul><ul><li>vid= new Video(640, 480); </li></ul><ul><li>cam=Camera.getCamera(); </li></ul><ul><li>cam.setMode(640, 480, 30); </li></ul><ul><li>vid.attachCamera(cam); </li></ul><ul><li>addChild(vid); </li></ul><ul><li>} </li></ul><ul><li>private function setupBitmapData():void </li></ul><ul><li>{ </li></ul><ul><li>bmd = new BitmapData(640, 480); </li></ul><ul><li>bmd.draw(vid); </li></ul><ul><li>raster = new FLARRgbRaster_BitmapData(bmd); </li></ul><ul><li>detector = new FLARSingleMarkerDetector(fparams,mpattern, 80); </li></ul><ul><li>} </li></ul>
  24. 24. Demo application <ul><li>private function setupPV3D():void </li></ul><ul><li>{ </li></ul><ul><li>scene = new Scene3D(); </li></ul><ul><li>camera = new FLARCamera3D(fparams); </li></ul><ul><li>container = new FLARBaseNode(); </li></ul><ul><li>scene.addChild(container); </li></ul><ul><li>var pl:PointLight3D = new PointLight3D(); </li></ul><ul><li>pl.x = 1000; </li></ul><ul><li>pl.y = 1000; </li></ul><ul><li>pl.z = -1000; </li></ul><ul><li>var ml:MaterialsList = new MaterialsList({all: new FlatShadeMaterial(pl)}); </li></ul><ul><li>var cube1:Cube = new Cube(ml,30,30,30); </li></ul><ul><li>var cube2:Cube = new Cube(ml,30,30,30); </li></ul><ul><li>cube2.z = 50; </li></ul><ul><li>var cube3:Cube = new Cube(ml,30,30,30); </li></ul><ul><li>cube3.z = 100; </li></ul><ul><li>container.addChild(cube1); </li></ul><ul><li>container.addChild(cube2); </li></ul><ul><li>container.addChild(cube3); </li></ul><ul><li>bre = new BasicRenderEngine(); </li></ul><ul><li>trans = new FLARTransMatResult(); </li></ul><ul><li>vp = new Viewport3D(); </li></ul><ul><li>addChild(vp); </li></ul><ul><li>} </li></ul><ul><li>private function loop(e:Event):void </li></ul><ul><li>{ </li></ul><ul><li>bmd.draw(vid); </li></ul><ul><li>try </li></ul><ul><li>{ </li></ul><ul><li>if(detector.detectMarkerLite(raster, 80) && detector.getConfidence()>0.5) </li></ul><ul><li>{ </li></ul><ul><li>vp.visible = true; </li></ul><ul><li>detector.getTransformMatrix(trans); </li></ul><ul><li>container.setTransformMatrix(trans); </li></ul><ul><li>bre.renderScene(scene, camera, vp); </li></ul><ul><li>} </li></ul><ul><li>else </li></ul><ul><li>{ </li></ul><ul><li>vp.visible = false </li></ul><ul><li>} </li></ul><ul><li>} </li></ul><ul><li>catch(e:Error){} </li></ul><ul><li>} </li></ul>
  25. 25. Demo application Loading of different Papervision3D objects
  26. 26. Future of FLARToolKit
  27. 27. Alchemy version of FLARToolKit <ul><li>Done by Ryo Iizuka (aka nyatla) </li></ul><ul><li>- 2 times faster than AS3 version. </li></ul><ul><li>- You can use more polygons to create 3D models. </li></ul><ul><li>- It runs on more slower PCs. </li></ul><ul><li>- Committed at alchemy branch. </li></ul><ul><li> </li></ul>
  28. 28. Links <ul><li>FLARToolkit project web page by Saqoosha </li></ul><ul><li> </li></ul><ul><li>Google group: FLARToolKit-userz . </li></ul><ul><li> </li></ul><ul><li>FlarToolkit/Flash Augmented Reality Getting Started by Mikko Haapoja </li></ul><ul><li> </li></ul>
  29. 29. Thank you <ul><li>Questions </li></ul>Contacts: Roman Protsyk Teodorovych [email_address] Senior Software Developer