FLAR Workflow


Published on

By now you have heard about Flash Augmented Reality and how it is taking the Flash Development community by storm! Whether you are looking for how to get started, how to improve your own experiments or have a client who desperately needs AR on their site, this session is for you.

In this keynote I cover getting up and running as well as the ideal workflow for testing/deploying your creation. I also cover the basics then quickly move into how to build a FLAR Emulator for easy testing/debugging as well as general usability/performance issues. Finally we will look at my own experiments with AR, how they were built and highlight some of the best uses of Flash AR today.

The goal of this presentation is to teach you how to build a solid reusable foundation for all of your Flash AR projects which will allow you to quickly prototype your ideas. All code covered in this session is open source and free to use. Documentation on how it works will also be handed out as well.

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

  • FLAR Workflow

    1. 1. FLAR Workflow
    2. 2. Who is Jesse Freeman?
    3. 3. Technical Architect at Roundarch 10 Years of Flash experience Worked for Heavy.com, Major League Baseball, The New York Jets, AKQA, HBO, FOX, Arista Records and BBDO Specialize in Flash workflow, large scale xml driven Flash apps and building Flash Frameworks. Twitter - @TheFlashBum
    4. 4. Let’s Talk about Augmented Reality
    5. 5. Augmented Reality (AR) is a field of computer research which deals with the combination of real-world and computer-generated data (virtual reality), where computer graphics objects are blended into real footage in real time.
    6. 6. What is FLAR? • Flash Augmented Reality allows us superimpose graphics over video. • In Flash, this is usually done with a webcam and a marker card. • When you hold the card up to the webcam, the FLARToolKit is able to detect the orientation of the marker and superimpose, in this case, a 3d model on top of it.
    7. 7. Jargon
    8. 8. FLARToolKit is the name of the Library we will use in order to implement AR in our project. This library was created by Saqoosha and is based on NyARToolkit 2.0.0. It is available as open source under the GNU General Public License.
    9. 9. camera_para.dat is a binary file that is required by the FLARToolkit in order to run. It represents setting and configuration values for web cams.
    10. 10. Marker or Pattern is the name of the graphic FLAR will analyze and calculate its orientation from. This graphic is a special box with a black border and a graphic inside. You need this in order to get AR working.
    11. 11. My Background with FLAR
    12. 12. • Was inspired by the AR GE Flash Site. • Began writing a library for quickly creating FLAR projects and wanted a way to Debug/Test them. • Was asked by O'Reilly's InsideRIA.com to write a introduction for using FLARToolKit.
    13. 13. NASA + FLAR
    14. 14. • While at my last job at @radical.media, they selected by the Southwest Research Institute to launch a website for a future NASA mission to Jupiter called Juno. • My job was to evaluate the technology, understand its limitations/advantages to create a strategy for implementing it into the JUNO site. • I created 2 AR demos for NASA to review as a proof of concept of the technology’s potential.
    15. 15. NASA Demo I
    16. 16. AR Solar System
    17. 17. Why talk about workflow and Augmented Reality?
    18. 18. Building AR apps have 3 workflow roadblocks
    19. 19. I Compiling, launching, and accessing the camera is time consuming
    20. 20. II Creating and printing out new markers for testing kills trees and is “expensive”
    21. 21. III How can we prototype new ideas without writing a lot of new code?
    22. 22. We need to make a tool
    23. 23. FLARVision Emulator Flash Augmented Reality + PaperVision 3D
    24. 24. Architectural Approach
    25. 25. Incapsulate Repetitive Tasks • Setting up FLARToolKit • Setting up Papervision • Creating a video source • Triggering debug mode • Reusable code base
    26. 26. Setting Up The FLARToolKit
    27. 27. Steps Involved • Load the camera file • Load the marker file • Set a canvas width, height • Define marker width (scale) • Create a source (bitmap used to detect the marker) • Create a Transform Matrix • Set the threshold & confidence when detect the marker
    28. 28. Thats a lot of ... .. to set up!
    29. 29. ARDetector Class /** * Creates the AR Detector class and have it load in the camera.data * and pattern.pat files. * */ protected function createFlarDetector():void { arDetector = new ARDetector(); arDetector.addEventListener(Event.COMPLETE, onActivate); arDetector.setup('data/camera_para.dat', 'data/flarlogo.pat'); } /** * */ protected function onActivate(event:Event):void { init(); }
    30. 30. Encapsulation the process of compartmentalizing the elements of an abstraction that constitute its structure and behavior; encapsulation serves to separate the contractual interface of an abstraction and its implementation.
    31. 31. ARDetector Interface public interface IARDetector { function get flarParam():FLARParam; function set src(target:BitmapData):void; function setup(cameraURL:String, markerURL:String):void; function calculateTransformMatrix( resultMat:FLARTransMatResult):void; function detectMarker( threshold:int = 90, confidence:Number = .5):Boolean; }
    32. 32. Setting up Papervision • Everyone has their own way of doing it. • I like to keep my options open. • Create a simple method in the Main class anyone can override with their own settings. • Default setup should be exactly what you need to get up and running quickly.
    33. 33. Setup Method public function setupPapervision():void { scene = new Scene3D( ); camera = new FLARCamera3D( arDetector.flarParam ); // Create the Viewport viewport = new Viewport3D( stage.stageWidth, stage.stageHeight, true ); addChild( viewport ); // The base node is where all PV3D object should be attached to. baseNode = new FLARBaseNode( ); scene.addChild( baseNode ); create3dObjects( ); renderer = new BasicRenderEngine( ); addChild( new StatsView( renderer ) ); }
    34. 34. create3dObjects Method /** * This default function is where 3d Objects should be added to PV3D's * scenes. */ protected function create3dObjects():void { var plane:Plane = new Plane( new WireframeMaterial( 0xff0000 ), 80, 80 ); plane.rotationX = 180; baseNode.addChild( plane ); }
    35. 35. Have it your way! • If you want to customize your pv3d setup simply override the createPapervision method. • If you want to add your own custom 3d object simply override the creat3dObject method.
    36. 36. Video Source(s) • FLARToolKit relies on analyzing a bitmap to detect if the marker is present. • In a normal setup we would draw the WebCam “video” feed into a bitmap and pass the bitmapdata over to the ARDetector. • Why use just a Video feed?
    37. 37. MarkerEmulator • First I setup a second papervision instance with a single plane. • Next I used a JPG of the marker for the plane's texture. • Then I have the plane follow the mouse's movement. • Finally I pass the PV3d BitmapData into our ARDetector.
    38. 38. Marker Emulator as a Source
    39. 39. Debug Mode • So now we have 2 feeds one from the web cam and the other from our second papervision instance • We can switch between the two at any time by pressing the spacebar. • There is only one bitmapdata instance the ARDetector watches so we simply switch between the two sources when we sample the bitmapdata
    40. 40. ARDetecor Source /** * Creates a Bitmap for us to scan for valid markers. * */ protected function createCaptureSource():void { capturedSrc = new Bitmap( new BitmapData( arDetector.width, arDetector.height, false, 0 ), PixelSnapping.AUTO, true ); arDetector.src = capturedSrc.bitmapData; addChild( capturedSrc ); }
    41. 41. Switching between feeds /** * Updates the capturedSrc with new bitmap data. */ protected function updateCaptureBitmap():void { if(debug || ! video) { cardEmulator.render( ); capturedSrc.bitmapData.draw( cardEmulator.viewport ); } else { capturedSrc.bitmapData.draw( video ); } }
    42. 42. Lets see a demo
    43. 43. Now what?
    44. 44. Use Case • So I have this idea for a AR project • I want to show a cardboard iPod when I detect the marker • I don’t want to spend a lot of time configuring the project. • How do I write the least amount of code to test out my 3d model?
    45. 45. CPod FLAREmulator Demo
    46. 46. CPod Demo Code package { import com.flashartofwar.pv3d.componets.CPodContainer; public class FLAREmulatorDemo extends FLAREmulator { public function FLAREmulatorDemo() { super( ); } override protected function create3dObjects():void { var cpod:CPodContainer = new CPodContainer(); cpod.attachTo(baseNode) ; } } }
    47. 47. And that is just one of the many workflow tools I build...
    48. 48. Automating Builds
    49. 49. Why Use ANT? • Ant is cross platform • Ant works across multiple IDE’s • Ant can be versioned • Ant helps automat complex/repetitive tasks.
    50. 50. Back To AR In Flash
    51. 51. Things To Keep In Mind • Detecting the marker with FLAR is CPU intensive. • True 3d in Flash is not native so we rely on software-based 3d engines like PV3d & Away 3d. • Webcam video playback at high resolutions is intensive. There is no hardware acceleration for live video.
    52. 52. The Future of FLAR
    53. 53. • There is a branch of FLARToolKit that uses Alchemy to speed up the detection process. • We need better 3d support from Flash Player. • More developers and open source projects. • Continued interested from companies and exposure to AR outside of the Flash Platform.
    54. 54. Virtual Physical Computing - VPC -
    55. 55. • In physical computing you use an external IO device to control what happens on the screen. • In Flash we are limited to a mouse and keyboard. • FLAR opens up a new way to manipulate objects on the screen, especially 3d objects.
    56. 56. USPS Virtual Box Simulator by AKQA
    57. 57. FLAR Innovators
    58. 58. Mikko Haapoja http://www.mikkoh.com/blog/
    59. 59. Eric Socolofsky - FLARManager http://transmote.com
    60. 60. James Alliban http://jamesalliban.wordpress.com/
    61. 61. Seb Lee-Delisle http://sebleedelisle.com/
    62. 62. FLAR Resources
    63. 63. http://www.libspark.org/wiki/saqoosha/FLARToolKit/en
    64. 64. http://flartoolkitdocs.org/
    65. 65. http://flaremulator.flashartofwar.com
    66. 66. The Fine Print
    67. 67. This Is Just A Workflow Tool • This is not production ready code • This was not meant to ever be used by someone who is not a developer • I cut a lot of corners to make this fit my own needs, you should too • The extensibility of this FLAREmulator may have been grossly exaggerated • Use at your own risk!
    68. 68. Thank You For Watching My Site - http://flashbum.com My Blog - http://flashartofwar.com Source Code - http://github.com/theflashbum/FLAREmulator