JSARToolKit / LiveChromaKey / LivePointers - Next gen of AR

7,553 views

Published on

I gave a talk about the next generation of AR. Pure ActionScript 3.0 libraries of LiveChromaKey and LivePointers made it at the SparkStudy/09 (Spark project勉強会#09)
Try this at: http://www.kawa.net/text/spark/09/spark.html

Published in: Technology, Design
1 Comment
4 Likes
Statistics
Notes
No Downloads
Views
Total views
7,553
On SlideShare
0
From Embeds
0
Number of Embeds
3,446
Actions
Shares
0
Downloads
23
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

JSARToolKit / LiveChromaKey / LivePointers - Next gen of AR

  1. 1. Spark project 勉強会 #09 JSARToolKit / LiveChromaKey 2009.05.27 Yusuke Kawasaki 川﨑 有亮 (kawanet) http://www.kawa.net/ http://www.kawa.net/ 1
  2. 2. Yusuke Kawasaki(川﨑 有亮) • Perl monger – XML::TreePP, XML::FeedPP, ... • JavaScript addict – Wiimote over HTTP, Gainer, ... • ActionScript 3.0 newbie – Today! • Media Technology Labs – Recruit Co., Ltd. in Japan http://www.kawa.net/ http://twitter.com/kawa0117 http://www.kawa.net/ 2
  3. 3. §1 - JSARToolKit JavaScript Augmented Reality http://www.kawa.net/ 3
  4. 4. JSARToolKit • JSAR(ジャバスクリプト拡張現実) – JavaScript Augmented Reality • JSARToolKit – My first work using ActionScript 3.0 – Powered by SWFObject! – A library to run AR by JavaScript – A proxy wrapper for a Flash app using FLARToolKit – Debuted at OSDC.TW 2009 in Taipei • Genealogy – 2008.03 – NyARToolKit for Java - by A⻁@nyatla.jp – 2008.05 – FLARToolKit for ActionScript3 by Saqoosha – 2009.04 – JSARToolKit for JavaScript by Kawanet • Saqoosha さん、本当にありがとうございます!!_o_ http://www.kawa.net/ 4
  5. 5. JSARToolKit Synopsis It needs just several lines of JavaScript code to run AR. When a marker found, it calls onDetect callback function. < s c r i p t t y p e = quot; t e x t / j a v a s c r i p t quot; s r c = quot; s w f o b j e c t . j s quot; > < / s c r i p t > < s c r i p t t y p e = quot; t e x t / j a v a s c r i p t quot; s r c = quot; j s a r . j s quot; > < / s c r i p t > < s c r i p t t y p e = quot; t e x t / j a v a s c r i p t quot; > < ! - - v a r p a t s = [ ' j s a r l o g o . p a t ' ] ; v a r j s a r = n e w J S A R ( ' j s a r _ h e r e ' ) ; j s a r . d r a w M a r k e r R e c t = t r u e ; j s a r . o n D e t e c t e d = f u n c t i o n ( r e s u l t ) { . . . } ; j s a r . i n i t ( ) ; j s a r . s e t M a r k e r ( p a t s ) ; - - > < / s c r i p t > < d i v i d = quot; j s a r _ h e r e quot; > < / d i v > http://www.kawa.net/ 5
  6. 6. JSAR returns a JavaScript Object { quot; d e t e c t e d quot; : t r u e , quot; m a r k e r s quot; : [ { quot; c o d e i d quot; : 0 , quot; d i r e c t i o n quot; : 2 , quot; c o n f i d e n c e quot; : 0 . 5 1 2 6 3 7 , quot; v e r t e x quot; : [ { quot; x quot; : 3 4 6 , quot; y quot; : 2 0 1 } , { quot; x quot; : 2 7 7 , quot; y quot; : 1 9 1 } , { quot; x quot; : 3 5 3 , quot; y quot; : 1 3 0 } , { quot; x quot; : 3 9 5 , quot; y quot; : 1 5 4 } ] } , ] , } http://www.kawa.net/ 6
  7. 7. JSARToolKit Layers Flash Layer jsar.swf HTML Canvas Layer HTML Div Layer http://www.kawa.net/ 7
  8. 8. AiR Xiaolongbao(エア⼩籠包) 2009.04.19 @ Taipei (OSDC.TW) http://www.kawa.net/ 8
  9. 9. §2 - LiveChromaKey Bluescreen-less augmented IN reality http://www.kawa.net/ 9
  10. 10. LiveChromaKey • A new image synthesizing engine for AR – 2009.05 released by Kawanet (me!) • Pure ActionScripe 3.0 – No library dependencies – No JavaScript • Only webcam needed – No blue back screen – No need to print AR marker PDF http://www.kawa.net/ 10
  11. 11. Step 1/3 - Stationary background At first, connect you webcam and run LiveChromaKey. Never move until it recognizes stationary background. http://www.kawa.net/ 11
  12. 12. Step 2/3 – Bluescreen on the fly It generates bluescreen automatically. Live Video – Background = Bluescreen - ⇒ http://www.kawa.net/ 12
  13. 13. Step 3/3 - Transparent foreground Get the foreground image as a translarent Sprite. Live Video – Bluescreen = Foreground - ⇒ http://www.kawa.net/ 13
  14. 14. LiveChromaKey Synopsis LiveChromaKey provides four kinds of Sprites. v a r c h r o m a k e y : L C K _ C o r e = n e w L C K _ C o r e ( ) ; c h r o m a k e y . i n i t ( ) ; v a r s p L i v e : S p r i t e = c h r o m a k e y . g e t L i v e ( ) ; v a r s p B a c k : S p r i t e = c h r o m a k e y . g e t B a c k g r o u n d ( ) ; v a r s p M a s k : S p r i t e = c h r o m a k e y . g e t M a s k ( ) ; v a r s p F o r e : S p r i t e = c h r o m a k e y . g e t F o r e g r o u n d ( ) ; t h i s . a d d C h i l d ( s p L i v e ) ; t h i s . a d d C h i l d ( s p B a c k ) ; t h i s . a d d C h i l d ( s p M a s k ) ; t h i s . a d d C h i l d ( s p F o r e ) ; http://www.kawa.net/ 14
  15. 15. Example: Travelling in Egypt egypt.swf http://www.kawa.net/ 15
  16. 16. Sprites augmented IN reality The key feature of LiveChromaKey is to insert Sprites augmented between background and foreground images. Sprite http://www.kawa.net/ 16
  17. 17. Example: Slideshow in reality minority.swf http://www.kawa.net/ 17
  18. 18. Multiarmed deity (or 美川健⼀) exile.swf http://www.kawa.net/ 18
  19. 19. ひとり おそ松くん exile.swf http://www.kawa.net/ 19
  20. 20. ひとり Exile exile.swf http://www.kawa.net/ 20
  21. 21. §3 - LivePointers Live Color Pointer Detection Library http://www.kawa.net/ 21
  22. 22. LivePointers • Live Color Pointer Detection Library – will be released soon! • Pure ActionScripe 3.0 – No library dependencies – No JavaScript • Webcam as a 3D human interface – No need to print AR marker PDF http://www.kawa.net/ 22
  23. 23. Pointer = Pixel Cluster • Ex. – Cluster #0 – 250 pixels – RGB: 25D985 – H: 152 – S: 0.83 – V: 0.85 http://www.kawa.net/ 23
  24. 24. Labeled Pointer Detection • Labeled pointer needs a guide color. • ex. fingercap – 0x2197A9 (uint) – ¥105(キャン・ドゥ) • Fingercap is definitely important user interface device of the future. http://www.kawa.net/ 24
  25. 25. LivePointers(⽇本語のみスマソ) • 最短距離法 (Nearest Neighbor Clustering) をベース にして、ActionScript 3.0 でリアルタイムに表⽰でき る程度に単純化したクラスタ分析処理を⾏う。 • 画⾯内の単⾊のカラーマーカーを検出します。 – ⽩・⿊・グレー・淡⾊は検出しません。 – (S・V値が⾼い)くっきりした⾊が得意です。 – 模様は検出しません。(⾊のみを認識する) – 通常は、画⾯内の全ての検出⽤ガイドとしてRGBで ⾊指定した「ラベル付マーカー」を利⽤する。 • 3次元⼊⼒のポインティングデバイスとして利⽤可能。 http://www.kawa.net/ 25
  26. 26. Now on the Spark! JSARToolKit http://www.libspark.org/svn/js/JSARToolKit/ LiveChromaKey http://www.libspark.org/svn/as3/LiveChromaKey/ LivePointers ... coming soon! http://www.kawa.net/ 26
  27. 27. Thank you! Yusuke Kawasaki 川﨑 有亮 (kawanet) http://www.kawa.net/ http://www.kawa.net/ 27

×