Spark project 勉強会 #09
JSARToolKit / LiveChromaKey
2009.05.27
Yusuke Kawasaki
川﨑 有亮 (kawanet)
http://www.kawa.net/
http://www.kawa.net/ 1

Yusuke Kawasaki（川﨑 有亮）
• Perl monger
– XML::TreePP, XML::FeedPP, ...
• JavaScript addict
– Wiimote over HTTP, Gainer, ...
• ActionScript 3.0 newbie
– Today!
• Media Technology Labs
– Recruit Co., Ltd. in Japan
http://www.kawa.net/
http://twitter.com/kawa0117
http://www.kawa.net/ 2

§1 - JSARToolKit
JavaScript Augmented Reality
http://www.kawa.net/ 3

JSARToolKit
• JSAR（ジャバスクリプト拡張現実）
– JavaScript Augmented Reality
• JSARToolKit
– My first work using ActionScript 3.0
– Powered by SWFObject!
– A library to run AR by JavaScript
– A proxy wrapper for a Flash app using FLARToolKit
– Debuted at OSDC.TW 2009 in Taipei
• Genealogy
– 2008.03 – NyARToolKit for Java - by A⻁@nyatla.jp
– 2008.05 – FLARToolKit for ActionScript3 by Saqoosha
– 2009.04 – JSARToolKit for JavaScript by Kawanet
• Saqoosha さん、本当にありがとうございます！！_o_
http://www.kawa.net/ 4

JSARToolKit Synopsis
It needs just several lines of JavaScript code to run AR.
When a marker found, it calls onDetect callback function.
< s c r i p t t y p e = \" t e x t / j a v a s c r i p t \" s r c = \" s w f o b j e c t . j s \" > < / s c r i p t >
< s c r i p t t y p e = \" t e x t / j a v a s c r i p t \" s r c = \" j s a r . j s \" > < / s c r i p t >
< s c r i p t t y p e = \" t e x t / j a v a s c r i p t \" > < ! - -
v a r p a t s = [ ' j s a r l o g o . p a t ' ] ;
v a r j s a r = n e w J S A R ( ' j s a r _ h e r e ' ) ;
j s a r . d r a w M a r k e r R e c t = t r u e ;
j s a r . o n D e t e c t e d = f u n c t i o n ( r e s u l t ) { . . . } ;
j s a r . i n i t ( ) ;
j s a r . s e t M a r k e r ( p a t s ) ;
- - > < / s c r i p t >
< d i v i d = \" j s a r _ h e r e \" > < / d i v >
http://www.kawa.net/ 5

JSAR returns a JavaScript Object
{
\" d e t e c t e d \" : t r u e ,
\" m a r k e r s \" : [
{
\" c o d e i d \" : 0 ,
\" d i r e c t i o n \" : 2 ,
\" c o n f i d e n c e \" : 0 . 5 1 2 6 3 7 ,
\" v e r t e x \" : [
{ \" x \" : 3 4 6 , \" y \" : 2 0 1 } ,
{ \" x \" : 2 7 7 , \" y \" : 1 9 1 } ,
{ \" x \" : 3 5 3 , \" y \" : 1 3 0 } ,
{ \" x \" : 3 9 5 , \" y \" : 1 5 4 }
]
} ,
] ,
}
http://www.kawa.net/ 6

JSARToolKit Layers
Flash Layer jsar.swf
HTML Canvas Layer
HTML Div Layer
http://www.kawa.net/ 7

AiR Xiaolongbao（エア⼩籠包）
2009.04.19 @ Taipei (OSDC.TW)
http://www.kawa.net/ 8

§2 - LiveChromaKey
Bluescreen-less augmented IN reality
http://www.kawa.net/ 9

LiveChromaKey
• A new image synthesizing engine for AR
– 2009.05 released by Kawanet (me!)
• Pure ActionScripe 3.0
– No library dependencies
– No JavaScript
• Only webcam needed
– No blue back screen
– No need to print
AR marker PDF
http://www.kawa.net/ 10

Step 1/3 - Stationary background
At first, connect you webcam and run LiveChromaKey.
Never move until it recognizes stationary background.
http://www.kawa.net/ 11

Step 2/3 – Bluescreen on the fly
It generates bluescreen automatically.
Live Video – Background = Bluescreen
－ ⇒
http://www.kawa.net/ 12

Step 3/3 - Transparent foreground
Get the foreground image as a translarent Sprite.
Live Video – Bluescreen = Foreground
－ ⇒
http://www.kawa.net/ 13

LiveChromaKey Synopsis
LiveChromaKey provides four kinds of Sprites.
v a r c h r o m a k e y : L C K _ C o r e = n e w L C K _ C o r e ( ) ;
c h r o m a k e y . i n i t ( ) ;
v a r s p L i v e : S p r i t e = c h r o m a k e y . g e t L i v e ( ) ;
v a r s p B a c k : S p r i t e = c h r o m a k e y . g e t B a c k g r o u n d ( ) ;
v a r s p M a s k : S p r i t e = c h r o m a k e y . g e t M a s k ( ) ;
v a r s p F o r e : S p r i t e = c h r o m a k e y . g e t F o r e g r o u n d ( ) ;
t h i s . a d d C h i l d ( s p L i v e ) ;
t h i s . a d d C h i l d ( s p B a c k ) ;
t h i s . a d d C h i l d ( s p M a s k ) ;
t h i s . a d d C h i l d ( s p F o r e ) ;
http://www.kawa.net/ 14

Example: Travelling in Egypt
egypt.swf
http://www.kawa.net/ 15

Sprites augmented IN reality
The key feature of LiveChromaKey is to insert
Sprites augmented between background and
foreground images.
Sprite
http://www.kawa.net/ 16

Example: Slideshow in reality
minority.swf
http://www.kawa.net/ 17

Multiarmed deity (or 美川健⼀)
exile.swf
http://www.kawa.net/ 18

ひとり おそ松くん
exile.swf
http://www.kawa.net/ 19

ひとり Exile
exile.swf
http://www.kawa.net/ 20

§3 - LivePointers
Live Color Pointer Detection Library
http://www.kawa.net/ 21

LivePointers
• Live Color Pointer Detection Library
– will be released soon!
• Pure ActionScripe 3.0
– No library dependencies
– No JavaScript
• Webcam as a 3D human interface
– No need to print AR marker PDF
http://www.kawa.net/ 22

Pointer = Pixel Cluster
• Ex.
– Cluster #0
– 250 pixels
– RGB: 25D985
– H: 152
– S: 0.83
– V: 0.85
http://www.kawa.net/ 23

Labeled Pointer Detection
• Labeled pointer needs a guide color.
• ex. fingercap
– 0x2197A9 (uint)
– ￥105（キャン・ドゥ）
• Fingercap is definitely
important user interface
device of the future.
http://www.kawa.net/ 24

LivePointers（⽇本語のみスマソ）
• 最短距離法 (Nearest Neighbor Clustering) をベース
にして、ActionScript 3.0 でリアルタイムに表⽰でき
る程度に単純化したクラスタ分析処理を⾏う。
• 画⾯内の単⾊のカラーマーカーを検出します。
– ⽩・⿊・グレー・淡⾊は検出しません。
– （S・V値が⾼い）くっきりした⾊が得意です。
– 模様は検出しません。（⾊のみを認識する）
– 通常は、画⾯内の全ての検出⽤ガイドとしてRGBで
⾊指定した「ラベル付マーカー」を利⽤する。
• 3次元⼊⼒のポインティングデバイスとして利⽤可能。
http://www.kawa.net/ 25

Now on the Spark!
JSARToolKit
http://www.libspark.org/svn/js/JSARToolKit/
LiveChromaKey
http://www.libspark.org/svn/as3/LiveChromaKey/
LivePointers
... coming soon!
http://www.kawa.net/ 26

Thank you!
Yusuke Kawasaki
川﨑 有亮 (kawanet)
http://www.kawa.net/
http://www.kawa.net/ 27