study Latent Doodle Space

  • 936 views
Uploaded on

study

study

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
936
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • 1. William Baxter, (OLM Digital 的主任研究員 ), 主要研究 , NPR, HCI( 人機互動 ?), and physically based models. 2. Ken- ichi Anjyo , 安生健一 (OLM Digital 的技術總監 ) OLM Digital Inc., 日本電腦繪圖公司 . 開發和 visual effect 和 in-house software
  • “ 手描き線画の学習モデル “ 左邊有五幅塗鴉 ( 注意筆數相同 ). 右下方為解析 latent space 後 , 微調 latent space 的結果 . Latent space, 潛在空間 , 隱性變數 , 潛在變數 .. 能模仿”手描塗鴉 , 字跡等的”等線條為主的學習系統 到這裡 , 大概可以猜出是 PCA 或其他多變量分析的技巧解析出 latent space. 那以 PCA, 分析的是什麼 ? 輸出的是什麼 ? 某些以 texton 模擬畫風的 (Leung and Malik’s Method), 拿 48 個 filter 的結果 , 假設畫作上的像素是這 48 變量 (f1,f2…f48) 的線性組合 . 以此分析出另外鑑別力高的主成分 , 作為 texton. 但在這裡 , 針對線條相同為主 ( 畫風相似 ) 的畫作 , 找出線上對應點群 , 作為變量 . 再拿這些變量作 PCA.. 比較直覺 . 而如何自動地找對應 stroke, 找對應點等 . 成為不同畫間 , stroke assignment 的問題 . 所以 , 主要流程會是 , 手繪數張”筆劃數相同” ,” 圖形相仿”的塗鴉 . Assignment 的演算法自動比對出 stroke to stroke, point to point. 再以 PCA, 和 GPLVM 分析出 latent space. 生成新圖 , 基本上是屬於 deform 的領域 . 但ㄧ般處理 deform, 會注意變化區和非變化區的局部性 , 否則會造成 deform 程度不如預期 比如 , 抿嘴的右下角圖 , 是整體鋸齒狀的 deform. 但以 deform 的角度觀之 , 頭髮不應呈鋸齒狀變化 ( 可能本文指針對 point 分析 , 並沒有針對 方向 ,gradient 等分析 ) 並非學習了某種畫風後 , 可將 source 模擬成該畫風的 NPR (Leung and Malik’s Method)
  • Stamp tool, 橡皮章工具 . 各大影像編輯軟體皆有 . 有些 stamp tool, 除了印章圖像外 , 還有可調整的參數 本文主張 , latent space 可以生成”相似”但不”相同”於輸入範例的圖 . 如 < 由 PCA 找出來的前兩各主成分 , 隨機稍微改變 , 就可以生成本圖
  • 本文 ,, 1. Stroke correspondence, 本文提供的新的有效率的演算法 . 2. 找出 latent doodle space. 以操控 latent variable 產生 drawing. ( 但被歸類在 deform, 卻沒有像 deform 會區分 rigid 區域 ) 這部分 , 提出找 latent doodle space 的觀念 , 也實作系統 . 算是將 latent space 應用在 也實驗了 PCA, thin spline plate RBF, GPLVM 等方法
  • [Sig75] = [BW75], “Computer animation of free form images” [Sig92] = [SG92], “A physically based approach to 2-D shape blending” [Ieee02]=[BMP02], “Shape matching and object recognition using shape contexts” [UIST01]=[Kg01], a method to express the topology of the space of valid drawings. 需要請 artist 去 training, 以建構 而本文主要是 find correspondence and construct a latent space. 這篇 paper 蠻重要的 .
  • [Nerosci91]=[TP91],”Eigenface for recognition”. 醫學影像裡 , 使用 PCA, 分析出可供辨識的主要成分 [IEEE98]= [RCB98],”Verbs and adverbs: Multidimensional motion interpolation”. 使用 RBF 作為 supervised regression technique. 該文中 , 使用者需對””標示 verb, adverb. [Wol98],””,survey of the PCA, RBF… [Law04],””, Gaussian Process Latent Variable Model (GPLVM), 由 Gaussian process likelihood model 解析出來的 latent variable, 自動找出最佳位置 . 這是一個機率模型 , 非線性的 PCA [Computer Vision02]=[GMHP04], “Style-based inverse kinematics”, 風格化的運動學
  • 以筆劃順序先訂 cluter. Cost function
  • In other words, cluster I is considered connected to cluster j if the average distance between the pairs of strokes assigned to I and j is below the connectivity threshold.
  • Schneider: 1990 年發表的 , “an algorithm for automatically fitting digitized curve”
  • RBF: basic function is thin-plate-spline ()fast GPLVM: GPLVM 和 GP 都是 probabilistic regression ( 迴歸 ) techniques. 這意味著 , 他們假設輸入帶有些 unknown noise. 所以 , 不是完全由 input 內插得 . GP 和 GPLVM 都用 non-linear optimization 去決定 model 的最佳 hyperparameter. 包刮這些 noise level. 這些 noise level 能
  • . 由 user 評價 latent space 的結果好壞 , 以此調整更正確的 drawing style. 就像 Kovar 的 paper . 目前只能接受相同粗細的 . 能接受 free-form hand-drawn sketch, 可以畫動畫初稿 , 就像 deform 的技術製作出動畫 .

Transcript

  • 1. Latent Doodle Space William Baxter 1 , Ken-ichi Anjyo 2 OLM Digital, Inc. EUROGRAPHICS,2006 Presented by C.M. Hsu OLM Digital, Inc. EUROGRAPHICS,2006
  • 2. Abstract
  • 3. Abstract
    • Major tech.
      • A heuristic algorithm to match strokes between the inputs.
      • Extract a low dimensional latent doodle space from the inputs.
  • 4. Applications The Randomized Stamp Tool Input Output
  • 5. Applications Handwriting synthesis Output Input
  • 6. Outline
    • Overview
    • Related Work
    • Stroke Matching Algorithm
    • Building the Latent Doodle Space
    • Conclusions and Future Work
    • Demo
  • 7. Overview
  • 8. Overview 3. Reverse the parameterization of strokes to improve the point-to-point correspondence 4. Build the latent control space
    • 1. Similar to Computer-assisted in-between algorithms
    • 2. N-way correspondences, not pair
    • 3. Competitive-learning algorithm
      • K-means-like
      • Match stokes based on Kuhn-Munkres method
      • O(N 3 )
    1. Assignments of stroke correspondence 2. Resample corresponding strokes with the same number of sample points Synthesis PCA thin plate spline RBF PCA GP Feature vector GPLVM
  • 9. Related Work
  • 10. Related Work – Stroke Correspondences
    • The order of strokes between two images are identical, [Burtnyk N., SIG75]
    • Closed shape only, [Sederberg, SIG92]
  • 11. Related Work – A low-dimensional latent space
    • Eigenfaces by PCA , [Turk, Nerosci91]
    • Multidimensional motion interpolation by Radial basis functions ( RBF ) [Rose et al.,IEEE98]
    • The Gaussian Process Latent Variable Model ( GPLVM ), [Lawrence, NIPS04]
    • Create keyframe from ex. by GPLVM , [Grochow K., CV04]
  • 12. Related Work
    • Create many drawings from a few ex., [Kovar, UIST01]
  • 13. Stroke Matching Algorithm
    • Finding Stroke Correspondences
    • The Assignment Cost Matrix
    • Stroke Re-sampling and Alignment
  • 14. Finding Stroke Correspondences
    • K-means like
      • Initialize stroke-to-cluster assignment
        • Clustering by the drawn order of strokes simply
      • Update the cost matrix
      • Reassign stroke based on new clusters
        • Linear assignment problem (strokes  clusters)
          • Constrain: one stroke per drawing to each cluster
          • Kuhn-Munkres algorithm, (N 3 ), N as number of strokes
      • If reassignments made, goto 2
  • 15. The Assignment Cost Matrix
    • E=e d +e c +e t
      • Translation error, e d
        • The mean of the stroke differs from the mean of the cluster
      • Orientation and eccentricity error,e c
        • The covariance axe of the stroke differs from those of the cluster
      • Topological matching cost, e t
        • The connectivity of the stroke differs from the connectivity of the members of the cluster
  • 16. Translation error
    • e d : The mean of a stroke differs form the mean of the cluster
  • 17. Orientation and eccentricity error
    • e c : The covariance axe of the stroke differing from those of the cluster
    Stroke 1 Stroke 2
  • 18. Topological matching cost
    • e t : Connectivity cost of the stroke differing from
    • the connectivity of the members of the cluster
    • C s j : the number of strokes  the ith stroke in the same drawing.
    • C c j : the number of clusters  the ith cluster by average distance.
    S2 S1 S3 S2 is connected to S1, c s 2 =1 C2 is connected to C1, c c 2 =1 C2 C1
  • 19. Stroke Re-sampling and Alignment
    • Same number of points on the corresponding strokes
      • for RBF, Gaussian process regression
    • Reverse backward strokes
      • The total distance error between two strokes is lower when the point ordering is reversed
  • 20. Building the Latent Doodle Space PCA 2 principal components thin plate spline RBF PCA 2 principal components GP Feature Vector m = { p1,p2,..pn} of Stroke m
    • RBF
    • Gaussian
    • Thin plate spline, r 4 logr
    best GPLVM input output
  • 21. Conclusions and Future Work
    • Using machine learning
      • Find good assignment weights in cost function
      • Ex: S.T. like a support-vector classifier could be trainned to assign strokes to clusters.
      • Allow user to appraise the products form as latent space. [Kovar, UIST01]
    • Accept scanned drawing
    • Accept completely free-form hand-drawn sketch without the line constrain of uniform width.
  • 22. End