Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

群衆の知を引き出すための機械学習(第4回ステアラボ人工知能セミナー)

1,300 views

Published on

講演者: 馬場雪乃先生(京都大学)

Published in: Technology

群衆の知を引き出すための機械学習(第4回ステアラボ人工知能セミナー)

  1. 1. 3/41 ● ● ● - -
  2. 2. 4/41 ● - mammal placental carnivore canine dog working dog husky vehicle craft watercraft sailing vessel sailboat trimaran Figure 1: A snapshot of two root-to-leaf branches of ImageNet: the top row is from the mammal subtree; the bottom row is from the vehicle subtree. For each synset, 9 randomly sampled images are presented. Summary of selected subtrees Imagenet Cat SubtreeESP Cat Subtree
  3. 3. 5/41 ●
  4. 4. 6/41 Ranjay Krishna et al. 8 Ranjay Kris Fig. 6: From all of the region descriptions, we extract all objects mentioned. For example, from the region de- scription “man jumping over a fire hydrant,” we extract man and fire hydrant. Fig. 8: Our dataset also captures the relations interactions between objects in our images. In ample, we show the relationship jumping o tween the objects man and fire hydrant. over on the object fire hydrant. Each Ranjay Krishna et al.
  5. 5. 7/41 ● - ● (e.g., how much to reward workers, whether to reject their work, or impose a reputation penalty) their power is attenuated due to factors such as lack of direct supervision and visibility into their work behavior, lack of nuanced and individualized rewards, and the difficulty of imposing stringent and lasting sanctions (since workers can leave Th w va m be pa to pr an st m A w ea B te cr sp ne fo th un fo pr cu ex th ef cl in [6 Figure 2: Proposed framework for future crowd work processes to support complex and interdependent work.
  6. 6. 8/41 ● o o ● ●
  7. 7. 9/41 ● - ● - - Microtasks and Crowdsourcing #chi4good, CHI 2016, San Jose, CA, USA
  8. 8. 11/41 ● ● ● FALSE TRUE TRUE TRUE? FALSE?
  9. 9. 12/41 ● TRUE TRUE TRUE FALSE TRUE TRUE TRUE FALSE TRUE ? ? ?
  10. 10. 13/41 ● 𝑗 ● TRUE FALSE TRUE FALSE ↵j 1 ↵j 0 ↵j 1 ↵j 0
  11. 11. 14/41 ● TRUE TRUE TRUE FALSE TRUE TRUE TRUE FALSE TRUE ? ? ?
  12. 12. 15/41 ● - - ● wj ( , + ) xi [0, + ) Pr [yij = ti] = 1 1 + exp ( wjxi)
  13. 13. 16/41 ● ● Pr [yij = 1] = w> j xi ⌧j j ! xi wj
  14. 14. 17/41 ● - - ● (d) ARJ (b) SRJ (c) CF Accurate (28%) Mosty accurate (12%) Outer class biased (29%) Middle class biased (29%) uth Predicted Truth Predicted Truth Predicted Truth Predicted Truth PredictedTruth PredictedTruth Predicteduth Predicted Truth PredictedTruth PredictedTruth Predicted 0 0 0 11 0.5 0 11 0.5 0 11 0 0.5 -1 01 -1 0 1 -1 01 1 0 -1-1 01 1 0 -1 1 0 -1 1 0 -1 -1 01 -1 01 -1 0 1 0 -1-1 01 0 -10 -1 1 0 -1 11 1 0 1 234 0 12 3 34 0 1 234 012 3 34 0 1 234 012 3 34 0.8 0.4 0 0.8 0.4 0 0.8 0.4 0 0.8 Middle class biased (34%)Accurate (28%) Mosty accurate (12%) Outer class biased (29%) Decisive (5%)Conservative (4%)Calibrated (91%)
  15. 15. 18/41 ● ● TRUE FALSE TRUE FALSE : j 10 j 10 j 11 j 01 j 00
  16. 16. 19/41 ● ●
  17. 17. 20/41 ● ● - 860 43 … 43 215 215 860 860 860 860 215 215 215 215
  18. 18. 21/41 “A silver tabby cat is howling with his mouth wide open’’ “A sleeping cat’’ “Dreaming of becoming a lion’’ 4.7 1.2 2.6
  19. 19. 22/41 ● ● ● 2.6 “Dreaming of becoming a lion’’
  20. 20. 23/41 ● - - ● - - µa a r r
  21. 21. 25/41 ● ● - -
  22. 22. 26/41 ● - ● -
  23. 23. 27/41 ● ● - - ● - -
  24. 24. 28/41 ● - - healthline Figure 3: Example ad to attract users time a user clicks on the a we record a conversion ev the advertising system. T the system to optimize th mizing the number of con contribution yield, instea the number of clicks. Although optimizing fo Figure 1: Screenshot of the Quizz system.
  25. 25. 29/41 ● ● - -
  26. 26. 30/41
  27. 27. 31/41 qmw qz qua qb qM QqN TI K e UKu u E D P 2 V qt qJ '2 V'I ' E qmyM K Figure 1: Fine Grained Truth Discovery Model for Crowd- sourced Data Aggregation. • For the q-th question (q = 1, 2, · · · , Q) – Draw a topic zq ∼ Multi(θ) to question q. Le topical words in qu ground words, θk φkw = p(w|k) be topic k. Then the times in question q ity of background independent, and t der topic k and wo p(wq|k, y where V is the num V w=1(nw q,y=1 + n dent, and the proba p(w|θ, φ, φ′ ) = 𝑞 𝑞 𝑚 𝑞 𝑢 𝑘 𝑞 𝑢 𝜎 𝑥 = 1/ 1 + exp −𝑥 (⇢kueku bq) 𝑘 𝑢 𝑘 𝑢 𝑞
  28. 28. 32/41 ● - ● ● Q1. “The accident triggered the establishment of the Law for the …” ☐ ☑☑ ☑
  29. 29. ● 𝑥0, 𝑦0 ● 𝑥0, 𝑦03, 𝑦04, ⋯ ● - - 34/41 Pr [yi = 1 | xi] = 1 1 + exp ( w>xi) Pr [yij | yi = 1] = ⇣ ↵j 1 ⌘yij ⇣ 1 ↵j 1 ⌘(1 yij )
  30. 30. 35/41 ● ● ● Visual Recognition with Humans in the Loop 3 er vision is helpful Computer vision is not helpfuler vision is helpful Computer vision is not helpful The bird is a  Black‐footed  Albatross Is the belly  white? yes Are the eyes  white? yes Th bi d i Is the beak cone‐shaped? yes Is the upper‐tail brown? yes Is the breast solid colored? no Is the breast striped? yes I h h hi ?The bird is a  Parakeet Auklet Is the throat white? yes The bird is a Henslow’s Sparrow amples of the visual 20 questions game on the 200 class Bird dataset. ponses (shown in red) to questions posed by the computer (shown in blue) drive up recognition accuracy. In the left image, computer vision algorithms he bird species correctly without any user interaction. In the middle image, vision reduces the number of questions to 2. In the right image, computer ides little help. YES YES NO
  31. 31. 36/41 ● - Crowd-Machine Learning Classifiers Justin Cheng and Michael S. Bernstein Stanford University {jcccf, msb}@cs.stanford.edu ing classifiers: clas- en description of a predictive features tures using machine rate and use human- lassifiers enable fast that can improve on judgment, and ac- extraction is not yet arning platform, in- nformative features, d examples, an ap- g. The crowd’s ef- s of the input space Figure 1. Flock is a hybrid crowd-machine learning platform that capi- talizes on analogical encoding to guide crowds to nominate effective fea-
  32. 32. 37/41 ● - ● - - …
  33. 33. 38/41 0.4 0.6 0.8 1.0 Aug 19 Aug 26 Sep 02 Sep 09 Sep 16 提出日 AUC 参加者1 参加者2 参加者3 参加者4 参加者5 参加者6 参加者7 参加者8 参加者9 参加者10 参加者11 参加者12 参加者13 参加者14 参加者15 参加者16 NLDR LM 統合 AUC=0.982 AUC=0.946 AUC=0.717
  34. 34. 39/41 ● - ● ●
  35. 35. 40/41 Figure 3. An example analysis microtask shows a single chart (a) along with chart-reading subtasks (b) an annotation subtask (c) and a feature-oriented explanation prompt designed to encourage workers Figure 3. An example analysis microtask shows a single chart (a) along with chart-reading subtasks (b) an annotation subtask (c) and a feature-oriented explanation prompt designed to encourage workers
  36. 36. 41/41 ● - ● - ● - - -

×