Paco Nathan, profile picture
Uploaded byPaco Nathan
PDF, PPTX1,582 views

Humans in the loop: AI in open source and industry

The document discusses the integration of AI in various educational media and its potential to personalize learning experiences and facilitate knowledge sharing among industry experts. It highlights recent advancements in AI technology, such as deep learning and machine learning, and addresses the challenges of adapting to rapid technological changes. Furthermore, it emphasizes the need for human involvement in AI applications and the importance of innovative methods for continuous organizational learning.

Embed presentation

Download as PDF, PPTX
Humans  in  the  loop   AI  in  open  source  and  industry Paco  Nathan  @pacoid   Dir,  Learning  Group  @  O’Reilly  Media   #NikeTechTalks    Portland  2017-­‐08-­‐10
2
Research  questions: ▪ How  do  we  personalize  learning  experiences,  across  
 ebooks,  videos,  conferences,  computable  content,  live   online  courses,  case  studies,  expert  AMAs,  etc.   ▪ How  do  we  help  experts  —  by  definition,  really  busy   people  —  share  knowledge  with  their  peers  in  industry?   ▪ How  do  we  manage  the  role  of  editors  at  human  scale,  
 while  technology  and  delivery  media  evolve  rapidly?   ▪ How  do  we  help  organizations  learn  and  transform   continuously?   ▪ Can  we  accomplish  these  goals  by  leveraging  AI  in  Media? 3
4
5 UX  for  content  discovery:   • partly  generated  +  curated  by  humans   • partly  generated  +  curated  by  AI  apps
AI:  why  now? 6
AI  is  real,  but  why  now? ▪ Big  Data:  machine  data  (1997-­‐ish)   ▪ Big  Compute:  cloud  computing  (2006-­‐ish)   ▪ Big  Models:  deep  learning  (2009-­‐ish)   The  confluence  of  factors  created  a  business  
 environment  where  AI  could  become  mainstream   AR/VR  combined  with  embedded  computing  and   reinforcement  learning  may  bring  it  to  a  next  level 7
Benchmark:  achieving  human  parity 2016-­‐10-­‐12:  Microsoft  researchers  reach  human  
 parity  in  conversational  speech  recognition   Achieving  Human  Parity  in  Conversational  Speech   Recognition
 W.  Xiong,  et  al.    Microsoft   8
Big  picture ▪ The  current  state  of  machine  intelligence  3.0
 Shivon  Zilis,  James  Cham    Bloomberg  Beta  (annual  landscape)   ▪ The  Future  of  Machine  Intelligence
 David  Beyer    Amplify  Partners  (report)   ▪ Artificial  Intelligence:  Teaching  Machines  to  Think  Like  People
 Jack  Clark    Open  AI  (report)   ▪ The  AI  Conf
 O’Reilly  Media  and  Intel  partnership  (industry  conference) 9
10
“Consider  the  shift  from  steam  to  electric  power:
    it  took  a  generation  before  factory  managers
    understood  they  could  reconfigure  the  physical
    arrangement.      AI  may  be  quicker  adoption,  but  faces  similar
    extremes  of  cognitive  embrace.”              –  David  Beyer    Amplify  Partners 11
Immediate  impact  of  AI 12 personal  op-­‐ed:  the  combination  of  advances  with   UX,  DevOps,  AI  together  –  specifically  –  is  taking  off   the  table  some  previous  needs  for  what  we’d  called   “software  engineering”  –  which  must  now  undergo   major  changes
The  leaderboard 13
2017  highlights  from  leading  teams ▪ TensorFlow:  Machine  learning  for  everyone
 Rajat  Monga    Google   ▪ Distributed  deep  learning  on  AWS  using  MXNet
 Anima  Anandkumar    Amazon   ▪ Squeezing  deep  learning  onto  mobile  phones
 Anirudh  Koul    Microsoft 14
15 Artificial  intelligence  in  the  software  engineering  workflow
 Peter  Norvig    Google  
16 Can  machines  spot  diseases  faster  than  expert  humans?
 Suchi  Saria    Johns  Hopkins  U
17 Cars  that  coordinate  with  people
 Anca  Dragan    UC  Berkeley
18 Strategies  for  integrating  people  and  machine  learning  in   online  systems
 Jason  Laska    Clara  Labs  
19 AI  for  manufacturing:  Today  and  tomorrow
 David  Rogers    Sight  Machine
20 Harnessing  the  power  of  artificial  intelligence  to  diagnose   diseases
 Kavya  Kopparapu    GirlsComputingLeague
Now  trending 21
22 Current  themes  among  leading  AI  teams:   ▪ scale  up  to  solve  complex  problems  (big  models)     ▪ optimize  to  deploy  consumer  products  (low  power) Trending  strategy…
23 Most  popular  content,  among  thousands  
 of  enterprise  organizations:   Hands-­‐On  Machine  Learning  with  scikit-­‐learn   and  TensorFlow
 Aurélien  Géron   Python  FTW.   Along  with  Keras,  PyTorch,  Caffe,  etc. Trending  methods…
UC  Berkeley  RISELab 24 ▪ https://rise.cs.berkeley.edu/     ▪ enable  machines  to  take  rapid,  intelligent   actions  based  on  real-­‐time  data  and  context   from  the  world  around  them   ▪ shift  away  from  prior  emphasis  on  JVM-­‐based   frameworks  during  AMPLab  period  (Spark)   ▪ major  focus  on  reinforcement  learning   Ray:  a  distributed  execution  framework   for  emerging  AI  applications
Increasing  role  of  the  hardware  interface 25 ▪ earlier  generations  of  virtualization  abstracted  away  
 hardware;  however,  containers  allow  direct  access   ▪ with  DL,  application  software  must  access  the  latest  
 hardware  features  directly  –  to  be  competitive   ▪ vendors  anticipate  adv.  math  needs  for  low-­‐level  hardware,   looking  beyond  DL  –  e.g.,  multi-­‐linear  algebra  libraries     ▪ Scaling  machine  learning  (O’Reilly  Data  Show,  21:43)
 Reza  Zadeh    Stanford  /  Matroid
Emerging  themes:  transfer  learning ▪ transfer  learning:  when  you  can  solve  a  task  well,  
 transfer  understanding  to  solve  related  problems   ▪ remove  final  classification  layer,  then  extract  
 next-­‐to-­‐last  layer  of  a  CNN:
 tensorflow.org/tutorials/image_recognition   ▪ leverage  a  network  pre-­‐trained  on  a  large  dataset:
 blog.keras.io/building-­‐powerful-­‐image-­‐classification-­‐ models-­‐using-­‐very-­‐little-­‐data.html 26
Emerging  themes:  GANs ▪ generative  adversarial  networks:  neural  networks  
 compete  against  each  other  in  a  zero-­‐sum  game   ▪ example:  CycleGAN    (see  AI  NY  2017)   27
“Generative  Adversarial  Networks  for  Beginners” 28
LSTM  used  to  generate  content 29 Long  short-­‐term  memory  (LSTM)  allows  recurrent   neural  networks  to  learn  sequences  of  data,  such  
 as  in  streams  of  voice  or  text.   Imagine  feeding  scripts  (semi-­‐structured  data)  from  
 a  film  genre  through  an  LSTM,  then  generating  new   output…
LSTM  used  to  generate  content 30 http://benjamin.wtf/ Sunspring It’s  No  Game
LSTM  in  music  composition  /  performance 31 https://github.com/IraKorshunova/folk-­‐rnn
Even  romance  novels… 32
How  do  people  learn? 33
34 White  paper:  "How  do  you  learn?"
Peer  Teaching  through  a  range  of  Media ▪ books,  videos   ▪ live  online  courses   ▪ conferences   ▪ AMAs   ▪ computable  content   ▪ case  studies   ▪ articles   ▪ podcast  interviews   ▪ chat  forums 35
Example:  "Learn  alongside  innovators,  thought-­‐by-­‐thought"
Example:  "How  great  companies  make  change  happen"
Example:  "Why  self  assessments  improve  learning"
Key  insight  for  AI  in  Media: ▪ any  content  which  can  represented  
 as  text  can  be  parsed  by  NLP,  then   manipulated  by  available  AI  tooling     ▪ labeled  images  get  really  interesting   ▪ text  or  images  within  a  context  have  
 inherent  structure   ▪ representation  of  that  kind  of  structure   is  rare  in  the  Media  vertical  –  so  far 39
Beyond  deep  learning… 40
Ontology ▪ provides  context  which  Deep  Learning  lacks   ▪ aka,  “knowledge  graph”  –  a  computable  thesaurus   ▪ maps  the  semantics  of  business  relationships   ▪ S/V/O:  “nouns”,  some  “verbs”,  a  few  “adjectives”   ▪ conversational  interfaces  (e.g.,  Google  Assistant)   improve  UX  by  importing  ontologies   ▪ the  hard  part,  a  relatively  expensive  investment 41
Which  parts  do  people  or  machines  do  best? 42 team  goal:  maintain  structural  correspondence  between  the  layers   big  win  for  AI:  inferences  across  the  graph human  scale   primary  structure   control  points   testability machine  generated  data  products   ~80%  of  the  graph
Ontology 43
Open  source  tooling 44
Components 45 ▪ rdflib  +  NetworkX:  ontology  graph  represented  as  N3  “turtle”   ▪ PyTextRank:  NLP  parsing,  feature  vectors,  summarization   ▪ Jupyter  +  nbtransom:  human-­‐in-­‐the-­‐loop  ML  pipelines   ▪ Apache  Spark:  sort,  partitioning,  task  management   ▪ scikit-­‐learn:  machine  learning  models   ▪ gensim:  vector  embedding  /  deep  learning   ▪ datasketch:  approximation  algorithms   ▪ Flask,  React,  Node.js:  microservices,  UI  web  components   ▪ Redis:  in-­‐memory  indexing,  full-­‐text  search

PyTextRank 46 TextRank  (R  Mihalcea,  P  Tarau,  2004)  a  graph  algorithm   that  extracts  key  phrases  and  summarizes  texts  –  for  NLP   which  is  improved  over  use  of  keywords,  n-­‐grams,  etc.   ▪ construct  a  graph  from  a  paragraph  of  text   ▪ run  PageRank  on  that  graph   ▪ extract  the  highly  ranked  phrases   Python  implementation  atop  spaCy,  NetworkX,  datasketch:   ▪ https://pypi.python.org/pypi/pytextrank/
PyTextRank 47
Working  with  text  and  NLP 48 ▪ parsing   ▪ named  entity  recognition   ▪ vector  embedding   ▪ smarter  indexing   ▪ summarization  (especially  video)   ▪ semantic  similarity  to  suggest  curriculum   ▪ speed  development  of  assessments   ▪ query  expansion   ▪ amending  ontology
A  plug  for  InnerSource… 49 We  thought  the  introduction  of  data  science  had  run   headlong  into  enterprise  silos  and  lingering  tech  debt.  
 As  if!!     Introduction  of  AI  exacerbates  that  problem  even   more.  Suggested  responses:   ▪ InnerSourceCommons.org  open  source  practices  
 within  enterprise   ▪ design  patterns  for  working  across  silos   ▪ think:  “good  house  rules  for  guests”  as  other  
 teams  submit  PRs  on  your  code  repos
Beyond  text… 50
A  generational  shift? ▪ We’re  12  years  beyond  the  introduction   of  YouTube  …  anyone  raising  tweens  
 now  probably  knows  about  YouTubers   ▪ Below  a  certain  age  demographic,  people   tend  to  rely  more  on  video  and  audio   sources  for  information,  while  perhaps   print  is  gaining  more  for  entertainment.   Mobile  certainly  has  huge  impact  there. 51
{"graf": [[21, "let", "let", "VB", 1, 48], [0, "'s", "'s", "PRP", 0, 49], "take", "take", "VB", 1, 50], [0, "a", "a", "DT", 0, 51], [23, "look", "l "NN", 1, 52], [0, "at", "at", "IN", 0, 53], [0, "a", "a", "DT", 0, 54], [ "few", "few", "JJ", 1, 55], [25, "examples", "example", "NNS", 1, 56], [0 "often", "often", "RB", 0, 57], [0, "when", "when", "WRB", 0, 58], [11, "people", "people", "NNS", 1, 59], [2, "are", "be", "VBP", 1, 60], [26, " "first", "JJ", 1, 61], [27, "learning", "learn", "VBG", 1, 62], [0, "abou "about", "IN", 0, 63], [28, "Docker", "docker", "NNP", 1, 64], [0, "they" "they", "PRP", 0, 65], [29, "try", "try", "VBP", 1, 66], [0, "and", "and" 0, 67], [30, "put", "put", "VBP", 1, 68], [0, "it", "it", "PRP", 0, 69], "in", "in", "IN", 0, 70], [0, "one", "one", "CD", 0, 71], [0, "of", "of", 0, 72], [0, "a", "a", "DT", 0, 73], [24, "few", "few", "JJ", 1, 74], [31, "existing", "existing", "JJ", 1, 75], [18, "categories", "category", "NNS 76], [0, "sometimes", "sometimes", "RB", 0, 77], [11, "people", "people", 1, 78], [9, "think", "think", "VBP", 1, 79], [0, "it", "it", "PRP", 0, 80 "'s", "be", "VBZ", 1, 81], [0, "a", "a", "DT", 0, 82], [32, "virtualizati "virtualization", "NN", 1, 83], [19, "tool", "tool", "NN", 1, 84], [0, "l "like", "IN", 0, 85], [33, "VMware", "vmware", "NNP", 1, 86], [0, "or", " "CC", 0, 87], [34, "virtualbox", "virtualbox", "NNP", 1, 88], [0, "also", "also", "RB", 0, 89], [35, "known", "know", "VBN", 1, 90], [0, "as", "as" 0, 91], [0, "a", "a", "DT", 0, 92], [36, "hypervisor", "hypervisor", "NN" 93], [0, "these", "these", "DT", 0, 94], [2, "are", "be", "VBP", 1, 95], "tools", "tool", "NNS", 1, 96], [0, "which", "which", "WDT", 0, 97], [2, "be", "VBP", 1, 98], [37, "emulating", "emulate", "VBG", 1, 99], [38, "hardware", "hardware", "NN", 1, 100], [0, "for", "for", "IN", 0, 101], [ "virtual", "virtual", "JJ", 1, 102], [40, "software", "software", "NN", 1 103]], "id": "001.video197359", "sha1": "4b69cf60f0497887e3776619b922514f2e5b70a8"} Video  transcription 52 {"count": 2, "ids": [32, 19], "pos": "np", "rank": 0.0194, "text": "virtualization tool"} {"count": 2, "ids": [40, 69], "pos": "np", "rank": 0.0117, "text": "software applications"} {"count": 4, "ids": [38], "pos": "np", "rank": 0.0114, "text": "hardware"} {"count": 2, "ids": [33, 36], "pos": "np", "rank": 0.0099, "text": "vmware hypervisor"} {"count": 4, "ids": [28], "pos": "np", "rank": 0.0096, "text": "docker"} {"count": 4, "ids": [34], "pos": "np", "rank": 0.0094, "text": "virtualbox"} {"count": 10, "ids": [11], "pos": "np", "rank": 0.0049, "text": "people"} {"count": 4, "ids": [37], "pos": "vbg", "rank": 0.0026, "text": "emulating"} {"count": 2, "ids": [27], "pos": "vbg", "rank": 0.0016, "text": "learning"} Transcript: let's take a look at a few examples often when people are first learning about Docker they try and put it in one of a few existing categories sometimes people think it's a virtualization tool like VMware or virtualbox also known as a hypervisor these are tools which are emulating hardware for virtual software Confidence: 0.973419129848 39 KUBERNETES 0.8747 coreos 0.8624 etcd 0.8478 DOCKER CONTAINERS 0.8458 mesos 0.8406 DOCKER 0.8354 DOCKER CONTAINER 0.8260 KUBERNETES CLUSTER 0.8258 docker image 0.8252 EC2 0.8210 docker hub 0.8138 OPENSTACK orm:Docker a orm:Vendor; a orm:Container; a orm:Open_Source; a orm:Commercial_Software; owl:sameAs dbr:Docker_%28software%29; skos:prefLabel "Docker"@en;
Fave  mediated  learning  experience:  audio+ 53
How  will  a  next  generation  learn?
Humans  in  the  loop 55
Active  learning ▪ special  case  of  semi-­‐supervised  machine  learning   ▪ send  difficult  calls  /  edge  cases  to  experts;  
 let  algorithms  handle  routine  decisions   ▪ works  well  in  use  cases  which  have  lots  of  
 inexpensive,  unlabeled  data   ▪ e.g.,  abundance  of  content  to  be  classified,  
 where  the  cost  of  labeling  is  the  expense   ▪ https://en.wikipedia.org/wiki/ Active_learning_(machine_learning) 56
Active  learning Data  preparation  in  the  age  of  deep  learning
 oreilly.com/ideas/data-­‐preparation-­‐in-­‐the-­‐ age-­‐of-­‐deep-­‐learning
 Luke  Biewald    CrowdFlower
 O’Reilly  Data  Show,  2017-­‐05-­‐04   send  human  workers  cases  where  machine  learning   algorithms  signal  uncertainty  (low  probability  scores)     or  when  your  ensemble  of  machine  learning   algorithms  signals  disagreement 58
Human-­‐in-­‐the-­‐loop  design  pattern   Building  a  business  that  combines  human  experts   and  data  science
 oreilly.com/ideas/building-­‐a-­‐business-­‐that-­‐ combines-­‐human-­‐experts-­‐and-­‐data-­‐science-­‐2
 Eric  Colson    StitchFix
 O’Reilly  Data  Show,  2016-­‐01-­‐28   “what  machines  can’t  do  are  things  around  cognition,
    things  that  have  to  do  with  ambient  information,  or
    appreciation  of  aesthetics,  or  even  the  ability  to
    relate  to  another  human”
 
 59
Weak  supervision Creating  large  training  data  sets  quickly
 oreilly.com/ideas/creating-­‐large-­‐ training-­‐data-­‐sets-­‐quickly
 Alex  Ratner    Stanford
 O’Reilly  Data  Show,  2017-­‐06-­‐08   Snorkel:  “data  programming”  as  another  
 instance  of  human-­‐in-­‐the-­‐loop
 github.com/HazyResearch/snorkel   conferences.oreilly.com/strata/strata-­‐ny/ public/schedule/detail/61849 60
Collaboration  through  Jupyter 61 Notebooks  get  used  to  manage  ML  pipelines,   where  machines  +  people  collaborate  on  docs   ▪ “Human-­‐in-­‐the-­‐loop  design  pattern”
 talk  @  JupyterCon  NY  2017   ▪ experts  adjust  parameters  in  ML  pipelines   ▪ machines  write  structured  “logs”  of  ML   modeling  and  evaluation   ▪ experts  run  `jupyter  notebook`  via  SSH  tunnel  
 for  remote  monitoring  and  updates   ▪ https://pypi.python.org/pypi/nbtransom
Collaboration  through  Jupyter 62 ML#Pipelines Jupyter#kernel Browser SSH#tunnel
Collaboration  through  Jupyter ▪ running  notebooks  via  SSH   tunnel  removes  the  need  for   dedicated  UIs   ▪ this  work  anticipates  upcoming   collaborative  document  features   in  JupyterLab:   Realtime  collaboration  for   JupyterLab  using  Google  Drive
 Ian  Rose    UC  Berkeley
Expert  review ▪ ML  pipelines  report  results:  recognizing  content,  adding  annotations,   requesting  more  examples  when  “confused”   ▪ Human-­‐in-­‐the-­‐loop  experts  –  potentially,  Customer  Service  –  
 review  decisions,  especially  edge  cases,  then  train  through  examples   ▪ The  system  iterates 64
What’s  the  point  of  using  AI  in  Media? ▪ more  work,  quicker,  than  could  be  performed  
 by  editors  –  who  are  already  super-­‐busy  people   ▪ exceeding  human  parity,  as  a  benchmark   ▪ helps  relieve  pressure  on  organizations,  
 as  learning  curves  accelerate   ▪ augments  some  of  our  most  valuable
 experts,  so  they  can  get  more  done 65
Human-­‐in-­‐the-­‐loop  as  a  management  strategy 66 personal  op-­‐ed:  the  “game”  isn’t  to  replace  people  –   instead  it’s  about  leveraging  AI  to  augment  staff,  
 so  organizations  can  retain  people  with  valuable   domain  expertise,  making  their  contributions  and   expertise  even  more  vital
Why  we’ll  never  run  out  of  jobs 67
Strata  Data   NY,  Sep  25-­‐28
 SG,  Dec  4-­‐7
 SJ,  Mar  5-­‐8,  2018
 UK,  May  21-­‐24,  2018   The  AI  Conf   SF,  Sep  17-­‐20
 NY,  Apr  29-­‐May  2,  2018   JupyterCon   NY,  Aug  22-­‐25   OSCON  (returns!)   PDX,  Jul  16-­‐19,  2018 68
69 contact  the  speaker  for  
 conf  discount  coupons
70 Learn  Alongside
 Innovators Just  Enough  Math Building  Data   Science  Teams Hylbert-­‐Speys How  Do  You  Learn? updates,  reviews,  conference  summaries…   liber118.com/pxn/
 @pacoid
Humans in the loop: AI in open source and industry

More Related Content

PDF
Humans in a loop: Jupyter notebooks as a front-end for AI
byPaco Nathan
 
PDF
Human-in-the-loop: a design pattern for managing teams that leverage ML
byPaco Nathan
 
PDF
Human in the loop: a design pattern for managing teams working with ML
byPaco Nathan
 
PDF
Human-in-a-loop: a design pattern for managing teams which leverage ML
byPaco Nathan
 
PDF
Data By The People, For The People
byDaniel Tunkelang
 
PDF
Data Scientist 101 BI Dutch
byJos van Dongen
 
KEY
Intro to Data Science for Enterprise Big Data
byPaco Nathan
 
PPTX
Data Science, Machine Learning, and H2O
bySri Ambati
 
Humans in a loop: Jupyter notebooks as a front-end for AI
byPaco Nathan
 
Human-in-the-loop: a design pattern for managing teams that leverage ML
byPaco Nathan
 
Human in the loop: a design pattern for managing teams working with ML
byPaco Nathan
 
Human-in-a-loop: a design pattern for managing teams which leverage ML
byPaco Nathan
 
Data By The People, For The People
byDaniel Tunkelang
 
Data Scientist 101 BI Dutch
byJos van Dongen
 
Intro to Data Science for Enterprise Big Data
byPaco Nathan
 
Data Science, Machine Learning, and H2O
bySri Ambati
 

What's hot

PDF
Deep Learning for Recommender Systems
byMarcel Kurovski
 
PDF
Web Scale Named Entity Mining
byFrancois Pouilloux
 
PPTX
Three Tools for "Human-in-the-loop" Data Science
byAditya Parameswaran
 
PDF
Cloudera Breakfast: Advanced Analytics Part II: Do More With Your Data
byCloudera, Inc.
 
PDF
Data Scientist Enablement roadmap 1.0
byDr. Mohan K. Bavirisetty
 
PPTX
AI on Big Data
byJongwook Woo
 
PPTX
Big Data and Data Intensive Computing: Use Cases
byJongwook Woo
 
PPTX
Big Data and Data Intensive Computing on Networks
byJongwook Woo
 
PPTX
Crowdsourced Data Processing: Industry and Academic Perspectives
byAditya Parameswaran
 
PDF
Sql saturday el salvador 2016 - Me, A Data Scientist?
byFabricio Quintanilla
 
PDF
Big Data and Predictive Analysis
byJongwook Woo
 
PDF
How to Interview a Data Scientist
byDaniel Tunkelang
 
PDF
Introduction to Machine Learning - WeCloudData
byWeCloudData
 
PDF
Collections Databases; Making the system work for you
byirowson
 
PDF
Introduction to Deep Learning and AI at Scale for Managers
byDataWorks Summit
 
PDF
Democratizing Advanced Analytics Propels Instant Analysis Results to the Ubiq...
byDana Gardner
 
PDF
Full-Stack Data Science: How to be a One-person Data Team
byGreg Goltsov
 
PDF
Python's Role in the Future of Data Analysis
byPeter Wang
 
PDF
Intro to Machine Learning with H2O and AWS
bySri Ambati
 
PDF
MIT Deep Learning Basics: Introduction and Overview by Lex Fridman
byPeerasak C.
 
Deep Learning for Recommender Systems
byMarcel Kurovski
 
Web Scale Named Entity Mining
byFrancois Pouilloux
 
Three Tools for "Human-in-the-loop" Data Science
byAditya Parameswaran
 
Cloudera Breakfast: Advanced Analytics Part II: Do More With Your Data
byCloudera, Inc.
 
Data Scientist Enablement roadmap 1.0
byDr. Mohan K. Bavirisetty
 
AI on Big Data
byJongwook Woo
 
Big Data and Data Intensive Computing: Use Cases
byJongwook Woo
 
Big Data and Data Intensive Computing on Networks
byJongwook Woo
 
Crowdsourced Data Processing: Industry and Academic Perspectives
byAditya Parameswaran
 
Sql saturday el salvador 2016 - Me, A Data Scientist?
byFabricio Quintanilla
 
Big Data and Predictive Analysis
byJongwook Woo
 
How to Interview a Data Scientist
byDaniel Tunkelang
 
Introduction to Machine Learning - WeCloudData
byWeCloudData
 
Collections Databases; Making the system work for you
byirowson
 
Introduction to Deep Learning and AI at Scale for Managers
byDataWorks Summit
 
Democratizing Advanced Analytics Propels Instant Analysis Results to the Ubiq...
byDana Gardner
 
Full-Stack Data Science: How to be a One-person Data Team
byGreg Goltsov
 
Python's Role in the Future of Data Analysis
byPeter Wang
 
Intro to Machine Learning with H2O and AWS
bySri Ambati
 
MIT Deep Learning Basics: Introduction and Overview by Lex Fridman
byPeerasak C.
 

Similar to Humans in the loop: AI in open source and industry

PPTX
4MCA- UNIT1-AI-and Machine learning.pptx
byendrisyimer02
 
PPTX
AI_presentation_AICT[1] [Read-Only].pptx
byiu0501250137
 
PPTX
AI_presentation_AICT[1] [Read-Only].pptx
byiu0501250137
 
PPTX
AI_presentation_AICT[1] [Read-Only].pptx
byiu0501250137
 
PDF
Artificial Intelligence in Business Management
byDharun Krishna K B
 
PPTX
Artificial Intelligence in Emerging Technology
bySitanathBiswas
 
PPTX
TrendsOfAI.pptx
byHarishYadav34879
 
PPTX
PresentationonArtificialintelligence.pptx
byNikhilPatil812222
 
PDF
Artificial Intelligence.pdf nhbhbuhuuhjuj
byarpitdhagate08
 
PPTX
Artificial Intelligence (AI) kunal yadav.pptx
byKunalYadav65140
 
PDF
Introduction to AI Evolution Handcrafted .pdf
bySaad Ali
 
PPTX
ojguifhgjfilhjsbgbkjfbvlbslkjjblkjfjsbbfgs
bysamjohnson7350
 
PPTX
ARTIFICIAL INTELLIGENCE PRESENTATION BY STUDENTS OF IIM
bymidtownbaker
 
PPTX
HICAS AI Unit 1 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bymohammedraashidn
 
PPTX
The Evolution and Impact of Artificial Intelligence
bymomnaqayyum01
 
PPTX
Exploring the Impact of AI Technology Today
byTop10Explorers
 
PPTX
AI Presentation for developers artificial intelligence
byyusufalzuobi
 
PDF
Empirical AI Research
byDeakin University
 
PPTX
CS302_Unit1-Till EnVIRONMENT_For_First_Sem
byaneeshdhavade1
 
PDF
Conviction LP Letter - Dec 2023 [Redacted]
byRazin Mustafiz
 
4MCA- UNIT1-AI-and Machine learning.pptx
byendrisyimer02
 
AI_presentation_AICT[1] [Read-Only].pptx
byiu0501250137
 
AI_presentation_AICT[1] [Read-Only].pptx
byiu0501250137
 
AI_presentation_AICT[1] [Read-Only].pptx
byiu0501250137
 
Artificial Intelligence in Business Management
byDharun Krishna K B
 
Artificial Intelligence in Emerging Technology
bySitanathBiswas
 
TrendsOfAI.pptx
byHarishYadav34879
 
PresentationonArtificialintelligence.pptx
byNikhilPatil812222
 
Artificial Intelligence.pdf nhbhbuhuuhjuj
byarpitdhagate08
 
Artificial Intelligence (AI) kunal yadav.pptx
byKunalYadav65140
 
Introduction to AI Evolution Handcrafted .pdf
bySaad Ali
 
ojguifhgjfilhjsbgbkjfbvlbslkjjblkjfjsbbfgs
bysamjohnson7350
 
ARTIFICIAL INTELLIGENCE PRESENTATION BY STUDENTS OF IIM
bymidtownbaker
 
HICAS AI Unit 1 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
bymohammedraashidn
 
The Evolution and Impact of Artificial Intelligence
bymomnaqayyum01
 
Exploring the Impact of AI Technology Today
byTop10Explorers
 
AI Presentation for developers artificial intelligence
byyusufalzuobi
 
Empirical AI Research
byDeakin University
 
CS302_Unit1-Till EnVIRONMENT_For_First_Sem
byaneeshdhavade1
 
Conviction LP Letter - Dec 2023 [Redacted]
byRazin Mustafiz
 

More from Paco Nathan

PDF
Computable Content
byPaco Nathan
 
PDF
Computable Content: Lessons Learned
byPaco Nathan
 
PDF
SF Python Meetup: TextRank in Python
byPaco Nathan
 
PDF
Use of standards and related issues in predictive analytics
byPaco Nathan
 
PDF
Data Science in 2016: Moving Up
byPaco Nathan
 
PDF
Data Science Reinvents Learning?
byPaco Nathan
 
PDF
Jupyter for Education: Beyond Gutenberg and Erasmus
byPaco Nathan
 
PDF
GalvanizeU Seattle: Eleven Almost-Truisms About Data
byPaco Nathan
 
PDF
Microservices, containers, and machine learning
byPaco Nathan
 
PDF
GraphX: Graph analytics for insights about developer communities
byPaco Nathan
 
PDF
Graph Analytics in Spark
byPaco Nathan
 
PDF
Apache Spark and the Emerging Technology Landscape for Big Data
byPaco Nathan
 
PDF
QCon São Paulo: Real-Time Analytics with Spark Streaming
byPaco Nathan
 
PDF
Strata 2015 Data Preview: Spark, Data Visualization, YARN, and More
byPaco Nathan
 
PDF
A New Year in Data Science: ML Unpaused
byPaco Nathan
 
PDF
Microservices, Containers, and Machine Learning
byPaco Nathan
 
PDF
Databricks Meetup @ Los Angeles Apache Spark User Group
byPaco Nathan
 
PDF
How Apache Spark fits into the Big Data landscape
byPaco Nathan
 
PDF
What's new with Apache Spark?
byPaco Nathan
 
PDF
How Apache Spark fits in the Big Data landscape
byPaco Nathan
 
Computable Content
byPaco Nathan
 
Computable Content: Lessons Learned
byPaco Nathan
 
SF Python Meetup: TextRank in Python
byPaco Nathan
 
Use of standards and related issues in predictive analytics
byPaco Nathan
 
Data Science in 2016: Moving Up
byPaco Nathan
 
Data Science Reinvents Learning?
byPaco Nathan
 
Jupyter for Education: Beyond Gutenberg and Erasmus
byPaco Nathan
 
GalvanizeU Seattle: Eleven Almost-Truisms About Data
byPaco Nathan
 
Microservices, containers, and machine learning
byPaco Nathan
 
GraphX: Graph analytics for insights about developer communities
byPaco Nathan
 
Graph Analytics in Spark
byPaco Nathan
 
Apache Spark and the Emerging Technology Landscape for Big Data
byPaco Nathan
 
QCon São Paulo: Real-Time Analytics with Spark Streaming
byPaco Nathan
 
Strata 2015 Data Preview: Spark, Data Visualization, YARN, and More
byPaco Nathan
 
A New Year in Data Science: ML Unpaused
byPaco Nathan
 
Microservices, Containers, and Machine Learning
byPaco Nathan
 
Databricks Meetup @ Los Angeles Apache Spark User Group
byPaco Nathan
 
How Apache Spark fits into the Big Data landscape
byPaco Nathan
 
What's new with Apache Spark?
byPaco Nathan
 
How Apache Spark fits in the Big Data landscape
byPaco Nathan
 

Recently uploaded

PDF
Rustici Software: eLearning standards in the age of AI
byRustici Software
 
PPTX
NTG - Data Center Management System Software
byMustafa Kuğu
 
PDF
Post-Hackathon-Learnings-Maximizing-Impact-Beyond-the-Event.pdf
byishantyadav1111
 
PDF
AI and Zero Trust: What it takes to do it right
byMark Simos
 
PPTX
TechSprint Inauguration at SJB Institute of Technology held on 18 December 2025
bysuhasspgdg
 
PDF
Traditional-Security-Models-No-Longer-Work.pptx (1).pdf
byOhhproJunction
 
PDF
CI CD Observability, Metrics and DORA - Shifting Left and Cleaning Up! - Febr...
byPeter Souter
 
PDF
What Is the Azure AI Foundry and Why Does It Matter for Enterprises?
byjohncarterjn
 
PDF
UiPath Automation Developer Associate Training Series 2026 - Session 2
byDianaGray10
 
PDF
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
PPTX
Career-Opportunities in Industrial Arts Grade 8 PPT Lesson 2
byFSBTLEDNathanVince
 
PPTX
The Lex Wire Precedent: A Technical Standard for Machine-Mediated Authority ...
byJeff Howell
 
PDF
Advanced SELinux Management - RHCSA (RH134).pdf
byLinuxCert Guru
 
PPTX
Authority After Search: How AI Systems Reconstruct Trust, Expertise, and Legi...
byJeff Howell
 
PPTX
Beyond the Algorithm: Designing Human-Centric Public Service with AI
byAlan Dix
 
PDF
Computer-Based Training (CBT) The Backbone of Modern Technical & Defence Trai...
bycomputerbasedtrainin1
 
PPTX
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
PDF
Parental Control App for Phones_ The Complete 2026 Guide for Safer, Smarter P...
byRyan Cooper
 
PDF
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
PPTX
Working Session — Build a Document Understanding Automation Using an OOTB ML ...
byDianaGray10
 
Rustici Software: eLearning standards in the age of AI
byRustici Software
 
NTG - Data Center Management System Software
byMustafa Kuğu
 
Post-Hackathon-Learnings-Maximizing-Impact-Beyond-the-Event.pdf
byishantyadav1111
 
AI and Zero Trust: What it takes to do it right
byMark Simos
 
TechSprint Inauguration at SJB Institute of Technology held on 18 December 2025
bysuhasspgdg
 
Traditional-Security-Models-No-Longer-Work.pptx (1).pdf
byOhhproJunction
 
CI CD Observability, Metrics and DORA - Shifting Left and Cleaning Up! - Febr...
byPeter Souter
 
What Is the Azure AI Foundry and Why Does It Matter for Enterprises?
byjohncarterjn
 
UiPath Automation Developer Associate Training Series 2026 - Session 2
byDianaGray10
 
Title Installing Windows, Linux, MacOS.pdf
byGeraldAbadajos
 
Career-Opportunities in Industrial Arts Grade 8 PPT Lesson 2
byFSBTLEDNathanVince
 
The Lex Wire Precedent: A Technical Standard for Machine-Mediated Authority ...
byJeff Howell
 
Advanced SELinux Management - RHCSA (RH134).pdf
byLinuxCert Guru
 
Authority After Search: How AI Systems Reconstruct Trust, Expertise, and Legi...
byJeff Howell
 
Beyond the Algorithm: Designing Human-Centric Public Service with AI
byAlan Dix
 
Computer-Based Training (CBT) The Backbone of Modern Technical & Defence Trai...
bycomputerbasedtrainin1
 
Introduction to Artificial Intelligence (AI).pptx
byammar414783
 
Parental Control App for Phones_ The Complete 2026 Guide for Safer, Smarter P...
byRyan Cooper
 
Dev Dives: Build and deploy agentic automations - the unified way
byUiPathCommunity
 
Working Session — Build a Document Understanding Automation Using an OOTB ML ...
byDianaGray10
 

Humans in the loop: AI in open source and industry

  • 1.
    Humans  in  the  loop   AI  in  open  source  and  industry Paco  Nathan  @pacoid   Dir,  Learning  Group  @  O’Reilly  Media   #NikeTechTalks    Portland  2017-­‐08-­‐10
  • 2.
  • 3.
    Research  questions: ▪ How  do  we  personalize  learning  experiences,  across  
 ebooks,  videos,  conferences,  computable  content,  live   online  courses,  case  studies,  expert  AMAs,  etc.   ▪ How  do  we  help  experts  —  by  definition,  really  busy   people  —  share  knowledge  with  their  peers  in  industry?   ▪ How  do  we  manage  the  role  of  editors  at  human  scale,  
 while  technology  and  delivery  media  evolve  rapidly?   ▪ How  do  we  help  organizations  learn  and  transform   continuously?   ▪ Can  we  accomplish  these  goals  by  leveraging  AI  in  Media? 3
  • 4.
  • 5.
    5 UX  for  content  discovery:   • partly  generated  +  curated  by  humans   • partly  generated  +  curated  by  AI  apps
  • 6.
  • 7.
    AI  is  real,  but  why  now? ▪ Big  Data:  machine  data  (1997-­‐ish)   ▪ Big  Compute:  cloud  computing  (2006-­‐ish)   ▪ Big  Models:  deep  learning  (2009-­‐ish)   The  confluence  of  factors  created  a  business  
 environment  where  AI  could  become  mainstream   AR/VR  combined  with  embedded  computing  and   reinforcement  learning  may  bring  it  to  a  next  level 7
  • 8.
    Benchmark:  achieving  human  parity 2016-­‐10-­‐12:  Microsoft  researchers  reach  human  
 parity  in  conversational  speech  recognition   Achieving  Human  Parity  in  Conversational  Speech   Recognition
 W.  Xiong,  et  al.    Microsoft   8
  • 9.
    Big  picture ▪ The  current  state  of  machine  intelligence  3.0
 Shivon  Zilis,  James  Cham    Bloomberg  Beta  (annual  landscape)   ▪ The  Future  of  Machine  Intelligence
 David  Beyer    Amplify  Partners  (report)   ▪ Artificial  Intelligence:  Teaching  Machines  to  Think  Like  People
 Jack  Clark    Open  AI  (report)   ▪ The  AI  Conf
 O’Reilly  Media  and  Intel  partnership  (industry  conference) 9
  • 10.
  • 11.
    “Consider  the  shift  from  steam  to  electric  power:
    it  took  a  generation  before  factory  managers
    understood  they  could  reconfigure  the  physical
    arrangement.      AI  may  be  quicker  adoption,  but  faces  similar
    extremes  of  cognitive  embrace.”              –  David  Beyer    Amplify  Partners 11
  • 12.
    Immediate  impact  of  AI 12 personal  op-­‐ed:  the  combination  of  advances  with   UX,  DevOps,  AI  together  –  specifically  –  is  taking  off   the  table  some  previous  needs  for  what  we’d  called   “software  engineering”  –  which  must  now  undergo   major  changes
  • 13.
  • 14.
    2017  highlights  from  leading  teams ▪ TensorFlow:  Machine  learning  for  everyone
 Rajat  Monga    Google   ▪ Distributed  deep  learning  on  AWS  using  MXNet
 Anima  Anandkumar    Amazon   ▪ Squeezing  deep  learning  onto  mobile  phones
 Anirudh  Koul    Microsoft 14
  • 15.
    15 Artificial  intelligence  in  the  software  engineering  workflow
 Peter  Norvig    Google  
  • 16.
    16 Can  machines  spot  diseases  faster  than  expert  humans?
 Suchi  Saria    Johns  Hopkins  U
  • 17.
    17 Cars  that  coordinate  with  people
 Anca  Dragan    UC  Berkeley
  • 18.
    18 Strategies  for  integrating  people  and  machine  learning  in   online  systems
 Jason  Laska    Clara  Labs  
  • 19.
    19 AI  for  manufacturing:  Today  and  tomorrow
 David  Rogers    Sight  Machine
  • 20.
    20 Harnessing  the  power  of  artificial  intelligence  to  diagnose   diseases
 Kavya  Kopparapu    GirlsComputingLeague
  • 21.
  • 22.
    22 Current  themes  among  leading  AI  teams:   ▪ scale  up  to  solve  complex  problems  (big  models)     ▪ optimize  to  deploy  consumer  products  (low  power) Trending  strategy…
  • 23.
    23 Most  popular  content,  among  thousands  
 of  enterprise  organizations:   Hands-­‐On  Machine  Learning  with  scikit-­‐learn   and  TensorFlow
 Aurélien  Géron   Python  FTW.   Along  with  Keras,  PyTorch,  Caffe,  etc. Trending  methods…
  • 24.
    UC  Berkeley  RISELab 24 ▪https://rise.cs.berkeley.edu/     ▪ enable  machines  to  take  rapid,  intelligent   actions  based  on  real-­‐time  data  and  context   from  the  world  around  them   ▪ shift  away  from  prior  emphasis  on  JVM-­‐based   frameworks  during  AMPLab  period  (Spark)   ▪ major  focus  on  reinforcement  learning   Ray:  a  distributed  execution  framework   for  emerging  AI  applications
  • 25.
    Increasing  role  of  the  hardware  interface 25 ▪ earlier  generations  of  virtualization  abstracted  away  
 hardware;  however,  containers  allow  direct  access   ▪ with  DL,  application  software  must  access  the  latest  
 hardware  features  directly  –  to  be  competitive   ▪ vendors  anticipate  adv.  math  needs  for  low-­‐level  hardware,   looking  beyond  DL  –  e.g.,  multi-­‐linear  algebra  libraries     ▪ Scaling  machine  learning  (O’Reilly  Data  Show,  21:43)
 Reza  Zadeh    Stanford  /  Matroid
  • 26.
    Emerging  themes:  transfer  learning ▪ transfer  learning:  when  you  can  solve  a  task  well,  
 transfer  understanding  to  solve  related  problems   ▪ remove  final  classification  layer,  then  extract  
 next-­‐to-­‐last  layer  of  a  CNN:
 tensorflow.org/tutorials/image_recognition   ▪ leverage  a  network  pre-­‐trained  on  a  large  dataset:
 blog.keras.io/building-­‐powerful-­‐image-­‐classification-­‐ models-­‐using-­‐very-­‐little-­‐data.html 26
  • 27.
    Emerging  themes:  GANs ▪generative  adversarial  networks:  neural  networks  
 compete  against  each  other  in  a  zero-­‐sum  game   ▪ example:  CycleGAN    (see  AI  NY  2017)   27
  • 28.
  • 29.
    LSTM  used  to  generate  content 29 Long  short-­‐term  memory  (LSTM)  allows  recurrent   neural  networks  to  learn  sequences  of  data,  such  
 as  in  streams  of  voice  or  text.   Imagine  feeding  scripts  (semi-­‐structured  data)  from  
 a  film  genre  through  an  LSTM,  then  generating  new   output…
  • 30.
    LSTM  used  to  generate  content 30 http://benjamin.wtf/ Sunspring It’s  No  Game
  • 31.
    LSTM  in  music  composition  /  performance 31 https://github.com/IraKorshunova/folk-­‐rnn
  • 32.
  • 33.
    How  do  people  learn? 33
  • 34.
    34 White  paper:  "How  do  you  learn?"
  • 35.
    Peer  Teaching  through  a  range  of  Media ▪ books,  videos   ▪ live  online  courses   ▪ conferences   ▪ AMAs   ▪ computable  content   ▪ case  studies   ▪ articles   ▪ podcast  interviews   ▪ chat  forums 35
  • 36.
    Example:  "Learn  alongside  innovators,  thought-­‐by-­‐thought"
  • 37.
    Example:  "How  great  companies  make  change  happen"
  • 38.
    Example:  "Why  self  assessments  improve  learning"
  • 39.
    Key  insight  for  AI  in  Media: ▪ any  content  which  can  represented  
 as  text  can  be  parsed  by  NLP,  then   manipulated  by  available  AI  tooling     ▪ labeled  images  get  really  interesting   ▪ text  or  images  within  a  context  have  
 inherent  structure   ▪ representation  of  that  kind  of  structure   is  rare  in  the  Media  vertical  –  so  far 39
  • 40.
  • 41.
    Ontology ▪ provides  context  which  Deep  Learning  lacks   ▪ aka,  “knowledge  graph”  –  a  computable  thesaurus   ▪ maps  the  semantics  of  business  relationships   ▪ S/V/O:  “nouns”,  some  “verbs”,  a  few  “adjectives”   ▪ conversational  interfaces  (e.g.,  Google  Assistant)   improve  UX  by  importing  ontologies   ▪ the  hard  part,  a  relatively  expensive  investment 41
  • 42.
    Which  parts  do  people  or  machines  do  best? 42 team  goal:  maintain  structural  correspondence  between  the  layers   big  win  for  AI:  inferences  across  the  graph human  scale   primary  structure   control  points   testability machine  generated  data  products   ~80%  of  the  graph
  • 43.
  • 44.
  • 45.
    Components 45 ▪ rdflib  +  NetworkX:  ontology  graph  represented  as  N3  “turtle”   ▪ PyTextRank:  NLP  parsing,  feature  vectors,  summarization   ▪ Jupyter  +  nbtransom:  human-­‐in-­‐the-­‐loop  ML  pipelines   ▪ Apache  Spark:  sort,  partitioning,  task  management   ▪ scikit-­‐learn:  machine  learning  models   ▪ gensim:  vector  embedding  /  deep  learning   ▪ datasketch:  approximation  algorithms   ▪ Flask,  React,  Node.js:  microservices,  UI  web  components   ▪ Redis:  in-­‐memory  indexing,  full-­‐text  search

  • 46.
    PyTextRank 46 TextRank  (R  Mihalcea,  P  Tarau,  2004)  a  graph  algorithm   that  extracts  key  phrases  and  summarizes  texts  –  for  NLP   which  is  improved  over  use  of  keywords,  n-­‐grams,  etc.   ▪ construct  a  graph  from  a  paragraph  of  text   ▪ run  PageRank  on  that  graph   ▪ extract  the  highly  ranked  phrases   Python  implementation  atop  spaCy,  NetworkX,  datasketch:   ▪ https://pypi.python.org/pypi/pytextrank/
  • 47.
  • 48.
    Working  with  text  and  NLP 48 ▪ parsing   ▪ named  entity  recognition   ▪ vector  embedding   ▪ smarter  indexing   ▪ summarization  (especially  video)   ▪ semantic  similarity  to  suggest  curriculum   ▪ speed  development  of  assessments   ▪ query  expansion   ▪ amending  ontology
  • 49.
    A  plug  for  InnerSource… 49 We  thought  the  introduction  of  data  science  had  run   headlong  into  enterprise  silos  and  lingering  tech  debt.  
 As  if!!     Introduction  of  AI  exacerbates  that  problem  even   more.  Suggested  responses:   ▪ InnerSourceCommons.org  open  source  practices  
 within  enterprise   ▪ design  patterns  for  working  across  silos   ▪ think:  “good  house  rules  for  guests”  as  other  
 teams  submit  PRs  on  your  code  repos
  • 50.
  • 51.
    A  generational  shift? ▪We’re  12  years  beyond  the  introduction   of  YouTube  …  anyone  raising  tweens  
 now  probably  knows  about  YouTubers   ▪ Below  a  certain  age  demographic,  people   tend  to  rely  more  on  video  and  audio   sources  for  information,  while  perhaps   print  is  gaining  more  for  entertainment.   Mobile  certainly  has  huge  impact  there. 51
  • 52.
    {"graf": [[21, "let","let", "VB", 1, 48], [0, "'s", "'s", "PRP", 0, 49], "take", "take", "VB", 1, 50], [0, "a", "a", "DT", 0, 51], [23, "look", "l "NN", 1, 52], [0, "at", "at", "IN", 0, 53], [0, "a", "a", "DT", 0, 54], [ "few", "few", "JJ", 1, 55], [25, "examples", "example", "NNS", 1, 56], [0 "often", "often", "RB", 0, 57], [0, "when", "when", "WRB", 0, 58], [11, "people", "people", "NNS", 1, 59], [2, "are", "be", "VBP", 1, 60], [26, " "first", "JJ", 1, 61], [27, "learning", "learn", "VBG", 1, 62], [0, "abou "about", "IN", 0, 63], [28, "Docker", "docker", "NNP", 1, 64], [0, "they" "they", "PRP", 0, 65], [29, "try", "try", "VBP", 1, 66], [0, "and", "and" 0, 67], [30, "put", "put", "VBP", 1, 68], [0, "it", "it", "PRP", 0, 69], "in", "in", "IN", 0, 70], [0, "one", "one", "CD", 0, 71], [0, "of", "of", 0, 72], [0, "a", "a", "DT", 0, 73], [24, "few", "few", "JJ", 1, 74], [31, "existing", "existing", "JJ", 1, 75], [18, "categories", "category", "NNS 76], [0, "sometimes", "sometimes", "RB", 0, 77], [11, "people", "people", 1, 78], [9, "think", "think", "VBP", 1, 79], [0, "it", "it", "PRP", 0, 80 "'s", "be", "VBZ", 1, 81], [0, "a", "a", "DT", 0, 82], [32, "virtualizati "virtualization", "NN", 1, 83], [19, "tool", "tool", "NN", 1, 84], [0, "l "like", "IN", 0, 85], [33, "VMware", "vmware", "NNP", 1, 86], [0, "or", " "CC", 0, 87], [34, "virtualbox", "virtualbox", "NNP", 1, 88], [0, "also", "also", "RB", 0, 89], [35, "known", "know", "VBN", 1, 90], [0, "as", "as" 0, 91], [0, "a", "a", "DT", 0, 92], [36, "hypervisor", "hypervisor", "NN" 93], [0, "these", "these", "DT", 0, 94], [2, "are", "be", "VBP", 1, 95], "tools", "tool", "NNS", 1, 96], [0, "which", "which", "WDT", 0, 97], [2, "be", "VBP", 1, 98], [37, "emulating", "emulate", "VBG", 1, 99], [38, "hardware", "hardware", "NN", 1, 100], [0, "for", "for", "IN", 0, 101], [ "virtual", "virtual", "JJ", 1, 102], [40, "software", "software", "NN", 1 103]], "id": "001.video197359", "sha1": "4b69cf60f0497887e3776619b922514f2e5b70a8"} Video  transcription 52 {"count": 2, "ids": [32, 19], "pos": "np", "rank": 0.0194, "text": "virtualization tool"} {"count": 2, "ids": [40, 69], "pos": "np", "rank": 0.0117, "text": "software applications"} {"count": 4, "ids": [38], "pos": "np", "rank": 0.0114, "text": "hardware"} {"count": 2, "ids": [33, 36], "pos": "np", "rank": 0.0099, "text": "vmware hypervisor"} {"count": 4, "ids": [28], "pos": "np", "rank": 0.0096, "text": "docker"} {"count": 4, "ids": [34], "pos": "np", "rank": 0.0094, "text": "virtualbox"} {"count": 10, "ids": [11], "pos": "np", "rank": 0.0049, "text": "people"} {"count": 4, "ids": [37], "pos": "vbg", "rank": 0.0026, "text": "emulating"} {"count": 2, "ids": [27], "pos": "vbg", "rank": 0.0016, "text": "learning"} Transcript: let's take a look at a few examples often when people are first learning about Docker they try and put it in one of a few existing categories sometimes people think it's a virtualization tool like VMware or virtualbox also known as a hypervisor these are tools which are emulating hardware for virtual software Confidence: 0.973419129848 39 KUBERNETES 0.8747 coreos 0.8624 etcd 0.8478 DOCKER CONTAINERS 0.8458 mesos 0.8406 DOCKER 0.8354 DOCKER CONTAINER 0.8260 KUBERNETES CLUSTER 0.8258 docker image 0.8252 EC2 0.8210 docker hub 0.8138 OPENSTACK orm:Docker a orm:Vendor; a orm:Container; a orm:Open_Source; a orm:Commercial_Software; owl:sameAs dbr:Docker_%28software%29; skos:prefLabel "Docker"@en;
  • 53.
    Fave  mediated  learning  experience:  audio+ 53
  • 54.
    How  will  a  next  generation  learn?
  • 55.
  • 56.
    Active  learning ▪ special  case  of  semi-­‐supervised  machine  learning   ▪ send  difficult  calls  /  edge  cases  to  experts;  
 let  algorithms  handle  routine  decisions   ▪ works  well  in  use  cases  which  have  lots  of  
 inexpensive,  unlabeled  data   ▪ e.g.,  abundance  of  content  to  be  classified,  
 where  the  cost  of  labeling  is  the  expense   ▪ https://en.wikipedia.org/wiki/ Active_learning_(machine_learning) 56
  • 58.
    Active  learning Data  preparation  in  the  age  of  deep  learning
 oreilly.com/ideas/data-­‐preparation-­‐in-­‐the-­‐ age-­‐of-­‐deep-­‐learning
 Luke  Biewald    CrowdFlower
 O’Reilly  Data  Show,  2017-­‐05-­‐04   send  human  workers  cases  where  machine  learning   algorithms  signal  uncertainty  (low  probability  scores)     or  when  your  ensemble  of  machine  learning   algorithms  signals  disagreement 58
  • 59.
    Human-­‐in-­‐the-­‐loop  design  pattern   Building  a  business  that  combines  human  experts   and  data  science
 oreilly.com/ideas/building-­‐a-­‐business-­‐that-­‐ combines-­‐human-­‐experts-­‐and-­‐data-­‐science-­‐2
 Eric  Colson    StitchFix
 O’Reilly  Data  Show,  2016-­‐01-­‐28   “what  machines  can’t  do  are  things  around  cognition,
    things  that  have  to  do  with  ambient  information,  or
    appreciation  of  aesthetics,  or  even  the  ability  to
    relate  to  another  human”
 
 59
  • 60.
    Weak  supervision Creating  large  training  data  sets  quickly
 oreilly.com/ideas/creating-­‐large-­‐ training-­‐data-­‐sets-­‐quickly
 Alex  Ratner    Stanford
 O’Reilly  Data  Show,  2017-­‐06-­‐08   Snorkel:  “data  programming”  as  another  
 instance  of  human-­‐in-­‐the-­‐loop
 github.com/HazyResearch/snorkel   conferences.oreilly.com/strata/strata-­‐ny/ public/schedule/detail/61849 60
  • 61.
    Collaboration  through  Jupyter 61 Notebooks  get  used  to  manage  ML  pipelines,   where  machines  +  people  collaborate  on  docs   ▪ “Human-­‐in-­‐the-­‐loop  design  pattern”
 talk  @  JupyterCon  NY  2017   ▪ experts  adjust  parameters  in  ML  pipelines   ▪ machines  write  structured  “logs”  of  ML   modeling  and  evaluation   ▪ experts  run  `jupyter  notebook`  via  SSH  tunnel  
 for  remote  monitoring  and  updates   ▪ https://pypi.python.org/pypi/nbtransom
  • 62.
  • 63.
    Collaboration  through  Jupyter ▪running  notebooks  via  SSH   tunnel  removes  the  need  for   dedicated  UIs   ▪ this  work  anticipates  upcoming   collaborative  document  features   in  JupyterLab:   Realtime  collaboration  for   JupyterLab  using  Google  Drive
 Ian  Rose    UC  Berkeley
  • 64.
    Expert  review ▪ ML  pipelines  report  results:  recognizing  content,  adding  annotations,   requesting  more  examples  when  “confused”   ▪ Human-­‐in-­‐the-­‐loop  experts  –  potentially,  Customer  Service  –  
 review  decisions,  especially  edge  cases,  then  train  through  examples   ▪ The  system  iterates 64
  • 65.
    What’s  the  point  of  using  AI  in  Media? ▪ more  work,  quicker,  than  could  be  performed  
 by  editors  –  who  are  already  super-­‐busy  people   ▪ exceeding  human  parity,  as  a  benchmark   ▪ helps  relieve  pressure  on  organizations,  
 as  learning  curves  accelerate   ▪ augments  some  of  our  most  valuable
 experts,  so  they  can  get  more  done 65
  • 66.
    Human-­‐in-­‐the-­‐loop  as  a  management  strategy 66 personal  op-­‐ed:  the  “game”  isn’t  to  replace  people  –   instead  it’s  about  leveraging  AI  to  augment  staff,  
 so  organizations  can  retain  people  with  valuable   domain  expertise,  making  their  contributions  and   expertise  even  more  vital
  • 67.
    Why  we’ll  never  run  out  of  jobs 67
  • 68.
    Strata  Data   NY,  Sep  25-­‐28
 SG,  Dec  4-­‐7
 SJ,  Mar  5-­‐8,  2018
 UK,  May  21-­‐24,  2018   The  AI  Conf   SF,  Sep  17-­‐20
 NY,  Apr  29-­‐May  2,  2018   JupyterCon   NY,  Aug  22-­‐25   OSCON  (returns!)   PDX,  Jul  16-­‐19,  2018 68
  • 69.
    69 contact  the  speaker  for  
 conf  discount  coupons
  • 70.
    70 Learn  Alongside
 Innovators Just  Enough  Math Building  Data   Science  Teams Hylbert-­‐Speys How  Do  You  Learn? updates,  reviews,  conference  summaries…   liber118.com/pxn/
 @pacoid