Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Deep Learning: A Critical Appraisal (2018)

3,190 views

Published on

Deep Learning: A Critical Appraisal, Gary Marcus, 2018.
[Link] https://arxiv.org/abs/1801.00631

Published in: Engineering

Deep Learning: A Critical Appraisal (2018)

  1. 1. Terry Taewoong Um (terry.t.um@gmail.com) University of Waterloo Department of Electrical & Computer Engineering Terry T. Um DEEP LEARNING : A CRITICAL APPRAISAL 1 Gary Marcus, New York University
  2. 2. TODAY’S PAPER Terry Taewoong Um (terry.t.um@gmail.com)
  3. 3. CONTENTS Terry Taewoong Um (terry.t.um@gmail.com) 1. Is deep learning approaching a wall? 2. What deep learning is, and what it does well 3. Limits on the scope of deep learning (10 concerns for deep learning) 4. Potential risks of excessive hype 5. What would be better? 6. Conclusions
  4. 4. 1. IS DEEP LEARNING APPROACHING A WALL? Terry Taewoong Um (terry.t.um@gmail.com) Andrew Ng (and UM TAE-NG… it’s me, Terry Taewoong Um…)
  5. 5. 1. IS DEEP LEARNING APPROACHING A WALL? Terry Taewoong Um (terry.t.um@gmail.com)
  6. 6. Terry Taewoong Um (terry.t.um@gmail.com) OUTLINE
  7. 7. 2. WHAT DL IS, AND WHAT IT DOES WELL Terry Taewoong Um (terry.t.um@gmail.com)
  8. 8. 3. LIMITS ON THE SCOPE OF DEEP LEARNING Terry Taewoong Um (terry.t.um@gmail.com) Deep learning thus far 3.1. is data hungry 3.2. is shallow & has limited capacity for transfer 3.3. has no natural way to deal with hierarchical structure 3.4. has struggled with open-ended inference 3.5. is not sufficiently transparent 3.6. has not been well integrated with prior knowledge 3.7. cannot inherently distinguish causation from correlation 3.8. presumes a largely stable world 3.9. its answer often cannot be fully trusted 3.10. is difficult to engineer with
  9. 9. 3.1. DL IS DATA HUNGRY Terry Taewoong Um (terry.t.um@gmail.com)
  10. 10. 3.2. DL IS SHALLOW AND HAS LIMITED CAPACITY FOR TRANSFER Terry Taewoong Um (terry.t.um@gmail.com)
  11. 11. 3.3. DL HAS NO NATURAL WAY TO DEAL WITH HIERARCHICAL STRUCTURE Terry Taewoong Um (terry.t.um@gmail.com)
  12. 12. 3.4. DL HAS STRUGGLED WITH OPEN -ENDED INFERENCE Terry Taewoong Um (terry.t.um@gmail.com) 3.5. DL IS NOT SUFFICIENTLY TRANSPARENT
  13. 13. 3.6. DL HAS NOT BEEN WELL INTEGRATED WITH PRIOR KNOWLEDGE Terry Taewoong Um (terry.t.um@gmail.com)
  14. 14. 3.7. DL CANNOT INHERENTLY DISTINGUISH CAUSATION FROM CORRELATION Terry Taewoong Um (terry.t.um@gmail.com)
  15. 15. 3.8. DL PRESUMES A LARGELY STABLE WORLD, IN WAYS THAT MAY BE PROBLEMATIC Terry Taewoong Um (terry.t.um@gmail.com) 3.9. DL WORKS WELL AS AN APPROX., BUT, ITS ANSWER OFTEN CANNOT BE FULLY TRUSTED
  16. 16. 3.10. DL IS DIFFICULT TO ENGINEER WITH Terry Taewoong Um (terry.t.um@gmail.com) 3.11. DISCUSSION
  17. 17. Terry Taewoong Um (terry.t.um@gmail.com)
  18. 18. Terry Taewoong Um (terry.t.um@gmail.com)
  19. 19. 3. LIMITS ON THE SCOPE OF DEEP LEARNING Terry Taewoong Um (terry.t.um@gmail.com) Deep learning thus far 3.1. is data hungry 3.2. is shallow & has limited capacity for transfer 3.3. has no natural way to deal with hierarchical structure 3.4. has struggled with open-ended inference 3.5. is not sufficiently transparent 3.6. has not been well integrated with prior knowledge 3.7. cannot inherently distinguish causation from correlation 3.8. presumes a largely stable world 3.9. its answer often cannot be fully trusted 3.10. is difficult to engineer with
  20. 20. 4. POTENTIAL RISKS OF EXCESSIVE HYPE Terry Taewoong Um (terry.t.um@gmail.com)
  21. 21. 5. WHAT WOULD BE BETTER Terry Taewoong Um (terry.t.um@gmail.com) 5.1. Unsupervised learning 5.2. Symbol-manipulation a& the need for hybrid models 5.3. More insight from cognitive and develop mental psychology 5.4. Bolder challenges
  22. 22. 5. WHAT WOULD BE BETTER Terry Taewoong Um (terry.t.um@gmail.com) 5.1. Unsupervised learning 5.2. Symbol-manipulation a& the need for hybrid models 5.3. More insight from cognitive and develop mental psychology 5.4. Bolder challenges
  23. 23. 5.1 UNSUPERVISED LEARNING
  24. 24. 5.2 SYMBOL-MANIPULATION & THE NEED FOR HYBRID MODELS Terry Taewoong Um (terry.t.um@gmail.com)
  25. 25. 5.3 MORE INSIGHT FROM COGNITIVE AND DEVELOPMENTAL PSYCHOLOGY Terry Taewoong Um (terry.t.um@gmail.com)
  26. 26. 5.4 BOLDER CHALLENGES Terry Taewoong Um (terry.t.um@gmail.com)
  27. 27. 6. CONCLUSIONS Terry Taewoong Um (terry.t.um@gmail.com)

×