Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The Rise of Machine Intelligence

531 views

Published on

Big Thought Leaders Colloquium Series – Spring 2017
Jackson State University
Jackson, MS
April 11, 2017

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

The Rise of Machine Intelligence

  1. 1. “The Rise of Machine Intelligence” Big Thought Leaders Colloquium Series – Spring 2017 Jackson State University Jackson, MS April 11, 2017 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD http://lsmarr.calit2.net 1
  2. 2. Abstract Over the next decade, we can expect autonomous machine intelligence to become pervasive in our society. To help spur that future, Calit2 has set up a Pattern Recognition Laboratory with a variety of novel low-energy processors that can execute real-time trained neural networks in the exploding mobile environment of drones, robots, and self- driving cars. However, the training of these neural networks requires massive amounts of Big Data and computing time. To support this need the NSF-funded Pacific Research Platform (PRP), which connects two dozen research universities at 100-1000 times the speed of the commodity Internet, is creating a new community of computer science machine learning researchers and proposing using the optical fiber backbone of the PRP to create a distributed Graphics Processing Unit computing “cloud.” Finally, I will speculate on the exponentially growing machine intelligence and how it will increasingly inter-operate with human intelligence.
  3. 3. Things Are About to Get Very Interesting… Source: Hans Moravec www.transhumanist.com/volume1/power_075.jpg Smarr Slide from 2001
  4. 4. The Defining Issue in IT for the Coming Decades May 5, 2015August 25, 2015
  5. 5. Traffic Control for Drone Air Delivery is Under Development by NASA, Amazon, & Google
  6. 6. Self-Driving Cars Have Appeared on the Market
  7. 7. I Am Living in the Self-Driving Future Autopilot at 71 MPH
  8. 8. Streaming Data From the Tesla Fleet Trains Self-Driving Algorithms: The “Hive-Mind” Advantage Note: Google Self-Driving Cars Have Only Driven 1.5 Million Miles
  9. 9. One Major Source of Jobs May Be the First Victim of Driverless Vehicles
  10. 10. DOD: “Perdix Drones Share One Distributed Brain for Decision-Making, Adapting to Each Other Like Swarms in Nature.”
  11. 11. The Planetary-Scale Computer Fed by a Trillion Sensors Will Drive a Global Industrial Internet www-bsac.eecs.berkeley.edu/frontpagefiles/BSACGrowingMEMS_Markets_%20SEMI.ORG.html Next Decade One Trillion “Within the next 20 years the Industrial Internet will have added to the global economy an additional $15 trillion.” --General Electric www.ge.com/docs/chapters/Industrial_Internet.pdf
  12. 12. What is the Cyberinfrastructure Needed For The World of Big Data Autonomous Machines? • Massive Multi-Architecture Cloud Computing • Trained Neural Nets Downloaded onto Robots with NvN ML Accelerators • Robots Use Neural Nets to Navigate with Real-Time Data Streams • Swarm Input to Update Training on Neural Nets
  13. 13. For ¾ of a Century, Computing Has Relied on von Neumann’s Architecture
  14. 14. Is It Time to Radically Expand Our Computer Architectures? NCSA 1988 Supercomputer Architectures Remain von Neumann Shared Memory CPU Plus SIMD Co-Processor NCSA 2016
  15. 15. The DOE and NSF Petascale Supercomputers All Are Built with CPU/GPU Nodes
  16. 16. Like Supercomputers, Commercial Cloud Providers Are Adding GPU Accelerators All Use Double Precision Nvidia Tesla GPUs
  17. 17. But, Commercial Cloud Providers Are Also Introducing NvN Accelerators: Microsoft is Using Field Programmable Gate Arrays (FPGAs) • Microsoft Installs FPGAs into Bing Servers – FPGAs are a Non von Neumann (NvN) Architecture – Improved the Ops/Sec of a Critical Component of Bing’s Search Engine by Nearly 2x – Many Other Applications and Services Can be Accelerated as Well www.microsoft.com/en-us/research/project/project-catapult/
  18. 18. The Democratization of Deep Learning: Google’s TensorFlow https://exponential.singularityu.org/medicine/big-data-machine-learning-with-jeremy-howard/ From Programming Computers Step by Step To Achieve a Goal To Showing the Computer Some Examples of What You Want It to Achieve and Then Letting the Computer Figure It Out On Its Own --Jeremy Howard, Singularity Univ. 2015
  19. 19. Google Designed a NvN Machine Learning Accelerator
  20. 20. AI is Advancing at an Unprecedented Pace: Deep Learning Algorithms Working on Massive Datasets 1.5 Years! Training on 30M Moves, Then Playing Against Itself Google Used TPUs to Achieve the Go Victory
  21. 21. Exascale (1000 PetaFLOPs) Will Blend Traditional HPC and Data Analytics: U.S. Committed to Building by 2025 “High Performance Computing Will Evolve Towards a Hybrid Model, Integrating Emerging Non-von Neumann Architectures, with Huge Potential in Pattern Recognition, Streaming Data Analysis, and Unpredictable New Applications.” Horst Simon, Deputy Director, U.S. Department of Energy’s Lawrence Berkeley National Laboratory
  22. 22. Realtime Simulation of Human Brain Possible Within the Next Ten Years With Exascale Supercomputer Horst Simon, Deputy Director, Lawrence Berkeley National Laboratory’s National Energy Research Scientific Computing Center Fastest Supercomputer Trend Line Tianhe-2
  23. 23. Using Nanotechnology to Read Out the Living Brain Is Accelerating Under the Federal Brain Initiative www.whitehouse.gov/infographics/brain-initiative
  24. 24. Reverse Engineering of the Brain: Large Scale Microscopy of Mammal Brains Reveals Complex Connectivity Source: Rat Cerebellum Image, Mark Ellisman, UCSD Neuron Cell Bodies Neuronal Dendritic Overlap Region
  25. 25. The Rise of Brain-Inspired Computers: Left & Right Brain Computing: Arithmetic vs. Pattern Recognition Adapted from D-Wave
  26. 26. Brain-Inspired Processors Are Accelerating the non-von Neumann Architecture Era “On the drawing board are collections of 64, 256, 1024, and 4096 chips. ‘It’s only limited by money, not imagination,’ Modha says.” Source: Dr. Dharmendra Modha Founding Director, IBM Cognitive Computing Group August 8, 2014
  27. 27. Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab For Machine Learning on non-von Neumann Processors “On the drawing board are collections of 64, 256, 1024, and 4096 chips. ‘It’s only limited by money, not imagination,’ Modha says.” Source: Dr. Dharmendra Modha Founding Director, IBM Cognitive Computing Group August 8, 2014 UCSD ECE Professor Ken Kreutz-Delgado Brings the IBM TrueNorth Chip to Start Calit2’s Qualcomm Institute Pattern Recognition Laboratory September 16, 2015
  28. 28. Our Pattern Recognition Lab is Exploring Mapping Machine Learning Algorithm Families Onto Novel Architectures • Deep & Recurrent Neural Networks (DNN, RNN) • Graph Theoretic • Reinforcement Learning (RL) • Clustering and Other Neighborhood-Based • Support Vector Machine (SVM) • Sparse Signal Processing and Source Localization • Dimensionality Reduction & Manifold Learning • Latent Variable Analysis (PCA, ICA) • Stochastic Sampling, Variational Approximation • Decision Tree Learning Source: Prof. Ken Kreutz-Delgado, Director PRL, UCSD
  29. 29. New Brain-Inspired Non-von Neumann Processors Are Emerging: KnuEdge is Essentially a Cloud-on-a-Chip That Scales to 512K Chips www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html www.calit2.net/newsroom/release.php?id=2704 “KnuEdge and Calit2 have worked together since the early days of the KnuEdge LambdaFabric processor, when key personnel and technology from UC San Diego provided the genesis for the first processor design.” www.calit2.net/newsroom/release.php?id=2726 June 6, 2016 KnuEdge Has Provided Processor to Calit2’s PRL
  30. 30. Neurobiological Systems are Flexible and Scalable 30 Animal # Neurons Flatworm 302 Medicinal leech 10,000 Pond snail 11,000 Sea slug 18,000 Fruit fly 100,000 Lobster 100,000 Ant 250,000 Honey bee 960,000 Cockroach 1,000,000 Frog 16,000,000 Mouse 75,000,000 Bat 110,000,000 Octopus 300,000,000 Human 100,000,000,000 Elephant 200,000,000,000 Current generation machine learning capabilities Where KnuEdge wants to be in 2021: MindScale. Source: Doug Palmer, CTO KnuEdge
  31. 31. Building a Big Data Cyberinfrastructure for Machine Learning
  32. 32. Based on Community Input and on ESnet’s Science DMZ Concept, NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways Red 2012 CC-NIE Awardees Yellow 2013 CC-NIE Awardees Green 2014 CC*IIE Awardees Blue 2015 CC*DNI Awardees Purple Multiple Time Awardees Source: NSF
  33. 33. Next Step: The Pacific Research Platform Creates a Regional End-to-End Science-Driven “Big Data Superhighway” System NSF CC*DNI Grant $5M 10/2015-10/2020 PI: Larry Smarr, UC San Diego Calit2 Co-Pis: • Camille Crittenden, UC Berkeley CITRIS, • Tom DeFanti, UC San Diego Calit2, • Philip Papadopoulos, UCSD SDSC, • Frank Wuerthwein, UCSD Physics and SDSC Letters of Commitment from: • 50 Researchers from 15 Campuses • 32 IT/Network Organization Leaders
  34. 34. FIONAs and FIONettes – Flash I/O Network Appliances: Linux PCs Optimized for Big Data Over Distance FIONAs Are Science DMZ Data Transfer Nodes (DTNs) & Also Compute/Visualization/ML Nodes Phil Papadopoulos & Tom DeFanti Joe Keefe & John Graham FIONAS—40G, $8,000 FIONette—1G, $1,000
  35. 35. PRP Continues to Expand Rapidly While Increasing Connectivity: 1 1/2 Years of Progress – 12 Sites to 23 Sites January 29, 2016 March 29, 2016 Connected 23 DMZs at 10G and 40G, demonstrating disk-to-disk GridFTP at ~7.5G and 12.5G respectively
  36. 36. PRP’s First 1.5 Years: Connecting Campus Application Teams and Devices
  37. 37. Adding a Cognitive Hardware and Software Ecosystem To the Pacific Research Platform • Working With 30 CSE Machine Learning Researchers – Goal is 320 Game GPUs in 32-40 FIONAs at 10 PRP Campuses – PRP Couples FIONAs with GPUs into a Condor-Managed Cloud • PRP Access to Emerging Processors – IBM TrueNorth, KnuEdge, FPGA, and Qualcomm Snapdragon • Software Including a Wide Range of Open ML Algorithms • Metrics for Performance of Processors and Algorithms Source: Tom DeFanti, Calit2 Multiple Proposals Under Review FIONA with 8-Game GPUs
  38. 38. Single vs. Double Precision GPUs: Gaming vs. Supercomputing 8 x 1080 Ti: 1 million GPU core-hours every two days, 500 million for $15K in 3yrs
  39. 39. Contextual Robots Need Low Energy Neuromorphic Processors That Can See and Learn Wirelessly Tied Into the Planetary Cloud Computer Professor Tajana Rosing
  40. 40. Kurzweil’s Theory of Mind: The Human Neocortex is a Self-Organizing Hierarchical System of Pattern Recognizers “There are ~300M Pattern Recognizers in the Human Neocortex.” In the Emerging Synthetic Neocortex, “Why Not a Billion? Or a Trillion?” November 13, 2012
  41. 41. Should We Give Robots Autonomy?
  42. 42. This Next Decade’s Computing Transition Will Not Be Just About Technology "Those disposed to dismiss an 'AI takeover' as science fiction may think again after reading this original and well- argued book." —Martin Rees, Past President, Royal Society If our own extinction is a likely, or even possible, outcome of our technological development, shouldn't we proceed with great caution? – Bill Joy Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks. – Steven Hawking
  43. 43. For Further Information: http://lsmarr.calit2.net/

×