The document discusses the anticipated rise of autonomous machine intelligence and the establishment of a pattern recognition laboratory aimed at enhancing real-time trained neural networks for applications like drones and self-driving cars. It emphasizes the critical need for vast amounts of data and computing power, supported by initiatives like the Pacific Research Platform, which seeks to create a high-speed computing environment for machine learning research. The text speculates on the integration of machine and human intelligence and the future implications of AI advancements on society and labor.

1. “The Rise of
Machine Intelligence”
Big Thought Leaders Colloquium Series – Spring 2017
Jackson State University
Jackson, MS
April 11, 2017
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
http://lsmarr.calit2.net
1

2. Abstract
Over the next decade, we can expect autonomous machine intelligence to become
pervasive in our society. To help spur that future, Calit2 has set up a Pattern Recognition
Laboratory with a variety of novel low-energy processors that can execute real-time
trained neural networks in the exploding mobile environment of drones, robots, and self-
driving cars. However, the training of these neural networks requires massive amounts of
Big Data and computing time. To support this need the NSF-funded Pacific Research
Platform (PRP), which connects two dozen research universities at 100-1000 times the
speed of the commodity Internet, is creating a new community of computer science
machine learning researchers and proposing using the optical fiber backbone of the PRP
to create a distributed Graphics Processing Unit computing “cloud.” Finally, I will
speculate on the exponentially growing machine intelligence and how it will increasingly
inter-operate with human intelligence.

3. Things Are About to Get Very Interesting…
Source: Hans Moravec
www.transhumanist.com/volume1/power_075.jpg
Smarr Slide from 2001

4. The Defining Issue in IT for the Coming Decades
May 5, 2015August 25, 2015

5. Traffic Control for Drone Air Delivery
is Under Development by NASA, Amazon, & Google

6. Self-Driving Cars
Have Appeared on the Market

7. I Am Living in the Self-Driving Future
Autopilot at 71 MPH

8. Streaming Data From the Tesla Fleet Trains Self-Driving Algorithms:
The “Hive-Mind” Advantage
Note: Google
Self-Driving Cars
Have Only Driven
1.5 Million Miles

9. One Major Source of Jobs
May Be the First Victim of Driverless Vehicles

10. DOD: “Perdix Drones Share One Distributed Brain for Decision-Making,
Adapting to Each Other Like Swarms in Nature.”

11. The Planetary-Scale Computer Fed by a Trillion Sensors
Will Drive a Global Industrial Internet
www-bsac.eecs.berkeley.edu/frontpagefiles/BSACGrowingMEMS_Markets_%20SEMI.ORG.html
Next Decade
One Trillion “Within the next 20 years
the Industrial Internet
will have added
to the global economy
an additional $15 trillion.”
--General Electric
www.ge.com/docs/chapters/Industrial_Internet.pdf

12. What is the Cyberinfrastructure Needed
For The World of Big Data Autonomous Machines?
• Massive Multi-Architecture Cloud Computing
• Trained Neural Nets Downloaded onto Robots with NvN ML Accelerators
• Robots Use Neural Nets to Navigate with Real-Time Data Streams
• Swarm Input to Update Training on Neural Nets

13. For ¾ of a Century, Computing Has Relied
on von Neumann’s Architecture

14. Is It Time to Radically Expand
Our Computer Architectures?
NCSA 1988
Supercomputer Architectures Remain von Neumann
Shared Memory CPU Plus SIMD Co-Processor
NCSA 2016

15. The DOE and NSF Petascale Supercomputers
All Are Built with CPU/GPU Nodes

16. Like Supercomputers, Commercial Cloud Providers
Are Adding GPU Accelerators
All Use Double Precision Nvidia Tesla GPUs

17. But, Commercial Cloud Providers Are Also Introducing NvN Accelerators:
Microsoft is Using Field Programmable Gate Arrays (FPGAs)
• Microsoft Installs FPGAs into Bing Servers
– FPGAs are a Non von Neumann (NvN) Architecture
– Improved the Ops/Sec of a Critical Component of Bing’s Search Engine by Nearly 2x
– Many Other Applications and Services Can be Accelerated as Well
www.microsoft.com/en-us/research/project/project-catapult/

18. The Democratization of Deep Learning:
Google’s TensorFlow
https://exponential.singularityu.org/medicine/big-data-machine-learning-with-jeremy-howard/
From Programming Computers
Step by Step To Achieve a Goal
To Showing the Computer
Some Examples of
What You Want It to Achieve
and Then Letting the Computer
Figure It Out On Its Own
--Jeremy Howard, Singularity Univ.
2015

19. Google Designed a NvN
Machine Learning Accelerator

20. AI is Advancing at an Unprecedented Pace:
Deep Learning Algorithms Working on Massive Datasets
1.5 Years!
Training on 30M Moves,
Then Playing Against Itself
Google Used TPUs to Achieve the Go Victory

21. Exascale (1000 PetaFLOPs) Will Blend Traditional HPC and Data Analytics:
U.S. Committed to Building by 2025
“High Performance Computing Will Evolve
Towards a Hybrid Model,
Integrating Emerging Non-von Neumann Architectures,
with Huge Potential in Pattern Recognition,
Streaming Data Analysis,
and Unpredictable New Applications.”
Horst Simon, Deputy Director,
U.S. Department of Energy’s
Lawrence Berkeley National Laboratory

22. Realtime Simulation of Human Brain Possible
Within the Next Ten Years With Exascale Supercomputer
Horst Simon, Deputy Director,
Lawrence Berkeley National Laboratory’s
National Energy Research Scientific Computing Center
Fastest
Supercomputer
Trend Line
Tianhe-2

23. Using Nanotechnology to Read Out the Living Brain
Is Accelerating Under the Federal Brain Initiative
www.whitehouse.gov/infographics/brain-initiative

24. Reverse Engineering of the Brain:
Large Scale Microscopy of Mammal Brains Reveals Complex Connectivity
Source: Rat Cerebellum Image, Mark Ellisman, UCSD
Neuron
Cell Bodies
Neuronal Dendritic
Overlap Region

25. The Rise of Brain-Inspired Computers:
Left & Right Brain Computing: Arithmetic vs. Pattern Recognition
Adapted from D-Wave

26. Brain-Inspired Processors
Are Accelerating the non-von Neumann Architecture Era
“On the drawing board are collections of 64, 256, 1024, and 4096 chips.
‘It’s only limited by money, not imagination,’ Modha says.”
Source: Dr. Dharmendra Modha
Founding Director, IBM Cognitive Computing Group
August 8, 2014

27. Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab
For Machine Learning on non-von Neumann Processors
“On the drawing board are collections of 64, 256, 1024, and 4096 chips.
‘It’s only limited by money, not imagination,’ Modha says.”
Source: Dr. Dharmendra Modha
Founding Director, IBM Cognitive Computing Group
August 8, 2014
UCSD ECE Professor Ken Kreutz-Delgado Brings
the IBM TrueNorth Chip
to Start Calit2’s Qualcomm Institute
Pattern Recognition Laboratory
September 16, 2015

28. Our Pattern Recognition Lab is Exploring Mapping
Machine Learning Algorithm Families Onto Novel Architectures
• Deep & Recurrent Neural Networks (DNN, RNN)
• Graph Theoretic
• Reinforcement Learning (RL)
• Clustering and Other Neighborhood-Based
• Support Vector Machine (SVM)
• Sparse Signal Processing and Source Localization
• Dimensionality Reduction & Manifold Learning
• Latent Variable Analysis (PCA, ICA)
• Stochastic Sampling, Variational Approximation
• Decision Tree Learning
Source: Prof. Ken Kreutz-Delgado, Director PRL, UCSD

29. New Brain-Inspired Non-von Neumann Processors Are Emerging:
KnuEdge is Essentially a Cloud-on-a-Chip That Scales to 512K Chips
www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html
www.calit2.net/newsroom/release.php?id=2704
“KnuEdge and Calit2
have worked together
since the early days of
the KnuEdge LambdaFabric
processor, when key
personnel and technology
from UC San Diego
provided the genesis for
the first processor design.”
www.calit2.net/newsroom/release.php?id=2726
June 6, 2016
KnuEdge Has Provided
Processor to Calit2’s PRL

30. Neurobiological Systems
are Flexible and Scalable
30
Animal # Neurons
Flatworm 302
Medicinal leech 10,000
Pond snail 11,000
Sea slug 18,000
Fruit fly 100,000
Lobster 100,000
Ant 250,000
Honey bee 960,000
Cockroach 1,000,000
Frog 16,000,000
Mouse 75,000,000
Bat 110,000,000
Octopus 300,000,000
Human 100,000,000,000
Elephant 200,000,000,000
Current generation
machine learning
capabilities
Where KnuEdge
wants to be in 2021:
MindScale.
Source: Doug Palmer, CTO KnuEdge

31. Building a Big Data Cyberinfrastructure
for Machine Learning

32. Based on Community Input and on ESnet’s Science DMZ Concept,
NSF Has Funded Over 100 Campuses to Build Local Big Data Freeways
Red 2012 CC-NIE Awardees
Yellow 2013 CC-NIE Awardees
Green 2014 CC*IIE Awardees
Blue 2015 CC*DNI Awardees
Purple Multiple Time Awardees
Source: NSF

33. Next Step: The Pacific Research Platform Creates
a Regional End-to-End Science-Driven “Big Data Superhighway” System
NSF CC*DNI Grant
$5M 10/2015-10/2020
PI: Larry Smarr, UC San Diego Calit2
Co-Pis:
• Camille Crittenden, UC Berkeley CITRIS,
• Tom DeFanti, UC San Diego Calit2,
• Philip Papadopoulos, UCSD SDSC,
• Frank Wuerthwein, UCSD Physics and SDSC
Letters of Commitment from:
• 50 Researchers from 15 Campuses
• 32 IT/Network Organization Leaders

34. FIONAs and FIONettes – Flash I/O Network Appliances:
Linux PCs Optimized for Big Data Over Distance
FIONAs Are
Science DMZ Data Transfer Nodes (DTNs) &
Also Compute/Visualization/ML Nodes
Phil Papadopoulos & Tom DeFanti
Joe Keefe & John Graham
FIONAS—40G, $8,000
FIONette—1G, $1,000

35. PRP Continues to Expand Rapidly While Increasing Connectivity:
1 1/2 Years of Progress – 12 Sites to 23 Sites
January 29, 2016 March 29, 2016
Connected 23 DMZs at 10G and 40G,
demonstrating disk-to-disk GridFTP
at ~7.5G and 12.5G respectively

36. PRP’s First 1.5 Years:
Connecting Campus Application Teams and Devices

37. Adding a Cognitive Hardware and Software Ecosystem
To the Pacific Research Platform
• Working With 30 CSE Machine Learning Researchers
– Goal is 320 Game GPUs in 32-40 FIONAs at 10 PRP Campuses
– PRP Couples FIONAs with GPUs into a Condor-Managed Cloud
• PRP Access to Emerging Processors
– IBM TrueNorth, KnuEdge, FPGA, and Qualcomm Snapdragon
• Software Including a Wide Range of Open ML Algorithms
• Metrics for Performance of Processors and Algorithms
Source: Tom DeFanti, Calit2
Multiple Proposals Under Review
FIONA with 8-Game GPUs

38. Single vs. Double Precision GPUs:
Gaming vs. Supercomputing
8 x 1080 Ti: 1 million GPU
core-hours every two days,
500 million for $15K in 3yrs

39. Contextual Robots Need Low Energy Neuromorphic Processors That
Can See and Learn Wirelessly Tied Into the Planetary Cloud Computer
Professor Tajana Rosing

40. Kurzweil’s Theory of Mind:
The Human Neocortex is a Self-Organizing
Hierarchical System of Pattern Recognizers
“There are ~300M
Pattern Recognizers
in the Human Neocortex.”
In the Emerging
Synthetic Neocortex,
“Why Not a Billion?
Or a Trillion?”
November 13, 2012

41. Should We Give Robots Autonomy?

42. This Next Decade’s Computing Transition
Will Not Be Just About Technology
"Those disposed to dismiss
an 'AI takeover' as science
fiction may think again after
reading this original and well-
argued book." —Martin Rees,
Past President, Royal Society
If our own extinction is
a likely, or even possible,
outcome of our
technological development,
shouldn't we proceed with
great caution? – Bill Joy
Success in creating AI would be
the biggest event in human
history. Unfortunately, it might
also be the last, unless we learn
how to avoid the risks.
– Steven Hawking

43. For Further Information:
http://lsmarr.calit2.net/