This document provides a summary of a lecture given by Dr. Larry Smarr on the past, present, and future of supercomputing over the last 50 years. The summary discusses: - How Smarr solved equations for colliding black holes in the 1970s using a megaFLOPs computer, whereas today collisions are detected using petaFLOPs supercomputers - a billion fold increase in speed. - How Smarr's research has evolved from modeling astrophysical phenomena to mapping the human gut microbiome using terabytes of sequencing data and hundreds of thousands of core-hours of supercomputing. - Emerging trends in brain-inspired computing architectures and non-von Neumann systems that are better suited to tasks

1. “Fifty Years of Supercomputing:
From Colliding Black Holes to Dynamic Microbiomes
to the Exascale”
Director’s Distinguished Lecture
National Center for Supercomputing Applications
University of Illinois at Urbana-Champaign
September 16, 2016
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor,
Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
http://lsmarr.calit2.net
1

2. Abstract
For the last thirty years, NCSA has played a critical role in bringing computational science
and scientific visualization to the national user community. I will embed those three
decades in the 50 year period 1975 to 2025, beginning with my solving Einstein's equations
for colliding black holes on the megaFLOPs CDC 6600 and ending with the Exascale
supercomputer. This 50 years spans a period in which we will have seen a one trillion-fold
increase in supercomputer speed. Today we see the rise of data science and a new
generation of architectures that balance traditional HPC with data analytics and pattern
recognition. I will illustrate that with my current research into the microbial ecology
dynamics within our human body. Finally, I will describe a number of trends in the decade
to come, in which brain-inspired computing will become more prevalent in both software
and hardware.

3. Personal Reflections on
a Half Century of Supercomputing
• This Lecture Will Not Attempt to Cover in Detail the History of:
– Supercomputers
– NCSA
– NSF’s HPC Program
– Black Hole Collisions
– Microbiomes
– Scientific Visualization
– Cyberinfrastructure
– March to the Exascale
– Machine Learning and Artificial Intelligence
• Rather I Will Discuss Significant Trends I Have Been Involved With

4. Forty Years of Computing Gravitational Waves From Colliding Black Holes –
One Billion Times Increase in Supercomputer Speed!
1977
L. Smarr and K. Eppley
Gravitational Radiation Computed
from an Axisymmetric
Black Hole Collision
40 Years
2016
LIGO Consortium
Spiral Black Hole Collision
MegaFLOPS PetaFLOPS
Holst, et al. Bull. Amer. Math. Soc 53, 513-554 (1916)

5. Documenting The Unmet Supercomputing Needs
of A Broad Range of Disciplines Led to the NCSA Proposal to NSF
1982 1983
1985

6. NCSA Numerical Astrophysics Group
Used NCSA Supercomputers to Explain Cosmic Phenomena
Mike Norman, Charles Evans, Roger Ove, John Hawley,
Dean Sumi, Rob Wolff, Larry Smarr
Gas Accretion Onto a Black Hole
Creates “Exhaust Channels”
Cosmic Jets
Emerge from
Galactic Centers
Collision of Neutron Stars

7. Exponential Increases in Supercomputer Speed and
Visualization Technology Drive Understanding and Applications
Source: Donna Cox, Robert Patterson, Bob Wilhelmson, NCSA
1987
2005
Showed Thunderstorms Arise
from Solving Physics Equations
Vastly Higher Resolution
Uncovers Birth of Tornadoes

8. From Scientific Visualization of Supercomputing Science
to Movie Special Effects
http://access.ncsa.uiuc.edu/
http://movies.warnerbros.com/twister
www.jurassicpark.com; www.jamescameron.org
www.cinemenium.com/perfectstorm/
NCSA 1987
1993
1996
2000
Computer Graphics
From NCSA to ILM
1991
Stefen Fangmeier

9. The Electronic Visualization Lab’s CAVE Virtual Reality System:
Fully Immersive Science and Fantasy Worlds
CAVE conceived in 1991 by Tom DeFanti and Dan Sandin (EVL co-directors) and
implemented by Carolina Cruz-Neira (Ph.D. student)
Crayoland
Colliding Galaxies QUAKE II
The CAVE
• EVL Invents ‘91
• Debuts SIGGRAPH ’92
• National Access NCSA ‘93

10. Industry Has Been a Close Collaborator in NCSA Advances:
Example - Caterpillar
Real Time Linked Virtual Reality and Audio-Video
Between NCSA, Peoria, Houston, and Germany
www.sv.vt.edu/future/vt-cave/apps/CatDistVR/DVR.html
1996
Distributed Virtual Reality for Global-Scale Collaborative Prototyping

11. Visual Supercomputing Goal:
Make Pattern Recognition Analysis as Powerful as Simulation
Colliding Galaxies (Smithsonian IMAX)-Donna Cox, Bob Patterson,
NCSA-From “Cosmic Voyage”-Nominated for Academy Award 1997
• Virtual Director in CAVE
• 1000 Hour SDSC Supercomputer Run to Generate Data
• Tens of Thousands of Hours of NCSA SGI Time to Render Data
• Cross-Country Transfer to IMAX Film of Massive Amounts of Data
Sponsored by Motorola for the Smithsonian

12. NCSA’s National Supercomputing Leadership
With Blue Waters
March 28, 2012November 5, 2008

13. NCSA Blue Waters Continues
NCSA’s Exponential Growth in Supercomputer Power
NCSA Blue Waters
2012
10,000,000
1,000,000
Smarr 2000 Slide

14. Twenty-Five Years from Bleeding Edge Research
to Consumer Mass Market
• 1990 Leading Edge University Research Center-NCSA
– Supercomputer GigaFLOPS Cray Y-MP ($15M)
– Megabit/s NSFnet Backbone
• 2015 Global Market for Smartphones
– Gigaherz Smartphones ($150)
– Megabit/s Wireless Cellular Internet
But, NSF Blue Waters Petascale Supercomputer
Is Over 1 Million Times Faster than Cray Y-MP!
Enormous Growth in Parallelism
Processors: Y-MP 4, Blue Waters 400,000
www.ncsa.uiuc.edu/BlueWaters/system.html

15. Big Data Analytics is a Key Component
of The Future of Supercomputing

16. Global Scientific Instruments Will Produce Ultralarge Datasets Continuously
Requiring Dedicated Optic Fiber and Supercomputers
Large Synoptic Survey Telescope
www.lsst.org/sites/default/files/documents/DM%20Introduction%20-%20Kantor.pdf
Tracks ~40B Objects,
Creates 10M Alerts/Night
Within 1 Minute of Observing
2x40Gb/s

17. There are 100 billion stars in the
Andromeda galaxy…
…and 100 billion galaxies in the
known universe.

18. It’s a microbial world…
…there are 100 million times as many bacteria on Earth
as stars in the universe.
Microbiology is the ultimate Big Data science!

20. Most of Life’s Evolutionary Time
Was in the Microbial World
You
Are
Here
Source: Carl Woese, et al
Tree of Life Derived from 16S rRNA Sequences
Carl Woese Mentored Me
on the Microbiome
While I Was at UIUC

21. Your Body Has 10 Times
As Many Microbe Cells As DNA-Bearing Human Cells
Your Microbiome is
Your “Near-Body” Environment
and its Cells
Contain 100x as Many DNA Genes
As Your Human Cells
Inclusion of the “Dark Matter” of the Body
Will Radically Alter Medicine

22. As a Model for the Precision Medicine Initiative,
I Have Tracked My Internal Biomarkers To Understand My Body’s Dynamics
My Quarterly
Blood Draw
Calit2 64 Megapixel VROOM

23. Longitudinal Time Series Revealed
Oscillatory Behavior in an Immune Variable Which is Antibacterial
Normal Range
<7.3 µg/mL
124x Upper Limit for Healthy
Lactoferrin is a Protein Shed from Neutrophils -
An Antibacterial that Sequesters Iron
Typical
Lactoferrin Value
for
Active
Inflammatory
Bowel Disease
(IBD)

24. To Map Out the Dynamics of Autoimmune Microbiome Ecology
Couples Next Generation Genome Sequencers to Big Data Supercomputers
Source: Weizhong Li, UCSD
Our Team Used 25 CPU-years
to Compute
Comparative Gut Microbiomes
Starting From
2.7 Trillion DNA Bases
of My Samples
and Healthy and IBD Subjects
Illumina HiSeq 2000 at JCVI
SDSC Gordon Data Supercomputer

25. The Supercomputer Converts Tens of Billions of DNA Fragments
Into Relative Abundance of Hundreds of Microbial Species
Average Over 250 Healthy People
From NIH Human Microbiome Project
Note Log Scale
Clostridium difficile

26. Calit2’s Qualcomm Institute Has Developed
Interactive Scalable Visualization for Biological Networks
20,000 Samples
60,000 OTUs
18 Million Edges
Runs Native on QI’s 64Million Pixels Tiled Wall
Software Written by Philip Weber, Calit2’s QI

27. Visually Analyzing Our Supercomputing Data Reveals Vast Changes
in the Human Gut Microbiome in Health and Disease
Ileal
Crohn’s
Healthy
Ulcerative
Colitis
www.sandia.gov/~smartin/presentations/OpenOrd.pdf
Source:
Philip Weber,
QI, UCSD

28. President Obama Announces National Microbiome Initiative
May 13, 2016

29. Complexity of Computing First Gut Microbiome Dynamics
Versus First Dynamics of Colliding Black Holes
• My 1975 PhD Dissertation
– Solving Einstein’s Equations of General Relativity for Colliding Black Holes and Grav Waves
– CDC 6600 Megaflop/s
– Hundred Hours of Computing
• Rob Knight & Smarr Gut Microbiome Map Using 800,000 Core-Hours on SDSC’s Comet
– Mapping From Illumina Sequencing to Taxonomy and Gene Abundance Dynamics
– Comet Petaflop/s
– Comet Core is 40,000x CDC6600 Speed
– ~Million Core-Hours
– 10,000x Supercomputer Time
• Gut Microbiome Takes ~ ½ Billion Times the Compute Power
of Early Solutions of Dynamic General Relativity

30. The Future Foundation of Medicine
is an Exponential Scaling-Up of the Number of Deeply Quantified Humans
Source: @EricTopol
Twitter 9/27/2014

31. President Obama Has Committed The United States
To Building an Exascale (1000 PetaFLOPs) Computing System by 2025
• “Accelerating delivery of a capable exascale computing system”
• “Increasing coherence between the technology base used for
modeling & simulation and that used for data analytic computing”

32. Is It Time to Radically Expand
Our Computer Architectures?
NCSA 1988
Supercomputer Architectures Remain von Neumann
Shared Memory CPU Plus SIMD Co-Processor
NCSA 2016

33. The Future of Supercomputing Will Blend Traditional HPC and Data Analytics
Integrating Non-von Neumann Architectures
“High Performance Computing Will Evolve
Towards a Hybrid Model,
Integrating Emerging Non-von Neumann Architectures,
with Huge Potential in Pattern Recognition,
Streaming Data Analysis,
and Unpredictable New Applications.”
Horst Simon, Deputy Director,
U.S. Department of Energy’s
Lawrence Berkeley National Laboratory

34. Brain-Inspired Processors
Are Accelerating the non-von Neumann Architecture Era
“On the drawing board are collections of 64, 256, 1024, and 4096 chips.
‘It’s only limited by money, not imagination,’ Modha says.”
Source: Dr. Dharmendra Modha
Founding Director, IBM Cognitive Computing Group
August 8, 2014

35. Calit2’s Qualcomm Institute Has Established a Pattern Recognition Lab
For Machine Learning on non-von Neumann Processors
“On the drawing board are collections of 64, 256, 1024, and 4096 chips.
‘It’s only limited by money, not imagination,’ Modha says.”
Source: Dr. Dharmendra Modha
Founding Director, IBM Cognitive Computing Group
August 8, 2014
UCSD ECE Professor Ken Kreutz-Delgado Brings
the IBM TrueNorth Chip
to Start Calit2’s Qualcomm Institute
Pattern Recognition Laboratory
September 16, 2015

36. New Brain-Inspired Non-von Neumann Processors Are Emerging:
KnuEdge Has Provided Processor to Calit2’s PRL
www.tomshardware.com/news/knuedge-announces-knuverse-and-knupath,31981.html
www.calit2.net/newsroom/release.php?id=2704
“KnuEdge and Calit2
have worked together
since the early days of
the KnuEdge LambdaFabric
processor, when key
personnel and technology
from UC San Diego
provided the genesis for
the first processor design.”
www.calit2.net/newsroom/release.php?id=2726
June 6, 2016

37. The Rise of Brain-Inspired Computers – Driver 1:
Left & Right Brain Computing: Arithmetic vs. Pattern Recognition
Adapted from D-Wave

38. The Rise of Brain-Inspired Computers – Driver 2:
Realtime Simulation of Human Brain Possible With Exascale Supercomputer
Horst Simon, Deputy Director,
Lawrence Berkeley National Laboratory’s
National Energy Research Scientific Computing Center
Fastest
Supercomputer
Trend Line
Tianhe-2

39. • Exascale Power Consumption of >20–30 MW
• The Human Brain Runs on 20 W
• Our Brain is a Million Times More Power Efficient!
Horst Simon, Deputy Director,
Lawrence Berkeley National Laboratory’s
National Energy Research Scientific Computing Center
The Rise of Brain-Inspired Computers – Driver 3:
The Exascale Power Conundrum

40. The Rise of Brain-Inspired Computers – Driver 4:
Reverse Engineering of the Brain Is Accelerating
www.whitehouse.gov/infographics/brain-initiative

41. Interactively Exploring Microscope Images of Brains:
40Gbps From NCMIR to Calit2 64Mpixel Wall
Data and Image Source: Mark Ellisman, NCMIR, UCSD

42. Artificial Intelligence (AI) is Advancing at a Amazing Pace:
Deep Learning Algorithms Working on Massive Datasets
Training on 30M Moves,
Then Playing Against Itself
Less Than
2 Years!

43. Google Released Its AI Software as Open Source
Accelerating Development
https://exponential.singularityu.org/medicine/big-data-machine-learning-with-jeremy-howard/
From Programming Computers
Step by Step To Achieve a Goal
To Showing the Computer
Some Examples of
What You Want It to Achieve
and Then Letting the Computer
Figure It Out On Its Own
--Jeremy Howard, Singularity Univ.
2015
November 9, 2015

44. Corporate Cloud Providers
Are Already Using Non von Neumann Accelerators
www.microsoft.com/en-us/research/project/project-catapult/
https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html
Microsoft Installs FPGAs into Bing Servers and 432 into TAAC for Academic Access

45. Deep Learning Will Provide Artificial Intelligence Personalized Assistants
to Coach Us to Wellness
Where Medicine Coaching is Now
Where Wellness Coaching is Going
January 10, 2014

46. Can a Planetary Supercomputer with Artificial Intelligence
Transform Our Sickcare System to Healthcare?
Using this data, the planetary computer will be able
to build a computational model of your body
and compare your sensor stream with millions of others.
Besides providing early detection of internal changes
that could lead to disease,
cloud-powered voice-recognition wellness coaches
could provide continual personalized support on lifestyle
choices, potentially staving off disease
and making health care affordable for everyone.
ESSAY
An Evolution Toward a Programmable
Universe
By LARRY SMARR
Published: December 5, 2011

47. In Spite of the Enormous Promise,
A Global Debate is Underway About the Dangers of Superintelligence
"Those disposed to dismiss
an 'AI takeover' as science
fiction may think again after
reading this original and well-
argued book." —Martin Rees,
Past President, Royal Society
If our own extinction is
a likely, or even possible,
outcome of our
technological development,
shouldn't we proceed with
great caution? – Bill Joy
Success in creating AI would be
the biggest event in human
history. Unfortunately, it might
also be the last, unless we learn
how to avoid the risks.
– Steven Hawking

48. Things Are About to Get Very Interesting…
Source: Hans Moravec
www.transhumanist.com/volume1/power_075.jpg
Smarr Slide from 2001