AI for All: Biology is eating the world & AI is eating Biology

Biology is eating the
world & AI is eating
Biology
Pradeep K Dubey
Intel Senior Fellow, IEEE Fellow
Director, Parallel Computing Labs

Intel All.AI 2021 @ Population Scale Virtual Summit 2
Machines:
Crunch
Numbers
Humans:
Make
Decisions

Machines:
Crunch
Numbers
Humans:
Make
Decisions
Division of Labor Between Man and Machine Is Getting Disrupted:
Faster than Anyone Predicted!
Machines:
Number Crunching
AND
Decision Making

FROM
A World of
analytical
models
Computational Fluid Dynamics
Start with Mathematical Model
Model  Simulate  Predict
Start with Data
Initial State  Increment  Steer
TO
A World of
Data driven
Models
Event Detection from Social Media
Inside - Out Outside - In

• Effectiveness of AI relies on how well model structure matches the underlying invariant (structure) of the
high-dimensional task objective
• A good set of implicit or explicit inductive bias incorporating domain knowledge
• Such as, CNNs for vision and attention networks for NLP or emerging GNNs
• Training time: How well we manage exploitation versus exploration to get to the most generalizable
(flatter) minima
• Avoiding typical solver attraction to sharp minima
• Higher-order methods
What makes AI effective in practice
5

better understanding of interiors and
evolution of RED GIANT stars
Accurately extract seismic parameters from 1000
spectra in under 10 secs
Measuring the frequency separation ∆ν and period separation ∆Π in red-giant stars using Machine learning, under submission at Science Advances
Department of Astronomy and Astrophysics, Tata Institute of Fundamental, Center for Space Science, NYUAD Institute, New York University Abu Dhabi, Division of Solar and Plasma Astrophysics, NAOJ,
Mitaka, Tokyo, Japan, Parallel Computing Lab, Intel Labs, Bangalore, India

Convergence of Revolutions
Daphne Koller*: https://www.youtube.com/watch?v=V6bSlPNwrKo&feature=youtu.be
Advances in
CELL
biology &
creation of
immense
amount of
data
Advances in
ML to
analyZE
large scale
data and
leverage To
make
Prediction

AI is Eating Biology
8
Biology is experiencing its “AI moment”
Publications involving AI methods (e.g. deep learning, NLP, computer vision, RL) in biology are growing
21000 papers in 2020 alone
> 50% YoY since 2019
Papers since 2019 = 25% of all output
since 2000
https://pubs.acs.org/doi/10.1021/acs.jcim.1c01114

Understand mechanisms, Design Interventions:
Massive Compute Appetite
Big Data: Astronomical or Genomical
https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002195
Algorithmic, Computational & Data Management
Requirements
>1000x
growth
IN COMPUTE
NEEDEDTO MATCH
DEMAND
100’s of TB/s
MEMORY BW AT
100’S OF GB
CAPACITY
Process 100’s of exabytes of
multi-modal data
e.g., Learning on Large Graphs,
structure learning, regulatory
networks, Combinatorial
optimizations…
Secure, Privacy preserving,
Federated

Accelerating Graph Neural
Networks on Xeon
Supercomputing’21 - distGNN: Scalable Distributed Training
for Large-Scale Graph Neural Networks
Full batch Training ~2-3.7x faster on 1s-CLX (1s) for GraphSAGE on OGB-Products & Reddit ~83x for distributed training on 128 sockets on OGB-
Papers
Cascade Lake Xeon: Intel® Xeon® Platinum 8280 Processor 38.5M Cache, 2.70 GHz, 28 cores
[arXiv’20, arXiv’21, SC’21]
DGL v 0.5.3
GraphSAGE on Reddit
GraphSAGE on OGB-Products
OGB-Papers: 100 Million Node Graph
Roofline: Upper &
lower bound
DGL v 0.5.3

LamBdaZero
 Search space 10^18 vs internet 10^9
 Combinatorial Optimization at scale
 Uses ML and HPC to accelerate screening of drug-like
molecules
 @MILA with Prof. Yoshua Bengio
[Intel-MILA announcement]

Bao*: Making Learned Query Optimization Practical
* Paper: https://arxiv.org/abs/2004.03814 , Code: https://learned.systems/bao

Bao outperforms them all!
SIGMOD’21: Best Paper
(Data Management)*
In collab with Prof. Tim
Kraska@MIT
* SIGMOD’21 Best Paper Announcement: https://2021.sigmod.org/sigmod_best_papers.shtml

BWA-MEM2* : An Accelerated
version of BWA MEM
(BWA-MEM has 950K+ Downloads, 70K
Users WW)
15
Higher is better
 In collaboration with Dr Heng Li, Author BWA-MEM
 Reference genome: GRCh38; Read dataset: 50x WGS ERR194147 (NA12878/HG001)
from Illumina HiSeq 2000
Sequence alignment
Cascade Lake Xeon: Intel® Xeon® Platinum 8280 Processor 38.5M Cache, 2.70 GHz, 28 cores
Ice Lake Xeon, ICX: Intel® Xeon® Platinum 8380 Processor 60MB Cache, 2.40 GHz, 40 cores
9.8
15.8
22.1
8.9
2s CLX 2s CLX 2s ICX 1 A100
BWA-MEM BWA-MEM2 Clara Parabricks BWA-
MEM
Throughput in genomes/day for 50x WGS
Higher is better
2.25x
2.5x
Source of Clara Parabricks results: https://at-cg.github.io/posts/ParaBricks-WGS/
Enabling Community Worldwide
https://github.com/bwa-mem2/bwa-mem2
horticulture
nutrition
In production use by Cancer, Ageing and Somatic
Mutations, Wellcome Sanger Institute; tested on ~88
Billion reads

MM2-Fast Accelerates
MINIMAP2 on Xeon by 3.1
Cascade Lake Xeon, CLX: Intel® Xeon® Platinum 8280 Processor 38.5M Cache, 2.70 GHz, 28 cores
[bioRxiv’21]
MM2-Fast Branch in
Minimap2 repo
In collaboration with Dr Heng Li, Author Minimap2
Reference genome: GRCh38; Read dataset: ONT, PacBio HiFi and PacBio CLR datasets derived from human trio benchmark genomes HG002, HG003 and HG004 as given at https://precision.fda.gov/challenges/10/view
and https://github.com/genome-in-a-bottle/giab_data_indexes
Minimap2 has >
100k Downloads

9x speedup for Analysis of Single Cell ATAC-
SEQ Data
Denoising and peak calling on noisy
ATAC-Seq data
Cascade Lake Xeon, CLX: Intel® Xeon® Platinum 8280 Processor 38.5MB Cache, 2.70 GHz, 28 cores
Cooper Lake Xeon, CPX: Intel® Xeon® Platinum 8380H Processor 38.5MB Cache, 2.90 GHz, 28 cores
Higher is better
1.8x
2.3x
Source of Clara Parabricks performance: [Nvidia, 2020] AtacWorks: A deep convolutional neural network toolkit for epigenomics
2.3x speedup over NVIDIA
Clara Parabricks on DGX-1
box (8 card V100) with 16
sockets of Cooper Lake
1.8x speedup over NVIDIA
Clara Parabricks on DGX-1
box (8 card V100) with 16
sockets of Ice Lake
[arXiv’21,
bioRxiv’21]

Brain tumor segmentation finds tumors from
MRIs
Sheller, M.J., Edwards, B., Reina, G.A. et al. Federated learning in medicine: facilitating multi-institutional
collaborations without sharing patient data. Sci Rep 10, 12598 (2020).
Intel-UPenn Collaboration
How much better does each institution do
when training on the full data vs. just their
own data?
17%
BETTER
2.6%
BETTER
on their own validation data
on the hold-out BraTS data
Other names and brands may be claimed as the property of others

1. Privacy Preserved Machine Learning for data
and model privacy / protection
2. Privacy/Confidentiality Preservation
3. Attestation and integrity
4. Federation deployment
5. Federated nodes software stacks for TTM
6. Curation tools and deployment automation
github.com/intel/openfl
openfl.readthedocs.io/
 Enables greatest access to data
 Any company can host a privacy
preserved federation
 Complete software and platform
offering time to market deployment

: a Benchmark Suite For
 Many GenomicsBench benchmarks have abundant data parallelism, but significant irregularity
makes it challenging to achieve good performance.
 12 representative kernels spanning the major steps in short-read and long-read sequence
analysis pipelines
 FM-index, Banded Smith-Waterman, deBruijn graphs, Pair HMM, DP Chaining, SIMD Partial Order
Alignment, Adaptive Banded Signal to Event Alignment, Genomic Relationship Matrix, Neural networks
based Basecalling, Neural networks based variant calling, Kmer counting, Pileup counting
Open-sourced and under active development:
https://github.com/arun-sub/genomicsbench
Xeon Optimized implementations of kernels under active development at:
https://github.com/IntelLabs/Trans-Omics-Acceleration-Library

DISCUSSIONS

AI for All: Biology is eating the world & AI is eating Biology

More Related Content

What's hot

Similar to AI for All: Biology is eating the world & AI is eating Biology

More from Intel® Software

Recently uploaded

AI for All: Biology is eating the world & AI is eating Biology

Editor's Notes