Cloud platforms are missing out on the revolution in new hardware and network technologies for realising vastly richer computational, communication, and storage resources. Technologies such as
field-programmable gate arrays (FPGA), general-purpose graphics processing units (GPGPU), network middleboxes, and solid-state disks promise increased performance, reduced energy consumption, and lower
cost profiles. However, their heterogeneity and complexity makes integrating them into the standard cloud Platform as a Service (PaaS) framework a fundamental challenge. This talk introduces the HARNESS project, whose aim is to explore how to bring these innovative and heterogeneous resources into cloud platforms.
6. http://www.harness-project.eu/
Goal: Programmable and Manageable
GPU-based
parallel-thread
engines
FPGA-based
shared dataflow
engines
Solid-state
disk drives
ASIC-based
OpenFlow
switching fabric
Middleboxes for
in-network aggregation
and storage
7. http://www.harness-project.eu/
Approach: Enrich IaaS and PaaS
Provide an IaaS layer that can manage
heterogeneous resources
– computation, communication and storage
– resource allocation and scheduling
Provide a PaaS layer that can exploit
heterogeneous resources
– multi-tenancy
– application development
– cross-resource allocation and scheduling
8. http://www.harness-project.eu/
Driving Use Cases
basis for demonstration and validation
shared memory
cache cache cache
CPUs CPUs CPUs…
I/O
Delta Merge for SAP HANA
in-memory OLTP and OLAP
queries for “big data” analytics
Reverse Time Migration (RTM)
scientific computation for the
geosciences
……
f1f1
fnfn
y E {−1,1}y E {−1,1}
predictpredict
updateupdate
…
f1
fn
y E {−1,1}
predict
update
…
Share state
Aggregate
Iterate
…
Parallelize
…
Preprocess
AdPredictor Machine Learning
open-source “map/reduce”
data-flow distributed computation
9. http://www.harness-project.eu/
Driving Use Cases
basis for demonstration and validation
shared memory
cache cache cache
CPUs CPUs CPUs…
I/O
Delta Merge for SAP HANA
in-memory OLTP and OLAP
queries for “big data” analytics
Reverse Time Migration (RTM)
scientific computation for the
geosciences
……
f1f1
fnfn
y E {−1,1}y E {−1,1}
predictpredict
updateupdate
…
f1
fn
y E {−1,1}
predict
update
…
Share state
Aggregate
Iterate
…
Parallelize
…
Preprocess
AdPredictor Machine Learning
open-source “map/reduce”
data-flow distributed computationO(109) entries in daily Web visit logO(109) entries in daily Web visit log
two weeks on 300 multi-core nodestwo weeks on 300 multi-core nodes
20% of cycles and 10s of seconds locking20% of cycles and 10s of seconds locking