Very Large Scale Global Optimization with Randomised Optimisation Algorithms in DAPHNE

Introduction ROA DAPHNE Results Conclusion
33rd
International Electrotechnical and Computer Science Conference ERK 2024
Computer and Information Science, CS. 1 “Streaming Computational Intelligence”
Congress Center Bernardin, Portorož, Slovenia, 26–27 September, 2024
Room D — Obala 2, Portorož, Slovenia „11:00 Thursday, 26th
Very Large Scale Global
Optimization with Randomised
Optimisation Algorithms in DAPHNE
Aleš Zamuda
University of Maribor
<ales.zamuda@um.si>
Acknowledgement. This work is supported by project DAPHNE (Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC and Machine Learning) funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 957407 and ARIS (Slovenian Research And Innovation Agency)
programme P2-0041 (Computer Systems, Methodologies, and Intelligent Services). AZ gratefully acknowledges the HPC RIVR consortium (www.hpc-rivr.si) and EuroHPC JU (eurohpc-ju.europa.eu) for funding this research by providing computing resources of the HPC system Vega at the Institute of Information Science (www.izum.si).
0.2 0.4 0.6 0.8 1
·105
0
0.5
1
·105
Dimension D [integer]
Runtime
[seconds]
Runtime of ROA in DAPHNE
depending on
problem dimension size
ROA in DAPHNE
Aleš Zamuda 7@aleszamuda Very Large Scale Global Optimization with Randomised Optimisation Algorithms in DAPHNE @ ERK 2024 (Portorož) 1/ 34

33rd
—
Initial Introduction
& Outline
—

Introduction & Outline: Content
1 (1 minutes) Part I: Background — Randomised
Optimisation Algorithms (ROA)
2 (7 minutes) Part II: ROA in DAPHNE on EuroHPC Vega
3 (4 minutes) Part III: VLSGO ROA in DAPHNE on Vega MODA
4 (2 minutes) Part IV: Conclusion with Takeaways
5 (≈3 minutes) Questions, Misc
6 (Appendix) Additional lecture materials

Introduction: Aims of this Talk — HPC & Benchmarking
A closer observation of execution times for
workloads processed in [2] is provided in
Fig. 1, where it is seen that the execution
time (color of the patches) changes for
different benchmark executions.
Fig. 1: Execution time of full benchmarks
for different instances of optimization
algorithms. Each patch presents one full
benchmark execution to evaluate an
optimization algorithm.
Warmup Highlights on (Generative) AI w/ ChatGPT+Synthesia: visiting Canaries/ASHPC/WCCI
Photo/video: 1) generative animation 2) HPC generated introduction (ASHPC23); 3) with underwater glider at ULPGC SIANI; 4) infront
SIANI; 5) with autonomous sailboat at SIANI;
If 2023 was about Generative AI, is 2024 on CI omnia Robotics?
• Therefore, it is useful to consider speeding up of benchmarking through vectorization of the
tasks that a benchmark is comprised of — e.g.:
• parallell data cleaning part of an individual ML tile [1] or
• synchronization between tasks when executing parallell geospatial processing [3].
• To enable the possibilities of data cleaning (preprocessing) as well as geospatial processing in
parallell ( ERK’06 a), such opportunities ﬁrst need to be found or designed, if none yet exist
for a problem tackled.
• Therefore, this contribution will highlight some experiences with ﬁnding and designing
parallell ML pipelines for vectorization and observe speedup.
• The speeding up focus will be on optimization algorithms within such ML pipelines.
a
A. Zamuda and N. Guid. “Modeliranje, simulacija in upodabljanje gozdov”. In: Zbornik petnajste mednarodne Elektrotehniške in
računalniške konference ERK 2006, 25. – 27. september 2006, Portorož, Slovenija. Ljubljana: IEEE Region 8, Slovenska sekcija IEEE,
2006. Zvezek B. 2006, pp. 391–392.
[2] Zamuda, A, Brest, J., Self-adaptive control parameters’ randomization frequency and propagations in differential evolution. Swarm and Evolutionary
Computation 25C, 72-99 (2015).
[3] Zamuda, A., Hernández Sosa, J. D., Success history applied to expert system for underwater glider path planning using differential evolution. Expert Systems
with Applications 119, 155-170 (2019).

33rd
—
I. Background — Randomised Optimisation
Algorithms (ROA)
—

ROA and Implementations in DAPHNE
• Differential Evolution (DE) is a ROA, a ﬂoating point
encoding Evolutionary Algorithm (EA) ROA for global
optimization over continuous spaces,
• through generations, the evolution process
improves population of vectors,
• iteratively by combining a parent individual and several other
individuals of the same population, using evolutionary
operators.
• We choose the strategy DE/rand/1/bin
• mutation: vi,G+1 = xr1,G + F × (xr2,G − xr3,G),
• crossover:
ui,j,G+1 =
(
vi,j,G+1 if rand(0, 1) ≤ CR or j = jrand
xi,j,G otherwise
,
• selection: xi,G+1 =
(
ui,G+1 if f(ui,G+1) < f(xi,G)
xi,G otherwise
,
Previously: ASHPC24, CoBCom:
• ROA in DAPHNE Benchmarked
(ASHPC24: Apple M1/M3, CoBCom,
July 2024 & D8.3 August: Vega/Slurm)
• Testing: convergence of a ML system;
ROA: Randomised Optimisation
Algorithm
• As seen from the plots, the ﬁtness
values are convergent, optimizing.
-350
-300
-250
-200
-150
-100
-50
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
A convergence plot for
the function f2, on
different independent
runs.
-50
-40
-30
-20
-10
0
10
0 50 100 150 200 250 300
Fitness, run 1
Fitness, run 2
Fitness, run 3
Fitness, run 4
Fitness, run 5
Fitness, run 6
Fitness, run 7
Fitness, run 8
Fitness, run 9
An example ROA run, with convergence
plot for the HappyCat function (f6),
on different independent runs.
f2 =
P
x5 + 1 + max(x, 0), 0

f6 =
P
x2, 0

− 10
2
0.125
+
P
x2, 0

/2 +
P
(x)

/10 + 0.5

33rd
—
DAPHNE: VLSGO ROA in DAPHNE on Vega MODA
—

DAPHNE Partners: Project Consortium
Project Consortium
13 partner institutions
from 7 countries
• DM, ML, HPC
• Academia industry
• Different application
domains
14
• Technical University Berlin1
University of Maribor (UM): UM FERI research team DAPHNE (lead: A. Zamuda), SLING connection (EuroHPC Vega).
https://feri.um.si/en/research/international-and-structural-funds-projects/integrated-data-analysis-pipelines-for-large-scale-data-management-hpc-and-machine-learning/
1
Publication Ofﬁce of the European Union. “Fact Sheet : Integrated Data Analysis Pipelines for Large-Scale Data Management, HPC, and
Machine Learning”. In: CORDIS – EU research results. 2024, https://cordis.europa.eu/project/id/957407. DOI: 10.3030/957407.

DAPHNE: Overview (Generic Aspect of the Project)
Overview: Generic Aspect of the Project
• Deployment Challenges
• Hardware Challenges
• DM+ML+HPC share compilation
and runtime techniques /
converging cluster hardware
• End of Dennard scaling:
P = α CFV2 (power density 1)
• End of Moore’s law
• Amdahl’s law: sp = 1/s
 Increasing Specialization
#1 Data
Representations
Sparsity Exploitation
from Algorithms to HW
dense
graph
sparse
compressed
#2 Data
Placement
Local vs distributed
CPUs/
NUMA
GPUs
FPGAs/
ASICs
#3 Data
(Value) Types
FP32, FP64, INT8,
INT32, INT64, UINT8,
BF16, TF32, FlexPoint
[NVIDIA
A100]
 DAPHNE Overall Objective:
Open and extensible system infrastructure
Different
Systems/
Libraries
Dev Teams Programming Models
Resource
Managers
Cluster
Under-
utilization
Data/File
Exchange
3 lessons learnt so far
choices made, methodology

DAPHNE: Functionalities (from Language Abstractions
to Distributed Vectorized Execution and Use Cases)
y
Functionality Introduction: from Language Abstractions to
Distributed Vectorized Execution and Use Cases
• Federated matrices/frames + distribution primitives
• Hierarchical vectorized pipelines and scheduling
• Coordinator
(spawns distributed fused pipeline)
• #1 Prepare Inputs
(N/A, repartition, broadcasts,
slices broadcasts as necessary)
• #2 Coarse-grained Tasks
(tasks run vectorized pipeline)
• #3 Combine Outputs
(N/A, all-reduce, rbind/cbind)
Node 1
X
[1:
100M]
Node 2
X
[100M:
200M]
colmu
colsd
y
y
(X)
XTX
XTy
dc = DaphneContext()
G = dc.from_numpy(npG)
G = (G != 0)
c = components(G, 100, True).compute()
Python API DaphneLib
def components(G, maxi, verbose) {
n = nrow(G); // get the number of vertexes
maxi = 100;
c = seq(1, n); // init vertex IDs
diff = inf; // init diff to +Infinity
iter = 1;
// iterative computation of connected components
while(diff0 iter=maxi) {
u = max(rowMaxs(G * t(c)), c); // neighbor prop
diff = sum(u != c); // # of changed vertexes
c = u; // update assignment
iter = iter + 1;
}
}
Domain-specific Language DaphneDSL
Multiple dispatch of functions/kernels
Example Use Cases
• DLR Earth Observation
• ESA Sentinel-1/2 datasets  4PB/year
• Training of local climate zone classifiers on So2Sat LCZ42
(15 experts, 400K instances, 10 labels each, ~55GB HDF5)
• ML pipeline: preprocessing,
ResNet-20, climate models
• IFAT Semiconductor Ion Beam Tuning
• KAI Semiconductor Material Degradation
• AVL Vehicle Development Process (ejector geometries, KPIs)
• ML-assisted simulations, data cleaning, augmentation
• Cleaning during exploratory query processing

About HPC: Vega Supercomputer (TOP500) EuroHPCs
Demo at Euro-PAR 20232, also on EuroHPC Vega3 OpenStack; ASHPCs 2021-244
ASHPC23: EuroHPC Vega tour
Ales Zamuda
@a�eszamuda
While visiting today I had the honor visiting the spectacular
MareNostrum supercomputers and their installation. From observing
afar in ‘06 as v1v2 were deployed w/ 4294 Tflops and v3 passing the
Pflop, this tour ‘23 to v4 and v5 was sourcely. Thanks
#sors
�BSC_CNS
@rosabadia
Ales Zamuda ·
@a�eszamuda Sep 12
Show this thread
Today I am delighted to present a Severo Ochoa Research Seminar (SORS) at
Barcelona Supercomputing Center #BSC, titled: EuroHPC AI in DAPHNE
(host: Rosa Badia @rosabadia, Workflows and Distributed Computing Group
Manager, CS, BSC) bsc.es/research-and-d… #presenting @daphne_eu
4:20 PM · Sep 12, 2023 · Views
165
View post engagements
5
Post your reply Reply
Post
Ales Zamuda on X: While visiting @BSC_CNS today I had the hono... https://twitter.com/aleszamuda/status/1701601792952512938
1 od 2 19. 09. 23, 11:14
• Towards HPC ROA in DAPHNE: June 2023, ASHPC23: 2 extended abstract
submitted (ROA are sought in DAPHNE; ROA role in GenAI),
• August 2023, Euro-Par 2023: initial demo created for ROA, conﬁrming that ROA
can run in DAPHNE (Apple M1)
• September 2023, seminars at Alicante (UA) and Barcelona (BSC): role of ROA in
NLP (incl. GenAI), ROA deployment preparations base start for Mare Nostrum 5
(not opened yet at that time)
• January 2024, HiPEAC 2024: presentation of the initial benchmarking of ROA in
DAPHNE (Apple M3)
• June 2024, ASHPC 2024: presentation of the further benchmarking of ROA in
DAPHNE (Apple M3)
• July 2024, CoBCom 2024: Vega ROA in DAPHNE
2
A. Vontzalidis et al. “DAPHNE Runtime: Harnessing Parallelism for Integrated Data Analysis Pipelines”. In: Euro-Par 2023: Parallel Processing
Workshops. Ed. by Demetris Zeinalipour et al. Lecture Notes in Computer Science, vol. 14352. Cham: Springer, 2024, pp. 242–246.
3
Aleš Zamuda and Mark Dokter. “Deploying DAPHNE Computational Intelligence on EuroHPC Vega for Benchmarking Randomised Optimisation
Algorithms”. In: International Conference on Broadband Communications for Next Generation Networks and Multimedia Applications (CoBCom).
2024, pp. 1–8.
4
A. Zamuda. “Parallelization of benchmarking using HPC: text summarization in natural language processing (NLP), glider piloting in deep-sea
missions, and search algorithms in computational intelligence (CI)”. In: Austrian-Slovenian HPC Meeting 2021 - ASHPC21. 2021, p. 35; Aleš Zamuda.
“Generative AI Using HPC in Text Summarization and Energy Plants”. In: Austrian-Slovenian HPC Meeting 2023–ASHPC23. 2023, p. 5.

CoBCom 2024: Identified DAPHNE ROA Opportunities
• Deploying Randomised Optimisation Algorithms
(ROA) in DAPHNE on EuroHPC Vega allows
• benchmarking and research in novel and
innovative models for Artificial Intelligece
(AI).
• The methods that are supported in DAPHNE
allow seamless distribution of AI memory
• that is required when an AI algorithm run requires a
large memory that can be distributed across different
HPC nodes.
• Using DAPHNE, the benchmarking can be not
only run in
• the Runtime Environment on an HPC that is much larger
than a regular laptop computer,
• but also gather monitoring data of the workload while
the algorithm is running,
• to obtain a benchmarking profile, allowing an
informed scientific observation of a novel
algorithm under test.
To discuss these results of the proposed approach from
a more distant context:
• we provide listing the main advantages (potentials for scaling)
and limitations (newly establishing language).
• Namely, the potential for scaling the ROA was successfully
benchmarked (scaling through tasks in Slurm) and as a limitation,
• we can mention that DAPHNE is a new language and the ROA
deployed still has limitations and does not include more
advanced fitness functions
0 1 2 3 4 5 6 7 8 9
Combined power from 110 MW to 975 MW, step 0.01 MW 104
0
100
200
300
400
500
600
700
Individual
output
(power
[MW]
or
unit
total
cost
[$])
Cost, TC / 3
Powerplant P1 power
Powerplant P2 power
Powerplant P3 power
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1 10 100 1000 10000
ROUGE-1R
ROUGE-2R
ROUGE-LR
ROUGE-SU4R
Fitness (scaled)
Real examples: science and HPC
• improved scheduling of workload in distributed multi-node
Slurm tasks, and comparisons of benchmarking resultsa
, which
are among our ongoing research work.
150
200
250
300
350
400
450
500
550
16 32 48 64 80
Seconds
to
compute
a
workload
Number of tasks (equals 16 times the SLURM --nodes parameter)
Summarizer workload
a
Zamuda, “Parallelization of benchmarking using HPC: text
summarization in natural language processing (NLP), glider piloting in
deep-sea missions, and search algorithms in computational intelligence
(CI)”; Zamuda, “Generative AI Using HPC in Text Summarization and
Energy Plants”.

33rd
—
CoBCom (revisited, D ≤ 1, 000):
ROA in DAPHNE on Vega —
D ≤ 1, 000 Contribution, Results Discussion
—

CoBCom 2024: Deployment Setup
Deployment command full instruction text:

CoBCom 2024: Fitness Function in LLVM
Example LLVM code for lowering of the fitness function implementation
(HappyCat function):
In line 1 the eval f6 function definition begins and in line 108 it ends, including
the constants definition (lines 2–7), allocations (e.g. in lines 8 and 14), and the
specific calls to the implemented kernels (e.g. for matrix operations like
addition/subtraction in lines 23/29 or multiplication/division in lines 17/83).

CoBCom 2024: Code Setup
Cloning the DAPHNE main repository,
from daphne-eu repository of daphne-eu at GitHub.
• The DAPHNE system is downloaded as source code,
cloning DAPHNE main repository:
In line 1, the git command is invoked, then the remote code is cloned into the
local ﬁle system in lines 2–9; and a change of directory ends it in line 11.

CoBCom 2024: Container Setup (Singularity)
A singularity image is compiled locally and transferred to the Vega so that it can
be used later for compilation of DAPHNE.
• Building the build environment image (Singularity container):
In line 1, the singularity command is invoked, then the build proceeds at lines
2–22 and completes by creating the daphneeu.sif image until line 24.

CoBCom 2024: Container Transfer to Target Machine
When the container image is compiled, the image is copied to Vega, where the
two-factor authentication and checking of the user access certificate take place.
• Transferring the built Singularity container image:
In line 1, the image file is specified, while authentication takes place in lines 2–3,
then copying proceeds in line 4.

CoBCom 2024: Compilation
on Target Machine Within Singularity Container
With the singularity image and source code inplace, the DAPHNE system is then
compiled on the target system (Vega) from source code.
• Building the DAPHNE system on Vega:
In line 1, the compilation is started, then there are thousands of lines of output
(not printed here), and the build then ﬁnishes.

CoBCom 2024: ROA Deployment on Target Machine
After the images are prepared, a ROA is implemented in roa.d and run with different
conﬁgurations.
• Deployment of ROAs with DAPHNE using Slurm on Vega:
Sample conﬁgurations with for...do are seen in lines surrounding the srun command
in lines 6–10 and saving of outputs and timings at lines 11 and 12, respectively.
For each task, up to 1 GB memory and 10 minutes node use are requested to run the
workload using the container.

CoBCom 2024: Data Flowchart
• To further explain the deployment and benchmarking of ROA in DAPHNE for our
use case, we also provide a data flowchart
• it is seen in the Figure on the right and shows how the data flows in the ROA use case.
• The ROA@HappyCat center part in red color
• is completely addressed by the DAPHNE system, within the core of the use case.
• as the DE parameters D, G, NP, and function for fitness
evaluation (HappyCat), are provided
• The pipeline generates data analysis reports
• (e.g. in PDF format, in the bottom of the flowchart),
• after it builds: the Singularity image from Docker platform
and DAPHNE from GitHub source;
• and runs the ROA tasks over Slurm (flowchart top).
• Then contribute in generating the reports:
• The collection of logs and cleanup (after waiting of the
tasks completion)
• and lookup into the Slurm database to see resource use.

CoBCom 2024: Flowchart Details (Code Recap)
• First, the DAPHNE system is downloaded as source code, cloning DAPHNE
main repository as daphne-eu repository by daphne-eu at GitHub
• see Figure on page 16: in line 1, the git command is invoked, then the remote code is cloned into
the local file system in lines 2–9; and a change of directory ends it in line 11.
• Then, a singularity image is compiled locally and transferred to the Vega
so that it can be used later for compilation of DAPHNE
• see Figure on page 17: in line 1, the singularity command is invoked,
then the build proceeds at lines 2–22
and completes by creating the daphneeu.sif image until line 24.
• When the container image is compiled, the image is copied to Vega, where the
two-factor authentication and checking of the user access certificate take place
• see Figure on page 18: in line 1, the image file is specified,
while authentication takes place in lines 2–3, then copying proceeds in line 4.
• With the singularity image and source code inplace, the DAPHNE system is then compiled on the target
system (Vega)
• from source code, see Figure on page 19: in line 1, the compilation is started,
then there are thousands of lines of output (not printed here), and the build then finishes.
• After the images are prepared, a ROA is implemented in roa.d
and run with different configurations, as seen in Figure on page 20:
• sample configurations with for...do are seen in lines surrounding the srun command in lines
6–10 and saving of outputs and timings at lines 11 and 12, respectively.
• For each task, up to 1 GB memory and 10 minutes node use are requested to run the workload using the
container.

CoBCom 2024: Experiment Setup and Convergence Results
• The optimisation results from the
ROA runs (fitness convergence
through generations) as
explained in the above
deployment preparation,
• as a set of convergence graphs, in
configurations with dimensions
D ∈ {10, 100, 1000} and population
sizes NP ∈ {10, 100, 1000}.
• For each of the runs plotted, we
observe that the fitness function
optimised by the ROA is successfully
improving, hence, the ROA CI is
performing its main functionality
of optimisation.
• The respective timings of the real
time to allocate and execute
different job variations as reported
by time command.
Convergent optimisation runs (fitness on vertical axis) using DAPHNE ROA on
different independent seeds:

CoBCom 2024: Experimental Results — Timing
• We can observe that the configuration of
D = 1000 and NP = 1000 in
case (i) has the far highest time
requirements overall for these cases.
• When observing each of the subfigures
separately, we see some limited degree of
variation in job allocation and execution
waiting time from 2 to 22 seconds, but
these are much less than the case (i) that
always reported timings above 100 seconds
(with only run 5 and 7 above 200 seconds,
but still below maximum requested
allocation of 10 minutes).
• We also further inspected the Slurm
database to profile run 7 and see that
while it consumed 92.139 Wh in 9m 42s,
• the task has spent only 8 seconds waiting to
be allocated, on empty current user queue,
which further demonstrates fast Vega task
allocations, practical in this use case.
Times (in seconds) of running optimisation runs, for different configurations:
The respective timings of the real time to allocate and execute different job variations
as reported by time command.

CoBCom 2024: Experimental Results — Timing (Combined)
• Also, when running just a subset of the jobs
with much more similar timings (e.g. jobs
with D = 100, NP = 100) for much more
independent runs,
• the speed up is mostly capped by
the longest running job.
• While the responsiveness of the Slurm
scheduler varies slightly due to HPC
workload of all running jobs,
• the batching of the set of jobs however greatly
reduces the time required to execute a batch,
• compared to just sequentially running each job.
• Also, as allocation time is important for HPC
users so that they do not wait for their
results longer than running sequentially,
• the allocation times by far did not exceed the
combined time,
• i.e. the speed up was signiﬁcant also from the
user perspective.
Combined time (left bar in the plot) vs. batched time (right):
500
1000
1500
2000
2500
3000
Time
To compare timings, we observe the combined time of
processing all batched jobs,
• compared to running them with Slurm,
demonstrating the speedup of real time needed by
runnning the tasks in parallell, and hence, scaling.

33rd
—
III. ERK: ROA in DAPHNE on Vega for VLSGO
(D ≥ 10, 000) — Contribution, Results Discussion
—

ERK 2024: ROA Fitness Implementation and Deployment
• Fitness function implementation in DaphneDSL:
def e v a l f ( x : matrixf64 ) − matrixf64 {
return ( (sum( x * x , 0) − 10) ˆ 2 ) ˆ 0.125 + ( sum( x * x , 0) / 2 + sum( x ) ) / 10 + 0 . 5 ;
}

}
• Deployment (uses Singularity for DAPHNE system):
multiplier of 288 minutes per scale S was used.
for S in {1..10}; do
for NP in 1000; do
D=${S}0000
echo D=$D NP=$NP
for RNi in {1..10}; do
echo −n .
{ time srun −−mpi=none
–time ((288∗S)) −−
mem=${S}G
. . / daphneeu . s i f
. / run−daphne . sh roa . d
D=$D NP=$NP RNi=$RNi
results−D−$D−NP−$NP−RNi−$RNi−out . t x t
} 2 time−D−$D−NP−$NP−RNi−$RNi−out . t x t
done #RNi
done #NP
done #D
wait

}
• Deployment (uses Singularity for DAPHNE system):
multiplier of 288 minutes per scale S was used.
for S in {1..10}; do
for NP in 1000; do
D=${S}0000
echo D=$D NP=$NP
for RNi in {1..10}; do
echo −n .
{ time srun −−mpi=none
–time ((288∗S)) −−
mem=${S}G
. . / daphneeu . s i f
. / run−daphne . sh roa . d
D=$D NP=$NP RNi=$RNi
results−D−$D−NP−$NP−RNi−$RNi−out . t x t
} 2 time−D−$D−NP−$NP−RNi−$RNi−out . t x t
done #RNi
done #NP
done #D
wait
DAPHNE system version used: July 26, 2024 (main branch, 548ea01).
DaphneDSL syntaxa:
• inspired by C/Java-like languages, case-sensitive,
• like Julia, Python NumPy, R, and Apache SystemDS DMLb,
• compiler hints for data/operator placement (i. e. local/distributed,
CPU/GPU/FPGA, computational storage)c.
a
DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines,
“DaphneDSL Language Reference”.
b
Boehm et al., “SystemDS: A Declarative Machine Learning System for the End-to-End Data Science
Lifecycle”.
c
DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines,
“DaphneDSL Language Reference”.

ERK 2024: Experiment Setup and Convergence Results
• The optimisation results from the ROA runs (fitness convergence through
generations) as explained in the above deployment preparation,
• as a set of convergence graphs, in configurations with dimensions D = 10000S, S ∈ {1, 2, 3, ..., 10}
and population size NP ∈ {1000}.
• For each of the runs plotted, we observe that the fitness function optimised by the ROA is
successfully improving, hence, the ROA CI is performing its main functionality of
optimisation.
• All jobs have successfully reached target generations of 300 for the VLSGO tasks,
except 4 cancelled earlier by Slurm due to timeout: on node cn0514 (jobs 31746064 on S = 5, RNi = 6 at G = 298; 31746065 on
S = 3, RNi = 6 and RNi = 1 both at G = 289) and cn0570 (31746113, G = 269).
Convergent optimisation runs (fitness on vertical axis) using DAPHNE ROA on function HappyCat, for different
independent seeds, D = 10000S, S ∈ {1, 2, 3, ..., 10} and population size NP ∈ {1000}:
-12000
-10000
-8000
-6000
-4000
-2000
0
2000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-16000
-14000
-12000
-10000
-8000
-6000
-4000
-2000
0
2000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-20000
-15000
-10000
-5000
0
5000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-25000
-20000
-15000
-10000
-5000
0
5000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-25000
-20000
-15000
-10000
-5000
0
5000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-25000
-20000
-15000
-10000
-5000
0
5000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-25000
-20000
-15000
-10000
-5000
0
5000
10000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-30000
-25000
-20000
-15000
-10000
-5000
0
5000
10000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-30000
-25000
-20000
-15000
-10000
-5000
0
5000
10000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
-35000
-30000
-25000
-20000
-15000
-10000
-5000
0
5000
10000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
• The respective timings of the real time to allocate and execute different job variations as reported by Umlaut monitor (next page).

ERK 2024: Experimental Results — Load Monitoring
• The respective timings of the real time to execute different job variations as reported by
Universal Machine Learning Analysis Utility (Umlaut) from GitHub
https://github.com/daphne-eu/umlaut; and their resource monitoring during execution.

ERK 2024: Experimental Results — Load Monitoring
• The respective timings of the real time to execute different job variations as reported by
Universal Machine Learning Analysis Utility (Umlaut) from GitHub
https://github.com/daphne-eu/umlaut; and their resource monitoring during execution.
• Total runtime
Total runtime over independent runs of initially D = 10.000, NP = 1000.
0 1000 2000 3000 4000 5000 6000 7000
Time taken in seconds
HPC time
from
2024-08-03 10:48:23
HPC time
from
2024-08-03 10:50:50
HPC time
from
2024-08-03 10:51:10
HPC time
from
2024-08-03 10:54:16
HPC time
from
2024-08-03 10:55:51
HPC time
from
2024-08-03 10:56:16
HPC time
from
2024-08-03 10:57:09
HPC time
from
2024-08-03 11:02:11
HPC time
from
2024-08-03 11:05:39
HPC time
from
2024-08-03 11:19:09
5770.01
5931.00
5929.37
6116.42
6211.01
6236.19
6265.27
6603.99
6823.02
7600.40
Metric: Time
• Run times over scaling level S for D, where runtime dash-dotted linear fit of data
points is seen increasing with S.
• Plots for CPU load
CPU load are drawn: for RNi = 1 (the first run) of D = 10.000 (smallest S) and
RNi = 4 (typical, median run) of D = 100.000 (largest S).
0h 0m 0.00s 0h 26m 24.40s 0h 52m 8.29s 1h 17m 55.01s 1h 43m 30.90s
Time elapsed since start of pipeline run
12.5
48.3
109.0
169.7
230.5
291.2
351.9
412.6
473.4
534.1
594.8
655.6
CPU
usage
in
%
Metric: CPU usage
Time elapsed since start of pipeline run
17.3
47.9
113.1
178.3
243.5
308.7
374.0
439.2
504.4
569.6
634.8
700.0
CPU
usage
in
%
Metric: CPU usage
D = 10.000, RNi = 1 D = 100.000, RNi = 5
• CPU load is baselined at 100% (one thread) and the jitters with higher loads are attributed to
multi-threaded executions of DAPHNE kernels with Basic Linear Algebra Subprograms (BLAS).
Deviations in running time among independent runs were observed and could be studied
further.
Times (in seconds) of running optimisation
runs, for different configurations:
0 20000 40000 60000 80000 100000
Time taken in seconds
HPC time ROA D=10000 NP=1000 RNi=4
from
2024-08-03 10:56:16
from
2024-08-03 12:59:54
from
2024-08-03 15:11:47
from
2024-08-03 17:23:23
from
2024-08-03 19:46:52
from
2024-08-03 22:22:57
from
2024-08-03 22:53:07
from
2024-08-04 03:46:32
from
2024-08-04 06:34:54
from
2024-08-04 12:29:47
6236.19
13638.10
21579.85
29462.71
38076.01
47448.61
49272.19
66865.54
76977.61
98269.75
Metric: Time
0.2 0.4 0.6 0.8 1
·105
0
0.5
1
·105
Runtime
[seconds]
depending on
ROA in DAPHNE
Total runtime
CPU load

ERK 2024: Experimental Results — Memory Monitoring
• Furthermore, usage of memory resources is monitored using Umlaut.
• As memory was seen increasing during the runa,
in meanwhile after experiments have already been done in August using latest July code
updates and by the time of revision of this paper in September, also for loopsb
,
the implementation of allocations has been upgraded in the language implementation with
newest software release (version 0.3) and additional regular updates.
• Successfully initially observed using Umlaut:
• the memory usage plot peeks at approximately 105.87 MB for these runs,
by initially rising to roughly 89 MB
and then staying almost all of the time at that usage,
slightly varying because of iterative allocations.
a
Benjamin Steinwender et al. Integrated Data Analysis Pipelines for Large-Scale Data Management,
HPC, and Machine Learning : D8.3 Benchmarking Results all Use Case Studies. Tech. rep. Version 2.1.
KAI (KAI Kompetenzzentrum Automobil- und Industrieelektronik GmbH), DLR (Deutsches Zentrum für
Luft- und Raumfahrt EV), IFAT (Infineon Technologies Austria AG), AVL (AVL List GmbH), and UM
(Univerza v Mariboru), 2024.
b
Borko Bošković, Janez Brest, and Aleš Zamuda. “Loops of the Domain-specific Programming
Language DaphneDSL”. In: Proceedings of 33rd International Electrotechnical and Computer Science
Conference. 2024, (in print).
Seconds elapsed since start of pipeline run
87.44
89.11
90.79
92.46
94.14
95.82
97.49
99.17
100.84
102.52
104.20
105.87
Memory
usage
in
MB
Metric: Memory usage
Memory usage profiles during the run
execution of DAPHNE ROA on function
HappyCat, D = 10.000, NP = 1000, and 10
RNi configuration values.

33rd
—
IV: Conclusion
with Takeaways
—

ERK 2024: Conclusion Takeaways
• Presented a deployment of DAPHNE (Integrated Data Analysis Pipelines for
Large-Scale Data Management, HPC and Machine Learning)
on EuroHPC Vega, running ROA CI tasks with Slurm.
• An example ROA benchmarking scenario was benchmarked
using HappyCat function on VLSGO (very large dimensions)
• and comparing overall run times and monitoring was discussed.
• In further detail:
• the insight explanation to resource monitoring
of the applied HappyCat function was presented,
• the deployment preparation was reviewed step by step from a ﬂowchart,
and
• running of ROA was discussed for VLSGO
• from CI conﬁguration and convergence perspective,
• from HPC perspective (e.g. timing and other resource use and monitoring),
• as well as in perspective from current main advantages and limitations for
prospective users and ongoing work.

Further Research — After ERK 2024
• Future work includes
• further deployment (e.g. to additional hardware),
• benchmarking, and
• extending the ROA scenario and library
• as well as research in other use cases,
• especially from CI and remote sensing, including
• underwater missions like ocean glider path planning
• and text summarization.
• Next week after ERK’24, Bernardin (Daphnia): DAPHNE travels to Athens.
0.2 0.4 0.6 0.8 1
·105
0
0.5
1
·105
Runtime
[seconds]
depending on
ROA in DAPHNE
-35000
-30000
-25000
-20000
-15000
-10000
-5000
0
5000
10000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
10000
1x106
1x108
1x1010
1x1012
0 1 2 3 4 5 6 7 8 9
FES
Digits
f1
f2
f3
f4
f5
f6
f7
f8
f9
f10
0 1 2 3 4 5 6 7 8 9
0
100
200
300
400
500
600
700
Individual
output
(power
[MW]
or
unit
total
cost
[$])
Cost, TC / 3
Powerplant P1 power
Powerplant P2 power
Powerplant P3 power
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1 10 100 1000 10000
ROUGE-1R
ROUGE-2R
ROUGE-LR
ROUGE-SU4R
Fitness (scaled)
150
200
250
300
350
400
450
500
550
16 32 48 64 80
Seconds
to
compute
a
workload
Summarizer workload

Thank you. Questions?
Thanks!
Acknowledgement: this work is supported by DAPHNE, funded by the European Union’s Horizon 2020
research and innovation programme under grant agreement No 957407.
0.2 0.4 0.6 0.8 1
·105
0
0.5
1
·105
Runtime
[seconds]
depending on
ROA in DAPHNE
-35000
-30000
-25000
-20000
-15000
-10000
-5000
0
5000
10000
0 50 100 150 200 250 300
Run 1
Run 2
Run 3
Run 4
Run 5
Run 6
Run 7
Run 8
Run 9
Run 10
10000
1x106
1x108
1x1010
1x1012
0 1 2 3 4 5 6 7 8 9
FES
Digits
f1
f2
f3
f4
f5
f6
f7
f8
f9
f10
0 1 2 3 4 5 6 7 8 9
0
100
200
300
400
500
600
700
Individual
output
(power
[MW]
or
unit
total
cost
[$])
Cost, TC / 3
Powerplant P1 power
Powerplant P2 power
Powerplant P3 power
-0.05
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
1 10 100 1000 10000
ROUGE-1R
ROUGE-2R
ROUGE-LR
ROUGE-SU4R
Fitness (scaled)
150
200
250
300
350
400
450
500
550
16 32 48 64 80
Seconds
to
compute
a
workload
Summarizer workload

Very Large Scale Global Optimization with Randomised Optimisation Algorithms in DAPHNE

More Related Content

Similar to Very Large Scale Global Optimization with Randomised Optimisation Algorithms in DAPHNE

More from University of Maribor

Recently uploaded

Very Large Scale Global Optimization with Randomised Optimisation Algorithms in DAPHNE