DASH: A C++ PGAS Library for Distributed Data Structures and Parallel Algorit...Menlo Systems GmbH
We present DASH, a C++ template library that offers distributed data structures and parallel algorithms and implements a compiler-free PGAS (partitioned global address space) approach.
DASH offers many productivity and performance features such as global-view data structures, efficient support for the owner-computes model, flexible multidimensional data distribution schemes and inter-operability with STL (standard template library) algorithms. DASH also features a flexible representation of the parallel target machine and allows the exploitation of several hierarchically organized levels of locality through a concept of Teams. We evaluate DASH on a number of benchmark applications and we port a scientific proxy application using the MPI two-sided model to DASH.
We find that DASH offers excellent productivity and performance and demonstrate scalability up to 9800 cores.
DASH: A C++ PGAS Library for Distributed Data Structures and Parallel Algorit...Menlo Systems GmbH
We present DASH, a C++ template library that offers distributed data structures and parallel algorithms and implements a compiler-free PGAS (partitioned global address space) approach.
DASH offers many productivity and performance features such as global-view data structures, efficient support for the owner-computes model, flexible multidimensional data distribution schemes and inter-operability with STL (standard template library) algorithms. DASH also features a flexible representation of the parallel target machine and allows the exploitation of several hierarchically organized levels of locality through a concept of Teams. We evaluate DASH on a number of benchmark applications and we port a scientific proxy application using the MPI two-sided model to DASH.
We find that DASH offers excellent productivity and performance and demonstrate scalability up to 9800 cores.
Inspired by Runar Bjarnason's scala exchange 2017 keynote https://skillsmatter.com/skillscasts/10746-keynote-composing-programs, I made a slide deck on Functor laws. Download for better quality.
Inspired by Runar Bjarnason's scala exchange 2017 keynote https://skillsmatter.com/skillscasts/10746-keynote-composing-programs, I made a slide deck on Functor laws. Download for better quality.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Enchancing adoption of Open Source Libraries. A case study on Albumentations.AIVladimir Iglovikov, Ph.D.
Presented by Vladimir Iglovikov:
- https://www.linkedin.com/in/iglovikov/
- https://x.com/viglovikov
- https://www.instagram.com/ternaus/
This presentation delves into the journey of Albumentations.ai, a highly successful open-source library for data augmentation.
Created out of a necessity for superior performance in Kaggle competitions, Albumentations has grown to become a widely used tool among data scientists and machine learning practitioners.
This case study covers various aspects, including:
People: The contributors and community that have supported Albumentations.
Metrics: The success indicators such as downloads, daily active users, GitHub stars, and financial contributions.
Challenges: The hurdles in monetizing open-source projects and measuring user engagement.
Development Practices: Best practices for creating, maintaining, and scaling open-source libraries, including code hygiene, CI/CD, and fast iteration.
Community Building: Strategies for making adoption easy, iterating quickly, and fostering a vibrant, engaged community.
Marketing: Both online and offline marketing tactics, focusing on real, impactful interactions and collaborations.
Mental Health: Maintaining balance and not feeling pressured by user demands.
Key insights include the importance of automation, making the adoption process seamless, and leveraging offline interactions for marketing. The presentation also emphasizes the need for continuous small improvements and building a friendly, inclusive community that contributes to the project's growth.
Vladimir Iglovikov brings his extensive experience as a Kaggle Grandmaster, ex-Staff ML Engineer at Lyft, sharing valuable lessons and practical advice for anyone looking to enhance the adoption of their open-source projects.
Explore more about Albumentations and join the community at:
GitHub: https://github.com/albumentations-team/albumentations
Website: https://albumentations.ai/
LinkedIn: https://www.linkedin.com/company/100504475
Twitter: https://x.com/albumentations
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
3. User-defined Functions
• Functions look exactly like scripts, but for ONE difference
Functions must have a function declaration
Help file
Function declaration
InputsOutputs
Courtesy of The MathWorks, Inc. Used with permission.
4. User-defined Functions
• Some comments about the function declaration
• No need for return: MATLAB 'returns' the variables whose
names match those in the function declaration
• Variable scope: Any variables created within the function
but not returned disappear after the function stops running
function [x, y, z] = funName(in1, in2)
Must have the reserved
word: function
Function name should
match MATLAB file
name
If more than one output,
must be in brackets
Inputs must be specified
5. Functions: overloading
• We're familiar with
» zeros
» size
» length
» sum
• Look at the help file for size by typing
» help size
• The help file describes several ways to invoke the function
D = SIZE(X)
[M,N] = SIZE(X)
[M1,M2,M3,...,MN] = SIZE(X)
M = SIZE(X,DIM)
6. Functions: overloading
• MATLAB functions are generally overloaded
Can take a variable number of inputs
Can return a variable number of outputs
• What would the following commands return:
» a=zeros(2,4,8); %n-dimensional matrices are OK
» D=size(a)
» [m,n]=size(a)
» [x,y,z]=size(a)
» m2=size(a,2)
• You can overload your own functions by having variable
input and output arguments (see varargin, nargin,
varargout, nargout)
7. Functions: Excercise
• Write a function with the following declaration:
function plotSin(f1)
• In the function, plot a sin wave with frequency f1, on the
range [0,2π]:
• To get good sampling, use 16 points per period.
( )1sin f x
0 1 2 3 4 5 6 7
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
9. Relational Operators
• MATLAB uses mostly standard relational operators
equal ==
not equal ~=
greater than >
less than <
greater or equal >=
less or equal <=
• Logical operators elementwise short-circuit (scalars)
And & &&
Or | ||
Not ~
Xor xor
All true all
Any true any
• Boolean values: zero is false, nonzero is true
• See help . for a detailed list of operators
10. if/else/elseif
• Basic flow-control, common to all languages
• MATLAB syntax is somewhat unique
IF
if cond
commands
end
ELSE
if cond
commands1
else
commands2
end
ELSEIF
if cond1
commands1
elseif cond2
commands2
else
commands3
end
• No need for parentheses: command blocks are between
reserved words
Conditional statement:
evaluates to true or false
11. for
• for loops: use for a known number of iterations
• MATLAB syntax:
for n=1:100
commands
end
• The loop variable
Is defined as a vector
Is a scalar within the command block
Does not have to have consecutive values (but it's usually
cleaner if they're consecutive)
• The command block
Anything between the for line and the end
Loop variable
Command block
12. while
• The while is like a more general for loop:
Don't need to know number of iterations
• The command block will execute while the conditional
expression is true
• Beware of infinite loops!
WHILE
while cond
commands
end
13. Exercise: Conditionals
• Modify your plotSin(f1) function to take two inputs: plotSin(f1,f2)
• If the number of input arguments is 1, execute the plot command
you wrote before. Otherwise, display the line 'Two inputs were
given'
• Hint: the number of input arguments are in the built-in variable
nargin
15. Plot Options
• Can change the line color, marker style, and line style by
adding a string argument
» plot(x,y,’k.-’);
• Can plot without connecting the dots by omitting line style
argument
» plot(x,y,’.’)
• Look at help plot for a full list of colors, markers, and
linestyles
color marker line-style
16. Playing with the Plot
to select lines
and delete or
change
properties
to zoom in/out
to slide the plot
around
to see all plot
tools at once
Courtesy of The MathWorks, Inc. Used with permission.
17. Line and Marker Options
• Everything on a line can be customized
» plot(x,y,'--s','LineWidth',2,...
'Color', [1 0 0], ...
'MarkerEdgeColor','k',...
'MarkerFaceColor','g',...
'MarkerSize',10)
• See doc line_props for a full list of
properties that can be specified
-4 -3 -2 -1 0 1 2 3 4
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
You can set colors by using
a vector of [R G B] values
or a predefined color
character like 'g', 'k', etc.
18. Cartesian Plots
• We have already seen the plot function
» x=-pi:pi/100:pi;
» y=cos(4*x).*sin(10*x).*exp(-abs(x));
» plot(x,y,'k-');
• The same syntax applies for semilog and loglog plots
» semilogx(x,y,'k');
» semilogy(y,'r.-');
» loglog(x,y);
• For example:
» x=0:100;
» semilogy(x,exp(x),'k.-');
0 10 20 30 40 50 60 70 80 90 100
10
0
10
10
10
20
10
30
10
40
10
50
19. -1
-0.5
0
0.5
1
-1
-0.5
0
0.5
1
-10
-5
0
5
10
3D Line Plots
• We can plot in 3 dimensions just as easily as in 2
» time=0:0.001:4*pi;
» x=sin(time);
» y=cos(time);
» z=time;
» plot3(x,y,z,'k','LineWidth',2);
» zlabel('Time');
• Use tools on figure to rotate it
• Can set limits on all 3 axes
» xlim, ylim, zlim
20. Axis Modes
• Built-in axis modes
» axis square
makes the current axis look like a box
» axis tight
fits axes to data
» axis equal
makes x and y scales the same
» axis xy
puts the origin in the bottom left corner (default for plots)
» axis ij
puts the origin in the top left corner (default for
matrices/images)
21. Multiple Plots in one Figure
• To have multiple axes in one figure
» subplot(2,3,1)
makes a figure with 2 rows and three columns of axes, and
activates the first axis for plotting
each axis can have labels, a legend, and a title
» subplot(2,3,4:6)
activating a range of axes fuses them into one
• To close existing figures
» close([1 3])
closes figures 1 and 3
» close all
closes all figures (useful in scripts/functions)
22. Copy/Paste Figures
• Figures can be pasted into other apps (word, ppt, etc)
• Edit copy options figure copy template
Change font sizes, line properties; presets for word and ppt
• Edit copy figure to copy figure
• Paste into document of interest
Courtesy of The MathWorks, Inc. Used with permission.
23. Saving Figures
• Figures can be saved in many formats. The common ones
are:
.fig preserves all
information
.bmp uncompressed
image
.eps high-quality
scaleable format
.pdf compressed
image
Courtesy of The MathWorks, Inc. Used with permission.
24. Advanced Plotting: Exercise
• Modify the plot command in your plotSin function to use
squares as markers and a dashed red line of thickness 2
as the line. Set the marker face color to be black
(properties are LineWidth, MarkerFaceColor)
• If there are 2 inputs, open a new figure with 2 axes, one on
top of the other (not side by side), and activate the top one
(subplot)
0 1 2 3 4 5 6 7
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.2
0.4
0.6
0.8
1
plotSin(6) plotSin(1,2)
26. Visualizing matrices
• Any matrix can be visualized as an image
» mat=reshape(1:10000,100,100);
» imagesc(mat);
» colorbar
• imagesc automatically scales the values to span the entire
colormap
• Can set limits for the color axis (analogous to xlim, ylim)
» caxis([3000 7000])
27. Colormaps
• You can change the colormap:
» imagesc(mat)
default map is jet
» colormap(gray)
» colormap(cool)
» colormap(hot(256))
• See help hot for a list
• Can define custom colormap
» map=zeros(256,3);
» map(:,2)=(0:255)/255;
» colormap(map);
28. Surface Plots
• It is more common to visualize surfaces in 3D
• Example:
• surf puts vertices at specified points in space x,y,z, and
connects all the vertices to make a surface
• The vertices can be denoted by matrices X,Y,Z
• How can we make these matrices
loop (DUMB)
built-in function: meshgrid
( ) ( ) ( )
[ ] [ ]
f x,y sin x cos y
x , ; y ,π π π π
=
∈ − ∈ −
2 4 6 8 10 12 14 16 18 20
2
4
6
8
10
12
14
16
18
20 -3
-2
-1
0
1
2
3
2 4 6 8 10 12 14 16 18 20
2
4
6
8
10
12
14
16
18
20 -3
-2
-1
0
1
2
3
29. surf
• Make the x and y vectors
» x=-pi:0.1:pi;
» y=-pi:0.1:pi;
• Use meshgrid to make matrices (this is the same as loop)
» [X,Y]=meshgrid(x,y);
• To get function values,
evaluate the matrices
» Z =sin(X).*cos(Y);
• Plot the surface
» surf(X,Y,Z)
» surf(x,y,Z);
30. surf Options
• See help surf for more options
• There are three types of surface shading
» shading faceted
» shading flat
» shading interp
• You can change colormaps
» colormap(gray)
31. contour
• You can make surfaces two-dimensional by using contour
» contour(X,Y,Z,'LineWidth',2)
takes same arguments as surf
color indicates height
can modify linestyle properties
can set colormap
» hold on
» mesh(X,Y,Z)
32. Exercise: 3-D Plots
• Modify plotSin to do the following:
• If two inputs are given, evaluate the following function:
• y should be just like x, but using f2. (use meshgrid to get
the X and Y matrices)
• In the top axis of your subplot, display an image of the Z
matrix. Display the colorbar and use a hot colormap. Set
the axis to xy (imagesc, colormap, colorbar, axis)
• In the bottom axis of the subplot, plot the 3-D surface of Z
(surf)
( ) ( )1 2sin sinZ f x f y= +
33. Specialized Plotting Functions
• MATLAB has a lot of specialized plotting functions
• polar-to make polar plots
» polar(0:0.01:2*pi,cos((0:0.01:2*pi)*2))
• bar-to make bar graphs
» bar(1:10,rand(1,10));
• quiver-to add velocity vectors to a plot
» [X,Y]=meshgrid(1:10,1:10);
» quiver(X,Y,rand(10),rand(10));
• stairs-plot piecewise constant functions
» stairs(1:10,rand(1,10));
• fill-draws and fills a polygon with specified vertices
» fill([0 1 0.5],[0 0 1],'r');
• see help on these functions for syntax
• doc specgraph – for a complete list
35. Revisiting find
• find is a very important function
Returns indices of nonzero values
Can simplify code and help avoid loops
• Basic syntax: index=find(cond)
» x=rand(1,100);
» inds = find(x>0.4 & x<0.6);
• inds will contain the indices at which x has values between
0.4 and 0.6. This is what happens:
x>0.4 returns a vector with 1 where true and 0 where false
x<0.6 returns a similar vector
The & combines the two vectors using an and
The find returns the indices of the 1's
36. Example: Avoiding Loops
• Given x= sin(linspace(0,10*pi,100)), how many of the
entries are positive?
Using a loop and if/else
count=0;
for n=1:length(x)
if x(n)>0
count=count+1;
end
end
Being more clever
count=length(find(x>0));
length(x) Loop time Find time
100 0.01 0
10,000 0.1 0
100,000 0.22 0
1,000,000 1.5 0.04
• Avoid loops!
• Built-in functions will make it faster to write and execute
37. Efficient Code
• Avoid loops
This is referred to as vectorization
• Vectorized code is more efficient for MATLAB
• Use indexing and matrix operations to avoid loops
• For example, to sum up every two consecutive terms:
» a=rand(1,100);
» b=zeros(1,100);
» for n=1:100
» if n==1
» b(n)=a(n);
» else
» b(n)=a(n-1)+a(n);
» end
» end
Slow and complicated
» a=rand(1,100);
» b=[0 a(1:end-1)]+a;
Efficient and clean.
Can also do this using
conv
38. End of Lecture 2
(1) Functions
(2) Flow Control
(3) Line Plots
(4) Image/Surface Plots
(5) Vectorization
Vectorization makes
coding fun!