Artificial intelligence (AI) has already been attracting the attention of deep tech investors for some years. The reasons why are clear. In its ‘Sizing The Prize’ analysis of artificial intelligence (AI), PwC forecast that AI will contribute $15.7 trillion to the global economy by 2030, with the ‘AI boost’ available to most national economies being approximately 26%. But what investors often overlook is that AI is not singular. Many individual components must work together to create AI. At its core artificial intelligence consists essentially of detecting statistical patterns in signals with many dimensions, such as analysis of audio frequencies (voice recognition) or high-resolution images (face recognition). The repetition of this search in order to detect these patterns is the basis of artificial intelligence. There are usually three components to AI: First, given a data set, learning what the patterns are. Second, building a model that can detect these patterns. Third, model deployment to the target environment. Traditionally, data mining or learning was done by experts in the matter who would develop some sort of classifier or detector based on certain features, and then try to see their correlations. This process was tedious and time consuming. https://klepsydra.com/cityam-ai-on-the-edge/

1. SMALLSAT 2021 PRESENTATION
DR PABLO GHIGLINO
pablo.ghiglino@klepsydra.com
www.klepsydra.com
A Low Power And High Performance Arti
fi
cial Intelligence
Approach To Increase Guidance Navigation And Control
Robustness

2. KLEPSYDRA AI IN ACTION
The demo:
• Pose estimation of 67P/
Churyumov–Gerasimenko
asteroid.
• Using an AI deep neural network
(DNN)
• Using real and synthetically
generated data from Rosetta
mission.
• Comparison of three AI inference
engines Klepsydra AI,
TensorFlowLite and OpenCV-
CNN
• Three identical computers,
running the same model with
the same input data and FPS.

3. KLEPSYDRA AI OVERVIEW
Klepsydra AI
Performance
analysis
Language
bindings
Trained Model
Basic features
Advanced features
Images Sensor Data
Timeseries

4. TRADING SOFTWARE VS EDGE SOFTWARE
Trading Systems
Edge Systems
• Bigger computer did not solve the
problem
• Can be solved using cutting-edge
lock-free programming techniques
• Top investment banks make billions
using these techniques.
• Very few developers have the required
skills
Computer
Usage
Low Medium
Data volume
Saturation

5. THE TECHNOLOGY
Klepsydra SDK
Sensors
External
Comms
Other Events
Application
Operating System
Patent pending technology
Klepsydra SDK
• 8x more real-time throughput
• 50% less CPU consumption
• No extra hardware or cloud

6. Event Loop
Sensor Multiplexer
Two main data
processing approaches
Producer 1
Consumer 1 Consumer 2
Producer 2
Producer 3
Consumer
Producer 1
6

7. Cobham GR716 Microcontroller
7
CPU vs Data processing rate 8 producers
CPU
(%) 25,00
43,75
62,50
81,25
100,00
Processing Rate (Hz)
0,00 1,25 2,50 3,75 5,00
Safe Queue Klepsydra
Traditional concurrent queue
Klepsydra’s Eventloop
Power consumption vs Data Processing
Power
(%)
10
33
55
78
100
Data processing rate (Hz)
0 10 20 30 40
Traditional
edge software
Klepsydra
Technical Spec:
• Processor: GR716
• OS: RTEMS 5
• Middleware: Memory data sharing
Benchmark Scenario:
• Multi-sensor data processing
• Concurrent Queue and Klepsydra’s processing
engine

8. APPROACHES TO CONCURRENT
ALGORITHMIC EXECUTION
Parallelisation Pipeline

9. BENCHMARK DESCRIPTION
Description
• Given an input matrix, a number of sequential multiplications will be
performed:
• Step 1: A => B = A x A => Step 2 : C = B x B…
• Matrix A randomly generated on each new sequence
Parameters:
• Matrix dimensions: 100x100
• Data type: Float, integer
• Number of multiplications per matrix: [10, 60]
• Processing frequency: [2Hz - 100Hz]
Technical Spec
• Computer: Odroid XU4
• OS: Ubuntu 18.04

10. TESTING SCENARIOS
Input
Matrix
B = A x A C = B x B
Output
Matrix
Input
Matrix B = A x A
Output
Matrix
C = B x B
Klepsydra Parallel Streaming Setup
OpenMP Sequential Setup
{
Thread 1
{
Thread 2
{
Vectorised
{
Vectorised

11. FLOAT PERFORMANCE RESULTS I
CPU Usage. 10 Steps
0,0
22,5
45,0
67,5
90,0
Publishing Rate (Hz)
2,00 26,50 51,00 75,50 100,00
OpenMp Klepsydra
Throughput. 10 Steps
0,00
25,00
50,00
75,00
100,00
Publishing Rate (Hz)
2,00 26,50 51,00 75,50 100,00
OpenMp Klepsydra
Latency. 10 Steps
0,00
12,50
25,00
37,50
50,00
Publishing Rate (Hz)
2,00 26,50 51,00 75,50 100,00
OpenMp Klepsydra
Throughput. 20 Steps
0,00
10,00
20,00
30,00
40,00
Publishing Rate (Hz)
2,00 11,50 21,00 30,50 40,00
OpenMp Klepsydra
Latency. 20 Steps
0,00
27,50
55,00
82,50
110,00
Publishing Rate (Hz)
2,00 11,50 21,00 30,50 40,00
OpenMp Klepsydra
CPU Usage. 20 Steps
0,0
22,5
45,0
67,5
90,0
Publishing Rate (Hz)
2,00 11,50 21,00 30,50 40,00
OpenMp Klepsydra

12. FLOAT PERFORMANCE RESULTS II
CPU Usage. 30 Steps
0,0
20,0
40,0
60,0
80,0
Publishing Rate (Hz)
2,00 6,50 11,00 15,50 20,00
OpenMp Klepsydra
Throughput. 30 Steps
0,00
5,00
10,00
15,00
20,00
Publishing Rate (Hz)
2,00 6,50 11,00 15,50 20,00
OpenMp Klepsydra
CPU Usage. 40 Steps
0,0
17,5
35,0
52,5
70,0
Publishing Rate (Hz)
2,00 5,00 8,00 11,00 14,00
OpenMp Klepsydra
Throughput. 40 Steps
0,00
3,50
7,00
10,50
14,00
Publishing Rate (Hz)
2,00 5,00 8,00 11,00 14,00
OpenMp Klepsydra
Latency. 40 Steps
0,00
60,00
120,00
180,00
240,00
Publishing Rate (Hz)
2,00 5,00 8,00 11,00 14,00
OpenMp Klepsydra
Latency. 30 Steps
0,00
45,00
90,00
135,00
180,00
Publishing Rate (Hz)
2,00 6,50 11,00 15,50 20,00
OpenMp Klepsydra

13. FLOAT PERFORMANCE RESULTS III
CPU Usage. 50 Steps
0,0
15,0
30,0
45,0
60,0
Publishing Rate (Hz)
2,00 4,00 6,00 8,00 10,00
OpenMp Klepsydra
Throughput. 50 Steps
0,00
2,75
5,50
8,25
11,00
Publishing Rate (Hz)
2,00 4,00 6,00 8,00 10,00
OpenMp Klepsydra
Latency. 50 Steps
0,00
100,00
200,00
300,00
400,00
Publishing Rate (Hz)
2,00 4,00 6,00 8,00 10,00
OpenMp Klepsydra
CPU Usage. 60 Steps
0,0
15,0
30,0
45,0
60,0
Publishing Rate (Hz)
2,00 3,50 5,00 6,50 8,00
OpenMp Klepsydra
Throughput. 60 Steps
0,00
2,00
4,00
6,00
8,00
Publishing Rate (Hz)
2,00 3,50 5,00 6,50 8,00
OpenMp Klepsydra
Latency. 60 Steps
0,00
225,00
450,00
675,00
900,00
Publishing Rate (Hz)
2,00 3,50 5,00 6,50 8,00
OpenMp Klepsydra

14. KLEPSYDRA AI DATA PROCESSING
APPROACH
Input
Data
Layer Layer
Output
Data
Klepsydra AI threading model
{
Thread 1
{
Thread 2
Threading model consists of:
- Number of cores assigned to event loops
- Number of event loops per core
- Number of parallelisation threads for each layer
Most layers can
be parallelised
and are
vectorised.
Eventloops are
assigned to
cores

15. Performance tuning
Performance Criteria
• CPU usage
• RAM usage
• Throughput (output data rate)
• Latency
15
Performance parameters:
• pool_size
Size of the internal queues of the event loop publish/
subscribe pairs.
High throughput requires large numbers, i.e., more RAM
usage, low throughout requires smaller number, therefore
less RAM.
Performance parameters
• number_of_cores
Number of cores where event loops will be distributed (by
default one event loop per core). High throughput requires
more cores, i.e., more CPU usage, low throughput requires
low number of cores, therefore substantial reduction in
CPU usage.
Performance parameters
• number_of_parallel_threads
Number of threads assigned to parallelise layers. For low
latency requirements, assign large numbers (maximum =
number of cores), i.e., increase CPU usage. For no latency
requirements, use low numbers (minimum = 1), therefore
substantial reduction in CPU usage.

16. 16
Example of performance benchmarks
TensorFlow Klepsydra AI
Latency: 56ms
Latency: 35ms

17. KLEPSYDRA AI IN ACTION
The demo:
• Pose estimation of 67P/
Churyumov–Gerasimenko
asteroid.
• Using an AI deep neural network
(DNN)
• Using real and synthetically
generated data from Rosetta
mission.
• Comparison of three AI inference
engines Klepsydra AI,
TensorFlowLite and OpenCV-
CNN
• Three identical computers,
running the same model with
the same input data and FPS.

18. ROADMAP
Q2 2021
• No third party dependencies.
• Binaries are C/C++ only
• Custom format for models
Q3 2021
• FreeRTOS support (alpha version)
• Xilinx Ultrascale+ board
• Microchip SAM V71
Q4 2021
• PykeOS support (alpha version)
• Xilinx Zedboard
Q1 2022
• NVIDIA Jetson TX2 Support (alpha
release)
• Quantisation support
Q2 2022
• Graphs support
• Memory allocation new model
• C support
Legend:
Hard deadlines
Flexible dates

19. CONCLUSIONS
• The use of advanced lock-free algorithms for on-board data
processing allows a substantial increase in real-time data
throughput and a 50% reduction in power consumption.
• When combined with pipelining, it can enable ground
breaking performance improvement in AI algorithms.
• Further work will be done in the
fi
eld of GPU and FPGA, self-
tuning and graph AI models.

20. CONTACT INFORMATION
Dr Pablo Ghiglino
pablo.ghiglino@klepsydra.com
+41786931544
www.klepsydra.com
linkedin.com/company/klepsydra-technologies