SlideShare a Scribd company logo
Real time Linux programming
Mark Veltzer
mailto:mark@veltzer.net
Definitions
What is “real time”?
● From wikipedia: “real-time computing (RTC), or reactive
computing, is the study of hardware and software systems that
are subject to a "real-time constraint"— e.g. operational
deadlines from event to system response. Real-time programs
must guarantee response within strict time constraints.[1] Often
real-time response times are understood to be in the order of
milliseconds and sometimes microseconds. In contrast, a non-
real-time system is one that cannot guarantee a response time
in any situation, even if a fast response is the usual result.”
“Real time” hardware
● A hardware may guarantee real time
performance by utilizing a “hardware clock”
which is a crystal which delivers a pulse at a
constant rate.
● The engineers will make sure that the hardware
concludes it's business before the clock strikes
again.
● What you get is a guarantee for how long it
takes you to do a certain operation.
Terms
● “Hard real time” means a piece of software that must meet
time requirements and never miss even a single dead line.
● “Soft real time” is the same but for a system that will
continue to perform even if the deadlines are missed.
● “High performance” is a piece of software that has to
utilize the hardware to the maximum but where each
operation is not constrained.
● A piece of software may fall into more than one category
(think chess playing software).
Designing real time software
●
There are methodologies designed specifically for the design of
real time software such as MASCOT, HOOD, AADL, Ravenscar,
profile and others.
●
These methodologies deal with how you design your system:
decide on numbers of threads or process, the roles of each
process, the priorities of the threads and more.
● They also deal with how to debug a multi-threaded piece of
software that must meat the strict deadlines.
●
We will not discuss them further.
Real time and programming
language
● Real time systems may be written in any
language (including Java and Python contrary
to public opinion).
● Because care is needed to disable non-
deterministic aspects of the programming
language (e.g. garbage collector) then
programmers prefer more deterministic
languages like C and C++.
● No, C++ is not less deterministic than C.
Traditional real time operating
systems
● In traditional real time operating systems the
programmers write threads which are part of the
operating system kernel (operate in the same
space).
● These systems are kept simple by design so as
not to be non deterministic.
● These systems are fast and deterministic.
● They are also very hard to debug once the
software grows because of the single space.
Where latency/non determinism
come from?
● Hardware. Examples can be self monitoring
interrupts, Caches.
● Operating system. Examples can be operating
system threads, swap, long interrupt handlers,
page faults, laziness.
● Programming language. Examples can be
malloc/free(3), garbage collectors.
● The actual software. Examples can be “if”
statements, algorithm bugs, locking.
The art of real time programming
● All this means that to create a real time
system one must master all levels of the
system: from the hardware, through the
operating system, through the programming
language to the software.
● That is also why hard real time programming
is hard.
Linux and real time
Linux and real time
●
Linux was never designed from the beginning to be a real time system and
some would claim that UNIX philosophy is contrary to real time.
● But: Linux has been developing real time capabilities for over a decade.
●
The kernel patch has been written by Ingo Molnar and Thomas Gleixner.
● It is almost fully integrated into the mainline kernel (3.6).
● Is expected to be fully integrated in the next year or so.
●
Is usually applied as a patch to the kernel by the people building the real
time OS for you.
●
See https://rt.wiki.kernel.org/index.php/Main_Page
What does real time Linux give me?
● Interrupts are handled by the kernel in threads with a given priority.
● The ability to run threads at OS interrupt handler priority or higher.
● Hard Real time scheduling - separate from the soft scheduling that user space usually
uses.
● Priority inversion.
● Fine granularity of CPU affinity of user space and operating system threads.
● New real time clock API for better precision.
● Less interrupts for real time threads – the scheduler does not use regular tick
interrupts to check if real time threads need to stop running and be replaced by other
threads.
● Low latency of the kernel (kernel was reviewed and locations where interrupts were
disabled were improved)
● What is the worst time latency? Around 30usec for dedicated systems. More if you put
more drivers, less if you use less.
Page faults and their cost
● In a Linux system going to the kernel is
expensive (1usec region).
● The system is lazy by default and so uses the
MMU to fetch pages it has promised but has not
attached to the process.
● This happens in fetching pages for the
executable, data segment, stack, libraries,
mmap(2) and malloc(3) (because of mmap(2)).
● See examples.
Real time programming on Linux
Monitoring page faults
● The proc filesystem: /proc/[pid]/stat
● From the command line: ps -o min_flt
● Link with libproc.so and use it's API to monitor
your own page faults.
● Use top(1) and configure page faults as one of
the fields you see.
Avoiding page faults
● Stack: allocate a big buffer on the stack and
zero it or call a recursive function a lot.
● Malloc: zero the memory when you get it.
● Mmap: zero the memory or pass a flag to
mmap.
● Data segment: zero the memory.
● For everything including all of the above there
is also mlock(2).
Avoiding latency
● If you want a really clean CPU all to yourself:
– Put your thread on a CPU that does not get
hardware interrupts.
– Do this using the pthread affinity API.
– On embedded systems hardware interrupts
usually go to CPU 0.
– Use cat /proc/interrupts to make sure.
– Tune where interrupts go using echo to /proc/irq/
[irq_number]/smp_affinity
Avoiding latency (cont)
● On larger systems (Redhat,...) you usually have
a /usr/sbin/irqbalance distributing the interrupts
evenly between the cores.
● Either get rid of it (don't run it on boot in
/etc/init.d).
● Or configure it not to send interrupts to the CPU
you want to keep clean (use
IRQBALANCE_BANNED_CPUS in
/etc/default/irqbalance).
Measuring the entire systems
latency
● Is done using cyclictest(1) which part of the rt-
tests package.
● If you don't have it in your system vendors
repository then install it from source
http://git.kernel.org/cgit/linux/kernel/git/clrkwllms/rt-t
● It contains not only cyclicttest(1) but other very
useful tools.

More Related Content

What's hot

Real time operating systems (rtos) concepts 3
Real time operating systems (rtos) concepts 3Real time operating systems (rtos) concepts 3
Real time operating systems (rtos) concepts 3
Abu Bakr Ramadan
 
An End to Order
An End to OrderAn End to Order
An End to Order
Robert Burrell Donkin
 
An End to Order (many cores with java, session two)
An End to Order (many cores with java, session two)An End to Order (many cores with java, session two)
An End to Order (many cores with java, session two)
Robert Burrell Donkin
 
Real time operating systems (rtos) concepts 9
Real time operating systems (rtos) concepts 9Real time operating systems (rtos) concepts 9
Real time operating systems (rtos) concepts 9
Abu Bakr Ramadan
 
RTDroid_Presentation
RTDroid_PresentationRTDroid_Presentation
RTDroid_Presentation
Aswin Bharadwaj
 
RTAI - Earliest Deadline First
RTAI - Earliest Deadline FirstRTAI - Earliest Deadline First
RTAI - Earliest Deadline First
Stefano Bragaglia
 
RTOS- Real Time Operating Systems
RTOS- Real Time Operating Systems RTOS- Real Time Operating Systems
RTOS- Real Time Operating Systems
Bayar shahab
 
REAL TIME OPERATING SYSTEM
REAL TIME OPERATING SYSTEMREAL TIME OPERATING SYSTEM
REAL TIME OPERATING SYSTEM
prakrutijsh
 
Real time operating system
Real time operating systemReal time operating system
Real time operating system
Khuram Shahzad
 
Real time scheduling - basic concepts
Real time scheduling - basic conceptsReal time scheduling - basic concepts
Real time scheduling - basic concepts
Student
 
Windows process-scheduling
Windows process-schedulingWindows process-scheduling
Windows process-scheduling
Talha Shaikh
 
INTERRUPT LATENCY AND RESPONSE OF THE TASK
INTERRUPT LATENCY AND RESPONSE OF THE TASKINTERRUPT LATENCY AND RESPONSE OF THE TASK
INTERRUPT LATENCY AND RESPONSE OF THE TASK
JOLLUSUDARSHANREDDY
 
RTOS
RTOSRTOS
RT linux
RT linuxRT linux
RT linux
SARITHA REDDY
 
Real Time Kernels
Real Time KernelsReal Time Kernels
Real Time Kernels
Arnav Soni
 
Real time system tsp
Real time system tspReal time system tsp
Real time system tsp
Pradeep Kumar TS
 
Luca Abeni - Real-Time Virtual Machines with Linux and kvm
Luca Abeni - Real-Time Virtual Machines with Linux and kvmLuca Abeni - Real-Time Virtual Machines with Linux and kvm
Luca Abeni - Real-Time Virtual Machines with Linux and kvm
linuxlab_conf
 
Basic functions & types of RTOS ES
Basic functions & types of  RTOS ESBasic functions & types of  RTOS ES
Basic functions & types of RTOS ES
JOLLUSUDARSHANREDDY
 
Real time-embedded-system-lec-02
Real time-embedded-system-lec-02Real time-embedded-system-lec-02
Real time-embedded-system-lec-02
University of Computer Science and Technology
 
Windows process scheduling presentation
Windows process scheduling presentationWindows process scheduling presentation
Windows process scheduling presentation
Talha Shaikh
 

What's hot (20)

Real time operating systems (rtos) concepts 3
Real time operating systems (rtos) concepts 3Real time operating systems (rtos) concepts 3
Real time operating systems (rtos) concepts 3
 
An End to Order
An End to OrderAn End to Order
An End to Order
 
An End to Order (many cores with java, session two)
An End to Order (many cores with java, session two)An End to Order (many cores with java, session two)
An End to Order (many cores with java, session two)
 
Real time operating systems (rtos) concepts 9
Real time operating systems (rtos) concepts 9Real time operating systems (rtos) concepts 9
Real time operating systems (rtos) concepts 9
 
RTDroid_Presentation
RTDroid_PresentationRTDroid_Presentation
RTDroid_Presentation
 
RTAI - Earliest Deadline First
RTAI - Earliest Deadline FirstRTAI - Earliest Deadline First
RTAI - Earliest Deadline First
 
RTOS- Real Time Operating Systems
RTOS- Real Time Operating Systems RTOS- Real Time Operating Systems
RTOS- Real Time Operating Systems
 
REAL TIME OPERATING SYSTEM
REAL TIME OPERATING SYSTEMREAL TIME OPERATING SYSTEM
REAL TIME OPERATING SYSTEM
 
Real time operating system
Real time operating systemReal time operating system
Real time operating system
 
Real time scheduling - basic concepts
Real time scheduling - basic conceptsReal time scheduling - basic concepts
Real time scheduling - basic concepts
 
Windows process-scheduling
Windows process-schedulingWindows process-scheduling
Windows process-scheduling
 
INTERRUPT LATENCY AND RESPONSE OF THE TASK
INTERRUPT LATENCY AND RESPONSE OF THE TASKINTERRUPT LATENCY AND RESPONSE OF THE TASK
INTERRUPT LATENCY AND RESPONSE OF THE TASK
 
RTOS
RTOSRTOS
RTOS
 
RT linux
RT linuxRT linux
RT linux
 
Real Time Kernels
Real Time KernelsReal Time Kernels
Real Time Kernels
 
Real time system tsp
Real time system tspReal time system tsp
Real time system tsp
 
Luca Abeni - Real-Time Virtual Machines with Linux and kvm
Luca Abeni - Real-Time Virtual Machines with Linux and kvmLuca Abeni - Real-Time Virtual Machines with Linux and kvm
Luca Abeni - Real-Time Virtual Machines with Linux and kvm
 
Basic functions & types of RTOS ES
Basic functions & types of  RTOS ESBasic functions & types of  RTOS ES
Basic functions & types of RTOS ES
 
Real time-embedded-system-lec-02
Real time-embedded-system-lec-02Real time-embedded-system-lec-02
Real time-embedded-system-lec-02
 
Windows process scheduling presentation
Windows process scheduling presentationWindows process scheduling presentation
Windows process scheduling presentation
 

Viewers also liked

Gcc
GccGcc
Volatile
VolatileVolatile
Volatile
Mark Veltzer
 
Gcc opt
Gcc optGcc opt
Gcc opt
Mark Veltzer
 
Effective cplusplus
Effective cplusplusEffective cplusplus
Effective cplusplus
Mark Veltzer
 
Linux logging
Linux loggingLinux logging
Linux logging
Mark Veltzer
 
Streams
StreamsStreams
Streams
Mark Veltzer
 
Multicore
MulticoreMulticore
Multicore
Mark Veltzer
 

Viewers also liked (7)

Gcc
GccGcc
Gcc
 
Volatile
VolatileVolatile
Volatile
 
Gcc opt
Gcc optGcc opt
Gcc opt
 
Effective cplusplus
Effective cplusplusEffective cplusplus
Effective cplusplus
 
Linux logging
Linux loggingLinux logging
Linux logging
 
Streams
StreamsStreams
Streams
 
Multicore
MulticoreMulticore
Multicore
 

Similar to Realtime

Testing real-time Linux. What to test and how
Testing real-time Linux. What to test and how Testing real-time Linux. What to test and how
Testing real-time Linux. What to test and how
Chirag Jog
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
National Cheng Kung University
 
FreeRTOS introduction
FreeRTOS introductionFreeRTOS introduction
FreeRTOS introduction
Jegadeesh Vontlin
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
National Cheng Kung University
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Quantum Leaps, LLC
 
Rtos by shibu
Rtos by shibuRtos by shibu
Rtos by shibu
Shibu Krishnan
 
Værktøjer udviklet på AAU til analyse af SCJ programmer
Værktøjer udviklet på AAU til analyse af SCJ programmerVærktøjer udviklet på AAU til analyse af SCJ programmer
Værktøjer udviklet på AAU til analyse af SCJ programmer
InfinIT - Innovationsnetværket for it
 
Real time system basic concept
Real time system basic conceptReal time system basic concept
Real time system basic concept
MOUMITA GHOSH
 
rtosbyshibu-131026100746-phpapp01.pdf
rtosbyshibu-131026100746-phpapp01.pdfrtosbyshibu-131026100746-phpapp01.pdf
rtosbyshibu-131026100746-phpapp01.pdf
reemasajin1
 
Chapter 19 - Real Time Systems
Chapter 19 - Real Time SystemsChapter 19 - Real Time Systems
Chapter 19 - Real Time Systems
Wayne Jones Jnr
 
Real Time Operating system (RTOS) - Embedded systems
Real Time Operating system (RTOS) - Embedded systemsReal Time Operating system (RTOS) - Embedded systems
Real Time Operating system (RTOS) - Embedded systems
Hariharan Ganesan
 
Lab3F22.pdf
Lab3F22.pdfLab3F22.pdf
Lab3F22.pdf
ssuser0e9cc4
 
Os Concepts
Os ConceptsOs Concepts
Os Concepts
rajesh panda
 
intro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptxintro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptx
ssuser413a98
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Miro Samek
 
Lecture1
Lecture1Lecture1
Lecture1
tt_aljobory
 
Rtos 8051
Rtos 8051Rtos 8051
Autosar Basics hand book_v1
Autosar Basics  hand book_v1Autosar Basics  hand book_v1
Autosar Basics hand book_v1
Keroles karam khalil
 
Concept of thread
Concept of threadConcept of thread
Concept of thread
Munmun Das Bhowmik
 
Four Ways to Improve Linux Performance IEEE Webinar, R2.0
Four Ways to Improve Linux Performance IEEE Webinar, R2.0Four Ways to Improve Linux Performance IEEE Webinar, R2.0
Four Ways to Improve Linux Performance IEEE Webinar, R2.0
Michael Christofferson
 

Similar to Realtime (20)

Testing real-time Linux. What to test and how
Testing real-time Linux. What to test and how Testing real-time Linux. What to test and how
Testing real-time Linux. What to test and how
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
 
FreeRTOS introduction
FreeRTOS introductionFreeRTOS introduction
FreeRTOS introduction
 
Making Linux do Hard Real-time
Making Linux do Hard Real-timeMaking Linux do Hard Real-time
Making Linux do Hard Real-time
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
 
Rtos by shibu
Rtos by shibuRtos by shibu
Rtos by shibu
 
Værktøjer udviklet på AAU til analyse af SCJ programmer
Værktøjer udviklet på AAU til analyse af SCJ programmerVærktøjer udviklet på AAU til analyse af SCJ programmer
Værktøjer udviklet på AAU til analyse af SCJ programmer
 
Real time system basic concept
Real time system basic conceptReal time system basic concept
Real time system basic concept
 
rtosbyshibu-131026100746-phpapp01.pdf
rtosbyshibu-131026100746-phpapp01.pdfrtosbyshibu-131026100746-phpapp01.pdf
rtosbyshibu-131026100746-phpapp01.pdf
 
Chapter 19 - Real Time Systems
Chapter 19 - Real Time SystemsChapter 19 - Real Time Systems
Chapter 19 - Real Time Systems
 
Real Time Operating system (RTOS) - Embedded systems
Real Time Operating system (RTOS) - Embedded systemsReal Time Operating system (RTOS) - Embedded systems
Real Time Operating system (RTOS) - Embedded systems
 
Lab3F22.pdf
Lab3F22.pdfLab3F22.pdf
Lab3F22.pdf
 
Os Concepts
Os ConceptsOs Concepts
Os Concepts
 
intro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptxintro, definitions, basic laws+.pptx
intro, definitions, basic laws+.pptx
 
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded SoftwareBeyond the RTOS: A Better Way to Design Real-Time Embedded Software
Beyond the RTOS: A Better Way to Design Real-Time Embedded Software
 
Lecture1
Lecture1Lecture1
Lecture1
 
Rtos 8051
Rtos 8051Rtos 8051
Rtos 8051
 
Autosar Basics hand book_v1
Autosar Basics  hand book_v1Autosar Basics  hand book_v1
Autosar Basics hand book_v1
 
Concept of thread
Concept of threadConcept of thread
Concept of thread
 
Four Ways to Improve Linux Performance IEEE Webinar, R2.0
Four Ways to Improve Linux Performance IEEE Webinar, R2.0Four Ways to Improve Linux Performance IEEE Webinar, R2.0
Four Ways to Improve Linux Performance IEEE Webinar, R2.0
 

Recently uploaded

The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
digitalxplive
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
Anant Gupta
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Zilliz
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
Tatiana Al-Chueyr
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Nicolás Lopéz
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
HackersList
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
Priyanka Aash
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
Steven Carlson
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
aakash malhotra
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
huseindihon
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
SynapseIndia
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
Ivanti
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
Edge AI and Vision Alliance
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
313mohammedarshad
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
Axel Rennoch
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
Safe Software
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
Neo4j
 

Recently uploaded (20)

The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
The Rise of AI in Cybersecurity How Machine Learning Will Shape Threat Detect...
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes..."Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
"Mastering Graphic Design: Essential Tips and Tricks for Beginners and Profes...
 
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and OllamaTirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
Tirana Tech Meetup - Agentic RAG with Milvus, Llama3 and Ollama
 
Best Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdfBest Practices for Effectively Running dbt in Airflow.pdf
Best Practices for Effectively Running dbt in Airflow.pdf
 
Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024Vertex AI Agent Builder - GDG Alicante - Julio 2024
Vertex AI Agent Builder - GDG Alicante - Julio 2024
 
WhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring AppsWhatsApp Spy Online Trackers and Monitoring Apps
WhatsApp Spy Online Trackers and Monitoring Apps
 
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
(CISOPlatform Summit & SACON 2024) Keynote _ Power Digital Identities With AI...
 
Vulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive OverviewVulnerability Management: A Comprehensive Overview
Vulnerability Management: A Comprehensive Overview
 
Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024Three New Criminal Laws in India 1 July 2024
Three New Criminal Laws in India 1 July 2024
 
find out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challengesfind out more about the role of autonomous vehicles in facing global challenges
find out more about the role of autonomous vehicles in facing global challenges
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxRPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptx
 
July Patch Tuesday
July Patch TuesdayJuly Patch Tuesday
July Patch Tuesday
 
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
“Deploying Large Language Models on a Raspberry Pi,” a Presentation from Usef...
 
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptxIntroduction-to-the-IAM-Platform-Implementation-Plan.pptx
Introduction-to-the-IAM-Platform-Implementation-Plan.pptx
 
The importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT StandardizationThe importance of Quality Assurance for ICT Standardization
The importance of Quality Assurance for ICT Standardization
 
Data Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining DataData Integration Basics: Merging & Joining Data
Data Integration Basics: Merging & Joining Data
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfBT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdf
 

Realtime

  • 1. Real time Linux programming Mark Veltzer mailto:mark@veltzer.net
  • 3. What is “real time”? ● From wikipedia: “real-time computing (RTC), or reactive computing, is the study of hardware and software systems that are subject to a "real-time constraint"— e.g. operational deadlines from event to system response. Real-time programs must guarantee response within strict time constraints.[1] Often real-time response times are understood to be in the order of milliseconds and sometimes microseconds. In contrast, a non- real-time system is one that cannot guarantee a response time in any situation, even if a fast response is the usual result.”
  • 4. “Real time” hardware ● A hardware may guarantee real time performance by utilizing a “hardware clock” which is a crystal which delivers a pulse at a constant rate. ● The engineers will make sure that the hardware concludes it's business before the clock strikes again. ● What you get is a guarantee for how long it takes you to do a certain operation.
  • 5. Terms ● “Hard real time” means a piece of software that must meet time requirements and never miss even a single dead line. ● “Soft real time” is the same but for a system that will continue to perform even if the deadlines are missed. ● “High performance” is a piece of software that has to utilize the hardware to the maximum but where each operation is not constrained. ● A piece of software may fall into more than one category (think chess playing software).
  • 6. Designing real time software ● There are methodologies designed specifically for the design of real time software such as MASCOT, HOOD, AADL, Ravenscar, profile and others. ● These methodologies deal with how you design your system: decide on numbers of threads or process, the roles of each process, the priorities of the threads and more. ● They also deal with how to debug a multi-threaded piece of software that must meat the strict deadlines. ● We will not discuss them further.
  • 7. Real time and programming language ● Real time systems may be written in any language (including Java and Python contrary to public opinion). ● Because care is needed to disable non- deterministic aspects of the programming language (e.g. garbage collector) then programmers prefer more deterministic languages like C and C++. ● No, C++ is not less deterministic than C.
  • 8. Traditional real time operating systems ● In traditional real time operating systems the programmers write threads which are part of the operating system kernel (operate in the same space). ● These systems are kept simple by design so as not to be non deterministic. ● These systems are fast and deterministic. ● They are also very hard to debug once the software grows because of the single space.
  • 9. Where latency/non determinism come from? ● Hardware. Examples can be self monitoring interrupts, Caches. ● Operating system. Examples can be operating system threads, swap, long interrupt handlers, page faults, laziness. ● Programming language. Examples can be malloc/free(3), garbage collectors. ● The actual software. Examples can be “if” statements, algorithm bugs, locking.
  • 10. The art of real time programming ● All this means that to create a real time system one must master all levels of the system: from the hardware, through the operating system, through the programming language to the software. ● That is also why hard real time programming is hard.
  • 12. Linux and real time ● Linux was never designed from the beginning to be a real time system and some would claim that UNIX philosophy is contrary to real time. ● But: Linux has been developing real time capabilities for over a decade. ● The kernel patch has been written by Ingo Molnar and Thomas Gleixner. ● It is almost fully integrated into the mainline kernel (3.6). ● Is expected to be fully integrated in the next year or so. ● Is usually applied as a patch to the kernel by the people building the real time OS for you. ● See https://rt.wiki.kernel.org/index.php/Main_Page
  • 13. What does real time Linux give me? ● Interrupts are handled by the kernel in threads with a given priority. ● The ability to run threads at OS interrupt handler priority or higher. ● Hard Real time scheduling - separate from the soft scheduling that user space usually uses. ● Priority inversion. ● Fine granularity of CPU affinity of user space and operating system threads. ● New real time clock API for better precision. ● Less interrupts for real time threads – the scheduler does not use regular tick interrupts to check if real time threads need to stop running and be replaced by other threads. ● Low latency of the kernel (kernel was reviewed and locations where interrupts were disabled were improved) ● What is the worst time latency? Around 30usec for dedicated systems. More if you put more drivers, less if you use less.
  • 14. Page faults and their cost ● In a Linux system going to the kernel is expensive (1usec region). ● The system is lazy by default and so uses the MMU to fetch pages it has promised but has not attached to the process. ● This happens in fetching pages for the executable, data segment, stack, libraries, mmap(2) and malloc(3) (because of mmap(2)). ● See examples.
  • 16. Monitoring page faults ● The proc filesystem: /proc/[pid]/stat ● From the command line: ps -o min_flt ● Link with libproc.so and use it's API to monitor your own page faults. ● Use top(1) and configure page faults as one of the fields you see.
  • 17. Avoiding page faults ● Stack: allocate a big buffer on the stack and zero it or call a recursive function a lot. ● Malloc: zero the memory when you get it. ● Mmap: zero the memory or pass a flag to mmap. ● Data segment: zero the memory. ● For everything including all of the above there is also mlock(2).
  • 18. Avoiding latency ● If you want a really clean CPU all to yourself: – Put your thread on a CPU that does not get hardware interrupts. – Do this using the pthread affinity API. – On embedded systems hardware interrupts usually go to CPU 0. – Use cat /proc/interrupts to make sure. – Tune where interrupts go using echo to /proc/irq/ [irq_number]/smp_affinity
  • 19. Avoiding latency (cont) ● On larger systems (Redhat,...) you usually have a /usr/sbin/irqbalance distributing the interrupts evenly between the cores. ● Either get rid of it (don't run it on boot in /etc/init.d). ● Or configure it not to send interrupts to the CPU you want to keep clean (use IRQBALANCE_BANNED_CPUS in /etc/default/irqbalance).
  • 20. Measuring the entire systems latency ● Is done using cyclictest(1) which part of the rt- tests package. ● If you don't have it in your system vendors repository then install it from source http://git.kernel.org/cgit/linux/kernel/git/clrkwllms/rt-t ● It contains not only cyclicttest(1) but other very useful tools.