SlideShare a Scribd company logo

Observability of the SCADA Systems Using Elastic APM, Reactive Streams and Asynchronous Communication, ICALEPCS+PyData meetup in Hamburg, October 2023

Igor Khokhryakov
Igor Khokhryakov
Igor KhokhryakovSoftware Architector at TANGO Controls Collaboration

As modern control systems grow in complexity, ensuring observability and traceability becomes more challenging. To meet this challenge, we present a novel solution that seamlessly integrates with multiple SCADA frameworks to provide end-to-end visibility into complex system interactions. Our solution utilizes Elastic APM to monitor and trace the performance of system components, allowing for real-time analysis and diagnosis of issues. In addition, our solution is built using reactive design principles and asynchronous communication, enabling it to scale to meet the demands of large, distributed systems. This paper will describe our approach and discuss how it can be applied to various use cases, including particle accelerators and other scientific facilities. We will also discuss the benefits of our solution, such as improved system observability and traceability, reduced downtime, and better resource allocation. We believe that our approach represents a significant step forward in the development of modern control systems, and we look forward to sharing our work with the community at ICALEPCS 2023.

Observability of the SCADA Systems Using Elastic APM, Reactive Streams and Asynchronous Communication, ICALEPCS+PyData meetup in Hamburg, October 2023

1 of 15
Download to read offline
Observability of the SCADA
Systems Using Elastic APM,
Reactive Streams and
Asynchronous Communication
Igor Khokhriakov, DESY
Olga Merkulova, DESY
Victoria Mazalova, CFEL
Alexander Nozik, JetBrains Research
11 Oct 2023 2
Who am I and why you should care?
●
15+ yrs in Software Development
●
10+ yrs in Scientific Software
Development
●
3+ yrs in Tango-Controls Kernel
development
●
1+ yrs in writing Tango RFC
●
500+ citations
●
Event-driven systems architect
●
Reactive programming advocate
11 Oct 2023 3
Goal
11 Oct 2023 4
First approach –
a dedicated micro-service
Axsis
Observer
https://github.com/waltz-controls/axsis-observer
●
Language & Design: Developed as a Java-
based microservice.
●
Purpose: Enhances the observability of
SCADA systems.
●
Operation:
●
Listens to various events and messages.
●
Operates on a designated channel.
●
Processes events and messages asynchronously.
●
Integration with Elastic APM:
●
Monitors and traces system components
●
Provides real-time performance insights.
11 Oct 2023 5
Second approach –
injected APM agents into
applications
-- APM agents
An APM (Application Performance
Management) agent is a software component
designed to monitor and collect data on the
performance of applications.
It tracks various metrics such as response
times, error rates, and transaction volumes.
11 Oct 2023 6
Java
●
Straightforward using Java JVM agent:
java -javaagent:/path/to/elastic-apm-agent.jar 
-Delastic.apm.service_name=my-application 
-Delastic.apm.application_packages=org.example 
-Delastic.apm.server_url=http://localhost:8200 
-jar your-application.jar
Ad

Recommended

Comparison of Current Service Mesh Architectures
Comparison of Current Service Mesh ArchitecturesComparison of Current Service Mesh Architectures
Comparison of Current Service Mesh ArchitecturesMirantis
 
OpenStack Neutron Havana Overview - Oct 2013
OpenStack Neutron Havana Overview - Oct 2013OpenStack Neutron Havana Overview - Oct 2013
OpenStack Neutron Havana Overview - Oct 2013Edgar Magana
 
Xpdays: Kubernetes CI-CD Frameworks Case Study
Xpdays: Kubernetes CI-CD Frameworks Case StudyXpdays: Kubernetes CI-CD Frameworks Case Study
Xpdays: Kubernetes CI-CD Frameworks Case StudyDenys Vasyliev
 
DEVNET-1166 Open SDN Controller APIs
DEVNET-1166	Open SDN Controller APIsDEVNET-1166	Open SDN Controller APIs
DEVNET-1166 Open SDN Controller APIsCisco DevNet
 
An Introduction to Eclipse Kura - Eclipse Day Florence 2014
An Introduction to Eclipse Kura - Eclipse Day Florence 2014An Introduction to Eclipse Kura - Eclipse Day Florence 2014
An Introduction to Eclipse Kura - Eclipse Day Florence 2014Eurotech
 
Triangle Devops Meetup 10/2015
Triangle Devops Meetup 10/2015Triangle Devops Meetup 10/2015
Triangle Devops Meetup 10/2015aspyker
 
Sumo Logic Cert Jam - Advanced Metrics with Kubernetes
Sumo Logic Cert Jam - Advanced Metrics with KubernetesSumo Logic Cert Jam - Advanced Metrics with Kubernetes
Sumo Logic Cert Jam - Advanced Metrics with KubernetesSumo Logic
 

More Related Content

Similar to Observability of the SCADA Systems Using Elastic APM, Reactive Streams and Asynchronous Communication, ICALEPCS+PyData meetup in Hamburg, October 2023

Service Meshes with Istio
Service Meshes with IstioService Meshes with Istio
Service Meshes with IstioRandyGupta
 
Accelerating SDN Applications with Open Source Network Overlays
Accelerating SDN Applications with Open Source Network OverlaysAccelerating SDN Applications with Open Source Network Overlays
Accelerating SDN Applications with Open Source Network OverlaysCumulus Networks
 
Tungsten Fabric Overview
Tungsten Fabric OverviewTungsten Fabric Overview
Tungsten Fabric OverviewMichelle Holley
 
Zou Layered VO PDCAT2008 V0.5 Concise
Zou Layered VO PDCAT2008 V0.5 ConciseZou Layered VO PDCAT2008 V0.5 Concise
Zou Layered VO PDCAT2008 V0.5 Conciseyongqiangzou
 
Edge Computing Platforms and Protocols - Ph.D. thesis
Edge Computing Platforms and Protocols - Ph.D. thesisEdge Computing Platforms and Protocols - Ph.D. thesis
Edge Computing Platforms and Protocols - Ph.D. thesisNitinder Mohan
 
TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6Sravanthi N
 
Kubernetes: Dive into the Future of Infrastructure
Kubernetes: Dive into the Future of InfrastructureKubernetes: Dive into the Future of Infrastructure
Kubernetes: Dive into the Future of InfrastructureGlobalLogic Ukraine
 
Overview of OpenDaylight Container Orchestration Engine Integration
Overview of OpenDaylight Container Orchestration Engine IntegrationOverview of OpenDaylight Container Orchestration Engine Integration
Overview of OpenDaylight Container Orchestration Engine IntegrationMichelle Holley
 
20141111_SOS3_Gallo
20141111_SOS3_Gallo20141111_SOS3_Gallo
20141111_SOS3_GalloAndrea Gallo
 
Comparison of existing cni plugins for kubernetes
Comparison of existing cni plugins for kubernetesComparison of existing cni plugins for kubernetes
Comparison of existing cni plugins for kubernetesAdam Hamsik
 
Open shift and docker - october,2014
Open shift and docker - october,2014Open shift and docker - october,2014
Open shift and docker - october,2014Hojoong Kim
 
Introduction to containers, k8s, Microservices & Cloud Native
Introduction to containers, k8s, Microservices & Cloud NativeIntroduction to containers, k8s, Microservices & Cloud Native
Introduction to containers, k8s, Microservices & Cloud NativeTerry Wang
 
SALSA: A Framework for Dynamic Configuration of Cloud Services
SALSA: A Framework for Dynamic Configuration of Cloud ServicesSALSA: A Framework for Dynamic Configuration of Cloud Services
SALSA: A Framework for Dynamic Configuration of Cloud ServicesDuc-Hung LE
 
Necos keynote ii_mobislice
Necos keynote ii_mobisliceNecos keynote ii_mobislice
Necos keynote ii_mobisliceAugusto Neto
 
All About Microservices and OpenSource Microservice Frameworks
All About Microservices and OpenSource Microservice FrameworksAll About Microservices and OpenSource Microservice Frameworks
All About Microservices and OpenSource Microservice FrameworksMohammad Asif Siddiqui
 
4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes
4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes
4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetesJuraj Hantak
 
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-orsCharacterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-orsLee Calcote
 

Similar to Observability of the SCADA Systems Using Elastic APM, Reactive Streams and Asynchronous Communication, ICALEPCS+PyData meetup in Hamburg, October 2023 (20)

Service Meshes with Istio
Service Meshes with IstioService Meshes with Istio
Service Meshes with Istio
 
Accelerating SDN Applications with Open Source Network Overlays
Accelerating SDN Applications with Open Source Network OverlaysAccelerating SDN Applications with Open Source Network Overlays
Accelerating SDN Applications with Open Source Network Overlays
 
Cisco project ideas
Cisco   project ideasCisco   project ideas
Cisco project ideas
 
Tungsten Fabric Overview
Tungsten Fabric OverviewTungsten Fabric Overview
Tungsten Fabric Overview
 
Zou Layered VO PDCAT2008 V0.5 Concise
Zou Layered VO PDCAT2008 V0.5 ConciseZou Layered VO PDCAT2008 V0.5 Concise
Zou Layered VO PDCAT2008 V0.5 Concise
 
Edge Computing Platforms and Protocols - Ph.D. thesis
Edge Computing Platforms and Protocols - Ph.D. thesisEdge Computing Platforms and Protocols - Ph.D. thesis
Edge Computing Platforms and Protocols - Ph.D. thesis
 
TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6TechTalk_Cloud Performance Testing_0.6
TechTalk_Cloud Performance Testing_0.6
 
Kubernetes: Dive into the Future of Infrastructure
Kubernetes: Dive into the Future of InfrastructureKubernetes: Dive into the Future of Infrastructure
Kubernetes: Dive into the Future of Infrastructure
 
01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf01-06 OCRE Test Suite - Fernandes.pdf
01-06 OCRE Test Suite - Fernandes.pdf
 
Overview of OpenDaylight Container Orchestration Engine Integration
Overview of OpenDaylight Container Orchestration Engine IntegrationOverview of OpenDaylight Container Orchestration Engine Integration
Overview of OpenDaylight Container Orchestration Engine Integration
 
20141111_SOS3_Gallo
20141111_SOS3_Gallo20141111_SOS3_Gallo
20141111_SOS3_Gallo
 
Comparison of existing cni plugins for kubernetes
Comparison of existing cni plugins for kubernetesComparison of existing cni plugins for kubernetes
Comparison of existing cni plugins for kubernetes
 
Open shift and docker - october,2014
Open shift and docker - october,2014Open shift and docker - october,2014
Open shift and docker - october,2014
 
Introduction to containers, k8s, Microservices & Cloud Native
Introduction to containers, k8s, Microservices & Cloud NativeIntroduction to containers, k8s, Microservices & Cloud Native
Introduction to containers, k8s, Microservices & Cloud Native
 
SALSA: A Framework for Dynamic Configuration of Cloud Services
SALSA: A Framework for Dynamic Configuration of Cloud ServicesSALSA: A Framework for Dynamic Configuration of Cloud Services
SALSA: A Framework for Dynamic Configuration of Cloud Services
 
One pk whitepaper
One pk whitepaperOne pk whitepaper
One pk whitepaper
 
Necos keynote ii_mobislice
Necos keynote ii_mobisliceNecos keynote ii_mobislice
Necos keynote ii_mobislice
 
All About Microservices and OpenSource Microservice Frameworks
All About Microservices and OpenSource Microservice FrameworksAll About Microservices and OpenSource Microservice Frameworks
All About Microservices and OpenSource Microservice Frameworks
 
4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes
4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes
4. CNCF kubernetes Comparison of-existing-cni-plugins-for-kubernetes
 
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-orsCharacterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
Characterizing and contrasting kuhn tey-ner awr-kuh-streyt-ors
 

More from Igor Khokhryakov

X-Environment Metadata Management
X-Environment Metadata ManagementX-Environment Metadata Management
X-Environment Metadata ManagementIgor Khokhryakov
 
Waltz-Controls presentation for Canadian Light Source
Waltz-Controls presentation for Canadian Light SourceWaltz-Controls presentation for Canadian Light Source
Waltz-Controls presentation for Canadian Light SourceIgor Khokhryakov
 
Waltz presentation at ASU and ANL
Waltz presentation at ASU and ANLWaltz presentation at ASU and ANL
Waltz presentation at ASU and ANLIgor Khokhryakov
 
1st Tango Users Group Russia meeting, overview
1st Tango Users Group Russia meeting, overview1st Tango Users Group Russia meeting, overview
1st Tango Users Group Russia meeting, overviewIgor Khokhryakov
 
Applying Enterprise technologies in Science
Applying Enterprise technologies in ScienceApplying Enterprise technologies in Science
Applying Enterprise technologies in ScienceIgor Khokhryakov
 

More from Igor Khokhryakov (7)

X-Environment Metadata Management
X-Environment Metadata ManagementX-Environment Metadata Management
X-Environment Metadata Management
 
Waltz-Controls presentation for Canadian Light Source
Waltz-Controls presentation for Canadian Light SourceWaltz-Controls presentation for Canadian Light Source
Waltz-Controls presentation for Canadian Light Source
 
Waltz presentation at ASU and ANL
Waltz presentation at ASU and ANLWaltz presentation at ASU and ANL
Waltz presentation at ASU and ANL
 
TangoWebapp insights
TangoWebapp insightsTangoWebapp insights
TangoWebapp insights
 
1st Tango Users Group Russia meeting, overview
1st Tango Users Group Russia meeting, overview1st Tango Users Group Russia meeting, overview
1st Tango Users Group Russia meeting, overview
 
GitHub/git+Cmake+Clion
GitHub/git+Cmake+ClionGitHub/git+Cmake+Clion
GitHub/git+Cmake+Clion
 
Applying Enterprise technologies in Science
Applying Enterprise technologies in ScienceApplying Enterprise technologies in Science
Applying Enterprise technologies in Science
 

Recently uploaded

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...confluent
 
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTSi-engage
 
Slide Deck - Milestone 9 alx mils .pptx
Slide Deck  - Milestone 9 alx mils .pptxSlide Deck  - Milestone 9 alx mils .pptx
Slide Deck - Milestone 9 alx mils .pptxYassineBissaoui1
 
Manual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12FxManual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12Fxjavierdavidvelasco17
 
owasp top 10 security risk categories and CWE
owasp top 10 security risk categories and CWEowasp top 10 security risk categories and CWE
owasp top 10 security risk categories and CWEArun Voleti
 
India's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdf
India's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdfIndia's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdf
India's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdfgranitesrijan
 
BotSE2022-Natarajan.pdf
BotSE2022-Natarajan.pdfBotSE2022-Natarajan.pdf
BotSE2022-Natarajan.pdfnatarajan8993
 
SATToSE_2023_Presentation_slideshare.pdf
SATToSE_2023_Presentation_slideshare.pdfSATToSE_2023_Presentation_slideshare.pdf
SATToSE_2023_Presentation_slideshare.pdfnatarajan8993
 
Get Your Hands Off the Teams Work.pdf
Get Your Hands Off the Teams Work.pdfGet Your Hands Off the Teams Work.pdf
Get Your Hands Off the Teams Work.pdfAngela Johnson
 
Steps to Build a PWA with Odoo.pdf
Steps to Build a PWA with Odoo.pdfSteps to Build a PWA with Odoo.pdf
Steps to Build a PWA with Odoo.pdfayushinwizards
 
unit 1 lecture 1 - Introduction - Software Engineering Myths.pdf
unit 1 lecture 1 - Introduction - Software Engineering Myths.pdfunit 1 lecture 1 - Introduction - Software Engineering Myths.pdf
unit 1 lecture 1 - Introduction - Software Engineering Myths.pdfStephenTec
 
unit I lecture 5 - Software Development Life Cycle.pdf
unit I lecture 5 - Software Development Life Cycle.pdfunit I lecture 5 - Software Development Life Cycle.pdf
unit I lecture 5 - Software Development Life Cycle.pdfStephenTec
 
Self scaling Multi cloud nomad workloads
Self scaling Multi cloud nomad workloadsSelf scaling Multi cloud nomad workloads
Self scaling Multi cloud nomad workloadsBram Vogelaar
 
Microsoft 365 De Security pdf
Microsoft 365 De Security pdfMicrosoft 365 De Security pdf
Microsoft 365 De Security pdfMarkus Moeller
 
MSR2022_Hackathon.pdf
MSR2022_Hackathon.pdfMSR2022_Hackathon.pdf
MSR2022_Hackathon.pdfnatarajan8993
 
unit I lecture 2 - Software Engineering Ethics - Software Process.pdf
unit I lecture 2 - Software Engineering Ethics - Software Process.pdfunit I lecture 2 - Software Engineering Ethics - Software Process.pdf
unit I lecture 2 - Software Engineering Ethics - Software Process.pdfStephenTec
 
Enabling Enterprise-wide OT Data access with Matrikon Data Broker.pdf
Enabling Enterprise-wide OT Data access  with Matrikon Data Broker.pdfEnabling Enterprise-wide OT Data access  with Matrikon Data Broker.pdf
Enabling Enterprise-wide OT Data access with Matrikon Data Broker.pdfJohn Archer
 
unit I lecture 3 - Software Process Models.pdf
unit I lecture 3 - Software Process Models.pdfunit I lecture 3 - Software Process Models.pdf
unit I lecture 3 - Software Process Models.pdfStephenTec
 

Recently uploaded (20)

Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
Industry 4.0: Building the Unified Namespace with Confluent, HiveMQ and Spark...
 
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
 
Slide Deck - Milestone 9 alx mils .pptx
Slide Deck  - Milestone 9 alx mils .pptxSlide Deck  - Milestone 9 alx mils .pptx
Slide Deck - Milestone 9 alx mils .pptx
 
Manual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12FxManual de la Mezcladora SoundCraft Notepad -12Fx
Manual de la Mezcladora SoundCraft Notepad -12Fx
 
owasp top 10 security risk categories and CWE
owasp top 10 security risk categories and CWEowasp top 10 security risk categories and CWE
owasp top 10 security risk categories and CWE
 
India's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdf
India's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdfIndia's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdf
India's_Generative_AI_Startup_Landscape_Report_2023_Inc42 (1).pdf
 
BotSE2022-Natarajan.pdf
BotSE2022-Natarajan.pdfBotSE2022-Natarajan.pdf
BotSE2022-Natarajan.pdf
 
SATToSE_2023_Presentation_slideshare.pdf
SATToSE_2023_Presentation_slideshare.pdfSATToSE_2023_Presentation_slideshare.pdf
SATToSE_2023_Presentation_slideshare.pdf
 
Get Your Hands Off the Teams Work.pdf
Get Your Hands Off the Teams Work.pdfGet Your Hands Off the Teams Work.pdf
Get Your Hands Off the Teams Work.pdf
 
Steps to Build a PWA with Odoo.pdf
Steps to Build a PWA with Odoo.pdfSteps to Build a PWA with Odoo.pdf
Steps to Build a PWA with Odoo.pdf
 
unit 1 lecture 1 - Introduction - Software Engineering Myths.pdf
unit 1 lecture 1 - Introduction - Software Engineering Myths.pdfunit 1 lecture 1 - Introduction - Software Engineering Myths.pdf
unit 1 lecture 1 - Introduction - Software Engineering Myths.pdf
 
unit I lecture 5 - Software Development Life Cycle.pdf
unit I lecture 5 - Software Development Life Cycle.pdfunit I lecture 5 - Software Development Life Cycle.pdf
unit I lecture 5 - Software Development Life Cycle.pdf
 
Self scaling Multi cloud nomad workloads
Self scaling Multi cloud nomad workloadsSelf scaling Multi cloud nomad workloads
Self scaling Multi cloud nomad workloads
 
Microsoft 365 De Security pdf
Microsoft 365 De Security pdfMicrosoft 365 De Security pdf
Microsoft 365 De Security pdf
 
MSR2022_Hackathon.pdf
MSR2022_Hackathon.pdfMSR2022_Hackathon.pdf
MSR2022_Hackathon.pdf
 
unit I lecture 2 - Software Engineering Ethics - Software Process.pdf
unit I lecture 2 - Software Engineering Ethics - Software Process.pdfunit I lecture 2 - Software Engineering Ethics - Software Process.pdf
unit I lecture 2 - Software Engineering Ethics - Software Process.pdf
 
Enabling Enterprise-wide OT Data access with Matrikon Data Broker.pdf
Enabling Enterprise-wide OT Data access  with Matrikon Data Broker.pdfEnabling Enterprise-wide OT Data access  with Matrikon Data Broker.pdf
Enabling Enterprise-wide OT Data access with Matrikon Data Broker.pdf
 
unit I lecture 3 - Software Process Models.pdf
unit I lecture 3 - Software Process Models.pdfunit I lecture 3 - Software Process Models.pdf
unit I lecture 3 - Software Process Models.pdf
 
Features of IETM Software -Code and Pixels
Features of IETM Software -Code and PixelsFeatures of IETM Software -Code and Pixels
Features of IETM Software -Code and Pixels
 
Importance Of Smaket In Your Buussiness
Importance Of Smaket In Your BuussinessImportance Of Smaket In Your Buussiness
Importance Of Smaket In Your Buussiness
 

Observability of the SCADA Systems Using Elastic APM, Reactive Streams and Asynchronous Communication, ICALEPCS+PyData meetup in Hamburg, October 2023

  • 1. Observability of the SCADA Systems Using Elastic APM, Reactive Streams and Asynchronous Communication Igor Khokhriakov, DESY Olga Merkulova, DESY Victoria Mazalova, CFEL Alexander Nozik, JetBrains Research
  • 2. 11 Oct 2023 2 Who am I and why you should care? ● 15+ yrs in Software Development ● 10+ yrs in Scientific Software Development ● 3+ yrs in Tango-Controls Kernel development ● 1+ yrs in writing Tango RFC ● 500+ citations ● Event-driven systems architect ● Reactive programming advocate
  • 3. 11 Oct 2023 3 Goal
  • 4. 11 Oct 2023 4 First approach – a dedicated micro-service Axsis Observer https://github.com/waltz-controls/axsis-observer ● Language & Design: Developed as a Java- based microservice. ● Purpose: Enhances the observability of SCADA systems. ● Operation: ● Listens to various events and messages. ● Operates on a designated channel. ● Processes events and messages asynchronously. ● Integration with Elastic APM: ● Monitors and traces system components ● Provides real-time performance insights.
  • 5. 11 Oct 2023 5 Second approach – injected APM agents into applications -- APM agents An APM (Application Performance Management) agent is a software component designed to monitor and collect data on the performance of applications. It tracks various metrics such as response times, error rates, and transaction volumes.
  • 6. 11 Oct 2023 6 Java ● Straightforward using Java JVM agent: java -javaagent:/path/to/elastic-apm-agent.jar -Delastic.apm.service_name=my-application -Delastic.apm.application_packages=org.example -Delastic.apm.server_url=http://localhost:8200 -jar your-application.jar
  • 7. 11 Oct 2023 7 Python import elasticapm kApmServerUrl = os.getenv('APM_SERVER_HOST', 'http://localhost:8200') kEnvironment = os.getenv('MODE', default='simulation') kApmClient = elasticapm.Client(service_name='axsis-magix', environment=kEnvironment, server_url=kApmServerUrl) elasticapm.instrument() https://github.com/waltz-controls/axsis-xes/blob/master/axsis.magix.py
  • 8. 11 Oct 2023 8 NodeJs and JS client ● NodeJs: ● JS: <script src="https://<your-cdn-host>.com/path/to/elastic-apm-rum.umd.min-<version>.js" crossorigin></script> <script> elasticApm.init({ serviceName: '<instrumented-app>', serverUrl: '<apm-server-url>', }) </script> import { init as initApm } from '@elastic/apm-rum' const apm = initApm({ // Set required service name (allowed characters: a-z, A-Z, 0-9, -, _, and space serviceName: '', // Set custom APM Server URL (default: http://localhost:8200) serverUrl: 'http://localhost:8200', // Set service version (required for sourcemap feature) serviceVersion: '' })
  • 9. 11 Oct 2023 9 Both have their pros and cons ● Dedicated micro-service easier to maintain – Not very suited for complex systems ● Embedded gives a better picture – Requires code ownership for non-JVM languages
  • 10. 11 Oct 2023 10 Context Khokhriakov, Merkulova, Mazalova, Nozik. “A novel solution for controlling hardware components of accelerators and beamlines” Volume 29 | Part 3 | May 2022 | | 10.1107/S1600577522002685
  • 11. 11 Oct 2023 11 Summary ● Observability and Application Performance Monitoring are essential for complex long running application ● Easy to achieve using plugins and 3rd party libraries ● Infrastructure requires competences and maintenance ● Tango-Controls moves towards integration with OpenTelemetry out of the box
  • 12. 11 Oct 2023 12 Acknowledgments *This work was supported by the consortium DAPHNE4NFDI in the context of the work of the NFDI e.V. The consortium is funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - project number 460248799
  • 13. 11 Oct 2023 13 “Harnessing the Power of Ubuntu and MicroK8s: Case Studies from San Diego Supercomputer Center and Center for Free Electron Lasers” by Igor Khokhriakov
  • 14. 11 Oct 2023 14 Join 2,5K+ followers on LinkedIn