SlideShare a Scribd company logo
1 of 25
Download to read offline
MONITORING
DISTRIBUTED HPC
DO YUO FNID TIHS SMILPE TO RAED ?
Q1 Q2 Q3 Q4
X Y X Y X Y X Y
10.0 8.04 10.0 9.14 10.0 7.46 8.0 6.58
8.0 6.95 8.0 8.14 8.0 6.77 8.0 5.76
13.0 7.58 13.0 8.74 13.0 12.74 8.0 7.71
9.0 8.81 9.0 8.77 9.0 7.11 8.0 8.84
11.0 8.33 11.0 9.26 11.0 7.81 8.0 8.47
14.0 9.96 14.0 8.10 14.0 8.84 8.0 7.04
6.0 7.24 6.0 6.13 6.0 6.08 8.0 5.25
4.0 4.26 4.0 3.10 4.0 5.39 19.0 12.50
12.0 10.84 12.0 9.13 12.0 8.15 8.0 5.56
7.0 4.82 7.0 7.26 7.0 6.42 8.0 7.91
5.0 5.68 5.0 4.74 5.0 5.73 8.0 6.89
Frank Anscombe

1918 - 2001
Standard
Expected
ANSCOMBE'S QUARTET
Q1 Q2 Q3 Q4
X Y X Y X Y X Y
Mean 9 7.50 9 7.50 9 7.50 9 7.50
Var 11 4.125 11 4.125 11 4.125 11 4.125
Cor 0.816 0.816 0.816 0.816
Σ(^2) 13.7627 13.7763 13.7562 13.7425
LinReg y = 0.5x + 3 y = 0.5x + 3 y = 0.5x + 3 y = 0.5x + 3
Coefficient 0.67 0.67 0.67 0.67
Intercept 3.00009 ± 0.909545 3.00091 ± 0.909545 3.00245 ± 0.909545 3.00173 ± 0.909545
Slope 0.500091 ± 0.0953463 0.500000 ± 0.0953463 0.499727 ± 0.0953463 0.499909 ± 0.0953463
0
11
0 14
y = 0.5001x + 3.0001
R² = 0.6665
Q1
0
11
0 14
y = 0.5x + 3.0009
R² = 0.6662
Q2
0
13
0 14
y = 0.4997x + 3.0025
R² = 0.6663
Q3
0
13
0 20
y = 0.4999x + 3.0017
R² = 0.6667
Q4
GRAPH != KPI
t0
BATCH PROCESSING TIMELINE
t0 t10
BATCH PROCESSING TIMELINE
Metrics

Collection
t10 t45
Metrics

Parsing
BATCH PROCESSING TIMELINE
Metrics

Collection
t0
t10 t45
Action
t60
Metrics

Parsing
BATCH PROCESSING TIMELINE
Metrics

Collection
t0
t0
STREAM PROCESSING TIMELINE
t7
STREAM PROCESSING TIMELINE
Metrics

Parsing
t0
t7 t15
Action
STREAM PROCESSING TIMELINE
Metrics

Parsing
t0
#1 INFRASTRUCTURE MATTERS
#2 NO TIME TO CONSULT HQ
A
B
C
#3 NO WAY TO CONSULT HQ
SETUP
NUDNIK - INFRASTRUCTURE HEARTBEAT
▸ https://pypi.org/project/nudnik/
▸ pip install nudnik
▸ https://aur.archlinux.org/packages/
nudnik/
▸ pacman -S nudnik
▸ git clone

https://github.com/salosh/nudnik
NUDNIK
MASSAGE TYPES - BASELINE
A
B
C
D
E
20ms
10ms
20ms
10ms
10ms
20ms
10ms
10ms
▸Tiny fingerprint
▸GRPC / REST
▸Multiplatform
NUDNIK
MASSAGE TYPES - LOAD
A
B
C
D
E
10ms
10ms
10ms
10ms
10ms
10ms
20ms
20ms
▸CPU
▸Memory
▸Disk
▸Network
▸Executable
NUDNIK
MASSAGE TYPES - CHAOS
A
B
C
D
E10%
10%
+? ms
? %
▸Set failure %
▸Set latency
▸.. Or randomise
12ms
NUDNIK
REPORTING
▸InfluxDB
▸ElasticSearch
▸Prometheus PG
▸Text csv / TTY
AB
TS
C
A
MULTILAYER DISTRIBUTION
B
C
Node A Node B
A
SCALE PLANNING / TESTING
B C
RuOK
Distributed HPC monitoring

More Related Content

Similar to Distributed HPC monitoring

(597922903) trabajo sobre aguas residuales 1
(597922903) trabajo sobre aguas residuales 1(597922903) trabajo sobre aguas residuales 1
(597922903) trabajo sobre aguas residuales 1Juliana Hincapié Rojo
 
Tarea 2 hidraulica iii-cabrera arias roberto alejandro
Tarea 2 hidraulica iii-cabrera arias roberto alejandroTarea 2 hidraulica iii-cabrera arias roberto alejandro
Tarea 2 hidraulica iii-cabrera arias roberto alejandroAlejandro Cabrera
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfbin971209zhou
 
Volvo EC55B Compact Excavator Service Repair Manual
Volvo EC55B Compact Excavator Service Repair ManualVolvo EC55B Compact Excavator Service Repair Manual
Volvo EC55B Compact Excavator Service Repair Manualfujdfjjskrtekme
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdffujsekddmdmdm
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdffyhsejkdm8u
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfttf99929781
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdff8usejkdmdd8i
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdffjskemdmmded
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdffujsekmdd9dik
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdffujsekmd9idd1
 
Tablas normal chi cuadrado y t student 1-semana 6
Tablas normal chi cuadrado y t student 1-semana 6Tablas normal chi cuadrado y t student 1-semana 6
Tablas normal chi cuadrado y t student 1-semana 6Karla Diaz
 
Econometrics project mcom and mphill
Econometrics project  mcom and mphillEconometrics project  mcom and mphill
Econometrics project mcom and mphilljunaidsuri
 
Peramalan Data Time Series #2
Peramalan Data Time Series #2Peramalan Data Time Series #2
Peramalan Data Time Series #2Adhitya Akbar
 
Rigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería Estructural
Rigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería EstructuralRigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería Estructural
Rigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería EstructuralIRMA Trejo Amado
 

Similar to Distributed HPC monitoring (20)

Regression project
Regression projectRegression project
Regression project
 
(597922903) trabajo sobre aguas residuales 1
(597922903) trabajo sobre aguas residuales 1(597922903) trabajo sobre aguas residuales 1
(597922903) trabajo sobre aguas residuales 1
 
Tarea 2 hidraulica iii-cabrera arias roberto alejandro
Tarea 2 hidraulica iii-cabrera arias roberto alejandroTarea 2 hidraulica iii-cabrera arias roberto alejandro
Tarea 2 hidraulica iii-cabrera arias roberto alejandro
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual
Volvo EC55B Compact Excavator Service Repair ManualVolvo EC55B Compact Excavator Service Repair Manual
Volvo EC55B Compact Excavator Service Repair Manual
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Volvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdfVolvo EC55B Compact Excavator Service Repair Manual.pdf
Volvo EC55B Compact Excavator Service Repair Manual.pdf
 
Metode nakayasu
Metode nakayasuMetode nakayasu
Metode nakayasu
 
30 x 11
30 x 1130 x 11
30 x 11
 
Tablas normal chi cuadrado y t student 1-semana 6
Tablas normal chi cuadrado y t student 1-semana 6Tablas normal chi cuadrado y t student 1-semana 6
Tablas normal chi cuadrado y t student 1-semana 6
 
Econometrics project mcom and mphill
Econometrics project  mcom and mphillEconometrics project  mcom and mphill
Econometrics project mcom and mphill
 
Peramalan Data Time Series #2
Peramalan Data Time Series #2Peramalan Data Time Series #2
Peramalan Data Time Series #2
 
BASIC STATISTIC
BASIC STATISTICBASIC STATISTIC
BASIC STATISTIC
 
GARCH
GARCHGARCH
GARCH
 
Rigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería Estructural
Rigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería EstructuralRigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería Estructural
Rigidez de Dos Pórticos para Dos Niveles de Piso - Albañilería Estructural
 

More from Salo Shp

Outside The Wire
Outside The WireOutside The Wire
Outside The WireSalo Shp
 
Scaling IO-bound microservices
Scaling IO-bound microservicesScaling IO-bound microservices
Scaling IO-bound microservicesSalo Shp
 
Scaling the #2ndhalf
Scaling the #2ndhalfScaling the #2ndhalf
Scaling the #2ndhalfSalo Shp
 
Infrastructure Fuzzing
Infrastructure FuzzingInfrastructure Fuzzing
Infrastructure FuzzingSalo Shp
 
Cyber Oriented Engineering
Cyber Oriented EngineeringCyber Oriented Engineering
Cyber Oriented EngineeringSalo Shp
 
Remote secured storage
Remote secured storageRemote secured storage
Remote secured storageSalo Shp
 
Pluggable Monitoring
Pluggable MonitoringPluggable Monitoring
Pluggable MonitoringSalo Shp
 

More from Salo Shp (7)

Outside The Wire
Outside The WireOutside The Wire
Outside The Wire
 
Scaling IO-bound microservices
Scaling IO-bound microservicesScaling IO-bound microservices
Scaling IO-bound microservices
 
Scaling the #2ndhalf
Scaling the #2ndhalfScaling the #2ndhalf
Scaling the #2ndhalf
 
Infrastructure Fuzzing
Infrastructure FuzzingInfrastructure Fuzzing
Infrastructure Fuzzing
 
Cyber Oriented Engineering
Cyber Oriented EngineeringCyber Oriented Engineering
Cyber Oriented Engineering
 
Remote secured storage
Remote secured storageRemote secured storage
Remote secured storage
 
Pluggable Monitoring
Pluggable MonitoringPluggable Monitoring
Pluggable Monitoring
 

Recently uploaded

Prach: A Feature-Rich Platform Empowering the Autism Community
Prach: A Feature-Rich Platform Empowering the Autism CommunityPrach: A Feature-Rich Platform Empowering the Autism Community
Prach: A Feature-Rich Platform Empowering the Autism Communityprachaibot
 
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...IJAEMSJORNAL
 
KCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosKCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosVictor Morales
 
A brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision ProA brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision ProRay Yuan Liu
 
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.elesangwon
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
Plastifloor Park Deck Waterproofing System_Flyer
Plastifloor Park Deck Waterproofing System_FlyerPlastifloor Park Deck Waterproofing System_Flyer
Plastifloor Park Deck Waterproofing System_FlyerPlasti-Chemie GmbH
 
Pyrolysis process control: temperature control design and application for opt...
Pyrolysis process control: temperature control design and application for opt...Pyrolysis process control: temperature control design and application for opt...
Pyrolysis process control: temperature control design and application for opt...IJECEIAES
 
Javier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptxJavier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptxJavier Fernández Muñoz
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionSneha Padhiar
 
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdfLivre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdfsaad175691
 
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxTriangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
priority interrupt computer organization
priority interrupt computer organizationpriority interrupt computer organization
priority interrupt computer organizationchnrketan
 
Defining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxDefining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxAshwiniTodkar4
 
LM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbmsLM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbmsBalaKrish12
 
Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...
Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...
Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...Amil baba
 
Design and Analysis of Algorithms Lecture Notes
Design and Analysis of Algorithms Lecture NotesDesign and Analysis of Algorithms Lecture Notes
Design and Analysis of Algorithms Lecture NotesSreedhar Chowdam
 

Recently uploaded (20)

Prach: A Feature-Rich Platform Empowering the Autism Community
Prach: A Feature-Rich Platform Empowering the Autism CommunityPrach: A Feature-Rich Platform Empowering the Autism Community
Prach: A Feature-Rich Platform Empowering the Autism Community
 
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
Guardians of E-Commerce: Harnessing NLP and Machine Learning Approaches for A...
 
KCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitosKCD Costa Rica 2024 - Nephio para parvulitos
KCD Costa Rica 2024 - Nephio para parvulitos
 
A brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision ProA brief look at visionOS - How to develop app on Apple's Vision Pro
A brief look at visionOS - How to develop app on Apple's Vision Pro
 
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
2022 AWS DNA Hackathon 장애 대응 솔루션 jarvis.
 
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptxCurve setting (Basic Mine Surveying)_MI10412MI.pptx
Curve setting (Basic Mine Surveying)_MI10412MI.pptx
 
Plastifloor Park Deck Waterproofing System_Flyer
Plastifloor Park Deck Waterproofing System_FlyerPlastifloor Park Deck Waterproofing System_Flyer
Plastifloor Park Deck Waterproofing System_Flyer
 
Pyrolysis process control: temperature control design and application for opt...
Pyrolysis process control: temperature control design and application for opt...Pyrolysis process control: temperature control design and application for opt...
Pyrolysis process control: temperature control design and application for opt...
 
Javier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptxJavier_Fernandez_CARS_workshop_presentation.pptx
Javier_Fernandez_CARS_workshop_presentation.pptx
 
Cost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based questionCost estimation approach: FP to COCOMO scenario based question
Cost estimation approach: FP to COCOMO scenario based question
 
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdfLivre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
Livre Implementing_Six_Sigma_and_Lean_A_prac([Ron_Basu]_).pdf
 
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxTriangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
 
Versatile Engineering Construction Firms
Versatile Engineering Construction FirmsVersatile Engineering Construction Firms
Versatile Engineering Construction Firms
 
priority interrupt computer organization
priority interrupt computer organizationpriority interrupt computer organization
priority interrupt computer organization
 
Defining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptxDefining the Clouds for entriprises.pptx
Defining the Clouds for entriprises.pptx
 
Neometrix Optical Balloon Theodolite.pptx
Neometrix Optical Balloon Theodolite.pptxNeometrix Optical Balloon Theodolite.pptx
Neometrix Optical Balloon Theodolite.pptx
 
LM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbmsLM7_ Embedded Sql and Dynamic SQL in dbms
LM7_ Embedded Sql and Dynamic SQL in dbms
 
Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...
Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...
Uk-NO1 Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Exp...
 
ASME-B31.4-2019-estandar para diseño de ductos
ASME-B31.4-2019-estandar para diseño de ductosASME-B31.4-2019-estandar para diseño de ductos
ASME-B31.4-2019-estandar para diseño de ductos
 
Design and Analysis of Algorithms Lecture Notes
Design and Analysis of Algorithms Lecture NotesDesign and Analysis of Algorithms Lecture Notes
Design and Analysis of Algorithms Lecture Notes
 

Distributed HPC monitoring