SlideShare a Scribd company logo
1 of 21
Clarizen adventures with the wild GC
during the holiday season
@tamir_dresher
Chief Architect @ Clarizen
2
Chief Architect @ @tamir_dresher
Tamir Dresher
My Books:
Software Engineering Lecturer
Ruppin Academic Center
tamir.dresher@clarizen.com
CLARIZEN - ENTERPRISE WORK COLLABORATION
I N T E R N A L & E X T E R N A L C O L L A B O R AT I O N
A U T O M AT E D W O R K F L O W E N G I N E
REPORTS & DASHBOARDS
MOBILE APPS & BOTS
APPSMARKETPLACE
INTEGRATIONS
RESOURCE
MANAGEMENT
TIME &
EXPENSES
FINANCIAL
PLANNING
DEMAND
MANAGEMENT
PROJECT
MANAGEMENT
PORTFOLIO
MANAGEMENT
ENTERPRISES FROM
124 COUNTRIES
99.95%
UPTIME
ENTERPRISE GRADE
SaaS SOLUTION
FOUNDED IN
2006
4
5
Loading…
Loading…
Loading…
Loading…
Loading… Loading…
Loading…
Loading…
Loading…
Loading…
6
Web App
Refresh
Dashboard
Job
/Dashboard/{dashboardId}
7
https://www.elastic.co/products/kibana
8
https://www.logicmonitor.com/
9
https://devblogs.microsoft.com/dotnet/gc-performance-counters/
10
11
12
.NET Garbage Collector (GC)
12
Gen 0 Gen 1 Gen 2
Gen 0 is full, start garbage collection!New objects are allocated in Gen 0Dead objects are foundStill living objects are promoted
Large object heap
13
INVESTIGATION
Disabling Background GC
14
https://github.com/microsoft/perfview
15
Long PauseGen 0 GC
https://github.com/microsoft/perfview
16
17
GC Threads 1..11 (all the same stack trace):
0:002> k
# Child-SP RetAddr Call Site
00 00000000`00bbf478 000007fe`fd8c10ac ntdll!ZwWaitForSingleObject+0xa
01 00000000`00bbf480 000007fe`f9232c77 KERNELBASE!WaitForSingleObjectEx+0x79
02 00000000`00bbf520 000007fe`f9232c2f clr!CLREventWaitHelper2+0x3c
03 00000000`00bbf560 000007fe`f9232bb0 clr!CLREventWaitHelper+0x1f
04 00000000`00bbf5c0 000007fe`f939cae5 clr!CLREventBase::WaitEx+0x7c
05 00000000`00bbf650 000007fe`f939d891 clr!SVR::t_join::join+0x10f
06 00000000`00bbf6b0 000007fe`f939ce57 clr!SVR::gc_heap::mark_phase+0x385
07 00000000`00bbf750 000007fe`f939d0c6 clr!SVR::gc_heap::gc1+0xa7
08 00000000`00bbf7a0 000007fe`f939c854 clr!SVR::gc_heap::garbage_collect+0x870
...
GC Thread 12:
0:013> k
# Child-SP RetAddr Call Site
00 00000000`0c17ebc8 000007fe`fd8c10ac ntdll!ZwWaitForSingleObject+0xa
01 00000000`0c17ebd0 000007fe`f9232c77 KERNELBASE!WaitForSingleObjectEx+0x79
02 00000000`0c17ec70 000007fe`f9232c2f clr!CLREventWaitHelper2+0x3c
03 00000000`0c17ecb0 000007fe`f9232bb0 clr!CLREventWaitHelper+0x1f
04 00000000`0c17ed10 000007fe`f939cae5 clr!CLREventBase::WaitEx+0x7c
05 00000000`0c17eda0 000007fe`f939df75 clr!SVR::t_join::join+0x10f
06 00000000`0c17ee00 000007fe`f939d879 clr!SVR::gc_heap::scan_dependent_handles+0x65
07 00000000`0c17ee30 000007fe`f939ce57 clr!SVR::gc_heap::mark_phase+0x36d
08 00000000`0c17eed0 000007fe`f939d0c6 clr!SVR::gc_heap::gc1+0xa7
09 00000000`0c17ef20 000007fe`f939c854 clr!SVR::gc_heap::garbage_collect+0x870
usually caused by managed heap corruption
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools
18
0:000> .exr -1
ExceptionAddress: 00000000778e7b9f
(ntdll!RtlpLowFragHeapAllocFromContext+0x000000000000013f)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 0000000000000001
Parameter[1]: 0000000000160fe0
Attempt to write to address 0000000000160fe0
0:000> k
# Child-SP RetAddr Call Site
00 00000000`00160fb0 00000000`778e7974
ntdll!RtlpLowFragHeapAllocFromContext+0x13f
01 00000000`00161090 000007fe`fdc7ca20 ntdll!RtlAllocateHeap+0xe4
02 00000000`001611a0 00000000`00000002 SomeAntivirusDLL+0x6ca20
03 00000000`001611a8 00000000`00000208 0x2
04 00000000`001611b0 01d4c79a`ffff0000 0x208
05 00000000`001611b8 00000000`00328a10 0x01d4c79a`ffff0000
06 00000000`001611c0 00000000`00000208 0x328a10
07 00000000`001611c8 000007fe`fdc6c2c1 0x208
08 00000000`001611d0 00000000`00000008 SomeAntivirusDLL+0x5c2c1
09 00000000`001611d8 000007fe`fddd8816 0x8
...
https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools
19
Disabling antivirus software
20
Conclusion
• Invest in monitoring –
Logs, application metrics, infrastructure metrics.
• Don’t be too proud to hire an expert.
• Invest in your developers diagnostic skills.
• Sometimes you can really blame the anti-virus.
¯_(ツ)_/¯
21
Thank you!
@tamir_dresher
Chief Architect @ Clarizen

More Related Content

Similar to Clarizen GC adventures during the holidays

TMPA-2017: Defect Report Classification in Accordance with Areas of Testing
TMPA-2017: Defect Report Classification in Accordance with Areas of TestingTMPA-2017: Defect Report Classification in Accordance with Areas of Testing
TMPA-2017: Defect Report Classification in Accordance with Areas of TestingIosif Itkin
 
Deploy your Multi-tier Application in Cloud Foundry
Deploy your Multi-tier Application in Cloud FoundryDeploy your Multi-tier Application in Cloud Foundry
Deploy your Multi-tier Application in Cloud Foundrycornelia davis
 
Using Cluster Analysis for Characteristics Detection in Software Defect Reports
Using Cluster Analysis for Characteristics Detection in Software Defect ReportsUsing Cluster Analysis for Characteristics Detection in Software Defect Reports
Using Cluster Analysis for Characteristics Detection in Software Defect ReportsIosif Itkin
 
Star bed 2018.07.19
Star bed 2018.07.19Star bed 2018.07.19
Star bed 2018.07.19Ruo Ando
 
AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018
AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018
AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018Andy Davies
 
FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
FØCAL Boston AiR - Computer Vision Tracing and Hardware SimulationFØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
FØCAL Boston AiR - Computer Vision Tracing and Hardware SimulationFØCAL
 
Lazy Join Optimizations Without Upfront Statistics with Matteo Interlandi
Lazy Join Optimizations Without Upfront Statistics with Matteo InterlandiLazy Join Optimizations Without Upfront Statistics with Matteo Interlandi
Lazy Join Optimizations Without Upfront Statistics with Matteo InterlandiDatabricks
 
Velocity NY 2016 - Devops: Who Does What?
Velocity NY 2016 - Devops: Who Does What?Velocity NY 2016 - Devops: Who Does What?
Velocity NY 2016 - Devops: Who Does What?cornelia davis
 
PAC 2020 Santorin - Andreas Grabner
PAC 2020 Santorin - Andreas Grabner PAC 2020 Santorin - Andreas Grabner
PAC 2020 Santorin - Andreas Grabner Neotys
 
The bare minimum that you should know about web application security testing ...
The bare minimum that you should know about web application security testing ...The bare minimum that you should know about web application security testing ...
The bare minimum that you should know about web application security testing ...Ken DeSouza
 
From Software Engineering To Machine Learning
From Software Engineering To Machine LearningFrom Software Engineering To Machine Learning
From Software Engineering To Machine LearningAlexey Grigorev
 
C programming 28 program
C programming 28 programC programming 28 program
C programming 28 programF Courses
 
Overcoming (organizational) scalability issues in your Prometheus ecosystem
Overcoming (organizational) scalability issues in your Prometheus ecosystemOvercoming (organizational) scalability issues in your Prometheus ecosystem
Overcoming (organizational) scalability issues in your Prometheus ecosystemQAware GmbH
 
OSDC 2017 | Modern Secrets Management with Vault by Seth Vargo
OSDC 2017 | Modern Secrets Management with Vault by Seth VargoOSDC 2017 | Modern Secrets Management with Vault by Seth Vargo
OSDC 2017 | Modern Secrets Management with Vault by Seth VargoNETWAYS
 
OSDC 2017 - Seth Vargo - Taming the modern data center
OSDC 2017 - Seth Vargo - Taming the modern data centerOSDC 2017 - Seth Vargo - Taming the modern data center
OSDC 2017 - Seth Vargo - Taming the modern data centerNETWAYS
 
OSDC 2017 | Taming the Modern Data Center by Seth Vargo
OSDC 2017 | Taming the Modern Data Center by Seth VargoOSDC 2017 | Taming the Modern Data Center by Seth Vargo
OSDC 2017 | Taming the Modern Data Center by Seth VargoNETWAYS
 
UKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tipsUKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tipsConnor McDonald
 

Similar to Clarizen GC adventures during the holidays (20)

TMPA-2017: Defect Report Classification in Accordance with Areas of Testing
TMPA-2017: Defect Report Classification in Accordance with Areas of TestingTMPA-2017: Defect Report Classification in Accordance with Areas of Testing
TMPA-2017: Defect Report Classification in Accordance with Areas of Testing
 
Deploy your Multi-tier Application in Cloud Foundry
Deploy your Multi-tier Application in Cloud FoundryDeploy your Multi-tier Application in Cloud Foundry
Deploy your Multi-tier Application in Cloud Foundry
 
Using Cluster Analysis for Characteristics Detection in Software Defect Reports
Using Cluster Analysis for Characteristics Detection in Software Defect ReportsUsing Cluster Analysis for Characteristics Detection in Software Defect Reports
Using Cluster Analysis for Characteristics Detection in Software Defect Reports
 
Star bed 2018.07.19
Star bed 2018.07.19Star bed 2018.07.19
Star bed 2018.07.19
 
AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018
AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018
AB Testing, Ads and other 3rd party tags - SmashingConf London - 2018
 
FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
FØCAL Boston AiR - Computer Vision Tracing and Hardware SimulationFØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
FØCAL Boston AiR - Computer Vision Tracing and Hardware Simulation
 
Lazy Join Optimizations Without Upfront Statistics with Matteo Interlandi
Lazy Join Optimizations Without Upfront Statistics with Matteo InterlandiLazy Join Optimizations Without Upfront Statistics with Matteo Interlandi
Lazy Join Optimizations Without Upfront Statistics with Matteo Interlandi
 
Velocity NY 2016 - Devops: Who Does What?
Velocity NY 2016 - Devops: Who Does What?Velocity NY 2016 - Devops: Who Does What?
Velocity NY 2016 - Devops: Who Does What?
 
URL to HTML
URL to HTMLURL to HTML
URL to HTML
 
PAC 2020 Santorin - Andreas Grabner
PAC 2020 Santorin - Andreas Grabner PAC 2020 Santorin - Andreas Grabner
PAC 2020 Santorin - Andreas Grabner
 
The bare minimum that you should know about web application security testing ...
The bare minimum that you should know about web application security testing ...The bare minimum that you should know about web application security testing ...
The bare minimum that you should know about web application security testing ...
 
From Software Engineering To Machine Learning
From Software Engineering To Machine LearningFrom Software Engineering To Machine Learning
From Software Engineering To Machine Learning
 
C programming 28 program
C programming 28 programC programming 28 program
C programming 28 program
 
Performance Risk Management
Performance Risk ManagementPerformance Risk Management
Performance Risk Management
 
Overcoming (organizational) scalability issues in your Prometheus ecosystem
Overcoming (organizational) scalability issues in your Prometheus ecosystemOvercoming (organizational) scalability issues in your Prometheus ecosystem
Overcoming (organizational) scalability issues in your Prometheus ecosystem
 
OSDC 2017 | Modern Secrets Management with Vault by Seth Vargo
OSDC 2017 | Modern Secrets Management with Vault by Seth VargoOSDC 2017 | Modern Secrets Management with Vault by Seth Vargo
OSDC 2017 | Modern Secrets Management with Vault by Seth Vargo
 
OSDC 2017 - Seth Vargo - Taming the modern data center
OSDC 2017 - Seth Vargo - Taming the modern data centerOSDC 2017 - Seth Vargo - Taming the modern data center
OSDC 2017 - Seth Vargo - Taming the modern data center
 
OSDC 2017 | Taming the Modern Data Center by Seth Vargo
OSDC 2017 | Taming the Modern Data Center by Seth VargoOSDC 2017 | Taming the Modern Data Center by Seth Vargo
OSDC 2017 | Taming the Modern Data Center by Seth Vargo
 
UKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tipsUKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tips
 
Paris data ladies #6
Paris data ladies #6Paris data ladies #6
Paris data ladies #6
 

More from Tamir Dresher

NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdfNET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdfTamir Dresher
 
Tamir Dresher - DotNet 7 What's new.pptx
Tamir Dresher - DotNet 7 What's new.pptxTamir Dresher - DotNet 7 What's new.pptx
Tamir Dresher - DotNet 7 What's new.pptxTamir Dresher
 
Tamir Dresher - What’s new in ASP.NET Core 6
Tamir Dresher - What’s new in ASP.NET Core 6Tamir Dresher - What’s new in ASP.NET Core 6
Tamir Dresher - What’s new in ASP.NET Core 6Tamir Dresher
 
Tamir Dresher - Async Streams in C#
Tamir Dresher - Async Streams in C#Tamir Dresher - Async Streams in C#
Tamir Dresher - Async Streams in C#Tamir Dresher
 
Anatomy of a data driven architecture - Tamir Dresher
Anatomy of a data driven architecture - Tamir Dresher   Anatomy of a data driven architecture - Tamir Dresher
Anatomy of a data driven architecture - Tamir Dresher Tamir Dresher
 
Debugging tricks you wish you knew Tamir Dresher - Odessa 2019
Debugging tricks you wish you knew   Tamir Dresher - Odessa 2019Debugging tricks you wish you knew   Tamir Dresher - Odessa 2019
Debugging tricks you wish you knew Tamir Dresher - Odessa 2019Tamir Dresher
 
From zero to hero with the actor model - Tamir Dresher - Odessa 2019
From zero to hero with the actor model  - Tamir Dresher - Odessa 2019From zero to hero with the actor model  - Tamir Dresher - Odessa 2019
From zero to hero with the actor model - Tamir Dresher - Odessa 2019Tamir Dresher
 
Tamir Dresher - Demystifying the Core of .NET Core
Tamir Dresher  - Demystifying the Core of .NET CoreTamir Dresher  - Demystifying the Core of .NET Core
Tamir Dresher - Demystifying the Core of .NET CoreTamir Dresher
 
Breaking the monolith to microservice with Docker and Kubernetes (k8s)
Breaking the monolith to microservice with Docker and Kubernetes (k8s)Breaking the monolith to microservice with Docker and Kubernetes (k8s)
Breaking the monolith to microservice with Docker and Kubernetes (k8s)Tamir Dresher
 
.Net december 2017 updates - Tamir Dresher
.Net december 2017 updates - Tamir Dresher.Net december 2017 updates - Tamir Dresher
.Net december 2017 updates - Tamir DresherTamir Dresher
 
Testing time and concurrency Rx
Testing time and concurrency RxTesting time and concurrency Rx
Testing time and concurrency RxTamir Dresher
 
Building responsive application with Rx - confoo - tamir dresher
Building responsive application with Rx - confoo - tamir dresherBuilding responsive application with Rx - confoo - tamir dresher
Building responsive application with Rx - confoo - tamir dresherTamir Dresher
 
.NET Debugging tricks you wish you knew tamir dresher
.NET Debugging tricks you wish you knew   tamir dresher.NET Debugging tricks you wish you knew   tamir dresher
.NET Debugging tricks you wish you knew tamir dresherTamir Dresher
 
From Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir Dresher
From Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir DresherFrom Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir Dresher
From Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir DresherTamir Dresher
 
Building responsive applications with Rx - CodeMash2017 - Tamir Dresher
Building responsive applications with Rx  - CodeMash2017 - Tamir DresherBuilding responsive applications with Rx  - CodeMash2017 - Tamir Dresher
Building responsive applications with Rx - CodeMash2017 - Tamir DresherTamir Dresher
 
Debugging tricks you wish you knew - Tamir Dresher
Debugging tricks you wish you knew  - Tamir DresherDebugging tricks you wish you knew  - Tamir Dresher
Debugging tricks you wish you knew - Tamir DresherTamir Dresher
 
Rx 101 - Tamir Dresher - Copenhagen .NET User Group
Rx 101  - Tamir Dresher - Copenhagen .NET User GroupRx 101  - Tamir Dresher - Copenhagen .NET User Group
Rx 101 - Tamir Dresher - Copenhagen .NET User GroupTamir Dresher
 
Cloud patterns - NDC Oslo 2016 - Tamir Dresher
Cloud patterns - NDC Oslo 2016 - Tamir DresherCloud patterns - NDC Oslo 2016 - Tamir Dresher
Cloud patterns - NDC Oslo 2016 - Tamir DresherTamir Dresher
 
Reactiveness All The Way - SW Architecture 2015 Conference
Reactiveness All The Way - SW Architecture 2015 ConferenceReactiveness All The Way - SW Architecture 2015 Conference
Reactiveness All The Way - SW Architecture 2015 ConferenceTamir Dresher
 
Rx 101 Codemotion Milan 2015 - Tamir Dresher
Rx 101   Codemotion Milan 2015 - Tamir DresherRx 101   Codemotion Milan 2015 - Tamir Dresher
Rx 101 Codemotion Milan 2015 - Tamir DresherTamir Dresher
 

More from Tamir Dresher (20)

NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdfNET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
NET Aspire - NET Conf IL 2024 - Tamir Dresher.pdf
 
Tamir Dresher - DotNet 7 What's new.pptx
Tamir Dresher - DotNet 7 What's new.pptxTamir Dresher - DotNet 7 What's new.pptx
Tamir Dresher - DotNet 7 What's new.pptx
 
Tamir Dresher - What’s new in ASP.NET Core 6
Tamir Dresher - What’s new in ASP.NET Core 6Tamir Dresher - What’s new in ASP.NET Core 6
Tamir Dresher - What’s new in ASP.NET Core 6
 
Tamir Dresher - Async Streams in C#
Tamir Dresher - Async Streams in C#Tamir Dresher - Async Streams in C#
Tamir Dresher - Async Streams in C#
 
Anatomy of a data driven architecture - Tamir Dresher
Anatomy of a data driven architecture - Tamir Dresher   Anatomy of a data driven architecture - Tamir Dresher
Anatomy of a data driven architecture - Tamir Dresher
 
Debugging tricks you wish you knew Tamir Dresher - Odessa 2019
Debugging tricks you wish you knew   Tamir Dresher - Odessa 2019Debugging tricks you wish you knew   Tamir Dresher - Odessa 2019
Debugging tricks you wish you knew Tamir Dresher - Odessa 2019
 
From zero to hero with the actor model - Tamir Dresher - Odessa 2019
From zero to hero with the actor model  - Tamir Dresher - Odessa 2019From zero to hero with the actor model  - Tamir Dresher - Odessa 2019
From zero to hero with the actor model - Tamir Dresher - Odessa 2019
 
Tamir Dresher - Demystifying the Core of .NET Core
Tamir Dresher  - Demystifying the Core of .NET CoreTamir Dresher  - Demystifying the Core of .NET Core
Tamir Dresher - Demystifying the Core of .NET Core
 
Breaking the monolith to microservice with Docker and Kubernetes (k8s)
Breaking the monolith to microservice with Docker and Kubernetes (k8s)Breaking the monolith to microservice with Docker and Kubernetes (k8s)
Breaking the monolith to microservice with Docker and Kubernetes (k8s)
 
.Net december 2017 updates - Tamir Dresher
.Net december 2017 updates - Tamir Dresher.Net december 2017 updates - Tamir Dresher
.Net december 2017 updates - Tamir Dresher
 
Testing time and concurrency Rx
Testing time and concurrency RxTesting time and concurrency Rx
Testing time and concurrency Rx
 
Building responsive application with Rx - confoo - tamir dresher
Building responsive application with Rx - confoo - tamir dresherBuilding responsive application with Rx - confoo - tamir dresher
Building responsive application with Rx - confoo - tamir dresher
 
.NET Debugging tricks you wish you knew tamir dresher
.NET Debugging tricks you wish you knew   tamir dresher.NET Debugging tricks you wish you knew   tamir dresher
.NET Debugging tricks you wish you knew tamir dresher
 
From Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir Dresher
From Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir DresherFrom Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir Dresher
From Zero to the Actor Model (With Akka.Net) - CodeMash2017 - Tamir Dresher
 
Building responsive applications with Rx - CodeMash2017 - Tamir Dresher
Building responsive applications with Rx  - CodeMash2017 - Tamir DresherBuilding responsive applications with Rx  - CodeMash2017 - Tamir Dresher
Building responsive applications with Rx - CodeMash2017 - Tamir Dresher
 
Debugging tricks you wish you knew - Tamir Dresher
Debugging tricks you wish you knew  - Tamir DresherDebugging tricks you wish you knew  - Tamir Dresher
Debugging tricks you wish you knew - Tamir Dresher
 
Rx 101 - Tamir Dresher - Copenhagen .NET User Group
Rx 101  - Tamir Dresher - Copenhagen .NET User GroupRx 101  - Tamir Dresher - Copenhagen .NET User Group
Rx 101 - Tamir Dresher - Copenhagen .NET User Group
 
Cloud patterns - NDC Oslo 2016 - Tamir Dresher
Cloud patterns - NDC Oslo 2016 - Tamir DresherCloud patterns - NDC Oslo 2016 - Tamir Dresher
Cloud patterns - NDC Oslo 2016 - Tamir Dresher
 
Reactiveness All The Way - SW Architecture 2015 Conference
Reactiveness All The Way - SW Architecture 2015 ConferenceReactiveness All The Way - SW Architecture 2015 Conference
Reactiveness All The Way - SW Architecture 2015 Conference
 
Rx 101 Codemotion Milan 2015 - Tamir Dresher
Rx 101   Codemotion Milan 2015 - Tamir DresherRx 101   Codemotion Milan 2015 - Tamir Dresher
Rx 101 Codemotion Milan 2015 - Tamir Dresher
 

Recently uploaded

Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Andreas Granig
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningVitsRangannavar
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...MyIntelliSource, Inc.
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio, Inc.
 
Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...aditisharan08
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfjoe51371421
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number SystemsJheuzeDellosa
 
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyEngage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyFrank van der Linden
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWave PLM
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptkotipi9215
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...stazi3110
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityNeo4j
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software DevelopersVinodh Ram
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)OPEN KNOWLEDGE GmbH
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackVICTOR MAESTRE RAMIREZ
 
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝soniya singh
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataBradBedford3
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfPower Karaoke
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 

Recently uploaded (20)

Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024Automate your Kamailio Test Calls - Kamailio World 2024
Automate your Kamailio Test Calls - Kamailio World 2024
 
cybersecurity notes for mca students for learning
cybersecurity notes for mca students for learningcybersecurity notes for mca students for learning
cybersecurity notes for mca students for learning
 
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...
 
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...Unit 1.1 Excite Part 1, class 9, cbse...
Unit 1.1 Excite Part 1, class 9, cbse...
 
why an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdfwhy an Opensea Clone Script might be your perfect match.pdf
why an Opensea Clone Script might be your perfect match.pdf
 
What is Binary Language? Computer Number Systems
What is Binary Language?  Computer Number SystemsWhat is Binary Language?  Computer Number Systems
What is Binary Language? Computer Number Systems
 
Engage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The UglyEngage Usergroup 2024 - The Good The Bad_The Ugly
Engage Usergroup 2024 - The Good The Bad_The Ugly
 
What is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need ItWhat is Fashion PLM and Why Do You Need It
What is Fashion PLM and Why Do You Need It
 
chapter--4-software-project-planning.ppt
chapter--4-software-project-planning.pptchapter--4-software-project-planning.ppt
chapter--4-software-project-planning.ppt
 
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...Call Girls In Mukherjee Nagar 📱  9999965857  🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
Call Girls In Mukherjee Nagar 📱 9999965857 🤩 Delhi 🫦 HOT AND SEXY VVIP 🍎 SE...
 
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
Building a General PDE Solving Framework with Symbolic-Numeric Scientific Mac...
 
EY_Graph Database Powered Sustainability
EY_Graph Database Powered SustainabilityEY_Graph Database Powered Sustainability
EY_Graph Database Powered Sustainability
 
Professional Resume Template for Software Developers
Professional Resume Template for Software DevelopersProfessional Resume Template for Software Developers
Professional Resume Template for Software Developers
 
Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)Der Spagat zwischen BIAS und FAIRNESS (2024)
Der Spagat zwischen BIAS und FAIRNESS (2024)
 
Cloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStackCloud Management Software Platforms: OpenStack
Cloud Management Software Platforms: OpenStack
 
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
Call Girls in Naraina Delhi 💯Call Us 🔝8264348440🔝
 
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer DataAdobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
Adobe Marketo Engage Deep Dives: Using Webhooks to Transfer Data
 
The Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdfThe Evolution of Karaoke From Analog to App.pdf
The Evolution of Karaoke From Analog to App.pdf
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 

Clarizen GC adventures during the holidays

  • 1. Clarizen adventures with the wild GC during the holiday season @tamir_dresher Chief Architect @ Clarizen
  • 2. 2 Chief Architect @ @tamir_dresher Tamir Dresher My Books: Software Engineering Lecturer Ruppin Academic Center tamir.dresher@clarizen.com
  • 3. CLARIZEN - ENTERPRISE WORK COLLABORATION I N T E R N A L & E X T E R N A L C O L L A B O R AT I O N A U T O M AT E D W O R K F L O W E N G I N E REPORTS & DASHBOARDS MOBILE APPS & BOTS APPSMARKETPLACE INTEGRATIONS RESOURCE MANAGEMENT TIME & EXPENSES FINANCIAL PLANNING DEMAND MANAGEMENT PROJECT MANAGEMENT PORTFOLIO MANAGEMENT ENTERPRISES FROM 124 COUNTRIES 99.95% UPTIME ENTERPRISE GRADE SaaS SOLUTION FOUNDED IN 2006
  • 4. 4
  • 10. 10
  • 11. 11
  • 12. 12 .NET Garbage Collector (GC) 12 Gen 0 Gen 1 Gen 2 Gen 0 is full, start garbage collection!New objects are allocated in Gen 0Dead objects are foundStill living objects are promoted Large object heap
  • 15. 15 Long PauseGen 0 GC https://github.com/microsoft/perfview
  • 16. 16
  • 17. 17 GC Threads 1..11 (all the same stack trace): 0:002> k # Child-SP RetAddr Call Site 00 00000000`00bbf478 000007fe`fd8c10ac ntdll!ZwWaitForSingleObject+0xa 01 00000000`00bbf480 000007fe`f9232c77 KERNELBASE!WaitForSingleObjectEx+0x79 02 00000000`00bbf520 000007fe`f9232c2f clr!CLREventWaitHelper2+0x3c 03 00000000`00bbf560 000007fe`f9232bb0 clr!CLREventWaitHelper+0x1f 04 00000000`00bbf5c0 000007fe`f939cae5 clr!CLREventBase::WaitEx+0x7c 05 00000000`00bbf650 000007fe`f939d891 clr!SVR::t_join::join+0x10f 06 00000000`00bbf6b0 000007fe`f939ce57 clr!SVR::gc_heap::mark_phase+0x385 07 00000000`00bbf750 000007fe`f939d0c6 clr!SVR::gc_heap::gc1+0xa7 08 00000000`00bbf7a0 000007fe`f939c854 clr!SVR::gc_heap::garbage_collect+0x870 ... GC Thread 12: 0:013> k # Child-SP RetAddr Call Site 00 00000000`0c17ebc8 000007fe`fd8c10ac ntdll!ZwWaitForSingleObject+0xa 01 00000000`0c17ebd0 000007fe`f9232c77 KERNELBASE!WaitForSingleObjectEx+0x79 02 00000000`0c17ec70 000007fe`f9232c2f clr!CLREventWaitHelper2+0x3c 03 00000000`0c17ecb0 000007fe`f9232bb0 clr!CLREventWaitHelper+0x1f 04 00000000`0c17ed10 000007fe`f939cae5 clr!CLREventBase::WaitEx+0x7c 05 00000000`0c17eda0 000007fe`f939df75 clr!SVR::t_join::join+0x10f 06 00000000`0c17ee00 000007fe`f939d879 clr!SVR::gc_heap::scan_dependent_handles+0x65 07 00000000`0c17ee30 000007fe`f939ce57 clr!SVR::gc_heap::mark_phase+0x36d 08 00000000`0c17eed0 000007fe`f939d0c6 clr!SVR::gc_heap::gc1+0xa7 09 00000000`0c17ef20 000007fe`f939c854 clr!SVR::gc_heap::garbage_collect+0x870 usually caused by managed heap corruption https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools
  • 18. 18 0:000> .exr -1 ExceptionAddress: 00000000778e7b9f (ntdll!RtlpLowFragHeapAllocFromContext+0x000000000000013f) ExceptionCode: c0000005 (Access violation) ExceptionFlags: 00000000 NumberParameters: 2 Parameter[0]: 0000000000000001 Parameter[1]: 0000000000160fe0 Attempt to write to address 0000000000160fe0 0:000> k # Child-SP RetAddr Call Site 00 00000000`00160fb0 00000000`778e7974 ntdll!RtlpLowFragHeapAllocFromContext+0x13f 01 00000000`00161090 000007fe`fdc7ca20 ntdll!RtlAllocateHeap+0xe4 02 00000000`001611a0 00000000`00000002 SomeAntivirusDLL+0x6ca20 03 00000000`001611a8 00000000`00000208 0x2 04 00000000`001611b0 01d4c79a`ffff0000 0x208 05 00000000`001611b8 00000000`00328a10 0x01d4c79a`ffff0000 06 00000000`001611c0 00000000`00000208 0x328a10 07 00000000`001611c8 000007fe`fdc6c2c1 0x208 08 00000000`001611d0 00000000`00000008 SomeAntivirusDLL+0x5c2c1 09 00000000`001611d8 000007fe`fddd8816 0x8 ... https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/debugger-download-tools
  • 20. 20 Conclusion • Invest in monitoring – Logs, application metrics, infrastructure metrics. • Don’t be too proud to hire an expert. • Invest in your developers diagnostic skills. • Sometimes you can really blame the anti-virus. ¯_(ツ)_/¯

Editor's Notes

  1. Top 10 customers alone represent 6 different industries: (in rank order) Financial Services Technology (HW / SW / SaaS) Agriculture Retail Travel & Hospitality Automotive So how does Clarizen’s solution bring everything together and empower customers to drive business agility. Our comprehensive suite helps you manage your work. Starting with managing demand and evaluating portfolio investments, assigning resources and optimizing them to address business needs. Robust project execution which includes tracking people’s time and expenses, as well as costs alongside revenues. We allow you to collaborate on a shared platform with your customers and partners. We know that highly collaborative project execution delivers higher success rates. These are powered by robust workflow engine. While solutions are delivered out of the box based on your specific needs, we also offer the ability to customize to your bsuiness needs. The outputs can be consumed through our web and mobile app in reports and dashboards. We also have an extension number of add-ons through the Apps Marketplace. As we address critical business processes, we recognize the need to integrate with other systems of record. We provide 3 methods of integration: RestAPIs, built-in iPaas platform (iHub) or 3rd party iPaas Platforms. And we have pre-built integrations with the most common work management tools, such as Doc Mgmt tools (SP, Box, Google), Salesforce (for PS orgs), JIRA & ServiceNow, as well as ERP systems (Oracle, Netsuite, etc).