SlideShare a Scribd company logo
1 of 34
Tales from the WebLogic Front Line(What NOT to do with WebLogic) Steve Millidge C2B2 Consulting Limited
Agenda Health Warning Setting the Scene Common Anti-patterns Ain’t misbehaving This could kill
C2B2 Consulting Fast Performance and Scalability Reliable High Availability and Recoverability Manageable Monitoring and Supporting Secure Security
First Day At Work
It’s All Gone to *#*@!
Meet the Project Manager
Philosophy Don’t Panic you didn’t #@#@ it up. Suspect Configuration  over Coding Suspect bespoke code over WebLogic code WebLogic is FAST Computers are Logical and Rational
Common Anti-patterns
Too Many Datasources
Too Many Datasources Data Source per Component All to the Same Database JNDI Name per Component C1 C1 C1 DS1 DS1 DS1 Oracle
Why It’s Bad Requires XA for all Datasources Developers receive strange “XA” exception Untick Honour Global Transactions Makes Exception “go away” Datasources now in autocommit DATA CORRUPTION!! Requires more DB connections BIND same Datasource to multiple JNDI locations
Inadvertent TX Coupling
Inadvertent TX Coupling WLS 1 WLS 2 EJB 1 REQUIRED Invoke EJB 2 REQUIRED Oracle 1 Oracle 2
Why It’s BAD Results in “Strange Exceptions” if not All XA Is the REALLY what you want XA is very heavyweight And not guaranteed XA requires particular database configuration You have 4 minimum XA resources to coordinate Fix using REQUIRES NEW
Clustering is TOO Heavy
WLS Cluster WLS Session Clustering WLS 1 WLS 2 WLS 3 WLS 4 1 2 Don’t Use the Session as a Cache USE A CACHE
Too Much logging
Too Much Logging Too Synchronized New Application 30% Slower Tx Timeouts High CPU Many Threads waiting on Logging Monitor Debug turned on, on one node In Memory Buffer Each Thread Logged to in memory queue Writer thread writes messages to file Under heavy load memory queues grew unbounded 1.5Gb heap in log messages
Ain’tMisbehavin’
Session Shenanigans Serialization Session Attributes must be Serializable See this one all the time No Failover!!! Working as per Spec No Put You must Put dirtied objects back in the Session Don’t Get modify then no Put No Failover!!! Working as per Spec
Exceptions and TX What happens to the TX when you throw an Exception from an EJB method? Application ExceptionCommit System ExceptionRollback
Poisoned JMS
Poisoned JMS Eats CPU Prevents Delivery Kills Server Performance JMS Q RepeatForever Set Redeliver Delay Set Redelivery Limit Set Error Destination Exception
Poisoned JMS (Paging) WLS Keeps Messages in Memory Causes OutOfMemory Exceptions Configure Paging Thresholds  Configure at JMS Server level
This Can Kill You
Too Much RAM -XMS2G –XMX2G 32 bit Can be Too Much O/S Reserves RAM VM Starts up When you need it! HOTSPOT DUMP
Node Manager Configuration Not Good Out of the Box Auto Kill If Failed = TRUE Auto Restart = TRUE Restart Delay should be set Restart Max should be set Set Panic Action to System.exit
Disappearing Datasource Start WebLogic when DB not Available Datasource is not created Connection Creation Retry Frequency Seconds should be non-zero
Summary Production Configuration is Complex Call in the Professionals
Example Bullet Slide Bullet 1

More Related Content

Similar to Tales From the Web Logic Front Line

Clustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And AvailabilityClustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And Availability
ConSanFrancisco123
 
Sql 2012 always on
Sql 2012 always onSql 2012 always on
Sql 2012 always on
dilip nayak
 
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
Jeffrey West
 
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012
Eonblast
 
download it from here
download it from heredownload it from here
download it from here
webhostingguy
 
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod ColledgeDb As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
sqlserver.co.il
 
Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0
Yogi Kulkarni
 

Similar to Tales From the Web Logic Front Line (20)

Locking and Race Conditions in Web Applications
Locking and Race Conditions in Web ApplicationsLocking and Race Conditions in Web Applications
Locking and Race Conditions in Web Applications
 
Large Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBossLarge Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBoss
 
Real World Single Page App - A Knockout Case Study
Real World Single Page App - A Knockout Case StudyReal World Single Page App - A Knockout Case Study
Real World Single Page App - A Knockout Case Study
 
Clustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And AvailabilityClustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And Availability
 
Sql 2012 always on
Sql 2012 always onSql 2012 always on
Sql 2012 always on
 
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
 
Weblogic - Data management in application servers
Weblogic - Data management in application serversWeblogic - Data management in application servers
Weblogic - Data management in application servers
 
MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMaker
 
Java one 2015 [con3339]
Java one 2015 [con3339]Java one 2015 [con3339]
Java one 2015 [con3339]
 
MySQL HA Alternatives 2010
MySQL  HA  Alternatives 2010MySQL  HA  Alternatives 2010
MySQL HA Alternatives 2010
 
GOTO Night with Todd Montgomery: Aeron: What, why and what next?
GOTO Night with Todd Montgomery: Aeron: What, why and what next?GOTO Night with Todd Montgomery: Aeron: What, why and what next?
GOTO Night with Todd Montgomery: Aeron: What, why and what next?
 
Medium Trust for Umbraco
Medium Trust for UmbracoMedium Trust for Umbraco
Medium Trust for Umbraco
 
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012
 
download it from here
download it from heredownload it from here
download it from here
 
We're going on a bug hunt! Experts Talk Manchester 2018
We're going on a bug hunt! Experts Talk Manchester 2018We're going on a bug hunt! Experts Talk Manchester 2018
We're going on a bug hunt! Experts Talk Manchester 2018
 
ExpertTalks Manchester September 2018
ExpertTalks Manchester September 2018ExpertTalks Manchester September 2018
ExpertTalks Manchester September 2018
 
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod ColledgeDb As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
 
Ceph - High Performance Without High Costs
Ceph - High Performance Without High CostsCeph - High Performance Without High Costs
Ceph - High Performance Without High Costs
 
Apache Kafka – (Pattern and) Anti-Pattern
Apache Kafka – (Pattern and) Anti-PatternApache Kafka – (Pattern and) Anti-Pattern
Apache Kafka – (Pattern and) Anti-Pattern
 
Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0
 

More from C2B2 Consulting

Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
C2B2 Consulting
 
Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
C2B2 Consulting
 

More from C2B2 Consulting (20)

Monitoring Oracle SOA Suite - UKOUG Tech15 2015
Monitoring Oracle SOA Suite - UKOUG Tech15 2015Monitoring Oracle SOA Suite - UKOUG Tech15 2015
Monitoring Oracle SOA Suite - UKOUG Tech15 2015
 
Hands-on Performance Tuning Lab - Devoxx Poland
Hands-on Performance Tuning Lab - Devoxx PolandHands-on Performance Tuning Lab - Devoxx Poland
Hands-on Performance Tuning Lab - Devoxx Poland
 
Monitoring Oracle SOA Suite
Monitoring Oracle SOA SuiteMonitoring Oracle SOA Suite
Monitoring Oracle SOA Suite
 
Advanced queries on the Infinispan Data Grid
Advanced queries on the Infinispan Data Grid Advanced queries on the Infinispan Data Grid
Advanced queries on the Infinispan Data Grid
 
Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
 
Building WebLogic Domains With WLST
Building WebLogic Domains With WLSTBuilding WebLogic Domains With WLST
Building WebLogic Domains With WLST
 
Hands-on Performance Workshop - The science of performance
Hands-on Performance Workshop - The science of performanceHands-on Performance Workshop - The science of performance
Hands-on Performance Workshop - The science of performance
 
Jsr107 come, code, cache, compute!
Jsr107 come, code, cache, compute!Jsr107 come, code, cache, compute!
Jsr107 come, code, cache, compute!
 
JBoss Clustering on OpenShift
JBoss Clustering on OpenShiftJBoss Clustering on OpenShift
JBoss Clustering on OpenShift
 
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...
 
Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
 
Oracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at Scale
Oracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at ScaleOracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at Scale
Oracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at Scale
 
Java Middleware Surgery
Java Middleware Surgery Java Middleware Surgery
Java Middleware Surgery
 
Jax London 2013
Jax London 2013Jax London 2013
Jax London 2013
 
Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...
Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...
Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...
 
'Deploying with GlassFish & Docker'
'Deploying with GlassFish & Docker' 'Deploying with GlassFish & Docker'
'Deploying with GlassFish & Docker'
 
'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit'
'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit' 'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit'
'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit'
 
'New JMS features in GlassFish 4.0' by Nigel Deakin
'New JMS features in GlassFish 4.0' by Nigel Deakin'New JMS features in GlassFish 4.0' by Nigel Deakin
'New JMS features in GlassFish 4.0' by Nigel Deakin
 
Coherence sig-nfr-web-tier-scaling-using-coherence-web
Coherence sig-nfr-web-tier-scaling-using-coherence-webCoherence sig-nfr-web-tier-scaling-using-coherence-web
Coherence sig-nfr-web-tier-scaling-using-coherence-web
 
JUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at Scale
JUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at ScaleJUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at Scale
JUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at Scale
 

Recently uploaded

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Recently uploaded (20)

Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 

Tales From the Web Logic Front Line

  • 1. Tales from the WebLogic Front Line(What NOT to do with WebLogic) Steve Millidge C2B2 Consulting Limited
  • 2. Agenda Health Warning Setting the Scene Common Anti-patterns Ain’t misbehaving This could kill
  • 3. C2B2 Consulting Fast Performance and Scalability Reliable High Availability and Recoverability Manageable Monitoring and Supporting Secure Security
  • 5. It’s All Gone to *#*@!
  • 6.
  • 7.
  • 9.
  • 10.
  • 11. Philosophy Don’t Panic you didn’t #@#@ it up. Suspect Configuration over Coding Suspect bespoke code over WebLogic code WebLogic is FAST Computers are Logical and Rational
  • 14. Too Many Datasources Data Source per Component All to the Same Database JNDI Name per Component C1 C1 C1 DS1 DS1 DS1 Oracle
  • 15. Why It’s Bad Requires XA for all Datasources Developers receive strange “XA” exception Untick Honour Global Transactions Makes Exception “go away” Datasources now in autocommit DATA CORRUPTION!! Requires more DB connections BIND same Datasource to multiple JNDI locations
  • 17. Inadvertent TX Coupling WLS 1 WLS 2 EJB 1 REQUIRED Invoke EJB 2 REQUIRED Oracle 1 Oracle 2
  • 18. Why It’s BAD Results in “Strange Exceptions” if not All XA Is the REALLY what you want XA is very heavyweight And not guaranteed XA requires particular database configuration You have 4 minimum XA resources to coordinate Fix using REQUIRES NEW
  • 20. WLS Cluster WLS Session Clustering WLS 1 WLS 2 WLS 3 WLS 4 1 2 Don’t Use the Session as a Cache USE A CACHE
  • 22. Too Much Logging Too Synchronized New Application 30% Slower Tx Timeouts High CPU Many Threads waiting on Logging Monitor Debug turned on, on one node In Memory Buffer Each Thread Logged to in memory queue Writer thread writes messages to file Under heavy load memory queues grew unbounded 1.5Gb heap in log messages
  • 24. Session Shenanigans Serialization Session Attributes must be Serializable See this one all the time No Failover!!! Working as per Spec No Put You must Put dirtied objects back in the Session Don’t Get modify then no Put No Failover!!! Working as per Spec
  • 25. Exceptions and TX What happens to the TX when you throw an Exception from an EJB method? Application ExceptionCommit System ExceptionRollback
  • 27. Poisoned JMS Eats CPU Prevents Delivery Kills Server Performance JMS Q RepeatForever Set Redeliver Delay Set Redelivery Limit Set Error Destination Exception
  • 28. Poisoned JMS (Paging) WLS Keeps Messages in Memory Causes OutOfMemory Exceptions Configure Paging Thresholds Configure at JMS Server level
  • 30. Too Much RAM -XMS2G –XMX2G 32 bit Can be Too Much O/S Reserves RAM VM Starts up When you need it! HOTSPOT DUMP
  • 31. Node Manager Configuration Not Good Out of the Box Auto Kill If Failed = TRUE Auto Restart = TRUE Restart Delay should be set Restart Max should be set Set Panic Action to System.exit
  • 32. Disappearing Datasource Start WebLogic when DB not Available Datasource is not created Connection Creation Retry Frequency Seconds should be non-zero
  • 33. Summary Production Configuration is Complex Call in the Professionals

Editor's Notes

  1. These are things we’ve seen people do a lot which you shouldn’t do.
  2. We see many examples of lots and lots of datasources