SlideShare a Scribd company logo
1 of 34
Tales from the WebLogic Front Line(What NOT to do with WebLogic) Steve Millidge C2B2 Consulting Limited
Agenda Health Warning Setting the Scene Common Anti-patterns Ain’t misbehaving This could kill
C2B2 Consulting Fast Performance and Scalability Reliable High Availability and Recoverability Manageable Monitoring and Supporting Secure Security
First Day At Work
It’s All Gone to *#*@!
Meet the Project Manager
Philosophy Don’t Panic you didn’t #@#@ it up. Suspect Configuration  over Coding Suspect bespoke code over WebLogic code WebLogic is FAST Computers are Logical and Rational
Common Anti-patterns
Too Many Datasources
Too Many Datasources Data Source per Component All to the Same Database JNDI Name per Component C1 C1 C1 DS1 DS1 DS1 Oracle
Why It’s Bad Requires XA for all Datasources Developers receive strange “XA” exception Untick Honour Global Transactions Makes Exception “go away” Datasources now in autocommit DATA CORRUPTION!! Requires more DB connections BIND same Datasource to multiple JNDI locations
Inadvertent TX Coupling
Inadvertent TX Coupling WLS 1 WLS 2 EJB 1 REQUIRED Invoke EJB 2 REQUIRED Oracle 1 Oracle 2
Why It’s BAD Results in “Strange Exceptions” if not All XA Is the REALLY what you want XA is very heavyweight And not guaranteed XA requires particular database configuration You have 4 minimum XA resources to coordinate Fix using REQUIRES NEW
Clustering is TOO Heavy
WLS Cluster WLS Session Clustering WLS 1 WLS 2 WLS 3 WLS 4 1 2 Don’t Use the Session as a Cache USE A CACHE
Too Much logging
Too Much Logging Too Synchronized New Application 30% Slower Tx Timeouts High CPU Many Threads waiting on Logging Monitor Debug turned on, on one node In Memory Buffer Each Thread Logged to in memory queue Writer thread writes messages to file Under heavy load memory queues grew unbounded 1.5Gb heap in log messages
Ain’tMisbehavin’
Session Shenanigans Serialization Session Attributes must be Serializable See this one all the time No Failover!!! Working as per Spec No Put You must Put dirtied objects back in the Session Don’t Get modify then no Put No Failover!!! Working as per Spec
Exceptions and TX What happens to the TX when you throw an Exception from an EJB method? Application ExceptionCommit System ExceptionRollback
Poisoned JMS
Poisoned JMS Eats CPU Prevents Delivery Kills Server Performance JMS Q RepeatForever Set Redeliver Delay Set Redelivery Limit Set Error Destination Exception
Poisoned JMS (Paging) WLS Keeps Messages in Memory Causes OutOfMemory Exceptions Configure Paging Thresholds  Configure at JMS Server level
This Can Kill You
Too Much RAM -XMS2G –XMX2G 32 bit Can be Too Much O/S Reserves RAM VM Starts up When you need it! HOTSPOT DUMP
Node Manager Configuration Not Good Out of the Box Auto Kill If Failed = TRUE Auto Restart = TRUE Restart Delay should be set Restart Max should be set Set Panic Action to System.exit
Disappearing Datasource Start WebLogic when DB not Available Datasource is not created Connection Creation Retry Frequency Seconds should be non-zero
Summary Production Configuration is Complex Call in the Professionals
Example Bullet Slide Bullet 1

More Related Content

Similar to Tales From the Web Logic Front Line

Clustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And AvailabilityClustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And Availability
ConSanFrancisco123
 
Sql 2012 always on
Sql 2012 always onSql 2012 always on
Sql 2012 always on
dilip nayak
 
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
Jeffrey West
 
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012
Eonblast
 
download it from here
download it from heredownload it from here
download it from here
webhostingguy
 
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod ColledgeDb As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
sqlserver.co.il
 
Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0
Yogi Kulkarni
 

Similar to Tales From the Web Logic Front Line (20)

Locking and Race Conditions in Web Applications
Locking and Race Conditions in Web ApplicationsLocking and Race Conditions in Web Applications
Locking and Race Conditions in Web Applications
 
Large Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBossLarge Scale Migration from WebLogic to JBoss
Large Scale Migration from WebLogic to JBoss
 
Real World Single Page App - A Knockout Case Study
Real World Single Page App - A Knockout Case StudyReal World Single Page App - A Knockout Case Study
Real World Single Page App - A Knockout Case Study
 
Clustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And AvailabilityClustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And Availability
 
Sql 2012 always on
Sql 2012 always onSql 2012 always on
Sql 2012 always on
 
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
WebLogic Developer Webcast 5: Troubleshooting and Testing with WebLogic, Soap...
 
Weblogic - Data management in application servers
Weblogic - Data management in application serversWeblogic - Data management in application servers
Weblogic - Data management in application servers
 
MySQL HA with PaceMaker
MySQL HA with  PaceMakerMySQL HA with  PaceMaker
MySQL HA with PaceMaker
 
Java one 2015 [con3339]
Java one 2015 [con3339]Java one 2015 [con3339]
Java one 2015 [con3339]
 
MySQL HA Alternatives 2010
MySQL  HA  Alternatives 2010MySQL  HA  Alternatives 2010
MySQL HA Alternatives 2010
 
GOTO Night with Todd Montgomery: Aeron: What, why and what next?
GOTO Night with Todd Montgomery: Aeron: What, why and what next?GOTO Night with Todd Montgomery: Aeron: What, why and what next?
GOTO Night with Todd Montgomery: Aeron: What, why and what next?
 
Medium Trust for Umbraco
Medium Trust for UmbracoMedium Trust for Umbraco
Medium Trust for Umbraco
 
VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012VoltDB and Erlang - Tech planet 2012
VoltDB and Erlang - Tech planet 2012
 
download it from here
download it from heredownload it from here
download it from here
 
ExpertTalks Manchester September 2018
ExpertTalks Manchester September 2018ExpertTalks Manchester September 2018
ExpertTalks Manchester September 2018
 
We're going on a bug hunt! Experts Talk Manchester 2018
We're going on a bug hunt! Experts Talk Manchester 2018We're going on a bug hunt! Experts Talk Manchester 2018
We're going on a bug hunt! Experts Talk Manchester 2018
 
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod ColledgeDb As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
Db As Behaving Badly... Worst Practices For Database Administrators Rod Colledge
 
Ceph - High Performance Without High Costs
Ceph - High Performance Without High CostsCeph - High Performance Without High Costs
Ceph - High Performance Without High Costs
 
Apache Kafka – (Pattern and) Anti-Pattern
Apache Kafka – (Pattern and) Anti-PatternApache Kafka – (Pattern and) Anti-Pattern
Apache Kafka – (Pattern and) Anti-Pattern
 
Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0Perfomance tuning on Go 2.0
Perfomance tuning on Go 2.0
 

More from C2B2 Consulting

Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
C2B2 Consulting
 
Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
C2B2 Consulting
 

More from C2B2 Consulting (20)

Monitoring Oracle SOA Suite - UKOUG Tech15 2015
Monitoring Oracle SOA Suite - UKOUG Tech15 2015Monitoring Oracle SOA Suite - UKOUG Tech15 2015
Monitoring Oracle SOA Suite - UKOUG Tech15 2015
 
Hands-on Performance Tuning Lab - Devoxx Poland
Hands-on Performance Tuning Lab - Devoxx PolandHands-on Performance Tuning Lab - Devoxx Poland
Hands-on Performance Tuning Lab - Devoxx Poland
 
Monitoring Oracle SOA Suite
Monitoring Oracle SOA SuiteMonitoring Oracle SOA Suite
Monitoring Oracle SOA Suite
 
Advanced queries on the Infinispan Data Grid
Advanced queries on the Infinispan Data Grid Advanced queries on the Infinispan Data Grid
Advanced queries on the Infinispan Data Grid
 
Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
 
Building WebLogic Domains With WLST
Building WebLogic Domains With WLSTBuilding WebLogic Domains With WLST
Building WebLogic Domains With WLST
 
Hands-on Performance Workshop - The science of performance
Hands-on Performance Workshop - The science of performanceHands-on Performance Workshop - The science of performance
Hands-on Performance Workshop - The science of performance
 
Jsr107 come, code, cache, compute!
Jsr107 come, code, cache, compute!Jsr107 come, code, cache, compute!
Jsr107 come, code, cache, compute!
 
JBoss Clustering on OpenShift
JBoss Clustering on OpenShiftJBoss Clustering on OpenShift
JBoss Clustering on OpenShift
 
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the ...
 
Through the JMX Window
Through the JMX WindowThrough the JMX Window
Through the JMX Window
 
Oracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at Scale
Oracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at ScaleOracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at Scale
Oracle Coherence & WebLogic 12c Web Sockets: Delivering Real Time Push at Scale
 
Java Middleware Surgery
Java Middleware Surgery Java Middleware Surgery
Java Middleware Surgery
 
Jax London 2013
Jax London 2013Jax London 2013
Jax London 2013
 
Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...
Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...
Oracle SOA Suite Performance Tuning- UKOUG Application Server & Middleware SI...
 
'Deploying with GlassFish & Docker'
'Deploying with GlassFish & Docker' 'Deploying with GlassFish & Docker'
'Deploying with GlassFish & Docker'
 
'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit'
'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit' 'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit'
'JMS @ Data Grid? Hacking the Glassfish messaging for fun & profit'
 
'New JMS features in GlassFish 4.0' by Nigel Deakin
'New JMS features in GlassFish 4.0' by Nigel Deakin'New JMS features in GlassFish 4.0' by Nigel Deakin
'New JMS features in GlassFish 4.0' by Nigel Deakin
 
Coherence sig-nfr-web-tier-scaling-using-coherence-web
Coherence sig-nfr-web-tier-scaling-using-coherence-webCoherence sig-nfr-web-tier-scaling-using-coherence-web
Coherence sig-nfr-web-tier-scaling-using-coherence-web
 
JUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at Scale
JUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at ScaleJUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at Scale
JUDCon 2013- JBoss Data Grid and WebSockets: Delivering Real Time Push at Scale
 

Recently uploaded

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 

Recently uploaded (20)

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024Extensible Python: Robustness through Addition - PyCon 2024
Extensible Python: Robustness through Addition - PyCon 2024
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
THE BEST IPTV in GERMANY for 2024: IPTVreel
THE BEST IPTV in  GERMANY for 2024: IPTVreelTHE BEST IPTV in  GERMANY for 2024: IPTVreel
THE BEST IPTV in GERMANY for 2024: IPTVreel
 
Intro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджераIntro in Product Management - Коротко про професію продакт менеджера
Intro in Product Management - Коротко про професію продакт менеджера
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Strategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering TeamsStrategic AI Integration in Engineering Teams
Strategic AI Integration in Engineering Teams
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 

Tales From the Web Logic Front Line

  • 1. Tales from the WebLogic Front Line(What NOT to do with WebLogic) Steve Millidge C2B2 Consulting Limited
  • 2. Agenda Health Warning Setting the Scene Common Anti-patterns Ain’t misbehaving This could kill
  • 3. C2B2 Consulting Fast Performance and Scalability Reliable High Availability and Recoverability Manageable Monitoring and Supporting Secure Security
  • 5. It’s All Gone to *#*@!
  • 6.
  • 7.
  • 9.
  • 10.
  • 11. Philosophy Don’t Panic you didn’t #@#@ it up. Suspect Configuration over Coding Suspect bespoke code over WebLogic code WebLogic is FAST Computers are Logical and Rational
  • 14. Too Many Datasources Data Source per Component All to the Same Database JNDI Name per Component C1 C1 C1 DS1 DS1 DS1 Oracle
  • 15. Why It’s Bad Requires XA for all Datasources Developers receive strange “XA” exception Untick Honour Global Transactions Makes Exception “go away” Datasources now in autocommit DATA CORRUPTION!! Requires more DB connections BIND same Datasource to multiple JNDI locations
  • 17. Inadvertent TX Coupling WLS 1 WLS 2 EJB 1 REQUIRED Invoke EJB 2 REQUIRED Oracle 1 Oracle 2
  • 18. Why It’s BAD Results in “Strange Exceptions” if not All XA Is the REALLY what you want XA is very heavyweight And not guaranteed XA requires particular database configuration You have 4 minimum XA resources to coordinate Fix using REQUIRES NEW
  • 20. WLS Cluster WLS Session Clustering WLS 1 WLS 2 WLS 3 WLS 4 1 2 Don’t Use the Session as a Cache USE A CACHE
  • 22. Too Much Logging Too Synchronized New Application 30% Slower Tx Timeouts High CPU Many Threads waiting on Logging Monitor Debug turned on, on one node In Memory Buffer Each Thread Logged to in memory queue Writer thread writes messages to file Under heavy load memory queues grew unbounded 1.5Gb heap in log messages
  • 24. Session Shenanigans Serialization Session Attributes must be Serializable See this one all the time No Failover!!! Working as per Spec No Put You must Put dirtied objects back in the Session Don’t Get modify then no Put No Failover!!! Working as per Spec
  • 25. Exceptions and TX What happens to the TX when you throw an Exception from an EJB method? Application ExceptionCommit System ExceptionRollback
  • 27. Poisoned JMS Eats CPU Prevents Delivery Kills Server Performance JMS Q RepeatForever Set Redeliver Delay Set Redelivery Limit Set Error Destination Exception
  • 28. Poisoned JMS (Paging) WLS Keeps Messages in Memory Causes OutOfMemory Exceptions Configure Paging Thresholds Configure at JMS Server level
  • 30. Too Much RAM -XMS2G –XMX2G 32 bit Can be Too Much O/S Reserves RAM VM Starts up When you need it! HOTSPOT DUMP
  • 31. Node Manager Configuration Not Good Out of the Box Auto Kill If Failed = TRUE Auto Restart = TRUE Restart Delay should be set Restart Max should be set Set Panic Action to System.exit
  • 32. Disappearing Datasource Start WebLogic when DB not Available Datasource is not created Connection Creation Retry Frequency Seconds should be non-zero
  • 33. Summary Production Configuration is Complex Call in the Professionals

Editor's Notes

  1. These are things we’ve seen people do a lot which you shouldn’t do.
  2. We see many examples of lots and lots of datasources