Legacy Renewal of Central Framework in the Enterprise

GeeCon 2014
Renewal of Legacy Frameworks
An Experience Report
Anatole Tresch
May 2014

Renewal of Central Frameworks – An Experience Report May 2014
Bio
Anatole Tresch
Consultant, Coach
Credit Suisse
Core Framework Engineer,
Technical Architect
Specification Lead JSR 354
Driving Java EE Config
Twitter/Google+: @atsticks
atsticks@java.net
anatole.tresch@credit-suisse.com
Java Config Discussion https://groups.google.com/forum/#!forum/java-config
Java Config Blog: http://javaeeconfig.blogspot.ch
Zurich Java Community
(https://plus.google.com/u/0/communities/106363890913137321158)
Zurich Hackergarten
(https://plus.google.com/u/0/103936743237262644768/posts)
2

Credit Suisse
 Global Bank with Headquarter in Zurich, Clients in Private Banking,
Investment Banking and Asset Management.
 Shares as Credit Suisse Group AG (CSGN) in Switzerland
(SIX) and as American Depositary Shares (CS) in New York
(NYSE).
 47,400 Employees.
 Ratings: Moody’s A2, Standard & Poor’s A, Fitch Ratings A.
3

Renewal of Central Frameworks
Agenda
4
Where we started
Objectives and Constraints
Challenges
CS Java Application Platform
Implementation
Summary

Where we started
5

The Credit Suisse IT
From a Bird’s View
6
On-, Offshore ca. 20’000+ Employees in IT
Multiple global distributed Operation Centers
Protocol-Mix: Corba, WS, Restful, RMI, …
Technology-Mix: Java, DotNet, PL/1, Oracle, Sybase,
MS SQL Server, DB2, IMS, Smalltalk, Python,
MS Access
In-House and COTS Solutions
Total > 1000 applications

Java Core Frameworks
Deployment
7
Global Deployment
Part of the Java Application Platform
Infrastructure (JAP)
2-3 Major Releases active in parallel
Delivery: 3 Jars + Scripts
Developer-Desktop: Windows
Production: Solaris/Linux

Positioning within the Software and Hardware Stack
8
Infrastructure-Code!

External Interfaces
2 Mio LOC
1.5 Mio LOC
Employees
Web
Web
Web
Business
1 Mio LOC
500k LOC
Java in Credit Suisse
A typical Application-Stack
9
Clients
Entry
Host
Web
Business
CSIB (synchron) MQ Series (asynchron)
Web
Business
Business
COTS
Batch
Batch
TA Monitoring
Entry
Entry
Web Web
Web
Business
Business
Batch
Web
Business
Business
Business
Core Frameworks
Colors = versions

The Java Core Frameworks’s Functionality
Context Management (Tracing, Accounting)
Exception Propagation (incl. Trace-Context)
High Performance Logging, Audit Logging
Platform-Abstraction Layer
Configuration Management (Deployment, Application)
JMX Registration und Naming Services
Go for the money - JSR 354 - http://java.net/projects/javamoney 23th October 2013 10
i18n
Common Utilities

Functionality of the Java Core Frameworks – a Simulation
Coonntteexxtt
ConteExrtror
11
Entry
Web
Business Business2 External Host
Service Bus
Exception
Audit
Audit
Audit Audit
Audit
Log
Log
Log
Log
Log
Log
Log
MessageBus
Log
Audit

From Daily Business
12
Increasing Support Efforts
Non solvable Problems
Complicated, invasive APIs
Missing or complicated SPIs
Implementation- and Runtime Dependencies
Lack of Flexibility and Extendability
Slow, uncomprehensive Code
Missing, outdated or uncomprehensive documentation
…

Quality Issues
API?
13
public interface Manageable {
/**
* Invokes different methods in all classes that implement this
* interface. What method actually is performed is determined by the
* argument <B>cmd</B>. The Stringarray options contains the
* arguments for invoking the method.
* @param command
* The method that should be invoked.
* @param options
* The command line arguments for the command.
* @throws Throwable
* If an error occurs.
* @return The return value depends on the method that is invoked.
*/
public Serializable invoke(String command, String[] options)
throws Throwable;
}

Quality Issues
SPI?
14
/**
* Manages the storage of a {@link CoreContextHolder} object. Extending this
* class, allows to support different storage behaviours. A CoreContextHolder
* Store is used in {@link CoreContext}. Hint: Abstract class
* is used to be visible package local only.
*/
abstract class CoreContextHolderStore {
/** Returns the previous stored CoreContextHolder. */
abstract CoreContextHolder getCoreContextHolder();
/** Stores a CoreContextHolder. */
abstract void setCoreContextHolder(CoreContextHolder holder);
/** Factory to get CoreContextHolderStore instance. */
final static class Factory {
enum Type { DEFAULT, SINGLETON, THREADLOCAL, WORKCONTEXT };
private final static Factory SINGLETON = new Factory();
/** type of factory. */
private Type instanceType = Type.DEFAULT;
/** store created last time */
private CoreContextHolderStore storeCreated = null;
static Factory getSingleton() { return SINGLETON; }
…

Quality Issues
Comprehensivness/Complexity
15
public void loadDefaultFromRescource(String[] resources)
throws ConfigException{
String environment = System.getProperty(PROPERTY_ENVIRONMENT);
environment = ENVIRONMENT_MAP.get(environment);
String rescourcePath = null;
for (int i = 0; i < resources.length; i++) {
rescourcePath = CONFIG_IN_JAR_FOLDERS_START + environment +
CONFIG_IN_JAR_FOLDERS_END;
try {
loadDefaultFromRescource(rescourcePath + resources[i]); }
catch (ConfigException e) {
if("BASE0130".equals(e.getExceptionContext().getMessageID())) {
rescourcePath = CONFIG_IN_JAR_FOLDERS_START +
FOLDER_ENV_GLOBAL + CONFIG_IN_JAR_FOLDERS_END;
loadDefaultFromRescource(rescourcePath + resources[i]);
}
}
}
}

Quality Issues
Other Metrics
16
17’000 LOC / 8721 Statements
204 Typen, 1400 Methoden
23 Packet Cycles !
Strong Coupling
Big Artifacts (> 1500 LOC)
Only a Few Interfaces
No Annotations
Missing Information Hiding
…

Where we started
Conclusion
17

18

Technical Objectives
Improved Stability, Maintainability and Extendibility
Improved Performance and Scalability
Improved Usability
Convention over Configuration
Easy Testing (independent tests!)
Reduction of API Footprint (decoupling)
Technical Innovation (IoC)
Maximal Backward Compatibility, smooth Migration Path
19

Compatibility
Backward Compatibility:
Context Propagation
Serialization (Context, Exceptions)
Build and behavioral Compatibility (API)
Reduced Compatibility Requirements on Implementation
Artifacts
Forward Compatibility:
Context Propagation
Serialization (Context, Exceptions)
20

Organizational Objectives
Reduce Support Efforts
Simplify Framework Development
Continuous Build
(Semi-) Automatic Deployment
Building up Know How
21

Challenges
22
Quality Culture
Non-Standard Tool-Chain
Stakeholder Management
Time & Budget
Skills

CS Java Application Platform
The Environment
23

Java Application Platform JAP
Idea and Motivation
24
Applikation Logic
GUI, Business Logic, DB Schemas, Configuration, etc.
Infrastructure Design/Configuration
HW, OS, Middleware, Network
Systems Management
Operating Manuals
Development Tools and Processes
Security Services and Processes
Integration Services and Processes
Applikation Platform (AP)
Optimized for similar applications,
Based on Hosting Platforms
Hosting Platforms
Provide Common Services
• Computation CHP (Hardware, OS)
• Persistence DHP (Databases)

JAP Lifecycle Management
Why Lifecycle Management?
25
• Applications and Components
without managed Lifecycle
• Uncoordinated technical
Upgrades
• Incompatibilities
• Managed Lifecycle
• Coordinated and tested technical
Upgrades (HW+SW!)
• Constant Decommissioning of old
Platform Releases
• New Releases wont impact
existing applications

JAP Lifecycle Management
Release Planning
26

Implementation
27

Implementation
Release Planning
Major Releases synchronous with JAP Platform (periodically all 2-3
Years)
Patch Releases 2-3/Year
Emergency Releases as necessary
1 Major Framework Release =
Proposals, Requirements, Feature Selection beforehand
6 months Implementation and Component Tests
6 months Integration Testing and Documentation
28

Implementation
Specification/Design
High Level Design (Make or “Buy”)
Existing Functionality in Java EE 5/6, Weblogic?
Third Party / OSS Libraries? Functional Gap?
Low Level Design
 Minimize APIs, go for Effective Java
 Immutable Value Objects
 Thread-Save with Low Latency
 Multi-Tenancy, Multi-Language, Multi-Timezone etc.
 Decide on Adaption of existing APIs vs. Recreation and Coexistence
 Consider Operational Aspects
29

Implementation
Team and Organization
Agile Process (no Offshoring)
2 Full-Time Programmers
Pair Programming as Needed
Hire the Best you get!
30

Implementation
Further Details
Extend and Improve Existing Tests
Improve/write Component Tests instead of Smoke Tests
Create Compatibility and Interop Tests
Ensure Independent Test Execution (Test Isolation)
Ensure Integration Tests (across multiple releases!)
Integrate early!
API
Deprecate old APIs (for at least(!) one Release)
Check Backward Compatibility with Sonar/clirr continously
31

Implementation
Quality Management
32
 Static Code Analysis:
 Continuous and integrated, on each code change, that compiled
 Based on Sonar, with customized Profile and Custom Rules
 Coverage Analysis of Component Tests
 Basic Integrationstests
 Regular Check of Metrics, Alarms!
 Peer-Reviews of all Code Changes
 Integration Tests with big Pilots

Implementation
Framework Release Cycles
Similar to Platform Releases
Playground 1 (first drop)
Playground 2 (functional)
Alpha (API stable)
Beta (only patches)
Release Candidate
Final Release
Patch Releases (regular updated)
Emergency Releases (as needed)
33

Implementation
Fist Run 2010 - 5.0.0 -> 6.0.0
Partial API Renewal and API Reduction
Defined SPIs
Implementation Modularization and Encapsulation (Spring/Lenz)
Focus on
Critical Functionalities
Stabilize Existing Code Base
Focus on Stability, reduce Complexity
3 Jars + Scripts
34
-> Only Partial Renewal
-> significant Compatibility Breaks
-> Massive Performance Gains

Implementation
Second Run 2012 - 6.0.0 -> 7.0.0
HW/OS Change from Solaris to Linux
Several small Improvements
Fixing of Known Issues (sometimes with behavioral changes!)
Low Latency Requirements
Introduction of a Runtime Platform Abstraction Layer
Replacement of Spring with a leaner Container
Introduction of Continuous Build based on Maven
7 Jars + Scripts
QC with Sonar
35
-> Renewal still ongoing
-> only a few compatibility breaks
-> New, efficient tool chain

Implementation
Third Run 2013/2014 - v7.0.0 -> 8.0.0 (today)
Introduction of Java EE 6 (formerly EE 5)
Complex Integration of Low Level Features / CDI
New Annotation based API
Long Lasting Deprecations (New IOC Paradigma)
Renewal of Further Areas Untouched so far
Full CDI-Support
36
-> Renewal Finished
-> only few Compatibility Issues
-> reduced API Footprint (Annotations)
-> modern API (CDI)

Conclusion
37
……..?

Conclusion
Key Factors
38
Functional and Effective Tooling
Continuous Build with Integrated Quality Measurement
Agil and Effective Collaboration Culture
Focus on task: minimize Maintenance for former Releases
Ensure enough Budget and Time
Define a realistic Scope
Managed Environment (Lifecycle and Platform Management)
Use complex Test-Pilots early!

Conclusion
Did we met our Objectives?
-50%
Yes.
100 x faster
-50%
Yes.
Yes EE6/7
supported
39
Significant Reduction of Support Efforts
Establishment of Maintainability and Extendibility
Improved Runtime Behavior (Performance, Memory)
Reduction (Optimization) of API Footprint
Simplify Testing
Enable Technical Progress
Improved Quality
Backward Compatibility, if possible
Yes, but more
artifacts & LOC!
mostly

Conclusion
Actual Metrics
40

Conclusion
API Evolution
41
OLD:
Configuration appCfg = ConfigHelper.getConfigManager()
.getApplicationConfig();
Map<String,String> config = null;
if(appCfg.containsArea("a.b.c")){
config = appCfg.getArea("a.b.c", false);
}
NEW:
@Configured
@ConfigRoot(value="a.b.c", recursive=false)
@Optional
private Map<String,String> config;

Conclusion
API Evolution 2
42
OLD:
private static final TraceLogger tl =
TraceLoggerHelper.getTracLogger(
getClass(), TraceType.SERVICE);
public void bookSettlement(Settlement s){
tl.entering(getClass(), "bookSettlement", s);
// perform…
tl.exiting(getClass(), "bookSettlement", s);
}
NEW:
@TraceLogged(TraceType.SERVICE)
public void bookSettlement(Settlement s){
// perform…
}

Q & A
43

The End
44

Legacy Renewal of Central Framework in the Enterprise

More Related Content

What's hot

Similar to Legacy Renewal of Central Framework in the Enterprise

More from Anatole Tresch

Recently uploaded

Legacy Renewal of Central Framework in the Enterprise

Editor's Notes