Download to read offline
![Figure 1.1: Typical Enterprise Architecture
volume of data produced when profiling even a single user system can be extremely
large. When profiling multi-user enterprise applications, where a typical load may be
in the order of thousands, the amount of data produced can be truly overwhelming.
Often developers are required to sift through and correlate this information looking
for bottlenecks in their systems. Furthermore, even when developers find issues in
their applications using these tools, it is common that they are unsure as to how to go
about rectifying the issue. There is a clear need for more advanced performance tools
that not only profile the running system, but that also analyse the data produced to
identify potential issues in the application. While there has been research in the area of
debugging tools (e.g. [95] [145] [55] [14] [47]) which allow for automatic low-level bug
detection, often it is the case that low-level optimizations or fixes will not be enough to
enhance the system efficiency such that performance requirements are met. In many
situations an overhaul of the system design is required.
There are a large number of well known design mistakes that are consistently made
by developers building these systems. Such issues have been documented in the form
of software design antipatterns [36]. Similar to software design patterns, which doc-
ument best practices in software development, software antipatterns document com-
mon mistakes made by developers when building software systems. However, as well
as documenting the mistake, antipatterns also document the corresponding solution
to the problem. Thus not only can they be used to identify issues in software systems,
but they can also be used to rectify these issues by applying the solution provided. A
more complete and detailed definition of software patterns and antipatterns is given
in sections 2.6 and 2.7 respectively.
3](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-15-2048.jpg)
![1.3 Thesis Contributions and Statement
The first major contribution of this thesis is an approach for the automatic detection of
design and deployment antipatterns in systems built using component based enter-
prise frameworks [125] [129] [130] [131] [132] [133]. This approach builds on current
performance tools by performing analysis on the data collected (i.e. run-time infor-
mation and component meta-data). The analysis reconstructs the system design and
identifies performance design flaws within it. The approach has been implemented
for the JEE technology in the form of the PAD tool, however it could potentially be
applied to other component based enterprise frameworks (e.g. CCM). This solution
has been successfully applied to both a sample and a real JEE application and has a
number of key advantages.
Firstly, it reduces and makes sense of the data collected by many of today’s perfor-
mance profilers. This work makes use of statistical analysis and data mining tech-
niques to summarise the data collected and to find patterns of interest that might
suggest performance problems. Thus, it takes the onus away from developers who
currently have to carry out this tedious task manually.
Secondly, while most of today’s performance tools tend to focus on identifying low
level hotspots and programming errors (e.g. memory leaks, deadlocks), this work
focuses on analysing the system from a performance design perspective. Since design
has such a significant effect on performance [43] it is essential that work is carried out
in this area.
Thirdly, unlike with many of today’s performance tools, problems identified are anno-
tated with descriptions of the issue detected, as well as a solution that can be applied
to alleviate the problem. This approach of identifying and presenting antipatterns to
developers helps them understand the mistakes that have been made, and the under-
lying reason as to why performance was affected. Developers can learn from using
our tool, and thus it may be less likely that the same mistakes are made in the future.
This approach also allows developers to easily rectify the situation by applying the
solution provided. In fact, the antipatterns presented provide a high level language
that developers and management alike can use to discuss such problems when they
occur.
The second major contribution of this work is a portable, low overhead, non-
intrusive, end-to-end run-time path tracer for JEE systems [128]. This is the first com-
pletely portable approach for collecting end-to-end run-time paths across all server
side tiers of a distributed JEE application. It is non-intrusive insofar as it does not re-
quire any modifications to the application or middleware source code. The monitor-
ing approach instead makes use of standard JEE mechanisms to intercept calls made
to the instrumented components. A run-time path [44] contains the control flow (i.e.
the ordered sequence of methods called required to service a user request), resources
5](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-17-2048.jpg)
![and performance characteristics associated with servicing a request. Such information
is utilised to detect antipatterns by our PAD tool. By analysing run-time paths one can
easily see how system resources are being used, how the different components in the
system interact and how user requests traverse through the different tiers that make
up the system. In fact these paths can also be used for Object Tracking, i.e. to monitor
particular objects’ life cycles across the different user requests. In this work we show
how run-time paths can be used to manually and automatically reverse engineer a
JEE application. We also show how the reconstructed design can be used for either
manual or automatic identification of performance design flaws. For example, the
PAD tool makes use of run-time paths to identify the (run-time) component relation-
ships, communication patterns and object usage patterns in a JEE system. Results are
given for this monitoring approach which show that it produces a low overhead on
the instrumented system and that it can be applied in a portable manner.
The third and final major contribution of this work is a technique for the identifica-
tion of interesting communication patterns in a collection of run-time paths [126].
More precisely, we have applied a data mining technique, Frequent Sequence Mining
(FSM) to identify sequences of interest (e.g. frequently repeating method sequences
and resource intensive loops) across a transactional database of run-time paths by us-
ing alternative support counting techniques. In this work we also discuss scalability
problems (in terms of both the algorithm runtime and the amount of data produced)
related to applying FSM to run-time paths and give solutions to these issues. We
show how the sequences identified, can be used to highlight design flaws in enter-
prise applications, that lead to poor system performance. The PAD tool makes use of
this analysis technique to identify interesting component communication patterns in
a JEE system that may indicate the presence of particular antipatterns.
Following the above contributions the fundamental thesis of this work can be stated
as follows:
Performance design and deployment antipatterns can be automatically detected in component
based enterprise systems by analysing run-time data and component meta-data.
1.4 Key Assumptions and Scope
The work in this thesis is focused on component based systems as defined in section
2.2. As such, it is highly likely that the source code of the application to be analysed
is not available in its entirety, as components may have been developed by third par-
ties. Thus we assume source code is not available for analysis of the system. For such
systems bytecode analysis may also be problematic due to security restrictions or li-
censing constraints. Instead, we assume that a running implementation of the system
to be analysed is available such that dynamic data can be collected and utilised for
analysis.
6](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-18-2048.jpg)
![We also assume that a realistic testing scenario is available which reflects how the sys-
tem will be used in production. We do not address the issue of how such testing sce-
narios can be obtained in this work, however, research in this area already exists. For
example Weyuker and Voklos have outlined an approach for the development of per-
formance test cases [176]. In this literature five typical steps required to develop per-
formance test cases are outlined. Alternatively, Ho et al. [92] propose an evolutionary
approach to performance test design based on their Performance Requirements Evolu-
tion Model. The authors claim that more precise and realistic performance tests can be
created incrementally during the development process through customer communi-
cation or performance model solving. In addition, agile development techniques such
as test-driven development [25] promote the design of test cases before developers
begin to code. Recently work has been presented which discusses how performance
tests can be incorporated into the test driven development process [96] allowing for
early availability of performance testing scenarios.
Our approach is applicable to applications built on component based enterprise
frameworks. However our research has thus far only been applied to synchronous
components, and has not, for example, been applied to message driven beans which
are asynchronous components in the JEE technology. Thus, our scope is limited to
synchronous applications. Our plans for future work outline how this approach could
potentially be applied to asynchronous components (see section 8.3).
7](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-19-2048.jpg)
![2.1 Performance of Software Systems
The performance of a software system has been described as an indicator of how well
the system meets its requirements for timeliness [154]. Smith and Williams [154] de-
scribe timeliness as being measured in either response time or throughput, where re-
sponse time is defined as the time required to respond to a request and throughput
is defined as the number of requests that can be processed in some specific time in-
terval. Furthermore they define two important dimensions to software performance
timeliness, responsiveness and scalability. Responsiveness is defined as the ability of
a system to meet its objectives for response time or throughput. The ability to con-
tinue to meet these objectives, as the demand on the system increases, is defined as
the systems scalability. The aim of performance engineering is to build systems that
are both responsive and scalable.
To date a vast amount of performance engineering research has focused on system
analysis through performance models. Performance models are created based on sys-
tem artifacts and various relevant estimates. Some of the most common performance
model classes are Queuing Networks (QN) (or extensions, such as Extended QN and
Layered QN), Stochastic Petri Nets (SPN), and Stochastic Process Algebras (SPA). Per-
formance models can be evaluated using simulation techniques or analytical methods,
in order to predict performance indices, such as throughput, response times, or re-
source utilization. A comprehensive survey of modelling approaches for performance
prediction is presented in [15].
However modelling today’s enterprise applications with a high degree of accuracy
can be a difficult and time consuming task. This results from the fact that these sys-
tems are often very large and complex and made up of black box components, the
internals of which are generally unknown (e.g. application servers). Performance
metrics required to populate performance models can thus not be easily obtained.
For enterprise applications accurate performance metrics can often only be obtained
through performance testing of a running system [58]. Recently Liu et al. [76] have
used a combination of performance modelling and benchmarking techniques to al-
low for population of performance models of enterprise applications. Their initial
results give accurate performance prediction for the small sample systems. A draw-
back of this approach is the lack of tool support which would allow for this technique
to be easily reused. From our experiences with large software houses, it seems that
performance modelling of enterprise applications is rarely performed. Most opt for
performance testing using available performance testing tools.
Work in the area of performance testing, however, has been very much lacking [176]
and thus performance testing, especially in the case of large enterprise applications,
can be a difficult task [76] [161]. This comes from the fact that today’s performance
testing tools are quite limited, insofar as they generally focus on simply collecting
10](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-22-2048.jpg)
![data from a running system (i.e. profiling) and presenting this data to the user. These
tools tend to focus on low level programming bugs and do not address many of the
issues that lead to poor system performance (e.g. design issues).
2.2 Component Based Software
2.2.1 Software Components
There are numerous definitions of what software components are or should be 1. To be
specific, for the purpose of this thesis, we use Szyperski’s definition of a software com-
ponent: ”A software component is a unit of composition with contractually specified
interfaces and explicit context dependencies only. A software component can be de-
ployed independently and is subject to composition by third parties” [157]. Defining a
software component as a unit of composition simply means that the purpose of a com-
ponent is to be composed with other components. A component based application is
assembled from a set of collaborating components. To be able to compose components
into applications, each component must provide one or more interfaces which provide
a contract between the component and its environment. The interface clearly defines
which services the component provides and therefore defines its responsibility. Usu-
ally, software depends on a specific context, such as available database connections
or other system resources being available. For example, other components that must
be available for a specific component to collaborate with. In order to support com-
posability of components, component dependencies need to be explicitly specified. A
component can be independently deployed, i.e. it is self-contained and changes to
the implementation of a component do not require changes to other components. Of
course, this is only true as long as the component interface remains compatible. Fi-
nally assemblers of component based applications, are not necessarily the developers
of the different components. That is, components can be deployed by third parties
and are intended to be reused.
This definition of a software component leaves many details open, for example, how
components interact, what language(s) can be used for their development, for what
platform. Component frameworks further define the notion of a component, by de-
tailing these issues.
2.2.2 Component Frameworks
The key goal of component technology is independent deployment and assembly of
components. Component frameworks are the most important step for achieving this
1Beyond Objects column of Software Development magazine, articles by Bertrand Meyer and
Clemens Szyperski, archived at http://www.ddj.com/
11](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-23-2048.jpg)
![aim [157]. They support components conforming to certain standards (or component
models) and allow instances of these components to be plugged into the component
framework. The component framework establishes environmental conditions for the
component instances and regulates the interaction between component instances. A
key contribution of component frameworks is partial enforcement of architectural
principles. By forcing component instances to perform certain tasks via mechanisms
under control of a component framework the component framework can enforce its
policies on the component instances. This approach helps prevent a number of classes
of subtle errors that can otherwise occur.
There are numerous component frameworks that exist today. Examples include EJB,
CCM, SOFA [140] and Fractal [37]. Each framework contains its own component
model, i.e. a set of features that components satisfy. Component models generally con-
form to either flat component models or hierarchical models. Flat component models
(e.g. EJB, CCM) define only primitive components whereby indivisible entities are di-
rectly implemented in a programming language. Hierarchical models (SOFA, Fractal)
also define composite components which are created via nesting of other components.
The research in this thesis aims at solving issues related to flat component models. In
particular we focus on EJB. EJB is part of a wider enterprise framework (Java Enter-
prise Edition) for building enterprise level applications (see section 2.3). EJB has been
selected since it is a well established technology that is currently used in industry to
develop enterprise applications. There is also a body of work detailing best practices
and bad practices for this technology (see sections 2.6 and 2.7). On the other hand the
hierarchical component models have mainly been used by the research community
and best practices in these areas are less well defined.
EJB is considered a contextual composition framework. Contextual composition
frameworks allow components to specify boundary conditions describing properties
that the runtime context must meet [166]. Composition performed by such frame-
works is based on the creation of contexts and placement of component instances
in the appropriate contexts. For example, a component framework for transactional
computation can be formed by supporting transactional attribution of components
(for example ”this component’s instances need to be used in a new transaction”) and
transactional enactment at the boundary of contexts. This approach can be used to cre-
ate frameworks for any other properties such as security, load balancing, management
etc.
Any component instance in a context can potentially be accessed from outside its con-
text. This context however, gets an opportunity to intercept all messages crossing the
context boundaries. Intercepting instances (e.g. objects) inside a context remains in-
visible to instances both external and internal to this context.
Current technology support for contextual composition includes Microsoft Transac-
12](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-24-2048.jpg)
![same specification, application developers can replace the driver that they are using
with another one without having to rewrite their application.
2.4 The Enterprise JavaBean Technology
The Enterprise Java Beans architecture is a component architecture for the develop-
ment and deployment of component based distributed applications. It is designed to
simplify and reduce the costs of the development and management processes of large-
scale, distributed applications. Applications built using this technology are capable
of being scalable, transactional, and multi-user secure. EJB provides the distributed
platform support and common services such as transactions, security, persistence and
lifecycle management. EJB also defines a flexible component model which allows for
components of different types that are suitable for specific tasks. Developers make use
of the different component types to implement the application business logic. Subse-
quently, EJBs are deployed and managed by EJB containers, as part of a JEE applica-
tion server. EJB containers provide middleware services and manage the EJB lifecycle
during runtime. These processes can be configured via XML documents, referred to
as EJB deployment descriptors. Physically EJB consists of two things [148]:
The specification7 which defines:
• The distinct ”EJB Roles” that are assumed by the component architecture.
• A component model
• A set of contracts: component-platform and component-client
A set of Java Interfaces:
• Components and application servers must conform to these interfaces. This al-
lows all conforming components to inter-operate. Also the application server
can manage any components that conform to the interfaces.
2.4.1 The EJB Roles
The EJB specification defines the following roles which are assumed by the component
architecture:
• Enterprise Bean Provider: The enterprise bean provider is typically an applica-
tion domain expert. The bean provider develops the reusable enterprise beans
that typically implement business tasks or business entities.
7The Enterprise Java Bean Specification version 2.0, http://java.sun.com/products/ejb/docs.html
15](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-27-2048.jpg)
![• Application Assembler: The application Assembler combines enterprise beans
into larger deployable application units.
• Deployer: The Deployer takes the ejb-jar files produced by either the Bean
Provider or the Application Assembler and deploys the enterprise beans con-
tained in the ejb-jar files in a specific operational environment. The operational
environment includes the EJB Server and Container.
• EJB Service Provider and EJB Container Provider: The container supplies an EJB
container (the application server). This is the runtime environment in which
the beans live. The container provides middleware services to the beans and
manages them. The server provider is the same as the container provider. Sun
has not yet differentiated between them.
• System Administrator: The system administrator is responsible for the upkeep
and monitoring of the deployed system and may make use of runtime monitor-
ing and management tools provided by the EJB server provider
2.4.2 The EJB Component Model
EJB is built on top of object technology (Java). An EJB component consists of a busi-
ness interface, an implementation class, a home interface and configuration settings
(defined in an XML deployment descriptor). All of these, except for the deployment
descriptor, are Java artifacts (i.e. classes or interfaces).
The EJB implementation class contains the bean business logic written by the Enter-
prise Bean Provider. The EJB implementation class is a Java object that conforms to a
well defined interface and obeys certain rules. The interface it conforms to depends
on the bean type. The rules are necessary in order for the bean to be able to run in a
container.
Access to the implementation class can be obtained using a the EJB home interface.
The home interface defines methods for creating, destroying and finding EJBs (i.e.
lifecycle methods). The home interface can either be local or remote. Local interfaces
allow access from clients within the same JVM whereas remote interfaces allow for
access from remote clients (e.g. on another JVM running on the same machine or on
a JVM running on a physically distributed machine). In fact an EJB component can
have both local and remote interfaces, however this is not recommended [161].
The bean implementation business methods are exposed through the business inter-
face. Similar to the home interface the business interface can be exposed locally or
remotely (or both).
An EJB component also requires configuration settings for deployment. These settings
are defined in a XML deployment descriptor. The information in the deployment
16](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-28-2048.jpg)
![The management of an EJB’s lifecycle is a complex process and differs from bean type
to bean type. Factors which influence the bean lifecycle management include the load
in the system and the container configuration settings 11. Figure 2.4 illustrates the
lifecycle of a stateless session bean.
When the container starts the application it creates a pool of bean instances. The pool
size can be determined by setting the value in the configuration settings. If a bean
instance is required by a client, an instance is assigned from the bean pool. If no in-
stances are available the container can create more instances until the pool has reached
its maximum size (which is also configurable). If the bean pool has already reached
its maximum size and there are still no instances available the client will be put in a
queue until an instance becomes available. The pool configuration settings can have
a major impact on the system performance and should be tuned according to the ex-
pected load on the system.
The lifecycle of a stateful session bean and of an entity bean are similar but slightly
more complicated than that of the stateless session (since they can both be passivated).
More details on these lifecycles can be found in the literature [148]. It is sufficient to
say for the purposes of this thesis that the configurations settings in relation to EJB
lifecycles can have a major impact on the system performance and need to be carefully
considered.
2.5 Software Architecture
A large number of definitions exist for the term software architecture 12. One of most
cited definitions is by Bass et al. [21] and states that:
”The software architecture of a program or computing system is the structure or struc-
tures of the system, which comprise software elements, the externally visible proper-
ties of those elements, and the relationships among them.”
Bass et al. [21] also outline a number of implications of this definition: Firstly, a soft-
ware architecture is essentially an abstraction since it embodies information about the
relationship between elements, and externally visible properties that are exposed to
other elements, but it omits internal element information or information that does not
pertain to the elements’ interactions. Secondly, the definition makes it evident that
systems can and do consist of more than one structure. Thirdly, it is implied by the
definition that every software system has an architecture since every system can be
shown to compose of elements and relationships between them. Fourthly, the exter-
nal behaviour of each element is part of the architecture and finally the definition is
indifferent as to whether the architecture for a system is a good or bad one.
11PreciseJava, http://www.precisejava.com/
12Software Engineering Institute, Carnegie Mellon, list of software architecture definitions,
http://www.sei.cmu.edu/architecture/definitions.html
22](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-34-2048.jpg)
![A software architecture is important for a number of reasons. Firstly it becomes a
vehicle for communication among the system’s stakeholders [21]. System stakehold-
ers are those concerned with the system (e.g. users, customers, software developers,
management etc.). A software architecture is a common abstraction of the system
and can serve as a lingua franca, i.e. an intermediate language that all stakeholders
can use to discuss various aspects of the system. The different stakeholders of the
system are often concerned with different system characteristics. An architecture pro-
vides a common language in which these different concerns can be expressed. Since
stakeholders can be interested in different system characteristics it is important for
the architecture to provide different views [51] that let them consider the architecture
from different perspectives. For example, a functional view might contain an abstrac-
tion of the different system functions and their relations whereas a code view may
give an abstraction of the code in terms of objects or classes (or higher level subsys-
tems or modules) and their relationships. Different stakeholders make use of different
views to analyse the architecture according to their needs. Typical views include a
functional view, a concurrency view, a code view, a physical view etc [52]. Kruchten
[104] introduced the 4+1 view model to describe a software architecture using five
concurrent views. Views are essentially a mechanism that allow for the separation of
concerns within the architecture allowing for the analysis of the architecture from dif-
ferent perspectives. Architecture description languages (ADLs) [50] can be utilised to
describe a software architecture. There have been many attempts to design such lan-
guages. However while some have been employed in real word projects none have
been widely adopted [21]. The literature [115] provides a comparison of ADLS.
Another important reason for system architecture is that it creates a realisation of early
design decisions and allows for system architects to analyse the suitability of these de-
cisions in relation to the system requirements (e.g. performance, security, flexibility)
[21]. These early design decisions manifested in the system architecture can not only
impact the quality attributes of the system but can also place constraints on the actual
system implementation i.e. some technologies may be more suitable for particular
architectures. The initial architecture can even have an impact on the organisational
structure of the team (or teams) building the application [21]. One of the earliest de-
sign decisions is often to choose a suitable architectural style. An architectural style
defines a vocabulary of components (e.g. clients, servers, databases) and connector
types (e.g. procedure calls, database protocols), and a set of constraints on how they
can be combined [152]. Architectural styles are found repeatedly in practice to address
similar sets of demands.
Finally, software architectures are also reusable assets that can be applied repeatedly
to other systems exhibiting similar requirements [21].
23](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-35-2048.jpg)
![2.6 Software Patterns
The current use of the term pattern in software engineering is derived from work by
Christopher Alexander [168] in the field of contemporary architecture. Alexander’s
notion of a pattern was adopted by a number of software engineering researchers [26]
[71] and became popular in this field mainly after work published by Gamma et. al
[72]. Gabriel 13 gives the following definition of a pattern: ”Each pattern is a three-part
rule, which expresses a relation between a certain context, a certain system of forces
which occurs repeatedly in that context, and a certain software configuration which
allows these forces to resolve themselves.” This definition is consistent with Alexan-
der’s original definition [168] which states that a ”pattern is a three part rule which
expresses a relation between a certain context a problem and a solution.” Alexander
expands his definition to say that a problem relates to a certain system of forces which
occurs repeatedly in a context and that the problem solution can be considered as a
certain configuration which allows these forces to resolve themselves. While patterns
have been documented for a number of different domains (such as patterns for con-
temporary architecture [168] or organisational patterns [54]) we are mainly interested
in software patterns. Software patterns are usually documented according to a pattern
template. Common templates for describing patterns include the Alexandrian form
[168] and the GoF form [72]. A given template contains a number of elements that
describe the pattern e.g. name, problem, context, forces, solution, examples, resulting
context, rationale, related patterns and known uses 14.
Buschmann et al. [42] document a number of properties or benefits of patterns. While
they focus on patterns for software architecture many of the properties hold for soft-
ware patterns in general e.g.:
• A pattern addresses a recurring problem that arises in specific situations, and
presents a solution to it [42].
• Patterns document existing, well proven experience. That is, they document
solutions learned through experience and avoid the need for less experienced
developers to ”reinvent the wheel” time and time again [72].
• Patterns provide a common vocabulary and understanding for design principles
[72]. Similar, to the way a software architecture can serve as a vehicle for com-
munication (see section 2.5 above), pattern names can become part of a design
language and can act as a lingua franca facilitating discussion of design issues
and their solutions [42].
• Patterns support the construction of software with defined properties [42]. Pat-
terns assist developers in meeting both functional and non-functional require-
13The Hillside Group, Pattern Definitions, http://www.hillside.net/patterns/definition.html
14Patterns and Software: Essential Concepts and Terminology, by Brad Appleton,
http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html
24](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-36-2048.jpg)
![ments since they can provide a skeleton of functional behaviour while at the
same time they can explicitly address non-functional requirements e.g. reuse-
ability, maintainability etc.
Software patterns can be documented at various levels of abstraction. For example,
Buschmann et al. [42] discuss patterns at three different levels of abstraction, i.e.,
architectural patterns, design patterns and coding patterns or idioms. Architectural
level patterns are concerned with system structure. They describe predefined sets of
subsystems, specify their responsibilities and include rules and guidelines for organ-
ising the relationships between them. Design patterns on the other hand tend to be at
the level of objects and classes (or micro-architectures) and are used for refining sub-
systems or components of a software system. In the literature [72] they are defined
as ”descriptions of communicating objects and classes that are customized to solve a
general design problem in a particular context.” Eden and Kazman have also distin-
guished between architecture and design stating that architecture is concerned with
non-local issues whereas design is concerned with local issues [62]. Coding patterns
or idioms are lower level patterns specific to a programming language [53].
Since their introduction in the area of object oriented software development [72], pat-
terns have been documented for a range of systems and technologies 15. For examples
pattern catalogs exits in areas such as enterprise systems [67] [93], embedded-systems
[142], telecommunication systems [171] to name but a few. Many technology specific
patterns (or idioms) also exist (e.g. for Java [80], Ajax [110] and Microsoft technolo-
gies 16). In fact pattern catalogs even exist with particular quality attributes in mind
(e.g. security [150], performance [154]). Alur et al. [7] provide a catalog of patterns for
the JEE technology which document best practices for the design and implementation
of JEE applications. Other literature in this area also exists [113] 17. The design of a
JEE application plays a major role in the overall system performance. For example, it
has previously been shown how the system design can influence a JEE system’s scal-
ability [43]. In fact it is well known and recent reports 18 19 also indicate that poor
system design is a major reason as to why JEE systems often fail to meet performance
requirements. Another reason why poor software design is particularly undesirable
is that unlike with lower level software bugs, for example, poor software design can
be particularly difficult to rectify late in development and as such can lead to major
project delays. Software design best practices documented in the form of patterns can
be used to help avoid design issues when developing JEE applications.
15Handbook of Software Architecture, http://www.booch.com/architecture/index.jsp
16Enterprise Solution Patterns Using Microsoft .NET, http://msdn2.microsoft.com/en-
us/library/ms998469.aspx
17The Server Side Pattern Repository, http://www.theserverside.com/patterns/index.tss,
18Ptak, Noel and Associates, ”The State of J2EE Application Management: Analysis of 2005 Benchmark
Survey”, http://www.ptaknoelassociates.com/members/J2EEBenchmarkSurvey2005.pdf,
19Jasmine Noel, ”J2EE Lessons Learned ”, SoftwareMag.com, The Software IT Journal, January, 2006.
http://www.softwaremag.com/L.cfm?doc=2006-01/2006-01j2ee
25](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-37-2048.jpg)
![2.7 Software Antipatterns
Antipatterns, first suggested by Koenig [101], have been defined by Brown et al. [36]
as: ”a literary form that describes a commonly occurring solution to a problem that
generates decidedly negative consequences.” The authors [36] also state that when
documented properly an ”antipattern describes a general form, the primary causes
which led to the general form; symptoms describing how to recognize the general
form; the consequences of the general form; and a refactored solution describing how
to change the antipattern into a healthier situation.” Software design antipatterns thus
provide the opportunity for developers to learn from past experiences. They docu-
ment software design mistakes that tend to consistently reoccur. However, as well
as documenting the mistake, antipatterns also document the corresponding solution.
Thus they allow developers to identify design issues in their system, and to rectify
these issues with the corresponding solution provided in the antipattern description.
Antipatterns are complementary to software patterns and often show situations where
patterns are misused. In fact, as technologies evolve often patterns can become stale
[60], i.e., what was once a best practice can in some instances become a bad practice.
Examples from the JEE technology include the caching with a Service Locator pat-
tern [7], which was recommended for J2EE 1.2 but is not recommended for J2EE 1.3
20. Another example is the Composite Entity pattern [7] which has become obsolete
since EJB version 2.x [61]. Figure 2.5 [36] shows the relationship between patterns and
antipatterns.
Software design antipatterns, like software design patterns, have been documented
at a number of different levels. For example Brown et al. [36] introduced a num-
ber of technology independent object oriented development antipatterns (as well as
higher level architectural and management antipatterns). Technology specific antipat-
terns have also been documented (e.g. Java [160], J2EE [161] [61]). Antipatterns for
systems built on service oriented architectures (SOA) have also been recently doc-
umented 21. As with software design patterns, some antipattern catalogs focus on
particular software quality attributes only. For example Smith and Williams have pre-
sented a number of performance related antipatterns [154], while Kis has presented
antipatterns focusing on security [99]. For the purpose of this thesis we focus mainly
on performance related antipatterns for enterprise systems. In particular we focus on
performance antipatterns related to design and deployment for JEE applications.
Similar to software design antipatterns, Fowler and Beck introduced the notion of
code smells [68]. Code smells are lower level symptoms of problems at the code level.
20B. Woolf. IBM WebSphere Developer Technical Journal: Eliminate caching in service locator
implementations in J2EE 1.3.,
http://www-128.ibm.com/developerworks/websphere/techjournal/0410 woolf/0410 woolf.html,
October 2004
21SOA antipatterns,Jenny Ang, Luba Cherbakov and Mamdouh Ibrahim, November 2005,
http://www-128.ibm.com/developerworks/webservices/library/ws-antipatterns/
26](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-38-2048.jpg)
![Figure 2.5: Patterns, Antipatterns and their Relationship
They may not necessarily be a problem but often their presence in the code indicate
that problems exist. Catalogs of code smells are available in the literature 22 23.
2.8 Performance Tools
When design issues lead to poor performance, developers require testing tools to iden-
tify why the system is performing poorly. Developers use performance testing tools
to try to understand system behaviour and to discover how their system makes use of
the system resources. Application level performance testing tools fall into two main
categories, i.e. workload generation tools and performance profilers.
2.8.1 Workload Generation
In order to evaluate the performance characteristics of an application under devel-
opment a realistic workload is required to mimic how the system would be utilised
by clients in a production environment. To achieve this a synthetic workload can be
automatically generated using a workload generator. Workload generators fall into
two main categories, traced based approaches and analytical approaches [17]. Traced
based approaches make use of server log files to characterise the workload of an ap-
plication, whereas analytical approaches are based on mathematical models which are
22A Taxonomy of Code Smells, http://www.soberit.hut.fi/mmantyla/BadCodeSmellsTaxonomy.htm
23Smells within Classes, http://wiki.java.net/bin/view/People/SmellsToRefactorings
27](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-39-2048.jpg)
![usually based on statistical methods [124]. There are advantages and disadvantages
associated with both approaches. For example, traced based approaches are consid-
ered relatively easy to implement and are based on activity from a known system.
However disadvantages relate to the fact that this approach treats the workload as a
black box and as such insight into the workload characteristics can be difficult to ob-
tain. Also it can be difficult to modify the workload to simulate future or alternative
conditions. Furthermore during development realistic logs may not be available to
base the traced based workload generation upon. Analytical approaches on the other
hand can be used to create synthetic workloads and do not suffer from the drawbacks
outlined above. However they can be more difficult to construct as an understanding
of the characteristics of the expected workload is required. The most commonly used
workload generators are the analytically based commercial profilers e.g. Apache’s
JMeter 24 or Mercury Loadrunner25. The literature [136] gives a representative subset
of the workload generators currently available in the open literature.
2.8.2 Profiling Tools
Next we explain what we mean by the term profiling and discuss the different ways
and levels of granularity in which profiling information can be collected (see section
2.8.2.1). We also give an overview of the different categories of profilers that are avail-
able for the Java technology (see section 2.8.2.2).
Profiling [169], is the ability to monitor and trace events that occur during run time.
This includes the ability to track the cost of these events, as well as the ability to at-
tribute the cost of the events to specific parts of the program. A profiler, for example,
may obtain information about what part of the program consumes the most CPU time,
or about the parts of the program which allocate the most amount of memory. Perfor-
mance profilers can be used in conjunction with load generators to monitor a running
system and obtain the required information for performance analysis. Profilers are
often described as either exact profilers or sampling based profilers [135] [30]. Ex-
act profiling also referred to as full profiling [64] or full instrumentation 26, captures
all events of a given type that are produced during program execution (e.g. method
invocations). Sampling based profilers on the other hand select a part of the entire
event population with the aim of determining the characteristics of the whole pro-
gram. Sampling usually involves selecting a subset of events for profiling based on
certain criteria (e.g. hot paths) or time intervals [64]. Exact profiling has the advantage
of being more precise than sampling but carries a higher performance overhead.
24Jakarta Apache JMeter http://jakarta.apache.org/jmeter/index.html.
25Mercury Loadrunner. http://mercury.com
26http://profiler.netbeans.org/docs/help/5.5/custom instrumetation
28](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-40-2048.jpg)
![2.8.2.1 Recording Information
Regardless of the profiling approach however, information must be recorded by the
profiling tool. Performance metrics can be recorded at different levels of granular-
ity. At the lowest level hardware counters can be utilised to obtain performance met-
rics from the underlying hardware on which the program executes [8] [10] [156] [88].
Hardware counters can be utilised to record events, such as instructions executed, cy-
cles executed, pipeline stalls, cache misses, etc. One of the main advantages of using
hardware counters is that an application can be profiled without the need to modify or
instrument it. Also the overhead associated with using hardware counters for profil-
ing is generally quite low [10]. A disadvantage of hardware counters is that they rely
on platform specific features and thus they are generally not portable across different
hardware. Another issue with this type of profiling is that the information may be
too low level for higher level program analysis. Information such as virtual memory
management requests, and signals caused by segmentation violations can be obtained
at the operating system (OS) level [88]. OS level information can be recorded by sys-
tem level tools 27 or libraries 28. In situations where hardware counters or OS level
information is unavailable, or the information they produce is undesirable, informa-
tion can be obtained at a higher level. For today’s enterprise Java applications such
information can be recorded at a number of different levels i.e. at the JVM level, the
middleware level or the application level.
JVM level information is generally recorded by either instrumenting the JVM or by
using an agent-based approach that requests notification of events from the virtual
machine. The former very often requires access to the JVM source code such that it
can be modified to record the information required [12] [33]. A disadvantage of this
approach is that it requires an understanding of the complex JVM internals. Also this
approach generally ties the user to a particular JVM and is thus not portable. One of
the main advantages of instrumenting the JVM is that access to JVM level information
is not restricted as with the agent-based approaches.
Agent based approaches have been made popular through standard interfaces that
allow for a profiler agent to request performance related information from a running
JVM. The Java Virtual Machine Profiler Interface (JVMPI) [169] is an example of such
an interface (see figure 2.6). The JVMPI is a two-way function call interface between
the JVM and an in-process profiler agent. The profiler agent is responsible for commu-
nication between the JVM and the profiler front end. The profiler agent can register
with the JVM to be notified when particular events occur and upon notification can
call back into the JVM to obtain additional information. For example a notification
may be received when a method is entered (or exited) and a call back may be made to
27Performance Monitoring Tools for Linux, David Gavin Jan, 1998,
http://www.linuxjournal.com/article/2396
28Windows Management Instrumentation, http://www.microsoft.com/whdc/system/pnppwr/wmi/default.mspx
29](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-41-2048.jpg)
![Figure 2.6: JVMPI Architecture
obtain the current stack trace at this point. The main advantage of standard interfaces
is that they are implemented by different JVM vendors. While the JVMPI was an ex-
perimental interface for Java 1.2, it was implemented by most JVM vendors and effec-
tively became standard. Thus profilers built using JVMPI are portable across different
JVM implementations. A disadvantage of standard interfaces is that they are fixed
interfaces and, as such, can only enable predefined types of profiling [135] or event
notifications. Another major issue with JVMPI in particular was that when using a
JVMPI agent the JVM could not run at full speed and was required to run in a debug-
ging mode. As such this profiling approach was not generally suitable for production
systems. Another major draw back of the JVMPI approach was that notifications could
not be tailored to profile selectively. If, for example, the profiler agent requested to be
notified on method entry events, all method entry events would be reported to the
agent. This lead to high overhead performance tools. The Java Virtual Machine Tools
Interface (JVMTI) 29 will replace JVMPI in Java 1.6 (the JVMPI is currently available
in Java 1.5). While at an architectural level the JVMTI looks similar to the JVMPI (i.e.
it also consists of call back functions and a profiler agent) it is quite different and im-
proves upon many of the limitations of JVMPI. Firstly, it allows for the JVM to run
at full speed and does not require it to run in debug mode. It also promotes the use
of bytecode instrumentation for many of the event based capabilities of the JVMPI.
Using bytecode instrumentation one can be more selective when profiling the appli-
cation and can instrument only the parts of the application that require analysis. This
avoids the ”all or nothing” approach of the JVMPI and thus reduces the profiler over-
head. In fact the JVMTI allows for dynamic bytecode instrumentation 30, which means
that the application can be instrumented as it runs. An issue that remains, however,
with both JVMPI and JVMTI is that they are native interfaces and while the profil-
ing agents (which must be written in native code) are portable across different JVMs
29The Java Virtual Machine Tools Interface, http://java.sun.com/j2se/1.5.0/docs/guide/jvmti/jvmti.html
30Ian Formanek and Gregg Sporar, Dynamic Bytecode Instrumentation A new way to profile Java
applications, December 15, 2005, http://www.ddj.com/dept/java/184406433
30](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-42-2048.jpg)
![they are not portable across different platforms. The java.lang.instrument interface 31
is another standard interface (as of Java 1.5) which allows for the interception of the
JVM classloading process through a non-native agent. Since the agent is non-native
it is portable across different platforms. Java.lang.instrument allows for the agent to
monitor the classloading process and to instrument the classes such that they can call
back into the agent libraries.
Recording performance information for Java applications at the middleware level can
also be achieved through standard mechanisms. Java Management Extensions (JMX)
technology 32 is a standard technology that allows for the management of Java re-
sources through so called MBeans. MBeans, also known as managed beans, are Java
objects that are used to represent and manage JMX resources. A JMX resource can be
any application, device or Java object. In order to manage an MBean it must be reg-
istered with a JMX agent. JMX agents directly control registered MBeans and make
them available to remote management applications. While the JMX technology can
potentially be used to manage a wide range of different resources it has been heavily
used, in particular, to manage the resources of JEE application servers. In fact, ac-
cording to the JEE Management specification (Java Service Request 77 33) application
servers are required to expose this data through the JMX technology. Profilers built us-
ing JMX can collect data on the state of the different system resources (e.g. object pool
sizes, thread queues, database connectivity information) and because JMX is standard
they are portable across the different application server implementations.
Non standard hooks or mechanisms have also been used to collect information at the
middleware level. Often middleware vendors provide these mechanisms to enhance
the capabilities of their products. For example, IBM provides non standard features
for the Websphere application server in the form of the Performance Monitoring In-
frastructure (PMI). PMI is available for the Websphere application server and allows
for the collection of performance information on the server resources. The informa-
tion can be exposed to performance profiling tools through a number of different in-
terfaces 34. The main issue with non-standard features, that allow for the collection
of performance information, is that they are not portable across different vendors’ im-
plementations of the middleware, and thus can result in vendor lock in.
Where the information required is not available through standard or non-standard
features the middleware itself can be the subject of instrumentation. This can be
achieved by manually modifying the source code if it is available [45] [105]. However
31J2SE 5.0 in a Nutshell, Calvin Austin, May 2004,http://java.sun.com/developer/technicalArticles/releases/j2se15,
32Java Management Extensions Technology,http://java.sun.com/javase/technologies/core/mntr-
mgmt/javamanagement/
33Java Service Request 77, J2EE management, http://www.jcp.org/en/jsr/detail?id=77
34Srini Rangaswamy, Ruth Willenborg and Wenjian Qiao, IBM WebSphere De-
veloper Technical Journal: Writing a Performance Monitoring Tool Using Web-
Sphere Application Server’s Performance Monitoring Infrastructure API, 13 Feb 2002,
http://www.ibm.com/developerworks/websphere/techjournal/0202 rangaswamy/rangaswamy.html
31](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-43-2048.jpg)
![as with JVM modification, instrumenting the middleware requires an understanding
of the middleware behaviour which can be extremely complex. Also modification of
the application server source code is not a portable approach.
Instrumenting at the application level to monitor performance can be achieved in a
number of different ways. The most basic way to instrument an application is to mod-
ify the source code of the application to log the required information. IBM encourages
instrumentation of the application at the time of development such that traces can be
recorded through the Application Response Measurement API 35 36. This approach
can be cumbersome and can require a major effort during development. Another ma-
jor drawback of this approach is that it can not be applied after development if source
code is not available.
More automated approaches have also been presented that remove the burden of hav-
ing to manually modify the source. For example the Gprof profiling tool [79] allows
for automatic instrumentation of an application at compile time for applications writ-
ten in C or C++. It relies upon specific support from the gcc compiler to insert instru-
mentation at the start of each method. Similarly binary rewriters and program trans-
formation systems can be used to automate the instrumentation process [22] [134]
[155] [170]. It has been argued [135] that such approaches are cumbersome to use
since they operate at a low level. Binary rewriters for example require further code
to be written while program transformation systems require the creation of complex
rewrite rules.
Application level instrumentation for Java systems can be achieved using bytecode
instrumentation (BCI). BCI allows for instrumentation of the compiled bytecodes and
thus access to the application source code is not required. JVMPI and JVMTI allow
for the instrumentation of bytecode before load time and at load time. Similarly the
java.lang.instrument interface allows for BCI at load time. The JVMTI even allows
for BCI to occur at run-time (i.e. dynamic BCI). Although these interfaces provide a
mechanism for the modification of bytecodes, the actual instrumentation on the class
file is generally performed through a third party BCI library (e.g. [57]). Developers
could alternatively make use of Aspect Orient Programming (AOP) libraries 37 38 to
instrument their applications. AOP libraries are at a higher level than the BCI libraries
and allow for modification of Java classes without a detailed knowledge of the byte-
code class structure. It has recently been shown how the AspectJ AOP library can be
effectively applied to instrument Java applications for profiling [135]. AspectJ allows
for both compile time and load time instrumentation and in fact makes use of BCI
libraries to instrument the application. Both BCI and AOP approaches are used by
35IBM InfoCenter, ARM 4.0 API concepts,
http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp?topic=/eicaw/section1.htm
36The Open Group, Application Response Management Standard,
http://www.opengroup.org/tech/management/arm/
37AspectJ, http://www.eclipse.org/aspectj/
38Aspectwerkz, http://aspectwerkz.codehaus.org/
32](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-44-2048.jpg)
![a number of today’s profiling tools to automatically instrument and profile running
applications (e.g. [78] 39).
2.8.2.2 Java Profilers
Independent of the instrumentation techniques (e.g. event based agents, BCI, AOP),
the most commonly used profilers for the Java technology can generally be split into 3
main categories: basic profilers, graphically enhanced J2SE profilers and JEE profilers.
Next we discuss each of the different categories. We begin by discussing the most
basic profilers and finish with the most sophisticated.
The are a large number of freely available (and in many cases open source) basic pro-
filers. The profilers in this category collect metrics and present them in a very basic
format. An example is the hprof profiler which comes as standard with the sun JVMs
40. While the hprof profiler output looks quite basic, it makes use of the JVMPI/JVMTI
interfaces (depending on the tool version) and can collect a vast amount of informa-
tion made available through these interfaces. Basic profilers, such as hprof, are most
suited for analysing small scale single user applications since the information output
is presented in a basic format. In situations where a vast amount of information is pro-
duced, as with multi-user applications, basic profilers are generally unsuitable since it
can be difficult to manage and understand the information produced.
A more sophisticated category of profilers are the (generally commercial) more graph-
ically enhanced J2SE profilers. The profilers generally collect the same information as
the basic profilers but present the data in a more user friendly manner. Examples
include the Jprobe 41 and Optimizeit 42 profilers. These profilers tend to capture infor-
mation at the Java class level (very often making use of JVMPI/JVMTI) and provide
views that help with performance analysis. For example these profilers generally pro-
vide heap analysers that help with memory leak detection and thread analysers for
deadlock detection. Most also provide call graphs which show object dependencies.
There are a large number of profilers that exist in this category 43. While these tools
are effective for single user applications, as with the most basic tools they are limited
when profiling larger multi-user enterprise applications. One of the main problems is
that they monitor the system at the class level. However, for enterprise systems the
number of classes loaded by the JVM can be in the order of thousands. The classes
can generally be broken down into the following categories: application level classes
(written by the application developers), middleware classes (corresponding to con-
tainer services) and lower level Java library classes. A major issue is that, while de-
39JRat the Java Runtime Analysis Toolkit, http://jrat.sourceforge.net/index.html
40Kelly O’Hair, HPROF: A Heap/CPU Profiling Tool in J2SE 5.0, November 18, 2004,
http://java.sun.com/developer/technicalArticles/Programming/HPROF.html
41Jprobe, http://www.quest.com/jprobe/
42Optimizeit, http://info.borland.com/techpubs/optimizeit/
43Java Performance Tuning Website, http://www.javaperformancetuning.com/resources
33](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-45-2048.jpg)
![velopers are generally interested in obtaining information on their own code, it can
be very difficult for developers to distinguish their code from lower level middleware
and library calls. Another issue related to profiling enterprise applications is that of-
ten these applications are distributed and made up of a number of different servers.
The (J2SE) profilers in this category do not profile the application across JVMs and
thus developers need to spend time correlating data from a number of profiler logs in
order to understand the entire system.
JEE profilers such as Performasure 44 or JProfiler 45 overcome this issue of muddling
middleware and application level calls by presenting data at a higher level of abstrac-
tion. Many of these tools also profile across JVM’s and give a transaction centric view
of the applications. The JEE profilers are also generally known as Application Perfor-
mance Management tools 46, since they are usually suitable for managing systems in
a production environment.
The biggest problem with the current profiling tools, from the most basic to the
most sophisticated is that they do not perform any meaningful analysis on the data
collected. The most sophisticated tools merely perform statistical analysis on per-
formance metrics and give average resources consumed by the different compo-
nents/classes. When profiling large enterprise applications the amount of informa-
tion recorded can be truly overwhelming. It can be difficult and time consuming for
developers that have to analyse the data produced by these tools in search of partic-
ular problems. Furthermore even when developers manage to identify issues often
they are unsure as to how to go about solving them. The work in this thesis comple-
ments such profiling tools. By performing different analyses the data they produce
performance design and deployment antipatterns can be automatically identified.
Another problem with the current tools is that they tend to focus on identifying low
level performance bugs that exist in the system. For example the more sophisticated
tools provide views that help with memory leak detection and deadlock detection.
The tools, however, do not focus on analysing the system from a design perspective.
Thus design flaws can easily go unnoticed. While identifying and fixing low level
bugs in the system will very often improve the system performance, it is common
that this is not enough to meet performance requirements. Often, an overhaul of the
system design is required to improve performance significantly [43].
44Performasure, http://www.quest.com/performasure/
45JProfiler, http://www.ej-technologies.com/products/jprofiler/overview.html
46Jack Shirazi, Application Performance Management, July 2004,
http://www.javaperformancetuning.com/articles/apm1b.shtml
34](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-46-2048.jpg)
![2.9 Reverse Engineering
The term reverse engineering covers a wide range of activities [163] including design
recovery, architecture recovery, feature analysis, impact analysis, clone detection etc.
For the purpose of this research we are mainly interested in design and architecture
recovery.
To analyse the system design, developers can make use of design documentation.
However unfortunately it is common that design documents are never created, or
are created early in the development process and become out of date as the system
evolves. Furthermore, with agile development processes such as extreme program-
ming, documentation is not generally maintained throughout the development pro-
cess [24]. In such circumstances reverse engineering can be used to extract the design
from the system such that a representation of the design is available.
Reverse engineering has been defined by Chikofsky and Cross [48] as ”the process of
analyzing a subject system to:
• Identify the system’s components and their interrelationships and
• Create representations of the system in another form or at a higher level of ab-
straction.”
Earlier approaches to reverse engineering mainly relied on static analysis (e.g. [122]
[143] [81] [158]). Static analysis techniques generally extract information from the
source code (such as component dependencies) without the need to execute the ap-
plication. In fact a static call graph extractor has recently been developed for the EJB
technology [165]. There are a number of issues related to applying source code/static
analysis to enterprise applications. Firstly, the entire source code of enterprise appli-
cations is often not readily available. This can occur as a result of outsourcing or from
using third party components. This problem is however overcome in the approach
presented in the literature [165], by extracting a static model by analysing the byte-
code of the application. Although if components have been developed by third parties
security or license issues may not allow for this approach to be applied. Secondly, and
more importantly, static models show all possible dependencies that could potentially
be present during the system execution. With enterprise applications the number of
possible dependencies can be very high and many of the potential dependencies may
never actually occur during execution. Thirdly, as claimed by the authors in the liter-
ature [149], there is an inherent mismatch between static, code-based structures (such
as classes and packages) and the runtime structures that are the essence of most sys-
tem descriptions. In fact, the actual runtime structures may not even be known until
the program executes (e.g., Dynamic Linked Libraries not under direct control of the
implementers may be dynamically loaded). The authors go on to claim that determin-
ing the actual runtime architectural configuration of a system using static analysis is,
35](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-47-2048.jpg)
![in general, undecidable.
To overcome this problem dynamic analysis can be applied to extract run-time depen-
dencies from the application. The advantages of dynamic analysis for reverse engi-
neering purposes has been well documented [146]. For example, dynamic approaches
can be more precise, insofar as only dependencies that are actually recorded during
execution exist in a dynamic model. Dynamic analysis also limits the scope of investi-
gation, similar to program slicing [23]. Dynamic analysis is also particularly useful in
relation to analysing the system from a performance perspective, since, performance
metrics can be obtained from the running system. This performance related informa-
tion can not be obtained easily when analysing the system source code.
Dynamic analysis techniques [147] [177] have generally made use of program traces
which can be analysed to extract dependencies. A major obstacle for performing this
analysis on large distributed enterprise systems is that such traces can be difficult to
obtain. Enterprise applications generally comprise of a large number of components
running on physically distributed servers. Thus obtaining an execution trace for these
systems is not a trivial task.
Recently Chen et al. [45] introduced an approach for the collection of run-time paths
in JEE systems in the form of the Pinpoint tool. A run-time path [44] contains the
ordered sequence of methods called required to service a user request, and thus con-
tain information required for dynamic reverse engineering. A major drawback of the
Pinpoint approach is that it is intrusive and can only be applied for a particular appli-
cation server implementation. In section 4.2.5.2 we discuss the pinpoint tool in more
detail and give related work in this area. We also show how this information can
be obtained in non-intrusive portable manner such that it can be applied to any JEE
application running on different middleware implementations.
2.10 Design Pattern and Antipattern Detection
Research in the area of software pattern detection has spanned the past decade. Com-
parison of pattern detection methods is often performed with respect to criteria such
as flexibility, efficiency and accuracy [139]. However efficiency and flexibility are of-
ten unimportant if accuracy can not be guaranteed. Accuracy in relation to software
pattern detection is usually defined in terms of precision and recall which are both
defined in terms of true/false positives/negatives [139]. True positives tp are defined
as pattern instances detected that exist in the system under test, while false positives
fp are defined as pattern instances detected that do not actually exist in the system.
On the other hand, true negatives tn are defined as pattern instances not detected that
do not exist in the system, while false negatives fn are defined as pattern instances not
detected that actually do exist in the system. Petterson et al. [139] define precision as
36](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-48-2048.jpg)
![P =
tp
tp+fp
and recall as R =
tp
tp+fn
.
Over the past decade a wide number of software pattern detection approaches have
been presented and generally use either static analysis or a combination of static and
dynamic analysis. The static approaches (e.g. [5] [103] [98] [153] [151] [13]) aim to
extract structural information such as class inheritance, interface hierarchies, associ-
ations etc. Most approaches analyse the source code of an application [103] [11] [16]
[65] [98] [5] [13] [172], although some have used graphical representations as input
[56] [27]. The application and design patterns are generally expressed using structural
representations, and matching is performed using either meta-programming, graph
re-write systems, logic programming, relational algebra, or parsing techniques [138].
According to the literature [138], the matching techniques mainly impact the efficiency
and do not greatly effect accuracy. An alternative technique is presented by Buchli and
Nierstrasz [39] who use (static) formal concept analysis (FCA) to identify the presence
of groups of classes which instantiate a common, repeated pattern. Patterns identi-
fied are analysed to find a match with known patterns. Tonella and Antoniol [164]
have presented a similar approach based on FCA. The main advantage of approaches
based on FCA is that they do not require a knowledge base of design pattern rep-
resentations. The combined (static and dynamic) approaches generally make use of
dynamic behavioural information to improve the accuracy of static analysis by reduc-
ing the levels of false positives [89] [90] [91] [174] [175] [137]. It has been argued that
static analysis approaches are limited to the recognition of static parts of design pat-
terns, and as a result, the dynamic behavior of patterns is disregarded, which leads to
high levels of false positives during recognition [174]. Similarly software metrics have
also been utilised to reduce the numbers of false positives [11]. The literature [139]
reports on the accuracy of the state of the art in design pattern detection techniques.
Similarly, Vok´ac [172] gives a comparison of the accuracy of a number of detection
approaches.
Efficiency is another important attribute of pattern detection approaches. With pattern
detection the size of the exploration space for large software systems can be a major
issue. A combinatorial explosion can occur due to the great number of system classes
and the multiple roles that classes can play in a specific design pattern. As a result
efficiency issues have often been reported in applying detection techniques in practice
(e.g. [13] [70] [138]). A number of papers have been presented with particular focus on
reducing the detection search space. For example, Niere et al. [123] increase efficiency
by replacing a number of constraints with a smaller number of weaker constraints.
This approach allows more variants of a particular pattern to be identified during the
detection process and increases the number of false positives. They address this issue
by attaching a ranking to each pattern candidate retrieved. Tsantalis et al. [167] take
an alternative approach and partition the system into clusters of hierarchies. They
then apply a similarity scoring algorithm to the smaller subsystems, rather than to the
entire system. Petterson et. al [138] present a method to increase efficiency based on
37](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-49-2048.jpg)
![planar graphs. Planar graphs are described as graphs drawable in two-dimensions
without any crossing edges. Both the structure of a program and a design pattern
can be represented as a graph. The detection of a pattern involves deciding if the
pattern graph is isomorphic to a subgraph of the program graph. This is known as the
subgraph-isomorphism problem and is NP-Complete for general graphs. Petterson et
al. use a filtering strategy to remove unneeded edges from program graphs with the
aim of producing planar graphs. Detecting patterns in planar graphs can be solved in
linear time.
Less work has been undertaken in the area of design antipattern detection. How-
ever many of the techniques for software pattern detection outlined above can also
be applied to detect design flaws. In most of the approaches discussed above the
patterns and application are described in terms of structural (and in some instances
behavioural) representations which are analysed to identify instances of the pattern.
Design flaws can also be described in this way and thus the same approaches can be
used to identify them. For example, work by Gueheneuc et al. [83] shows how defects
can be identified by creating abstract models of known design patterns. The models
(representing entities and relationships) are used to identify similar sets of entities in
the source of the application, which represent distorted design patterns. Similarly,
the work presented by Ciupke [49], extracts a model from the source code to iden-
tify design issues. The model defines the different entities and relationships that may
occur in the design of the program. The information contained in the model is con-
verted to Prolog facts. Queries formulated as Prolog rules examine this information
and identify design issues. An approach by Moha et al. has suggested the use of
formal concept analysis (which has also been applied for pattern detection) and soft-
ware metrics, to identify and correct a well known deign antipattern (The Blob) [117].
Metrics have also been applied by Marinsecu [114] for the purpose of design flaw de-
tection. Munro [121] has applied metrics for the detection of bad smells. Alternatively,
the ARBRIUM antipattern detection tool [63] has been used to detect antipatterns in
use case models by making use of the Object Constraint Language (OCL). Antipat-
terns can be expressed using OCL and the integrity of the models are assessed.
The techniques outlined above, for both pattern and antipattern detection, mainly rely
heavily on static source code analysis. Even where dynamic analysis is applied it is
generally used only to add behavioural information to a static model of the system
such that detection can be more precise. There is a major issue applying such tech-
niques to component based systems, since source code is often not available, or at least
may not be available for analysis due to security or licensing constraints. Also, when
analysing components at the component level we are not interested in their internal
structural make up (e.g. inheritance relationships) since components are considered
as black boxes and we may not be aware of their internal workings. We are more in-
terested in how they communicate with each other and the underlying middleware.
However since components can be subject to dynamic binding this information often
38](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-50-2048.jpg)
![can be difficult to obtain statically. True inter-component dependencies can in many
cases only be determined at run-time. Furthermore, since we are interested in per-
formance antipatterns we need to be able to reason about how the system resources
are being used. Such performance information can generally not be obtained by static
analysis alone.
The goal of this research is to show how performance design flaws can be identified
at the component level in enterprise applications using run-time data and component
meta-data. The flaws we detect are presented to the application developer with sug-
gested refactorings and contextual information. Our aim is to bring a greater level of
understanding to developers in relation to such problems than is currently available
from today’s enterprise performance tools.
2.11 Knowledge Discovery in Databases and Data Mining
When monitoring multi-user enterprise applications, the amount of data produced
can be truly overwhelming 47. Currently a major effort is often required by devel-
opers who must correlate vast amounts of data in order to identify problems in their
systems. Such tasks often require repetitive labourious work and should ideally be
automated so developers and system testers can put their minds to more useful tasks.
The field of Knowledge Discovery in Databases [66] combines techniques from pat-
tern recognition, statistics, machine learning, databases, and visualization to auto-
matically extract concepts, concept interrelations, and patterns of interest from large
databases. The basic task is to extract knowledge, or information, from lower level
data (databases). The basic tools used to extract patterns from data are called Data
Mining methods [87], while the process of surrounding the usage of these tools (in-
cluding pre-processing, selection, and transformation of the data) and the interpreta-
tion of patterns into ”knowledge” is the KDD process. This extracted knowledge is
subsequently applied to support human decision-making, e.g., prediction and classi-
fication tasks, summarize the contents of databases, or explain observed phenomena.
The application of KDD systems alleviates the problem of manually analyzing the
large amounts of collected data which decision-makers face presently. KDD systems
have been implemented and are currently in use in finance, fraud detection, market
data analysis, astronomy, diagnosis, manufacturing, and biology. The KDD process
(figure 2.7) is made up of the following sub processes: selection, pre-processing, trans-
formation, data mining and interpretation.
The proposed antipattern detection approach (see chapter 3) in this thesis can be con-
sidered as an instance of the KDD process. The monitoring stage collects data and
forwards it on for analysis. The analysis stage implements pre-processing, transfor-
47Michael Baum, ”Migrating J2EE Applications from Development to Production Is it becoming more
complex?”, Java Developers Journal, November, 2005, http://jdj.sys-con.com/read/152269.htm
39](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-51-2048.jpg)
![Figure 2.7: The KDD Process
mation, and performs the data mining techniques. The detection stage interprets the
discovered patterns and presents the discovered knowledge. Data mining techniques
from the areas of frequent pattern mining and clustering are applied in our work.
2.11.1 Frequent Pattern Mining and Clustering
The aim of frequent pattern mining methods is generally to find all frequent itemsets,
i.e. those that appear in at least a given percentage of all transactions. Agrawal et al.
first introduced a solution to this problem in the form of the Apriori algorithm [4]. The
frequent item set mining problem is usually formulated as follows:
Let I = i1, i2, ..., im be a set of items. Let D be a set of transactions T where each
transaction is a set of items such that T ⊆ I. A unique identifier TID is given to each
transaction. A transaction T is said to contain X, a set of items in I, if X ⊆ T. We can
also say that the itemset X appears in the transaction T. The itemset X is said to be
frequent if it appears in at least a given percentage of all transactions.
The Apriori algorithm is based on the principle: an itemset can only be frequent if
all its subsets are also frequent. Since its introduction by Agrawal et al. [4] Frequent
Itemset Mining (FIM) has received much attention. Within the past decade, hundreds
of research papers have been published presenting new algorithms or improvements
on existing algorithms to solve these mining problems more efficiently [75]. Most
of these papers present algorithms which are proposed as being faster than previous
existing algorithms. Efficient techniques in the field of data mining are certainly im-
portant, considering the recent advances in computing, communication and digital
storage technologies which today make it possible to easily store massive volumes of
data. However there is a real need to show how such algorithms can be applied to
different domains to extract interesting and useful information, since it is irrelevant
how efficient an algorithm is, if it can not be put to a meaningful task.
In this research, we show how a generalisation of FIM, Frequent Sequence Mining
(FSM) [3] can be applied to data collected from monitoring enterprise applications to
identify patterns of interest that can be used to identify design flaws in the system.
The FIM and FSM problems are discussed in more detail in section 5.4.1.
40](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-52-2048.jpg)
![A second data mining technique, clustering [87], is also applied in this research. Clus-
tering is an unsupervised learning problem, whereby data is divided into groups of
similar objects. A cluster consists of a group of objects that are similar between them-
selves and dissimilar to objects of other groups. In section 5.5 we show how clustering
can be used to effectively summarise data produced during monitoring enterprise ap-
plications. A comprehensive overview of current clustering techniques in the field of
data mining can be found in the literature [28].
41](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-53-2048.jpg)
![Figure 3.1: PAD Tool Overview
3.1 Approach for the Automatic Detection of Performance
Antipatterns
The approach presented in this thesis, for the automatic detection of performance
design and deployment antipatterns, can be divided into 3 main modules (see fig-
ure 3.1): a monitoring module, an analysis module and a detection module. Our
work is concerned with identifying known antipatterns i.e. antipatterns that have
been already documented (e.g. [161] [61]). Our work is not concerned with pat-
tern/antipattern mining which is the task of identifying new, previously unknown
patterns/antipatterns in a systems 1.
The monitoring module is responsible for collecting information on the running sys-
tem. Information is collected on the application components, and on the underlying
middleware such that a complete picture of the running system can be obtained and
recreated. Information collected is forwarded to the analysis module where analysis
is performed off-line.
The analysis module is responsible for extracting a run-time design model of the sys-
tem from the monitoring data. A system’s run-time design can be defined as an in-
stantiation (or instantiations) of a system’s design decisions which have been made
1Patterns and Software: Essential Concepts and Terminology, by Brad Appleton,
http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html
44](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-56-2048.jpg)
![during the development process. A run-time design model captures structural and
behavioural information from an executing system. It contains information on the
components that are executed, as well as the dependencies and patterns of communi-
cation between components that occur at run-time. Figure 3.2 gives a run-time design
meta model. The meta model describes the different relationships that are captured
in a run-time design model. For a typical component based system (e.g. a JEE sys-
tem) the run-time design model will contain information such as: dynamic compo-
nent dependencies, run-time paths (see section 4.2.2 for a definition), communication
patterns and interactions between components and container services. A run-time
design model can also be enhanced with related performance metrics (e.g. method
memory consumption). The analysis module is responsible for populating the run-
time design model by extracting the information from the monitoring module output.
For example, during analysis component dependencies are identified and container
services reconstructed. Also during analysis data mining algorithms are applied to
identify relationships and patterns of interest in the monitoring data (e.g. frequent se-
quences). The analysis module is also responsible for summarising the data collected
during monitoring. The summarised data can be captured in the model to give a
concise overview of the data collected during monitoring. For example, performance
characteristics of components can be summarised using statistical analysis and added
to the run-time design model. The run-time design meta model is described in more
detail in section 5.6.
The output from the analysis module is a run-time design model of the system. The
extracted design model is analysed by the detection module. The detection module is
responsible for identifying any pre-defined antipatterns that exist in the model. An-
tipatterns are pre-defined and stored in an antipattern library. The solution can be
used to rectify any problems found. This approach has been realised for the JEE
technology in the form of the Performance Antipattern Detection (PAD) Tool. Any
antipatterns detected by the tool are presented with contextual information and the
corresponding antipattern solution.
3.1.1 Research Methodology and Validation Criteria
Four main research methods have been outlined for software engineering [74], i.e., the
scientific method, the engineering method, the empirical method and the analytical
method. The methods first presented by Basili [20] in the software engineering context
have been described by Wohlin et al. [178] as follows:
• ”The scientific method: The world is observed and a model is built based on the
observation, for example, a simulation model.
• The engineering method: The current solutions are studied and changes are pro-
posed, and then evaluated.
45](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-57-2048.jpg)
![Figure 3.2: Run-time Design Meta Model
• The empirical method: A model is proposed and evaluated through empirical
studies, for example, case studies or experiments.
• The analytical method: A formal theory is proposed and then compared with
the empirical observations.”
Traditionally software engineering research has made use of the analytical methods
[74], the engineering method has been mainly used in industry and the scientific
method has been used mostly in applied areas such as simulating telecommunica-
tion networks [178]. More recently, however there has been a call for more empirical
research in the field of software engineering [162] [74] [144] [100].
A number of authors have suggested that rigorous experimentation is required to
evaluate new technologies and their effects on organisations, processes and products
[100]. Wohlin et al. [178] suggest three main empirical strategies that can be carried
out, i.e. experiments, case studies, and surveys. Experiments are often referred to as
”research in the small” [100] since they are usually run in a laboratory environment
and are generally small in scale. The aim of an experiment is to manipulate one or
more variables while controlling the remaining variables at fixed levels. The effect of
manipulation is measured. Experiments tend to be small since they require a high
level of control over the test environment. Case studies on the other hand are referred
to as ”research in the typical” [100] since they tend to look at what is happening in a
46](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-58-2048.jpg)
![typical project. The level of control is generally lower in a case study than in an experi-
ment. The objective is often to monitor a particular project or activity. Case studies are
said to be observational studies whereas experiments are controlled studies. Finally
surveys are often referred to as ”research in the large” [100] since they try to capture
what is happening over a range of projects. They are usually performed by obtaining
information from a representative sample of the entire population through interviews
or questionnaires. The results are generalised to the population from which the sam-
ple was taken.
In this thesis we make use of both case studies and experiments to evaluate our re-
search. Experiments have been used to evaluate the overhead associated with our
monitoring approach (see section 7.2.3) as well as to assess the efficiency of our anal-
ysis module (see section 7.3.1). According to Wohlin et al. [178] the main strength of
an experiment is ”that it can investigate in which situations the claims are true and
they can provide a context in which certain standards, methods and tools are recom-
mended for use”.
Case studies on the other hand have been used to show how our monitoring and
analysis outputs can be used to manually identify antipatterns (see sections 7.2.1 and
7.3.2 respectively). Case studies have also been used to show how antipatterns can
be automatically identified in component based systems (see section 7.4). The main
advantage of using case studies is that they incorporate qualities that an experiment
can not such as scale, complexity, unpredictability and dynamism [178].
Below we have outlined a number of evaluation criteria which we evaluate our work
against. The evaluation criteria are the attributes which we believe are required, in
order for our approach to be considered useful. The evaluation criteria are as follows:
• Portable: The monitoring module should be portable across different vendors’
implementations of the middleware. This is particularly important for JEE,
whereby application servers are built by a large number of different vendors
2. In fact it is not unusual for an enterprise application to be made up of web,
application or database servers from a number of different software vendors. To
be able to apply our approach to truly heterogenous systems it is required that
the monitoring module is completely portable across the different implementa-
tions.
• Non-intrusive: The monitoring module should also be non-intrusive, that is, it
should not require any modifications to the application or middleware source
code. This is important for component based systems, since often it may not
be possible to make such modifications. For example most commercial applica-
tion servers are proprietary and access to the source code is unavailable. Also
2Application server product vendors, http://www.service-architecture.com/products/application servers.html
47](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-59-2048.jpg)
![tion patterns in the data collected. Specifically we show how FSM can be applied in an
efficient manner to run-time paths (section 7.3.1). We also show how, by modifying
the FSM algorithm, the algorithm output can be made more human readable (section
5.4.2). In fact in section 7.3.2 we show how the output from the FSM algorithm can
be used to manually identify design flaws in JEE applications. The FSM algorithm
takes a user configurable input (known as the support value) which determines the
threshold for the patterns it identifies. The support value can be used to filter out less
interesting patterns in the data. Finally the analysis module is also responsible for
summarising the monitoring data using clustering and statistical analysis techniques
(see data reduction section 5.5). This data reduction allows for the design extraction
techniques to be scalable when large amounts of data are collected by the monitoring
module (e.g. when the system is under load).
3.1.4 Detection
The extracted design model is used as input into the detection module (chapter 6). Our
detection mechanism is based on a rule engine approach. The design model is con-
verted to rule engine specific facts that can be loaded into the rule engine. Rules can
be input into the rule engine which describe the antipatterns that we want to detect
in the model. The rule-engine makes use of forward-chaining rules which are in the
form of ”if-then” statements. Rules are specified so that the rule’s conditions verify the
occurrence of a certain antipattern. Subsequently, when existing facts match a rule’s
condition clauses, the rule action is fired indicating that the antipattern was detected
(see section 6.2). Our predefined rules make use of configuration files which contain
user specified thresholds (section 6.3.2). User defined thresholds can be utilised to
filter out antipatterns that developers feel are too insignificant to address. The PAD
tool also contains a rule library for JEE performance design and deployment antipat-
terns (section 6.3.1) which users can extend by adding their own antipatterns rule de-
scriptions. The PAD tool has been applied to a sample and real large scale enterprise
application. Our results (see section 7.4) show how the approach scales when applied
to a real enterprise system and that the number of false positives remains low.
3.2 Antipatterns Revisited
While the concept of software antipatterns is generally understood by developers, the
individual antipattern instances are not as widely known. As documented in the lit-
erature [36], antipatterns consistently appear in software, especially in large complex
(enterprise) applications [161]. The work in this thesis outlines an approach for the
automatic detection of these well documented issues that consistently occur in enter-
prise applications.
50](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-62-2048.jpg)
![The benefits of highlighting antipatterns are manifold. Firstly, software antipatterns
document bad practices in software development as well as their corresponding so-
lutions. Thus, detecting instances of antipatterns and highlighting them to the devel-
oper, not only allows the developer to understand what is wrong with the application,
but it also allows the developer to easily rectify the situation by refactoring the applica-
tion with the solution provided. Secondly, antipatterns provide a high-level language
for discussing design issues. Developers can use antipattern names to clearly and ef-
ficiently discuss within the development team particular implementation issues that
may arise. Also, exposing developers to performance related antipatterns helps them
form a sense of performance intuition, since the underlying reason as to why a partic-
ular antipattern had an adverse effect on performance is underlined in the antipattern
description. Consequently developers are less likely to make the same mistake in the
future.
In the following subsection we discuss a number of different antipattern categories
and give a hierarchy of antipatterns that shows how the different categories that we
are interested in are related. We also show that a high percentage of enterprise an-
tipatterns tend to be concerned with performance issues. Finally in this sub section
we differentiate between design antipatterns, deployment antipatterns and program-
ming errors.
3.2.1 Antipattern Hierarchy
Figure 3.3 gives an antipattern hierarchy diagram. At the top of the diagram we have
high level technology independent software antipatterns. Brown et al. [36] introduced
a number of such antipatterns concerned with a range of software quality attributes
(such as reusability, maintainability, performance). More recently Smith and Williams
[154] introduced general performance antipatterns which solely focus on performance
concerns (i.e. level 2). The performance antipatterns presented in the literature [154]
are high level and language-independent antipatterns. They describe situations where
a sub-optimal performance design choice has been made. Instances of the antipatterns
documented by Smith and Williams, however, occur throughout different technolo-
gies. Many of these problems are especially common in enterprise technologies where
performance is often a major concern (level 3). JEE antipatterns have been presented
in [161] and [61]. The literature [161] concentrates on EJB antipatterns, while [61] lists
antipatterns concerned with a number of aspects of the JEE technology (i.e. Servlets,
JSP, EJB and Web Services). We have analysed the antipatterns from both sources.
From a total of 43 antipatterns documented in [161] we have identified 34 (79%) of
them to be performance related antipatterns (since according to the authors they have
an impact on system performance). From a total of 52 antipatterns documented in
the literature [61] we identified 28 performance antipatterns (54%). The high propor-
tion of antipatterns from [161] and [61], that are related to performance, is further
51](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-63-2048.jpg)
![Figure 3.3: Hierarchy of Antipatterns
evidence that performance is an important software quality attribute for enterprise
systems, and that poor performance design is common in such systems.
We further divided all JEE performance antipatterns identified into 3 different cat-
egories (level 4): (a) Performance programming errors, (b) performance design an-
tipatterns and (c) performance deployment antipatterns. Performance programming
errors (a) can be defined as common mistakes made by developers that result in de-
graded system performance. They yield no design trade-off and always have a neg-
ative impact on performance. Examples include memory leaks, deadlocks and im-
proper cleanup of resources such as database connections. Generally developers are
unaware of the presence of performance programming errors in the system. The Rot-
ting Session Garbage antipattern [161] is an example of a performance programming
error that is often made by developers using the EJB technology. This antipattern
occurs when a client fails to explicitly remove a stateful session bean when finished
using it. The orphaned bean will continue to live in the application server using up
system resources until it times out. Until then, the EJB container must manage the
52](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-64-2048.jpg)
![4.1 Chapter Overview
The monitoring stage of the antipattern detection process obtains information on (a)
component relationships, (b) component communication patterns, (c) component re-
source usage, (d) component object usage, (e) component meta-data and (f) server
resource usage. Using this information we can extract the required relationships to
reconstruct the systems run-time design for performance antipattern detection (see
figure 3.2 on page 46). The monitoring module consists of three main components,
a run-time path tracer, a server resource monitor and an meta-data extractor. Next
we detail the different techniques required to capture this information in a portable,
low-overhead and non-intrusive manner. Our monitoring approaches are applied to
a running application and do not require any analysis of the source code.
The run-time path tracer collects run-time paths which contain the information (a) (b)
(c) and (d) outlined above. More specifically a run-time path [44] contains the control
flow (i.e. the ordered sequence of methods called required to service a user request),
resources and performance characteristics associated with servicing a request. Run-
time paths are described in more detail in section 4.2.1. By analysing these paths one
can easily see how system resources are being used, how the different components
in the system interact and how user requests traverse through the different tiers that
make up the system. As part of this work a run-time path tracing tool has been de-
veloped, called COMPAS Java End to End Monitoring (JEEM). COMPAS JEEM mon-
itors the paths user requests take when traversing through the different tiers in the
system. Resource usage and performance characteristics of each request can also be
obtained using our approach. It builds upon the open source COMPAS monitoring
framework 1 [120] [118] and leverages and extends the capabilities of a number of
other open source projects2 [45]. In this chapter we discuss how run-time path trac-
ing has been achieved in a non-intrusive, low-overhead and portable manner. Section
4.2.2 and 4.2.3 gives our motivation and the considerations that should be undertaken
(independent of the underlying technology) for non-intrusive run-time path tracing.
Section 4.2.4 presents the core instrumentation process and the extension capabilities
available in the COMPAS monitoring framework. The end-to-end run-time path trac-
ing approach leverages these extension mechanisms to build upon the COMPAS mon-
itoring and instrumentation infrastructure. In section 4.2.5 we describe the extensions
made to the COMPAS framework and the overall architecture of COMPAS JEEM. In
this section we also detail the open source projects that we have extended and inte-
grated with COMPAS to achieve our aims. In section 4.2.5.3 we detail how run-time
path tracing can be performed in a truly distributed manner. The idea of Compo-
nent Object Tracking is introduced in section 4.2.5.8 and we discuss how this can be
achieved through dynamic bytecode instrumentation and the COMPAS ByteCode In-
1COMPAS, http://compas.sourceforge.net/
2P6Spy, http://www.p6spy.com/
56](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-68-2048.jpg)
![strumentation (BCI) tool. The portability of the run-time path tracing approach across
different component technologies is discussed in section 4.2.5.9.
Section 4.3 details the remaining two components of the monitoring module i.e. the
server resource monitor and the component meta-data extractor. Firstly in subsection
4.3.1 we detail how our server resource monitor obtains information on the server
resources at run-time through a Management EJB monitoring component. In this sec-
tion we discuss how this is achieved in a non-intrusive and portable manner through
the Java Management Extension interface. We also discuss the information obtained
that is required for the run-time design model (i.e. (f) server resource information)
and why this approach induces very little overhead on the running system. In subsec-
tion (4.3.2) we outline how information on the different components can be extracted
from the component meta-data (e). The component meta-data contains information
on how different components make use of the container services as well as structural
and functional information which is added to the run-time design model. This infor-
mation can be obtained by analysing the component deployment descriptors. This
task is performed off-line and is automated using an XML parser.
Finally in section 4.4 we compare our monitoring approach to related work. In partic-
ular we detail: alternative applications of run-time paths, related data structures for
the representation of dynamic calling behaviour and related tracing tools and tech-
niques that have been previously presented.
4.2 Run-Time Path Tracing
4.2.1 Run-Time Paths Overview
A run-time path is a data structure for the representation of run-time calling be-
haviour. A run-time path has been previously defined by Chen et al. [44] as containing
”the control flow, resources and performance characteristics associated with servicing
a user request”. The authors also state that run-time paths are recorded at run-time by
tracing requests through a live system, and that they span the system’s layers to di-
rectly access component and resource dependencies. Next we describe run-time paths
in more detail and explain how they related to other representations of run-time call-
ing behaviour. We also show a run-time path data structure and explain the various
fields that are contained with it.
A dynamic call trace [94] (see figure 4.1 (a)) is a recording of all calls and returns that
occur in a program’s execution. Dynamic call traces are unbounded. Their size is
proportional to the number of calls in an execution [8]. Figure 4.1 (b) shows a tree
representation of a dynamic call trace, often referred to as a dynamic call tree [8] [94].
Traversing the tree in preorder will give the ordered sequence of calls and returns as
they occurred during execution. Dynamic call traces/trees capture important infor-
57](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-69-2048.jpg)
![Figure 4.1: Dynamic Call Trace (a) and Corresponding Dynamic Call Tree (b)
mation for system comprehension such as, the sequence of calls, the context in which
a call was made and repeating calls or loops. An issue with dynamic call traces how-
ever is that they are unbounded and are space inefficient. More compact represen-
tations, such as dynamic call graphs and calling context trees, have been previously
presented to reduce the space overhead [8] [94] [79] however these representations
do not capture important information that is contained in the dynamic call trace. A
more detailed comparison of these representations is given in section 4.4. Run-time
paths are similar to dynamic call traces. In fact they contain the same information as
a dynamic call trace i.e. they contain the ordered sequence of all calls and returns that
occur during execution. However they also contain related resource and and perfor-
mance information for each call. In fact they can contain further information such as
the type of call made or the arguments passed in the call for example. Furthermore
run-time paths are generally associated with multi-user/multi-client systems. They
group together, into a single path, the sequence of calls that correspond to a given
user/client request. As such, run-time paths isolate the calls that occur in response
to a given request. This is particularly important in multi-user/multi-client applica-
tions whereby many simultaneous requests may be concurrently serviced. Chen et al.
[44] have described this as providing ”a vertical slice of the system from a request’s
perspective.” Finally, since they are associated with multi-user applications, run-time
paths often span a number of system layers. These layers can often be physically dis-
58](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-70-2048.jpg)
![class CallPath {
String pathID;
PathNode Root;
...
...
...
//other useful fields or methods
}
Figure 4.3: Run-Time Path Data Structure
class PathNode {
Map Details;
List Callees =new ArrayList();
...
...
//other useful fields or methods
}
Figure 4.4: A Run-Time Path’s PathNode Data Structure
4.2.2 Run-Time Path Tracing Motivation
The analysis module presented in chapter 5 makes use of run-time paths to automat-
ically extract information required for antipattern detection. Below are some alterna-
tive uses that run-time paths can be put to.
A run-time path shows how a user request is serviced, giving the ordered sequence of
events/calls and related performance and resource usage information. Thus run-time
path analysis can help greatly with system comprehension and can be applied for a
number of different purposes in this area. System comprehension can be a major issue
for developers of large enterprise applications. A lack of a system wide understand-
ing can lead to badly designed systems which perform poorly and can be difficult to
debug and maintain. In section 7.2.1 we show how run-time paths can be used to
reverse engineer and quickly deduce the overall system structure of enterprise appli-
cations. Analysing run-time paths to deduce system structure is advantageous for a
number of reasons. Firstly analysing application source code can be difficult since a
large number of files can be involved. Even when using code comprehension tools
3 [77] determining the chain and order of method calls by stepping through source
code can be a complex and time consuming task. In addition, some paths might be
very difficult to determine from static analysis, in particular in systems which con-
tain workflow engines that select execution targets based on run-time conditions. In
comparison COMPAS JEEM presents all calls from the main software components
that make up a user request in a single run-time path. It maintains the order of calls
3Juliet Java code comprehension tool, http://infotectonica.com/
60](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-72-2048.jpg)
![and the call sequence can be easily observed by traversing the run-time path tree-like
structure (see figure 4.2). Another advantage of using run-time paths, for system com-
prehension, is that they can be well represented in a diagrammatic format (see figure
4.12). The diagram construction process can be easily automated reducing the effort
required on the part of the developer further [97]. As shown in section 7.2.1, such dia-
grams are useful for identifying performance design issues or antipatterns that might
exist in enterprise applications.
Run-time paths have also been put to other uses such as fault detection and diagnosis,
workload modelling, and adaptive monitoring. In section 4.4 we will discuss this
related work in more detail.
4.2.3 Run-Time Path Tracing Considerations
Today’s enterprise applications are generally required to handle high loads of concur-
rent users. Run-time path tracing in such systems involves tracing each of these user
requests as they pass through the different software tiers that make up the entire ap-
plication. To achieve this user requests must be identifiable across the entire request
and the order of the calls that make up the requests needs to be maintained.
A system needs to meet the following requirements to be able to perform run-time
path tracing:
R1 the system must be able to identify new user requests entering the system
R2 the system is required to tag the request with request specific information (RSI) so
that calls can be mapped to the originating requests
R3 the system requires the ability to piggy back the RSI with the request, such that
the request can be tracked across the entire system
R4 a monitoring framework that provides for request interception is required
Next we detail how run-time path tracing can be achieved through the above require-
ments. The user request identification mechanism (R1) is required to determine when
a new request enters the system such that it can be (R2) tagged with request specific
information (RSI). The RSI contains a unique ID so that the request can be identified
and distinguished from other concurrent requests in the system. The RSI also con-
tains a sequence number that is used to order the calls that make up the request. The
sequence number is incremented (by the tracer’s run-time path ordering logic) every
time a component method is called and is reset upon a new user request.
User requests also need to be tracked (R3), so that, as the request passes through the
different software components that make up the system, it can be identified and the
order of its calls maintained. In order to be able to track the request the RSI must be
61](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-73-2048.jpg)
![attached to it, in such a way as to be available at every call point along the request.
This can be particularly challenging when a system is distributed across a network.
A monitoring framework (R4) is required to intercept and log calls made to the soft-
ware components that make up the request. The RSI, along with component details
are logged upon a method invocation. The component details describe the compo-
nent that is called, giving information such as what component method was invoked,
what arguments were passed etc. The logged information can be subsequently used
to construct the run-time paths offline. The reconstruction process makes use of the
RSI data to (a) determine what calls make up each run-time path and (b) to order the
calls.
Thus far, the above requirements have generally been met in a manner that involves
instrumenting either the software application itself or the underlying middleware [82]
[45]. Manual instrumentation of the application source is undesirable since instru-
menting the code can be time consuming and cumbersome and the effort must be
repeated for each new system under development. Since the JEE specification does
not explicitly address the requirements for run-time path tracing, middleware layer
approaches for JEE have involved using non-standard mechanisms provided by spe-
cific vendors that are not portable across the technology and that often require the
source code of the server to be available (such that it can be recompiled). For example
the JBoss group recognised the need for R4 by providing its interceptor-based com-
ponent architecture [105] which allows for the interception of calls to business tier
JEE components. IBM has also recently addressed the need for R3 through its Work
Area Service [102]. The biggest issue with using non-standard mechanisms, however,
is that developers get locked in to using a particular middleware implementation and
lose the flexibility of being able to quickly change their underlying middleware imple-
mentation should the need arise. Consequently, such approaches can not be applied
to systems that are made up of application servers from numerous different vendors.
In the following sections we present an approach that, through the COMPAS monitor-
ing framework, takes advantage of a number of JEE standard mechanisms to meet the
requirements outlined above for JEE systems in a non-intrusive and portable manner.
While our implementation focuses on the JEE technology, only the framework instru-
mentation process and request tracking are JEE specific, the remainder of the work
presented below can be applied to other component technologies (e.g. CCM). This is
discussed in detail in section 4.2.5.9.
4.2.4 COMPAS Monitoring Framework
COMPAS [118] is intended as a foundation for building enterprise-level performance
management solutions for component-based applications. It was designed to pro-
vide a common monitoring platform across different application server implementa-
62](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-74-2048.jpg)
![tions. This is achieved by leveraging the underlying properties of component-based
platforms in order to enable non-intrusive instrumentation of enterprise applications.
Two main techniques are used to minimise the overhead of the monitoring infrastruc-
ture: asynchronous communication and adaptive activation. The former is employed
in the entire infrastructure by the use of an event-based architecture with robust mes-
sage handling entities that prevent the occurrence of locks in the target application.
The latter technique uses execution models captured from the target application to
drive the activation and deactivation of the monitoring probes. By appropriately min-
imising the number of active probes (i.e. through the alert management and adaptive
monitoring process), the total overhead is reduced while preserving complete target
coverage (see [119] [118]). As the COMPAS infrastructure is designed to be used as
a foundation for performance management tools, its design is highly extensible and
based on decoupled communication mechanisms.
The most important functional entity of the monitoring infrastructure is the monitor-
ing probe. The probe is conceptually a proxy element with a 1 to 1 relationship with
its target component. In JEE, target components are JSPs, Servlets, EJBs and EIS com-
ponents deployed in a target application. The probe is implemented as a proxy layer
surrounding the target component with the purpose of intercepting all method invo-
cations and lifecycle events. The process of augmenting a target component with the
proxy layer is referred to as probe insertion. This section focuses on the core COMPAS
capabilities, initially limited to EJB components only, whereas the following sections
present the extensions that allow COMPAS to instrument and monitor the remaining
JEE target components, grouped under the name COMPAS JEEM.
4.2.4.1 Portable EJB Instrumentation
COMPAS uses component meta-data to derive the internal structure of the target en-
tities. The component meta-data is placed in deployment descriptors that contain
structural as well as behavioral information about the encompassing EJBs. By lever-
aging this data, it is possible to obtain the internal class-structure of each component,
which is needed for instrumentation. The instrumentation is performed by a ”proxy
layer” attached to the target application through a process called COMPAS Probe In-
sertion (CPI). As all the information needed for probe insertion is obtained from the
meta-data, there is no need for source code or proprietary application server hooks.
Therefore, the effect on the target environment is minimal and user intervention in the
probe insertion process is not required. COMPAS is in this respect non-intrusive, as it
does not require changes to the application code or to the runtime environment as the
majority of other approaches do.
The CPI process examines the Target Application’s structure and uses component
meta-data to generate the proxy layer. For each component in the target application,
a monitoring probe is specifically generated based on the component’s functional and
63](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-75-2048.jpg)
![Figure 4.6: COMPAS JEEM Architecture
• Server-Side FEP: facilitates the extension of the functionality available to or pro-
vided by a COMPAS Monitoring Probe. In addition, it facilitates the creation
of new probe types. The majority of extensions introduced by JEEM use server-
side FEPs. In particular, JEEM makes use of Servlet filter probes and JDBC driver
probes to feed information into the COMPAS infrastructure. Apart from these
probes in the web and EIS tiers, JEEM also enhances existing EJB probes in or-
der to generate appropriate run-time path extraction information. Other exam-
ples of server-side FEPs include better time-stamp extraction techniques or ad-
vanced anomaly detection algorithms used in the COMPAS problem diagnosis
processes [118].
• Client-Side FEP: facilitate the extension of the functionality available to or pro-
vided by the COMPAS Monitoring Dispatcher. The run-time path construction
and analysis module introduced by JEEM uses such a client-side extension and
benefits from the information made available by the distributed infrastructure to
the client-side. Other examples of extensions that can be added using client-side
FEPs include specialised GUI consoles or integration into wider-scope perfor-
mance tools.
As indicated, the additional functionality that COMPAS JEEM provides over the basic
COMPAS framework uses several FEPs, across all JEE tiers. The following sections
present in detail these JEEM extensions.
66](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-78-2048.jpg)
![4.2.5 COMPAS Extensions
As detailed in section 4.2.4 the current implementation of the COMPAS framework
has the ability to monitor the business tier of JEE applications. We have made a num-
ber of extensions to the COMPAS framework using the COMPAS FEPs to allow for
the tracing of run-time paths across all tiers of a JEE system for multi-user workloads.
The extensions made fall into three different categories, namely, monitoring, run-time
path tracing and analysis. The overall architecture of COMPAS JEEM is shown in
figure 4.6.
4.2.5.1 Monitoring
The COMPAS JEEM monitoring approach is completely portable and non-intrusive
and has the ability to trace system wide JEE run-time paths. Basically it works by first
intercepting the calls made to the different components that make up the system and
second, recording their sequence. The interception of calls is performed by extend-
ing the COMPAS framework, using server-side FEPs, so that monitoring on all three
server side tiers is performed.
COMPAS uses the idea of a proxy layer to monitor EJB components [119]. The proxy
layer is a thin layer of indirection directly attached to the EJBs that captures any re-
quests to and responses from the components. When extending COMPAS we applied
the idea of intercepting requests to and responses from components to the web and
EIS tiers. This is performed in the web tier using intercepting filters and in the EIS tier
using wrapper components. The EJB probes, intercepting filters and wrapper compo-
nents can also be referred to as Interception Points (IPs).
Intercepting Filters A chain of pluggable filters can be created to implement com-
mon pre and post-processing tasks during a web page request by implementing the
Intercepting Filter pattern [7]. This patterns allows for the creation of pluggable fil-
ters to process common services in a standard manner without requiring changes to
core request processing code. The filters intercept incoming requests and outgoing re-
sponses, which allows for pre and post-processing respectively. The servlet 2.3 speci-
fication 4 includes a standard mechanism for building filter chains based on the Inter-
cepting Filters pattern. Consequently web applications running on servlet specifica-
tion compliant web servers can be augmented with filters for pre and post processing
without the need to modify source code.
Next we briefly detail the Intercepting Filter pattern which we use to monitor the web
tier of JEE applications. A UML sequence diagram of the Intercepting Filter pattern is
shown in figure 4.7.
The pattern contains the following participants: the Client, the FilterManager, the Fil-
4Java Servlet Specification, http://java.sun.com/products/servlet/download.html,
67](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-79-2048.jpg)
![Figure 4.7: Intercepting Filter
terChain, one or more Filters and finally the Target resource. The FilterManager sits
between the client and the Target resource. When the client makes a request for the
Target resource the FilterManager intercepts this request and creates a FilterChain. A
FilterChain is an ordered collection of independent Filters. The Filters perform the
pre and post processing logic and are mapped to a particular Target resource. The
FilterManager next makes a request to the FilterChain to process the filters and subse-
quently the Target resource.
The Intercepting Filter pattern can be implemented using a number of different strate-
gies. Servlet specification compliant web servers implement the Intercepting Filter
Standard Filter Strategy [7] whereby filters can be added declaratively through XML
deployment descriptors. The web server container contains the FilterManager and is
responsible for creating the different FilterChains that can be specified through the
XML deployment descriptors.
To monitor the web tier we have implemented a monitoring filter. The monitoring filter
intercepts incoming requests and outgoing responses and thus can log the beginning
and end of user requests. The monitoring filter can be added to the web application
under test automatically (see section 4.2.5.1 below).
Before the servlet 2.4 specification filters could only be applied to the initial client
68](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-80-2048.jpg)
![request. Filters could not be applied to inter servlet communications. As a result
the web tier was monitored as one software component on web servers prior to 2.4.
However the 2.4 servlet specification allows for filters to be applied to calls between
servlets (i.e. forwarded requests) 5 and thus our monitoring filters can collect the entire
web tier run-time path trace for servlet specification 2.4 compliant web servers. Since
this has been mandated by the servlet specification all future JEE environments will
support this feature.
EIS Tier Wrapper Components The monitoring of the EIS tier is performed using
wrapper components. This approach takes advantage of the proxy design pattern
[72]. EIS tier monitoring works by wrapping the JDBC database driver with a mon-
itoring layer. JDBC drivers implement the JBDC interfaces and classes of the JDBC
API. For each original JDBC driver class, a wrapper class that exposes the same inter-
face as the driver class is created. Rather than registering the JDBC driver class with
the application server as normal, the packaged wrapper classes are registered with
the application server. Calls made to the database layer from the application server
are thus intercepted by the wrapper classes. The original JDBC driver is instead regis-
tered with the wrapper classes (declaratively using a properties file). Any calls made
to the wrapper classes can thus be delegated to the original driver classes. The wrap-
per classes contain monitoring logic that captures all JDBC requests and responses.
The wrapper classes have been implemented by extending the P6Spy open source
framework 6.
As outlined above COMPAS JEEM’s monitoring framework consists of a number of
IPs. The IPs capture performance metrics and resource usage information along with
the control flow data required for run-time path tracing.
Adding The Monitoring Framework to a JEE Application: Adding the aforemen-
tioned filters and wrapper components does not require the server code or the appli-
cation source code to be available nor does it require special JVM hooks. Instead we
leverage standard JEE mechanisms and make use of the component meta-data that is
stored in XML files as mandated by the JEE specification. The monitoring components
can be automatically added to the packaged application. The basic instrumentation
script (i.e. the CPI process) provided by the original COMPAS monitoring framework
has been extended to (1) unpack the entire JEE application, (2) parse the XML deploy-
ment descriptors of the application (to obtain structural and functional information on
the different components that make up the system) and (3) repackage the application
with the COMPAS JEEM monitoring components inserted.
5Jason Hunter, ”Servlet 2.4: What’s in store”, Java World, March 2003,
http://www.javaworld.com/javaworld/jw-03-2003/jw-0328-servlet.html,
6P6Spy, http://www.p6spy.com/, accessed June 2006
69](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-81-2048.jpg)
![4.2.5.2 Run-time Path Tracing
The ability to trace system wide run-time paths for multi-user workloads has been
added to the capabilities of COMPAS JEEM. A generic run-time path tracing approach
has been outlined in section 4.2.3. This approach has previously been implemented by
Chen et al. in the Pinpoint open source project [45] [44]. However, a major drawback
of the pinpoint implementation is that it relies on instrumentation of the middleware
and it is therefore tied to its target application server, JBoss. In the following sections
we detail the Pinpoint implementation of the above approach and how we applied it
in a non-intrusive capacity.
Pinpoint - Intrusive run-time path Tracing: Pinpoint is a framework for problem de-
termination in internet service environments. It includes a runtime path tracer for
the JBoss application server. The requirements R1 R3 and R4 outlined in section 4.2.3
have been met in the Pinpoint implementation through instrumentation of the mid-
dleware. The following areas in the middleware were instrumented: the web server,
the application server, a Remote Method Invocation (RMI) library and the JBoss JDBC
Resource Adaptor [105]. Next we briefly detail the above instrumentation points, and
their responsibilities. R2 is the only requirement that has been met in a non-intrusive
capacity by the pinpoint framework and is described in section 4.2.5.2.
User requests enter the system in the web tier. Pinpoint instruments both the http
server and the servlet container. The instrumentation in the http server is responsi-
ble for determining when a new request enters the system (R1). It injects a unique ID
(R2) value into the local thread of the new incoming request. The servlet container in-
strumentation is responsible for logging calls to and responses from each JSP/servlet
(R4). The servlet container instrumentation is also responsible for the runtime path
ordering logic in the web tier.
The application server instrumentation is performed in the Pinpoint project using
JBoss interceptors [105]. The instrumentation logs calls to and from EJB components
(R4) and performs runtime path ordering logic when a method is called. The JDBC Re-
source Adaptor instrumentation has similar responsibilities in relation to JDBC calls.
Pinpoint also allows for tracing of run-time paths that span a number of threads
(which for example might be caused by a remote method invocation) by instrument-
ing a JBoss RMI library such that the unique request ID is marshalled across the remote
call from the local thread to the new (remote) thread (R3).
Non-Intrusive Run-time Path Tracing We require a non-intrusive approach for run-
time path tracing. In the following paragraphs we discuss how all the requirements
(outlined in section 4.2.3), monitoring (R4), user request identification (R1), user re-
quest tagging (R2) and user request tracking (R3) can be met in a non-intrusive capac-
ity.
70](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-82-2048.jpg)
![4.2.5.3 Tracking RSI in Distributed Environments
In this section, we analyze how calls from a client to a server can be tracked in a
distributed setting. The approach we take is based on piggy-backing the RSI onto
the request from a client to a remote server. In the following subsections we detail
how this has been achieved in a portable manner for JEE systems. This approach has
previously been implemented in the WebMon [82] profiling tool, a tool for profiling
JEE web applications. In the following sections we discuss how this can be applied
for COMPAS JEEM.
4.2.5.4 Piggy-backing data
There is no standard mechanism in JEE for piggy-backing data along a remote request.
Piggy backing mechanisms however are available in other technologies. For example
CORBA’s Portable Interceptors 8 allow for piggy-backing of data along a request. The
CORBA Portable Interceptors can be used to track user requests in distributed envi-
ronments. For example, it has previously been shown that they can be used to attach
additional security information onto requests [179].
We discuss how this problem can be solved for JEE systems by adding an additional
parameter to the remote request and allowing the client to pass the extra request data
through that additional parameter. This can be achieved without access to the appli-
cation’s source code as follows: A components’ meta-data specifies the component
interfaces. The interfaces can be reconstructed and rewritten from their bytecode rep-
resentation (or by analysing the XML deployment descriptor) to add a method that
allows for the passing of the additional information using an additional request pa-
rameter. Next the client component is also instrumented, to add this additional pa-
rameter, and the server component instrumented, to pass it to the instrumentation
infrastructure. In the next section, we give a detailed description how this technique
can be used to pass RSI across distribute component boundaries.
4.2.5.5 Client Side Interception
When a client component calls a distributed component in JEE, the following occurs:
The client calls a stub which is a client side proxy object. The stub masks the low
level networking issues from the client and forwards the request on to a server side
proxy object. The server side proxy element, known as a skeleton, masks the low
level networking issues from the distributed component. It also delegates the remote
request to the distributed component. The distributed component then handles the
request and returns control to the skeleton, which in turn returns control to the stub.
8Common Ojbect Request Broker Architecture: Core Specification, Version 3.0.3,
http://www.omg.org/technology/documents/
73](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-85-2048.jpg)
![public class SampleClient {
public static void main(String[] args)
throws Exception {
Properties props=System.getProperties();
Context ctx=new InitialContext(props);
Object obj=ctx.lookup("Sample");
SampleHome sh =(SampleHome)
PortableRemoteObject.narrow
(obj,SampleHome.class);
Sample s = sh.create();
System.out.println
(s.callBusinessMethod("SomeString"));
s.remove();
}
}
Figure 4.11: A Sample Bean Client
each method such that it contains an extra parameter. This extra parameter is used to
piggyback the data across the network. In the context of run-time path tracing once
the data has been sent across the network it can be attached to the current thread as
outlined in section 4.2.5.2.
4.2.5.7 Analysis
COMPAS JEEM contains run-time path tracing logic which non-intrusively logs in-
formation when methods are invoked. The monitoring framework uses the COMPAS
Monitoring Dispatcher to asynchronously log the information to a remote machine.
This allows for the information to be stored and analysed off-line. Analysing the data
remotely (as opposed to analysing it locally) reduces the performance impact on the
monitored system.
There are two main phases of information extraction from the collected data: run-time
path construction and advanced run-time path analysis. It is important to note that
the analysis part of the framework is completely independent of the approach used
for run-time path extraction. The only requirement is that the necessary information
(see Run-Time Path Construction below) is provided.
Run-Time Path Construction On every invocation of a component method, informa-
tion is logged. In the pinpoint framework this is known as an observation. The logged
information contains the following data:
• user request (unique) ID
• sequence number
• call depth
76](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-88-2048.jpg)
![• component details
• performance data
Run-time paths are constructed by grouping all observations by user request id. Each
such group represents an unordered list of the calls that make up a single run-time
path. By ordering the list according to the sequence number of each observation, the
observations can be arranged in the order that they occurred during program execu-
tion. A run-time paths data structure is subsequently created by traversing each or-
dered list. This completes the construction of the run-time paths. Further analysis can
be applied to this information for a number of purposes. For example, by analysing
the run-time paths developers can easily construct UML diagrams that can help to de-
duce the overall system structure (see sections 7.2.1). This output also forms the basis
of the automatic antipattern detection approach and allows for the identification of
component relationships, component communication patterns and the reconstruction
of container services (see chapter 5). Next we discuss how component object tracking
can be performed using this data.
4.2.5.8 Tracking Component’s Object Usage
Objects can also be tracked along run-time paths to allow for analysis of their usage
in the system. Such analysis can lead to identification of inefficient object usage (see
section 7.4.1). Figure 4.12 shows an example run-time path which tracks the lifecycle
of instances of the AccountDetails data transfer object.
The COMPAS JEEM tool is currently being extended (the new version is called COM-
PAS ByteCode Instrumentation [BCI]) to track selected objects across run-time paths.
The new version of the tool makes use of dynamic byte code instrumentation, through
the java.lang.management interface, to dynamically instrument selected objects. Tool
users can select particular classes to be tracked. When an object of the selected class
is created it is identified along with its corresponding position in the run-time path.
Whenever another method (other than the constructor [or creator method for EJBs])
is accessed this is also logged. Thus we can identify where objects are created along
the run-time paths and at what points they are accessed. We can effectively see how
objects are created, used and passed along the run-time path. Figure 4.12 show where
instances of the AccountDetails object are created and accessed along a JEE call path.
The object was created by an entity bean in the application tier and passed to the
web tier where a single method was accessed. This information is required to identify
a range of common antipatterns (for example to identify variations of the excessive
object allocation antipattern [154] which manifests itself in a number of different an-
tipatterns in enterprise applications). We have developed a proof of concept of this in-
strumentation by manually instrumenting objects that we required for tracking. The
77](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-89-2048.jpg)
![the EJB specification). The fact that component based enterprise frameworks specify
how certain component types should behave allows us to automatically identify this
type of unusual behaviour. Without this information automatic antipattern detection
is more difficult. For example, if instead we were monitoring an application made up
of Plain Old Java Objects (POJO’s) with no information describing how we expect the
objects to behave, it would be difficult to flag unusual behaviour. In such situations
domain or application specific information could instead be supplied by the appli-
cation developers. The PAD tool extracts the component meta-data from the XML
deployment files of the JEE applications automatically using an XML parsing library
13. This is in fact achieved during the COMPAS instrumentation process (see section
4.2.5.1).
The information obtained through the PAD tool’s meta-data extractor is used to pop-
ulate the run-time design model (figure 3.2, page 46). Functional and structural in-
formation is obtained on the various components (e.g. the component type, the com-
ponent’s methods, the interface type). This information can be directly added to the
run-time design model. Information on the container services that are required by the
different components is also collected. In section 5.3 we discuss how these services
can be reconstructed using the component meta-data and run-time path information
and subsequently added to the run-time design model.
4.4 Related Work
In this section we discuss and compare related work to the work presented in this
chapter. Firstly we outline how run-time paths have been previously used for a num-
ber of different purposes. We follow this by discussing alternative representations for
dynamic calling behaviour that have been previously presented and compare these to
run-time paths. Finally we discuss tracing tools and techniques that are related to the
run-time path tracing approach presented.
4.4.1 Applications of Run-Time Paths
In section 4.2.2 we have discussed a number of uses to which run-time paths have
been put to, for example, reverse engineering, information extraction for antipattern
detection, or system comprehension. Run-time paths however have also been utilised
in a number of other ways.
Chen et al. [45] have applied run-time paths to deal with fault detection and diagnosis
in large distributed systems. To detect faults they characterize distributions for nor-
mal paths and look for statistically significant deviations that might suggest failures.
To isolate faults to the components responsible (diagnosis) they search for correlations
13Java XML Libraries, http://java.sun.com/xml/
82](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-94-2048.jpg)
![(using data mining and machine learning techniques) in the run-time paths between
component use and failed requests. They have also applied run-time path analysis to
assess the impact of faults and to help with system evolution. Traditional approaches
to this problem have relied on static dependency models. However such dependency
models often do not capture the dynamic nature of today’s systems. Chen et al. use
dynamic tracing (i.e. run-time paths) to capture the dynamic nature of today’s con-
stantly evolving internet systems.
The literature [19] presents the use of run-time paths to address performance issues.
The Magpie project 14 collects run-time paths for distributed systems. It also measures
resource consumption. Magpie then builds stochastic workload models suitable for
performance prediction, tuning and diagnosis.
Another application of run-time paths (outside the area of system comprehension) can
be seen in recent work on autonomic monitoring [119]. Here Mos and Murphy intro-
duce an adaptive monitoring framework whereby instrumentation is performed by
low-overhead monitoring probes which are automatically activated and deactivated
based on run-time conditions. The approach makes use of dynamic models to ac-
tivate and deactivate monitoring probes, such that the monitoring overhead is kept
to a minimum under normal conditions. If a there is a sudden degradation in sys-
tem performance monitoring probes are automatically activated to identify the bottle-
neck component(s). The dynamic models are utilised to determine which monitoring
probes should be activated to identify the bottleneck. The dynamic models consist of
the monitored components and the dynamic, ordered relationships between them (i.e.
the dynamic models are equivalent to run-time paths).
4.4.2 Alternative Representations for Component Interactions
In this subsection we discuss alternative ways of representing component interactions
recorded at run-time. The authors Jerding et al. [94] discuss a spectrum of representa-
tions that can be used for this purpose, from the most accurate and space inefficient, to
the least accurate and most efficient. Dynamic call traces are said to be most accurate
and space inefficient while call graphs are least accurate and space efficient [8] [94].
A dynamic call trace is a recording of all calls and returns that occur in a program’s
execution. They capture important information such as the sequence of calls, the con-
text in which a call was made, and repeating calls or loops. However extracting the
call trace may not be feasible for long running programs since they are unbounded
and their size is proportional to the number of calls in an execution. Call graphs on
the other hand are space efficient but are less accurate. A call graph is a compact rep-
resentation of calling behavior that summarizes all possible component relationships.
While they are bounded by the size of the program, there is much interesting informa-
14Microsoft Research. Magpie. http://research. microsoft.com/projects/magpie/.
83](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-95-2048.jpg)
![tion about calling behavior that is dropped to gain compactness. The sequencing of
calls, the context in which certain calls are made, and repeated calls are all examples
of calling behavior that is lost. Call graphs are said to be context insensitive while
call traces are context sensitive [8]. Context sensitive profiling associates a metric with
sequence of executed calls. This can be used to separate measurements from different
invocations of components [8]. Profiling tools that are context insensitive can only
approximate a program’s context dependent behaviour. In fact the inaccuracy of pro-
filers such as gprof [79] and qpt [106] can be attributed to the fact that they used call
graphs to represent context dependent profile information in a context-independent
manner [141]. Another issue with call graphs is that they can contain infeasible paths
that never actually occurred during execution. A calling context tree is an intermedi-
ate representation that is more accurate than a call graph and less inefficient than the
dynamic call trace in terms of space requirements. In fact, it is a more compact repre-
sentation of the dynamic call trace and can represent all contexts in a given trace. The
calling context tree discards any redundant paths in a trace while preserving unique
contexts. Metrics from identical contexts are aggregated in order to improve space
efficiency. Calling context trees are context sensitive and thus do not suffer from the
same inaccuracies that can occur with call graphs. The breath of a calling context trees
are also bounded by the number of methods or procedures in an application. In the
absence of recursion the depth of calling context trees are also bounded [8].
As outlined in section 4.2.1 run-time paths are essentially dynamic call traces, that
contains performance metrics, and that are recorded for each user request in a multi-
user distributed application. They contain the same important information contained
in dynamic call traces, that describes the calling behaviour of an application. How-
ever they are also unbounded and thus can be space inefficient. To solve this issue a
set of run-time paths can be easily represented by a calling context tree in the same
way that a dynamic call trace is [8]. An issue with calling context trees however is
the fact that the do not maintain the exact sequence of events as they occurred dur-
ing execution. For this reason we have not used them in this work, since we require
that the exact sequence of events is maintained to identify particular antipatterns (e.g.
communication patterns discussed in section 6.1).
4.4.3 Run-Time Interaction Tracing Approaches
The closest run-time path tracing approach to our work is the pinpoint problem deter-
mination tool. In fact we have made use of the pinpoint tracing approach in COMPAS
JEEM and have extended it such that it can be applied in a non-intrusive and portable
manner. Pinpoint has been discussed in detail and compared to our approach in sec-
tion 4.2.5.2. Essentially its main drawback is that it instruments the middleware and
thus it is not portable across different middleware implementations.
Webmon [82] is a similar tool that also instruments the middleware to trace run-time
84](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-96-2048.jpg)
![paths. However as well as monitoring the server side components, webmon also in-
struments client side components. This is performed used a cookie-based approach
and allows for webmon to trace paths beginning in the browser, rather than on the
server side. Since instrumentation is performed on the client side this approach gives
a more accurate measurement of a client’s actual wait time when a request is made
to the system. Server side only approaches record the amount of time that the sys-
tem takes to process the request but does not record the time spent communicating
between the client and the system. The main drawback of the webmon approach is
that it is not portable. Another issue outlined by the authors is the fact that the instru-
mentation process was very time consuming.
The magpie project [19] also monitors paths through distributed systems. The initial
approach [19] uses a unique identifier to identify requests and propagates this iden-
tifier from one component to the next. A later version [18] avoids the need for global
identifiers and instead correlates component events based on an application specific
event schema. Both versions make use of kernel, middleware and application level in-
strumentation. The magpie project has been implemented for Microsoft technologies.
Another approach closely related to our work has been recently presented in the lit-
erature [78]. The InfraRed tool 15 makes use of AOP to instrument the application
and has the ability to capture run-time paths, (although it presents this information
in the form of a calling context tree). Similar to our approach, InfraRed makes use
of a ThreadLocal object to record the sequence of calls. Their approach is potentially
portable across different middleware implementations and is implemented using As-
pectJ 16. It has also been shown to have a low overhead. However, the authors suggest
an alternative approach to remote call tracking and do not marshal data across the net-
work. Instead they simply relate the calls via the method name. Using this approach
it is not clear how the order of calls can be maintained across remote calls when the
system is under load.
Briand et al. have presented an approach for reverse engineering sequence diagrams
from multi-threaded [34] and distributed java applications [35]. They record traces
using an AOP monitoring approach. To distinguish between local threads they make
use of a threadID identifier, however this information is not stored in a threadlocal
object. Instead a list of threadIDs are maintained in a text file. To allow for distributed
tracing they marshall data across the network using aspects. Similar to our approach
they intercept remote calls such that information can be sent across the network such
that one can identify the client method called that called a particular remote method.
The information they send across the network however differs from the information
we use. They make use of the identifier of the client node (clientNodeID), the thread
identifier (clientThreadID), the class name (clientClassName) and the object identi-
15InfraRed, http://infrared.sourceforge.net
16The AspectJ Project, http://www.eclipse.org/aspectj/
85](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-97-2048.jpg)
![fier (clientObjectID). However the approach essentially works in the same way since
their information is used to be able to identify the client thread responsible for remote
communication. Their approach has not been applied to JEE applications.
JFluid [59] (now part of the netbeans profiler 17) is another tool which produces calling
context trees. JFluid also makes use of a per thread data structure, which the authors
call a ThreadInfo object, to collect data for construction of the calling context tree.
JFluid can perform dynamic bytecode instrumentation using a modified JVM. Tool
users can selectively profile their applications by selecting a number of root methods.
Instrumentation is injected into all methods that are reachable from the root method
specified by the user. This results in a reduced set of methods (i.e. a subgraph) that are
profiled. However an issue can potentially arise that could result in a distorted calling
context tree. This issue comes from the fact that any method within a subgraph of
profiled methods can be called from a non-profiled method outside of the subgraph.
To make sure the calling context tree is not distorted, the instrumentation has to detect
whether a method is called in the profiled subgraph context or not. To achieve this the
JFluid tool maintains the depth of calls within the subgraph (in the ThreadInfo object)
to identify if calls are made to instrumented methods from non-instrumented meth-
ods. Their approach is based on the same principles as our point of entry detection
mechanism, which also maintains the call depth in a per thread data structure (i.e. the
ThreadLocal object).
Hall [85] has implemented an approach to collect call paths in the form of the CPProfJ
profiler. CPProfJ makes use of sampling to periodically interrupt a running appli-
cation and records the runtime stacks of all threads currently in the system. From
the information recorded call path profiles are constructed. Call paths are usually
represented by a list of methods enclosed in parentheses, e.g. (method1, method2,
method3), and are defined as representing ”all stack traces that could be present when
each of the calls within the call path is present (in call path order)” [84]. Call path pro-
files report statistics about how much run-time can be attributed to the different sets
of stack traces. Unlike with run-time paths however call paths are not end-to-end and
do not maintain the exact order of all events from the beginning to the end of a re-
quest. They are thus not suitable for our purposes. In fact call paths have mainly been
applied to address lower level issues than we focus on, such as inter-thread depen-
dencies.
Finally data mining techniques have also been applied to extract dynamic dependen-
cies from JEE systems. Agarwal et al. [1] use a data mining approach to extract re-
source dependencies from monitoring data. The output is similar to that of COMPAS
JEEM. The approach is also non-intrusive since the data required to extract the depen-
dencies is obtained from existing system monitoring data. Their approach relies on
the assumption that most system vendors provide a degree of built in instrumenta-
17The NetBeans Profiler Project, http://profiler.netbeans.org/
86](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-98-2048.jpg)
![is recorded along with the number of corresponding accesses made to this method.
The above information is added to the TrackedObject instance, which maintains a list
of objects, where each object in the list contains tracking statistics for a given run-time
path (RTPStatsList).
5.3 Reconstructing Run-time Container Services
Contextual component frameworks provide services to components at run-time (e.g.
checking and ensuring that component boundary constraints are being met). In EJB
such boundary constraints include security restrictions and transactional isolation
checks. For example, an EJB component method may have the following transactional
attribute: (transaction) Required (i.e. the method will participate in the client’s trans-
action, or, if a client transaction is not supplied, a new transaction will be started for
that method). Such attributes are defined during deployment, (specified in the XML
deployment descriptor) and do not change during run-time. By analysing the differ-
ent component attributes, along with run-time paths, it is possible to reconstruct the
different container checks as they occur along the paths. For example, by analysing
the transactional attributes of each component method along a particular run-time
path, one can easily reconstruct the transactional boundaries (i.e. where transactions
begin and end) along the path. This information can be used by developers to easily
identify how the container services are being used by their application. Since a high
proportion of the application run-time can be spent executing such container services
[9] it is important that the services are utilised in an efficient manner. Inefficient use of
such services can lead to well known antipatterns (e.g. the large transaction antipat-
tern [61]). A RunTimeContainerService data structure is created during analysis which
contains information in relation to the reconstructed services. The information can in-
clude the service type, the path ID in which the service occurred and the service start
and end points along the path, as well as the methods that make use of the service.
Next we give an example run-time path and the related component service require-
ments. We show how the run-time service is reconstructed and the information that is
extracted and added to an instance of the RunTimeContainerService data structure. Fig-
ure 5.4 (a) gives an example run-time path and corresponding transactional require-
ments for each EJB component method in the path. The transactional requirements
are extracted from the XML deployment descriptor. For each new run-time container
service reconstructed an instance of the RunTimeContainerService data structure (figure
5.4 (b)) is created. To populate the data structure the run-time path data structure is
traversed and the required information extracted. Figure 5.4 (c) gives the information
that would be extracted for the example run-time path. When a service is identi-
fied its start and end points are recorded, along with the methods within the service
boundaries. This information along with the path ID is added to the RunTimeContain-
96](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-108-2048.jpg)
![erService instance, which maintains a list of objects, where each object in the list con-
tains run-time container service statistics for a given run-time path (RTPStatsList). If a
reconstructed container service is identified which is equivalent to one which has al-
ready been identified (and a RunTimeContainerService data structure has already been
created) a new instance of the RunTimeContainerService data structure is not created.
Instead the reconstructed container service statistics are added to the instance that has
already been created. Two reconstructed container services are equivalent if the con-
tain the same sequence of methods in the MethodList, are of the same Length and
both are of the same ServiceType.
5.4 Identifying Component Communication Patterns in
Run-Time Paths using Frequent Sequence Mining
Identifying component relationships allows for the construction of class diagrams
which show a high level architecture of the application, as in figure 5.5. What is not
clear from the class diagram in figure 5.5, however, is the communication patterns or
frequency of calls between the different components in the system. This type of infor-
mation is often required when trying to identify particular performance issues in the
application. It is important to be able to identify parts of the application where there
are high or unusual levels of communication between components as knowledge of
such patterns can lead to opportunities for optimizations (see section 7.3). Commu-
nication patterns between components (e.g. long running loops) can be identified
by manually analysing run-time paths. An issue with manually analysing run-time
paths from enterprise applications, however, is that (a) the paths can be very long
and (b) there may be a large number of different paths that exist for a given system.
Rather than having to analyse the paths manually, it is desirable to be able to perform
automatic analysis, such that the amount of data to be examined by developers can
be reduced. From a performance design perspective we are interested in finding fre-
quently occurring method calls across the run-time paths that might suggest instances
of potential performance design flaws in the application. That is, frequent sequences
of method calls that are resource intensive. Identified design flaws can be refactored
or optimised such that the overall system performance can be improved.
5.4.1 Frequent Itemset Mining and Frequent Sequence Mining
Frequent Itemset Mining (FIM) [2] is particularly suited to finding patterns in trans-
actional data. Transactional data in the data mining context refers to a database of
transactional records. For example a database of different customer shopping trans-
actions on a given day (known as market basket data) or a database of individuals
banking transactions. In relation to this research a transactional database refers to a
97](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-109-2048.jpg)
![Figure 5.5: Class Diagram Showing Components Relationships
database of run-time paths. However it is important not to confuse a transaction in
the data mining context with the meaning of a transaction in the computer software
context [148]. Figure 5.7 shows a transaction in the data mining context with 9 items.
For the purpose of this research each item represents a component method call in a
run-time path. However the item could as easily be used to represent other informa-
tion such as shopping items in a customer’s basket. A transactional database consists
of numerous such transactions.
An issue with FIM in relation to applying it to run-time paths is the fact that it does not
take the order of items in the paths into account. Since a run-time path maintains the
order of the events that constitute it, it is important that our analysis technique also
respects this order. Mining frequent item sequences [3] in transactional data considers
the order of the items in the transactions and thus is more suited to finding patterns in
run-time paths. FSM is a general case of FIM. FIM is concerned with discovering all
frequent itemsets within a transactional database. Most FIM algorithms work on the
following principle: an itemset X of variables can only be frequent if all its subsets are
also frequent (i.e. the downward closure property). Using this principle the general
approach is to find all frequent sets of size 1. Assuming these are known, candidate
sets of size 2 can be generated (i.e. sets {A, B} such that {A} is frequent and {B}
98](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-110-2048.jpg)
![is frequent). The frequency of the candidate sets can then be calculated by scanning
the database. Infrequent candidate sets are removed. This gives the frequent sets
of size 2. Next the candidate sets of size 3 can be constructed and their frequency
calculated. This process can be repeated until no candidate sets are generated. This is
known as the Apriori algorithm [2]. Calculating the frequency of a candidate itemset
is referred to as determining the support count of the candidate. The support count of
the candidates are calculated by reading in each run-time path from the database and
determining if each candidate is contained within the run-time path. Where this is true
the support count is incremented. For item sets the containment relation corresponds
to the set inclusion (⊆) relation. In the case of item sequences, a sequence a1, a2....an
is contained in another sequence bj1, bj2....bjm if there exists integers 1 < j1 < j2 <
... < jn <= m such that a1 = bj1, a2 = bj2, ...an = bjn [3].
5.4.2 Support Counting for Run-Time Paths
The support counting performed in FSM is useful for identifying frequent sequences
that occur across different transactions. However an issue with this approach is that
it does not take into account sequences which might repeat within a given transac-
tion i.e. if a sequence appears twice in a particular transaction the support count will
have only been incremented by 1. In some situations it can be useful to also take into
account repeating sequences within the same transaction. This is especially the case
when applying FSM to run-time paths since often we would like to be able to iden-
tify repeating sequences of method calls within a given run-time path (e.g. loops).
Weighted support counting can be applied to increment the support of an item, such
that for every instance of the sequence within a given run-time path the support is
incremented. A consequence of this is that the support count performance suffers
significantly (see section 7.3.1). With non-weighted support the support count scans
each run-time path and stops as soon as it finds the item sequence. With weighted
support however each run-time path is searched completely for all instances of the
item sequence.
Another issue that arises from using weighted support is that the output generated
can be difficult to comprehend. This results from the fact that, with weighted support
counting, the support is incremented for every instance of the sequence that occurs in
a given transaction. This problem can be clearly seen from a simple example. Figure
5.6 shows a single transaction containing 9 items. For the purposes of the example we
give the non-weighted support count and weighted support counts for this transac-
tion in respect to the item sequence 1, 2, 3 . For non-weighted support the support
count value is 1, whereas for weighted support the value is 10. The weighted support
count can be represented mathematically as being at least: (l+(n−1))!
(n−1)!(l)!
where l is the se-
quence length and n is the number of non-overlapping sequences in the transaction.
Figure 5.6 shows the different sequences that are counted as part of the weighted sup-
99](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-111-2048.jpg)
![5.4.3 Further Criteria for Interestingness
In general for FSM the interestingness [87] of a sequence is directly proportional to the
frequency of the sequence. However when analysing run-time paths from a perfor-
mance perspective we are most interested in finding frequent sequences of methods
that consume a high amount of system resources. In this situation the criteria for in-
terestingness is not the frequency of the sequence but is in fact the amount of system
resources consumed by the methods that make up the sequence. In order to account
for the resource consumption per method we can add this information to the algo-
rithm input file such that the transactional database contains not only the item IDs but
also their corresponding resource consumption information (since resource consump-
tion per methods is collected during monitoring and contained within the run-time
paths). This information can be utilised during the support counting stage to influ-
ence the outcome of the FSM algorithm. During support counting, rather than sim-
ply incrementing the support count, we instead add the resource consumption values
of the methods in the candidate sequence to the support counter. This weights the
support count value by the resources consumed such that sequences that consume a
highest amount of resources are found. Section 7.3.1 gives results of applying this ap-
proach to run-time paths and shows how it can be used to quickly identify frequently
occurring resource intensive sequences. An issue with this approach however is that
the downward closure property is invalidated since there may be instances where an
item sequence may be frequent even though some of its sub sequences are not [159].
The literature [159] suggests a solution to this problem using the weighted downward
closure property for weighted association rule mining. Future work will investigate
the applicability of this approach to FSM. Our results show that even though we in-
validate the downward closure property the approach can be effective in finding se-
quences of resource intensive methods that occur across the transactions (see section
7.3.2).
5.4.4 Preprocessing for FSM Performance Improvement
In order to apply FSM to run-time paths it is required to convert the run-time paths
from their text format to a flat format or representation (which is used by the FSM
implementation [32], described in section 7.3). This conversion involves assigning
a unique ID to each distinct method in the run-time paths and listing these IDs in
a flat format in the same order as they would appear in the run-time path. How-
ever for enterprise systems the run-time path length can be very long (in the order of
thousands of items). Run-time paths of this size can have a major impact on the algo-
rithm performance particularly in relation to FSM implementations that make use of
weighted support or non-overlapping weighted support counting approaches. In or-
der to reduce the running time of the algorithm we applied a number of preprocessing
102](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-114-2048.jpg)
![run is generally quite short and can be achieved by either setting a high support count
(close to the min. support count for the most frequent sequences of size 2 or larger)
or by limiting the size of the sequences that are generated to a particular length. The
next step is to identify if any of the frequent sequences found, in the first run, occur
as consecutive sequences in the database. If this is true the consecutive sequences can
be replaced in the database by a new item ID (replacement item) which we use to rep-
resent this sequence in the next iteration of the FSM algorithm. Replacing frequent
consecutive sequences reduces the length of the run-time path and as a result the next
iteration of the FSM algorithm will be faster than the last. This iterative approach can
be repeated until either an acceptable performance run-time (for the FSM algorithm)
is attained or until no further improvements can be made. The performance improve-
ment depends on a number of factors such as the run-time path length, length of the
consecutive sequences replaced, number of consecutive sequences replaced, database
size etc. This approach is especially effective in databases of run-time paths where
long running loops may appear. On completion of the final iteration of the FSM
algorithm any replacement items that exist in the algorithm output can be replaced
with the original consecutive sequences to give the real frequent sequences that occur
within the data.
5.4.5 Closed Sequences
In some instances a large number of frequent sequences can be generated as a result
of the FSM algorithm (see section 7.3.1). Ordering the generated sequence by their
support value will however highlight the most significant sequences (i.e. the most
frequent or resource intensive). Often however a sequence will have the same support
as many of its subsequences, and thus they will all be presented together, if ordered
by support. This gives much redundant information which can limit the value of the
results produced. By identifying closed sequences only this problem can be avoided.
Closed Mining has been mainly applied to the FIM problem. An Itemset I is closed if
no proper superset of I exists that has the same support as I. The set of closed itemsets
is a compact representation of the frequent itemsets. All frequent itemsets together
with their supports can be generated if only the closed itemsets and their supports
are known. Closed Itemset Mining can significantly reduce the number of sequences
output and avoids the problem we outlined above. It has been shown that closed
itemsets can be generated in a post processing step [181]. However, more efficient
approaches have been presented which modify the FIM algorithm (e.g. [182]). Closed
mining can also be applied to sequences [180]. Our plans for future work involve
investigating these approaches such that they can be applied to run-time paths.
104](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-116-2048.jpg)
![5.4.6 PostProcessing
A number of postprocessing techniques can be applied to the data produced from
FSM. Postprocessing can be applied to give more value to the sequences such that
interesting sequences can be more easily identified. For example the confidence
of a sequence (rule) can be calculated as follows sequence: confidence (s1 =>
s2) = support(s1) ∪ support(s1)/support(s2) [40]. The confidence, C, of the se-
quence rule above holds true if: C% of run-time paths that contain s1 also contain
s2. The confidence of a rule is sometimes referred to as its accuracy. This infor-
mation is particularly useful when analysing design of enterprise systems since it
can be useful to identify methods that appear together with a certain level of confi-
dence. For example if we have remote method A and remote method B, and the rule:
support(MethodA) ∪ support(methodB)/Support(a) = 100%, we can be sure (with a
confidence of 100%) that method A and method B will always occur together. Seeing
as A and B are both remote methods and seeing as they occur together, there may be
an opportunity to combine these methods into a single more coarse grained remote
method call and thus cut down on network traffic.
Another postprocessing technique that can be applied, in relation to run-time paths,
is adding parental information, which is contained in the paths, to the identified se-
quences. For example parent components and methods of the sequence can be iden-
tified, i.e. components and methods that called the sequence in the run-time paths.
This information can be stored along with the sequence for future use.
5.4.7 Component Communication Information for the Extracted Design
Model
In chapter 7 we show how the information extracted by the FSM algorithm can also be
used for automatic identification of performance antipatterns. For automatic antipat-
tern detection the analysis module creates a FrequentSequence data structure which
contains information relating to the frequent sequences identified in the run-time
paths (see figure 5.1). An instance of the data structure contains the path IDs of the
sequence (i.e. in what paths the sequence occurs), the items (i.e. the component meth-
ods) that make up the sequence, the parents of the sequence, the support of the se-
quence (i.e. how often the sequence occurs, or how much resources were consumed).
The support of the sequence can be broken down further to reflect how often the se-
quence occurs in the different run-time paths. Confidence rules can also be created
where appropriate (e.g. for sequences containing remote methods).
105](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-117-2048.jpg)
![5.5 Data Reduction
The amount of data produced during monitoring large scale enterprise applications is
often too large for easy human analysis. Thus it is required that the data be reduced
and summarised such that it can be presented to developers and system testers in a
more meaningful format. Summarising the data also allows for the data to be more
easily integrated with a design model.
5.5.1 Clustering Run-time Paths
Run-time paths collected from enterprise applications can often be too long, or there
may be too many, for easy human analysis. Considering the number of software com-
ponents and sub components in an typical enterprise application (this can be in the
order of hundreds), the number of paths that can be taken through the system is gen-
erally quite large. Similarly the length of a run-time path corresponding to a partic-
ular user action can also be very long (since a large number of component method
calls may be needed to service a user request). The issue of path lengths is somewhat
addressed through frequent sequence mining, since repeating sequences in a given
path can be identified and represented in a more concise manner (see loops in figure
4.12 on page 78). To address the issue of having a large number of paths we can apply
another data mining technique called clustering [87]. Clustering is the classification
of similar objects into different groups, or more precisely, the partitioning of a data set
into subsets (clusters), so that the data in each subset share some common traits.
Although there can be a large number of unique paths through the system many of
these paths can be similar and may be constructed of many of the same sub-paths.
Using basic clustering analysis we can reduce the number of paths significantly into
common path groups. In section 7.3.3 we show how this can be achieved for run-time
paths, collected from monitoring a JEE system, when we cluster paths together that (a)
call the same component methods and (b) make use of the same components. During
analysis we create a Cluster data structure (see figure5.1). An instance of a Cluster data
structure contains all the run-time path IDs that belong to a particular cluster. This
data structure also contains information relating to the clustering criteria (e.g. for (a)
above the data structure contains the list of component methods that are called by the
run-time paths in a particular cluster). Instances of the cluster data structure can be
added to the design model. This allows a developer to see (a summary of) the different
paths that are taken through the system (e.g. at a method or component level), without
the need to analyse all run-time paths recorded.
106](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-118-2048.jpg)
![The extracted design model gives two main views into the system. A transactional
view and a hierarchical view. The transactional (RunTimePath) view of the system
shows the main paths (Clusters) through the system, the most frequent communication
patterns along these paths and how particular (Tracked) data objects are being created
and used along the transactions. From the transactional design view one can also
determine the container level services that are used along the different transactions.
A more abstract hierarchical view of the system can also be obtained from the design
model, by analysing the Component details and the component callers and callees. This
shows the different component types and relationships that make up the system. The
model also shows how the components are being made available by the underlying
container through the different container pools.
5.7 Related Work
In this section we discuss and compare related work. We focus on two related areas in
particular, reverse engineering and data mining.
5.7.1 Reverse Engineering
Our work applies analysis to reverse engineer the system’s run-time behaviour and
structure from data collected by our monitoring module. Our approach mainly makes
use of dynamic data to extract the required information for design recovery. In this
subsection we outline and compare a number of related approaches.
Our research is closely related to work presented by Schmerl et al. [149]. In this litera-
ture the authors present an approach for architecture recovery using runtime analysis.
Their approach requires an engineer to create a mapping specification between low
level and architecture events. A run-time engine takes as input the mapping specifica-
tion and monitoring events from a running system and produces architecture events.
Finally, the architecture events are used to automatically create an architecture de-
scription. The work focuses on identifying architectural structure and does not focus
heavily on architectural behaviour. The authors have applied their approach to dis-
cover the architecture of an EJB application. A number of drawbacks exist however
for this approach. Firstly, an effort is required by an engineer to develop the mapping
specification. For example it took an expert in their mapping specification language
three days to construct a mapping for a sample EJB application. The engineer is re-
quired to understand the format of the monitoring data, and the workings of the ap-
plication to be analysed. Furthermore, in the case of EJB applications, the engineer
requires an understanding of the application server on which the application is run-
ning. Secondly, the approach is not portable as a new specification is required per
application, albeit the authors claim only minor modifications of the specification are
108](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-120-2048.jpg)
![required for applications running on the same middleware. For example, in the case
of the EJB application, they claim that the same specification could be easily modified
with little effort such that it could be applied to another EJB application running on
the same application server. However this would not be the case if the system was
running on an alternative application server. Thus the approach is less portable across
different middleware implementations. However, a key advantage of this approach is
that it can be applied to any system that can be monitored to provide low level data
such as object creation, method invocation and instance variable assignment.
Briand et. al present work on reverse engineering sequence diagrams from distributed
[35] and multi-threaded [34] java applications. Their approach, similar to our analysis
module, is based on populating instances of a meta model with information collected
by monitoring a running system (their monitoring approach has been discussed in
section 4.4). More precisely, the data collected at run-time is used to create an instance
of a trace meta model which in turn is transformed into an instance of a scenario meta
model. The scenario meta model is an adaption and simplification of the UML se-
quence diagram meta model. The output of their approach is scenario diagrams. They
have performed a case study to show how reverse-engineered scenario diagrams can
be used to check the quality of an applications implementation and its conformance to
the intended design. Our monitoring tool, COMPAS JEEM, produces a similar output
(in the form of run-time paths), that when represented diagrammatically results in a
sequence diagram. We use this information (along with collected performance metrics
and component metadata) to populate an instance of our run-time design meta model
which gives a structural and behavioural view of the overall system design from a per-
formance design perspective. Their approach also addresses the issue of identifying
repeating sequences in a trace as a result of loops by instrumenting the control-flow
structures. On the other hand our approach makes use of FSM to identify repeating
patterns. An advantage of their approach is that they can distinguish the execution
of loops from incidental executions of identical sequences in different contexts. Our
approach can not make this distinction, however, it is less intrusive since it does not
require such low level instrumentation.
Static and dynamic data has been applied by Richner and Ducasse [146] to recover
high-level views of the system architecture through user defined queries. Both the
dynamic and static information is modelled in terms of Prolog facts. They provide a
logic layer made up of rules above the database of facts to allow sophisticated queries
about the structure and behaviour of the application. The main advantage of their
work is that it allows an engineer to drive the investigation of the application through
an iterative process, much like Murphy’s static relexion approach [122]. In compari-
son our approach makes use of mainly dynamic information. Although source code
analysis is not applied, we also make use of component meta-data to obtain structural
and functional information on the different components. We also extract information
pertaining to the components’ required container services.
109](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-121-2048.jpg)
![The reconstruction of container services using component meta-data and, what the
authors call, binding graphs has previously been applied by Trofin and Murphy [166]
to identify redundant boundary checks within contextual container services. Binding
graphs (similar to call graphs) contain component dependencies and can be collected
statically or dynamically. We reconstruct the container services to identify antipat-
terns rather than for redundancy checking, and in contrast to their approach our re-
construction process requires the order of calls to be maintained. Thus we make use
of run-time paths rather than binding graphs for our reconstruction process.
5.7.2 Data Mining
In this thesis we have applied both FSM and clustering to data produced from moni-
toring component based enterprise applications. FSM is applied to identify interesting
component communication patterns. We have modified the support counting tech-
nique of the FSM algorithm to identify repeating sequences within a given run-time
path such that patterns as a result of long running loops, for example, can be identi-
fied. Previous work has addressed this issue of counting the support of repeating se-
quences within data using a sliding window approach [111]. However this approach
has been applied to event streams and is not suitable for data that fits naturally into
distinct transactions (e.g. run-time paths).
We have also weighted the support counting to take into account performance metrics
(and not merely frequency) such that resource intensive patterns can be identified.
The idea of weighting support count to take account of alternative criteria other than
frequency has been previously been applied to association rules in the literature [159].
Other examples of weighting the support with criteria other than frequency include
[107] and [109] where the support is weighted by time values.
In this work we introduce the notion of non-overlapping weighted support. Non-
overlapping weighted support allows for a more human readable output of the FSM
algorithm when repeating sequences within a single run-time path are accounted for.
Similar to our non-overlapping weighted support approach, mining frequent closed
sequences can produce more compact results [173]. In fact the sequences produced
using our non-overlapping weighted support approach can be further compacted by
filtering the results produced for closed sequences only. Maximal sets are a subset of
all closed sets and maximal mining techniques can be applied to reduce the output
volume further [41].
To reduce the run-time of FSM when applied to long run-time paths we have ap-
plied a number of preprocessing techniques. In particular we have made use of
parental information to split the run-time paths into shorter subpaths. We have also
applied a technique called Consecutive Iterative Replacement (CIR) which replaces
frequent consecutive item sequences with replacement items to reduce the run-time
110](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-122-2048.jpg)
![path length. Reducing the run-time of sequential pattern mining algorithms by taking
advantage of consecutive sequences has been previously applied and documented in
the literature [108]. The authors propose an extension of the SPADE algorithm for FSM
called GO-SPADE. One advantage of our Consecutive Item Replacement approach is
that it works by modifying the database of run-time paths and no modification of the
FSM algorithm is required. To the best of our knowledge the use of parental informa-
tion has not previously been applied to improve the run-time of FSM algorithms by
reducing the length of the algorithm input data.
In this work we have also made use of clustering to reduce and summarise the data
collected during monitoring. That is, we have applied clustering and statistical anal-
ysis to run-time paths to group the large volume of different paths produced. Chen
et.al [45] have previously applied clustering and statistical anslysis to run-time paths
collected from monitoring enterprise systems for the purpose of problem determina-
tion (i.e. identifying faulty software components). They make use of data clustering
and statistical analysis techniques to correlate failures of requests to the components
most likely to have cause them.
111](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-123-2048.jpg)
![6.1 Antipatterns Categories
In this section we categorise the antipatterns, that we detect, based on the similarities
in the type of data used to detect them. This categorisation was a result of analysing
JEE performance design and deployment antipatterns from the literature 1 [161] and
[61]. In the following section we give examples of rules that can be used to detect
antipatterns from a number of the different categories.
1. Antipatterns Across or Within Run-Time Paths: In order to detect the antipat-
terns in this category, information is required on how often particular compo-
nents or services occur within the same, or across the different run-time paths.
For example, a large number of session beans occurring across the different run-
time paths may signify the existence of the Sessions-A-Plenty antipattern [61]
(i.e. the overzealous use of session beans, even when they are not required). An-
other example of an antipattern in this category is the Large/Small transaction
[61] antipattern whereby an inefficient transaction size is set resulting in either
large, long living transactions or many inefficient short living transactions.
2. Inter-Component Relationship Antipatterns: The inter-component relation-
ship antipatterns can be identified by analysing the relationships that exist
between the different components in the system. Where inappropriate rela-
tionships exist antipatterns can be detected. A typical example might be the
Customers-In-The-Kitchen Antipattern [161] where web tier components di-
rectly access persistent objects (e.g. Entity Beans). Other antipatterns that can be
detected by analysing inter-component relationships include the Needless Ses-
sion Antipattern (discussed in section 6.3), the Transparent Facade Antipattern
[61] and the Bloated Session Antipattern [61].
3. Antipatterns Related to Component Communication Patterns: This category
of antipatterns can be identified by analysing the communication patterns be-
tween particular component types. For example a high level of fine grained
chattiness between remote components (i.e. the Fine Grained Remote Calls An-
tipattern, which is also known as the Face Off Antipattern [161]). Another ex-
ample might be an unusually bulky or high amount of communication between
the business tier and the database which could signify the existence of an Appli-
cation Filter/Join Antipattern [161]. Other examples include the Eager Iterator
Antipattern [161].
4. Data Tracking Antipatterns: The data tracking category of antipatterns can be
detected by analysing how data objects are created and used across particular
run-time paths. A typical example of this antipattern is an unused data object,
1Precise Java Website, http://www.precisejava.com
113](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-125-2048.jpg)
![whereby a data object is created and passed along the run-time path, but the in-
formation which it contains is never accessed or used. An EJB specific variation
on this antipattern is called aggressive loading of entity beans, whereby an en-
tity bean is loaded but only a small proportion of the persistent data it contains
is ever accessed 2.
5. Pooling Antipatterns: Antipatterns in this category can be detected by
analysing the related pool size and queue information. For example, to deter-
mine if an inefficient pool size has been specified we need to consider the pool
size, the number of object instances being used, on average, and the average
queue length. Examples of these antipatterns include inefficient configuration of
any of the container pools (thread pools, session bean pools, entity bean pools,
database connection pools)2.
6. Intra-Component Antipatterns: Intra-component antipatterns can be detected
by analysing internal component details. Examples include the Simultaneous
Remote and Local Interfaces Antipattern [161] or the Rusty Keys antipattern
[161]. In both these cases the antipattern can be identified by analysing the com-
ponent meta data.
6.2 A Rule Engine Approach for Antipattern Detection
The detection module is responsible for identifying performance design and deploy-
ment antipatterns in the extracted design model. Detection can be achieved through
the use of a knowledge base or rule engine. The PAD tool makes use of the JESS rule
engine [69] for this purpose. JESS is a Java based rule engine, whereby information
can be loaded into the rule engine in the form of JESS facts. Rules can be defined to
match a number of facts in the knowledge base. If a rule matches the required facts,
the rule fires. When a rule fires, particular tasks can be performed. Such tasks are out-
lined in the rule definition. The information captured in the extracted design model
can be easily loaded into the rule engine by converting the entities in the model to JESS
facts. JESS will accept Java objects as input and automatically convert them to JESS
facts. During analysis the model entities are created in the form of Java objects. The
objects created are loaded into JESS by the detection module. To detect antipatterns
in the design, rules that describe the antipatterns must be written and loaded into the
rule engine.
Jess provides for both forward and backward-chaining rules. Rules are generally in
the form of ”if-then” statements, where ”if” a number of conditions (specified in the
rule) match facts in the database, ”then” the rule fires. Forward-chaining rules fire
when the conditions fully match the facts in the database. Backward-chaining rules
2Precise Java Website, http://www.precisejava.com
114](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-126-2048.jpg)
![There are two main drawbacks of using multi user mode however. Firstly, it requires
a load to be generated on the system. In most cases this requires the creation of an au-
tomated test environment (e.g. using load generation tools). Secondly, a large amount
of data is produced when monitoring enterprise systems under load. In particular,
our monitoring module produces a large amount of run-time path information. This
issue can be addressed however by applying the clustering and statistical analysis
techniques outlined in section 5.5.
6.5 Related Work
In this section we discuss and compare related work to the research that has been pre-
sented in this chapter and to our overall antipattern detection approach. We discuss
in particular alternative antipattern categorisations that have been presented, as well
as performance testing techniques, detection approaches for pattern and antipattern
identification and finally work in the area of antipattern detection closely related to
our approach.
6.5.1 Antipattern Categorisation
Antipatterns have been categorised in the various literature in which they have been
documented. For example, Brown et. al [36], distinguish between development, ar-
chitecture and project management antipatterns. The literature [161] and [61] both
categorise their antipatterns according to the related component types which are af-
fected. For example, antipatterns related to entity beans, antipatterns related to ses-
sion beans. In contrast we have taken technology specific (JEE) performance antipat-
terns and categorised them according to the data required to detect them. A similar
approach has previously been taken by Reimer et al., who have categorised program-
ming errors based on the algorithms used to detect them [145]. Our antipattern cate-
gorisation also differentiates between performance design antipatterns, performance
deployment antipatterns and performance programming errors. Similarly, Moha and
Gueheneuc [116] provide a taxonomy in which they attempt to clarify the difference
between errors, antipatterns, design defects and code smells. In their analysis they
define code smells as intra-class defects and design defects as inter-class defects. Our
antipattern categorisation is at a higher more abstract component level. Hallal et. al.
[86] provide a library of antipatterns for multi-threaded Java applications. They also
distinguish between errors and design antipatterns. They classify the multi-threaded
antipatterns that they present following a categorization, which reflects the effect of
the antipatterns on the outcome of the tested program.
119](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-131-2048.jpg)
![6.5.2 Performance Testing
Early analysis of systems from a performance perspective has mainly focussed on
modelling techniques whereby models of the system are created at an early stage us-
ing system artifacts and estimates of resource consumption, user load etc. In the re-
search community modelling has been applied to enterprise systems [168] [76], how-
ever in reality they are rarely used for performance analysis. This is due to the fact that
they can be inaccurate, and time consuming to build and maintain for large enterprise
applications.
Performance testing of running systems is generally the preferred choice of industry
today. Often, however, testing is left until late in the development cycle and as a result
issues identified can be expensive to fix and can lead to major project delays. Ideally
performance testing should be carried out as early as possible in the development cy-
cle. Recent research efforts have focused on early performance testing through the
creation of component stubs [58] or through the automatic generation of component
test-beds 3. This allows for a running system to be created, deployed and analysed
early in the development cycle such that performance tests can be run. These ap-
proaches allow for the system design to be analysed at an early stage in development
whereby performance issues can be identified even before much of the coding takes
place. Our work complements these initiatives since design level antipatterns can be
identified in a real running system or in test bed or stub applications. Our antipat-
tern detection approach can be applied early in the development process as soon as
a running system is in place and should be carried out iteratively as the application
evolves during development. The approach we present is suitable for any iterative
development processes whereby a running prototype of the application is produced
early in development (e.g extreme programming [24]).
Current tools for performance testing generally fall into the categories of load genera-
tion tools or performance profilers (see section 2.8.2). Current tools merely preset the
data collected to the user unanalysed. Thus tool users must perform analysis of the
data manually which can be a difficult task. Our work builds on current performance
profilers. The PAD tool performs automatic analysis of the data produced during
monitoring to automatically identify design issues.
6.5.3 Detection Techniques
The detection technique presented in this chapter makes use of a knowledge base/rule
engine based approach. A number of previous pattern detection approaches have
made use of the logic programming language Prolog for software analysis [103] [49].
Prolog allows for the creation of a knowledge base of facts and for the creation of
rules that can be used to query the information in the knowledge base. In general a
3AdaptiveCells/J, http://adaptivecellsj.sourceforge.net/
120](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-132-2048.jpg)
![software system’s design is represented by Prolog facts and rules are used to express
known patterns. Prolog works using backward-chaining. A disadvantage of using
Prolog for pattern detection is that the creation of new rules to detect new patterns
requires an understanding of the Prolog language. The literature [29] introduces the
Relational Manipulation Language (RML) which like Prolog is based on predicate cal-
culus. The authors claim that RML is simpler to use than Prolog and can be used for
different program analysis tasks such as pattern detection, antipattern detection, cal-
culation of design metrics etc. The authors have implemented the CrocoPat tool which
interprets programs written in RML. The tool was designed for the analysis of graph
models of software systems and it has been shown to scale well on large systems. In
comparison our detection approach makes use of very simple forward-chaining rules
in the form of ”if-then” statements. We do not require backward-chaining or the use of
logic programming. The analysis we perform prior to detection (see chapter 5) allows
for simple rules to be written. If analysis was not performed, our rules would need
to be more complex in order to identify the relationships which are extracted during
analysis.
A number of techniques have been applied to identify structural patterns in software.
For example, graph matching [138], graph rewrite systems [123], or meta program-
ming [11] [98]. The representations used to model the software structure include
graph models [138], abstract syntax graphs [123], or design description languages [11]
[98]. However such approaches are less suitable for the detection of design and de-
ployment antipatterns. This results from the fact that the design model we extract
from the running application contains information other than structural information
that can not be represented easily when applying these techniques. For example, be-
havioural information (e.g. repeating sequences of events), performance metrics, re-
constructed container services and object pool information. Instead we have made use
of a knowledge based approach which allows for this information to be represented in
the form of knowledge base facts. An alterative way of representing this information
might be to store it using a relational database. Vokac [172] has previously made use
of a relational database for the purpose of structural design pattern detection.
Formal concept analysis (FCA) has also been used for the purpose of pattern detection
[39] [164]. FCA does not require a library of predefined rules. Instead FCA works by
identifying groups of classes that share common relations. FCA approaches try to
infer patterns/antipatterns from the code or system design. In contrast, our approach
tries to identify antipatterns based on a library of defined antipatterns to be matched.
6.5.4 Antipattern Detection
Software pattern and antipattern detection techniques have been discussed in section
2.10. Many of the pattern detection techniques could potentially be applied to detect
instances of antipatterns. For example, with most of the pattern detection approaches
121](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-133-2048.jpg)
![discussed, the patterns and application are described in terms of structural (and in
some instances behavioural) representations which are analysed to identify instances
of the pattern. Since design flaws or antipatterns can also be described in this way,
the same approaches can be used to identify them. Some work however has focused
more specifically on antipattern detection [83] [49] [117] [114] (see section 2.10). The
aforementioned approaches all rely on extracting information from the system source
code.
A run-time antipattern detection approach very closely related to our work, that ex-
tracts information from a running system, has recently been presented. Eologic 4 have
developed an antipattern detection tool for JEE applications. This tool identifies a
number of general JEE antipatterns and presents the user with possible solutions.
This tool, similar to our approach, extracts a model from a running system 5. The
tool makes use of JMX and AOP to monitor a running system and to extract an Entity-
Relationship-Attribute model. A major drawback of the tool is that it is currently not
portable across different application servers. Also it does not detect antipatterns based
on communication patterns or object tracking. Furthermore it is designed to be used in
single user mode and does not perform data reduction when monitoring applications
under load.
Other work closely related to our antipattern detection work has been presented by
Bischofberger et al. [31] in the form of an architecture conformance checking tool
called Sotograph. Sotograph makes use of cycle analysis and metrics based analy-
sis to identify architectural level problems in the application. This is performed by
analysing the system source code. Many of the issues identified are similar to the an-
tipatterns that we try to detect. For example the author’s list architectural issues that
can be identified such as large artifacts, small artifacts and bottlenecks. These issues
are closely related to a number of antipatterns from the inter-component relationship
category that we detect using the PAD tool.
Similarly, IBM’s Structural Analysis for Java (SA4J) tool 6 analyses structural depen-
dencies of Java applications in order to identify structural antipatterns within the ap-
plication. SA4J measures the quality of the application structure by evaluating the
web of dependencies between packages, classes, and interfaces of a Java application.
The Smart Analysis Based Error Reduction Tool (SABER) [145] is also closely related
to our work. SABER uses a combination of domain specific knowledge and static
analysis to identify high impact coding errors in large commercial (JEE) applications.
This tool has been successfully applied to detect hundreds of errors in these appli-
cations. The SABER tool identifies lower level issues (e.g. improper clean up of re-
sources, concurrency issues) than detected by the PAD tool. However similar to our
4Eologic’s Eosense for JEE, http://www.eologic.com/eosense.shtml
5Derived Model Analysis,Detecting J2EE Problems Before They Happen, Alan West and Gordon
Cruickshank, December, 2006, http://dev2dev.bea.com/pub/a/2007/07/derived-model-analysis.html,
6IBM’s Structural Analysis for Java tool, http://www.alphaworks.ibm.com/tech/sa4j
122](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-134-2048.jpg)
![7.1 Chapter Overview
In this chapter we present a wide range of results that have been carried out in order to
validate our research. Section 7.2 gives results from a number of different tests carried
out on the COMPAS JEEM run-time path tracing tool (see section 4.2 for a description
of this tool). In subsection 7.2.1 we show how we applied COMPAS JEEM to a number
of JEE applications and how we could quickly deduce the system structure of the
applications by analysing the run-time paths collected. For one of the applications we
show that we could easily identify a number of performance design antipatterns from
the system structure that was deduced. We also show the portability of our approach
(in subsection 7.2.2) by applying it to applications running on application servers from
a number of different vendors. In subsection 7.2.3 we give details on the performance
overhead incurred by applying COMPAS JEEM to a JEE application.
Section 7.3 presents tests carried out on the analysis module components. In partic-
ular in section 7.3.1 results are given which show the performance run-time of the
different FSM approaches, which have been presented in section 5.4, with respect to a
number of different databases. In section 7.3.2 we also show how FSM was applied to
data collected from two JEE applications and how we used the results to successfully
(manually) identify performance design flaws within them. In particular using our
technique we quickly identified a real design flaw in an IBM application (beta ver-
sion). We also show (in section 7.3.3) how information collected during monitoring
was reduced by our analysis module’s data reduction mechanism, when information
was collected under load.
In section 7.4 we present results from applying our PAD tool to two JEE applications.
More precisely, we detail how the PAD tool was applied to two JEE applications i.e. a
sample JEE application (see section 7.4.1) and a real large scale enterprise system (see
section 7.4.2). Antipatterns from all the categories outlined in section 6.1 have been
detected. For each antipattern detected we give a brief description of the antipattern
and the antipattern category. We also give the related information (PAD output) which
is presented to the tool user upon detection. Using this information the tool user
can easily determine the severity of the problem and a decision can be made as to
whether refactoring is required or not. We do not show performance improvements
that can be attained by refactoring the antipatterns detected since these improvements
have already been well documented [161] [61]. Also performance improvements can
be very much application specific and vary greatly depending on the severity of the
antipattern.
Finally in section 7.5 we revisit our contributions claimed and the validation criteria
which we previously outlined. In this section we discuss how our validation criteria
have been met.
126](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-138-2048.jpg)
![The diagrams that we produced can be used for identifying performance design is-
sues within enterprise applications. For example, by analysing figure 7.2 we quickly
identified the existence of two EJB performance antipatterns in the Duke’s Bank ap-
plication. The diagram consists of four EJBs two stateful session beans and two entity
beans.
• Antipattern 1: The first antipattern we identified is known as the Conversa-
tional Baggage antipattern [161]. It identifies situations where a stateful session
bean is used but not required. This can be wasteful since there is a performance
overhead required to manage stateful sessions within the EJB container. Often a
(more lightweight) stateless session bean would suffice. Figure 7.2 shows that a
stateful session bean is invoked for each user request. However there is no ob-
vious need for a stateful session bean in any of the user actions we performed,
as state is not maintained across the different web pages returned for the related
user requests. The stateful session beans in figure 7.2 could easily be replaced by
more lightweight stateless session beans which is the suggested solution to this
antipattern.
• Antipattern 2: The second issue we identified with the Duke’s Bank application
was that the entity beans exposed remote interfaces. The entity beans in this
application (unlike the session beans) are only accessed by EJBs which would
most likely be deployed in the same container. Thus there is no real need for the
entity beans to expose remote interfaces. The beans can be refactored to expose
local interfaces and thus reduce the overhead associated with remote method
calls.
PlantsByWebsphere is a sample e-commerce application that comes installed with
Websphere 6.1. Websphere 6.1 contains a websever that conforms to the servlet 2.4
specification and thus can monitor inter web component calls. As above we per-
formed a number of user actions in the PlantsByWebsphere application and con-
structed a diagram by analysing the corresponding run-time paths. In total 19 user
actions were invoked. Figure 7.3 shows the diagram that was constructed. The dia-
gram shows the different web and business tier components that made up the applica-
tion (for simplicity we did not analyse the JDBC components for PlantsByWebsphere).
No antipatterns were identified from the diagrams for this application.
As shown in figures 7.2 and 7.3 COMPAS JEEM monitors only the main components
(JSPs, Servlets, EJBs and JDBC calls) that make up the J2EE system. Smaller utility
classes and the underlying middleware calls are omitted. This allows for run-time
paths (and thus diagrams) that contain only the main components of the applica-
tion. As a result our run-time paths and diagrams are not cluttered with unimportant
method calls that play only a minor role in the overall application design structure.
130](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-142-2048.jpg)
![7.2.3 Performance Overhead
To assess the performance overhead incurred by COMPAS JEEM we instrumented
the web and business tiers of the PlantsByWebsphere application described in section
7.2.1 and ran a number of performance tests. The test environment was made up of
three machines. The first machine (Pentium 4, 2.25GHz, 1GB RAM) was used for load
generation. Apache Jmeter was used as a load generation tool. PlantsByWebsphere
was installed on WebSphere 6.1 which ran on a second machine (Pentium M, 1.7GHz,
1GB RAM). Finally a third machine (Pentium 4, 1.6 GHz, 768 MB RAM) was employed
to log the data produced by COMPAS JEEM remotely such that the performance over-
head on the machine running the JEE application could be kept to a minimum. This is
common practice for production environments. To assess the performance overhead
we ran a number of test cases with different user loads for two versions of the sample
application (i.e. one version included COMPAS JEEM instrumentation and the other
version did not). The test cases consisted of a warm up period (to warm up the JVM as
suggested in the literature [38]), followed by a measurement period. During the differ-
ent test cases we observed the response time of the application for the following loads:
20 users, 50 users, 100 users, 150 users, 200 users. Users entered the system at a rate of
one user per second with a random delay of between 1-2 seconds. Each measurement
period lasted 30 minutes. For each of the given loads three test cases were run and an
average value obtained. To assess the performance overhead we analysed the aver-
age response time of the instrumented application versus the average response time
of the non-instrumented application for the same user load. The results are shown in
figure 7.5. Note that the response time for each run is an average value that has been
normalised based on the average response time for 20 users (non-instrumented).
Application Version 20 Users 50 Users 100 Users 150 Users 200 Users
Non-Instr. resp. time 1 1 1.1 4.12 X
Instr. resp. time 1 1 1.1 4.9 X
Overhead 0 0 0 19% X
Figure 7.5: Performance Overhead Test Results
At 20 and 50 users there was no noticeable difference (on average) in response time
for either the instrumented or non-instrumented version of the application. Thus there
was no noticeable overhead for these loads. At 100 users there was a 10% increase in
response time for both the instrumented and non-instrumented versions compared
with 20 users. Again there was no noticeable overhead that could be attributed to
COMPAS JEEM. At 150 users, however the system was heavily saturated and response
time increased significantly. For the non-instrumented application response time in-
creased by 310%. The instrumented version at 150 users showed an increase of 390%.
Therefore there was a 19% overhead that could be attributed to COMPAS JEEM at 150
users. At 200 users (for the default application server settings) the system under test
had reached its capacity for both the instrumented and non-instrumented application.
133](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-145-2048.jpg)
![able information for developers trying to solve design problems. From the output we
could see that the ResultSet.next() method was repeatedly being called across a num-
ber of transactions in sequences of various sizes (i.e. loops in the run-time paths). Our
FSM implementation was modified to record and output the transaction number in
which the sequences occurred. Using this information we could easily identify these
resource intensive loops and the particular run-time paths in which they existed. We
could also see from the output that the AccountControllerBean.getDetails() method
was not executed in a looping sequence and occurred at most once per transaction,
except in one instance where it was called twice.
The second application was an early beta version of the collaborative Workplace (en-
terprise) application from IBM. Again we collected run-time paths from the applica-
tion by performing different user actions. In total we collected 233 run-time paths from
the application. We did not collect performance information for these tests (due to our
limited access to the system) and thus had to rely on identifying the most frequent
sequences of methods to find design flaws in the application. We applied the FSM-
NO-CIR approach to the database (IBM-enterprise-app). Initially (min. support 400)
we identified that the most frequent method occurred 1464 times across the 233 trans-
actions. On closer inspection (min. support 30) we identified that the same method
made up a consecutive sequence of size 24 occurring 58 times across the transactions
(making up a total of 1392 calls). We made modifications to the original FSM imple-
mentation such that it listed the transactions in which the sequence occurs and how
often for each particular transaction. From this we could see that for one particular
transaction the sequence of size 24 occurred 33 times. The transaction length was
1617 (method calls). We identified 792 (24x33) of these calls were to one method. The
method being called was an EJB Entity bean method. Entity beans are used in JEE to
access persistent data in the database from the application tier. There are a number
of common design flaws that occur in these systems related to this kind of database
activity i.e. looping through the database from the application tier which is inefficient
[161]. On reporting that we had identified these potential loops in the application
tier to the IBM development team they confirmed that we had indeed identified a de-
sign issue which they subsequently rectified for the release. Our approach allowed
an individual unfamiliar with the internals of the application (e.g. a system tester) to
quickly identify potential problems in the application and the (run-time) context in
which these problems occurred. This information can be quickly used by developers
to refactor the application.
7.3.3 Data Reduction Results
To show the effectiveness of the data reduction techniques discussed in section 5.5 we
have applied them to data collected from a JEE application under load. For this test
we loaded the Duke’s Bank sample e-commerce application with 40 users for a five
142](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-154-2048.jpg)
![7.4 PAD Tool Results
In this section we present results attained from applying the PAD tool to two JEE
applications i.e. a sample JEE application and a real large scale enterprise system.
Section 7.4.1 details the test environment for our tests on the Duke’s Bank sample
application, modifications that we made to the application and our test results. A
number of modifications were made to Duke’s Bank in order to introduce a number
of antipatterns for detection purposes. Section 7.4.2 gives the test environment for our
tests on a real JEE application from IBM called Workplace and the results from these
tests. Antipatterns from all categories outlined in section 6.1 have been identified. For
each antipattern identified we describe the corresponding PAD tool output.
7.4.1 Antipatterns Detected in the Duke’s Bank Application
The first application we applied the PAD tool to, in order to identify performance
design and deployment antipatterns, was Duke’s Bank. Duke’s Bank is an online
banking application. When a user logs in to the Duke’s Bank application he/she can
perform the following actions: log on, view a list of accounts, view an individual ac-
count’s details, withdraw or lodge cash, transfer cash from one account to another or
finally log off. For our tests each of the different (8) user actions was performed. COM-
PAS JEEM was used to collect run-time paths. We instrumented the application source
code manually to perform object tracking, since COMPAS BCI could not run on the
1.4.1 JVM that was used. Duke’s Bank was deployed on the JBoss application server
(version 3.2.7) with a MySQL database (version 4.0.2) as the backend. Our MEJB mon-
itoring tool was used to interface with the application server to collect information
on the server resources. For multi user mode (which was required to identify the In-
correct Pool Size antipattern) the open source Apache JMeter load generation tool was
used to load the application. Two versions of dukes bank were tested, the original ver-
sion and a modified version with a number of antipatterns added. The original dukes
bank application consists of 6 EJBs (4 of these were invoked during the tests, see figure
7.2 for a class diagram). We also modified the original version of duke’s bank to add
a number of antipatterns to be detected by the PAD tool such that antipatterns from
all categories discussed in section 6.1 were present (see figure 7.10 for a class diagram
of the modified version of dukes bank). The antipatterns introduced are described
in detail below. For each antipattern detected we give its corresponding category as
described in section 6.1. In total 3 antipatterns were detected in the original version of
Duke’s Bank by the PAD tool:
• Conversational Baggage Antipattern [161] (category 1): This antipattern de-
scribes a situation where stateful sessions beans are being used but are not nec-
essarily required. Stateful session beans maintain state on the application server
between client requests and should be used only when there is a clear need to
144](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-156-2048.jpg)
![do so. Stateless session beans will scale better than stateful session beans and
should be used when state does not need to be maintained. Detection of this
antipattern involves flagging the creation of unusually high numbers of stateful
session beans across the run-time paths. A potential instance of this antipattern
was flagged by the PAD tool when Dukes Bank was analysed as the number
of stateful session beans was above the user defined threshold. This threshold
was set to zero for the dukes bank application since there is no noticeable state
maintained from one user action to the next. When the potential antipattern was
detected the PAD tool showed us that stateful session beans were used in 100%
of the run-time paths where the business tier is invoked. On closer inspection
of the application (source code) we saw that indeed the stateful sessions could
have been replaced with stateless sessions to improve scalability. In total 2 state-
ful sessions could have been replaced.
• Fine Grained Remote Calls (also known as the Face Off antipattern [161]) (cat-
egory 3): This antipattern describes a situation where a number fine grained calls
are consistently made from a client to the same remote bean. This results in a
high number of expensive remote calls. A better approach is (if possible) to make
a more coarse grained call that performs the combined tasks of the fine-grained
calls. Performing a single coarse grained call instead of a number of fine grained
calls will reduce network latency and thus the overall time required for the user
action to execute. An instance of this antipattern was identified by the PAD tool.
The tool identified a frequent sequence which contained fine grained remote
calls. In fact it identified that the remote methods AccountBean.getType and Ac-
countBean.getBalance appeared together 100% of the time they were called (i.e.
the sequence had a confidence value of 100%). Thus it would be more efficient
to combine these calls into a single coarse grained remote call. This antipattern
is far from severe in this instance, however the identification of this antipattern
shows that the tool can indeed identify antipatterns in this category. The rule to
identify this antipattern can in fact be modified with a user defined threshold to
only flag more severe instances.
• Remote Calls Locally (also known as Ubiquitous Distribution [161]) (category
2): This antipattern describes a situation where beans that run in the same JVM
as the client are called through their remote interfaces. In this situation the client
has to perform an expensive remote call even though the bean is local. While
some containers can optimize in this situation, this optimization is not a stan-
dard feature of JEE. A better approach would be to write the beans with local
interfaces (instead of remote, provided the beans are not also called from remote
clients) such that they can be accessed in a more efficient manner. The PAD tool
identified this relationship between components. In fact all (4) beans invoked in
the Duke’s Bank application are accessed through remote interfaces even though
their clients are within the same JVM.
145](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-157-2048.jpg)
 (category 2): The application was modified such that components
in the web tier were directly accessing entity bean components (see figure 7.10).
This antipattern can cause a number of different performance issues as docu-
mented in the antipattern description (e.g. problems with transaction manage-
ment). Furthermore it can create maintainability issues since it mixes presenta-
tion and business logic. The PAD tool identifies the inappropriate component
relationships and flags them as antipattern instances. The component identified
was the accountList.jsp which was modified to call the AccountBean (entity) di-
rectly.
• Needless Session Antipattern (category 2): Another antipattern added was the
needless session antipattern. This antipattern is described in section 6.3 and out-
lines a situation where a session bean is used but not required. In the Duke’s
Bank application we added a session bean that was being used to calculated the
current time and date. This function could have been easily carried out by a
POJO which would have been a more efficient solution. The antipattern was
detected by the PAD tool which reported that the session bean (CalculateDate-
Bean) had no relationships with any persistent components (e.g. entity beans or
database components) and thus was a potential needless session bean.
• Application Filter Antipattern [161] (category 3): The application filter antipat-
tern describes a situation where large amounts of data are retrieved from the
database tier and filtered in the application tier. However, databases are de-
signed to process filters a lot faster than can be done in the application tier. Thus
filters should, where possible, be implemented in the database tier. We modified
the Duke’s Bank application to include an application filter. The dukes bank
application can retrieve account information for a customer that has logged in.
When this is performed the application performs a search for the accounts that
are owned by the particular user. We created an application filter in this situa-
tion by modifying the SQL statement which filtered the information (using the
following SQL statement ”select account id from customer database where cus-
tomer id = ?”) to instead retrieve all accounts and send them to the application
tier for filtering. In the application tier, filtering of the data was performed by
checking the details of each account to see if it matched the id of the customer.
We applied the PAD tool to the modified version of Dukes Bank. The tool suc-
cessfully identified the application filter that had been added to the application.
The tool identified two frequent sequences of communication: (1) between a ses-
sion bean (AccountControllerBean) and an entity bean (AccountBean) (see figure
6.3 on page 117) and (2) between the same entity bean (AccountBean) and the
146](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-158-2048.jpg)
![database in these paths. The first sequence of size 1 (AccountBean.getDetails)
occurred 1180 times across the different run-time paths. The second sequence of
size 2 (ResultSet.next, ResultSet.getString) occurred 6336 times across the run-
time paths. It was evident from this information that an application filter existed
in the application, especially when considering that this the amount of activity
was occurring in single user mode. In fact the PAD tool identified that by mod-
ifying this filtering 7/11 of run-time paths collected during the test run were
affected.
• Unused Data Object/Aggressive Loading (category 4) 12: A second antipattern
was identified when the PAD tool was applied to the dukes bank application
modified with an application filter antipattern. The antipattern detected was the
unused data object antipattern/aggressive loading antipattern. This antipattern
describes a situation whereby information is loaded from the database, but the
data (or at least a high percentage of the data) is never used. A solution to this
problem can be to refactor the application such that information is not retrieved
if it is never actually required. Often however the information may be required a
small percentage of the time. In such circumstances the information can be lazy
loaded when it is required. This antipattern occurred in the application filter
above. During filtering a check is performed to check each account to see if its
owner id matched that of the (logged in) customer’s. To obtain each account’s
owner id, the application loads the account information from the database into
an entity bean. An AccountDetails Data Transfer Object (DTO) [7] is then cre-
ated by the entity bean with the account information. Finally during the filter-
ing the DTO is accessed to obtain the account owner id. Rather than loading all
the account information into the entity bean, and subsequently into the DTO, a
lazy loading approach can be used to load only the information that it generally
needed (i.e. the account owner id). If the remaining (unloaded) information is
required it can be loaded later. This antipattern will in fact be removed if the
application filter is pushed into the database tier as suggested in the antipat-
tern solution for the application filter. However there may be situations where it
may not be possible to easily remove the application filter and where lazy load-
ing can be applied. Lazy loading can in fact be applied in any situation where
only a small proportion of the entity bean fields are ever accessed. The PAD tool
provided the following information when this antipattern was identified: The
AccountDetails Object was created on average 1886 times across the 7 run-time
paths with a maximum value per path of 2400 times and a minimum value of
1200 times. The object has 8 accessor methods. Each method is given below with
the corresponding average number of times it was accessed per run-time path:
getCustomerID’s 1886 times, getType 1.4 times, getDescription 3 times, getBal-
ance 2.6 times, getAccountId 8.1 times, getCreditLine 2.3 times, getBeginBalance
12Precise Java Website, http://www.precisejava.com
147](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-159-2048.jpg)
: Another an-
15IBM Workplace, http://www.ibm.com/software/workplace
149](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-161-2048.jpg)
: The final antipattern type
detected was the bloated session bean antipattern. This antipattern is similar to
the well known God class antipattern [36]. It describes a situation in EJB sys-
tems where a session bean has become too bulky. Such session beans generally
implement methods that operate against a large number of abstractions. For
example one method may check the balance of an account while another may
approve an order, and yet another may apply a payment to an invoice. Such
sessions can create memory and latency issues since their creation and manage-
ment can come with much overhead due to their bulky nature. The PAD tool
identified two potential bloated session beans in the IBM Workplace applica-
tion. The first potential instance of this antipattern was a session bean which
had 8 entity bean relationships, 6 session bean relationships and was invoked in
11 of the 25 user actions executed. It also contained a high number of business
methods (47). The second instance of this antipattern was a session bean which
had 7 entity relationships 1 session relationship and was invoked in 6 of the 25
user actions. This session had 14 business methods. In general the rule of thumb
is that there should be a 1 to 1 relationship between session and entity beans.
From the information (above), presented by the PAD tool upon identification of
these potential antipattern instances, it seemed that these session beans were in
fact bloated session beans. Unfortunately we were unable to contact the devel-
opers originally responsible for this code. However we did discover that this
code was removed from later releases of the application which indicated that it
was indeed problematic.
No false positives were identified when we applied the PAD tool to the IBM work-
place application and in fact all antipatterns identified were addressed in the sub-
sequent versions of the application which suggested they were indeed problematic
pieces of code. Unfortunately we could not assess whether false negatives were iden-
tified in the application as we did not have access to the complete system source code
or documentation.
150](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-162-2048.jpg)
![actions. Further research is required in this area to identify how FSM can be
applied to highly dynamic enterprise systems.
• PAD Tool Output: The PAD tool currently presents all antipatterns identified
to the tool user with corresponding contextual information. The information
presented (e.g. antipattern description, performance metrics) can be used by the
tool user to easily determine the impact of the antipatterns detected. However
it would be more desirable that the performance impact assessment would be
automated. This would allow for the PAD tool to rank antipatterns instances
of the same type according to their performance impact and developers could
concentrate their efforts on addressing the most significant issues first. This will
form part of our ideas for future work.
• PAD Tool: There are a number of steps in the PAD tool detection process that
have not been fully automated and which require human intervention. A fully
automated tool is currently being developed.
• Message Driven Beans: In this work we do not address issues related to message
driven beans. We consider only session and entity beans. A number of antipat-
terns however exist for message driven beans 1 [161] [61] and research in this
area could form part of future work.
8.3 Future Work
In this section we detail interesting areas for future work in this area:
• Monitoring: We plan to address the limitations detailed above in relation to our
monitoring module. In particular we plan to remove the need for system re-
deployment during the instrumentation process. This can be achieved through
dynamic bytecode instrumentation. Dynamic bytecode instrumentation can be
performed for newer versions of the JVM (1.5+) through the JVMTI. Work in
this area is already underway and forms part of the COMPAS BCI tool. Another
limitation of our monitoring approach is that it has not been applied to asyn-
chronous components (such as message driven beans). We plan to investigate
how the run-time path tracing approach in particular can be applied to message
driven beans. This could most likely be achieved by modifying our proxy com-
ponents such that they intercept calls to the onMessage method of the message
driven beans.
• Integration with Real Time Correlation Engine: A proof of concept, real-time
correlation engine has recently been implemented with the IBM Dublin Soft-
1Precise Java, http://www.precisejava.com/
156](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-168-2048.jpg)
![ware Lab [46]. The tool correlates server logs from the different servers on a dis-
tributed system such that a single log is available to system testers at run-time.
The solves the problem of the system testers having to correlate a range of logs
from different machines in order to understand the results of their tests. The tool
can allow for correlation of log files by various criteria, (e.g. time stamp, user id,
error type) and allows for easier and quicker problem determination. Currently
an industry research project is underway to integrating this tool with COMPAS
JEEM, such that system testers can analyse both system level information (e.g.
the server logs) and application level information (e.g. run-time paths). The aim
is to give the system tester a complete picture of the system under test, i.e. both
at the system level and application level.
• Analysis: Future work can be carried out to address the limitations of the FSM
CIR approach by fully automating the replacement approach. The results from
applying FSM to run-time paths show that this information is indeed useful once
it can be extracted. An interesting area for future work would be to investigate
sliding window approaches for finding episodes in sequences [112] to assess
whether they can be modified such that they can be applied to a transactional
database (e.g. a database of run-time paths). To date this approach has generally
been applied to event sequence data. The benefit of this approach over the FSM
approach using weighted support is that it does not suffer from combinatorial
explosion when transactions grow in length and can not be reduced in size using
preprocessing techniques.
• Antipattern documentation: Technology specific antipatterns have been well
documented for the Java technology covering pitfalls for both base java and the
java enterprise technology. Enterprise applications are often built using other
technologies/enterprise frameworks (e.g. .Net, CCM, Spring 2). However, we
are unaware of any literature that details instances of antipatterns in the other
common enterprise technologies. We firmly believe that similar problems con-
sistently occur in the other enterprise technologies. Documenting these prob-
lems in the form of antipatterns would bring understanding to developers build-
ing systems using these technologies.
• Antipattern detection for other technologies: The creation of a catalog of antipat-
terns for alternative enterprise technologies would allow for the automatic an-
tipattern detection using the approach we have outlined in this thesis. Research
would be required to develop or identify a monitoring framework that could
perform call path tracing. Much of the call path tracing logic and mechanisms
we present could be reused. The analysis and detection modules are portable
across different technologies.
2The Spring Framework, http://www.springframework.org/
157](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-169-2048.jpg)
![REFERENCES
[1] M. K. Agarwal, M. Gupta , G. Kar, A. Neogi and A. Sailer, Mining Activity
Data for Dynamic Dependency Discovery in e-Business Systems, IEEE eTransac-
tionson Network and Service Management Journal, Vol.1 No.2, September,
2004
[2] R. Agrawal, H. Mannila, R. Srikant, H. Toivonen and A.I. Verkamo. Fast
discovery of association rules. In Advances in Knowledge Discovery and Data
Mining, 1996.
[3] R. Agrawal and R. Srikant. Mining sequential patterns. In P. S. Yu and A. L.
P. Chen, editors, In Proceedings 11th International Confonference on Data
Engineering, 1995
[4] R. Agrawal, T. Imielinski, and A.N. Swami. Mining association rules between
sets of items in large databases. In P. Buneman and S. Jajodia, editors, In Pro-
ceedings of the 1993 ACM SIGMOD International Conference on Manage-
ment of Data, volume 22(2) of SIGMOD Record, 1993.
[5] H. Albin-Amiot, P. Cointe, Y. G. Gueheneuc, N. Jussien. Instantiating and
Detecting Design Patterns: Putting Bits and Pieces Together, 16th IEEE Interna-
tional Conference on Automated Software Engineering (ASE’01), 2001.
[6] C. Alexander. A Timless Way of Building. Oxford University Press, 1979.
[7] D. Alur, J. Crupi and D. Malks. Core J2EE Patterns: Best Practices and Design
Strategies. Prentice Hall, Sun Microsystems Press, 2001
[8] G. Ammons, T. Ball and J.R. Larus, Exploiting hardware performance counters
with flow and context sensitive profiling. ACM Conference on Programming
Language Design and Implementation. ACM Press, New York, 1997.
[9] G. Ammons, J.D. Choi, M. Gupta, and N. Swamy. Finding and Removing
Performance Bottlenecks in Large Systems., In Proceedings of ECOOP, 2004.
[10] J.M. Anderson, L.M. Berc, J. Dean, S. Ghemawat, M.R. Henzinger, S.A. Le-
ung, R.L. Sites, M.T. Vandervoorde, C.A. Waldspurger and W.E. Weihl. Con-
tinuous profiling: Where have all the cycles gone? Proceedings of the Sympo-
sium on Operating Systems Principles. ACM Press, New York, 1997.
[11] G. Antoniol, G. Casazza, M. Di Penta, and R. Fiutem. Object-oriented design
patterns recovery. Journal of Systems and Software, 2001.
159](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-171-2048.jpg)
![[12] M. Arnold and D. Grove. Collecting and Exploiting High-Accuracy Call Graph
Profiles in Virtual Machines. International Symposium on Code Generation
and Optimization, San Jose, California, March, 2005.
[13] Z. Balanyi, R. Ferenc. Mining Design Patterns from C++ Source Code. 19th IEEE
International Conference on Software Maintenance, 2003.
[14] T. Ball and S. K. Rajamani. The SLAM project: debugging system software via
static analysis, In Proceedings of the 29th POPL, January 2002.
[15] S. Balsamo, A. Di Marco, P. Inverardi, M. Simeoni. Model-Based Performance
Prediction in Software Development: A Survey. IEEE Transactions on Software
Engineering, , pp. 295-310, Vol. 30, No. 5, MAY 2004.
[16] J. Bansiya. Automating Design-Pattern Identification. Dr. Dobb’s Journal, vol.
23, no. 6, June, 1998.
[17] P. Barford and M. Crovella, Generating representative web workloads for network
and server performance evaluation. In ACM SIGMETRICS, 1998.
[18] P. Barham, A. Donnelly, R. Isaacs and R. Mortier. Using Magpie for request
extraction and workload modelling, Symposium on Operating Systems Design
and Implementation, pp 259–272, San Francisco, CA, USA, December 2004.
[19] P. Barham, R. Isaacs, R. Mortier and D. Narayanan. Magpie: online modelling
and performance-aware systems. In the 9th Workshop on Hot Topics in Oper-
ating Systems, May, 2003.
[20] V.R. Basili. The Experimental Paradigm in Software Engineering. Experimen-
tal Software Engineering Issues: Cirtical Assessment and Future Directives,
Springer-Verlang, Lecture Notes in Computer Science, 1993.
[21] L. Bass, P. Clements and R Kazman. Software Architecture in Practice (2nd
Edition). Addison-Wesley, 2003.
[22] I. D. Baxter, C. Pidgeon, M. Mehlich. DMS: Program transformations for prac-
tical scalable software evolution. Proceedings of the IEEE International Confer-
ence on Software Engineering. IEEE Computer Society Press: Los Alamitos,
CA, 2004.
[23] J. Beck, D. Eichmann, Program and Interface Slicing for Reverse Engineering. In
Proceedings of the International Conference on Software Engineering, 1993.
[24] K. Beck. Extreme Programming Explained: Embrace Change. Addison-Wesley,
1999.
[25] K. Beck. Test Driven-Development by Example, Addison-Wesley, Boston, MA,
2003.
[26] K. Beck and W. Cunningham. Using Pattern Languages for Object-Oriented
Programs, Technical Report No. CR-87-43 submitted to OOPSLA-87 work-
shop on the Specification and Design for Object Oriented Programming,
September, 1987.
160](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-172-2048.jpg)
![[27] F. Bergenti and A. Poggi. Improving UML Designs Using Automatic Design
Pattern Detection. Proceedings of the 12th International Conference on Soft-
ware Enginneering and Knowledge Engineering, July, 2000.
[28] P. Berkhin. Survey of clustering data mining techniques. Technical report, Ac-
crue Software, San Jose, CA, 2002.
[29] D. Beyer, A. Noack and C. Lewerentz. Efficient Relational Calculation for Soft-
ware Analysis. IEEE Transactions on Software Engineering, vol. 31, no. 2,
February, 2005.
[30] W. Binder, Portable and accurate sampling profiling for Java. Software Practice
and Experience, Vol. 36, Issue 6, John Wiley and Sons, Inc., New York, NY,
USA, February, 2006.
[31] W. Bischofberger, J. Kuhl and S. Loffler. Sotogrpah - a Pragmatic Approach to
Source Code Architecture Conformance Checking. Proceedings First European
Workshop on Software Architecture, Lecture Notes in Computer Science
3047, Springer, 2004.
[32] F. Bodon, A Trie-based APRIORI Implementation for Mining Frequent Item Se-
quences. ACM SIGKDD Workshop on Open Source Data Mining Workshop
(OSDM’05), pages 56 - 65, Chicago, IL, USA. 2005
[33] M. D. Bond and K. S. McKinley. Continuous Path and Edge Profiling. 38th In-
ternational Symposium on Microarchitecture, Barcelona, November, 2005.
[34] L.C. Briand, Y. Labiche, and J. Leduc, Towards the Reverse Engineering of UML
Sequence Diagrams for Distributed, Multithreaded Java Software. Technical Re-
port SCE-04-04, Carleton Univ., http://www.sce.carleton.ca/Squall, Sept.
2004.
[35] L.C. Briand, Y. Labiche and J. Leduc. Toward the Reverse Engineering of UML
Sequence Diagrams for Distributed Java Software. IEEE Transactions on Soft-
ware Engineering, vol. 32, no. 9, September, 2006.
[36] W. J. Brown, R. C. Malveau, and T. J. Mowbray. AntiPatterns: Refactoring
Software, Architectures, and Projects in Crisis. Wiley, 1998.
[37] E. Bruneton, T. Coupaye, M. Leclercq, V. Qu´ema, and J.-B. Stefani. The Frac-
tal Component Model and Its Support in Java. Software Practice and Experi-
ence, special issue on Experiences with Auto-adaptive and Reconfigurable
Systems. 2006.
[38] A. Buble, L. Bulej and P. Tuma. CORBA Benchmarking: A Course with Hidden
Obstacles. Proceedings of IPDPS Workshop on Performance Modeling, Eval-
uation and Optimization of Parallel and Distributed Systems, Nice, France,
2003.
[39] F. Buchli. Detecting Software Patterns using Formal Concept Analysis, Diploma
Thesis, University of Bern, 2003.
[40] M. Burch, S. Diehl and P. Weissgerber. Visual data mining in software archives.
Proceedings of the 2005 ACM symposium on Software visualization, St.
Louis, Missouri, 2005.
161](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-173-2048.jpg)
![[41] D. Burdick, M. Calimlim, J. Flannick, J. Gehrke and T. Yiu, MAFIA: A Max-
imal Frequent Itemset Algorithm. IEEE Transactions on Knowledge and Data
Engineering, vol. 17, no. 11, pp. 1490-1504, Nov., 2005.
[42] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad and M. Stal. Pattern-
Oriented Software Architecture, A System of Patterns, Volume 1. Wiley, 1996.
[43] E. Cecchet, J. Marguerite and W. Zwaenepoel. Performance and Scalability of
EJB Applications. Proc. 17th ACM SIGPLAN Conference on Object-Oriented
Programming, Systems, Languages, and Applications, pp 246-261, Seattle,
Washington, USA, 2002.
[44] M. Chen, E. Kiciman, A. Accardi, A. Fox and E. Brewer. Using runtime paths
for macro analysis. In Proceedings 9th Workshop on Hot Topics in Operating
Systems, Lihue, HI, USA, May 2003
[45] M. Chen, E. Kiciman, E. Fratkin, A. Fox and E. Brewer: Pinpoint: Problem De-
termination in Large, Dynamic, Internet Services. In Proceedings International
Conference on Dependable Systems and Networks (IPDS Track), Washing-
ton, D.C., June 2002
[46] L. Chen, P. O’Sullivan, L. P. Bergman, V. Castelli, E. Labadie, P. Sohn, T. Par-
sons. Problem Determination in Large Enterprise Systems. At the Software En-
gineering Tools For Tomorrow (SWEFT) 2006 conference, T.J. Watson, New
York, Oct 17 - 19, 2006. (Abstract only available)
[47] H. Chen and D. Wagner. MOPS: an infrastructure for examining security prop-
erties of software. In Proceedings of the Ninth ACM Conference on Computer
and Communications Security, Washington, DC, November 2002.
[48] E. J. Chikofsky and J. H. Cross II. Reverse Engineering and Design Recovery: A
Taxonomy. IEEE Software, 7(1):13– 17, January/February 1990.
[49] O. Ciupke. Automatic detection of design problems in object-oriented reengineer-
ing, In proceedings Technology of Object-Oriented Languages and Systems,
1999.
[50] P. Clements. A Survey of Architecture Description Languages. In Proceedings of
the 8th International Workshop on Software Specification and Design, 1996.
[51] P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, R. Nord
and J. Stafford. Documenting Software Architectures: Views and Beyond. Boston:
Addison-Wesley, 2003.
[52] P. Clemeents, R. Kazman and M. Klein. Evaluating Software Architectures,
Methods and Case Studies Addison-Wesley, 2001.
[53] J. Coplien. Advanced C++ Programming Styles and Idioms. Addison-Wesley,
1991.
[54] J. Coplien and N. Harrison. Organizational Patterns of Agile Software Develop-
ment. Prentice-Hall, 2004.
[55] J. Corbett, M. Dwyer, J. Hatcliff, C. Pasareanu, Robby, S. Laubach, H. Zheng.
Bandera: Extracting Finite-state Models from Java Source Code, In Proc. of ICSE,
June 2000.
162](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-174-2048.jpg)
![[56] G. Costagliola, A. De Lucia, V. Deufemia, C. Gravino, and M. Risi. Design
Pattern Recovery by Visual Language Parsing, Proceedings of the Ninth Eu-
ropean Conference on Software Maintainance and Reengineering, March,
2005.
[57] M. Dahm. Byte code engineering with the BCEL API. Technical Report B-
17-98, Freie Universit¨at Berlin, 2001.
[58] G. Denaro, A. Polini, and W. Emmerich, 2004. Early performance testing of
distributed software applications. In Proceedings of the 4th international Work-
shop on Software and Performance, Redwood Shores, California, 2004.
[59] M. Dmitriev, Profiling Java applications using code hotswapping and dynamic
call graph revelation. In Proceedings of the 4th international workshop on
Software and performance, Redwood Shores, California, 2004.
[60] M. Dodani: Patterns of Anti-Patterns, in Journal of Object Technology, vol. 5,
no. 6, pp. 29-33, July - August 2006.
[61] B. Dudney, S. Asbury, J. K. Krozak and K. Wittkopf. J2EE Antipatterns, Wiley,
2003.
[62] A. H. Eden and R. Kazman. Architecture, Design, Implementation. In pro-
ceesings of the International Conference on Software Engineering, Portland,
USA, May, 2003.
[63] M. El-Attar and J. Miller. Matching Antipatterns to Improve the Quality of Use
Case Models. 14th IEEE International Requirements Engineering Conference,
Minnesota, USA, 2006.
[64] S. Elbaum and M. Diep,Profiling Deployed Software: Assessing Strategies and
Testing Opportunities. IEEE Transactions on Software Engineering, Vol. 31,
No. 4, April 2005.
[65] G. Florijn, M. Meijers, and P. van Winsen. Tool support for object-oriented pat-
terns. In Ecoop 1997: Object-Oriented Programming, volume 1241 of Lecture
Notes in Computer Science, 1997.
[66] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. The KDD process for extract-
ing useful knowledge from volumes of data. Communications of the ACM, vol-
ume 39, issue 11, pp27-34, November 1996
[67] M. Fowler. Patterns of Enterprise Application Architecture. Addison-Wesley
Professional, 2002
[68] M. Fowler. Refactoring: improving the design of existing code. Addison-Wesley,
1999.
[69] E. Friedman-Hill. Jess in Action. Manning Publications, July, 2003.
[70] L. Fulop, T. Gyovai, and R. Ferenc. Evaluating C++ Design Pattern Miner
Tools. In Sixth IEEE International Workshop on Source Code Analysis and
Manipulation, Sep. 2006.
163](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-175-2048.jpg)
![[71] E. Gamma , R. Helm, R. Johnson and J. Vlissides. Design Patterns: Abstrac-
tion and Reuse of Object-Oriented Design. European Conference on Object Ori-
ented Programming, 1993
[72] E. Gamma , R. Helm, R. Johnson and J. Vlissides: Design Patterns: Elements
of Reusable Object-Oriented Software, Addison-Wesley, 1995
[73] D. Garlan and M. Shaw. An introduction to software architecture. In Advances in
Software Engineering and Knowledge Engineering. Volume 1. World Scientific
Publishing Co., 1993.
[74] R. Glass. The Software Research Crisis. IEEE Software, November, 1994.
[75] B. Goethals, and M. J. Zaki , ”Advances in Frequent Itemset Mining Implemen-
tations: Report of FIMI 2003”, ACM SIGKDD Explorations; Vol. 6, nr. 1, 2003.
[76] I. Gorton and A. Liu, Software Component Quality Assessment in Practice: Suc-
cesses and Practical Impediments, the Proceeding of the International Confer-
ence on Software Engineering, U.S., May 2002
[77] I. Gorton and L. Zhu. Tool Support for Just-In-Time Architecture Reconstruction
and Evaluation: An Experience Report, Proc. 27th International Conference on
Software Engineering, St. Louis, Mousiri, USA, 2005
[78] K. Govindraj, S. Narayanan, B. Thomas, P Nair, and S. Peeru. On Using AOP
for Application Performance Management. In proceedings Fifth International
Conference on Aspect-Oriented Software Development (Industry Track),
Bonn, Germany, March 2006.
[79] S. Graham and M. McKusick, Gprof: A Call Graph Execution Profiler. ACM
SIGPLAN, vol. 17, no. 6, pp. 120-126, June 1982.
[80] M. Grand. Patterns in Java. vol. 2. New York, New York: Wiley: 1999.
[81] J. Grass. Object-oriented design archeology with CIA. Computing Systems,
1992.
[82] T. Gschwind, K. Eshghi, P. K. Garg and K. Wurster. WebMon: A Perfor-
mance Profiler for Web Transactions, Proc. Fourth IEEE International Work-
shop on Advanced Issues of E-Commerce and Web-Based Information Sys-
tems, Newport Beach, California, USA, June, 2002
[83] Y.G. Gueheneuc, H. Albin-Amiot. Using Design Patterns and Constraints to
Automate the Detection and Correction of Inter-Class Design Defects. 39th Inter-
national Conference and Exhibition on Technology of Object-Oriented Lan-
guages and Systems (TOOLS39), 2001.
[84] R.J. Hall. Call Path Refinement Profiles. IEEE Transactions on Software Engi-
neering, vol. 21, no. 6, June 1995.
[85] R.J. Hall. CPPROFJ: Aspect-capable Call Path Profiling of Multi-threaded Java
Applications. Proceedings of the 17 th IEEE International Conference on Au-
tomated Software Engineering, 2002.
164](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-176-2048.jpg)
![[86] H. H. Hallal, E. Alikacem, W. P. Tunney, S. Boroday, and A. Petrenko.
Antipattern-Based Detection of Deficiencies in Java Multithreaded Software, Pro-
ceedings of the Quality Software, Fourth International Conference on
(QSIC’04), IEEE Computer Society, Washington, DC, USA, 2004.
[87] D. Hand, H. Mannila, P. Smyth. Principles of Data Mining, MIT Press, 2001.
[88] M. Hauswirth, P.F. Sweeney, A. Diwan and M. Hind. Vertical profiling: Un-
derstanding the behavior of object-oriented applications. Proceedings of the 19th
Annual ACM SIGPLAN Conference on Object-oriented Programming, Sys-
tems, Languages, and Applications, 2004.
[89] D. Heuzeroth, T. Holl, G. Hogstrom, and W. Lowe. Automatic design pat-
tern detection. In 11th International Workshop on Program Comprehension
(IWPC), USA, May 2003. IEEE.
[90] D. Heuzeroth, T. Holl, and W. Lowe. Combining static and dynamic analyses
to detect interaction patterns. In Sixth International Conference on Integrated
Design and Process Technology (IDPT), Jun 2002.
[91] D. Heuzeroth, S. Mandel, and W. Lowe. Generating design pattern detectors
from pattern specifications. In 18th International Conference on Automated
Software Engineering (ASE). IEEE, Oct. 2003.
[92] C.W. Ho, M.J. Johnson, L. Williams and E.M. Maximilien. On Agile Perfor-
mance Requirements Specification and Testing. Proceedings of AGILE 2006.
[93] G. Hohpe, and B. Woolf: Enterprise Integration Patterns. Addison-Wesley
Professional, 2003
[94] D.F. Jerding, J.T. Stasko and T. Ball. Visualizing Interactions in Program Execu-
tions. In the proceedings of the International Conference on Software Engi-
neering, 1997.
[95] S.C. Johnson. Lint, a C program checker. Unix Programmer’s Manual, 4.2
Berkeley Software Distribution Supplementary Docs; U.C. Berkeley, 1984.
[96] M.J. Johnson, E.M. Maximilien, C.W. Ho and L. Williams. Incorporating Per-
formance Testing in Test-Driven Development. IEEE Software, May/June, 2007.
[97] S. Kamin and D. Hyatt. A Special Purpose Language for Picture Drawing, In
Proceedings of the Conference of Domain Specific Languages, Santa Bar-
bara, CA, USA, 1997
[98] R.K. Keller, R. Schauer, S. Robitaille, and P. Pag´e. Pattern-based reverse engi-
neering of design components. In Software Engineering, 1999. Proceedings of
the 1999 International Conference on, 1999.
[99] M. Kis. Information Security Antipatterns in Software Requirements Engineering.
In proceedings 9th Conference on Pattern language of Programs, 2002.
[100] B. Kitchenham, L. Pickard and S.L. Pfleeger. Case Studies for Method and Tool
Evaluation. IEEE Software, July, 1995.
[101] A. Koenig. Patterns and Antipatterns, Journal of Object Oriented Program-
ming, volume 8, number 1, 1995.
165](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-177-2048.jpg)
![[102] P. Kovari, D. Cerecedo Diaz ,F. C. H. Fernandes ,D. Hassan , K. Kawamura
, D. Leigh, N. Lin, D. Masicand, G. Wadley and P. Xu. WebSphere Application
Server Enterprise V5 and Programming Model Extensions WebSphere Handbook
Series, September, August, 2003
[103] C. Kramer and L. Prechelt. Design recovery by automated search for structural
design patterns in object-oriented software. In Reverse Engineering, 1996, Pro-
ceedings of the Third Working Conference on, 1996.
[104] P. Kruchten. The 4+1 View Model of Architecture. IEEE Software, volume 12,
issue 6, November, 1995.
[105] M. Kunnumpurath. JBoss Administration and Development Third Edition (3.2.x
Series, Apress, October 2003
[106] J. R. Larus and T. Ball. Rewriting executable files to measure program behavior.
Technical Report Computer Sciences Technical Report 1083, University of
Wisconsin-Madison, 1992.
[107] C.H. Lee, J.C. Ou and M.S. Chen. Progressive Weighted Miner: An Efficient
Method for Time-Constraint Mining. PAKDD, 2003.
[108] M. Leleu, C. Rigotti , J.F. Boulicaut and G. Euvrard. GO-SPADE: Mining Se-
quential Patterns over Datasets with Consecutive Repetitions. Proc. 3rd Int. Conf.
Machine Learning and Data Mining in Pattern Recognition, Leipzig, Ger-
many, 2003.
[109] S. Lo. Binary Prediction Based on Weighted Sequential Mining Method. Proc. Int.
Conf. on Web Intelligence, 2005.
[110] M. Mahemoff. Ajax Design Patterns. Sebastopol, California: O’Reilly, 2006.
[111] H. Mannila, H. Toivonen and A.I. Verkamo. Discovering Frequent Episodes in
Sequences, Proc. 1st Int. Conf. on Knowledge Discovery and Data Mining,
Montreal, Canada, 1995
[112] H. Mannila, H. Toivonen and A.I. Verkamo. ”Discovery of frequent episodes
in event sequences”, Data Mining and Knowledge Discovery 1(3): 259 - 289,
November 1997.
[113] F. Marinescu. EJB Design Patterns. Wiley, 2002.
[114] R. Marinescu. Measurement and Quality in Object-Oriented Design. PhD thesis,
Politehnica University of Timisoara, 2002.
[115] N. Medvidovic and R. N. Taylor. A Classification and Comparison Framework
for Software Architecture Description Languages. IEEE Transactions on Soft-
ware Engineering, vol. 26, no. 1, January, 2000.
[116] N. Moha and Y.G. Gu´eh´eneuc. On the Automatic Detection and Correction of
Design Defects. In Serge Demeyer, Kim Mens, Roel Wuyts, and St´ephane
Ducasse, editors, Proceedings of the 6th ECOOP workshop on Object-
Oriented Reengineering, July 2005.
166](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-178-2048.jpg)
![[117] N. Moha, J. Rezgui, Y.G. Gueheneuc, P. Valtchev and G. El Boussaidi. Using
FCA to Suggest Refactorings to Correct Design Defects. Proceedings of the 4th
International Conference On Concept Lattices and Their Applications (CLA
2006),In S. Ben Yahia and E. Mephu Nguifo Eds, Hammamet, Tunisia, 2006.
[118] A. Mos. A Framework for Adaptive Monitoring and Performance Management of
Component-Based Enterprise Applications. PhD thesis, Dublin City University,
Ireland, 2004.
[119] A. Mos and J. Murphy. COMPAS: Adaptive Performance Monitoring of
Component-Based Systems. Proc. Workshop on Remote Analysis and Mea-
surement of Software Systems at 26th International Conference on Software
Engineering, Edinburgh, Scotland, May 2004.
[120] A. Mos and J. Murphy. Performance Management in Component-Oriented Sys-
tems using a Model Driven Architecture Approach, Proc. 6th IEEE International
Enterprise Distributed Object Computing Conference, Lausanne, Switzer-
land, September 2002.
[121] M.J. Munro. Product metrics for automatic identifcation of bad smell design prob-
lems in java source-code. In Lanubile, F., Seaman, C., eds.: proceedings of the
11th International Software Metrics Symposium, IEEE Computer Society
Press, 2005.
[122] G.C. Murphy, D. Notkin, and K. Sullivan. Software Reflexion Models: Bridging
the Gap between Source and High-Level Models. Proc. SIGSOFT Symp. Founda-
tions of Software Eng., ACM Press, New York, 1995.
[123] J. Niere, J. P. Wedsack, and L. Wendehals. Handling Large Search Space in
Pattern-based Reverse Engineering. In 11th IEEE International Workshop on
Program Comprehension, May 2003.
[124] G. Papadimitriou, A. Vakali, G. Pallis, S. Petridou and A. Pomportsis, Sim-
ulation in Web data management, Applied system simulation: methodologies
and applications, Kluwer Academic Press, Norwell, MA, USA, 2003.
[125] T. Parsons and J. Murphy. Detecting Performance Antipatterns in Component
Based Enterprise Systems. Accepted for Publication in the Journal of Object
Technology.
[126] T. Parsons, J. Murphy, P. O’Sullivan, Applying Frequent Sequence Mining to
Identify Design Flaws in Enterprise Software Systems. In Proceedings 5th Intl.
Conference on Machine Learning and Data Mining, Leipzig, Germany, July
18-20, 2007.
[127] T. Parsons, J. Murphy, S. Pizzoli, P. O’Sullivan, A. Mos, Reverse Engineering
Distributed Enterprise Applications to Identify Common Design Flaws. At the
Software Engineering Tools For Tomorrow (SWEFT) 2006 Conference, T.J.
Watson, New York, Oct 17 - 19, 2006.
[128] T. Parsons, A. Mos and J. Murphy. Non-Intrusive End to End Run-time Path
Tracing for J2EE Systems. IEE Proc. Software, August, 2006.
167](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-179-2048.jpg)
![[129] T. Parsons and J. Murphy.. The 2nd International Middleware Doctoral Sympo-
sium: Detecting Performance Antipatterns in Component-Based Enterprise Sys-
tems. IEEE Distributed Systems Online, vol. 7, no. 3, March, 2006.
[130] T. Parsons. A Framework for Detecting Performance Design and Deployment An-
tipatterns in Component Based Enterprise Systems. Proc. 2nd Int’l Middleware
Doctoral Symp., ACM Press, art. no. 7, Grenoble, France, 2005.
[131] T. Parsons. A Framework for Detecting, Assessing and Visualizing Performance
Antipatterns in Component Based Systems. First Place at ACM SIGPLAN Stu-
dent Research Competition Graduate Division, In OOPSLA’04: Companion
to the 19th annual ACM SIGPLAN conference on Object-oriented program-
ming systems, languages, and applications, Vancouver, BC, Canada, 2004.
[132] T. Parsons and J. Murphy.. A Framework for Automatically Detecting and As-
sessing Performance Antipatterns in Component Based Systems using Run-Time
Analysis. The 9th International Workshop on Component Oriented Pro-
gramming, part of the 18th European Conference on Object Oriented Pro-
gramming. Oslo, Norway, June, 2004.
[133] T. Parsons and J. Murphy. Data Mining for Performance Antipatterns in Compo-
nent Based Systems Using Run-Time and Static Analysis. Transactions on Au-
tomatic Control and Control Science, Vol. 49 (63), No. 3, May 2004.
[134] D.J. Pearce, P. H. J. Kelly, T. Field, U. Harder. Gilk: A dynamic instrumentation
tool for the Linux kernel. Proceedings of the International TOOLS Conference.
Springer: Berlin, 2002.
[135] D. J. Pearce, M. Webster, R. Berry and Paul H. J. Kelly, Profiling with AspectJ.
Software Practice and Experience Vol. 37, Issue 7, John Wiley and Sons, Inc.
New York, NY, USA, June 2007.
[136] R. Pena-Ortiz, J. Sahuquillo, A. Pont and Jose A. Gil, Modeling Continuous
Changes of the User’s Dynamic Behavior in the WWW, Proceedings of the 5th
international workshop on Software and performance, Palma, Illes Balears,
Spain, 2005.
[137] N. Pettersson. Measuring precision for static and dynamic design pattern recog-
nition as a function of coverage. In ICSE Workshop on Dynamic Analysis, St.
Louis, USA, May 2005.
[138] N. Pettersson and W. L¨owe. Efficient and Accurate Software Pattern Detec-
tion. 13th Asia Pacific Software Engineering Conference (APSEC), Decem-
ber 2006.
[139] N. Pettersson, W. L¨owe and J. Nivre. On Evaluation of Accuracy in Pattern
Detection. First International Workshop on Design Pattern Detection for Re-
verse Engineering, October 2006.
[140] F. Pl´asil, D. B´alek and R. Janecek. SOFA/DCUP: Architecture for Compo-
nent Trading and Dynamic Updating. In proceedings of ICCDS.98, Annapolis,
Maryland, USA, IEEE CS Press, May 1998
[141] C. Ponder and R.J. Fateman. Inaccuracies in Program Profilers. Software Prac-
tice and Experience, May, 1998.
168](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-180-2048.jpg)
![[142] M. Pont. Patterns for Time-Triggered Embedded Systems. Harlow, England:
Addison-Wesley: 2001.
[143] A. Postma: A method for module architecture verification and its applica-
tion on a large component based system, Information and Software Tech-
nology 45(4), 2003, 171-194
[144] C. Potts. Software Engineering Research Revisited. IEEE Software, September,
1993.
[145] D. Reimer et al. SABER: Smart Analysis Based Error Reduction, Proceedings of
the ACM SIGSOFT international symposium on Software testing and anal-
ysis, 2004.
[146] T. Richner and S. Ducasse. Recovering High-Level Views of Object-Oriented
Applications from Static and Dynamic Information. Proceedings International
Conference on Software Maintenance, 1999.
[147] H. Ritch and H. M. Sneed. Reverse engineering programs via dynamic analysis.
In Proceedings of WCRE 1993. IEEE, May 1993.
[148] E. Roman, Scott W. Ambler and Tyler Jewell, Mastering Enterprise JavaBeans,
second edition, J.Wiley and Sons, USA and Canada, 2002.
[149] B. Schmerl, J. Aldrich, D. Garlan, R. Kazman, and H. Yan. Discovering Archi-
tectures from Running Systems. IEEE Transactions on Software Engineering,
July, 2006.
[150] M. Schumacher, E. Fernandez-Buglioni, D. Hypertson, F. Buschmann, and
P. Sommerlad, Security Patterns. Chichester, England: Wiley, 2006.
[151] J. Seemann and J. W. von Gudenberg. Pattern-based design recovery of Java
software. SIGSOFT 1998/FSE-6: Proceedings of the 6th ACM SIGSOFT in-
ternational symposium on Foundations of software engineering, 1998
[152] M. Shaw and D. Garlan. Software Architecture, Perspectives on an Emerging
Discipline. Prentice Hall, 1996.
[153] N. Shi and R. A. Olsson. Reverse Engineering of Design Patterns from Java
Source Code. ASE ’06: Proceedings of the 21st IEEE International Conference
on Automated Software Engineering, 2006.
[154] C. U. Smith, and L. Williams. Performance Solutions. Addison Wesley, 2002.
[155] A. Srivastava and A. Eustace. ATOM: A system for building customized pro-
gram analysis tools. Proceedings of the ACM Conference on Programming
Language Design and Implementation. ACM Press: New York, 1994.
[156] P.F. Sweeney, M. Hauswirth, B. Cahoon, P. Cheng, A. Diwan, D. Grove
and M. Hind. Using hardware performance monitors to understand the behavior
of Java applications. Virtual Machine Research and Technology Symposium.
USENIX Association: Berkeley, CA, USA, 2004.
[157] C. Szyperski. Component Software - Beyond Object-Oriented Programming,
Addison-Wesley, 1999
169](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-181-2048.jpg)
![[158] TakeFive Software GmbH. SNiFF+, 1996.
[159] F. Tao, F. Murtagh and M. Farid. Weighted Association Rule Mining using
weighted support and significance framework. KDD, Washington, D.C., 2003.
[160] B. Tate. Bitter Java. Manning Publications Co., Greenwich, CT, USA, 2002.
[161] B. Tate, M. Clarke, B. Lee and P. Linskey. Bitter EJB. Manning, 2003
[162] W.F. Tichy. Should Computer Scientists Experiment More?. IEEE Computer,
1998.
[163] P. Tonella, M. Torchiano, B. Du Bois and T Systa. Empirical studies in reverse
engineering: state of the art and future trends. Journal on Empirical Software
Engineering, 2007.
[164] P. Tonella and G. Antoniol. Object Oriented Design Pattern Inference. Proceed-
ings of IEEE Conference on Software Maintenance, 1999.
[165] Call Graph-Directed Boundary Condition Analysis in Contextual Composi-
tion Frameworks. PhD Thesis, University College Dublin, April, 2007.
[166] M. Trofin and J. Murphy. Removing Redundant Boundary Checks in Contextual
Composition Frameworks. Journal of Object Technology, vol. 5, no. 6, July-
August, 2006.
[167] N. Tsantalis, A. Chatzigeorgiou, G. Stephanides, and S. T. Halkidis. Design
Pattern Detection Using Similarity Scoring. IEEE Transactions on Software En-
gineering, November 2006.
[168] A. Ufimtsev and L. Murphy. Performance modeling of a JavaEE component
application using layered queuing networks: revised approach and a case study.
In Proceedings of the 2006 Conference on Specification and Verification of
Component-Based Systems, Portland, Oregon, 2006.
[169] D. Viswanathan and S. Liang, Java Virtual Machine Profiler Interface. IBM Sys-
tems Journal, Java Performance ,vol. 39, no. 1, 2000.
[170] E. Visser. Stratego: A language for program transformation based on rewriting
strategies. Proceedings of the International Conference on Rewriting Tech-
niques and Applications. Springer: Berlin, 2001.
[171] J. M. Vlissides, J. O. Coplien and N. L. Kerth. Pattern Languages of Program
Design 2 (Software Patterns Series). Addison-Wesley Professional, 1996.
[172] M. Vok´ac: An efficient tool for recovering Design Patterns from C++ Code. Jour-
nal of Object Technology, vol. 5, no. 1, 2006.
[173] J. Wang and J. Han, BIDE: efficient mining of frequent closed sequences. Proc.
20th Int. Conf. on Data Engineering, 2004
[174] L. Wendehals. Improving design pattern instance recognition by dynamic analy-
sis. In ICSE 2003 Workshop on Dynamic Analysis (WODA), Portland, USA,
May 2003.
[175] L.Wendehals and A. Orso. Recognizing Behavioral Patterns at Runtime using
Finite Automata. In Workshop on Dynamic Analysis (WODA), May 2006.
170](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-182-2048.jpg)
![[176] E. Weyuker and F. Vokolos. Experience with performance testring of software
systems: issues, an approach, and case study. IEEE Transactions on Software
Engineering, 2000.
[177] N. Wilde and M. C. Scully. Software reconnaisance: Mapping program features
to code. Software Maintenance: Research and Practice, 1992.
[178] C. Wohlin, P. Runeson, M. Host, M. C. Ohlsson, B. Regnell and A. Wesslen.
Experimentation in Software Engineering, An Introduction. Kluwer Academic
Publishers, 2000.
[179] E. Wohlstadter, S. Jackson and P. Devanbu, DADO: Enhancing Middleware to
Support Crosscutting Features in Distributed, Heterogeneous Systems, Proceed-
ings of the 25th International Conference on Software Engineering, May,
2003
[180] X. Yan, J. Han, and R. Afshar, CloSpan: Mining Closed Sequential Patterns in
Large Databases. In SIAM Int. Conf. on Data Mining , San Francisco, CA, May
2003.
[181] M.J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed associa-
tion rule mining. Technical Report 99-10, Department of Computer Science,
Rensselaer Polytechnic Institute, Troy, NY, USA, 1999.
[182] M. J. Zaki and C. Hsiao. Charm: An efficient algorithm for closed itemset mining.
In 2nd SIAM Int’l Conf. on Data Mining, 2002.
171](https://image.slidesharecdn.com/thesis-trevor-140604105744-phpapp01/75/Automatic-Detection-of-Performance-Design-and-Deployment-Antipatterns-in-Component-Based-Enterprise-Systems-183-2048.jpg)
Uploaded byTrevor Parsons
Automatic Detection of Performance Design and Deployment Antipatterns in Component Based Enterprise Systems
The document presents Trevor Parsons' PhD thesis on the automatic detection of performance design and deployment antipatterns in component-based enterprise systems. It explores methodologies for detecting these antipatterns, discusses the necessary background in software systems and their performance, and describes the implementation and evaluation of the proposed detection tools. Key contributions include the development of a rule engine approach and a comprehensive antipattern library.
More Related Content
Similar to Automatic Detection of Performance Design and Deployment Antipatterns in Component Based Enterprise Systems
Automatic Detection of Performance Design and Deployment Antipatterns in Component Based Enterprise Systems
- 1. Automatic Detection of PerformanceDesign and Deployment Antipatterns in Component Based Enterprise Systems by Trevor Parsons The thesis is submitted to University College Dublin for the degree of PhD in the College of Engineering, Mathematical and Physical Sciences. November 2007 School of Computer Science and Informatics Dr. J. Carthy. (Head of Department) Under the supervision of Dr. J. Murphy
- 2. CONTENTS Abstract vii Acknowledgements ix Listof Publications xi 1 Introduction 1 1.1 Background and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2 Thesis Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Thesis Contributions and Statement . . . . . . . . . . . . . . . . . . . . . 5 1.4 Key Assumptions and Scope . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Background 8 2.1 Performance of Software Systems . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Component Based Software . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2.1 Software Components . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2.2 Component Frameworks . . . . . . . . . . . . . . . . . . . . . . . 11 2.3 The Java Enterprise Edition . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.1 Web Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.3.2 Business Tier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.3 Enterprise Information System Tier . . . . . . . . . . . . . . . . . 14 2.4 The Enterprise JavaBean Technology . . . . . . . . . . . . . . . . . . . . 15 2.4.1 The EJB Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4.2 The EJB Component Model . . . . . . . . . . . . . . . . . . . . . 16 2.4.3 EJB Runtime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.4.4 Deployment Settings . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.5 Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.6 Software Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.7 Software Antipatterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.8 Performance Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.8.1 Workload Generation . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.8.2 Profiling Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.9 Reverse Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 2.10 Design Pattern and Antipattern Detection . . . . . . . . . . . . . . . . . 36 2.11 Knowledge Discovery in Databases and Data Mining . . . . . . . . . . . 39 2.11.1 Frequent Pattern Mining and Clustering . . . . . . . . . . . . . . 40 i
- 3. 3 Overview ofApproach 42 3.1 Approach for the Automatic Detection of Performance Antipatterns . . 44 3.1.1 Research Methodology and Validation Criteria . . . . . . . . . . 45 3.1.2 Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.1.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 3.1.4 Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2 Antipatterns Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2.1 Antipattern Hierarchy . . . . . . . . . . . . . . . . . . . . . . . . 51 4 Monitoring Required for Antipattern Detection 54 4.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.2 Run-Time Path Tracing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.2.1 Run-Time Paths Overview . . . . . . . . . . . . . . . . . . . . . . 57 4.2.2 Run-Time Path Tracing Motivation . . . . . . . . . . . . . . . . . 60 4.2.3 Run-Time Path Tracing Considerations . . . . . . . . . . . . . . 61 4.2.4 COMPAS Monitoring Framework . . . . . . . . . . . . . . . . . 62 4.2.5 COMPAS Extensions . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.3 Monitoring Server Resource Usage and Extracting Component Meta- Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.3.1 Using Java Management Extensions to Monitoring Server Re- source Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.3.2 Automatically Extracting Component Meta-Data . . . . . . . . . 80 4.4 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.4.1 Applications of Run-Time Paths . . . . . . . . . . . . . . . . . . . 82 4.4.2 Alternative Representations for Component Interactions . . . . 83 4.4.3 Run-Time Interaction Tracing Approaches . . . . . . . . . . . . . 84 5 Reconstructing the Systems Design for Antipattern Detection 88 5.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.2 Automatically Extracting Component Relationships and Object Usage Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.3 Reconstructing Run-time Container Services . . . . . . . . . . . . . . . . 96 5.4 Identifying Component Communication Patterns in Run-Time Paths using Frequent Sequence Mining . . . . . . . . . . . . . . . . . . . . . . . 97 5.4.1 Frequent Itemset Mining and Frequent Sequence Mining . . . . 97 5.4.2 Support Counting for Run-Time Paths . . . . . . . . . . . . . . . 99 5.4.3 Further Criteria for Interestingness . . . . . . . . . . . . . . . . . 102 5.4.4 Preprocessing for FSM Performance Improvement . . . . . . . . 102 5.4.5 Closed Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.4.6 PostProcessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.4.7 Component Communication Information for the Extracted De- sign Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.5 Data Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.5.1 Clustering Run-time Paths . . . . . . . . . . . . . . . . . . . . . . 106 5.5.2 Statistical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.6 The Reconstructed Design Model . . . . . . . . . . . . . . . . . . . . . . 107 5.7 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.7.1 Reverse Engineering . . . . . . . . . . . . . . . . . . . . . . . . . 108 5.7.2 Data Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6 Detecting Performance Design and Deployment Antipatterns 112 ii
- 4. 6.1 Antipatterns Categories. . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.2 A Rule Engine Approach for Antipattern Detection . . . . . . . . . . . . 114 6.3 Example Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.3.1 Antipattern Library . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.3.2 Filtering Using Threshold Values . . . . . . . . . . . . . . . . . . 118 6.4 PAD Tool User Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.5.1 Antipattern Categorisation . . . . . . . . . . . . . . . . . . . . . 119 6.5.2 Performance Testing . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.5.3 Detection Techniques . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.5.4 Antipattern Detection . . . . . . . . . . . . . . . . . . . . . . . . 121 7 Results and Evaluation 124 7.1 Chapter Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 7.2 COMPAS JEEM Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 7.2.1 Deducing System structure . . . . . . . . . . . . . . . . . . . . . 127 7.2.2 Portability Assessment . . . . . . . . . . . . . . . . . . . . . . . . 132 7.2.3 Performance Overhead . . . . . . . . . . . . . . . . . . . . . . . . 133 7.3 Analysis Module Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 7.3.1 FSM Performance Tests . . . . . . . . . . . . . . . . . . . . . . . . 135 7.3.2 Applying FSM to Identify Design Flaws . . . . . . . . . . . . . . 140 7.3.3 Data Reduction Results . . . . . . . . . . . . . . . . . . . . . . . . 142 7.4 PAD Tool Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 7.4.1 Antipatterns Detected in the Duke’s Bank Application . . . . . 144 7.4.2 Antipatterns Detected in the IBM Workplace Application - Beta Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 7.5 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 7.5.1 Overview of Contributions and Evaluation Criteria . . . . . . . 151 7.5.2 Validation of Contributions . . . . . . . . . . . . . . . . . . . . . 152 8 Conclusions 153 8.1 Thesis Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 8.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 8.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 References 171 A Antipattern Rule Library 172 A.1 Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 A.1.1 Category1: Antipatterns Across or Within Run-Time Paths . . . 172 A.1.2 Category2: Inter-Component Relationship Antipatterns . . . . . 174 A.1.3 Category3: Antipatterns Related to Component Communica- tion Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 A.1.4 Category4: Data Tracking Antipatterns . . . . . . . . . . . . . . 177 A.1.5 Category5: Pooling Antipatterns . . . . . . . . . . . . . . . . . . 177 A.1.6 Category6: Intra-Component Antipatterns . . . . . . . . . . . . 178 A.1.7 Adding Rules to The Rule Library . . . . . . . . . . . . . . . . . 178 A.2 Jess User Defined Functions provided by the PAD Tool . . . . . . . . . . 178 A.3 Configuration Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 B JEEM and FSM Implementation Source Code 180 iii
- 5. LIST OF FIGURES 1.1Typical Enterprise Architecture . . . . . . . . . . . . . . . . . . . . . . 3 2.1 Typical JEE Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.2 Client Invoking an EJB . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.3 Example EJB Deployment Descriptor . . . . . . . . . . . . . . . . . . . 20 2.4 Stateless Session Bean Lifecycle . . . . . . . . . . . . . . . . . . . . . . 21 2.5 Patterns, Antipatterns and their Relationship . . . . . . . . . . . . . . 27 2.6 JVMPI Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.7 The KDD Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.1 PAD Tool Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2 Run-time Design Meta Model . . . . . . . . . . . . . . . . . . . . . . . 46 3.3 Hierarchy of Antipatterns . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.1 Dynamic Call Trace (a) and Corresponding Dynamic Call Tree (b) . . 58 4.2 Example Run-Time Path . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.3 Run-Time Path Data Structure . . . . . . . . . . . . . . . . . . . . . . . 60 4.4 A Run-Time Path’s PathNode Data Structure . . . . . . . . . . . . . . 60 4.5 COMPAS Probe Insertion Process . . . . . . . . . . . . . . . . . . . . . 65 4.6 COMPAS JEEM Architecture . . . . . . . . . . . . . . . . . . . . . . . . 66 4.7 Intercepting Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 4.8 Remote Method Invocation . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.9 The Sample Bean’s Home Interface . . . . . . . . . . . . . . . . . . . . 75 4.10 A Wrapper for the Sample Bean’s Home Interface . . . . . . . . . . . . 75 4.11 A Sample Bean Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 4.12 Run-Time Path with Tracked Object, as a Sequence Diagram . . . . . . 78 4.13 JEEManagedObject . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 4.14 JDBCStats, JDBCConnectionStats, JDBCConnectionPoolStats . . . . . 81 5.1 Run-time Design Meta Model from Chapter 3 . . . . . . . . . . . . . . 91 5.2 Example Run-Time Path (a), Example Deployment Descriptors (b), Extract of Component Data Structure (c) and Data Extracted to Pop- ulate Data Structure (d) . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.3 Example Run-Time Path (a), Extract of the TrackedObject Data Struc- ture (b) and Information Extracted to Populate the TrackedObject Data Structure (c) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 iv
- 6. 5.4 Example Run-TimePath (a), Extract of the RunTimeContainerSer- vice Data Structure (b), and Information Extracted to Populate the RunTimeContainerService Data Structure (c) . . . . . . . . . . . . . . . 95 5.5 Class Diagram Showing Components Relationships . . . . . . . . . . 98 5.6 Example Transaction with Different Support Counting Approaches . 100 5.7 Hidden Elements in Transaction and Corresponding Support Counts 101 6.1 Rule to Detect Simultaneous Interfaces Antipattern . . . . . . . . . . . 115 6.2 Rule to Detect Needless Session Antipattern . . . . . . . . . . . . . . . 116 6.3 Rule to Detect Bulky or Unusual Levels of Database Communication 117 7.1 AccountList Run-Time Path and UML sequence diagram . . . . . . . 128 7.2 Diagram Showing Components in Duke’s Bank . . . . . . . . . . . . . 129 7.3 Diagram Showing Components in PlantsByWebsphere . . . . . . . . . 131 7.4 Portability Test Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 7.5 Performance Overhead Test Results . . . . . . . . . . . . . . . . . . . . 133 7.6 Test Results on K2 10 2 Database . . . . . . . . . . . . . . . . . . . . . 136 7.7 Test Results on K2 100 2 Database . . . . . . . . . . . . . . . . . . . . . 138 7.8 Test Results on Sun Database . . . . . . . . . . . . . . . . . . . . . . . . 139 7.9 Test Results on IBM Database . . . . . . . . . . . . . . . . . . . . . . . 140 7.10 Class Diagram of a Modified Version of Duke’s Bank with Commu- nication Patterns Highlighted . . . . . . . . . . . . . . . . . . . . . . . 143 A.1 The Transactions-A-Plenty Rule . . . . . . . . . . . . . . . . . . . . . . 172 A.2 The Conversational-Baggage Rule . . . . . . . . . . . . . . . . . . . . . 173 A.3 The Sessions-A-Plenty Rule . . . . . . . . . . . . . . . . . . . . . . . . . 173 A.4 The Needless-Session Rule . . . . . . . . . . . . . . . . . . . . . . . . . 174 A.5 The Remote-Calls-Locally Rule . . . . . . . . . . . . . . . . . . . . . . . 175 A.6 The Accessing-Entities-Directly Rule . . . . . . . . . . . . . . . . . . . 175 A.7 The Bloated-Session Rule . . . . . . . . . . . . . . . . . . . . . . . . . . 175 A.8 The Unusual-or-Bulky-Session-Entity-Communication Rule . . . . . . 176 A.9 The Fine-Grained-Remote-Calls Rule . . . . . . . . . . . . . . . . . . . 176 A.10 The Unused-Data-Object Rule . . . . . . . . . . . . . . . . . . . . . . . 177 A.11 The Incorrect-Pool-Size Rule . . . . . . . . . . . . . . . . . . . . . . . . 177 A.12 The Local-and-Remote-Intefaces-Simultaneously Rule . . . . . . . . . 178 v
- 7. List of Acronyms ADL:Architecture Description Language AOP: Aspect Oriented Programming API: Application Programming Interface AST: Abstract Syntax Tree BCI: Byte Code Instrumentation CCM: CORBA Component Model CPI: COMPAS Probe Insertion DTO: Data Transfer Object EIS: Enterprise Information Systems EJB: Enterprise Java Beans ERP: Enterprise Resource Planning FCA: Formal Concept Analysis FIM: Frequent Itemset Mining FSM: Frequent Sequence Mining HTML: HyperText Markup Language HTTP: HyperText Transfer Protocol J2EE: Java 2 Enterprise Edition J2SE: Java 2 Standard Edition JDBC: Java Database Connectivity JEE: Java Enterprise Edition JMS: Java Message Service JMX: Java Management Extensions JNDI: Java Naming and Directory Interface JSP: Java Server Pages JSR: Java Service Request JVM: Java Virtual Machine JVMPI: Java Virtual Machine Profiler Interface JVMTI: Java Virtual Machine Tools Interface KDD: Knowledge Discovery in Databases LQN: Layered Queuing Networks MTS: Microsoft Transaction Server OCL: Object Constraint Language OS: Operating System PAD: Performance Antipattern Detection PMI: Performance Monitoring Infrastructure POJO: Plain Old Java Object QN: Queuing Networks RDBMS: Relational Database Management Systems RMI: Remote Method Invocation RML Relational Manipulation Language SOA: Service Oriented Architecture SPA: Stochastic Process Algebras SPN: Stochastic Petri Nets SQL: Structured Query Language UML: Unified Modelling Language XML: Extensible Markup Language vi
- 8. ABSTRACT Enterprise applications arebecoming increasingly complex. In recent times they have moved away from monolithic architectures to more distributed systems made up of a collection of heterogonous servers. Such servers generally host numerous soft- ware components that interact to service client requests. Component based enterprise frameworks (e.g. JEE or CCM) have been extensively adopted for building such ap- plications. Enterprise technologies provide a range of reusable services that can assist developers building these systems. Consequently developers no longer need to spend time developing the underlying infrastructure of such applications, and can instead concentrate their efforts on functional requirements. Poor performance design choices, however, are common in enterprise applications and have been well documented in the form of software antipatterns. Design mistakes generally result from the fact that these multi-tier, distributed systems are extremely complex and often developers do not have a complete understanding of the entire ap- plication. As a result developers can be oblivious to the performance implications of their design decisions. Current performance testing tools fail to address this lack of system understanding. Most merely profile the running system and present large vol- umes of data to the tool user. Consequently developers can find it extremely difficult to identify design issues in their applications. Fixing serious design level performance problems late in development is expensive and can not be achieved through ”code op- timizations”. In fact, often performance requirements can only be met by modifying the design of the application which can lead to major project delays and increased costs. This thesis presents an approach for the automatic detection of performance design and deployment antipatterns in enterprise applications built using component based frameworks. Our main aim is to take the onus away from developers having to sift through large volumes of data, in search of performance bottlenecks in their applica- tions. Instead we automate this process. Our approach works by automatically recon- structing the run-time design of the system using advanced monitoring and analysis techniques. Well known (predefined) performance design and deployment antipat- terns that exist in the reconstructed design are automatically detected. Results of ap- plying our technique to two enterprise applications are presented. The main contributions of this thesis are (a) an approach for automatic detection of performance design and deployment antipatterns in component based enterprise frameworks, (b) a non-intrusive, portable, end-to-end run-time path tracing approach for JEE and (c) the advanced analysis of run-time paths using frequent sequence mining to automatically identify interesting communication patterns between com- ponents. vii
- 9. Dedicated to myparents, Tommy and Kay.
- 10. ACKNOWLEDGEMENTS Firstly I wouldlike to thank my supervisor, John Murphy for giving me the oppor- tunity to pursue this research, and also for all his help, encouragement and good humour along the way. I would also like to thank Liam Murphy who was always available for dialog and who has effectively acted as a second supervisor over the years. I would like to thank Andrew Lee for initially suggesting the notion of ”detect- ing antipatterns”, when I was back searching for research ideas. Also thanks to Peter Hughes for his input and feedback during the early days of my work. Next, I would like to thank my colleagues in Dublin City University, where this jour- ney first began. In particular, I would like to thank Ada Diaconescu, Mircea Trofin and Adrian Mos, from whom I learned so much during my initial two years as a re- searcher, for being fun colleagues, for always being available to bounce ideas off (even now that you have all unfortunately left Dublin) and for teaching me the important basics of the Romanian language. Furthermore I would like to thank Adrian for as- sisting me in extending some of his research ideas and for inviting me to INRIA for interesting discussions on my work. I would also like to thank Colm Devine, Adrian Fernandes and Cathal Furey (three of the four horsemen) for their engaging lunch time discussions (in the early days) on the human anatomy and other such topics. Thanks also to my DCU/UCD colleagues, Dave ”the brickie” McGuinness, Jenny Mc- Manis, Gabriel Muntean, Christina Muntean, Christina Thorpe, Alex Ufimtsev, Oc- tavian Ciuhandu, Lucian Patcas, Olga Ormand, Jimmy Noonan, Hamid Nafaa, Petr Hnetynka, Sean Murphy, John Fitzpatrick (for allowing me to introduce him to Las Ve- gas), John Bergin, Omar Ashagi (for teaching me basic Arabic) and Philip McGovern for all being fun colleagues and to all those who I have had the pleasure of working with over the years. Also thanks again to Sean Murphy for taking my questions over the years and especially for his help with some of the mathematical aspects of my research. Furthermore, thanks to all those in IBM who helped me during my work and granted me access to their environments. Thanks especially to Pat O’Sullivan and Simon Piz- zoli for their help, interest and invaluable feedback on my research. A special thanks to Claire Breslin for her endless support and patience, and for re- minding me about the more important things in life. Finally I would like to thank my parents, Tommy and Kay and brother, Tom, for their constant encouragement. I would especially like to thank my parents, to whom this work is dedicated. Without their unwavering love and support this work would not have been possible. ix
- 11. LIST OF PUBLICATIONS TrevorParsons, John Murphy. Detecting Performance Antipatterns in Component Based Enterprise Systems. Accepted for publication in the Journal of Object Technology. Trevor Parsons, John Murphy, Patrick O’Sullivan, Applying Frequent Sequence Mining to Identify Design Flaws in Enterprise Software Systems. In Proceedings of the 5th Inter- national Conference on Machine Learning and Data Mining, Leipzig, Germany, July 18-20, 2007. Trevor Parsons, John Murphy, Simon Pizzoli, Patrick O’Sullivan, Adrian Mos, Reverse Engineering Distributed Enterprise Applications to Identify Common Design Flaws. Pre- sented at the Software Engineering Tools For Tomorrow (SWEFT) 2006 Conference, T.J. Watson, New York, Oct 17 - 19, 2006. Liang Chen, Patrick O’Sullivan, Laurence P. Bergman, Vitorrio Castelli, Eric Labadie, Peter Sohn, Trevor Parsons. Problem Determination in Large Enterprise Systems. Pre- sented at the Software Engineering Tools For Tomorrow (SWEFT) 2006 conference, T.J. Watson, New York, Oct 17 - 19, 2006. (Abstract only available) Trevor Parsons, Adrian Mos, John Murphy. Non-Intrusive End to End Run-time Path Tracing for J2EE Systems. IEE Proceedings Software, August 2006 Trevor Parsons, John Murphy. The 2nd International Middleware Doctoral Symposium: Detecting Performance Antipatterns in Component-Based Enterprise Systems. IEEE Dis- tributed Systems Online, vol. 7, no. 3, March, 2006 Trevor Parsons. A Framework for Detecting Performance Design and Deployment Antipat- terns in Component Based Enterprise Systems. In Proceedings 2nd International Middle- ware Doctoral Symposium, ACM Press, art. no. 7, Grenoble, France, 2005 Trevor Parsons. A Framework for Detecting, Assessing and Visualizing Performance An- tipatterns in Component Based Systems. First Place at ACM SIGPLAN Student Research Competition Graduate Division, In OOPSLA’04: Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and ap- plications, pages 316-317, Vancouver, BC, Canada, 2004. x
- 12. Trevor Parsons, JohnMurphy. A Framework for Automatically Detecting and Assessing Performance Antipatterns in Component Based Systems using Run-Time Analysis. The 9th International Workshop on Component Oriented Programming, part of the 18th Eu- ropean Conference on Object Oriented Programming. Oslo, Norway, June 2004. Trevor Parsons, John Murphy. Data Mining for Performance Antipatterns in Component Based Systems Using Run-Time and Static Analysis. Transactions on Automatic Control and Control Science, Vol. 49 (63), No. 3, pp. 113-118 - ISSN 1224-600X, May 2004. xi
- 13. CHAPTER ONE Introduction Main Points • Performanceis a major issue during the development of enterprise applications. • System complexity leads to a lack of understanding and consequently poor de- sign decisions are commonly made by developers. • Poor system design is often responsible for a badly performing system. • Current performance testing tools do not address performance design issues and are limited. • There are a large number of well known design issues for enterprise systems. • Antipatterns document well known design issues and their corresponding solu- tion. • Thesis Contributions: – An approach for the automatic detection of performance design and de- ployment antipatterns in systems built on component based enterprise frameworks. – A portable, low overhead, non-intrusive, end-to-end run time path tracer for distributed JEE systems. – A technique for the identification of interesting communication patterns in a collection of run-time paths. 1
- 14. 1.1 Background andMotivation In the past software developers had to be extremely careful when developing their ap- plications as resources were often scarce and the management of such scare resources was a complex issue. Modern advances in software technologies, however, have al- lowed for developers to concentrate less on issues such as performance and resource management, and instead developers have been able to spend more time developing the functionality of their applications. An example of this can be seen in modern lan- guages (Java1, C#2) that provide garbage collection facilities, freeing developers from the task of having to manage memory, which had typically been a complex and time consuming exercise. Freeing developers from having to worry about what is happen- ing ”under the hood” allows them to concentrate more of their efforts on developing the functionality of a system. This is even more obvious with enterprise level com- ponent frameworks (e.g. JEE3 or CCM4) whereby the framework can be expected to handle complex underlying issues such as security, persistence, performance and con- currency to name but a few. Again the idea is to allow developers to concentrate on the application functionality such that the time to market is reduced. A downside of this advance in software technologies is that developers become less familiar with the mechanics of the underlying system, and as a result, can make decisions during development that have an adverse effect on the system. Performance is a major issue for developers building large scale multi user enterprise applications. In fact recent surveys have shown that a high percentage of enterprise projects fail to meet their performance requirements on time or within budget 5 6. This leads to project delays and higher development costs, and results from the fact that de- velopers often do not have a complete understanding of the overall system behaviour. Figure 1.1 shows a typical enterprise application made up of a number of different physically distributed servers. Each server can in turn be made up of a large number of software components that interact to service different client requests. Understand- ing the run-time behaviour of such systems can be a difficult task and consequently it is common that developers are unaware of the performance implications of their design decisions. Current development and testing tools fail to address this issue of understanding en- terprise system behaviour. For example most of today’s performance tools merely profile the running system and present performance metrics to the tool user. The 1The Java Technology, Sun Microsystems, http://java.sun.com/ 2The C# language, Microsoft, http://msdn2.microsoft.com/en-us/vcsharp/aa336809.aspx 3Java Enterprise Edition, Sun Microsystems, http://java.sun.com/javaee/ 4The CORBA Component Model specification, The Object Management Group, http://www.omg.org/technology/documents/formal/components.htm 5Ptak, Noel and Associates, ”The State of J2EE Application Management: Analysis of 2005 Benchmark Survey”, http://www.ptaknoelassociates.com/members/J2EEBenchmarkSurvey2005.pdf, 6Jasmine Noel, ”J2EE Lessons Learned ”, SoftwareMag.com, The Software IT Journal, January, 2006. http://www.softwaremag.com/L.cfm?doc=2006-01/2006-01j2ee 2
- 15. Figure 1.1: TypicalEnterprise Architecture volume of data produced when profiling even a single user system can be extremely large. When profiling multi-user enterprise applications, where a typical load may be in the order of thousands, the amount of data produced can be truly overwhelming. Often developers are required to sift through and correlate this information looking for bottlenecks in their systems. Furthermore, even when developers find issues in their applications using these tools, it is common that they are unsure as to how to go about rectifying the issue. There is a clear need for more advanced performance tools that not only profile the running system, but that also analyse the data produced to identify potential issues in the application. While there has been research in the area of debugging tools (e.g. [95] [145] [55] [14] [47]) which allow for automatic low-level bug detection, often it is the case that low-level optimizations or fixes will not be enough to enhance the system efficiency such that performance requirements are met. In many situations an overhaul of the system design is required. There are a large number of well known design mistakes that are consistently made by developers building these systems. Such issues have been documented in the form of software design antipatterns [36]. Similar to software design patterns, which doc- ument best practices in software development, software antipatterns document com- mon mistakes made by developers when building software systems. However, as well as documenting the mistake, antipatterns also document the corresponding solution to the problem. Thus not only can they be used to identify issues in software systems, but they can also be used to rectify these issues by applying the solution provided. A more complete and detailed definition of software patterns and antipatterns is given in sections 2.6 and 2.7 respectively. 3
- 16. 1.2 Thesis Overview Inlight of the limitations of current performance tools and of the benefits of software antipatterns, we have developed an approach to automatically identify performance design and deployment antipatterns in systems built on enterprise component-based frameworks. This approach takes the burden away from developers, of having to sift through large volumes of monitoring data in search of design flaws, and instead automates this process. Well known performance design flaws can be identified au- tomatically. Identified issues are presented with related contextual information and a corresponding solution to the problem such that the problem can be easily addressed. The approach works by reconstructing the run-time design of the application under test. The reconstructed design can be subsequently checked for well known pre- defined antipatterns. From a high level this is achieved (a) by monitoring the run- ning system to collect information required for antipattern detection, (b) by perform- ing analysis on the monitoring data to summarise it and to identify relationships and patterns in the data that might suggest potential design flaws, (c) by representing the analysed data in a design model of the system and (d) by loading the design into a rule engine such that antipatterns (pre-defined as rules) can be detected. The approach has been realised in the Performance Antipattern Detection (PAD) tool. The tool has been designed for the Java Enterprise Edition (JEE) technology. The remainder of the thesis is structured as follows: Chapter 2 gives background information on related technologies and related work. Chapter 3 gives a more de- tailed overview of our approach, discusses our research methodology and outlines a number of criteria that we use to validate our work. In this chapter we also give an overview of software design antipatterns, with particular focus on performance an- tipatterns. Chapter 4 outlines the different monitoring approaches that are required for antipattern detection in a component based enterprise system, and how they can be performed in a portable manner. Chapter 5 details a number of advanced anal- ysis techniques that are applied to identify interesting relationships and patterns in the run-time data. In particular it presents an approach for identifying frequent or re- source intensive communication patterns between components using techniques from the field of data mining. In this chapter we also show how the data collected from en- terprise systems under load can be reduced and summarised. Chapter 6 shows how a rule engine approach can be used to identify antipatterns in the reconstructed run- time design of the system. In this chapter we also categorise JEE performance design and deployment antipatterns into groups based on the data required to detect them. Chapter 7 presents different sets of results from a range of tests that we have per- formed to validate our research. Finally chapter 8 gives our conclusions and ideas on future work in this area. 4
- 17. 1.3 Thesis Contributionsand Statement The first major contribution of this thesis is an approach for the automatic detection of design and deployment antipatterns in systems built using component based enter- prise frameworks [125] [129] [130] [131] [132] [133]. This approach builds on current performance tools by performing analysis on the data collected (i.e. run-time infor- mation and component meta-data). The analysis reconstructs the system design and identifies performance design flaws within it. The approach has been implemented for the JEE technology in the form of the PAD tool, however it could potentially be applied to other component based enterprise frameworks (e.g. CCM). This solution has been successfully applied to both a sample and a real JEE application and has a number of key advantages. Firstly, it reduces and makes sense of the data collected by many of today’s perfor- mance profilers. This work makes use of statistical analysis and data mining tech- niques to summarise the data collected and to find patterns of interest that might suggest performance problems. Thus, it takes the onus away from developers who currently have to carry out this tedious task manually. Secondly, while most of today’s performance tools tend to focus on identifying low level hotspots and programming errors (e.g. memory leaks, deadlocks), this work focuses on analysing the system from a performance design perspective. Since design has such a significant effect on performance [43] it is essential that work is carried out in this area. Thirdly, unlike with many of today’s performance tools, problems identified are anno- tated with descriptions of the issue detected, as well as a solution that can be applied to alleviate the problem. This approach of identifying and presenting antipatterns to developers helps them understand the mistakes that have been made, and the under- lying reason as to why performance was affected. Developers can learn from using our tool, and thus it may be less likely that the same mistakes are made in the future. This approach also allows developers to easily rectify the situation by applying the solution provided. In fact, the antipatterns presented provide a high level language that developers and management alike can use to discuss such problems when they occur. The second major contribution of this work is a portable, low overhead, non- intrusive, end-to-end run-time path tracer for JEE systems [128]. This is the first com- pletely portable approach for collecting end-to-end run-time paths across all server side tiers of a distributed JEE application. It is non-intrusive insofar as it does not re- quire any modifications to the application or middleware source code. The monitor- ing approach instead makes use of standard JEE mechanisms to intercept calls made to the instrumented components. A run-time path [44] contains the control flow (i.e. the ordered sequence of methods called required to service a user request), resources 5
- 18. and performance characteristicsassociated with servicing a request. Such information is utilised to detect antipatterns by our PAD tool. By analysing run-time paths one can easily see how system resources are being used, how the different components in the system interact and how user requests traverse through the different tiers that make up the system. In fact these paths can also be used for Object Tracking, i.e. to monitor particular objects’ life cycles across the different user requests. In this work we show how run-time paths can be used to manually and automatically reverse engineer a JEE application. We also show how the reconstructed design can be used for either manual or automatic identification of performance design flaws. For example, the PAD tool makes use of run-time paths to identify the (run-time) component relation- ships, communication patterns and object usage patterns in a JEE system. Results are given for this monitoring approach which show that it produces a low overhead on the instrumented system and that it can be applied in a portable manner. The third and final major contribution of this work is a technique for the identifica- tion of interesting communication patterns in a collection of run-time paths [126]. More precisely, we have applied a data mining technique, Frequent Sequence Mining (FSM) to identify sequences of interest (e.g. frequently repeating method sequences and resource intensive loops) across a transactional database of run-time paths by us- ing alternative support counting techniques. In this work we also discuss scalability problems (in terms of both the algorithm runtime and the amount of data produced) related to applying FSM to run-time paths and give solutions to these issues. We show how the sequences identified, can be used to highlight design flaws in enter- prise applications, that lead to poor system performance. The PAD tool makes use of this analysis technique to identify interesting component communication patterns in a JEE system that may indicate the presence of particular antipatterns. Following the above contributions the fundamental thesis of this work can be stated as follows: Performance design and deployment antipatterns can be automatically detected in component based enterprise systems by analysing run-time data and component meta-data. 1.4 Key Assumptions and Scope The work in this thesis is focused on component based systems as defined in section 2.2. As such, it is highly likely that the source code of the application to be analysed is not available in its entirety, as components may have been developed by third par- ties. Thus we assume source code is not available for analysis of the system. For such systems bytecode analysis may also be problematic due to security restrictions or li- censing constraints. Instead, we assume that a running implementation of the system to be analysed is available such that dynamic data can be collected and utilised for analysis. 6
- 19. We also assumethat a realistic testing scenario is available which reflects how the sys- tem will be used in production. We do not address the issue of how such testing sce- narios can be obtained in this work, however, research in this area already exists. For example Weyuker and Voklos have outlined an approach for the development of per- formance test cases [176]. In this literature five typical steps required to develop per- formance test cases are outlined. Alternatively, Ho et al. [92] propose an evolutionary approach to performance test design based on their Performance Requirements Evolu- tion Model. The authors claim that more precise and realistic performance tests can be created incrementally during the development process through customer communi- cation or performance model solving. In addition, agile development techniques such as test-driven development [25] promote the design of test cases before developers begin to code. Recently work has been presented which discusses how performance tests can be incorporated into the test driven development process [96] allowing for early availability of performance testing scenarios. Our approach is applicable to applications built on component based enterprise frameworks. However our research has thus far only been applied to synchronous components, and has not, for example, been applied to message driven beans which are asynchronous components in the JEE technology. Thus, our scope is limited to synchronous applications. Our plans for future work outline how this approach could potentially be applied to asynchronous components (see section 8.3). 7
- 20. CHAPTER TWO Background In this chapterwe introduce related research areas and technologies. We begin by discussing the research area of performance engineering. Next we give an overview of component based software, giving a definition for a software component and dis- cussing component frameworks. We also give background information on the Java Enterprise Edition technology which is the enterprise framework that our work has been applied to. We focus specifically on the Enterprise Java Bean component tech- nology and give details in this area related to our research. We present an overview of the state of the art in performance tools discussing techniques for load generation and performance profiling. We particularly focus on performance profiling tools for the Java technology. Furthermore we give an overview of software architecture, software patterns, and software antipatterns. An overview of research in the area of reverse engineering is also presented. In this section we outline why previous approaches are less suitable for distributed component based applications. The current state of the art of research in the area of software pattern/antipattern detection is also discussed. Finally we introduce the area of knowledge discovery in databases, and data mining techniques relevant in this work. 8
- 21. Main Points • Currentperformance analysis techniques, e.g., modelling, are inaccurate and time consuming when applied to component based enterprise systems. Thus, in industry performance analysis is usually deferred until performance testing begins using the currently available performance testing tools. • Component technologies such as EJB are increasingly being adopted to provide for flexible, manageable and reusable solutions for complex software systems. However poor system performance is common in these systems. • System Architecture focuses on issues related to the overall system structure and is said to be non-local, whereas software design focuses on local issues . • Enterprise design plays a significant role in a system’s overall performance. Best practices in design have been well documented in the form of design patterns. • Well known design issues consistently occur in enterprise applications and have been well documented, along with their corresponding solution, in the form of design antipatterns. • Performance testing tools for complex-multi user enterprise applications are lim- ited and merely profile the running system, presenting vast amounts of data to the tool user. There is a clear need from more advanced tools, that take the onus away from the developer of having to sift through this data, and that automati- cally analyse the data produced. • Detailed documentation is generally not available for enterprise applications. Thus, it can be difficult for developers to comprehend the overall application design. • Current reverse engineering/design pattern detection/antipattern detection techniques are heavily based on static analysis and are unsuitable for compo- nent based systems • Data Mining techniques can be applied to extract knowledge from vast volumes of data. 9
- 22. 2.1 Performance ofSoftware Systems The performance of a software system has been described as an indicator of how well the system meets its requirements for timeliness [154]. Smith and Williams [154] de- scribe timeliness as being measured in either response time or throughput, where re- sponse time is defined as the time required to respond to a request and throughput is defined as the number of requests that can be processed in some specific time in- terval. Furthermore they define two important dimensions to software performance timeliness, responsiveness and scalability. Responsiveness is defined as the ability of a system to meet its objectives for response time or throughput. The ability to con- tinue to meet these objectives, as the demand on the system increases, is defined as the systems scalability. The aim of performance engineering is to build systems that are both responsive and scalable. To date a vast amount of performance engineering research has focused on system analysis through performance models. Performance models are created based on sys- tem artifacts and various relevant estimates. Some of the most common performance model classes are Queuing Networks (QN) (or extensions, such as Extended QN and Layered QN), Stochastic Petri Nets (SPN), and Stochastic Process Algebras (SPA). Per- formance models can be evaluated using simulation techniques or analytical methods, in order to predict performance indices, such as throughput, response times, or re- source utilization. A comprehensive survey of modelling approaches for performance prediction is presented in [15]. However modelling today’s enterprise applications with a high degree of accuracy can be a difficult and time consuming task. This results from the fact that these sys- tems are often very large and complex and made up of black box components, the internals of which are generally unknown (e.g. application servers). Performance metrics required to populate performance models can thus not be easily obtained. For enterprise applications accurate performance metrics can often only be obtained through performance testing of a running system [58]. Recently Liu et al. [76] have used a combination of performance modelling and benchmarking techniques to al- low for population of performance models of enterprise applications. Their initial results give accurate performance prediction for the small sample systems. A draw- back of this approach is the lack of tool support which would allow for this technique to be easily reused. From our experiences with large software houses, it seems that performance modelling of enterprise applications is rarely performed. Most opt for performance testing using available performance testing tools. Work in the area of performance testing, however, has been very much lacking [176] and thus performance testing, especially in the case of large enterprise applications, can be a difficult task [76] [161]. This comes from the fact that today’s performance testing tools are quite limited, insofar as they generally focus on simply collecting 10
- 23. data from arunning system (i.e. profiling) and presenting this data to the user. These tools tend to focus on low level programming bugs and do not address many of the issues that lead to poor system performance (e.g. design issues). 2.2 Component Based Software 2.2.1 Software Components There are numerous definitions of what software components are or should be 1. To be specific, for the purpose of this thesis, we use Szyperski’s definition of a software com- ponent: ”A software component is a unit of composition with contractually specified interfaces and explicit context dependencies only. A software component can be de- ployed independently and is subject to composition by third parties” [157]. Defining a software component as a unit of composition simply means that the purpose of a com- ponent is to be composed with other components. A component based application is assembled from a set of collaborating components. To be able to compose components into applications, each component must provide one or more interfaces which provide a contract between the component and its environment. The interface clearly defines which services the component provides and therefore defines its responsibility. Usu- ally, software depends on a specific context, such as available database connections or other system resources being available. For example, other components that must be available for a specific component to collaborate with. In order to support com- posability of components, component dependencies need to be explicitly specified. A component can be independently deployed, i.e. it is self-contained and changes to the implementation of a component do not require changes to other components. Of course, this is only true as long as the component interface remains compatible. Fi- nally assemblers of component based applications, are not necessarily the developers of the different components. That is, components can be deployed by third parties and are intended to be reused. This definition of a software component leaves many details open, for example, how components interact, what language(s) can be used for their development, for what platform. Component frameworks further define the notion of a component, by de- tailing these issues. 2.2.2 Component Frameworks The key goal of component technology is independent deployment and assembly of components. Component frameworks are the most important step for achieving this 1Beyond Objects column of Software Development magazine, articles by Bertrand Meyer and Clemens Szyperski, archived at http://www.ddj.com/ 11
- 24. aim [157]. Theysupport components conforming to certain standards (or component models) and allow instances of these components to be plugged into the component framework. The component framework establishes environmental conditions for the component instances and regulates the interaction between component instances. A key contribution of component frameworks is partial enforcement of architectural principles. By forcing component instances to perform certain tasks via mechanisms under control of a component framework the component framework can enforce its policies on the component instances. This approach helps prevent a number of classes of subtle errors that can otherwise occur. There are numerous component frameworks that exist today. Examples include EJB, CCM, SOFA [140] and Fractal [37]. Each framework contains its own component model, i.e. a set of features that components satisfy. Component models generally con- form to either flat component models or hierarchical models. Flat component models (e.g. EJB, CCM) define only primitive components whereby indivisible entities are di- rectly implemented in a programming language. Hierarchical models (SOFA, Fractal) also define composite components which are created via nesting of other components. The research in this thesis aims at solving issues related to flat component models. In particular we focus on EJB. EJB is part of a wider enterprise framework (Java Enter- prise Edition) for building enterprise level applications (see section 2.3). EJB has been selected since it is a well established technology that is currently used in industry to develop enterprise applications. There is also a body of work detailing best practices and bad practices for this technology (see sections 2.6 and 2.7). On the other hand the hierarchical component models have mainly been used by the research community and best practices in these areas are less well defined. EJB is considered a contextual composition framework. Contextual composition frameworks allow components to specify boundary conditions describing properties that the runtime context must meet [166]. Composition performed by such frame- works is based on the creation of contexts and placement of component instances in the appropriate contexts. For example, a component framework for transactional computation can be formed by supporting transactional attribution of components (for example ”this component’s instances need to be used in a new transaction”) and transactional enactment at the boundary of contexts. This approach can be used to cre- ate frameworks for any other properties such as security, load balancing, management etc. Any component instance in a context can potentially be accessed from outside its con- text. This context however, gets an opportunity to intercept all messages crossing the context boundaries. Intercepting instances (e.g. objects) inside a context remains in- visible to instances both external and internal to this context. Current technology support for contextual composition includes Microsoft Transac- 12
- 25. tion Server (MTS)2,EJB containers, and CCM containers. We introduce the EJB tech- nology in the following sections. The run-time services we discuss in section 5.3 are created as a result of contextual composition. 2.3 The Java Enterprise Edition The Java Enterprise Edition (JEE) is a component technology which defines a standard (or a set of standards) for developing multi-tier enterprise applications. JEE, formerly the Java 2 Enterprise Edition (J2EE), is an enterprise component framework for the Java technology. The specification promotes a multi-tiered distributed architecture for enterprise applications. Figure 2.1 shows a typical JEE architecture consisting of 4 main tiers: a client tier, a presentation or web tier, a business tier and an enterprise information systems tier. JEE specifies different component types for implementing the various enterprise application tiers. Naturally, clients reside in the client tier and can be in the form of stand-alone Java applications or web browsers. In the following subsections we detail each of the server-side tiers and give details on the components that they can consist of. Figure 2.1: Typical JEE Architecture 2.3.1 Web Tier The JEE web tier provides a run-time environment (or container) for web components. JEE web components are either servlets or pages created using the Java Servlet Pages 2Microsoft Corporation. Microsoft Transaction Server Transactional Component Services. http://www.microsoft.com/com/wpaper/revguide.asp. 13
- 26. technology (JSPs) 3.Servlets are Java programming language classes that dynamically process requests and construct responses. They allow for a combination of static and dynamic content within the web pages. JSP pages are text-based documents that ex- ecute as servlets but allow a more natural approach to creating the static content as they integrate seamlessly in HTML pages. JSPs and Servlets execute in a web con- tainer and can be accessed by clients over HTTP (e.g. a web browser). The servlet filter technology is a standard JEE mechanism that can be applied to components in the web tier to implement common pre and post-processing logic. It is discussed in detail in section 4.2.5.1. 2.3.2 Business Tier Enterprise Java Beans (EJBs) 4 are the business tier components and are used to handle business logic. Business logic is logic that solves or meets the needs of a particular business domain such as banking, retail, or finance for example. EJBs run in an EJB container and often interact with a database in the EIS tier in order to process requests. Clients of the EJBs can be either web components or stand alone applications. EJB is the core of the JEE platform and provides a number of complex services such as messaging, security, transactionality and persistence. These services are provided by the EJB container to any EJB component that requests them. More details on the EJB component model are given in section 2.4 2.3.3 Enterprise Information System Tier Enterprise information systems provide the information infrastructure critical to the business processes of an enterprise. Examples of EISs include relational databases, enterprise resource planning (ERP) systems, mainframe transaction processing sys- tems, and legacy database systems. The JEE Connector architecture 5 defines a stan- dard architecture for connecting the JEE platform to heterogeneous EIS systems. For example a Java Database Connectivity (JDBC) Connector is a JEE Connector Archi- tecture compliant connector that facilitates integration of databases with JEE appli- cation servers. JDBC 6 is an API and specification to which application developers and database driver vendors must adhere. Relational Database Management Systems (RDBMS) vendors or third party vendors develop drivers which adhere to the JDBC specification. Application developers make use of such drivers to communicate with the vendors’ databases using the JDBC API. The main advantage of JDBC is that it allows for portability and avoids vendor lock-in. Since all drivers must adhere to the 3Java Servlet Technology, http://java.sun.com/products/servlet/ 4Enterprise Java Bean Technology, http://java.sun.com/products/ejb/docs.html 5Java Connector Architecture, http://java.sun.com/j2ee/connector/ 6Java Database Connectivity Architecture, http://java.sun.com/products/jdbc/ 14
- 27. same specification, applicationdevelopers can replace the driver that they are using with another one without having to rewrite their application. 2.4 The Enterprise JavaBean Technology The Enterprise Java Beans architecture is a component architecture for the develop- ment and deployment of component based distributed applications. It is designed to simplify and reduce the costs of the development and management processes of large- scale, distributed applications. Applications built using this technology are capable of being scalable, transactional, and multi-user secure. EJB provides the distributed platform support and common services such as transactions, security, persistence and lifecycle management. EJB also defines a flexible component model which allows for components of different types that are suitable for specific tasks. Developers make use of the different component types to implement the application business logic. Subse- quently, EJBs are deployed and managed by EJB containers, as part of a JEE applica- tion server. EJB containers provide middleware services and manage the EJB lifecycle during runtime. These processes can be configured via XML documents, referred to as EJB deployment descriptors. Physically EJB consists of two things [148]: The specification7 which defines: • The distinct ”EJB Roles” that are assumed by the component architecture. • A component model • A set of contracts: component-platform and component-client A set of Java Interfaces: • Components and application servers must conform to these interfaces. This al- lows all conforming components to inter-operate. Also the application server can manage any components that conform to the interfaces. 2.4.1 The EJB Roles The EJB specification defines the following roles which are assumed by the component architecture: • Enterprise Bean Provider: The enterprise bean provider is typically an applica- tion domain expert. The bean provider develops the reusable enterprise beans that typically implement business tasks or business entities. 7The Enterprise Java Bean Specification version 2.0, http://java.sun.com/products/ejb/docs.html 15
- 28. • Application Assembler:The application Assembler combines enterprise beans into larger deployable application units. • Deployer: The Deployer takes the ejb-jar files produced by either the Bean Provider or the Application Assembler and deploys the enterprise beans con- tained in the ejb-jar files in a specific operational environment. The operational environment includes the EJB Server and Container. • EJB Service Provider and EJB Container Provider: The container supplies an EJB container (the application server). This is the runtime environment in which the beans live. The container provides middleware services to the beans and manages them. The server provider is the same as the container provider. Sun has not yet differentiated between them. • System Administrator: The system administrator is responsible for the upkeep and monitoring of the deployed system and may make use of runtime monitor- ing and management tools provided by the EJB server provider 2.4.2 The EJB Component Model EJB is built on top of object technology (Java). An EJB component consists of a busi- ness interface, an implementation class, a home interface and configuration settings (defined in an XML deployment descriptor). All of these, except for the deployment descriptor, are Java artifacts (i.e. classes or interfaces). The EJB implementation class contains the bean business logic written by the Enter- prise Bean Provider. The EJB implementation class is a Java object that conforms to a well defined interface and obeys certain rules. The interface it conforms to depends on the bean type. The rules are necessary in order for the bean to be able to run in a container. Access to the implementation class can be obtained using a the EJB home interface. The home interface defines methods for creating, destroying and finding EJBs (i.e. lifecycle methods). The home interface can either be local or remote. Local interfaces allow access from clients within the same JVM whereas remote interfaces allow for access from remote clients (e.g. on another JVM running on the same machine or on a JVM running on a physically distributed machine). In fact an EJB component can have both local and remote interfaces, however this is not recommended [161]. The bean implementation business methods are exposed through the business inter- face. Similar to the home interface the business interface can be exposed locally or remotely (or both). An EJB component also requires configuration settings for deployment. These settings are defined in a XML deployment descriptor. The information in the deployment 16
- 29. descriptor details thedifferent container services that are required by the EJB. For example, a deployment descriptor can be used to declare how the container should perform lifecycle management, persistence, transaction control, and security services. EJB 2.0 defines three different kinds of enterprise beans, namely session beans, entity beans and message-driven beans. Session Beans: A session bean is an action bean that performs work for its client, shielding the client from complexity by executing business tasks inside the server. A session bean has only one client. When the client terminates the session appears to terminate and is no longer associated with the client. The life of a session bean spans the length of the session (or conversation) between the session and the client. Session beans are not persistent and typically they do not survive application server crashes, or machine crashes. They are in memory objects that live and die with their surround- ing environments. Session beans hold conversations with clients. A conversation is an interaction between a client and the bean. The two subtypes of session beans are state- ful session beans and stateless session beans. Each is used to model different types of conversations. Stateful Session Beans: A stateful session bean is a bean that is designed to service business processes that span multiple method requests or transactions. Stateful ses- sion beans retain state on behalf of an individual client. If a stateful session bean’s state is changed during a method invocation, that same state will be available to that same client upon the following invocation. Stateless Session Beans: A stateless session bean is a bean that holds conversations that span a single method call. They are stateless because they do not hold multi- method conversations with their clients. Except during method invocation, all in- stances of a stateless bean are equivalent, allowing the EJB container to assign an in- stance to any client. Because stateless session beans can support multiple clients, they can offer better scalability for applications that require a large number of clients. Typ- ically, an application requires fewer stateless session beans than stateful session beans to support the same number of clients. Entity Beans: Entity beans are persistent data components. Entity beans are enterprise beans that know how to persist themselves permanently to a durable storage (e.g. a database). They are physical, storable parts of an enterprise. Entity beans differ from session beans in a number of ways. They are persistent, and allow shared access. They have a unique identifier, enabling a client to identify a particular entity bean. Entity beans can also persist in relationships with other entity beans. Entity beans can be per- sisted in two ways, either using Bean-Managed Persistence, or Container-Managed Persistence. Container-Managed persistent beans are the simplest for the bean de- veloper to create. All logic for synchronizing the bean’s state with the database is handled automatically by the container. Thus, the beans do not contain any database access calls, and as a result the bean’s code is not tied to a specific persistent storage 17
- 30. mechanism (database). ABean-managed persistent entity bean is an entity bean that must be persisted by hand. The component developer must write code to translate the in-memory fields into an underlying data store. Message-driven Beans: A message-driven bean is an enterprise bean that allows EJB applications to process messages asynchronously. They rely on the Java Message Ser- vice (JMS) technology 8. Message-driven beans act as JMS message listeners. The messages may be sent by any JEE component: an application client, another enter- prise bean, a Web component or by a JMS application or system that does not use JEE technology. A message-driven bean does not have component interfaces. The com- ponent interfaces are absent because the message-driven bean is not accessible via the Java RMI API; it responds only to asynchronous messages. One of the most impor- tant aspects of message-driven beans is that they can consume and process messages concurrently, because numerous instances of the MDB can execute concurrently in the container. This capability provides a significant advantage over traditional JMS clients. As discussed in section 1.4 we have not applied our research to asynchronous compo- nents. This is a direct result of the fact that our run-time path tracing approach (see section 4.2) can not currently be used to monitoring message driven beans. Our plans for future work suggest how this problem may be addressed (see section 8.3). 2.4.3 EJB Runtime An EJB component contains a bean implementation class, a business interface, a home interface and an XML deployment descriptor all of which are supplied by the bean provider. To integrate the component into the container environment the container automatically generates ”glue-code” that allows for the component to implicitly make use of the container services. In fact, enterprise beans are not fully-fledged remote ob- jects. When a client accesses an EJB, the client never invokes the methods directly on the actual bean instance. Instead, the invocation is intercepted by the EJB container and delegated to the bean instance. The interception is performed by the EJBObject. The EJBObject is generated by the container (either during deployment or at run-time) and provides the enterprise bean with networking capabilities and container services such as transactions and security. The EJBObject replicates and exposes every business method that the bean exposes. It is generated from the business interface supplied by the bean provider. Similarly an EJBHome object is generated from the home interface. The EJBHome object exposes the same methods as this interface and acts as a factory object for EJBObjects. That is, the EJBHome Object is responsible for creating and de- stroying EJBObjects. In order to understand how the various component constituents work together we give an example of the various steps that are performed by a client 8Java Message Service, from Sun Microsystems: http://java.sun.com/products/jms/ 18
- 31. and by thecontainer when a bean is invoked. To create an instance of an EJB a client must first obtain an instance of a EJBHome ob- ject, generated by the container. The EJBHome object is bound to the component name and available at run-time through the system’s naming directory, accessed through the Java Naming and Directory Interface (JNDI)9. Thus to invoke an EJB a client performs the following steps (see figure 2.2): (1) It first obtains a reference to the EJBHome object that the container has generated. The reference is looked up in the system-naming directory via JNDI. The client will call the required construction method on the home object. 2) The EJBHome object instructs the container to create a new instance or retrieve an existing instance of the component, and returns it to the client. The actual Java Object returned is an instance of the container-generated EJBObject class that corresponds to the bean’s component interface. 3) The client invokes the business method on the returned EJBObject, transparently, through the component interface. The EJBObject performs the required container ser- vices (specified in the XML deployment descriptor) and calls the corresponding busi- ness method on the bean’s implementation object, instance of the bean provider’s bean class. Figure 2.2: Client Invoking an EJB 9Java Naming and Directory Interface (JNDI), http://java.sun.com/products/jndi/ 19
- 32. Figure 2.3: ExampleEJB Deployment Descriptor 2.4.4 Deployment Settings As shown in figure 2.2 the container generated EJBObject intercepts and delegates all calls to the bean implementation. The EJBObject supplies the bean implementation with any required services as specified in the deployment descriptor. Figure 2.3 shows an extract from a deployment descriptor which specifies transactional attributes for a beans methods. Such settings can have a major impact on the system performance 10 and should be carefully considered. The container is also responsible for the management of the bean’s life cycle events. 10Performance Tuning EJB Applications - Part I by Mihir Kulkarni, February 2005, http://dev2dev.bea.com/pub/a/2005/02/perf tune session beans.html 20
- 33. Figure 2.4: StatelessSession Bean Lifecycle 21
- 34. The management ofan EJB’s lifecycle is a complex process and differs from bean type to bean type. Factors which influence the bean lifecycle management include the load in the system and the container configuration settings 11. Figure 2.4 illustrates the lifecycle of a stateless session bean. When the container starts the application it creates a pool of bean instances. The pool size can be determined by setting the value in the configuration settings. If a bean instance is required by a client, an instance is assigned from the bean pool. If no in- stances are available the container can create more instances until the pool has reached its maximum size (which is also configurable). If the bean pool has already reached its maximum size and there are still no instances available the client will be put in a queue until an instance becomes available. The pool configuration settings can have a major impact on the system performance and should be tuned according to the ex- pected load on the system. The lifecycle of a stateful session bean and of an entity bean are similar but slightly more complicated than that of the stateless session (since they can both be passivated). More details on these lifecycles can be found in the literature [148]. It is sufficient to say for the purposes of this thesis that the configurations settings in relation to EJB lifecycles can have a major impact on the system performance and need to be carefully considered. 2.5 Software Architecture A large number of definitions exist for the term software architecture 12. One of most cited definitions is by Bass et al. [21] and states that: ”The software architecture of a program or computing system is the structure or struc- tures of the system, which comprise software elements, the externally visible proper- ties of those elements, and the relationships among them.” Bass et al. [21] also outline a number of implications of this definition: Firstly, a soft- ware architecture is essentially an abstraction since it embodies information about the relationship between elements, and externally visible properties that are exposed to other elements, but it omits internal element information or information that does not pertain to the elements’ interactions. Secondly, the definition makes it evident that systems can and do consist of more than one structure. Thirdly, it is implied by the definition that every software system has an architecture since every system can be shown to compose of elements and relationships between them. Fourthly, the exter- nal behaviour of each element is part of the architecture and finally the definition is indifferent as to whether the architecture for a system is a good or bad one. 11PreciseJava, http://www.precisejava.com/ 12Software Engineering Institute, Carnegie Mellon, list of software architecture definitions, http://www.sei.cmu.edu/architecture/definitions.html 22
- 35. A software architectureis important for a number of reasons. Firstly it becomes a vehicle for communication among the system’s stakeholders [21]. System stakehold- ers are those concerned with the system (e.g. users, customers, software developers, management etc.). A software architecture is a common abstraction of the system and can serve as a lingua franca, i.e. an intermediate language that all stakeholders can use to discuss various aspects of the system. The different stakeholders of the system are often concerned with different system characteristics. An architecture pro- vides a common language in which these different concerns can be expressed. Since stakeholders can be interested in different system characteristics it is important for the architecture to provide different views [51] that let them consider the architecture from different perspectives. For example, a functional view might contain an abstrac- tion of the different system functions and their relations whereas a code view may give an abstraction of the code in terms of objects or classes (or higher level subsys- tems or modules) and their relationships. Different stakeholders make use of different views to analyse the architecture according to their needs. Typical views include a functional view, a concurrency view, a code view, a physical view etc [52]. Kruchten [104] introduced the 4+1 view model to describe a software architecture using five concurrent views. Views are essentially a mechanism that allow for the separation of concerns within the architecture allowing for the analysis of the architecture from dif- ferent perspectives. Architecture description languages (ADLs) [50] can be utilised to describe a software architecture. There have been many attempts to design such lan- guages. However while some have been employed in real word projects none have been widely adopted [21]. The literature [115] provides a comparison of ADLS. Another important reason for system architecture is that it creates a realisation of early design decisions and allows for system architects to analyse the suitability of these de- cisions in relation to the system requirements (e.g. performance, security, flexibility) [21]. These early design decisions manifested in the system architecture can not only impact the quality attributes of the system but can also place constraints on the actual system implementation i.e. some technologies may be more suitable for particular architectures. The initial architecture can even have an impact on the organisational structure of the team (or teams) building the application [21]. One of the earliest de- sign decisions is often to choose a suitable architectural style. An architectural style defines a vocabulary of components (e.g. clients, servers, databases) and connector types (e.g. procedure calls, database protocols), and a set of constraints on how they can be combined [152]. Architectural styles are found repeatedly in practice to address similar sets of demands. Finally, software architectures are also reusable assets that can be applied repeatedly to other systems exhibiting similar requirements [21]. 23
- 36. 2.6 Software Patterns Thecurrent use of the term pattern in software engineering is derived from work by Christopher Alexander [168] in the field of contemporary architecture. Alexander’s notion of a pattern was adopted by a number of software engineering researchers [26] [71] and became popular in this field mainly after work published by Gamma et. al [72]. Gabriel 13 gives the following definition of a pattern: ”Each pattern is a three-part rule, which expresses a relation between a certain context, a certain system of forces which occurs repeatedly in that context, and a certain software configuration which allows these forces to resolve themselves.” This definition is consistent with Alexan- der’s original definition [168] which states that a ”pattern is a three part rule which expresses a relation between a certain context a problem and a solution.” Alexander expands his definition to say that a problem relates to a certain system of forces which occurs repeatedly in a context and that the problem solution can be considered as a certain configuration which allows these forces to resolve themselves. While patterns have been documented for a number of different domains (such as patterns for con- temporary architecture [168] or organisational patterns [54]) we are mainly interested in software patterns. Software patterns are usually documented according to a pattern template. Common templates for describing patterns include the Alexandrian form [168] and the GoF form [72]. A given template contains a number of elements that describe the pattern e.g. name, problem, context, forces, solution, examples, resulting context, rationale, related patterns and known uses 14. Buschmann et al. [42] document a number of properties or benefits of patterns. While they focus on patterns for software architecture many of the properties hold for soft- ware patterns in general e.g.: • A pattern addresses a recurring problem that arises in specific situations, and presents a solution to it [42]. • Patterns document existing, well proven experience. That is, they document solutions learned through experience and avoid the need for less experienced developers to ”reinvent the wheel” time and time again [72]. • Patterns provide a common vocabulary and understanding for design principles [72]. Similar, to the way a software architecture can serve as a vehicle for com- munication (see section 2.5 above), pattern names can become part of a design language and can act as a lingua franca facilitating discussion of design issues and their solutions [42]. • Patterns support the construction of software with defined properties [42]. Pat- terns assist developers in meeting both functional and non-functional require- 13The Hillside Group, Pattern Definitions, http://www.hillside.net/patterns/definition.html 14Patterns and Software: Essential Concepts and Terminology, by Brad Appleton, http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html 24
- 37. ments since theycan provide a skeleton of functional behaviour while at the same time they can explicitly address non-functional requirements e.g. reuse- ability, maintainability etc. Software patterns can be documented at various levels of abstraction. For example, Buschmann et al. [42] discuss patterns at three different levels of abstraction, i.e., architectural patterns, design patterns and coding patterns or idioms. Architectural level patterns are concerned with system structure. They describe predefined sets of subsystems, specify their responsibilities and include rules and guidelines for organ- ising the relationships between them. Design patterns on the other hand tend to be at the level of objects and classes (or micro-architectures) and are used for refining sub- systems or components of a software system. In the literature [72] they are defined as ”descriptions of communicating objects and classes that are customized to solve a general design problem in a particular context.” Eden and Kazman have also distin- guished between architecture and design stating that architecture is concerned with non-local issues whereas design is concerned with local issues [62]. Coding patterns or idioms are lower level patterns specific to a programming language [53]. Since their introduction in the area of object oriented software development [72], pat- terns have been documented for a range of systems and technologies 15. For examples pattern catalogs exits in areas such as enterprise systems [67] [93], embedded-systems [142], telecommunication systems [171] to name but a few. Many technology specific patterns (or idioms) also exist (e.g. for Java [80], Ajax [110] and Microsoft technolo- gies 16). In fact pattern catalogs even exist with particular quality attributes in mind (e.g. security [150], performance [154]). Alur et al. [7] provide a catalog of patterns for the JEE technology which document best practices for the design and implementation of JEE applications. Other literature in this area also exists [113] 17. The design of a JEE application plays a major role in the overall system performance. For example, it has previously been shown how the system design can influence a JEE system’s scal- ability [43]. In fact it is well known and recent reports 18 19 also indicate that poor system design is a major reason as to why JEE systems often fail to meet performance requirements. Another reason why poor software design is particularly undesirable is that unlike with lower level software bugs, for example, poor software design can be particularly difficult to rectify late in development and as such can lead to major project delays. Software design best practices documented in the form of patterns can be used to help avoid design issues when developing JEE applications. 15Handbook of Software Architecture, http://www.booch.com/architecture/index.jsp 16Enterprise Solution Patterns Using Microsoft .NET, http://msdn2.microsoft.com/en- us/library/ms998469.aspx 17The Server Side Pattern Repository, http://www.theserverside.com/patterns/index.tss, 18Ptak, Noel and Associates, ”The State of J2EE Application Management: Analysis of 2005 Benchmark Survey”, http://www.ptaknoelassociates.com/members/J2EEBenchmarkSurvey2005.pdf, 19Jasmine Noel, ”J2EE Lessons Learned ”, SoftwareMag.com, The Software IT Journal, January, 2006. http://www.softwaremag.com/L.cfm?doc=2006-01/2006-01j2ee 25
- 38. 2.7 Software Antipatterns Antipatterns,first suggested by Koenig [101], have been defined by Brown et al. [36] as: ”a literary form that describes a commonly occurring solution to a problem that generates decidedly negative consequences.” The authors [36] also state that when documented properly an ”antipattern describes a general form, the primary causes which led to the general form; symptoms describing how to recognize the general form; the consequences of the general form; and a refactored solution describing how to change the antipattern into a healthier situation.” Software design antipatterns thus provide the opportunity for developers to learn from past experiences. They docu- ment software design mistakes that tend to consistently reoccur. However, as well as documenting the mistake, antipatterns also document the corresponding solution. Thus they allow developers to identify design issues in their system, and to rectify these issues with the corresponding solution provided in the antipattern description. Antipatterns are complementary to software patterns and often show situations where patterns are misused. In fact, as technologies evolve often patterns can become stale [60], i.e., what was once a best practice can in some instances become a bad practice. Examples from the JEE technology include the caching with a Service Locator pat- tern [7], which was recommended for J2EE 1.2 but is not recommended for J2EE 1.3 20. Another example is the Composite Entity pattern [7] which has become obsolete since EJB version 2.x [61]. Figure 2.5 [36] shows the relationship between patterns and antipatterns. Software design antipatterns, like software design patterns, have been documented at a number of different levels. For example Brown et al. [36] introduced a num- ber of technology independent object oriented development antipatterns (as well as higher level architectural and management antipatterns). Technology specific antipat- terns have also been documented (e.g. Java [160], J2EE [161] [61]). Antipatterns for systems built on service oriented architectures (SOA) have also been recently doc- umented 21. As with software design patterns, some antipattern catalogs focus on particular software quality attributes only. For example Smith and Williams have pre- sented a number of performance related antipatterns [154], while Kis has presented antipatterns focusing on security [99]. For the purpose of this thesis we focus mainly on performance related antipatterns for enterprise systems. In particular we focus on performance antipatterns related to design and deployment for JEE applications. Similar to software design antipatterns, Fowler and Beck introduced the notion of code smells [68]. Code smells are lower level symptoms of problems at the code level. 20B. Woolf. IBM WebSphere Developer Technical Journal: Eliminate caching in service locator implementations in J2EE 1.3., http://www-128.ibm.com/developerworks/websphere/techjournal/0410 woolf/0410 woolf.html, October 2004 21SOA antipatterns,Jenny Ang, Luba Cherbakov and Mamdouh Ibrahim, November 2005, http://www-128.ibm.com/developerworks/webservices/library/ws-antipatterns/ 26
- 39. Figure 2.5: Patterns,Antipatterns and their Relationship They may not necessarily be a problem but often their presence in the code indicate that problems exist. Catalogs of code smells are available in the literature 22 23. 2.8 Performance Tools When design issues lead to poor performance, developers require testing tools to iden- tify why the system is performing poorly. Developers use performance testing tools to try to understand system behaviour and to discover how their system makes use of the system resources. Application level performance testing tools fall into two main categories, i.e. workload generation tools and performance profilers. 2.8.1 Workload Generation In order to evaluate the performance characteristics of an application under devel- opment a realistic workload is required to mimic how the system would be utilised by clients in a production environment. To achieve this a synthetic workload can be automatically generated using a workload generator. Workload generators fall into two main categories, traced based approaches and analytical approaches [17]. Traced based approaches make use of server log files to characterise the workload of an ap- plication, whereas analytical approaches are based on mathematical models which are 22A Taxonomy of Code Smells, http://www.soberit.hut.fi/mmantyla/BadCodeSmellsTaxonomy.htm 23Smells within Classes, http://wiki.java.net/bin/view/People/SmellsToRefactorings 27
- 40. usually based onstatistical methods [124]. There are advantages and disadvantages associated with both approaches. For example, traced based approaches are consid- ered relatively easy to implement and are based on activity from a known system. However disadvantages relate to the fact that this approach treats the workload as a black box and as such insight into the workload characteristics can be difficult to ob- tain. Also it can be difficult to modify the workload to simulate future or alternative conditions. Furthermore during development realistic logs may not be available to base the traced based workload generation upon. Analytical approaches on the other hand can be used to create synthetic workloads and do not suffer from the drawbacks outlined above. However they can be more difficult to construct as an understanding of the characteristics of the expected workload is required. The most commonly used workload generators are the analytically based commercial profilers e.g. Apache’s JMeter 24 or Mercury Loadrunner25. The literature [136] gives a representative subset of the workload generators currently available in the open literature. 2.8.2 Profiling Tools Next we explain what we mean by the term profiling and discuss the different ways and levels of granularity in which profiling information can be collected (see section 2.8.2.1). We also give an overview of the different categories of profilers that are avail- able for the Java technology (see section 2.8.2.2). Profiling [169], is the ability to monitor and trace events that occur during run time. This includes the ability to track the cost of these events, as well as the ability to at- tribute the cost of the events to specific parts of the program. A profiler, for example, may obtain information about what part of the program consumes the most CPU time, or about the parts of the program which allocate the most amount of memory. Perfor- mance profilers can be used in conjunction with load generators to monitor a running system and obtain the required information for performance analysis. Profilers are often described as either exact profilers or sampling based profilers [135] [30]. Ex- act profiling also referred to as full profiling [64] or full instrumentation 26, captures all events of a given type that are produced during program execution (e.g. method invocations). Sampling based profilers on the other hand select a part of the entire event population with the aim of determining the characteristics of the whole pro- gram. Sampling usually involves selecting a subset of events for profiling based on certain criteria (e.g. hot paths) or time intervals [64]. Exact profiling has the advantage of being more precise than sampling but carries a higher performance overhead. 24Jakarta Apache JMeter http://jakarta.apache.org/jmeter/index.html. 25Mercury Loadrunner. http://mercury.com 26http://profiler.netbeans.org/docs/help/5.5/custom instrumetation 28
- 41. 2.8.2.1 Recording Information Regardlessof the profiling approach however, information must be recorded by the profiling tool. Performance metrics can be recorded at different levels of granular- ity. At the lowest level hardware counters can be utilised to obtain performance met- rics from the underlying hardware on which the program executes [8] [10] [156] [88]. Hardware counters can be utilised to record events, such as instructions executed, cy- cles executed, pipeline stalls, cache misses, etc. One of the main advantages of using hardware counters is that an application can be profiled without the need to modify or instrument it. Also the overhead associated with using hardware counters for profil- ing is generally quite low [10]. A disadvantage of hardware counters is that they rely on platform specific features and thus they are generally not portable across different hardware. Another issue with this type of profiling is that the information may be too low level for higher level program analysis. Information such as virtual memory management requests, and signals caused by segmentation violations can be obtained at the operating system (OS) level [88]. OS level information can be recorded by sys- tem level tools 27 or libraries 28. In situations where hardware counters or OS level information is unavailable, or the information they produce is undesirable, informa- tion can be obtained at a higher level. For today’s enterprise Java applications such information can be recorded at a number of different levels i.e. at the JVM level, the middleware level or the application level. JVM level information is generally recorded by either instrumenting the JVM or by using an agent-based approach that requests notification of events from the virtual machine. The former very often requires access to the JVM source code such that it can be modified to record the information required [12] [33]. A disadvantage of this approach is that it requires an understanding of the complex JVM internals. Also this approach generally ties the user to a particular JVM and is thus not portable. One of the main advantages of instrumenting the JVM is that access to JVM level information is not restricted as with the agent-based approaches. Agent based approaches have been made popular through standard interfaces that allow for a profiler agent to request performance related information from a running JVM. The Java Virtual Machine Profiler Interface (JVMPI) [169] is an example of such an interface (see figure 2.6). The JVMPI is a two-way function call interface between the JVM and an in-process profiler agent. The profiler agent is responsible for commu- nication between the JVM and the profiler front end. The profiler agent can register with the JVM to be notified when particular events occur and upon notification can call back into the JVM to obtain additional information. For example a notification may be received when a method is entered (or exited) and a call back may be made to 27Performance Monitoring Tools for Linux, David Gavin Jan, 1998, http://www.linuxjournal.com/article/2396 28Windows Management Instrumentation, http://www.microsoft.com/whdc/system/pnppwr/wmi/default.mspx 29
- 42. Figure 2.6: JVMPIArchitecture obtain the current stack trace at this point. The main advantage of standard interfaces is that they are implemented by different JVM vendors. While the JVMPI was an ex- perimental interface for Java 1.2, it was implemented by most JVM vendors and effec- tively became standard. Thus profilers built using JVMPI are portable across different JVM implementations. A disadvantage of standard interfaces is that they are fixed interfaces and, as such, can only enable predefined types of profiling [135] or event notifications. Another major issue with JVMPI in particular was that when using a JVMPI agent the JVM could not run at full speed and was required to run in a debug- ging mode. As such this profiling approach was not generally suitable for production systems. Another major draw back of the JVMPI approach was that notifications could not be tailored to profile selectively. If, for example, the profiler agent requested to be notified on method entry events, all method entry events would be reported to the agent. This lead to high overhead performance tools. The Java Virtual Machine Tools Interface (JVMTI) 29 will replace JVMPI in Java 1.6 (the JVMPI is currently available in Java 1.5). While at an architectural level the JVMTI looks similar to the JVMPI (i.e. it also consists of call back functions and a profiler agent) it is quite different and im- proves upon many of the limitations of JVMPI. Firstly, it allows for the JVM to run at full speed and does not require it to run in debug mode. It also promotes the use of bytecode instrumentation for many of the event based capabilities of the JVMPI. Using bytecode instrumentation one can be more selective when profiling the appli- cation and can instrument only the parts of the application that require analysis. This avoids the ”all or nothing” approach of the JVMPI and thus reduces the profiler over- head. In fact the JVMTI allows for dynamic bytecode instrumentation 30, which means that the application can be instrumented as it runs. An issue that remains, however, with both JVMPI and JVMTI is that they are native interfaces and while the profil- ing agents (which must be written in native code) are portable across different JVMs 29The Java Virtual Machine Tools Interface, http://java.sun.com/j2se/1.5.0/docs/guide/jvmti/jvmti.html 30Ian Formanek and Gregg Sporar, Dynamic Bytecode Instrumentation A new way to profile Java applications, December 15, 2005, http://www.ddj.com/dept/java/184406433 30
- 43. they are notportable across different platforms. The java.lang.instrument interface 31 is another standard interface (as of Java 1.5) which allows for the interception of the JVM classloading process through a non-native agent. Since the agent is non-native it is portable across different platforms. Java.lang.instrument allows for the agent to monitor the classloading process and to instrument the classes such that they can call back into the agent libraries. Recording performance information for Java applications at the middleware level can also be achieved through standard mechanisms. Java Management Extensions (JMX) technology 32 is a standard technology that allows for the management of Java re- sources through so called MBeans. MBeans, also known as managed beans, are Java objects that are used to represent and manage JMX resources. A JMX resource can be any application, device or Java object. In order to manage an MBean it must be reg- istered with a JMX agent. JMX agents directly control registered MBeans and make them available to remote management applications. While the JMX technology can potentially be used to manage a wide range of different resources it has been heavily used, in particular, to manage the resources of JEE application servers. In fact, ac- cording to the JEE Management specification (Java Service Request 77 33) application servers are required to expose this data through the JMX technology. Profilers built us- ing JMX can collect data on the state of the different system resources (e.g. object pool sizes, thread queues, database connectivity information) and because JMX is standard they are portable across the different application server implementations. Non standard hooks or mechanisms have also been used to collect information at the middleware level. Often middleware vendors provide these mechanisms to enhance the capabilities of their products. For example, IBM provides non standard features for the Websphere application server in the form of the Performance Monitoring In- frastructure (PMI). PMI is available for the Websphere application server and allows for the collection of performance information on the server resources. The informa- tion can be exposed to performance profiling tools through a number of different in- terfaces 34. The main issue with non-standard features, that allow for the collection of performance information, is that they are not portable across different vendors’ im- plementations of the middleware, and thus can result in vendor lock in. Where the information required is not available through standard or non-standard features the middleware itself can be the subject of instrumentation. This can be achieved by manually modifying the source code if it is available [45] [105]. However 31J2SE 5.0 in a Nutshell, Calvin Austin, May 2004,http://java.sun.com/developer/technicalArticles/releases/j2se15, 32Java Management Extensions Technology,http://java.sun.com/javase/technologies/core/mntr- mgmt/javamanagement/ 33Java Service Request 77, J2EE management, http://www.jcp.org/en/jsr/detail?id=77 34Srini Rangaswamy, Ruth Willenborg and Wenjian Qiao, IBM WebSphere De- veloper Technical Journal: Writing a Performance Monitoring Tool Using Web- Sphere Application Server’s Performance Monitoring Infrastructure API, 13 Feb 2002, http://www.ibm.com/developerworks/websphere/techjournal/0202 rangaswamy/rangaswamy.html 31
- 44. as with JVMmodification, instrumenting the middleware requires an understanding of the middleware behaviour which can be extremely complex. Also modification of the application server source code is not a portable approach. Instrumenting at the application level to monitor performance can be achieved in a number of different ways. The most basic way to instrument an application is to mod- ify the source code of the application to log the required information. IBM encourages instrumentation of the application at the time of development such that traces can be recorded through the Application Response Measurement API 35 36. This approach can be cumbersome and can require a major effort during development. Another ma- jor drawback of this approach is that it can not be applied after development if source code is not available. More automated approaches have also been presented that remove the burden of hav- ing to manually modify the source. For example the Gprof profiling tool [79] allows for automatic instrumentation of an application at compile time for applications writ- ten in C or C++. It relies upon specific support from the gcc compiler to insert instru- mentation at the start of each method. Similarly binary rewriters and program trans- formation systems can be used to automate the instrumentation process [22] [134] [155] [170]. It has been argued [135] that such approaches are cumbersome to use since they operate at a low level. Binary rewriters for example require further code to be written while program transformation systems require the creation of complex rewrite rules. Application level instrumentation for Java systems can be achieved using bytecode instrumentation (BCI). BCI allows for instrumentation of the compiled bytecodes and thus access to the application source code is not required. JVMPI and JVMTI allow for the instrumentation of bytecode before load time and at load time. Similarly the java.lang.instrument interface allows for BCI at load time. The JVMTI even allows for BCI to occur at run-time (i.e. dynamic BCI). Although these interfaces provide a mechanism for the modification of bytecodes, the actual instrumentation on the class file is generally performed through a third party BCI library (e.g. [57]). Developers could alternatively make use of Aspect Orient Programming (AOP) libraries 37 38 to instrument their applications. AOP libraries are at a higher level than the BCI libraries and allow for modification of Java classes without a detailed knowledge of the byte- code class structure. It has recently been shown how the AspectJ AOP library can be effectively applied to instrument Java applications for profiling [135]. AspectJ allows for both compile time and load time instrumentation and in fact makes use of BCI libraries to instrument the application. Both BCI and AOP approaches are used by 35IBM InfoCenter, ARM 4.0 API concepts, http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp?topic=/eicaw/section1.htm 36The Open Group, Application Response Management Standard, http://www.opengroup.org/tech/management/arm/ 37AspectJ, http://www.eclipse.org/aspectj/ 38Aspectwerkz, http://aspectwerkz.codehaus.org/ 32
- 45. a number oftoday’s profiling tools to automatically instrument and profile running applications (e.g. [78] 39). 2.8.2.2 Java Profilers Independent of the instrumentation techniques (e.g. event based agents, BCI, AOP), the most commonly used profilers for the Java technology can generally be split into 3 main categories: basic profilers, graphically enhanced J2SE profilers and JEE profilers. Next we discuss each of the different categories. We begin by discussing the most basic profilers and finish with the most sophisticated. The are a large number of freely available (and in many cases open source) basic pro- filers. The profilers in this category collect metrics and present them in a very basic format. An example is the hprof profiler which comes as standard with the sun JVMs 40. While the hprof profiler output looks quite basic, it makes use of the JVMPI/JVMTI interfaces (depending on the tool version) and can collect a vast amount of informa- tion made available through these interfaces. Basic profilers, such as hprof, are most suited for analysing small scale single user applications since the information output is presented in a basic format. In situations where a vast amount of information is pro- duced, as with multi-user applications, basic profilers are generally unsuitable since it can be difficult to manage and understand the information produced. A more sophisticated category of profilers are the (generally commercial) more graph- ically enhanced J2SE profilers. The profilers generally collect the same information as the basic profilers but present the data in a more user friendly manner. Examples include the Jprobe 41 and Optimizeit 42 profilers. These profilers tend to capture infor- mation at the Java class level (very often making use of JVMPI/JVMTI) and provide views that help with performance analysis. For example these profilers generally pro- vide heap analysers that help with memory leak detection and thread analysers for deadlock detection. Most also provide call graphs which show object dependencies. There are a large number of profilers that exist in this category 43. While these tools are effective for single user applications, as with the most basic tools they are limited when profiling larger multi-user enterprise applications. One of the main problems is that they monitor the system at the class level. However, for enterprise systems the number of classes loaded by the JVM can be in the order of thousands. The classes can generally be broken down into the following categories: application level classes (written by the application developers), middleware classes (corresponding to con- tainer services) and lower level Java library classes. A major issue is that, while de- 39JRat the Java Runtime Analysis Toolkit, http://jrat.sourceforge.net/index.html 40Kelly O’Hair, HPROF: A Heap/CPU Profiling Tool in J2SE 5.0, November 18, 2004, http://java.sun.com/developer/technicalArticles/Programming/HPROF.html 41Jprobe, http://www.quest.com/jprobe/ 42Optimizeit, http://info.borland.com/techpubs/optimizeit/ 43Java Performance Tuning Website, http://www.javaperformancetuning.com/resources 33
- 46. velopers are generallyinterested in obtaining information on their own code, it can be very difficult for developers to distinguish their code from lower level middleware and library calls. Another issue related to profiling enterprise applications is that of- ten these applications are distributed and made up of a number of different servers. The (J2SE) profilers in this category do not profile the application across JVMs and thus developers need to spend time correlating data from a number of profiler logs in order to understand the entire system. JEE profilers such as Performasure 44 or JProfiler 45 overcome this issue of muddling middleware and application level calls by presenting data at a higher level of abstrac- tion. Many of these tools also profile across JVM’s and give a transaction centric view of the applications. The JEE profilers are also generally known as Application Perfor- mance Management tools 46, since they are usually suitable for managing systems in a production environment. The biggest problem with the current profiling tools, from the most basic to the most sophisticated is that they do not perform any meaningful analysis on the data collected. The most sophisticated tools merely perform statistical analysis on per- formance metrics and give average resources consumed by the different compo- nents/classes. When profiling large enterprise applications the amount of informa- tion recorded can be truly overwhelming. It can be difficult and time consuming for developers that have to analyse the data produced by these tools in search of partic- ular problems. Furthermore even when developers manage to identify issues often they are unsure as to how to go about solving them. The work in this thesis comple- ments such profiling tools. By performing different analyses the data they produce performance design and deployment antipatterns can be automatically identified. Another problem with the current tools is that they tend to focus on identifying low level performance bugs that exist in the system. For example the more sophisticated tools provide views that help with memory leak detection and deadlock detection. The tools, however, do not focus on analysing the system from a design perspective. Thus design flaws can easily go unnoticed. While identifying and fixing low level bugs in the system will very often improve the system performance, it is common that this is not enough to meet performance requirements. Often, an overhaul of the system design is required to improve performance significantly [43]. 44Performasure, http://www.quest.com/performasure/ 45JProfiler, http://www.ej-technologies.com/products/jprofiler/overview.html 46Jack Shirazi, Application Performance Management, July 2004, http://www.javaperformancetuning.com/articles/apm1b.shtml 34
- 47. 2.9 Reverse Engineering Theterm reverse engineering covers a wide range of activities [163] including design recovery, architecture recovery, feature analysis, impact analysis, clone detection etc. For the purpose of this research we are mainly interested in design and architecture recovery. To analyse the system design, developers can make use of design documentation. However unfortunately it is common that design documents are never created, or are created early in the development process and become out of date as the system evolves. Furthermore, with agile development processes such as extreme program- ming, documentation is not generally maintained throughout the development pro- cess [24]. In such circumstances reverse engineering can be used to extract the design from the system such that a representation of the design is available. Reverse engineering has been defined by Chikofsky and Cross [48] as ”the process of analyzing a subject system to: • Identify the system’s components and their interrelationships and • Create representations of the system in another form or at a higher level of ab- straction.” Earlier approaches to reverse engineering mainly relied on static analysis (e.g. [122] [143] [81] [158]). Static analysis techniques generally extract information from the source code (such as component dependencies) without the need to execute the ap- plication. In fact a static call graph extractor has recently been developed for the EJB technology [165]. There are a number of issues related to applying source code/static analysis to enterprise applications. Firstly, the entire source code of enterprise appli- cations is often not readily available. This can occur as a result of outsourcing or from using third party components. This problem is however overcome in the approach presented in the literature [165], by extracting a static model by analysing the byte- code of the application. Although if components have been developed by third parties security or license issues may not allow for this approach to be applied. Secondly, and more importantly, static models show all possible dependencies that could potentially be present during the system execution. With enterprise applications the number of possible dependencies can be very high and many of the potential dependencies may never actually occur during execution. Thirdly, as claimed by the authors in the liter- ature [149], there is an inherent mismatch between static, code-based structures (such as classes and packages) and the runtime structures that are the essence of most sys- tem descriptions. In fact, the actual runtime structures may not even be known until the program executes (e.g., Dynamic Linked Libraries not under direct control of the implementers may be dynamically loaded). The authors go on to claim that determin- ing the actual runtime architectural configuration of a system using static analysis is, 35
- 48. in general, undecidable. Toovercome this problem dynamic analysis can be applied to extract run-time depen- dencies from the application. The advantages of dynamic analysis for reverse engi- neering purposes has been well documented [146]. For example, dynamic approaches can be more precise, insofar as only dependencies that are actually recorded during execution exist in a dynamic model. Dynamic analysis also limits the scope of investi- gation, similar to program slicing [23]. Dynamic analysis is also particularly useful in relation to analysing the system from a performance perspective, since, performance metrics can be obtained from the running system. This performance related informa- tion can not be obtained easily when analysing the system source code. Dynamic analysis techniques [147] [177] have generally made use of program traces which can be analysed to extract dependencies. A major obstacle for performing this analysis on large distributed enterprise systems is that such traces can be difficult to obtain. Enterprise applications generally comprise of a large number of components running on physically distributed servers. Thus obtaining an execution trace for these systems is not a trivial task. Recently Chen et al. [45] introduced an approach for the collection of run-time paths in JEE systems in the form of the Pinpoint tool. A run-time path [44] contains the ordered sequence of methods called required to service a user request, and thus con- tain information required for dynamic reverse engineering. A major drawback of the Pinpoint approach is that it is intrusive and can only be applied for a particular appli- cation server implementation. In section 4.2.5.2 we discuss the pinpoint tool in more detail and give related work in this area. We also show how this information can be obtained in non-intrusive portable manner such that it can be applied to any JEE application running on different middleware implementations. 2.10 Design Pattern and Antipattern Detection Research in the area of software pattern detection has spanned the past decade. Com- parison of pattern detection methods is often performed with respect to criteria such as flexibility, efficiency and accuracy [139]. However efficiency and flexibility are of- ten unimportant if accuracy can not be guaranteed. Accuracy in relation to software pattern detection is usually defined in terms of precision and recall which are both defined in terms of true/false positives/negatives [139]. True positives tp are defined as pattern instances detected that exist in the system under test, while false positives fp are defined as pattern instances detected that do not actually exist in the system. On the other hand, true negatives tn are defined as pattern instances not detected that do not exist in the system, while false negatives fn are defined as pattern instances not detected that actually do exist in the system. Petterson et al. [139] define precision as 36
- 49. P = tp tp+fp and recallas R = tp tp+fn . Over the past decade a wide number of software pattern detection approaches have been presented and generally use either static analysis or a combination of static and dynamic analysis. The static approaches (e.g. [5] [103] [98] [153] [151] [13]) aim to extract structural information such as class inheritance, interface hierarchies, associ- ations etc. Most approaches analyse the source code of an application [103] [11] [16] [65] [98] [5] [13] [172], although some have used graphical representations as input [56] [27]. The application and design patterns are generally expressed using structural representations, and matching is performed using either meta-programming, graph re-write systems, logic programming, relational algebra, or parsing techniques [138]. According to the literature [138], the matching techniques mainly impact the efficiency and do not greatly effect accuracy. An alternative technique is presented by Buchli and Nierstrasz [39] who use (static) formal concept analysis (FCA) to identify the presence of groups of classes which instantiate a common, repeated pattern. Patterns identi- fied are analysed to find a match with known patterns. Tonella and Antoniol [164] have presented a similar approach based on FCA. The main advantage of approaches based on FCA is that they do not require a knowledge base of design pattern rep- resentations. The combined (static and dynamic) approaches generally make use of dynamic behavioural information to improve the accuracy of static analysis by reduc- ing the levels of false positives [89] [90] [91] [174] [175] [137]. It has been argued that static analysis approaches are limited to the recognition of static parts of design pat- terns, and as a result, the dynamic behavior of patterns is disregarded, which leads to high levels of false positives during recognition [174]. Similarly software metrics have also been utilised to reduce the numbers of false positives [11]. The literature [139] reports on the accuracy of the state of the art in design pattern detection techniques. Similarly, Vok´ac [172] gives a comparison of the accuracy of a number of detection approaches. Efficiency is another important attribute of pattern detection approaches. With pattern detection the size of the exploration space for large software systems can be a major issue. A combinatorial explosion can occur due to the great number of system classes and the multiple roles that classes can play in a specific design pattern. As a result efficiency issues have often been reported in applying detection techniques in practice (e.g. [13] [70] [138]). A number of papers have been presented with particular focus on reducing the detection search space. For example, Niere et al. [123] increase efficiency by replacing a number of constraints with a smaller number of weaker constraints. This approach allows more variants of a particular pattern to be identified during the detection process and increases the number of false positives. They address this issue by attaching a ranking to each pattern candidate retrieved. Tsantalis et al. [167] take an alternative approach and partition the system into clusters of hierarchies. They then apply a similarity scoring algorithm to the smaller subsystems, rather than to the entire system. Petterson et. al [138] present a method to increase efficiency based on 37
- 50. planar graphs. Planargraphs are described as graphs drawable in two-dimensions without any crossing edges. Both the structure of a program and a design pattern can be represented as a graph. The detection of a pattern involves deciding if the pattern graph is isomorphic to a subgraph of the program graph. This is known as the subgraph-isomorphism problem and is NP-Complete for general graphs. Petterson et al. use a filtering strategy to remove unneeded edges from program graphs with the aim of producing planar graphs. Detecting patterns in planar graphs can be solved in linear time. Less work has been undertaken in the area of design antipattern detection. How- ever many of the techniques for software pattern detection outlined above can also be applied to detect design flaws. In most of the approaches discussed above the patterns and application are described in terms of structural (and in some instances behavioural) representations which are analysed to identify instances of the pattern. Design flaws can also be described in this way and thus the same approaches can be used to identify them. For example, work by Gueheneuc et al. [83] shows how defects can be identified by creating abstract models of known design patterns. The models (representing entities and relationships) are used to identify similar sets of entities in the source of the application, which represent distorted design patterns. Similarly, the work presented by Ciupke [49], extracts a model from the source code to iden- tify design issues. The model defines the different entities and relationships that may occur in the design of the program. The information contained in the model is con- verted to Prolog facts. Queries formulated as Prolog rules examine this information and identify design issues. An approach by Moha et al. has suggested the use of formal concept analysis (which has also been applied for pattern detection) and soft- ware metrics, to identify and correct a well known deign antipattern (The Blob) [117]. Metrics have also been applied by Marinsecu [114] for the purpose of design flaw de- tection. Munro [121] has applied metrics for the detection of bad smells. Alternatively, the ARBRIUM antipattern detection tool [63] has been used to detect antipatterns in use case models by making use of the Object Constraint Language (OCL). Antipat- terns can be expressed using OCL and the integrity of the models are assessed. The techniques outlined above, for both pattern and antipattern detection, mainly rely heavily on static source code analysis. Even where dynamic analysis is applied it is generally used only to add behavioural information to a static model of the system such that detection can be more precise. There is a major issue applying such tech- niques to component based systems, since source code is often not available, or at least may not be available for analysis due to security or licensing constraints. Also, when analysing components at the component level we are not interested in their internal structural make up (e.g. inheritance relationships) since components are considered as black boxes and we may not be aware of their internal workings. We are more in- terested in how they communicate with each other and the underlying middleware. However since components can be subject to dynamic binding this information often 38
- 51. can be difficultto obtain statically. True inter-component dependencies can in many cases only be determined at run-time. Furthermore, since we are interested in per- formance antipatterns we need to be able to reason about how the system resources are being used. Such performance information can generally not be obtained by static analysis alone. The goal of this research is to show how performance design flaws can be identified at the component level in enterprise applications using run-time data and component meta-data. The flaws we detect are presented to the application developer with sug- gested refactorings and contextual information. Our aim is to bring a greater level of understanding to developers in relation to such problems than is currently available from today’s enterprise performance tools. 2.11 Knowledge Discovery in Databases and Data Mining When monitoring multi-user enterprise applications, the amount of data produced can be truly overwhelming 47. Currently a major effort is often required by devel- opers who must correlate vast amounts of data in order to identify problems in their systems. Such tasks often require repetitive labourious work and should ideally be automated so developers and system testers can put their minds to more useful tasks. The field of Knowledge Discovery in Databases [66] combines techniques from pat- tern recognition, statistics, machine learning, databases, and visualization to auto- matically extract concepts, concept interrelations, and patterns of interest from large databases. The basic task is to extract knowledge, or information, from lower level data (databases). The basic tools used to extract patterns from data are called Data Mining methods [87], while the process of surrounding the usage of these tools (in- cluding pre-processing, selection, and transformation of the data) and the interpreta- tion of patterns into ”knowledge” is the KDD process. This extracted knowledge is subsequently applied to support human decision-making, e.g., prediction and classi- fication tasks, summarize the contents of databases, or explain observed phenomena. The application of KDD systems alleviates the problem of manually analyzing the large amounts of collected data which decision-makers face presently. KDD systems have been implemented and are currently in use in finance, fraud detection, market data analysis, astronomy, diagnosis, manufacturing, and biology. The KDD process (figure 2.7) is made up of the following sub processes: selection, pre-processing, trans- formation, data mining and interpretation. The proposed antipattern detection approach (see chapter 3) in this thesis can be con- sidered as an instance of the KDD process. The monitoring stage collects data and forwards it on for analysis. The analysis stage implements pre-processing, transfor- 47Michael Baum, ”Migrating J2EE Applications from Development to Production Is it becoming more complex?”, Java Developers Journal, November, 2005, http://jdj.sys-con.com/read/152269.htm 39
- 52. Figure 2.7: TheKDD Process mation, and performs the data mining techniques. The detection stage interprets the discovered patterns and presents the discovered knowledge. Data mining techniques from the areas of frequent pattern mining and clustering are applied in our work. 2.11.1 Frequent Pattern Mining and Clustering The aim of frequent pattern mining methods is generally to find all frequent itemsets, i.e. those that appear in at least a given percentage of all transactions. Agrawal et al. first introduced a solution to this problem in the form of the Apriori algorithm [4]. The frequent item set mining problem is usually formulated as follows: Let I = i1, i2, ..., im be a set of items. Let D be a set of transactions T where each transaction is a set of items such that T ⊆ I. A unique identifier TID is given to each transaction. A transaction T is said to contain X, a set of items in I, if X ⊆ T. We can also say that the itemset X appears in the transaction T. The itemset X is said to be frequent if it appears in at least a given percentage of all transactions. The Apriori algorithm is based on the principle: an itemset can only be frequent if all its subsets are also frequent. Since its introduction by Agrawal et al. [4] Frequent Itemset Mining (FIM) has received much attention. Within the past decade, hundreds of research papers have been published presenting new algorithms or improvements on existing algorithms to solve these mining problems more efficiently [75]. Most of these papers present algorithms which are proposed as being faster than previous existing algorithms. Efficient techniques in the field of data mining are certainly im- portant, considering the recent advances in computing, communication and digital storage technologies which today make it possible to easily store massive volumes of data. However there is a real need to show how such algorithms can be applied to different domains to extract interesting and useful information, since it is irrelevant how efficient an algorithm is, if it can not be put to a meaningful task. In this research, we show how a generalisation of FIM, Frequent Sequence Mining (FSM) [3] can be applied to data collected from monitoring enterprise applications to identify patterns of interest that can be used to identify design flaws in the system. The FIM and FSM problems are discussed in more detail in section 5.4.1. 40
- 53. A second datamining technique, clustering [87], is also applied in this research. Clus- tering is an unsupervised learning problem, whereby data is divided into groups of similar objects. A cluster consists of a group of objects that are similar between them- selves and dissimilar to objects of other groups. In section 5.5 we show how clustering can be used to effectively summarise data produced during monitoring enterprise ap- plications. A comprehensive overview of current clustering techniques in the field of data mining can be found in the literature [28]. 41
- 54. CHAPTER THREE Overview of Approach Inthis chapter we give an overview of the work presented in this thesis, that is, an approach for the automatic detection of performance design and deployment antipat- terns in component based systems. We briefly discuss the three main steps involved in this process, i.e. monitoring, analysis and detection. We outline the research method used, as well as a number of criteria that we have used to evaluate our approach. We also discuss how our approach meets these criteria. Following this, we further dis- cuss antipatterns detailing the advantages of automatically identifying them. In this chapter we also present a hierarchy of antipatterns, from high level abstract antipat- terns, to lower level technology specific ones. We focus in particular on performance antipatterns and show that a high percentage of antipatterns in the JEE technology are performance related. Finally we distinguish between design antipatterns, deployment antipatterns and programming errors. 42
- 55. Main Points • Automaticperformance antipattern detection for component based systems re- quires three main steps: monitoring, analysis and detection. • Our research approach makes use of empirical methods such as experiments and case studies. • A performance antipattern detection approach for component based enterprise systems should meet a number of criteria. Such an approach should be portable, non-intrusive, end-to-end, applicable to distributed systems and produce a low overhead on the system under test. The analysis should be efficient. Detection should be user configurable and produce low numbers of false positives. The above criteria should scale when the approach is applied to real large scale en- terprise applications. • Documented antipatterns conform to an antipattern hierarchy, from technology independent, antipatterns to technology and quality attribute specific antipat- terns. • A high percentage of JEE antipatterns are performance related. • JEE performance antipatterns can be categorised as being either performance programming errors, performance design antipatterns or performance deploy- ment antipatterns. 43
- 56. Figure 3.1: PADTool Overview 3.1 Approach for the Automatic Detection of Performance Antipatterns The approach presented in this thesis, for the automatic detection of performance design and deployment antipatterns, can be divided into 3 main modules (see fig- ure 3.1): a monitoring module, an analysis module and a detection module. Our work is concerned with identifying known antipatterns i.e. antipatterns that have been already documented (e.g. [161] [61]). Our work is not concerned with pat- tern/antipattern mining which is the task of identifying new, previously unknown patterns/antipatterns in a systems 1. The monitoring module is responsible for collecting information on the running sys- tem. Information is collected on the application components, and on the underlying middleware such that a complete picture of the running system can be obtained and recreated. Information collected is forwarded to the analysis module where analysis is performed off-line. The analysis module is responsible for extracting a run-time design model of the sys- tem from the monitoring data. A system’s run-time design can be defined as an in- stantiation (or instantiations) of a system’s design decisions which have been made 1Patterns and Software: Essential Concepts and Terminology, by Brad Appleton, http://www.cmcrossroads.com/bradapp/docs/patterns-intro.html 44
- 57. during the developmentprocess. A run-time design model captures structural and behavioural information from an executing system. It contains information on the components that are executed, as well as the dependencies and patterns of communi- cation between components that occur at run-time. Figure 3.2 gives a run-time design meta model. The meta model describes the different relationships that are captured in a run-time design model. For a typical component based system (e.g. a JEE sys- tem) the run-time design model will contain information such as: dynamic compo- nent dependencies, run-time paths (see section 4.2.2 for a definition), communication patterns and interactions between components and container services. A run-time design model can also be enhanced with related performance metrics (e.g. method memory consumption). The analysis module is responsible for populating the run- time design model by extracting the information from the monitoring module output. For example, during analysis component dependencies are identified and container services reconstructed. Also during analysis data mining algorithms are applied to identify relationships and patterns of interest in the monitoring data (e.g. frequent se- quences). The analysis module is also responsible for summarising the data collected during monitoring. The summarised data can be captured in the model to give a concise overview of the data collected during monitoring. For example, performance characteristics of components can be summarised using statistical analysis and added to the run-time design model. The run-time design meta model is described in more detail in section 5.6. The output from the analysis module is a run-time design model of the system. The extracted design model is analysed by the detection module. The detection module is responsible for identifying any pre-defined antipatterns that exist in the model. An- tipatterns are pre-defined and stored in an antipattern library. The solution can be used to rectify any problems found. This approach has been realised for the JEE technology in the form of the Performance Antipattern Detection (PAD) Tool. Any antipatterns detected by the tool are presented with contextual information and the corresponding antipattern solution. 3.1.1 Research Methodology and Validation Criteria Four main research methods have been outlined for software engineering [74], i.e., the scientific method, the engineering method, the empirical method and the analytical method. The methods first presented by Basili [20] in the software engineering context have been described by Wohlin et al. [178] as follows: • ”The scientific method: The world is observed and a model is built based on the observation, for example, a simulation model. • The engineering method: The current solutions are studied and changes are pro- posed, and then evaluated. 45
- 58. Figure 3.2: Run-timeDesign Meta Model • The empirical method: A model is proposed and evaluated through empirical studies, for example, case studies or experiments. • The analytical method: A formal theory is proposed and then compared with the empirical observations.” Traditionally software engineering research has made use of the analytical methods [74], the engineering method has been mainly used in industry and the scientific method has been used mostly in applied areas such as simulating telecommunica- tion networks [178]. More recently, however there has been a call for more empirical research in the field of software engineering [162] [74] [144] [100]. A number of authors have suggested that rigorous experimentation is required to evaluate new technologies and their effects on organisations, processes and products [100]. Wohlin et al. [178] suggest three main empirical strategies that can be carried out, i.e. experiments, case studies, and surveys. Experiments are often referred to as ”research in the small” [100] since they are usually run in a laboratory environment and are generally small in scale. The aim of an experiment is to manipulate one or more variables while controlling the remaining variables at fixed levels. The effect of manipulation is measured. Experiments tend to be small since they require a high level of control over the test environment. Case studies on the other hand are referred to as ”research in the typical” [100] since they tend to look at what is happening in a 46
- 59. typical project. Thelevel of control is generally lower in a case study than in an experi- ment. The objective is often to monitor a particular project or activity. Case studies are said to be observational studies whereas experiments are controlled studies. Finally surveys are often referred to as ”research in the large” [100] since they try to capture what is happening over a range of projects. They are usually performed by obtaining information from a representative sample of the entire population through interviews or questionnaires. The results are generalised to the population from which the sam- ple was taken. In this thesis we make use of both case studies and experiments to evaluate our re- search. Experiments have been used to evaluate the overhead associated with our monitoring approach (see section 7.2.3) as well as to assess the efficiency of our anal- ysis module (see section 7.3.1). According to Wohlin et al. [178] the main strength of an experiment is ”that it can investigate in which situations the claims are true and they can provide a context in which certain standards, methods and tools are recom- mended for use”. Case studies on the other hand have been used to show how our monitoring and analysis outputs can be used to manually identify antipatterns (see sections 7.2.1 and 7.3.2 respectively). Case studies have also been used to show how antipatterns can be automatically identified in component based systems (see section 7.4). The main advantage of using case studies is that they incorporate qualities that an experiment can not such as scale, complexity, unpredictability and dynamism [178]. Below we have outlined a number of evaluation criteria which we evaluate our work against. The evaluation criteria are the attributes which we believe are required, in order for our approach to be considered useful. The evaluation criteria are as follows: • Portable: The monitoring module should be portable across different vendors’ implementations of the middleware. This is particularly important for JEE, whereby application servers are built by a large number of different vendors 2. In fact it is not unusual for an enterprise application to be made up of web, application or database servers from a number of different software vendors. To be able to apply our approach to truly heterogenous systems it is required that the monitoring module is completely portable across the different implementa- tions. • Non-intrusive: The monitoring module should also be non-intrusive, that is, it should not require any modifications to the application or middleware source code. This is important for component based systems, since often it may not be possible to make such modifications. For example most commercial applica- tion servers are proprietary and access to the source code is unavailable. Also 2Application server product vendors, http://www.service-architecture.com/products/application servers.html 47
- 60. even with opensource middleware 3 4 where the source code is available it may be too time consuming for developers to gain an understanding of application server behaviour, such that modifications can be made. Similarly, modifications to the application components can be time consuming and cumbersome. Also with component based systems the application components may be from third parties (e.g. component vendors 5) and the source code may but unavailable or unmodifiable due to licensing constraints. • Low overhead: It is important that the impact on the running system under test is kept to a minimum. Performance tools that severely impact the system performance can not be run with realistic loads and thus can not obtain a realistic picture of the system as it is intended to run in a production environment. • End-to-end: Performance problems can occur in any of the different server side tiers that make up an enterprise application. In fact often they occur as a result of communication between the different component tiers. Thus it is important to analyse the end-to-end design of enterprise applications and not just to con- centrate on tuning particular components (e.g. the database). • Applicable to distributed systems: It is common that today’s enterprise appli- cations are physically distributed across a number of different machines. The approach we outline above should be applicable for such systems without the need for a developer to correlate a wide number of different log files from the different machines. • Efficient: Although, the analysis and detection is performed off-line, the per- formance of these modules should not exceed an acceptable level, which might inconvenience users. For example analysis that takes a very long time (hours/days) would not be convenient for developers working to tight dead- lines. • Low number of false positives: The number of false positives needs to be kept to a minimum. False negatives should also ideally be low, however false neg- atives can be difficult to measure since antipatterns may exist in a system that we are unaware of. This is especially the case when testing real systems where documentation may be incomplete or unavailable. • User configurable and extensible: Users need to be able to add their own mon- itoring information and antipattern descriptions since certain antipatterns may be either application or domain specific. Users should also be able to configure the antipattern detection process such that unimportant potential antipattern detections are filtered out. 3JBoss Application Server, Red Hat,http://www.jboss.org 4Jonas Application Server, Objectweb, http://wiki.jonas.objectweb.org/xwiki/bin/view/Main/WebHome 5WebCab Components, http://www.webcabcomponents.com/ 48
- 61. • Scalable: Theabove quality attributes should all scale when the tool is applied to large systems and to systems under load. Our approach, which has been implemented in the form of the PAD tool, meets the requirements outlined above. Next we discuss how this is achieved by giving a more detailed overview of the monitoring, analysis and detection modules. 3.1.2 Monitoring The monitoring module (chapter 4) is responsible for collecting run-time information on the different components that make up an enterprise application such that a de- tailed representation of the system can be recreated and potential antipatterns iden- tified. The monitoring approaches allow for (a) identification of component relation- ships, (b) identification of communication patterns between components, (c) tracking of objects (and how they are used) across components, (d) the collection of component performance metrics, (e) the collection of component meta data (e.g. container services used, bean type) and (f) the collection of information on server resources. Using this information we can extract the required relationships to reconstruct the system design for performance antipattern detection. The monitoring module consists of three main components, i.e. a run-time path tracer (section 4.2), a MEJB server resource moni- tor (section 4.3.1) and a meta-data extractor (section 4.3.2). The monitoring module is shown to be end-to-end (section 4.2.5.1), portable across different JEE implementa- tions (see section 7.2.2) and non-intrusive (see section 4.2.5.2). We also discuss how the monitoring can be applied across physically distributed systems (section 4.2.5.3). The monitoring module output is forwarded to the PAD analysis module for auto- matic design reconstruction. Section 7.2.1, however, shows how this output can be used to manually reconstruct run-time design models of the system, and how using these design models, antipatterns can be easily identified by a developer. Finally, re- sults from performance tests show that the monitoring approach results in a very low overhead on the system being monitoring (see section 7.2.3). 3.1.3 Analysis The analysis module (chapter 5) is responsible for extracting a run-time design model of the system and does so by performs the following actions on the monitoring data: (a) it automatically extracts the component relationships and component object usage patterns (section 5.2), (b) it automatically reconstructs the run-time container service boundaries (section 5.3), (c) it automatically identifies component communications patterns (section 5.4) and (d) it applies data reduction techniques (section 5.5). In par- ticular, in this chapter, we show how we apply the data mining technique, frequent sequence mining (FSM) to automatically identify interesting component communica- 49
- 62. tion patterns inthe data collected. Specifically we show how FSM can be applied in an efficient manner to run-time paths (section 7.3.1). We also show how, by modifying the FSM algorithm, the algorithm output can be made more human readable (section 5.4.2). In fact in section 7.3.2 we show how the output from the FSM algorithm can be used to manually identify design flaws in JEE applications. The FSM algorithm takes a user configurable input (known as the support value) which determines the threshold for the patterns it identifies. The support value can be used to filter out less interesting patterns in the data. Finally the analysis module is also responsible for summarising the monitoring data using clustering and statistical analysis techniques (see data reduction section 5.5). This data reduction allows for the design extraction techniques to be scalable when large amounts of data are collected by the monitoring module (e.g. when the system is under load). 3.1.4 Detection The extracted design model is used as input into the detection module (chapter 6). Our detection mechanism is based on a rule engine approach. The design model is con- verted to rule engine specific facts that can be loaded into the rule engine. Rules can be input into the rule engine which describe the antipatterns that we want to detect in the model. The rule-engine makes use of forward-chaining rules which are in the form of ”if-then” statements. Rules are specified so that the rule’s conditions verify the occurrence of a certain antipattern. Subsequently, when existing facts match a rule’s condition clauses, the rule action is fired indicating that the antipattern was detected (see section 6.2). Our predefined rules make use of configuration files which contain user specified thresholds (section 6.3.2). User defined thresholds can be utilised to filter out antipatterns that developers feel are too insignificant to address. The PAD tool also contains a rule library for JEE performance design and deployment antipat- terns (section 6.3.1) which users can extend by adding their own antipatterns rule de- scriptions. The PAD tool has been applied to a sample and real large scale enterprise application. Our results (see section 7.4) show how the approach scales when applied to a real enterprise system and that the number of false positives remains low. 3.2 Antipatterns Revisited While the concept of software antipatterns is generally understood by developers, the individual antipattern instances are not as widely known. As documented in the lit- erature [36], antipatterns consistently appear in software, especially in large complex (enterprise) applications [161]. The work in this thesis outlines an approach for the automatic detection of these well documented issues that consistently occur in enter- prise applications. 50
- 63. The benefits ofhighlighting antipatterns are manifold. Firstly, software antipatterns document bad practices in software development as well as their corresponding so- lutions. Thus, detecting instances of antipatterns and highlighting them to the devel- oper, not only allows the developer to understand what is wrong with the application, but it also allows the developer to easily rectify the situation by refactoring the applica- tion with the solution provided. Secondly, antipatterns provide a high-level language for discussing design issues. Developers can use antipattern names to clearly and ef- ficiently discuss within the development team particular implementation issues that may arise. Also, exposing developers to performance related antipatterns helps them form a sense of performance intuition, since the underlying reason as to why a partic- ular antipattern had an adverse effect on performance is underlined in the antipattern description. Consequently developers are less likely to make the same mistake in the future. In the following subsection we discuss a number of different antipattern categories and give a hierarchy of antipatterns that shows how the different categories that we are interested in are related. We also show that a high percentage of enterprise an- tipatterns tend to be concerned with performance issues. Finally in this sub section we differentiate between design antipatterns, deployment antipatterns and program- ming errors. 3.2.1 Antipattern Hierarchy Figure 3.3 gives an antipattern hierarchy diagram. At the top of the diagram we have high level technology independent software antipatterns. Brown et al. [36] introduced a number of such antipatterns concerned with a range of software quality attributes (such as reusability, maintainability, performance). More recently Smith and Williams [154] introduced general performance antipatterns which solely focus on performance concerns (i.e. level 2). The performance antipatterns presented in the literature [154] are high level and language-independent antipatterns. They describe situations where a sub-optimal performance design choice has been made. Instances of the antipatterns documented by Smith and Williams, however, occur throughout different technolo- gies. Many of these problems are especially common in enterprise technologies where performance is often a major concern (level 3). JEE antipatterns have been presented in [161] and [61]. The literature [161] concentrates on EJB antipatterns, while [61] lists antipatterns concerned with a number of aspects of the JEE technology (i.e. Servlets, JSP, EJB and Web Services). We have analysed the antipatterns from both sources. From a total of 43 antipatterns documented in [161] we have identified 34 (79%) of them to be performance related antipatterns (since according to the authors they have an impact on system performance). From a total of 52 antipatterns documented in the literature [61] we identified 28 performance antipatterns (54%). The high propor- tion of antipatterns from [161] and [61], that are related to performance, is further 51
- 64. Figure 3.3: Hierarchyof Antipatterns evidence that performance is an important software quality attribute for enterprise systems, and that poor performance design is common in such systems. We further divided all JEE performance antipatterns identified into 3 different cat- egories (level 4): (a) Performance programming errors, (b) performance design an- tipatterns and (c) performance deployment antipatterns. Performance programming errors (a) can be defined as common mistakes made by developers that result in de- graded system performance. They yield no design trade-off and always have a neg- ative impact on performance. Examples include memory leaks, deadlocks and im- proper cleanup of resources such as database connections. Generally developers are unaware of the presence of performance programming errors in the system. The Rot- ting Session Garbage antipattern [161] is an example of a performance programming error that is often made by developers using the EJB technology. This antipattern occurs when a client fails to explicitly remove a stateful session bean when finished using it. The orphaned bean will continue to live in the application server using up system resources until it times out. Until then, the EJB container must manage the 52
- 65. bean, which caninvolve the relatively expensive job of passivating the bean to make room for other active beans. In many situations fixing programming errors alone will not improve the overall system performance such that requirements are met. Often it is the case that the system design requires modification. Performance design (b) and deployment (c) antipatterns can be defined as instances of sub-optimal design or sub- optimal deployment settings that exist within the application i.e. situations where an inefficient design choice has been taken. In such situations an alternative more effi- cient design choice exists. Developers are often aware of having made these choices, but can be unaware of their consequences. Performance design and deployment an- tipatterns can be used to identify and resolve these situations since they document both the sub-optimal design (or deployment setting) and its corresponding optimal solution. We are interested in both design and deployment antipatterns since, with component based frameworks such as JEE, many decisions that were inherent in the design and coding of applications in the past, have been abstracted out into the deployment set- tings of the application. With EJB for example the granularity and type of transactions can be specified in the XML deployment descriptors of the application. As a result, when making deployment time configuration settings, different design trade-offs that can significantly impact performance must be considered. In section 6.1 we give the categories of performance design and deployment antipat- terns that our PAD tool can currently detect. In this section we categorise the antipat- terns further into groups related to the data required to detect them. 53
- 66. CHAPTER FOUR Monitoring Required for AntipatternDetection This chapter outlines the various monitoring components that are required to capture the information used for antipattern detection. The monitoring module is responsible for collecting information on the system under test such that a detailed representa- tion of the system can be recreated and potential antipatterns identified. To identify performance design and deployment antipatterns in component-based enterprise ap- plications we need to be able to identify (1) the components (and their types) that make up the system, (2) how the different components interact with each other and (3) how the components make use of the different middleware and system resources. The monitoring stage of the antipattern detection process obtains information on (a) component relationships, (b) component communication patterns, (c) component re- source usage, (d) component object usage, (e) component meta-data and (f) server resource usage. Using this information we can extract the required relationships to reconstruct the system design for performance antipattern detection. The monitor- ing module consists of three main components, a run-time call path tracer, a server resource monitor and a meta-data collector. In this chapter we detail the different techniques required to capture this information in a portable, low-overhead and non- intrusive manner. Our monitoring approaches are applied to a running application and do not require any analysis of the source code. The final section of this chapter outlines related work. 54
- 67. Main Points • Arun-time path is a data structure for the representation of run-time calling be- haviour. It contains control flow, resource usage and performance characteristics associated with servicing a user request. • Run-time paths can be applied for a number of different purposes, including reverse engineering, antipattern detection, problem determination, workload modelling, hotspot identification and adaptive monitoring. Thus it is impor- tant that there are mechanisms and tools that allow for the recording of run-time paths. • There are four main requirements for run-time path tracing of component based systems: request identification, request tagging, distributed request tracking and a request interception or monitoring mechanism. • Run-time path tracing requirements have been met in a non-intrusive, portable manner for JEE in the COMPAS JEEM tool. COMPAS JEEM takes advantage of a number of JEE standard mechanisms, and leverages and extends the capabilities of a number of open source projects to achieve this aim. The run-time path trac- ing approach is portable across the JEE technology and across other component technologies. • A solution for distributed run-time path tracing is presented for the JEE technol- ogy. • The state of server resources can be monitored through the Java Management Extensions interface • Functional and structural information on the JEE components can be automati- cally extracted from the component meta-data. This avoids the need for source code analysis. Information on how the component interacts with the different middleware services can also be obtained. 55
- 68. 4.1 Chapter Overview Themonitoring stage of the antipattern detection process obtains information on (a) component relationships, (b) component communication patterns, (c) component re- source usage, (d) component object usage, (e) component meta-data and (f) server resource usage. Using this information we can extract the required relationships to reconstruct the systems run-time design for performance antipattern detection (see figure 3.2 on page 46). The monitoring module consists of three main components, a run-time path tracer, a server resource monitor and an meta-data extractor. Next we detail the different techniques required to capture this information in a portable, low-overhead and non-intrusive manner. Our monitoring approaches are applied to a running application and do not require any analysis of the source code. The run-time path tracer collects run-time paths which contain the information (a) (b) (c) and (d) outlined above. More specifically a run-time path [44] contains the control flow (i.e. the ordered sequence of methods called required to service a user request), resources and performance characteristics associated with servicing a request. Run- time paths are described in more detail in section 4.2.1. By analysing these paths one can easily see how system resources are being used, how the different components in the system interact and how user requests traverse through the different tiers that make up the system. As part of this work a run-time path tracing tool has been de- veloped, called COMPAS Java End to End Monitoring (JEEM). COMPAS JEEM mon- itors the paths user requests take when traversing through the different tiers in the system. Resource usage and performance characteristics of each request can also be obtained using our approach. It builds upon the open source COMPAS monitoring framework 1 [120] [118] and leverages and extends the capabilities of a number of other open source projects2 [45]. In this chapter we discuss how run-time path trac- ing has been achieved in a non-intrusive, low-overhead and portable manner. Section 4.2.2 and 4.2.3 gives our motivation and the considerations that should be undertaken (independent of the underlying technology) for non-intrusive run-time path tracing. Section 4.2.4 presents the core instrumentation process and the extension capabilities available in the COMPAS monitoring framework. The end-to-end run-time path trac- ing approach leverages these extension mechanisms to build upon the COMPAS mon- itoring and instrumentation infrastructure. In section 4.2.5 we describe the extensions made to the COMPAS framework and the overall architecture of COMPAS JEEM. In this section we also detail the open source projects that we have extended and inte- grated with COMPAS to achieve our aims. In section 4.2.5.3 we detail how run-time path tracing can be performed in a truly distributed manner. The idea of Compo- nent Object Tracking is introduced in section 4.2.5.8 and we discuss how this can be achieved through dynamic bytecode instrumentation and the COMPAS ByteCode In- 1COMPAS, http://compas.sourceforge.net/ 2P6Spy, http://www.p6spy.com/ 56
- 69. strumentation (BCI) tool.The portability of the run-time path tracing approach across different component technologies is discussed in section 4.2.5.9. Section 4.3 details the remaining two components of the monitoring module i.e. the server resource monitor and the component meta-data extractor. Firstly in subsection 4.3.1 we detail how our server resource monitor obtains information on the server resources at run-time through a Management EJB monitoring component. In this sec- tion we discuss how this is achieved in a non-intrusive and portable manner through the Java Management Extension interface. We also discuss the information obtained that is required for the run-time design model (i.e. (f) server resource information) and why this approach induces very little overhead on the running system. In subsec- tion (4.3.2) we outline how information on the different components can be extracted from the component meta-data (e). The component meta-data contains information on how different components make use of the container services as well as structural and functional information which is added to the run-time design model. This infor- mation can be obtained by analysing the component deployment descriptors. This task is performed off-line and is automated using an XML parser. Finally in section 4.4 we compare our monitoring approach to related work. In partic- ular we detail: alternative applications of run-time paths, related data structures for the representation of dynamic calling behaviour and related tracing tools and tech- niques that have been previously presented. 4.2 Run-Time Path Tracing 4.2.1 Run-Time Paths Overview A run-time path is a data structure for the representation of run-time calling be- haviour. A run-time path has been previously defined by Chen et al. [44] as containing ”the control flow, resources and performance characteristics associated with servicing a user request”. The authors also state that run-time paths are recorded at run-time by tracing requests through a live system, and that they span the system’s layers to di- rectly access component and resource dependencies. Next we describe run-time paths in more detail and explain how they related to other representations of run-time call- ing behaviour. We also show a run-time path data structure and explain the various fields that are contained with it. A dynamic call trace [94] (see figure 4.1 (a)) is a recording of all calls and returns that occur in a program’s execution. Dynamic call traces are unbounded. Their size is proportional to the number of calls in an execution [8]. Figure 4.1 (b) shows a tree representation of a dynamic call trace, often referred to as a dynamic call tree [8] [94]. Traversing the tree in preorder will give the ordered sequence of calls and returns as they occurred during execution. Dynamic call traces/trees capture important infor- 57
- 70. Figure 4.1: DynamicCall Trace (a) and Corresponding Dynamic Call Tree (b) mation for system comprehension such as, the sequence of calls, the context in which a call was made and repeating calls or loops. An issue with dynamic call traces how- ever is that they are unbounded and are space inefficient. More compact represen- tations, such as dynamic call graphs and calling context trees, have been previously presented to reduce the space overhead [8] [94] [79] however these representations do not capture important information that is contained in the dynamic call trace. A more detailed comparison of these representations is given in section 4.4. Run-time paths are similar to dynamic call traces. In fact they contain the same information as a dynamic call trace i.e. they contain the ordered sequence of all calls and returns that occur during execution. However they also contain related resource and and perfor- mance information for each call. In fact they can contain further information such as the type of call made or the arguments passed in the call for example. Furthermore run-time paths are generally associated with multi-user/multi-client systems. They group together, into a single path, the sequence of calls that correspond to a given user/client request. As such, run-time paths isolate the calls that occur in response to a given request. This is particularly important in multi-user/multi-client applica- tions whereby many simultaneous requests may be concurrently serviced. Chen et al. [44] have described this as providing ”a vertical slice of the system from a request’s perspective.” Finally, since they are associated with multi-user applications, run-time paths often span a number of system layers. These layers can often be physically dis- 58
- 71. Figure 4.2: ExampleRun-Time Path tributed across multiple hardware components. An example run-time path is given in figure 4.2. The diagram shows a representation of a run-time path in a tree like structure. An indent from one level to the next means that the call above the indent is a parent of the indented call below, and any other calls below with the same level of indentation, until the parent call has returned. Figure 4.2 shows a sequence of calls that were invoked to service a user request in a typical JEE application. It contains further information such as the type of call (e.g. a web tier call, a business tier call or a database call) and performance metrics. Next we describe the run-time path data structure. A run-time path data structure (figure 4.3) essentially contains a path ID and a path node. The path ID is a unique identifier that distinguishes the run-time path from other paths in the system. A path node data structure (figure 4.4) represents a call in the path. It contains an object that holds information on the call details (e.g. component name, method name, arguments passed, performance metrics) and an ordered list of path nodes (callees). By traversing the path structure in a preorder fashion we obtain the ordered sequence of calls as they executed in the application. Both figure 4.3 and 4.4 contain only the essential fields that are required in these data structures. Typically a run-time path data structure can contain a number of other useful fields and/or methods. For example it may contain a component quick list which gives a list of all components that make up the path. Also it may contain useful methods for access and creation. Similarly a path node data structure may contain a number of other useful fields (e.g. call type) and methods. 59
- 72. class CallPath { StringpathID; PathNode Root; ... ... ... //other useful fields or methods } Figure 4.3: Run-Time Path Data Structure class PathNode { Map Details; List Callees =new ArrayList(); ... ... //other useful fields or methods } Figure 4.4: A Run-Time Path’s PathNode Data Structure 4.2.2 Run-Time Path Tracing Motivation The analysis module presented in chapter 5 makes use of run-time paths to automat- ically extract information required for antipattern detection. Below are some alterna- tive uses that run-time paths can be put to. A run-time path shows how a user request is serviced, giving the ordered sequence of events/calls and related performance and resource usage information. Thus run-time path analysis can help greatly with system comprehension and can be applied for a number of different purposes in this area. System comprehension can be a major issue for developers of large enterprise applications. A lack of a system wide understand- ing can lead to badly designed systems which perform poorly and can be difficult to debug and maintain. In section 7.2.1 we show how run-time paths can be used to reverse engineer and quickly deduce the overall system structure of enterprise appli- cations. Analysing run-time paths to deduce system structure is advantageous for a number of reasons. Firstly analysing application source code can be difficult since a large number of files can be involved. Even when using code comprehension tools 3 [77] determining the chain and order of method calls by stepping through source code can be a complex and time consuming task. In addition, some paths might be very difficult to determine from static analysis, in particular in systems which con- tain workflow engines that select execution targets based on run-time conditions. In comparison COMPAS JEEM presents all calls from the main software components that make up a user request in a single run-time path. It maintains the order of calls 3Juliet Java code comprehension tool, http://infotectonica.com/ 60
- 73. and the callsequence can be easily observed by traversing the run-time path tree-like structure (see figure 4.2). Another advantage of using run-time paths, for system com- prehension, is that they can be well represented in a diagrammatic format (see figure 4.12). The diagram construction process can be easily automated reducing the effort required on the part of the developer further [97]. As shown in section 7.2.1, such dia- grams are useful for identifying performance design issues or antipatterns that might exist in enterprise applications. Run-time paths have also been put to other uses such as fault detection and diagnosis, workload modelling, and adaptive monitoring. In section 4.4 we will discuss this related work in more detail. 4.2.3 Run-Time Path Tracing Considerations Today’s enterprise applications are generally required to handle high loads of concur- rent users. Run-time path tracing in such systems involves tracing each of these user requests as they pass through the different software tiers that make up the entire ap- plication. To achieve this user requests must be identifiable across the entire request and the order of the calls that make up the requests needs to be maintained. A system needs to meet the following requirements to be able to perform run-time path tracing: R1 the system must be able to identify new user requests entering the system R2 the system is required to tag the request with request specific information (RSI) so that calls can be mapped to the originating requests R3 the system requires the ability to piggy back the RSI with the request, such that the request can be tracked across the entire system R4 a monitoring framework that provides for request interception is required Next we detail how run-time path tracing can be achieved through the above require- ments. The user request identification mechanism (R1) is required to determine when a new request enters the system such that it can be (R2) tagged with request specific information (RSI). The RSI contains a unique ID so that the request can be identified and distinguished from other concurrent requests in the system. The RSI also con- tains a sequence number that is used to order the calls that make up the request. The sequence number is incremented (by the tracer’s run-time path ordering logic) every time a component method is called and is reset upon a new user request. User requests also need to be tracked (R3), so that, as the request passes through the different software components that make up the system, it can be identified and the order of its calls maintained. In order to be able to track the request the RSI must be 61
- 74. attached to it,in such a way as to be available at every call point along the request. This can be particularly challenging when a system is distributed across a network. A monitoring framework (R4) is required to intercept and log calls made to the soft- ware components that make up the request. The RSI, along with component details are logged upon a method invocation. The component details describe the compo- nent that is called, giving information such as what component method was invoked, what arguments were passed etc. The logged information can be subsequently used to construct the run-time paths offline. The reconstruction process makes use of the RSI data to (a) determine what calls make up each run-time path and (b) to order the calls. Thus far, the above requirements have generally been met in a manner that involves instrumenting either the software application itself or the underlying middleware [82] [45]. Manual instrumentation of the application source is undesirable since instru- menting the code can be time consuming and cumbersome and the effort must be repeated for each new system under development. Since the JEE specification does not explicitly address the requirements for run-time path tracing, middleware layer approaches for JEE have involved using non-standard mechanisms provided by spe- cific vendors that are not portable across the technology and that often require the source code of the server to be available (such that it can be recompiled). For example the JBoss group recognised the need for R4 by providing its interceptor-based com- ponent architecture [105] which allows for the interception of calls to business tier JEE components. IBM has also recently addressed the need for R3 through its Work Area Service [102]. The biggest issue with using non-standard mechanisms, however, is that developers get locked in to using a particular middleware implementation and lose the flexibility of being able to quickly change their underlying middleware imple- mentation should the need arise. Consequently, such approaches can not be applied to systems that are made up of application servers from numerous different vendors. In the following sections we present an approach that, through the COMPAS monitor- ing framework, takes advantage of a number of JEE standard mechanisms to meet the requirements outlined above for JEE systems in a non-intrusive and portable manner. While our implementation focuses on the JEE technology, only the framework instru- mentation process and request tracking are JEE specific, the remainder of the work presented below can be applied to other component technologies (e.g. CCM). This is discussed in detail in section 4.2.5.9. 4.2.4 COMPAS Monitoring Framework COMPAS [118] is intended as a foundation for building enterprise-level performance management solutions for component-based applications. It was designed to pro- vide a common monitoring platform across different application server implementa- 62
- 75. tions. This isachieved by leveraging the underlying properties of component-based platforms in order to enable non-intrusive instrumentation of enterprise applications. Two main techniques are used to minimise the overhead of the monitoring infrastruc- ture: asynchronous communication and adaptive activation. The former is employed in the entire infrastructure by the use of an event-based architecture with robust mes- sage handling entities that prevent the occurrence of locks in the target application. The latter technique uses execution models captured from the target application to drive the activation and deactivation of the monitoring probes. By appropriately min- imising the number of active probes (i.e. through the alert management and adaptive monitoring process), the total overhead is reduced while preserving complete target coverage (see [119] [118]). As the COMPAS infrastructure is designed to be used as a foundation for performance management tools, its design is highly extensible and based on decoupled communication mechanisms. The most important functional entity of the monitoring infrastructure is the monitor- ing probe. The probe is conceptually a proxy element with a 1 to 1 relationship with its target component. In JEE, target components are JSPs, Servlets, EJBs and EIS com- ponents deployed in a target application. The probe is implemented as a proxy layer surrounding the target component with the purpose of intercepting all method invo- cations and lifecycle events. The process of augmenting a target component with the proxy layer is referred to as probe insertion. This section focuses on the core COMPAS capabilities, initially limited to EJB components only, whereas the following sections present the extensions that allow COMPAS to instrument and monitor the remaining JEE target components, grouped under the name COMPAS JEEM. 4.2.4.1 Portable EJB Instrumentation COMPAS uses component meta-data to derive the internal structure of the target en- tities. The component meta-data is placed in deployment descriptors that contain structural as well as behavioral information about the encompassing EJBs. By lever- aging this data, it is possible to obtain the internal class-structure of each component, which is needed for instrumentation. The instrumentation is performed by a ”proxy layer” attached to the target application through a process called COMPAS Probe In- sertion (CPI). As all the information needed for probe insertion is obtained from the meta-data, there is no need for source code or proprietary application server hooks. Therefore, the effect on the target environment is minimal and user intervention in the probe insertion process is not required. COMPAS is in this respect non-intrusive, as it does not require changes to the application code or to the runtime environment as the majority of other approaches do. The CPI process examines the Target Application’s structure and uses component meta-data to generate the proxy layer. For each component in the target application, a monitoring probe is specifically generated based on the component’s functional and 63
- 76. structural properties. TheCPI process leverages deployment properties of component frameworks to discover and analyse target applications. Therefore, CPI is conceptu- ally portable across component frameworks such as EJB, .NET or CCM. Indeed, given a component system written for any such platforms, COMPAS would be able, with minimal modifications, to insert monitoring probes that match each of the application components. For components written for the EJB platform, the following meta-data is extracted and used to generate a monitoring probe for a target component (TC): • Component Name (bean name, for EJB) • Component Interface (Java interface implementing the services exposed to clients, for EJB) • Locality (local or remote, for EJB) • Component Type (stateless session, stateful session, entity or message-driven, for EJB) • Component Interface Methods (Java methods in the business interface, for EJB) • Component Creation Methods (ejbCreate() methods, for EJB) COMPAS provides a Probe Code Template (PCT), modifiable by the user, which rep- resents a template for the generated monitoring probes. It consists of extensible logic for initiating event-handling operations and placeholders for component-specific in- formation. Using the PCT and the extracted meta-data, the CPI process generates one probe for each Target Application Component. The placeholders in the template are replaced with the values extracted from the meta-data. The proxy layer is an instanti- ation of the PCT, using the TC meta-data values. The proxy layer (probe) is a thin layer of indirection directly attached to the TC. To fulfill its instrumentation functionality, the Probe employs the Instrumentation Layer that has the capability of processing the data captured by the Probe and performing such operations as event notifications. The Instrumentation Layer uses the COMPAS Probe Libraries for implementing most of its logic. A Modified Component (MC) re- sults after the CPI process has been applied to a TC, and this will enclose the original TC. In addition, it will contain the Probe and Instrumentation Layer artifacts. In order to ensure a seamless transition from the TC to the MC, the CPI transfers the TC meta- data to the MC. The MC meta-data will only be updated so as to ensure the proper functionality of the proxy layer (the bean class property must be updated to indicate the insertion of the Probe class). The CPI process is illustrated in figure 4.5. At run-time, probes collect and analyse performance and lifecycle events from the tar- get components and communicate with the central monitoring authority, the COM- PAS Monitoring Dispatcher. The Monitoring Dispatcher is the client-side entity re- 64
- 77. MC Probe TCPCT Probe Instrumentation Layer COMPAS ProbeLibraries TC + = (CPI) COMPAS MonitoringDispatcher network Figure 4.5: COMPAS Probe Insertion Process sponsible for mediating client access to the COMPAS probes by providing an abstrac- tion layer over the lower-level communication and control operations. It contains handlers for efficient processing and transformation of probe notifications into COM- PAS events. In addition, the Monitoring Dispatcher provides a control interface that allows transparent relaying of commands to the monitoring probes. The central role of the Monitoring Dispatcher is client-side data processing as illustrated in figure 4.6. 4.2.4.2 COMPAS Extension Points COMPAS Monitoring contains an instrumentation core and a set of extensions for coordinating and handling instrumentation events. The extensions are built upon the pluggable architecture of the instrumentation core by leveraging the COMPAS Frame- work Extension Points (FEPs) based on loosely coupled asynchronous communica- tion. Possible extensions that can be added to COMPAS include support for low-level instrumentation sources such as virtual machine profiling data, as well as high-level functional extensions such as elaborate data processing capabilities for complex anal- ysis of the monitoring data. Complex decision policies for improving the alert man- agement and adaptive monitoring process can also be implemented as extensions. Since the COMPAS infrastructure spans all the JEE server-side tiers and the client tier, it provides two types of framework extension points: 65
- 78. Figure 4.6: COMPASJEEM Architecture • Server-Side FEP: facilitates the extension of the functionality available to or pro- vided by a COMPAS Monitoring Probe. In addition, it facilitates the creation of new probe types. The majority of extensions introduced by JEEM use server- side FEPs. In particular, JEEM makes use of Servlet filter probes and JDBC driver probes to feed information into the COMPAS infrastructure. Apart from these probes in the web and EIS tiers, JEEM also enhances existing EJB probes in or- der to generate appropriate run-time path extraction information. Other exam- ples of server-side FEPs include better time-stamp extraction techniques or ad- vanced anomaly detection algorithms used in the COMPAS problem diagnosis processes [118]. • Client-Side FEP: facilitate the extension of the functionality available to or pro- vided by the COMPAS Monitoring Dispatcher. The run-time path construction and analysis module introduced by JEEM uses such a client-side extension and benefits from the information made available by the distributed infrastructure to the client-side. Other examples of extensions that can be added using client-side FEPs include specialised GUI consoles or integration into wider-scope perfor- mance tools. As indicated, the additional functionality that COMPAS JEEM provides over the basic COMPAS framework uses several FEPs, across all JEE tiers. The following sections present in detail these JEEM extensions. 66
- 79. 4.2.5 COMPAS Extensions Asdetailed in section 4.2.4 the current implementation of the COMPAS framework has the ability to monitor the business tier of JEE applications. We have made a num- ber of extensions to the COMPAS framework using the COMPAS FEPs to allow for the tracing of run-time paths across all tiers of a JEE system for multi-user workloads. The extensions made fall into three different categories, namely, monitoring, run-time path tracing and analysis. The overall architecture of COMPAS JEEM is shown in figure 4.6. 4.2.5.1 Monitoring The COMPAS JEEM monitoring approach is completely portable and non-intrusive and has the ability to trace system wide JEE run-time paths. Basically it works by first intercepting the calls made to the different components that make up the system and second, recording their sequence. The interception of calls is performed by extend- ing the COMPAS framework, using server-side FEPs, so that monitoring on all three server side tiers is performed. COMPAS uses the idea of a proxy layer to monitor EJB components [119]. The proxy layer is a thin layer of indirection directly attached to the EJBs that captures any re- quests to and responses from the components. When extending COMPAS we applied the idea of intercepting requests to and responses from components to the web and EIS tiers. This is performed in the web tier using intercepting filters and in the EIS tier using wrapper components. The EJB probes, intercepting filters and wrapper compo- nents can also be referred to as Interception Points (IPs). Intercepting Filters A chain of pluggable filters can be created to implement com- mon pre and post-processing tasks during a web page request by implementing the Intercepting Filter pattern [7]. This patterns allows for the creation of pluggable fil- ters to process common services in a standard manner without requiring changes to core request processing code. The filters intercept incoming requests and outgoing re- sponses, which allows for pre and post-processing respectively. The servlet 2.3 speci- fication 4 includes a standard mechanism for building filter chains based on the Inter- cepting Filters pattern. Consequently web applications running on servlet specifica- tion compliant web servers can be augmented with filters for pre and post processing without the need to modify source code. Next we briefly detail the Intercepting Filter pattern which we use to monitor the web tier of JEE applications. A UML sequence diagram of the Intercepting Filter pattern is shown in figure 4.7. The pattern contains the following participants: the Client, the FilterManager, the Fil- 4Java Servlet Specification, http://java.sun.com/products/servlet/download.html, 67
- 80. Figure 4.7: InterceptingFilter terChain, one or more Filters and finally the Target resource. The FilterManager sits between the client and the Target resource. When the client makes a request for the Target resource the FilterManager intercepts this request and creates a FilterChain. A FilterChain is an ordered collection of independent Filters. The Filters perform the pre and post processing logic and are mapped to a particular Target resource. The FilterManager next makes a request to the FilterChain to process the filters and subse- quently the Target resource. The Intercepting Filter pattern can be implemented using a number of different strate- gies. Servlet specification compliant web servers implement the Intercepting Filter Standard Filter Strategy [7] whereby filters can be added declaratively through XML deployment descriptors. The web server container contains the FilterManager and is responsible for creating the different FilterChains that can be specified through the XML deployment descriptors. To monitor the web tier we have implemented a monitoring filter. The monitoring filter intercepts incoming requests and outgoing responses and thus can log the beginning and end of user requests. The monitoring filter can be added to the web application under test automatically (see section 4.2.5.1 below). Before the servlet 2.4 specification filters could only be applied to the initial client 68
- 81. request. Filters couldnot be applied to inter servlet communications. As a result the web tier was monitored as one software component on web servers prior to 2.4. However the 2.4 servlet specification allows for filters to be applied to calls between servlets (i.e. forwarded requests) 5 and thus our monitoring filters can collect the entire web tier run-time path trace for servlet specification 2.4 compliant web servers. Since this has been mandated by the servlet specification all future JEE environments will support this feature. EIS Tier Wrapper Components The monitoring of the EIS tier is performed using wrapper components. This approach takes advantage of the proxy design pattern [72]. EIS tier monitoring works by wrapping the JDBC database driver with a mon- itoring layer. JDBC drivers implement the JBDC interfaces and classes of the JDBC API. For each original JDBC driver class, a wrapper class that exposes the same inter- face as the driver class is created. Rather than registering the JDBC driver class with the application server as normal, the packaged wrapper classes are registered with the application server. Calls made to the database layer from the application server are thus intercepted by the wrapper classes. The original JDBC driver is instead regis- tered with the wrapper classes (declaratively using a properties file). Any calls made to the wrapper classes can thus be delegated to the original driver classes. The wrap- per classes contain monitoring logic that captures all JDBC requests and responses. The wrapper classes have been implemented by extending the P6Spy open source framework 6. As outlined above COMPAS JEEM’s monitoring framework consists of a number of IPs. The IPs capture performance metrics and resource usage information along with the control flow data required for run-time path tracing. Adding The Monitoring Framework to a JEE Application: Adding the aforemen- tioned filters and wrapper components does not require the server code or the appli- cation source code to be available nor does it require special JVM hooks. Instead we leverage standard JEE mechanisms and make use of the component meta-data that is stored in XML files as mandated by the JEE specification. The monitoring components can be automatically added to the packaged application. The basic instrumentation script (i.e. the CPI process) provided by the original COMPAS monitoring framework has been extended to (1) unpack the entire JEE application, (2) parse the XML deploy- ment descriptors of the application (to obtain structural and functional information on the different components that make up the system) and (3) repackage the application with the COMPAS JEEM monitoring components inserted. 5Jason Hunter, ”Servlet 2.4: What’s in store”, Java World, March 2003, http://www.javaworld.com/javaworld/jw-03-2003/jw-0328-servlet.html, 6P6Spy, http://www.p6spy.com/, accessed June 2006 69
- 82. 4.2.5.2 Run-time PathTracing The ability to trace system wide run-time paths for multi-user workloads has been added to the capabilities of COMPAS JEEM. A generic run-time path tracing approach has been outlined in section 4.2.3. This approach has previously been implemented by Chen et al. in the Pinpoint open source project [45] [44]. However, a major drawback of the pinpoint implementation is that it relies on instrumentation of the middleware and it is therefore tied to its target application server, JBoss. In the following sections we detail the Pinpoint implementation of the above approach and how we applied it in a non-intrusive capacity. Pinpoint - Intrusive run-time path Tracing: Pinpoint is a framework for problem de- termination in internet service environments. It includes a runtime path tracer for the JBoss application server. The requirements R1 R3 and R4 outlined in section 4.2.3 have been met in the Pinpoint implementation through instrumentation of the mid- dleware. The following areas in the middleware were instrumented: the web server, the application server, a Remote Method Invocation (RMI) library and the JBoss JDBC Resource Adaptor [105]. Next we briefly detail the above instrumentation points, and their responsibilities. R2 is the only requirement that has been met in a non-intrusive capacity by the pinpoint framework and is described in section 4.2.5.2. User requests enter the system in the web tier. Pinpoint instruments both the http server and the servlet container. The instrumentation in the http server is responsi- ble for determining when a new request enters the system (R1). It injects a unique ID (R2) value into the local thread of the new incoming request. The servlet container in- strumentation is responsible for logging calls to and responses from each JSP/servlet (R4). The servlet container instrumentation is also responsible for the runtime path ordering logic in the web tier. The application server instrumentation is performed in the Pinpoint project using JBoss interceptors [105]. The instrumentation logs calls to and from EJB components (R4) and performs runtime path ordering logic when a method is called. The JDBC Re- source Adaptor instrumentation has similar responsibilities in relation to JDBC calls. Pinpoint also allows for tracing of run-time paths that span a number of threads (which for example might be caused by a remote method invocation) by instrument- ing a JBoss RMI library such that the unique request ID is marshalled across the remote call from the local thread to the new (remote) thread (R3). Non-Intrusive Run-time Path Tracing We require a non-intrusive approach for run- time path tracing. In the following paragraphs we discuss how all the requirements (outlined in section 4.2.3), monitoring (R4), user request identification (R1), user re- quest tagging (R2) and user request tracking (R3) can be met in a non-intrusive capac- ity. 70
- 83. An application canbe easily monitored (R4) by instrumenting middleware. Where the middleware source code is available it can be instrumented and recompiled such that the components running on top of the middleware are monitored and the required information logged (for instance performance metrics). The issue of monitoring the components in a non-intrusive manner is overcome using the COMPAS framework along with the monitoring extensions that we introduced above in section 4.2.5.1. The run-time path tracing approach discussed in section 4.2.3 requires user request tagging (R2), i.e. tagging each user request with a unique id. User requests are tagged by inserting a unique value into the local thread object when a new user request enters the system. The pinpoint tracing library performs this in a non-intrusive manner: A ThreadLocal object (introduced into Java in version 1.2) is associated with each thread in the system. The ThreadLocal objects can be used to hold a particular value (or object) for the lifetime of the thread. The pinpoint tracing library can be used to inject a RequestInfo object into the ThreadLocal object. The RequestInfo object contains the RSI (i.e. the unique Id of the user request and the sequence number of the current method call). Determining when a new user request arrives into the system can be easily achieved when instrumenting the middleware. If user requests are restricted to entering the system in the web tier, then the HTTP server (a component of the web server) can be instrumented such that each new request for a http connection can be tagged. A new request for a http connection represents a new user request. However, determining new user requests non-intrusively, does not allow manipulation of the HTTP server. To overcome this we implemented a point of entry detection mechanism. Point of entry detection is required to find if a call to a particular component is the point of entry into the system i.e. is this call as a result of a new client request. We have extended the Pinpoint tracing library to allow for point of entry detection (R1) in a non-intrusive manner. Our point of entry detection mechanism works by monitoring the depth of the calls made on the current thread. We have extended the pinpoint RequestInfo object to in- clude a Depth field to monitor the level of the current method call. The depth value is incremented when entering a method and decremented upon method exit. A depth of zero indicates that the current method is the first method called in the run-time path and is thus the point of entry into the system. When a call is made to a method with a depth of zero a unique ID can be assigned to that thread and remains associated with that thread until the call depth again reaches zero. Our point of entry detection mechanism is advantageous since (1) it is non-intrusive and (2) it can be applied to any of the tiers in a JEE system and unlike with the pinpoint approach outlined above new user requests must not be restricted to the web tier. The point of entry detection logic can be inserted into any of the IPs in figure 4.6 to identify new requests entering the system at any of the different tiers. 71
- 84. The remaining issueto be addressed in relation to non-intrusive run-time path tracing is user request tracking (R3). While intrusive approaches can afford the luxury of instrumenting RMI libraries to marshall request IDs across threads, this can not be achieved non-intrusively. Our approach instead makes the assumption that the web server and application server are co-located. The database can be deployed either locally or on a remote machine. This assumption is justifiable, since it is often the case that enterprise systems are deployed on co-located servers during development. It is common that such systems are only deployed in a distributed environment late in the development process when production testing takes place. In fact recently it is becoming more and more common for the web tier and business tier to be co-located even in (often clustered) operational environments. The web and business tier can be co-located to take advantage of faster local calls between the tiers. Thus, these environments typically contain multiple identical machines, each running an instance of the web server and the application server, optimised for local inter-communication. Servers that are co-located run on the same Java Virtual Machine (JVM). Thus, when servers are co-located, for a particular user request that consists of local calls, the same thread is used across the web server, EJB server and database driver. Thus using our approach the user request can be tracked across the different tiers, since the user re- quest can be identified by its unique ID stored in the ThreadLocal object. An issue arises, however, if the run-time path includes remote calls to components that are de- ployed locally. For instance a servlet may call an EJB that is deployed locally, through its remote interface. In this case it is possible that a new thread is spawned when the remote call occurs. In fact according to the RMI specification 7: ”A method dispatched by the RMI run-time to a remote object implementation may or may not execute in a separate thread.” In practice, however it seems that a new thread is not spawned when a remote call is made to a component that is in fact local to the callee. This can be clearly seen from the results of our portability tests (see section 7.2.2). We be- lieve that application server vendors take this decision not to spawn new threads since spawning a new thread when invoking local components through their remote inter- faces would most likely prove wasteful. The fact that new threads are not spawned allows for the tracing of user requests across all JEE tiers using the approach we out- lined above. A problem arises, however, if calls are made across distributed JVMs. In this situation a new thread is spawned on the remote JVM to handle any remote calls. The new thread does not contain the RSI and thus calls made to the remote component cannot be traced. 7Java Remote Method Invocation Specification, http://java.sun.com/j2se/1.4.2/docs/guide/rmi/ 72
- 85. 4.2.5.3 Tracking RSIin Distributed Environments In this section, we analyze how calls from a client to a server can be tracked in a distributed setting. The approach we take is based on piggy-backing the RSI onto the request from a client to a remote server. In the following subsections we detail how this has been achieved in a portable manner for JEE systems. This approach has previously been implemented in the WebMon [82] profiling tool, a tool for profiling JEE web applications. In the following sections we discuss how this can be applied for COMPAS JEEM. 4.2.5.4 Piggy-backing data There is no standard mechanism in JEE for piggy-backing data along a remote request. Piggy backing mechanisms however are available in other technologies. For example CORBA’s Portable Interceptors 8 allow for piggy-backing of data along a request. The CORBA Portable Interceptors can be used to track user requests in distributed envi- ronments. For example, it has previously been shown that they can be used to attach additional security information onto requests [179]. We discuss how this problem can be solved for JEE systems by adding an additional parameter to the remote request and allowing the client to pass the extra request data through that additional parameter. This can be achieved without access to the appli- cation’s source code as follows: A components’ meta-data specifies the component interfaces. The interfaces can be reconstructed and rewritten from their bytecode rep- resentation (or by analysing the XML deployment descriptor) to add a method that allows for the passing of the additional information using an additional request pa- rameter. Next the client component is also instrumented, to add this additional pa- rameter, and the server component instrumented, to pass it to the instrumentation infrastructure. In the next section, we give a detailed description how this technique can be used to pass RSI across distribute component boundaries. 4.2.5.5 Client Side Interception When a client component calls a distributed component in JEE, the following occurs: The client calls a stub which is a client side proxy object. The stub masks the low level networking issues from the client and forwards the request on to a server side proxy object. The server side proxy element, known as a skeleton, masks the low level networking issues from the distributed component. It also delegates the remote request to the distributed component. The distributed component then handles the request and returns control to the skeleton, which in turn returns control to the stub. 8Common Ojbect Request Broker Architecture: Core Specification, Version 3.0.3, http://www.omg.org/technology/documents/ 73
- 86. The stub, inturn, hands back to the client (see figure 4.8). Figure 4.8: Remote Method Invocation For client side interception an IP is responsible for adding an additional parameter to the remote request. Adding a client side IP can be accomplished by replacing the client stub used by the application with a custom wrapper. This custom wrapper pro- vides the functionality to intercept the client’s request invocation before invoking the server side EJB. The custom wrapper performs the necessary task of extracting the RSI from the ThreadLocal object (or adding it if it is the point of entry into the sys- tem) and sending this information as an extra parameter in the remote call. A sample home interface and a custom wrapper that accomplishes this is shown in Listings 4.99 and 4.10. A similar wrapper has to be provided for the bean’s remote interface. In order for the custom wrapper to be able to intercept the client side calls it must replace the stubs generated by the Java Virtual Machine. With JEE, the client obtains a reference to a remote component using the Java Naming Directory Interface (JNDI) naming service and obtains the stub by casting the reference to the component’s inter- face from a generic type to an RMI-IIOP interface type using the PortableRemoteOb- 9Listings 4.9, 4.10 and 4.11 are based on an example first written by Dr. Thomas Gschwind 74
- 87. ject.narrow method (seeListing 4.11). public interface SampleHome extends EJBHome { Sample create() throws RemoteException, CreateException; } Figure 4.9: The Sample Bean’s Home Interface public class WrappedSampleHome implements SampleHome{ private SampleHome home; public WrappedSampleHome(SampleHome home) { this.home=home; } public Sample create() throws RemoteException, CreateException { // intercept request before invocation WrappedSample obj=new WrappedSample (home.create()); // intercept request after invocation return obj; } public void remove(Object p0) throws RemoteException, RemoveException { home.remove(p0); } ... // other methods provided by home stub } Figure 4.10: A Wrapper for the Sample Bean’s Home Interface The JVM generated stub can be replaced with the wrapper by using a custom PortableRemoteObjectDelegate object. This is supported through a standard mech- anism in the JEE specification and can be configured as part of the Java runtime en- vironment. Thus this approach is portable across different application server imple- mentations. 4.2.5.6 Server Side Interception On the server side the IP is in the form of a proxy layer (or wrapper) as described in section 4.2.5.1. Using this approach all calls to and from the bean can be intercepted. The extra parameter (RSI) sent by the client is handled in the wrapper which overloads 75
- 88. public class SampleClient{ public static void main(String[] args) throws Exception { Properties props=System.getProperties(); Context ctx=new InitialContext(props); Object obj=ctx.lookup("Sample"); SampleHome sh =(SampleHome) PortableRemoteObject.narrow (obj,SampleHome.class); Sample s = sh.create(); System.out.println (s.callBusinessMethod("SomeString")); s.remove(); } } Figure 4.11: A Sample Bean Client each method such that it contains an extra parameter. This extra parameter is used to piggyback the data across the network. In the context of run-time path tracing once the data has been sent across the network it can be attached to the current thread as outlined in section 4.2.5.2. 4.2.5.7 Analysis COMPAS JEEM contains run-time path tracing logic which non-intrusively logs in- formation when methods are invoked. The monitoring framework uses the COMPAS Monitoring Dispatcher to asynchronously log the information to a remote machine. This allows for the information to be stored and analysed off-line. Analysing the data remotely (as opposed to analysing it locally) reduces the performance impact on the monitored system. There are two main phases of information extraction from the collected data: run-time path construction and advanced run-time path analysis. It is important to note that the analysis part of the framework is completely independent of the approach used for run-time path extraction. The only requirement is that the necessary information (see Run-Time Path Construction below) is provided. Run-Time Path Construction On every invocation of a component method, informa- tion is logged. In the pinpoint framework this is known as an observation. The logged information contains the following data: • user request (unique) ID • sequence number • call depth 76
- 89. • component details •performance data Run-time paths are constructed by grouping all observations by user request id. Each such group represents an unordered list of the calls that make up a single run-time path. By ordering the list according to the sequence number of each observation, the observations can be arranged in the order that they occurred during program execu- tion. A run-time paths data structure is subsequently created by traversing each or- dered list. This completes the construction of the run-time paths. Further analysis can be applied to this information for a number of purposes. For example, by analysing the run-time paths developers can easily construct UML diagrams that can help to de- duce the overall system structure (see sections 7.2.1). This output also forms the basis of the automatic antipattern detection approach and allows for the identification of component relationships, component communication patterns and the reconstruction of container services (see chapter 5). Next we discuss how component object tracking can be performed using this data. 4.2.5.8 Tracking Component’s Object Usage Objects can also be tracked along run-time paths to allow for analysis of their usage in the system. Such analysis can lead to identification of inefficient object usage (see section 7.4.1). Figure 4.12 shows an example run-time path which tracks the lifecycle of instances of the AccountDetails data transfer object. The COMPAS JEEM tool is currently being extended (the new version is called COM- PAS ByteCode Instrumentation [BCI]) to track selected objects across run-time paths. The new version of the tool makes use of dynamic byte code instrumentation, through the java.lang.management interface, to dynamically instrument selected objects. Tool users can select particular classes to be tracked. When an object of the selected class is created it is identified along with its corresponding position in the run-time path. Whenever another method (other than the constructor [or creator method for EJBs]) is accessed this is also logged. Thus we can identify where objects are created along the run-time paths and at what points they are accessed. We can effectively see how objects are created, used and passed along the run-time path. Figure 4.12 show where instances of the AccountDetails object are created and accessed along a JEE call path. The object was created by an entity bean in the application tier and passed to the web tier where a single method was accessed. This information is required to identify a range of common antipatterns (for example to identify variations of the excessive object allocation antipattern [154] which manifests itself in a number of different an- tipatterns in enterprise applications). We have developed a proof of concept of this in- strumentation by manually instrumenting objects that we required for tracking. The 77
- 90. Figure 4.12: Run-TimePath with Tracked Object, as a Sequence Diagram COMPAS BCI approach which makes use of dynamic bytecode instrumentation is only available for newer JVMs (Java 1.5+). 4.2.5.9 COMPAS JEEM Portability One of the main advantages of our approach is that it is portable across the JEE spec- trum of platforms and can be applied independent of the underlying middleware im- plementation. However much of this approach is portable across different component technologies (such as CCM). In fact only the instrumentation process (CPI) and the request tracking (using the ThreadLocal approach) are specific to JEE. Therefore much of our implementation can be reused across technologies as long as an instrumenta- tion process and request tracking approach can be provided. The CORBA Portable Interceptors, for example, allow for the insertion of custom monitoring code (i.e. in- strumentation of the application) and the piggy backing of data along a request (which allows for request tracking). Thus our approach could also be applied in the CORBA environment by making use of the Portable Interceptors provided for through the CORBA standard and by reusing much of our implementation to perform run-time path tracing (i.e. the tagging logic, the point of entry detection mechanism, the run- time path ordering logic, the monitoring dispatcher, and the run-time path analysis modules). 78
- 91. 4.3 Monitoring ServerResource Usage and Extracting Component Meta-Data In this section we outline how the final two monitoring components have been im- plemented, i.e. the server resource monitor and the component meta-data extractor. We give details on the information they extract and how this relates to the run-time design model. We also discuss why the approaches are portable and why they do not have a major impact on the system performance. 4.3.1 Using Java Management Extensions to Monitoring Server Re- source Usage In enterprise frameworks such as JEE the different components that make up the ap- plication interact with the underlying middleware. The state of the server resources that service these components can significantly impact the overall system performance (e.g. thread pools, database connection pools, object pools). According to the JEE Management specification (JSR 77 10) application servers are required to expose this data through the Java Management Extensions (JMX) technology 11. The foundation of the JMX technology is the Managaed Bean (MBean). JMX allows for a centralised management of MBeans, which act as wrappers for applications, components, or re- sources in a distributed network. This functionality is provided by a MBean server which serves as a registry for all MBeans. MBeans provide attributes, operations, and notifications. A client (e.g. a management console) can read and change the attributes, invoke operations, and receive notifications when certain events occur. The JEE management model exposes MBeans for all well-known concepts from JEE. Examples include the JVM, EJBs, Sevlets and JDBC Drivers. The JEEManagedObject, shown in figure 4.13 12, is the base model of all managed objects in the JEE Manage- ment Model. JSR77 compliant application servers provide access to all managed object instances through a Management EJB (MEJB) component. The MEJB is a stateless ses- sion bean, which can be accessed either locally or remotely. The JSR77 also defines a Performance Data Framework which specifies a data model as well the performance data requirements of the JEE Management Model. Performance statistics on the var- ious parts of the JEE application server can thus be easily obtained at runtime. For example, figure 4.14 shows the JDBCStats type that is expected to be provided by a JDBC resource. We collected various performance statistics specified by the Performance Data Frame- work for the different JEE component pools (i.e. the thread pool, the database con- 10Java Service Request 77, J2EE management, http://www.jcp.org/en/jsr/detail?id=77 11Java Management Extensions Technology,http://java.sun.com/javase/technologies/core/mntr- mgmt/javamanagement/ 12Figures 4.13 and 4.14 are from the JEE management specification 79
- 92. Figure 4.13: JEEManagedObject nectionpool, and the various EJB pools) and use this information to populate the run-time design meta model shown in figure 3.2 on page 46. Pool statistics include in- formation on the pool size, free pool size, pool queue size, passivation levels etc. This information can be directly added to the design model and can be utilised to identify deployment related antipatterns (see section 7.4.1). The monitoring module makes use of a custom JMX monitoring tool to log such information. Since the data recorded by the tool is already made available by the application server, the performance impact of a JMX monitoring tool is minimal. 4.3.2 Automatically Extracting Component Meta-Data Component based enterprise frameworks are particularly suited for antipattern de- tection since (a) they specify a component model that defines how the different com- ponents types in the model should be used (e.g. using entity beans for persistence) and (b) they generally contain meta-data on the components that make up the system. EJB meta-data contains structural and functional information on each of the EJB’s de- ployed in the system. For example, information on the EJB component type (i.e. is the bean a stateless session bean, a stateful session bean, an entity bean or a message 80
- 93. Figure 4.14: JDBCStats,JDBCConnectionStats, JDBCConnectionPoolStats driven bean). The meta-data also contains information on the container services re- quired by the bean’s business methods (e.g. whether the bean requires security checks, whether the bean should be invoked in a transactional context, whether the bean can be accessed remotely). This meta-data is contained in the XML deployment descrip- tors that are used to configure the application during deployment (see figure 2.3 on page 20). Thus the meta-data can be obtained without having to access the source code of the application. The data can be used to reason about the behaviour of cer- tain components. For example, if from our run-time profiling we see that a particular component is frequently communicating with a database, from the meta-data we can check the component type. If this component turns out to be a stateful session bean we could flag this as a potential antipattern, since stateful session beans are not designed to frequently access persistent data (as outlined in the component model specified by 81
- 94. the EJB specification).The fact that component based enterprise frameworks specify how certain component types should behave allows us to automatically identify this type of unusual behaviour. Without this information automatic antipattern detection is more difficult. For example, if instead we were monitoring an application made up of Plain Old Java Objects (POJO’s) with no information describing how we expect the objects to behave, it would be difficult to flag unusual behaviour. In such situations domain or application specific information could instead be supplied by the appli- cation developers. The PAD tool extracts the component meta-data from the XML deployment files of the JEE applications automatically using an XML parsing library 13. This is in fact achieved during the COMPAS instrumentation process (see section 4.2.5.1). The information obtained through the PAD tool’s meta-data extractor is used to pop- ulate the run-time design model (figure 3.2, page 46). Functional and structural in- formation is obtained on the various components (e.g. the component type, the com- ponent’s methods, the interface type). This information can be directly added to the run-time design model. Information on the container services that are required by the different components is also collected. In section 5.3 we discuss how these services can be reconstructed using the component meta-data and run-time path information and subsequently added to the run-time design model. 4.4 Related Work In this section we discuss and compare related work to the work presented in this chapter. Firstly we outline how run-time paths have been previously used for a num- ber of different purposes. We follow this by discussing alternative representations for dynamic calling behaviour that have been previously presented and compare these to run-time paths. Finally we discuss tracing tools and techniques that are related to the run-time path tracing approach presented. 4.4.1 Applications of Run-Time Paths In section 4.2.2 we have discussed a number of uses to which run-time paths have been put to, for example, reverse engineering, information extraction for antipattern detection, or system comprehension. Run-time paths however have also been utilised in a number of other ways. Chen et al. [45] have applied run-time paths to deal with fault detection and diagnosis in large distributed systems. To detect faults they characterize distributions for nor- mal paths and look for statistically significant deviations that might suggest failures. To isolate faults to the components responsible (diagnosis) they search for correlations 13Java XML Libraries, http://java.sun.com/xml/ 82
- 95. (using data miningand machine learning techniques) in the run-time paths between component use and failed requests. They have also applied run-time path analysis to assess the impact of faults and to help with system evolution. Traditional approaches to this problem have relied on static dependency models. However such dependency models often do not capture the dynamic nature of today’s systems. Chen et al. use dynamic tracing (i.e. run-time paths) to capture the dynamic nature of today’s con- stantly evolving internet systems. The literature [19] presents the use of run-time paths to address performance issues. The Magpie project 14 collects run-time paths for distributed systems. It also measures resource consumption. Magpie then builds stochastic workload models suitable for performance prediction, tuning and diagnosis. Another application of run-time paths (outside the area of system comprehension) can be seen in recent work on autonomic monitoring [119]. Here Mos and Murphy intro- duce an adaptive monitoring framework whereby instrumentation is performed by low-overhead monitoring probes which are automatically activated and deactivated based on run-time conditions. The approach makes use of dynamic models to ac- tivate and deactivate monitoring probes, such that the monitoring overhead is kept to a minimum under normal conditions. If a there is a sudden degradation in sys- tem performance monitoring probes are automatically activated to identify the bottle- neck component(s). The dynamic models are utilised to determine which monitoring probes should be activated to identify the bottleneck. The dynamic models consist of the monitored components and the dynamic, ordered relationships between them (i.e. the dynamic models are equivalent to run-time paths). 4.4.2 Alternative Representations for Component Interactions In this subsection we discuss alternative ways of representing component interactions recorded at run-time. The authors Jerding et al. [94] discuss a spectrum of representa- tions that can be used for this purpose, from the most accurate and space inefficient, to the least accurate and most efficient. Dynamic call traces are said to be most accurate and space inefficient while call graphs are least accurate and space efficient [8] [94]. A dynamic call trace is a recording of all calls and returns that occur in a program’s execution. They capture important information such as the sequence of calls, the con- text in which a call was made, and repeating calls or loops. However extracting the call trace may not be feasible for long running programs since they are unbounded and their size is proportional to the number of calls in an execution. Call graphs on the other hand are space efficient but are less accurate. A call graph is a compact rep- resentation of calling behavior that summarizes all possible component relationships. While they are bounded by the size of the program, there is much interesting informa- 14Microsoft Research. Magpie. http://research. microsoft.com/projects/magpie/. 83
- 96. tion about callingbehavior that is dropped to gain compactness. The sequencing of calls, the context in which certain calls are made, and repeated calls are all examples of calling behavior that is lost. Call graphs are said to be context insensitive while call traces are context sensitive [8]. Context sensitive profiling associates a metric with sequence of executed calls. This can be used to separate measurements from different invocations of components [8]. Profiling tools that are context insensitive can only approximate a program’s context dependent behaviour. In fact the inaccuracy of pro- filers such as gprof [79] and qpt [106] can be attributed to the fact that they used call graphs to represent context dependent profile information in a context-independent manner [141]. Another issue with call graphs is that they can contain infeasible paths that never actually occurred during execution. A calling context tree is an intermedi- ate representation that is more accurate than a call graph and less inefficient than the dynamic call trace in terms of space requirements. In fact, it is a more compact repre- sentation of the dynamic call trace and can represent all contexts in a given trace. The calling context tree discards any redundant paths in a trace while preserving unique contexts. Metrics from identical contexts are aggregated in order to improve space efficiency. Calling context trees are context sensitive and thus do not suffer from the same inaccuracies that can occur with call graphs. The breath of a calling context trees are also bounded by the number of methods or procedures in an application. In the absence of recursion the depth of calling context trees are also bounded [8]. As outlined in section 4.2.1 run-time paths are essentially dynamic call traces, that contains performance metrics, and that are recorded for each user request in a multi- user distributed application. They contain the same important information contained in dynamic call traces, that describes the calling behaviour of an application. How- ever they are also unbounded and thus can be space inefficient. To solve this issue a set of run-time paths can be easily represented by a calling context tree in the same way that a dynamic call trace is [8]. An issue with calling context trees however is the fact that the do not maintain the exact sequence of events as they occurred dur- ing execution. For this reason we have not used them in this work, since we require that the exact sequence of events is maintained to identify particular antipatterns (e.g. communication patterns discussed in section 6.1). 4.4.3 Run-Time Interaction Tracing Approaches The closest run-time path tracing approach to our work is the pinpoint problem deter- mination tool. In fact we have made use of the pinpoint tracing approach in COMPAS JEEM and have extended it such that it can be applied in a non-intrusive and portable manner. Pinpoint has been discussed in detail and compared to our approach in sec- tion 4.2.5.2. Essentially its main drawback is that it instruments the middleware and thus it is not portable across different middleware implementations. Webmon [82] is a similar tool that also instruments the middleware to trace run-time 84
- 97. paths. However aswell as monitoring the server side components, webmon also in- struments client side components. This is performed used a cookie-based approach and allows for webmon to trace paths beginning in the browser, rather than on the server side. Since instrumentation is performed on the client side this approach gives a more accurate measurement of a client’s actual wait time when a request is made to the system. Server side only approaches record the amount of time that the sys- tem takes to process the request but does not record the time spent communicating between the client and the system. The main drawback of the webmon approach is that it is not portable. Another issue outlined by the authors is the fact that the instru- mentation process was very time consuming. The magpie project [19] also monitors paths through distributed systems. The initial approach [19] uses a unique identifier to identify requests and propagates this iden- tifier from one component to the next. A later version [18] avoids the need for global identifiers and instead correlates component events based on an application specific event schema. Both versions make use of kernel, middleware and application level in- strumentation. The magpie project has been implemented for Microsoft technologies. Another approach closely related to our work has been recently presented in the lit- erature [78]. The InfraRed tool 15 makes use of AOP to instrument the application and has the ability to capture run-time paths, (although it presents this information in the form of a calling context tree). Similar to our approach, InfraRed makes use of a ThreadLocal object to record the sequence of calls. Their approach is potentially portable across different middleware implementations and is implemented using As- pectJ 16. It has also been shown to have a low overhead. However, the authors suggest an alternative approach to remote call tracking and do not marshal data across the net- work. Instead they simply relate the calls via the method name. Using this approach it is not clear how the order of calls can be maintained across remote calls when the system is under load. Briand et al. have presented an approach for reverse engineering sequence diagrams from multi-threaded [34] and distributed java applications [35]. They record traces using an AOP monitoring approach. To distinguish between local threads they make use of a threadID identifier, however this information is not stored in a threadlocal object. Instead a list of threadIDs are maintained in a text file. To allow for distributed tracing they marshall data across the network using aspects. Similar to our approach they intercept remote calls such that information can be sent across the network such that one can identify the client method called that called a particular remote method. The information they send across the network however differs from the information we use. They make use of the identifier of the client node (clientNodeID), the thread identifier (clientThreadID), the class name (clientClassName) and the object identi- 15InfraRed, http://infrared.sourceforge.net 16The AspectJ Project, http://www.eclipse.org/aspectj/ 85
- 98. fier (clientObjectID). Howeverthe approach essentially works in the same way since their information is used to be able to identify the client thread responsible for remote communication. Their approach has not been applied to JEE applications. JFluid [59] (now part of the netbeans profiler 17) is another tool which produces calling context trees. JFluid also makes use of a per thread data structure, which the authors call a ThreadInfo object, to collect data for construction of the calling context tree. JFluid can perform dynamic bytecode instrumentation using a modified JVM. Tool users can selectively profile their applications by selecting a number of root methods. Instrumentation is injected into all methods that are reachable from the root method specified by the user. This results in a reduced set of methods (i.e. a subgraph) that are profiled. However an issue can potentially arise that could result in a distorted calling context tree. This issue comes from the fact that any method within a subgraph of profiled methods can be called from a non-profiled method outside of the subgraph. To make sure the calling context tree is not distorted, the instrumentation has to detect whether a method is called in the profiled subgraph context or not. To achieve this the JFluid tool maintains the depth of calls within the subgraph (in the ThreadInfo object) to identify if calls are made to instrumented methods from non-instrumented meth- ods. Their approach is based on the same principles as our point of entry detection mechanism, which also maintains the call depth in a per thread data structure (i.e. the ThreadLocal object). Hall [85] has implemented an approach to collect call paths in the form of the CPProfJ profiler. CPProfJ makes use of sampling to periodically interrupt a running appli- cation and records the runtime stacks of all threads currently in the system. From the information recorded call path profiles are constructed. Call paths are usually represented by a list of methods enclosed in parentheses, e.g. (method1, method2, method3), and are defined as representing ”all stack traces that could be present when each of the calls within the call path is present (in call path order)” [84]. Call path pro- files report statistics about how much run-time can be attributed to the different sets of stack traces. Unlike with run-time paths however call paths are not end-to-end and do not maintain the exact order of all events from the beginning to the end of a re- quest. They are thus not suitable for our purposes. In fact call paths have mainly been applied to address lower level issues than we focus on, such as inter-thread depen- dencies. Finally data mining techniques have also been applied to extract dynamic dependen- cies from JEE systems. Agarwal et al. [1] use a data mining approach to extract re- source dependencies from monitoring data. The output is similar to that of COMPAS JEEM. The approach is also non-intrusive since the data required to extract the depen- dencies is obtained from existing system monitoring data. Their approach relies on the assumption that most system vendors provide a degree of built in instrumenta- 17The NetBeans Profiler Project, http://profiler.netbeans.org/ 86
- 99. tion for monitoring.A major drawback of this approach however is that it is statistical and unlike COMPAS JEEM it is not exact, and at higher loads the number of false dependencies increase significantly. 87
- 100. CHAPTER FIVE Reconstructing the Systems Designfor Antipattern Detection In this chapter we discuss how we automatically extract the different relationships (that make up a reconstructed design model of the system) from the monitored data. In particular we detail how we automatically identify inter-component relationships, inter-component communication patterns and object usage patterns. In addition, we show how run-time container services can be reconstructed and added to the design model and how our analysis approach summarises and reduces the amount of data produced during monitoring. A number of advanced analysis techniques are applied to meet these aims. In particular we apply two techniques from the field of data min- ing, i.e. Frequent Sequence Mining (FSM) and Clustering. FSM is applied to run- time paths to identify interesting component communication patterns, for example, resource intensive loops. Issues related to applying this technique to run-time paths are discussed and solutions are provided. Clustering and statistical analysis tech- niques are applied for the purpose of data reduction. Following this we describe the run-time design model of the system that captures the information extracted during analysis. Finally, we discuss and compare related work. 88
- 101. Main Points • Dynamiccomponent dependencies and component object usage information can be automatically extracted from runtime paths. • Run-time container services can be automatically reconstructed from run-time path information and component meta-data. • FSM can be used to automatically identify component communication patterns in run-time paths. However, applying FSM to run-time paths creates scalability issues in terms of both the FSM output and the algorithm run-time. • FSM with non-overlapping support counting can be applied to allow for a more human readable output of the FSM algorithm. • FSM can be applied to identify both frequent sequences and resource intensive sequences. • Preprocessing techniques can be applied to run-time paths to reduce the run- time path lengths an ultimately the FSM algorithm run time. • All run-time paths recorded during monitoring can be clustered into groups to give a concise method level and component level view of the system. • The extracted run-time design model contains the following information: – component dependencies – component communication patterns – component object usage patterns – component structural and functional information – reconstructed container services – run-time server resource information 89
- 102. 5.1 Chapter Overview Inthis chapter we discuss how we automatically extract the different relationships (that make up a reconstructed design model of the system) from the monitored data. In particular we detail how we automatically identify inter-component relationships, inter-component communication patterns and object usage patterns. In addition, we show how run-time container services can be reconstructed and added to the design model and how our analysis approach summarises and reduces the amount of data produced during monitoring. More specifically section 5.2 outlines how the run-time component relationships and component object usage patterns can be extracted from the monitoring data by traversing the run-time paths collected. In section 5.3 we detail how the run-time con- tainer service boundaries can be automatically reconstructed by analysing the run- time paths and the component meta-data. The output of this analysis shows how the different container services are being used at runtime by the application components. The automatic extraction of component communication patterns requires more com- plex analysis than the analysis techniques in sections 5.2 and 5.3. For this purpose we make use of a data mining technique, frequent sequence mining (FSM). In section 5.4 we discuss how we applied FSM to run-time paths, to identify sequential patterns of interest (e.g. frequent sequences, resource intensive sequences) within the data. We outline how the FSM implementation was modified to identify sequences of interest and to allow for a more human readable output from the algorithm. Algorithm run- time performance issues that arise with our approach are also outlined and a number of preprocessing techniques are described that can be used to alleviate these problems. Furthermore in this section we also discuss how postprocessing techniques can be ap- plied to give more value to the FSM output. Section 5.5 shows how the amount of data produced during monitoring can be significantly reduced by applying clustering and statistical analysis techniques. This summarised data can be used to further enhance the design model. The penultimate section (5.6) in this chapter presents a description of the reconstructed design model of the system, and the information that it captures. Finally section 5.7 compares the work presented in this chapter to related research in the areas of reverse engineering and data mining. 5.2 Automatically Extracting Component Relationships and Object Usage Patterns Run-time paths capture the run-time behaviour of an application and can be easily represented in a graphical format. Figure 4.12 on page 78 shows a run-time path con- verted to a UML sequence diagram which captures the relationships between the dif- ferent components for a given user action. Run-time paths are represented at a code 90
- 103. Figure 5.1: Run-timeDesign Meta Model from Chapter 3 level by a tree like data structure. A root node represents the first component in the path, which has an ordered list of callees. Each callee is itself a node which can also have an ordered list of callees. The run-time path structure can be traversed to identify the different component relationships that exist within it. The run-time path structure is traversed by visiting each node as it appears in the path. For each node along the path we identify the direct caller of this node and the node’s callees. By analysing all run-time paths collected we can build up a collection of all the component rela- tionships that existed during execution (of a particular test run). This information can be represented in a UML class diagram which shows the overall system architec- ture of the executed components and 7.3). During analysis instances of a Component data structure are created which contain this information. The Component data struc- ture also contains information extracted from the components’ deployment descrip- tors (see section 4.3.2). The instances of the Component data structure are added to the run-time design model. The run-time design meta model diagram which describes the run-time design model from chapter 3 is shown again in this chapter in figure 5.1 for convenience. 91
- 104. Next we givean example run-time path and example deployment descriptors for the components in the path. We show the information that would be extracted during analysis of this run-time path. Figure 5.2 (a) gives a sample run-time path and figure 5.2 (b) gives an extract of the deployment descriptors of the components that make up the run-time path (note there is no deployment descriptor for the database compo- nents). The run-time path contains three different components. For each component an instance of the component data structure (figure 5.2 (c)) is created. The component data structure contains information such as the component name, type, list of callee components, list of caller components and a list of the executed component meth- ods. Figure 5.2 (c) gives an extract from a Java implementation of the data structure, omitting details related to constructors and accessor methods (note all data structures presented in this chapter are presented in this way). Two tasks are performed to pop- ulate this data structure: (a) the component type and component name are extracted from the deployment descriptor and (b) the run-time path data structure is traversed to identify a component’s callers, callees and executed methods. The PAD tool uses recursion to traverse the run-time path data structure. Figure 5.2 (d) shows the infor- mation which would be extracted for each component from the sample run-time path (a) and deployment descriptors (b). By repeating this process for all run-time paths recorded a picture of all the component relationships that existed during the test run can be built up. Object usage patterns can also be identified by traversing the run-time paths (pro- duced using COMPAS BCI, see section 4.2.5.8). For each object type that we track, we can mark any points along the path where an instance of this object is (a) created and (b) accessed. This information can be stored in a TrackedObject data structure. An instance of this data structure contains information on the object type, a list of the call paths where it has been created and accessed, a corresponding list of the object methods accessed in each path and (depending on the granularity of the information required) the points/positions along the path at which the objects were accessed and created. A diagrammatic representation of this information is shown in figure 4.12 on page 78. Next we give an example run-time path where an object is tracked and we show the information that is extracted and added to the run-time design model. Figure 5.3 (a) gives a run-time path where the AccountDetails object is being tracked. For each tracked object type an instance of the TrackedObject data structure (figure 5.3 (b)) is created. The TrackedObject instance is populated by traversing the run-time path data structure recursively. Each node along the path is visited and counters are in- cremented where instances of the tracked object type are accessed or created. In fact individual counters are maintained for each different accessor method. Figure 5.3 (c) gives the data that would be extracted for this example. When the run-time path is traversed the tracked object name and path ID are identified and the total number of accesses and creations are counted. Also for each accessor method the method name 92
- 105. Figure 5.2: ExampleRun-Time Path (a), Example Deployment Descriptors (b), Extract of Component Data Structure (c) and Data Extracted to Populate Data Structure (d) 93
- 106. Figure 5.3: ExampleRun-Time Path (a), Extract of the TrackedObject Data Structure (b) and Information Extracted to Populate the TrackedObject Data Structure (c) 94
- 107. Figure 5.4: ExampleRun-Time Path (a), Extract of the RunTimeContainerService Data Structure (b), and Information Extracted to Populate the RunTimeContainerService Data Structure (c) 95
- 108. is recorded alongwith the number of corresponding accesses made to this method. The above information is added to the TrackedObject instance, which maintains a list of objects, where each object in the list contains tracking statistics for a given run-time path (RTPStatsList). 5.3 Reconstructing Run-time Container Services Contextual component frameworks provide services to components at run-time (e.g. checking and ensuring that component boundary constraints are being met). In EJB such boundary constraints include security restrictions and transactional isolation checks. For example, an EJB component method may have the following transactional attribute: (transaction) Required (i.e. the method will participate in the client’s trans- action, or, if a client transaction is not supplied, a new transaction will be started for that method). Such attributes are defined during deployment, (specified in the XML deployment descriptor) and do not change during run-time. By analysing the differ- ent component attributes, along with run-time paths, it is possible to reconstruct the different container checks as they occur along the paths. For example, by analysing the transactional attributes of each component method along a particular run-time path, one can easily reconstruct the transactional boundaries (i.e. where transactions begin and end) along the path. This information can be used by developers to easily identify how the container services are being used by their application. Since a high proportion of the application run-time can be spent executing such container services [9] it is important that the services are utilised in an efficient manner. Inefficient use of such services can lead to well known antipatterns (e.g. the large transaction antipat- tern [61]). A RunTimeContainerService data structure is created during analysis which contains information in relation to the reconstructed services. The information can in- clude the service type, the path ID in which the service occurred and the service start and end points along the path, as well as the methods that make use of the service. Next we give an example run-time path and the related component service require- ments. We show how the run-time service is reconstructed and the information that is extracted and added to an instance of the RunTimeContainerService data structure. Fig- ure 5.4 (a) gives an example run-time path and corresponding transactional require- ments for each EJB component method in the path. The transactional requirements are extracted from the XML deployment descriptor. For each new run-time container service reconstructed an instance of the RunTimeContainerService data structure (figure 5.4 (b)) is created. To populate the data structure the run-time path data structure is traversed and the required information extracted. Figure 5.4 (c) gives the information that would be extracted for the example run-time path. When a service is identi- fied its start and end points are recorded, along with the methods within the service boundaries. This information along with the path ID is added to the RunTimeContain- 96
- 109. erService instance, whichmaintains a list of objects, where each object in the list con- tains run-time container service statistics for a given run-time path (RTPStatsList). If a reconstructed container service is identified which is equivalent to one which has al- ready been identified (and a RunTimeContainerService data structure has already been created) a new instance of the RunTimeContainerService data structure is not created. Instead the reconstructed container service statistics are added to the instance that has already been created. Two reconstructed container services are equivalent if the con- tain the same sequence of methods in the MethodList, are of the same Length and both are of the same ServiceType. 5.4 Identifying Component Communication Patterns in Run-Time Paths using Frequent Sequence Mining Identifying component relationships allows for the construction of class diagrams which show a high level architecture of the application, as in figure 5.5. What is not clear from the class diagram in figure 5.5, however, is the communication patterns or frequency of calls between the different components in the system. This type of infor- mation is often required when trying to identify particular performance issues in the application. It is important to be able to identify parts of the application where there are high or unusual levels of communication between components as knowledge of such patterns can lead to opportunities for optimizations (see section 7.3). Commu- nication patterns between components (e.g. long running loops) can be identified by manually analysing run-time paths. An issue with manually analysing run-time paths from enterprise applications, however, is that (a) the paths can be very long and (b) there may be a large number of different paths that exist for a given system. Rather than having to analyse the paths manually, it is desirable to be able to perform automatic analysis, such that the amount of data to be examined by developers can be reduced. From a performance design perspective we are interested in finding fre- quently occurring method calls across the run-time paths that might suggest instances of potential performance design flaws in the application. That is, frequent sequences of method calls that are resource intensive. Identified design flaws can be refactored or optimised such that the overall system performance can be improved. 5.4.1 Frequent Itemset Mining and Frequent Sequence Mining Frequent Itemset Mining (FIM) [2] is particularly suited to finding patterns in trans- actional data. Transactional data in the data mining context refers to a database of transactional records. For example a database of different customer shopping trans- actions on a given day (known as market basket data) or a database of individuals banking transactions. In relation to this research a transactional database refers to a 97
- 110. Figure 5.5: ClassDiagram Showing Components Relationships database of run-time paths. However it is important not to confuse a transaction in the data mining context with the meaning of a transaction in the computer software context [148]. Figure 5.7 shows a transaction in the data mining context with 9 items. For the purpose of this research each item represents a component method call in a run-time path. However the item could as easily be used to represent other informa- tion such as shopping items in a customer’s basket. A transactional database consists of numerous such transactions. An issue with FIM in relation to applying it to run-time paths is the fact that it does not take the order of items in the paths into account. Since a run-time path maintains the order of the events that constitute it, it is important that our analysis technique also respects this order. Mining frequent item sequences [3] in transactional data considers the order of the items in the transactions and thus is more suited to finding patterns in run-time paths. FSM is a general case of FIM. FIM is concerned with discovering all frequent itemsets within a transactional database. Most FIM algorithms work on the following principle: an itemset X of variables can only be frequent if all its subsets are also frequent (i.e. the downward closure property). Using this principle the general approach is to find all frequent sets of size 1. Assuming these are known, candidate sets of size 2 can be generated (i.e. sets {A, B} such that {A} is frequent and {B} 98
- 111. is frequent). Thefrequency of the candidate sets can then be calculated by scanning the database. Infrequent candidate sets are removed. This gives the frequent sets of size 2. Next the candidate sets of size 3 can be constructed and their frequency calculated. This process can be repeated until no candidate sets are generated. This is known as the Apriori algorithm [2]. Calculating the frequency of a candidate itemset is referred to as determining the support count of the candidate. The support count of the candidates are calculated by reading in each run-time path from the database and determining if each candidate is contained within the run-time path. Where this is true the support count is incremented. For item sets the containment relation corresponds to the set inclusion (⊆) relation. In the case of item sequences, a sequence a1, a2....an is contained in another sequence bj1, bj2....bjm if there exists integers 1 < j1 < j2 < ... < jn <= m such that a1 = bj1, a2 = bj2, ...an = bjn [3]. 5.4.2 Support Counting for Run-Time Paths The support counting performed in FSM is useful for identifying frequent sequences that occur across different transactions. However an issue with this approach is that it does not take into account sequences which might repeat within a given transac- tion i.e. if a sequence appears twice in a particular transaction the support count will have only been incremented by 1. In some situations it can be useful to also take into account repeating sequences within the same transaction. This is especially the case when applying FSM to run-time paths since often we would like to be able to iden- tify repeating sequences of method calls within a given run-time path (e.g. loops). Weighted support counting can be applied to increment the support of an item, such that for every instance of the sequence within a given run-time path the support is incremented. A consequence of this is that the support count performance suffers significantly (see section 7.3.1). With non-weighted support the support count scans each run-time path and stops as soon as it finds the item sequence. With weighted support however each run-time path is searched completely for all instances of the item sequence. Another issue that arises from using weighted support is that the output generated can be difficult to comprehend. This results from the fact that, with weighted support counting, the support is incremented for every instance of the sequence that occurs in a given transaction. This problem can be clearly seen from a simple example. Figure 5.6 shows a single transaction containing 9 items. For the purposes of the example we give the non-weighted support count and weighted support counts for this transac- tion in respect to the item sequence 1, 2, 3 . For non-weighted support the support count value is 1, whereas for weighted support the value is 10. The weighted support count can be represented mathematically as being at least: (l+(n−1))! (n−1)!(l)! where l is the se- quence length and n is the number of non-overlapping sequences in the transaction. Figure 5.6 shows the different sequences that are counted as part of the weighted sup- 99
- 112. Figure 5.6: ExampleTransaction with Different Support Counting Approaches port. The figure visualises the sequences as they appear in the transaction by repeat- ing the transaction with the sequences highlighted in different colours. The bracketed number that follows each transaction indicates the number of sequences that we visu- alise per line. Clearly it would be difficult to visualise all sequences without repeating the transaction. Furthermore mentally constructing the different sequences that are counted using weighted support can be a cumbersome and confusing task. This is even more evident when either the size of the sequence grows or when the number of non-overlapping sequences within the transaction increases. For example in transac- tion 1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 1, 2, 3, 4, 5 the item sequence 1, 2, 3, 4, 5 has a weighted support count of 21. Relating the weighted support count to the different sequences as they appear in a transaction can be particularly difficult when applying FSM to run- time paths from real enterprise applications where individual transactions may have thousands of items. To address this issue we introduce the notion of non-overlapping weighted support. Non-overlapping weighted support works by incrementing the support count for non-overlapping sequences only. This approach improves upon the weighted sup- port counting in two main ways: (a) the performance of the algorithm improves for databases with long transactions (see section 7.3.1) and (b) the output of the algorithm is more easily understood by somebody trying to interpret the results (without infor- mation loss). For instance, in the example given in figure 5.6 the weighted support for the sequence 1, 2, 3 is 10 where as the the non-overlapping weighted support is 100
- 113. 3. While relatingthe 10 instances of the sequence 1, 2, 3 to their actual occurrences in the transaction can be difficult, for non-overlapping weighted support this task can be quite simple. Figure 5.6 shows the non-overlapping sequences in the transac- tion. Since the sequences do not overlap humans can easily visualise them. The non- overlapping weighted support counting works in a similar way to weighted support counting. For weighted support counting the transactions are read in one at a time. Each candidate sequence is compared with the transaction. For every instance of the candidate sequence that is found in the transaction the support is incremented. For non-overlapping weighted support we prevent the support count being incremented for overlapping candidate sequences by keeping track of the start and end position of the last candidate sequence that was identified within the transaction. The support count is only incremented where the start position of the current candidate sequence is greater than the end position of the previous candidate sequence, that the support count was incremented for. We say there is no information loss since from the non-overlapping weighted support count output one can easily work out the value of the weighted support count by cal- culating the following equation: (l+(n−1))! (n−1)!(l)! where l is the sequence length and n is the non over-lapping weighted support. In situations where a sequence within the trans- action has within it a duplicate element (or elements) of the candidate sequence there will be some information lost, insofar as the real weighted support will be higher than (l+(n−1))! (n−1)!(l)! depending on at what position along the transaction the duplicate element(s) occur as in figure 5.7. We call these elements hidden elements. Figure 5.7: Hidden Elements in Transaction and Corresponding Support Counts 101
- 114. 5.4.3 Further Criteriafor Interestingness In general for FSM the interestingness [87] of a sequence is directly proportional to the frequency of the sequence. However when analysing run-time paths from a perfor- mance perspective we are most interested in finding frequent sequences of methods that consume a high amount of system resources. In this situation the criteria for in- terestingness is not the frequency of the sequence but is in fact the amount of system resources consumed by the methods that make up the sequence. In order to account for the resource consumption per method we can add this information to the algo- rithm input file such that the transactional database contains not only the item IDs but also their corresponding resource consumption information (since resource consump- tion per methods is collected during monitoring and contained within the run-time paths). This information can be utilised during the support counting stage to influ- ence the outcome of the FSM algorithm. During support counting, rather than sim- ply incrementing the support count, we instead add the resource consumption values of the methods in the candidate sequence to the support counter. This weights the support count value by the resources consumed such that sequences that consume a highest amount of resources are found. Section 7.3.1 gives results of applying this ap- proach to run-time paths and shows how it can be used to quickly identify frequently occurring resource intensive sequences. An issue with this approach however is that the downward closure property is invalidated since there may be instances where an item sequence may be frequent even though some of its sub sequences are not [159]. The literature [159] suggests a solution to this problem using the weighted downward closure property for weighted association rule mining. Future work will investigate the applicability of this approach to FSM. Our results show that even though we in- validate the downward closure property the approach can be effective in finding se- quences of resource intensive methods that occur across the transactions (see section 7.3.2). 5.4.4 Preprocessing for FSM Performance Improvement In order to apply FSM to run-time paths it is required to convert the run-time paths from their text format to a flat format or representation (which is used by the FSM implementation [32], described in section 7.3). This conversion involves assigning a unique ID to each distinct method in the run-time paths and listing these IDs in a flat format in the same order as they would appear in the run-time path. How- ever for enterprise systems the run-time path length can be very long (in the order of thousands of items). Run-time paths of this size can have a major impact on the algo- rithm performance particularly in relation to FSM implementations that make use of weighted support or non-overlapping weighted support counting approaches. In or- der to reduce the running time of the algorithm we applied a number of preprocessing 102
- 115. techniques to reducethe run-time path lengths. 5.4.4.1 Using Parental Information to Split Run-Time Paths The flat run-time path file contains the method IDs that make up the call path. From a flat run-time path one can determine the calls that make up the path and the sequence of the calls. Information is lost in the translation from a run-time path to the flat for- mat, however, since we can not determine the caller (or parent) of each method in the flat run-time path file. This information is clearly available in the original run-time path. Parental information is interesting from a performance analysis point of view, since frequent methods that have the same parent often provide opportunities for op- timization (e.g. loops, since every method in a particular loop has the same parent). In order to be able to identify such patterns in the data easily we can add the parental information to the flat run-time paths such that it is available in an intermediate rep- resentation. This is performed in a preprocessing step. During this process each item in a flat run-time path is augmented with the position of his parent (or caller) method. This information can be utilized, before applying the FSM algorithm. Since we are in- terested in method sequences with the same parent, at the preprocessing stage we can split the run-time paths in the intermediate representation into a number of smaller subpaths by their parent information. i.e. for each parent we create a new subpath that contains all the children of the parent. The FSM algorithm can then be applied to the subpaths to identify frequent or interesting sequences with the same parent. This has two main advantages: (a) it allows us to quickly focus on sequences of methods that have the same parent and which we are particularly interested in and (b) it re- duces length of each run-time path significantly which improves the performance of the search (see section 7.3.1). A downside to this approach is that frequent sequences with different parent methods are not identified. If it is required to keep a list of the run-time path IDs that a frequent sequence ap- peared in, it is possible to reconstruct the split run-time paths after the algorithm has completed and to list the frequent sequences along with the run-time paths in which they appear. Maintaining such a list is a common feature of mining algorithms and is also very useful in run-time path analysis where a developer may want to quickly analyse the run-time paths in which the frequent or resource intensive sequences oc- curred. 5.4.4.2 Consecutive Item Replacement Consecutive Item Replacement (CIR) can be used to further reduce the run-time path lengths. CIR is an iterative approach whereby we perform a number of runs of the FSM algorithm. The first run of the FSM algorithm is aimed at finding a small number of the most frequent sequences in the database. The amount of time spent in the first 103
- 116. run is generallyquite short and can be achieved by either setting a high support count (close to the min. support count for the most frequent sequences of size 2 or larger) or by limiting the size of the sequences that are generated to a particular length. The next step is to identify if any of the frequent sequences found, in the first run, occur as consecutive sequences in the database. If this is true the consecutive sequences can be replaced in the database by a new item ID (replacement item) which we use to rep- resent this sequence in the next iteration of the FSM algorithm. Replacing frequent consecutive sequences reduces the length of the run-time path and as a result the next iteration of the FSM algorithm will be faster than the last. This iterative approach can be repeated until either an acceptable performance run-time (for the FSM algorithm) is attained or until no further improvements can be made. The performance improve- ment depends on a number of factors such as the run-time path length, length of the consecutive sequences replaced, number of consecutive sequences replaced, database size etc. This approach is especially effective in databases of run-time paths where long running loops may appear. On completion of the final iteration of the FSM algorithm any replacement items that exist in the algorithm output can be replaced with the original consecutive sequences to give the real frequent sequences that occur within the data. 5.4.5 Closed Sequences In some instances a large number of frequent sequences can be generated as a result of the FSM algorithm (see section 7.3.1). Ordering the generated sequence by their support value will however highlight the most significant sequences (i.e. the most frequent or resource intensive). Often however a sequence will have the same support as many of its subsequences, and thus they will all be presented together, if ordered by support. This gives much redundant information which can limit the value of the results produced. By identifying closed sequences only this problem can be avoided. Closed Mining has been mainly applied to the FIM problem. An Itemset I is closed if no proper superset of I exists that has the same support as I. The set of closed itemsets is a compact representation of the frequent itemsets. All frequent itemsets together with their supports can be generated if only the closed itemsets and their supports are known. Closed Itemset Mining can significantly reduce the number of sequences output and avoids the problem we outlined above. It has been shown that closed itemsets can be generated in a post processing step [181]. However, more efficient approaches have been presented which modify the FIM algorithm (e.g. [182]). Closed mining can also be applied to sequences [180]. Our plans for future work involve investigating these approaches such that they can be applied to run-time paths. 104
- 117. 5.4.6 PostProcessing A numberof postprocessing techniques can be applied to the data produced from FSM. Postprocessing can be applied to give more value to the sequences such that interesting sequences can be more easily identified. For example the confidence of a sequence (rule) can be calculated as follows sequence: confidence (s1 => s2) = support(s1) ∪ support(s1)/support(s2) [40]. The confidence, C, of the se- quence rule above holds true if: C% of run-time paths that contain s1 also contain s2. The confidence of a rule is sometimes referred to as its accuracy. This infor- mation is particularly useful when analysing design of enterprise systems since it can be useful to identify methods that appear together with a certain level of confi- dence. For example if we have remote method A and remote method B, and the rule: support(MethodA) ∪ support(methodB)/Support(a) = 100%, we can be sure (with a confidence of 100%) that method A and method B will always occur together. Seeing as A and B are both remote methods and seeing as they occur together, there may be an opportunity to combine these methods into a single more coarse grained remote method call and thus cut down on network traffic. Another postprocessing technique that can be applied, in relation to run-time paths, is adding parental information, which is contained in the paths, to the identified se- quences. For example parent components and methods of the sequence can be iden- tified, i.e. components and methods that called the sequence in the run-time paths. This information can be stored along with the sequence for future use. 5.4.7 Component Communication Information for the Extracted Design Model In chapter 7 we show how the information extracted by the FSM algorithm can also be used for automatic identification of performance antipatterns. For automatic antipat- tern detection the analysis module creates a FrequentSequence data structure which contains information relating to the frequent sequences identified in the run-time paths (see figure 5.1). An instance of the data structure contains the path IDs of the sequence (i.e. in what paths the sequence occurs), the items (i.e. the component meth- ods) that make up the sequence, the parents of the sequence, the support of the se- quence (i.e. how often the sequence occurs, or how much resources were consumed). The support of the sequence can be broken down further to reflect how often the se- quence occurs in the different run-time paths. Confidence rules can also be created where appropriate (e.g. for sequences containing remote methods). 105
- 118. 5.5 Data Reduction Theamount of data produced during monitoring large scale enterprise applications is often too large for easy human analysis. Thus it is required that the data be reduced and summarised such that it can be presented to developers and system testers in a more meaningful format. Summarising the data also allows for the data to be more easily integrated with a design model. 5.5.1 Clustering Run-time Paths Run-time paths collected from enterprise applications can often be too long, or there may be too many, for easy human analysis. Considering the number of software com- ponents and sub components in an typical enterprise application (this can be in the order of hundreds), the number of paths that can be taken through the system is gen- erally quite large. Similarly the length of a run-time path corresponding to a partic- ular user action can also be very long (since a large number of component method calls may be needed to service a user request). The issue of path lengths is somewhat addressed through frequent sequence mining, since repeating sequences in a given path can be identified and represented in a more concise manner (see loops in figure 4.12 on page 78). To address the issue of having a large number of paths we can apply another data mining technique called clustering [87]. Clustering is the classification of similar objects into different groups, or more precisely, the partitioning of a data set into subsets (clusters), so that the data in each subset share some common traits. Although there can be a large number of unique paths through the system many of these paths can be similar and may be constructed of many of the same sub-paths. Using basic clustering analysis we can reduce the number of paths significantly into common path groups. In section 7.3.3 we show how this can be achieved for run-time paths, collected from monitoring a JEE system, when we cluster paths together that (a) call the same component methods and (b) make use of the same components. During analysis we create a Cluster data structure (see figure5.1). An instance of a Cluster data structure contains all the run-time path IDs that belong to a particular cluster. This data structure also contains information relating to the clustering criteria (e.g. for (a) above the data structure contains the list of component methods that are called by the run-time paths in a particular cluster). Instances of the cluster data structure can be added to the design model. This allows a developer to see (a summary of) the different paths that are taken through the system (e.g. at a method or component level), without the need to analyse all run-time paths recorded. 106
- 119. 5.5.2 Statistical Analysis Awide range of performance metrics can be collected when monitoring large enter- prise applications. For example our monitoring module can collect performance met- rics such as component memory usage, component CPU usage, and a wide range of server resources (object instance pools, thread pools, instance caches). During a test run the amount of information that is collected can be large, especially when a system is under heavy load. Statistical analysis can be used to summarise resource usage and server resource information. For example for each method we can get the average methods response time/CPU usage/memory usage, maximum and minimum val- ues, as well as the standard deviation. The same analysis can be applied to queues for server resources, e.g. object pool sizes, database connection pools. The data structures created during analysis can be enhanced with statistical values for the component methods/server resources that they contain. The statistical values are calculated by analysing the performance metrics collected with the run-time paths and by analysing the server resource usage information that is collected. Although the JEE Management Specification provides for a Performance Data Frame- work (see section 4.3.1) which mandates how server resources should be collected and presented, in practice statistical information is often not available. Thus, we are required to perform this analysis for server resources. 5.6 The Reconstructed Design Model The output from the analysis module is a detailed design model that captures the re- lationships and patterns identified during analysis, as well as the reconstructed con- tainer services, path clusters and statistical information. We call this a run-time design model since it captures a snapshot of system design decisions that are realised as the system executes during our test run. Figure 5.1 gives a diagram of the different data structures that are contained in the design model. Figure 5.1 contains 8 entities with the following relationships: A Component entity can call zero or more other Compo- nents, is made up of zero or more component Methods and can be associated with one or more object Pools. A RunTimePath entity is made up of one or more component Method calls. The RunTimePath can contain zero or more FrequentSequences (made up of a sequence of one or more Methods) , the FrequentSequences can in turn belong to 1 or more RunTimePaths. Likewise a RunTimePath can contain zero or more TrackedObjects and TrackedObjects can be tracked in one or more RunTimePaths. A RunTimePath can belong to a single Cluster and a Cluster is made up of one or more RunTimePaths. Fi- nally, a RunTimeContainerService consists of one or more Method calls within particular service boundary constraints (determined by meta-data) and can be present in one or more RunTimePaths. 107
- 120. The extracted designmodel gives two main views into the system. A transactional view and a hierarchical view. The transactional (RunTimePath) view of the system shows the main paths (Clusters) through the system, the most frequent communication patterns along these paths and how particular (Tracked) data objects are being created and used along the transactions. From the transactional design view one can also determine the container level services that are used along the different transactions. A more abstract hierarchical view of the system can also be obtained from the design model, by analysing the Component details and the component callers and callees. This shows the different component types and relationships that make up the system. The model also shows how the components are being made available by the underlying container through the different container pools. 5.7 Related Work In this section we discuss and compare related work. We focus on two related areas in particular, reverse engineering and data mining. 5.7.1 Reverse Engineering Our work applies analysis to reverse engineer the system’s run-time behaviour and structure from data collected by our monitoring module. Our approach mainly makes use of dynamic data to extract the required information for design recovery. In this subsection we outline and compare a number of related approaches. Our research is closely related to work presented by Schmerl et al. [149]. In this litera- ture the authors present an approach for architecture recovery using runtime analysis. Their approach requires an engineer to create a mapping specification between low level and architecture events. A run-time engine takes as input the mapping specifica- tion and monitoring events from a running system and produces architecture events. Finally, the architecture events are used to automatically create an architecture de- scription. The work focuses on identifying architectural structure and does not focus heavily on architectural behaviour. The authors have applied their approach to dis- cover the architecture of an EJB application. A number of drawbacks exist however for this approach. Firstly, an effort is required by an engineer to develop the mapping specification. For example it took an expert in their mapping specification language three days to construct a mapping for a sample EJB application. The engineer is re- quired to understand the format of the monitoring data, and the workings of the ap- plication to be analysed. Furthermore, in the case of EJB applications, the engineer requires an understanding of the application server on which the application is run- ning. Secondly, the approach is not portable as a new specification is required per application, albeit the authors claim only minor modifications of the specification are 108
- 121. required for applicationsrunning on the same middleware. For example, in the case of the EJB application, they claim that the same specification could be easily modified with little effort such that it could be applied to another EJB application running on the same application server. However this would not be the case if the system was running on an alternative application server. Thus the approach is less portable across different middleware implementations. However, a key advantage of this approach is that it can be applied to any system that can be monitored to provide low level data such as object creation, method invocation and instance variable assignment. Briand et. al present work on reverse engineering sequence diagrams from distributed [35] and multi-threaded [34] java applications. Their approach, similar to our analysis module, is based on populating instances of a meta model with information collected by monitoring a running system (their monitoring approach has been discussed in section 4.4). More precisely, the data collected at run-time is used to create an instance of a trace meta model which in turn is transformed into an instance of a scenario meta model. The scenario meta model is an adaption and simplification of the UML se- quence diagram meta model. The output of their approach is scenario diagrams. They have performed a case study to show how reverse-engineered scenario diagrams can be used to check the quality of an applications implementation and its conformance to the intended design. Our monitoring tool, COMPAS JEEM, produces a similar output (in the form of run-time paths), that when represented diagrammatically results in a sequence diagram. We use this information (along with collected performance metrics and component metadata) to populate an instance of our run-time design meta model which gives a structural and behavioural view of the overall system design from a per- formance design perspective. Their approach also addresses the issue of identifying repeating sequences in a trace as a result of loops by instrumenting the control-flow structures. On the other hand our approach makes use of FSM to identify repeating patterns. An advantage of their approach is that they can distinguish the execution of loops from incidental executions of identical sequences in different contexts. Our approach can not make this distinction, however, it is less intrusive since it does not require such low level instrumentation. Static and dynamic data has been applied by Richner and Ducasse [146] to recover high-level views of the system architecture through user defined queries. Both the dynamic and static information is modelled in terms of Prolog facts. They provide a logic layer made up of rules above the database of facts to allow sophisticated queries about the structure and behaviour of the application. The main advantage of their work is that it allows an engineer to drive the investigation of the application through an iterative process, much like Murphy’s static relexion approach [122]. In compari- son our approach makes use of mainly dynamic information. Although source code analysis is not applied, we also make use of component meta-data to obtain structural and functional information on the different components. We also extract information pertaining to the components’ required container services. 109
- 122. The reconstruction ofcontainer services using component meta-data and, what the authors call, binding graphs has previously been applied by Trofin and Murphy [166] to identify redundant boundary checks within contextual container services. Binding graphs (similar to call graphs) contain component dependencies and can be collected statically or dynamically. We reconstruct the container services to identify antipat- terns rather than for redundancy checking, and in contrast to their approach our re- construction process requires the order of calls to be maintained. Thus we make use of run-time paths rather than binding graphs for our reconstruction process. 5.7.2 Data Mining In this thesis we have applied both FSM and clustering to data produced from moni- toring component based enterprise applications. FSM is applied to identify interesting component communication patterns. We have modified the support counting tech- nique of the FSM algorithm to identify repeating sequences within a given run-time path such that patterns as a result of long running loops, for example, can be identi- fied. Previous work has addressed this issue of counting the support of repeating se- quences within data using a sliding window approach [111]. However this approach has been applied to event streams and is not suitable for data that fits naturally into distinct transactions (e.g. run-time paths). We have also weighted the support counting to take into account performance metrics (and not merely frequency) such that resource intensive patterns can be identified. The idea of weighting support count to take account of alternative criteria other than frequency has been previously been applied to association rules in the literature [159]. Other examples of weighting the support with criteria other than frequency include [107] and [109] where the support is weighted by time values. In this work we introduce the notion of non-overlapping weighted support. Non- overlapping weighted support allows for a more human readable output of the FSM algorithm when repeating sequences within a single run-time path are accounted for. Similar to our non-overlapping weighted support approach, mining frequent closed sequences can produce more compact results [173]. In fact the sequences produced using our non-overlapping weighted support approach can be further compacted by filtering the results produced for closed sequences only. Maximal sets are a subset of all closed sets and maximal mining techniques can be applied to reduce the output volume further [41]. To reduce the run-time of FSM when applied to long run-time paths we have ap- plied a number of preprocessing techniques. In particular we have made use of parental information to split the run-time paths into shorter subpaths. We have also applied a technique called Consecutive Iterative Replacement (CIR) which replaces frequent consecutive item sequences with replacement items to reduce the run-time 110
- 123. path length. Reducingthe run-time of sequential pattern mining algorithms by taking advantage of consecutive sequences has been previously applied and documented in the literature [108]. The authors propose an extension of the SPADE algorithm for FSM called GO-SPADE. One advantage of our Consecutive Item Replacement approach is that it works by modifying the database of run-time paths and no modification of the FSM algorithm is required. To the best of our knowledge the use of parental informa- tion has not previously been applied to improve the run-time of FSM algorithms by reducing the length of the algorithm input data. In this work we have also made use of clustering to reduce and summarise the data collected during monitoring. That is, we have applied clustering and statistical anal- ysis to run-time paths to group the large volume of different paths produced. Chen et.al [45] have previously applied clustering and statistical anslysis to run-time paths collected from monitoring enterprise systems for the purpose of problem determina- tion (i.e. identifying faulty software components). They make use of data clustering and statistical analysis techniques to correlate failures of requests to the components most likely to have cause them. 111
- 124. CHAPTER SIX Detecting Performance Design andDeployment Antipatterns In this chapter we discuss how antipatterns can be detected in an extracted run-time design model. The detection process is based on a rule engine approach whereby antipatterns are pre-defined, in an antipattern library, as rule-engine specific rules. Examples of how rules can be written for antipattern detection are provided. The approach is easily extensible and user configurable. In fact users can configure the de- tection module to filter uninteresting antipatterns through antipattern threshold con- figurations. User defined rules can also be added to the antipattern library such that domain or application specific antipatterns can be identified. The PAD tool usage modes (i.e. single user and multi user) are also discussed. In this chapter the JEE performance design and deployment antipatterns have also been categorised into groups related to the data required to detect them. The final section of this chapter discusses related work. Main Points • JEE performance design and deployment antipatterns are categorised into groups based on the data required to detect them. • Antipatterns are defined as rules and added to an antipattern library. The an- tipattern library can be extended with user defined antipattern descriptions. • Example rules are provided. • User defined threshold values can be used to filter out insignificant antipatterns. • The PAD Tool can be used in both single and multi user mode. For the detection of certain antipatterns multi user mode is required. 112
- 125. 6.1 Antipatterns Categories Inthis section we categorise the antipatterns, that we detect, based on the similarities in the type of data used to detect them. This categorisation was a result of analysing JEE performance design and deployment antipatterns from the literature 1 [161] and [61]. In the following section we give examples of rules that can be used to detect antipatterns from a number of the different categories. 1. Antipatterns Across or Within Run-Time Paths: In order to detect the antipat- terns in this category, information is required on how often particular compo- nents or services occur within the same, or across the different run-time paths. For example, a large number of session beans occurring across the different run- time paths may signify the existence of the Sessions-A-Plenty antipattern [61] (i.e. the overzealous use of session beans, even when they are not required). An- other example of an antipattern in this category is the Large/Small transaction [61] antipattern whereby an inefficient transaction size is set resulting in either large, long living transactions or many inefficient short living transactions. 2. Inter-Component Relationship Antipatterns: The inter-component relation- ship antipatterns can be identified by analysing the relationships that exist between the different components in the system. Where inappropriate rela- tionships exist antipatterns can be detected. A typical example might be the Customers-In-The-Kitchen Antipattern [161] where web tier components di- rectly access persistent objects (e.g. Entity Beans). Other antipatterns that can be detected by analysing inter-component relationships include the Needless Ses- sion Antipattern (discussed in section 6.3), the Transparent Facade Antipattern [61] and the Bloated Session Antipattern [61]. 3. Antipatterns Related to Component Communication Patterns: This category of antipatterns can be identified by analysing the communication patterns be- tween particular component types. For example a high level of fine grained chattiness between remote components (i.e. the Fine Grained Remote Calls An- tipattern, which is also known as the Face Off Antipattern [161]). Another ex- ample might be an unusually bulky or high amount of communication between the business tier and the database which could signify the existence of an Appli- cation Filter/Join Antipattern [161]. Other examples include the Eager Iterator Antipattern [161]. 4. Data Tracking Antipatterns: The data tracking category of antipatterns can be detected by analysing how data objects are created and used across particular run-time paths. A typical example of this antipattern is an unused data object, 1Precise Java Website, http://www.precisejava.com 113
- 126. whereby a dataobject is created and passed along the run-time path, but the in- formation which it contains is never accessed or used. An EJB specific variation on this antipattern is called aggressive loading of entity beans, whereby an en- tity bean is loaded but only a small proportion of the persistent data it contains is ever accessed 2. 5. Pooling Antipatterns: Antipatterns in this category can be detected by analysing the related pool size and queue information. For example, to deter- mine if an inefficient pool size has been specified we need to consider the pool size, the number of object instances being used, on average, and the average queue length. Examples of these antipatterns include inefficient configuration of any of the container pools (thread pools, session bean pools, entity bean pools, database connection pools)2. 6. Intra-Component Antipatterns: Intra-component antipatterns can be detected by analysing internal component details. Examples include the Simultaneous Remote and Local Interfaces Antipattern [161] or the Rusty Keys antipattern [161]. In both these cases the antipattern can be identified by analysing the com- ponent meta data. 6.2 A Rule Engine Approach for Antipattern Detection The detection module is responsible for identifying performance design and deploy- ment antipatterns in the extracted design model. Detection can be achieved through the use of a knowledge base or rule engine. The PAD tool makes use of the JESS rule engine [69] for this purpose. JESS is a Java based rule engine, whereby information can be loaded into the rule engine in the form of JESS facts. Rules can be defined to match a number of facts in the knowledge base. If a rule matches the required facts, the rule fires. When a rule fires, particular tasks can be performed. Such tasks are out- lined in the rule definition. The information captured in the extracted design model can be easily loaded into the rule engine by converting the entities in the model to JESS facts. JESS will accept Java objects as input and automatically convert them to JESS facts. During analysis the model entities are created in the form of Java objects. The objects created are loaded into JESS by the detection module. To detect antipatterns in the design, rules that describe the antipatterns must be written and loaded into the rule engine. Jess provides for both forward and backward-chaining rules. Rules are generally in the form of ”if-then” statements, where ”if” a number of conditions (specified in the rule) match facts in the database, ”then” the rule fires. Forward-chaining rules fire when the conditions fully match the facts in the database. Backward-chaining rules 2Precise Java Website, http://www.precisejava.com 114
- 127. however are goalseeking rules since, if a rule is only partially matched and the rule engine can determine that firing some other rule would cause it to be fully matched, the engine tries to fire that second rule. With backward-chaining rules the engine actively tries to make the rules fire. Rule-based systems such as Prolog based systems support backward-chaining rules. Forward-chaining rules are often described as rules that discover the conclusions drawn from an existing set of facts, whereas backward- chaining rules are described as rules that search for the premises from which existing facts can be derived. The antipattern rules that we have implemented are all forward-chaining rules. This approach was chosen as we wanted the rule library to be easily understandable and extensible. Forward-chaining rules are self contained and do not require other rules to fire in order for the rule to match a set of facts. Thus rules in the library and the corresponding antipatterns have a one to one relationship. This allows for users of our approach to easily relate each rule in the library to the corresponding antipattern it describes. (defrule detect-Local-and-Remote-Intrfs-Simultaneously-Antipattern (?C<-(component (has_local_interface ?LI&:(eq? LI"true")) (has_remote_interface ?RI&:(eq ?RI "true")))) => (printAPSolution "Local-and-Remote-Intrfs-Simultaneously" ?C)) Figure 6.1: Rule to Detect Simultaneous Interfaces Antipattern 6.3 Example Rules Next we give examples of how we specified antipattern rules for a number of differ- ent antipatterns from the categories above. The rules given have been used to detect instances of antipatterns in JEE applications as shown in section 7.4. JESS rules are written in a Lisp like syntax. A rule has two parts separated by the following sign: => The left hand side (LHS) of the rule consists of patterns that match facts. The right hand side (RHS) gives the functions to be executed when the pattern on the LHS is matched. The RHS of the rules shown in this section consists of a function call to the printAPSolution function. This function prints the antipattern description, solution and corresponding contextual information for the particular antipattern detected. The rule in figure 6.1 describes an antipattern from the intra-component antipatterns category. The antipattern described is the Local and Remote Interfaces Simultane- 115
- 128. ously antipattern, wherebya component exposes its business methods through both local and remote interfaces. The detection of this antipatterns is quite simple since it requires the matching of only one fact i.e. is there a component fact that has the value ”true” for both attributes ”has local interface” and ”has remote interface”. (defrule detect-Needless-Session-Antipattern ?C1<-(component (name ?N) (type ?T&:(eq ?T "Session")) (callees ?Callees)(callers ?Callers)) (component (name ?N2)(type ?T2&:(eq ?T2 "Entity"))|?T2&:(eq ?T2 "DB")) (not (test (existsInList ?N2 ?Callees))) (not (test (existsInList ?N2 ?Callers))) => (printAPSolution "Needless-Session" ?C1)) Figure 6.2: Rule to Detect Needless Session Antipattern The rule shown in figure 6.2 is from the inter-component relationship antipatterns category. It describes a situation where a session bean has been used but was not re- quired. In general a session bean is generally only required if there is interaction with the persistent tier (e.g. entity beans or the database components) or if other container services are required. Otherwise a plain old java object (POJO), which is less resource intensive, can be used. To identify this antipattern we try to identify session beans that exist but do not have any relationships with entity or database tier components. Further checks can be made to identify the use of container services. However, we have found that in many situations container services are used by sessions when they are in fact not required (e.g. setting transaction attributes to ”Required” by default), so instead in the rule below we check only for (persistent) component relationships. The rule in figure 6.2 (a) checks for a session bean component, C1 that has a list of caller and callee components, (b) checks for a second component (which we will call C2 for the purpose of the example), that is either an entity bean or a database component and (c) checks if C2 is a caller or callee of C1. JESS allows for the use of user defined functions which can be used to provide more complex functionality to the rules in a concise manner. The existsInList function in figure 6.2 is a user defined function which checks a list (argument 2) for a particular value (agrument 1). Without the use of such functions the rules can become overly complex and difficult to both write and com- prehend. The PAD tool provides a number of user defined JESS functions to allow for the easy construction of rules. 116
- 129. (defrule detect-Bulky-Session-Entity-Comms-antipattern (?C1<-(component (name?N1)(type ?T1&:(eq ?T1 "Session")) (callees ?Callees))) (?C2<-(component (name ?N2)(type ?T2&:(eq ?T2 "Entity")))) (?FS<-(frequentSequence(children ?Ch) (parents ?Ps) (methodIdLists ?Ms) (support ?S))) (test(existsInList ?N1 ?Ch)) (test(existsInList ?N2 ?Ps)) (test(flagHighResourceConsumption "frequentSequence" ?Ms ?S)) => (printAPSolution "Bulky-Session-Entity-Comms" ?FS ?C1 ?C2)) Figure 6.3: Rule to Detect Bulky or Unusual Levels of Database Communication The final rule example given in this section (figure 6.3) is a rule from the antipat- tern category concerned with component communication. In this rule we identify a relationship between a session bean and an entity bean in the form of a frequently re- peating sequence. If this relationship exists, the average resource consumption of the frequent sequence is calculated and is flagged if it is above a user defined threshold. The calculation of the resources consumed is performed by the flagHighResourceCon- sumption function, which is passed the list of methods in the sequence and the support of the sequence. The function refers to a user defined configuration file which specifies the acceptable threshold values. Alternatively if performance metrics are unavailable the frequency of the sequence alone can be used to identify the antipattern. 6.3.1 Antipattern Library An antipattern library is provided by the PAD tool. The library provides rules to detect antipatterns from all categories outlined in section 6.1. Rules can be easily modified or added to the library such that domain specific or application specific antipatterns can be detected. Details in relation to the rule library are given in the thesis appendix. Here we also briefly describe how rules can be added. The rule library also makes use of a number of user defined Jess functions which are available as part of the PAD tool. Details on the Jess functions are also given in the thesis appendix. 117
- 130. 6.3.2 Filtering UsingThreshold Values User defined threshold values can be used to either reduce or increase the levels of antipatterns detected. This can be particularly useful if a high level of performance antipatterns are being detected. A user may only be interested in the most significant antipatterns and can use the threshold values to filter out less significant antipattern instances. User defined thresholds can be defined in a single configuration file. Next we give a number of examples of how thresholds are used by the detection mod- ule to filter out uninteresting antipattern detections. In the antipattern outlined in fig- ure 6.3 a user may only be interested in sequences that consume over a certain amount of the system resources. Thus a user of the PAD tool can define the level of resource usage that should not be exceeded. If sequences are identified which exceed this level of resource consumption they will be flagged as antipatterns. The flagHighResource- Consumption function calculates the amount of resources consumed by the sequence and compares it to the user defined threshold value. Similarly a threshold value can be useful for filtering potential antipatterns from the pooling category. For example thresholds can be used to set levels above which queue sizes, for the various pools, should not exceed. Many such settings are very much application specific and thus it is essential that they are easily configurable. By modifying the configuration files users do not need to understand or edit the rule library. Examples of the types of settings that can be made in the configuration file are given in the thesis appendix. Similarly, the FSM support count can be modified by a user during analysis to increase or reduce the level of frequent sequences identified during the mining process. The FSM support count can be considered an analysis module user configurable threshold value. 6.4 PAD Tool User Modes The PAD tool can be used in two different monitoring modes. Either single user mode or multi user mode. Single user means that there is only one user in the system during monitoring (e.g. a single developer testing the application). Multi user mode requires a load on the system. Antipatterns from categories 1,2,3,4 and 6 can be detected in single user mode. All antipatterns can be detected in multi-user mode and in fact this mode is required to detect the antipatterns in category 5. An added advantage of using multi user mode is that accurate performance metrics can be given to the tool user on detection of an antipattern. Such metrics can be used by the tool user to quickly assess the impact of the detected pattern. Performance metrics can also be collected during single user mode, however they are less reliable since the system is being used in a less realistic manner. 118
- 131. There are twomain drawbacks of using multi user mode however. Firstly, it requires a load to be generated on the system. In most cases this requires the creation of an au- tomated test environment (e.g. using load generation tools). Secondly, a large amount of data is produced when monitoring enterprise systems under load. In particular, our monitoring module produces a large amount of run-time path information. This issue can be addressed however by applying the clustering and statistical analysis techniques outlined in section 5.5. 6.5 Related Work In this section we discuss and compare related work to the research that has been pre- sented in this chapter and to our overall antipattern detection approach. We discuss in particular alternative antipattern categorisations that have been presented, as well as performance testing techniques, detection approaches for pattern and antipattern identification and finally work in the area of antipattern detection closely related to our approach. 6.5.1 Antipattern Categorisation Antipatterns have been categorised in the various literature in which they have been documented. For example, Brown et. al [36], distinguish between development, ar- chitecture and project management antipatterns. The literature [161] and [61] both categorise their antipatterns according to the related component types which are af- fected. For example, antipatterns related to entity beans, antipatterns related to ses- sion beans. In contrast we have taken technology specific (JEE) performance antipat- terns and categorised them according to the data required to detect them. A similar approach has previously been taken by Reimer et al., who have categorised program- ming errors based on the algorithms used to detect them [145]. Our antipattern cate- gorisation also differentiates between performance design antipatterns, performance deployment antipatterns and performance programming errors. Similarly, Moha and Gueheneuc [116] provide a taxonomy in which they attempt to clarify the difference between errors, antipatterns, design defects and code smells. In their analysis they define code smells as intra-class defects and design defects as inter-class defects. Our antipattern categorisation is at a higher more abstract component level. Hallal et. al. [86] provide a library of antipatterns for multi-threaded Java applications. They also distinguish between errors and design antipatterns. They classify the multi-threaded antipatterns that they present following a categorization, which reflects the effect of the antipatterns on the outcome of the tested program. 119
- 132. 6.5.2 Performance Testing Earlyanalysis of systems from a performance perspective has mainly focussed on modelling techniques whereby models of the system are created at an early stage us- ing system artifacts and estimates of resource consumption, user load etc. In the re- search community modelling has been applied to enterprise systems [168] [76], how- ever in reality they are rarely used for performance analysis. This is due to the fact that they can be inaccurate, and time consuming to build and maintain for large enterprise applications. Performance testing of running systems is generally the preferred choice of industry today. Often, however, testing is left until late in the development cycle and as a result issues identified can be expensive to fix and can lead to major project delays. Ideally performance testing should be carried out as early as possible in the development cy- cle. Recent research efforts have focused on early performance testing through the creation of component stubs [58] or through the automatic generation of component test-beds 3. This allows for a running system to be created, deployed and analysed early in the development cycle such that performance tests can be run. These ap- proaches allow for the system design to be analysed at an early stage in development whereby performance issues can be identified even before much of the coding takes place. Our work complements these initiatives since design level antipatterns can be identified in a real running system or in test bed or stub applications. Our antipat- tern detection approach can be applied early in the development process as soon as a running system is in place and should be carried out iteratively as the application evolves during development. The approach we present is suitable for any iterative development processes whereby a running prototype of the application is produced early in development (e.g extreme programming [24]). Current tools for performance testing generally fall into the categories of load genera- tion tools or performance profilers (see section 2.8.2). Current tools merely preset the data collected to the user unanalysed. Thus tool users must perform analysis of the data manually which can be a difficult task. Our work builds on current performance profilers. The PAD tool performs automatic analysis of the data produced during monitoring to automatically identify design issues. 6.5.3 Detection Techniques The detection technique presented in this chapter makes use of a knowledge base/rule engine based approach. A number of previous pattern detection approaches have made use of the logic programming language Prolog for software analysis [103] [49]. Prolog allows for the creation of a knowledge base of facts and for the creation of rules that can be used to query the information in the knowledge base. In general a 3AdaptiveCells/J, http://adaptivecellsj.sourceforge.net/ 120
- 133. software system’s designis represented by Prolog facts and rules are used to express known patterns. Prolog works using backward-chaining. A disadvantage of using Prolog for pattern detection is that the creation of new rules to detect new patterns requires an understanding of the Prolog language. The literature [29] introduces the Relational Manipulation Language (RML) which like Prolog is based on predicate cal- culus. The authors claim that RML is simpler to use than Prolog and can be used for different program analysis tasks such as pattern detection, antipattern detection, cal- culation of design metrics etc. The authors have implemented the CrocoPat tool which interprets programs written in RML. The tool was designed for the analysis of graph models of software systems and it has been shown to scale well on large systems. In comparison our detection approach makes use of very simple forward-chaining rules in the form of ”if-then” statements. We do not require backward-chaining or the use of logic programming. The analysis we perform prior to detection (see chapter 5) allows for simple rules to be written. If analysis was not performed, our rules would need to be more complex in order to identify the relationships which are extracted during analysis. A number of techniques have been applied to identify structural patterns in software. For example, graph matching [138], graph rewrite systems [123], or meta program- ming [11] [98]. The representations used to model the software structure include graph models [138], abstract syntax graphs [123], or design description languages [11] [98]. However such approaches are less suitable for the detection of design and de- ployment antipatterns. This results from the fact that the design model we extract from the running application contains information other than structural information that can not be represented easily when applying these techniques. For example, be- havioural information (e.g. repeating sequences of events), performance metrics, re- constructed container services and object pool information. Instead we have made use of a knowledge based approach which allows for this information to be represented in the form of knowledge base facts. An alterative way of representing this information might be to store it using a relational database. Vokac [172] has previously made use of a relational database for the purpose of structural design pattern detection. Formal concept analysis (FCA) has also been used for the purpose of pattern detection [39] [164]. FCA does not require a library of predefined rules. Instead FCA works by identifying groups of classes that share common relations. FCA approaches try to infer patterns/antipatterns from the code or system design. In contrast, our approach tries to identify antipatterns based on a library of defined antipatterns to be matched. 6.5.4 Antipattern Detection Software pattern and antipattern detection techniques have been discussed in section 2.10. Many of the pattern detection techniques could potentially be applied to detect instances of antipatterns. For example, with most of the pattern detection approaches 121
- 134. discussed, the patternsand application are described in terms of structural (and in some instances behavioural) representations which are analysed to identify instances of the pattern. Since design flaws or antipatterns can also be described in this way, the same approaches can be used to identify them. Some work however has focused more specifically on antipattern detection [83] [49] [117] [114] (see section 2.10). The aforementioned approaches all rely on extracting information from the system source code. A run-time antipattern detection approach very closely related to our work, that ex- tracts information from a running system, has recently been presented. Eologic 4 have developed an antipattern detection tool for JEE applications. This tool identifies a number of general JEE antipatterns and presents the user with possible solutions. This tool, similar to our approach, extracts a model from a running system 5. The tool makes use of JMX and AOP to monitor a running system and to extract an Entity- Relationship-Attribute model. A major drawback of the tool is that it is currently not portable across different application servers. Also it does not detect antipatterns based on communication patterns or object tracking. Furthermore it is designed to be used in single user mode and does not perform data reduction when monitoring applications under load. Other work closely related to our antipattern detection work has been presented by Bischofberger et al. [31] in the form of an architecture conformance checking tool called Sotograph. Sotograph makes use of cycle analysis and metrics based analy- sis to identify architectural level problems in the application. This is performed by analysing the system source code. Many of the issues identified are similar to the an- tipatterns that we try to detect. For example the author’s list architectural issues that can be identified such as large artifacts, small artifacts and bottlenecks. These issues are closely related to a number of antipatterns from the inter-component relationship category that we detect using the PAD tool. Similarly, IBM’s Structural Analysis for Java (SA4J) tool 6 analyses structural depen- dencies of Java applications in order to identify structural antipatterns within the ap- plication. SA4J measures the quality of the application structure by evaluating the web of dependencies between packages, classes, and interfaces of a Java application. The Smart Analysis Based Error Reduction Tool (SABER) [145] is also closely related to our work. SABER uses a combination of domain specific knowledge and static analysis to identify high impact coding errors in large commercial (JEE) applications. This tool has been successfully applied to detect hundreds of errors in these appli- cations. The SABER tool identifies lower level issues (e.g. improper clean up of re- sources, concurrency issues) than detected by the PAD tool. However similar to our 4Eologic’s Eosense for JEE, http://www.eologic.com/eosense.shtml 5Derived Model Analysis,Detecting J2EE Problems Before They Happen, Alan West and Gordon Cruickshank, December, 2006, http://dev2dev.bea.com/pub/a/2007/07/derived-model-analysis.html, 6IBM’s Structural Analysis for Java tool, http://www.alphaworks.ibm.com/tech/sa4j 122
- 135. approach problems detectedare annotated with contextual information such that the issues identified can be better understood. For example the SABER tool provides con- textual path and data flow information which can explain how the defect occurred. 123
- 136. CHAPTER SEVEN Results and Evaluation Inthis chapter we present a range of different tests, and their corresponding results, that we have carried out to validate our work. Results are presented from tests on COMPAS JEEM that show it is a low-overhead and portable monitoring approach. We also show how the output from COMPAS JEEM can be used to manually reverse engineer and identify antipatterns in JEE applications. Furthermore results are pre- sented in relation to tests carried out on the analysis module components. We show how the analysis of the monitored data can be carried out in an efficient and timely manner. The analysis module test results show that the analysis module scales well in relation to data from real enterprise systems in terms of both the output produced and its run-time. In addition the usefulness of component communication patterns extracted during analysis has been highlighted. Our test results show the output from the analysis module can be used to (manually) identify serious performance design flaws in sample and real JEE applications. We also provide results that show how the analysis module can apply data reduction techniques to reduce the amount of data produced during monitoring when the PAD tool is used in multi-user mode. In the penultimate section we present results from applying the PAD tool to two JEE appli- cations. The results show how using our tool we automatically identified a number of performance design and deployment antipatterns in each application under test. Fi- nally we revisit our claimed contributions and discuss how they have been evaluated. 124
- 137. Main Points • Testhave been performed using COMPAS JEEM which show: – COMPAS JEEM is portable across a number of different application servers. Application servers do not spawn new threads for local calls to remote beans. Thus the ThreadLocal approach can be used to tag requests locally. – COMPAS JEEM is a low overhead monitoring approach. – COMPAS JEEM can be used to reverse engineer JEE applications and to manually identify design flaws. • Test have been performed in relation to the Analysis Module components which show: – FSM can be applied to run-time paths in an efficient manner if preprocess- ing is applied. – The output from applying FSM to run-time paths can be used to identify real issues. The FSM output was in fact used to manually identify perfor- mance design antipatterns in two JEE applications, i.e. a sample application and a real enterprise application. – Clustering can be successfully applied to reduce data collected when mon- itoring the system under load. • Test have been performed using the PAD tool: – Three antipattern types were automatically identified in a widely available JEE sample application. Numerous instances of two of the antipattern types were identified. – Antipatterns were successfully identified in a modified version of a JEE sample application. Antipatterns were added to the application for detec- tion purposes. All antipatterns added were identified. – Test results show how six antipattern types were identified in a real large scale JEE enterprise application. Thirty four instances of these antipattern types were identified. – Antipatterns from all categories outlined in chapter 6 have been identified. • The validation criteria outlined in chapter 3 have been met. 125
- 138. 7.1 Chapter Overview Inthis chapter we present a wide range of results that have been carried out in order to validate our research. Section 7.2 gives results from a number of different tests carried out on the COMPAS JEEM run-time path tracing tool (see section 4.2 for a description of this tool). In subsection 7.2.1 we show how we applied COMPAS JEEM to a number of JEE applications and how we could quickly deduce the system structure of the applications by analysing the run-time paths collected. For one of the applications we show that we could easily identify a number of performance design antipatterns from the system structure that was deduced. We also show the portability of our approach (in subsection 7.2.2) by applying it to applications running on application servers from a number of different vendors. In subsection 7.2.3 we give details on the performance overhead incurred by applying COMPAS JEEM to a JEE application. Section 7.3 presents tests carried out on the analysis module components. In partic- ular in section 7.3.1 results are given which show the performance run-time of the different FSM approaches, which have been presented in section 5.4, with respect to a number of different databases. In section 7.3.2 we also show how FSM was applied to data collected from two JEE applications and how we used the results to successfully (manually) identify performance design flaws within them. In particular using our technique we quickly identified a real design flaw in an IBM application (beta ver- sion). We also show (in section 7.3.3) how information collected during monitoring was reduced by our analysis module’s data reduction mechanism, when information was collected under load. In section 7.4 we present results from applying our PAD tool to two JEE applications. More precisely, we detail how the PAD tool was applied to two JEE applications i.e. a sample JEE application (see section 7.4.1) and a real large scale enterprise system (see section 7.4.2). Antipatterns from all the categories outlined in section 6.1 have been detected. For each antipattern detected we give a brief description of the antipattern and the antipattern category. We also give the related information (PAD output) which is presented to the tool user upon detection. Using this information the tool user can easily determine the severity of the problem and a decision can be made as to whether refactoring is required or not. We do not show performance improvements that can be attained by refactoring the antipatterns detected since these improvements have already been well documented [161] [61]. Also performance improvements can be very much application specific and vary greatly depending on the severity of the antipattern. Finally in section 7.5 we revisit our contributions claimed and the validation criteria which we previously outlined. In this section we discuss how our validation criteria have been met. 126
- 139. 7.2 COMPAS JEEMResults In this section we show how we applied COMPAS JEEM to a number of JEE applica- tions. Three subsections follow. The first subsection shows how COMPAS JEEM can be applied to reverse engineer JEE applications, the second provides a portability as- sessment of COMPAS JEEEM and the final subsection give details on the performance overhead incurred by applying COMPAS JEEM to a JEE application. 7.2.1 Deducing System structure We applied COMPAS JEEM to two JEE applications: (1) a sample online banking ap- plication, called Duke’s Bank 1 and (2) a sample e-commerce application, called Plants- ByWebsphere 2. Duke’s Bank is a sample application provided by Sun Microsystems as a showcase for the JEE technology. PlantsByWebsphere is a sample e-commerce application provided by IBM. We monitored both applications using COMPAS JEEM and performed a number of user actions. The run-time paths that corresponded to each user action were obtained. A diagramatic representation of each system was manually constructed by analysing the run-time paths. Duke’s Bank is an online banking application. Modifications were required to fix a number of bugs in this application such that tests could be carried out 3. When a user logs in to the Duke’s Bank application he/she can perform the following actions: view a list of accounts, view an individual account’s details, withdraw or lodge cash, transfer cash from one account to another or finally log off. Each of the different user actions was performed and the run-time paths recorded. Figure 7.1 shows the run- time path associated with the account list user action. It shows the interactions that occur between components to satisfy a user request (for the list of accounts that be- long to a particular customer). The run-time paths collected by COMPAS JEEM also contain related performance metrics (e.g. method execution time as shown in figure 7.1). Such information can be useful for identifying bottlenecks or performance design issues that exist in the application. Five different components make up the run-time path: accountList (web component), AccountControllerBean (session bean), Account- Bean (entity bean), PreparedStatement (JDBC component) and ResultSet (JDBC com- ponent). The run-time path can be easily traversed to identify the relationships that exist between the different components. By analysing six run-time paths we manually produced the diagram in figure 7.2. The paths analysed were produced as a result of a user logging in and performing the 1Duke’s Bank Sample Application, Sun Microsystems, http://java.sun.com/j2ee/tutorial/ 2How to Run WebSphere Application Server Version 5 Samples on DB2, Tom Krieger, April 2003, IBM Developerworks, http://www- 128.ibm.com/developerworks/websphere/library/techarticles/0304 krieger/krieger.html 3Duke’s Bank J2EE Application - Exceptions and Troubleshooting, Ada Diaconescu, http://adadiaconescu.there-you-are.com/dukes-bank/dukesBank-troubleshooting.htm 127
- 140. Figure 7.1: AccountListRun-Time Path and UML sequence diagram 128
- 141. available banking options.After studying the system documentation we expect that the system would be used in this manner. Figure 7.2 shows the design of the section of the application which is responsible for handling the different requests that can be made by banking customers that log in to the system. The diagram was constructed manually by traversing the run-time paths. The testing environment consisted of the JBoss application server (version 3.2.7) and a MySql database (version 4.0.2). Because the web server that comes as part of JBoss 3.2.7 implements the servlet 2.3 specification the web tier is monitored as one software component (see section 4.2.5.1 for further details on why this occurs) as shown in figure 7.2. Figure 7.2: Diagram Showing Components in Duke’s Bank 129
- 142. The diagrams thatwe produced can be used for identifying performance design is- sues within enterprise applications. For example, by analysing figure 7.2 we quickly identified the existence of two EJB performance antipatterns in the Duke’s Bank ap- plication. The diagram consists of four EJBs two stateful session beans and two entity beans. • Antipattern 1: The first antipattern we identified is known as the Conversa- tional Baggage antipattern [161]. It identifies situations where a stateful session bean is used but not required. This can be wasteful since there is a performance overhead required to manage stateful sessions within the EJB container. Often a (more lightweight) stateless session bean would suffice. Figure 7.2 shows that a stateful session bean is invoked for each user request. However there is no ob- vious need for a stateful session bean in any of the user actions we performed, as state is not maintained across the different web pages returned for the related user requests. The stateful session beans in figure 7.2 could easily be replaced by more lightweight stateless session beans which is the suggested solution to this antipattern. • Antipattern 2: The second issue we identified with the Duke’s Bank application was that the entity beans exposed remote interfaces. The entity beans in this application (unlike the session beans) are only accessed by EJBs which would most likely be deployed in the same container. Thus there is no real need for the entity beans to expose remote interfaces. The beans can be refactored to expose local interfaces and thus reduce the overhead associated with remote method calls. PlantsByWebsphere is a sample e-commerce application that comes installed with Websphere 6.1. Websphere 6.1 contains a websever that conforms to the servlet 2.4 specification and thus can monitor inter web component calls. As above we per- formed a number of user actions in the PlantsByWebsphere application and con- structed a diagram by analysing the corresponding run-time paths. In total 19 user actions were invoked. Figure 7.3 shows the diagram that was constructed. The dia- gram shows the different web and business tier components that made up the applica- tion (for simplicity we did not analyse the JDBC components for PlantsByWebsphere). No antipatterns were identified from the diagrams for this application. As shown in figures 7.2 and 7.3 COMPAS JEEM monitors only the main components (JSPs, Servlets, EJBs and JDBC calls) that make up the J2EE system. Smaller utility classes and the underlying middleware calls are omitted. This allows for run-time paths (and thus diagrams) that contain only the main components of the applica- tion. As a result our run-time paths and diagrams are not cluttered with unimportant method calls that play only a minor role in the overall application design structure. 130
- 143. Figure 7.3: DiagramShowing Components in PlantsByWebsphere 131
- 144. 7.2.2 Portability Assessment Inorder to show that our approach is truly portable we applied it to a number of application servers running JEE applications. The table in figure 7.4 shows results from applying our tool to three different application servers, JBoss, Websphere and OC4J. The table gives the server vendor, the server version, results from monitoring the web tier and results from monitoring the business tier. Server Vendor Version Web Tier Business Tier JBoss Group JBoss 3.2.7 As one comp. Remote + Local Comps. IBM Websphere 6.1 Inter comp. comms. Remote + Local Comps. Oracle OC4J 10.1.2.0.2 As one comp. Remote + Local Comps. Figure 7.4: Portability Test Results In relation to web tier monitoring we tested to see if the interactions between web components could be monitored. From the results table it can be seen that only the Websphere application server could achieve monitoring of the individual web tier components i.e. it monitored inter component communications in the web tier. In JBoss and OC4J the web tier was monitored as one software component. This can be explained by the fact that both the JBoss and OC4J versions that we used during testing contain servlet containers that conform to the servlet 2.3 specification. Later versions of the application servers will conform to the servlet 2.4 specification and will allow for complete web tier monitoring as is the case with Websphere 6.1. The testing of the business tier was designed to determine if calls made to a compo- nent through a remote interface from a local component (i.e. within the same JVM) could be traced using our technique based on the thread preservation in EJB remote calls, outlined in section 4.2.5.2. For the three applications servers that were tested, it was found that in situations where components were located within the same JVM all calls made between components could be traced regardless of whether the calls were through local or remote interfaces. Even though the RMI specification stipulates that, ”a method dispatched by the RMI run-time to a remote object implementation may or may not execute in a separate thread”, our results suggest that in the case of current application servers a new thread is not spawned for such calls when the objects are co-located. Since this is the case the approach we outline for tracing run-time paths above can be applied. The monitoring of the EIS tier is achieved be extending the P6Spy open source moni- toring framework. P6Spy can be used to monitor database statements of any applica- tion that uses JDBC, thus the EIS tier monitoring is portable across different database implementations and is independent of the web and application server tiers. 132
- 145. 7.2.3 Performance Overhead Toassess the performance overhead incurred by COMPAS JEEM we instrumented the web and business tiers of the PlantsByWebsphere application described in section 7.2.1 and ran a number of performance tests. The test environment was made up of three machines. The first machine (Pentium 4, 2.25GHz, 1GB RAM) was used for load generation. Apache Jmeter was used as a load generation tool. PlantsByWebsphere was installed on WebSphere 6.1 which ran on a second machine (Pentium M, 1.7GHz, 1GB RAM). Finally a third machine (Pentium 4, 1.6 GHz, 768 MB RAM) was employed to log the data produced by COMPAS JEEM remotely such that the performance over- head on the machine running the JEE application could be kept to a minimum. This is common practice for production environments. To assess the performance overhead we ran a number of test cases with different user loads for two versions of the sample application (i.e. one version included COMPAS JEEM instrumentation and the other version did not). The test cases consisted of a warm up period (to warm up the JVM as suggested in the literature [38]), followed by a measurement period. During the differ- ent test cases we observed the response time of the application for the following loads: 20 users, 50 users, 100 users, 150 users, 200 users. Users entered the system at a rate of one user per second with a random delay of between 1-2 seconds. Each measurement period lasted 30 minutes. For each of the given loads three test cases were run and an average value obtained. To assess the performance overhead we analysed the aver- age response time of the instrumented application versus the average response time of the non-instrumented application for the same user load. The results are shown in figure 7.5. Note that the response time for each run is an average value that has been normalised based on the average response time for 20 users (non-instrumented). Application Version 20 Users 50 Users 100 Users 150 Users 200 Users Non-Instr. resp. time 1 1 1.1 4.12 X Instr. resp. time 1 1 1.1 4.9 X Overhead 0 0 0 19% X Figure 7.5: Performance Overhead Test Results At 20 and 50 users there was no noticeable difference (on average) in response time for either the instrumented or non-instrumented version of the application. Thus there was no noticeable overhead for these loads. At 100 users there was a 10% increase in response time for both the instrumented and non-instrumented versions compared with 20 users. Again there was no noticeable overhead that could be attributed to COMPAS JEEM. At 150 users, however the system was heavily saturated and response time increased significantly. For the non-instrumented application response time in- creased by 310%. The instrumented version at 150 users showed an increase of 390%. Therefore there was a 19% overhead that could be attributed to COMPAS JEEM at 150 users. At 200 users (for the default application server settings) the system under test had reached its capacity for both the instrumented and non-instrumented application. 133
- 146. There were alarge number of errors at this user load and the results were ignored. As can be seen from the results above there is no obvious overhead when COMPAS JEEM is applied to a non-saturated system. This can be explained by the fact that COMPAS JEEM only monitors at the component level and does not instrument lower level utility classes (unlike most of today’s JVMPI based monitoring tools). Adding a small number of monitoring probes is insignificant in the context of large enterprise applications that generally contain very large numbers of classes. The sudden increase in overhead at 150 users can be explained by the fact that at this point the system seems to be already heavily saturated, with the response time over 4 times slower than at lower loads. Any further increase in load at this point adds to the bottleneck and increases response time significantly. Further testing is required to assess if COMPAS JEEM will scale for increased loads when more hardware is available. 7.3 Analysis Module Results In this section we provide results on the performance of the FSM implementations modified with the different support counting approaches and preprocessing tech- niques. We also provide results on how we applied FSM to two enterprise applica- tions to identify design flaws within them. Our test environment consisted of a single machine (Pentium M, 1.7GHz, 1GB RAM) running Windows XP. The FSM implemen- tation was run through the Cygwin environment 4. Run-times were recorded using the GNU time tool 5. Gprof was also used to inspect how the time was being spent within the FSM implementation 6. The FSM implementation used was a modified version of Ferenc Bodon’s open source apriori implementation for mining frequent sequences of items 7. The main modifi- cation made to this code involved the adding of non-overlapping support counting. This code is currently being reviewed for submission into the FIM template library 8. We also modified the original version of the FSM implementation to output the corresponding transaction IDs in which a frequent sequence occurred. The final subsection in section 7.3 shows how we applied clustering for data reduction purposes. Clustering is applied to data collected from a JEE application under load and shows how the number of paths collected can effectively reduced. 4Cygwin, http://www.cygwin.com/ 5GNU time command, http://www.gnu.org/software/time/time.html 6GNU Gprof, http://www.gnu.org/software/binutils/manual/gprof-2.9.1/gprof.html 7Apriori implementation for mining frequent sequence of items, http://www.cs.bme.hu/ bodon/en/index.html 8Frequent Itemset Mining Template library, http://www.cs.bme.hu/ bodon/en/index.html 134
- 147. 7.3.1 FSM PerformanceTests This section details a number of tests that were run on different databases to show the run-times of the various FSM approaches discussed above. For the purpose of the tests we make use of a number of transactional databases; two publicly available databases from the Open Source Data Mining (OSDM) repository 9 and two databases that we constructed from run-time paths collected from monitoring distributed enterprise ap- plications (also publicly available 10). The databases from the OSDM repository were generated from weblogs of a large web news portal. The reason we chose to make use of these two databases was to show how the different techniques performed on databases with significantly different transaction lengths. The database kosarak2-10- 2 (238209 transactions) for instance has a maximum transaction length of 10 items, whereas kosarak2-100-2 (604280 transactions) has a maximum transaction length of 100. The databases generated from run-time paths were used to show how the dif- ferent implementations performed in relation to data obtained from real enterprise applications of different sizes. The first of these databases, Sun-Microsystems-sample- app (72 transactions, max length 83), was generated from monitoring a sample enter- prise application that is available from Sun Microsystems (see section 7.4). This is a small enterprise application. The second of these, IBM-enterprise-app (233 transactions, max length 1617),was generated from monitoring a beta version of the IBM Workplace collaborative enterprise application 11. The different FSM implementations that we tested on the OSDM databases were FSM with non-weighted support counting (FSM-NW), FSM with weighted support count- ing (FSM-W), FSM with non-overlapping weighted support counting (FSM-NO) and FSM with non-overlapping weighted support counting using Consecutive Item Re- placement (FSM-NO-CIR). For each test we measured the run-time of the algorithm. Figure 7.6 gives results when we ran the different implementations on the kosarak2-10- 2 database. As expected FSM-NW performed best for all support counts. The FSM- NW run-times ranged from 10.66 seconds (for a min. support of 5) to 1.71 seconds (for a min. support of 50). This was expected since this approach looks for only the first instance of each item sequence per transaction whereas all the other approaches tested on this database continue searching (in different ways, see sections 5.4.2 and 5.4.4) after the first instance has been identified. Also the output produced by FSM is much smaller than with the other approaches since it only counts the first sequence in the transaction. For the remaining three FSM approaches the FSM-NO approach was slowest in all cases. The reason why FSM-NO is slowest for the kosarak2-10-2 database is because for this database a large number of candidates are generated (e.g. for min. support of 5, 80347 frequent sequences were generated for FSM-NO). The FSM imple- 9Open Source Data Mining repository, http://www.ilab.sztaki.hu/∼bodon/osdm/ 10Frequent Sequence Mining Implementation and Databases, Trevor Parsons, http://pel.ucd.ie/tparsons/fsm 11IBM Workplace, http://www.ibm.com/software/workplace 135
- 148. Figure 7.6: TestResults on K2 10 2 Database mentations store the candidates in a trie data structure. The transactions are analysed one by one to count the support of the candidates. For each candidate that exists in the transaction the trie is traversed and the support of the candidate is incremented. FSM- NO makes use of flags in the trie to prevent the counting of overlapping sequences. When we begin comparing a new transaction to the trie the flags in the trie must be reset. When the trie is big (i.e. a large number of candidate sequences exist) and there are a large number of transactions in the database, this (resetting) process can be time- consuming and can significantly affect the performance of the FSM implementation. In fact for the tests run on the kosarak2-10-2 database 86-97% of time of the FSM-NO approach was spent performing this task. For the kosarak2-10-2 database FSM-W was significantly quicker than FSM-NO. To reduce the FSM-NO run-time we applied CIR to reduce the length of the trans- actions in the database. CIR was performed by initially running the FSM-NO imple- mentation with a high min. support threshold (50). The results from this initial run are used to identify consecutive sequences that can be replaced for the subsequent runs of lower min. support threshold. In total, after the first run, the database was reduced in size by 4449 items by making 771 replacements of 12 consecutive item sequences varying in size from 6 consecutive items to 9 consecutive items (50 item sequences of 136
- 149. size 9 werereplaced, 104 of size 8, 236 of size 7 and 381 of size 6). This process resulted in a new database kosarak2-10-2*. Reducing the database by 4449 items, however, had no noticeable improvement for FSM-NO when we applied it to kosarak2-10-2* with a min. support set to 40. This can be explained by the fact that replacing 4449 items in the kosarak2-10-2 database is insignificant when we consider the database size (over 230,000 transactions varying in size from 1 to 10). After the second run of the algo- rithm (at min. support 40) the output was analysed and CIR was again performed by replacing 14 frequent item sequences. This time kosarak2-10-2* was reduced by 17020 items resulting in a new database kosarak2-10-2**. On running FSM-NO with a min. support of 10 a noticeable run-time improvement of 30 seconds (4.3%) was recorded. The CIR process was repeated after the 3rd run and kosarak2-10-2** was reduced in size by a 10204 items resulting in a new database kosarak2-10-2***. Using kosarak2-10-2*** for FSM-NO with a min. support of 5 gave a run-time improvement of 148 seconds (6.9%). When analysing the results above, however, it must be considered that for the CIR runs other than the first FSM-NO-CIR run (min. support 50 in this case) previous iterations of the CIR process were required. So in fact for FSM-NO-CIR with a min. support of 5, 3 previous runs of FSM-NO (with min. support of 10, 20,50) were re- quired. Thus, the 6.9% performance improvements with FSM-NO-CIR (min. support 5) over FSM-NO is in fact out-weighted by the fact that three previous runs of the FSM-NO algorithm are required. When considering the performance of FSM-NO-CIR we must take into account the time taken for the previous runs. Also the modification of the databases during the CIR process is also time consuming since this is currently semi-manual. Since this process could be easily automated, we will not consider this when comparing FSM-NO-CIR to other approaches. Figure 7.7 gives our results when we ran the different implementations on the kosarak2-100-2 database. For all kosarak2-100-2 tests the max. candidate size was set to 5. Again FSM-NW outperformed the other implementations, its run-times rang- ing between 1.5 and 17.4 seconds. However for kosarak2-100-2 FSM-W was by far the slowest approach in all cases. FSM-NO was significantly quicker than FSM-W. In fact for support count of 10,000 FSM-W ran almost 10 times slower with a differ- ence of 8071 seconds (2.24 hrs). This can be explained by the fact that the FSM-W approach produces a much larger number of frequent sequences than the FSM-NO approach for kosarak2-100-2, which results in a large trie data structure (e.g. for min. support of 50000 the number of frequent sequences resulting from FSM-W and FSM- NO is 8225 and 108 respectively). As a result the number of generated candidates is much larger which leads to more time being spent counting the supports of the candidates. Counting the weighted support of a given candidate sequence is time consuming when transactions are long. To understand why this is we must under- stand how weighted support counting works. For each candidate sequence the first item must be checked at each point along the transaction for a match. If the first item matches an item in the transaction, then the second item in the sequence is checked 137
- 150. Figure 7.7: TestResults on K2 100 2 Database against all items in the transaction that occur after the matched item. This process con- tinues for any remaining items in the candidate sequence until all complete matches of the sequence are found. When transactions are long this process can be very time consuming. In fact for kosarak2-100-2 over 99% of FSM-W was spent counting the sup- ports of the generated candidates for all support counts tested. In comparison with kosarak2-10-2, while the number of sequences output for FSM-W and FSM-NO is very high, since the transactions are much shorter, the time spent counting supports is less significant. The impact of applying CIR to the kosarak2-100-2 database for FSM-NO was as follows: after the first iteration of the FSM-NO implementation at min. sup- port of 100,000, the database was reduced by 75,801 items by replacing 6 frequent consecutive sequences. The run-time for the second run (min. support 50,000) was reduced by 16.3 seconds (16.3%). After the second run the database was reduced in size by a further 390,250 items by replacing 10 frequent consecutive sequences. The run-time improvement of the third run was 170.2 seconds (47.2%). After the third run the database was reduced by a further 318,612 items by replacing 8 frequent consec- utive sequences. The run-time improvement for FSM-NO with min. support 10,000 was 475.1 seconds (52.8%). For both theSun-Microsystems-sample-app and the IBM-enterprise-app databases we ran 138
- 151. Figure 7.8: TestResults on Sun Database the same FSM approaches as with the previous two databases. However since both these databases were created from run-time paths, we could also make use of parental information to split the transactions and thus reduce their length (see section 5.4.4.1). Applying the FSM-NO approach to the split transactions is referred to as the SPLIT approach for the remainder of this section. Figure 7.8 gives results when we ran the different implementations on the Sun-Microsystems-sample-app database. For all cases the SPLIT approach was quickest. After splitting the database the maximum transac- tion length had been reduced from 84 to 37 items. This approach produced a smaller number of frequent sequences than the other approaches (e.g. for a min. support of 10 the FSM, FSM-NO and SPLIT produced 32234, 55906 and 174 frequent sequences respectively). FSM-NO-CIR was the next quickest. After the initial run (min. sup- port 50) the database was reduced by 126 items. After the second run (min, support 40) the database was reduced by a further 40 items and finally after the 4th run (min. support 20) the database was reduced by a further 105 items. The FSM-W approach was slowest running at least 50 times slower than FSM-NO at min. support count 20. For min. support 20 the FSM-W implementation failed to complete within 4300 seconds and the test was stopped. Figure 7.9 gives results when we ran the different implementations on the IBM-enterprise-app database. For this database the SPLIT ap- 139
- 152. proach did nothave as significant an effect as with the Sun-Microsystems-sample-app. This was because even after splitting the transactions the transaction lengths were still very long (max transaction length had been reduced from approximately 1600 to 900 items). However there was still a speed up of 14701.49 seconds (58.8%) over the FSM- NO approach. The most significant run-time improvement however for this database was with the FSM-NO-CIR approach. After the first FSM-NO-CIR run (min. support 400) the database was reduced by 1044 items by replacing 348 occurrences of a fre- quent consecutive sequence of size 4. The run-time improvement for a min. support of 300 was 24983.04 seconds (99.9%). The run-time for the FSM-NO-CIR approach re- mained low up to a min. support of 25. The database was reduced again by 654 items after the min. support 100 run and 207 items after the min. support 50 run. For min support of 300 the FSM-W implementation failed to complete within 35000 seconds and was stopped. Figure 7.9: Test Results on IBM Database 7.3.2 Applying FSM to Identify Design Flaws In this subsection we show how we applied FSM to identify design flaws in two enter- prise applications. The first application is a sample Java enterprise application from 140
- 153. Sun Microsystems calledDukes Bank discussed previously in section 7.2.1. Dukes Bank is a sample e-commerce application that Sun provided to showcase the JEE tech- nology. We modified the application and introduced a number of delays in certain sections of the code to simulate different performance design issues. In particular we introduced a delay in the business tier of the application to simulate a performance de- sign issue in this part of the application (a typical example for distributed enterprise systems might be network latency as a result of making a number of remote calls). Next we introduced another delay into the JDBC wrapper to simulate a database is- sue. In fact we introduced a delay to give the impression of looping through database calls in a resource intensive fashion. This is a common problem in enterprise systems whereby a large amount of information is often retrieved from the database resulting in significant performance degradation. Often it is the case that the information could be retrieved in a more efficient manner. As a result it is important to be able to identify when this occurs. To identify performance issues in the application we monitored the running appli- cation and collected run-time paths corresponding to the different user actions that could be executed in the system. The run-time paths also contained performance in- formation. The performance data corresponded to the time spent in each method that made up the run-time path. The run-time paths were converted to flat transactions (augmented with the performance data). This resulted in the Sun-Microsystems-sample- app-performance database. Since we are interested in finding performance issues in the application, it was more interesting to identify sequences of resource intensive meth- ods rather than simply identifying frequent sequences. To identify resource intensive sequences we modified the FSM-NO implementation to use an alternative criteria of interest other than frequency, i.e. time spent in a sequence of methods (see section 5.4.3). Applying the modified FSM-NO implementation to the Sun-Microsystems-sample-app- performance database we quickly identified the most resource intensive methods in the system and also the most resource intensive method sequences. For the test we set the min. support to 100 milliseconds and the max. sequence length to 9 (i.e. the algorithm would complete after candidates of size 9 were generated and their support counted). The algorithm run-time was 177 seconds. Although a large number of sequences were generated (3324) the most resource intensive methods and sequences could be easily identified by sorting by resources consumed (i.e. support). We correctly identified the two design flaws that had been introduced into the application. In fact from the output we could see that out of a total time the application was running (831.2 seconds), 95% (787 seconds) of the time was being spent in the database tier delay (in method Result- Set.next()) that we introduced. 4% of the time was spent in the business tier delay (in method AccountControllerBean.getDetails()). However the real advantage of using FSM is that not only does it highlight the most resource intensive methods, but it also gives the sequences in which they occurred (i.e. the run-time context) which is invalu- 141
- 154. able information fordevelopers trying to solve design problems. From the output we could see that the ResultSet.next() method was repeatedly being called across a num- ber of transactions in sequences of various sizes (i.e. loops in the run-time paths). Our FSM implementation was modified to record and output the transaction number in which the sequences occurred. Using this information we could easily identify these resource intensive loops and the particular run-time paths in which they existed. We could also see from the output that the AccountControllerBean.getDetails() method was not executed in a looping sequence and occurred at most once per transaction, except in one instance where it was called twice. The second application was an early beta version of the collaborative Workplace (en- terprise) application from IBM. Again we collected run-time paths from the applica- tion by performing different user actions. In total we collected 233 run-time paths from the application. We did not collect performance information for these tests (due to our limited access to the system) and thus had to rely on identifying the most frequent sequences of methods to find design flaws in the application. We applied the FSM- NO-CIR approach to the database (IBM-enterprise-app). Initially (min. support 400) we identified that the most frequent method occurred 1464 times across the 233 trans- actions. On closer inspection (min. support 30) we identified that the same method made up a consecutive sequence of size 24 occurring 58 times across the transactions (making up a total of 1392 calls). We made modifications to the original FSM imple- mentation such that it listed the transactions in which the sequence occurs and how often for each particular transaction. From this we could see that for one particular transaction the sequence of size 24 occurred 33 times. The transaction length was 1617 (method calls). We identified 792 (24x33) of these calls were to one method. The method being called was an EJB Entity bean method. Entity beans are used in JEE to access persistent data in the database from the application tier. There are a number of common design flaws that occur in these systems related to this kind of database activity i.e. looping through the database from the application tier which is inefficient [161]. On reporting that we had identified these potential loops in the application tier to the IBM development team they confirmed that we had indeed identified a de- sign issue which they subsequently rectified for the release. Our approach allowed an individual unfamiliar with the internals of the application (e.g. a system tester) to quickly identify potential problems in the application and the (run-time) context in which these problems occurred. This information can be quickly used by developers to refactor the application. 7.3.3 Data Reduction Results To show the effectiveness of the data reduction techniques discussed in section 5.5 we have applied them to data collected from a JEE application under load. For this test we loaded the Duke’s Bank sample e-commerce application with 40 users for a five 142
- 155. Figure 7.10: ClassDiagram of a Modified Version of Duke’s Bank with Communica- tion Patterns Highlighted minute period. Each user logged onto the system, browsed their account information, and deposited funds onto different accounts. In total each user performed 8 differ- ent user actions. A total of 1081 run-time paths were collected during this period. To reduce the data produced we clustered the paths (a) by the component methods that were invoked in each path and (b) by the different components that were invoked in each path. After applying clustering criteria (a) we grouped the paths into 11 dif- ferent path clusters. That is, our cluster analysis reduced the 1081 paths recorded to 11 (component-method level) paths through the system. In this instance statistical analysis can be applied to the component methods contained in each cluster to give a summary of the performance metrics associated with the run-time paths in each clus- ter. Applying the single user mode approach to the same user actions results in 11 distinct call paths. Our results show that (in this instance) applying clustering analy- sis to data collected in multi-user mode can effectively reduce the number of distinct path clusters to the number of different paths observed in single user mode. The path clusters in multi-user mode in fact contain more useful information than the paths collected in single user mode, since they give more realistic performance metrics for each method that is invoked in the path. Applying clustering criteria (b) to the 1081 paths resulted in 8 path clusters. That is, at the more abstract component level, there were 8 different paths through the system. 143
- 156. 7.4 PAD ToolResults In this section we present results attained from applying the PAD tool to two JEE applications i.e. a sample JEE application and a real large scale enterprise system. Section 7.4.1 details the test environment for our tests on the Duke’s Bank sample application, modifications that we made to the application and our test results. A number of modifications were made to Duke’s Bank in order to introduce a number of antipatterns for detection purposes. Section 7.4.2 gives the test environment for our tests on a real JEE application from IBM called Workplace and the results from these tests. Antipatterns from all categories outlined in section 6.1 have been identified. For each antipattern identified we describe the corresponding PAD tool output. 7.4.1 Antipatterns Detected in the Duke’s Bank Application The first application we applied the PAD tool to, in order to identify performance design and deployment antipatterns, was Duke’s Bank. Duke’s Bank is an online banking application. When a user logs in to the Duke’s Bank application he/she can perform the following actions: log on, view a list of accounts, view an individual ac- count’s details, withdraw or lodge cash, transfer cash from one account to another or finally log off. For our tests each of the different (8) user actions was performed. COM- PAS JEEM was used to collect run-time paths. We instrumented the application source code manually to perform object tracking, since COMPAS BCI could not run on the 1.4.1 JVM that was used. Duke’s Bank was deployed on the JBoss application server (version 3.2.7) with a MySQL database (version 4.0.2) as the backend. Our MEJB mon- itoring tool was used to interface with the application server to collect information on the server resources. For multi user mode (which was required to identify the In- correct Pool Size antipattern) the open source Apache JMeter load generation tool was used to load the application. Two versions of dukes bank were tested, the original ver- sion and a modified version with a number of antipatterns added. The original dukes bank application consists of 6 EJBs (4 of these were invoked during the tests, see figure 7.2 for a class diagram). We also modified the original version of duke’s bank to add a number of antipatterns to be detected by the PAD tool such that antipatterns from all categories discussed in section 6.1 were present (see figure 7.10 for a class diagram of the modified version of dukes bank). The antipatterns introduced are described in detail below. For each antipattern detected we give its corresponding category as described in section 6.1. In total 3 antipatterns were detected in the original version of Duke’s Bank by the PAD tool: • Conversational Baggage Antipattern [161] (category 1): This antipattern de- scribes a situation where stateful sessions beans are being used but are not nec- essarily required. Stateful session beans maintain state on the application server between client requests and should be used only when there is a clear need to 144
- 157. do so. Statelesssession beans will scale better than stateful session beans and should be used when state does not need to be maintained. Detection of this antipattern involves flagging the creation of unusually high numbers of stateful session beans across the run-time paths. A potential instance of this antipattern was flagged by the PAD tool when Dukes Bank was analysed as the number of stateful session beans was above the user defined threshold. This threshold was set to zero for the dukes bank application since there is no noticeable state maintained from one user action to the next. When the potential antipattern was detected the PAD tool showed us that stateful session beans were used in 100% of the run-time paths where the business tier is invoked. On closer inspection of the application (source code) we saw that indeed the stateful sessions could have been replaced with stateless sessions to improve scalability. In total 2 state- ful sessions could have been replaced. • Fine Grained Remote Calls (also known as the Face Off antipattern [161]) (cat- egory 3): This antipattern describes a situation where a number fine grained calls are consistently made from a client to the same remote bean. This results in a high number of expensive remote calls. A better approach is (if possible) to make a more coarse grained call that performs the combined tasks of the fine-grained calls. Performing a single coarse grained call instead of a number of fine grained calls will reduce network latency and thus the overall time required for the user action to execute. An instance of this antipattern was identified by the PAD tool. The tool identified a frequent sequence which contained fine grained remote calls. In fact it identified that the remote methods AccountBean.getType and Ac- countBean.getBalance appeared together 100% of the time they were called (i.e. the sequence had a confidence value of 100%). Thus it would be more efficient to combine these calls into a single coarse grained remote call. This antipattern is far from severe in this instance, however the identification of this antipattern shows that the tool can indeed identify antipatterns in this category. The rule to identify this antipattern can in fact be modified with a user defined threshold to only flag more severe instances. • Remote Calls Locally (also known as Ubiquitous Distribution [161]) (category 2): This antipattern describes a situation where beans that run in the same JVM as the client are called through their remote interfaces. In this situation the client has to perform an expensive remote call even though the bean is local. While some containers can optimize in this situation, this optimization is not a stan- dard feature of JEE. A better approach would be to write the beans with local interfaces (instead of remote, provided the beans are not also called from remote clients) such that they can be accessed in a more efficient manner. The PAD tool identified this relationship between components. In fact all (4) beans invoked in the Duke’s Bank application are accessed through remote interfaces even though their clients are within the same JVM. 145
- 158. Next we describethe antipatterns that were added to Duke’s Bank and the informa- tion given by the PAD tool when they were detected: • Accessing Entities Directly [61](also known as Customers In The Kitchen [161] Antipattern) (category 2): The application was modified such that components in the web tier were directly accessing entity bean components (see figure 7.10). This antipattern can cause a number of different performance issues as docu- mented in the antipattern description (e.g. problems with transaction manage- ment). Furthermore it can create maintainability issues since it mixes presenta- tion and business logic. The PAD tool identifies the inappropriate component relationships and flags them as antipattern instances. The component identified was the accountList.jsp which was modified to call the AccountBean (entity) di- rectly. • Needless Session Antipattern (category 2): Another antipattern added was the needless session antipattern. This antipattern is described in section 6.3 and out- lines a situation where a session bean is used but not required. In the Duke’s Bank application we added a session bean that was being used to calculated the current time and date. This function could have been easily carried out by a POJO which would have been a more efficient solution. The antipattern was detected by the PAD tool which reported that the session bean (CalculateDate- Bean) had no relationships with any persistent components (e.g. entity beans or database components) and thus was a potential needless session bean. • Application Filter Antipattern [161] (category 3): The application filter antipat- tern describes a situation where large amounts of data are retrieved from the database tier and filtered in the application tier. However, databases are de- signed to process filters a lot faster than can be done in the application tier. Thus filters should, where possible, be implemented in the database tier. We modified the Duke’s Bank application to include an application filter. The dukes bank application can retrieve account information for a customer that has logged in. When this is performed the application performs a search for the accounts that are owned by the particular user. We created an application filter in this situa- tion by modifying the SQL statement which filtered the information (using the following SQL statement ”select account id from customer database where cus- tomer id = ?”) to instead retrieve all accounts and send them to the application tier for filtering. In the application tier, filtering of the data was performed by checking the details of each account to see if it matched the id of the customer. We applied the PAD tool to the modified version of Dukes Bank. The tool suc- cessfully identified the application filter that had been added to the application. The tool identified two frequent sequences of communication: (1) between a ses- sion bean (AccountControllerBean) and an entity bean (AccountBean) (see figure 6.3 on page 117) and (2) between the same entity bean (AccountBean) and the 146
- 159. database in thesepaths. The first sequence of size 1 (AccountBean.getDetails) occurred 1180 times across the different run-time paths. The second sequence of size 2 (ResultSet.next, ResultSet.getString) occurred 6336 times across the run- time paths. It was evident from this information that an application filter existed in the application, especially when considering that this the amount of activity was occurring in single user mode. In fact the PAD tool identified that by mod- ifying this filtering 7/11 of run-time paths collected during the test run were affected. • Unused Data Object/Aggressive Loading (category 4) 12: A second antipattern was identified when the PAD tool was applied to the dukes bank application modified with an application filter antipattern. The antipattern detected was the unused data object antipattern/aggressive loading antipattern. This antipattern describes a situation whereby information is loaded from the database, but the data (or at least a high percentage of the data) is never used. A solution to this problem can be to refactor the application such that information is not retrieved if it is never actually required. Often however the information may be required a small percentage of the time. In such circumstances the information can be lazy loaded when it is required. This antipattern occurred in the application filter above. During filtering a check is performed to check each account to see if its owner id matched that of the (logged in) customer’s. To obtain each account’s owner id, the application loads the account information from the database into an entity bean. An AccountDetails Data Transfer Object (DTO) [7] is then cre- ated by the entity bean with the account information. Finally during the filter- ing the DTO is accessed to obtain the account owner id. Rather than loading all the account information into the entity bean, and subsequently into the DTO, a lazy loading approach can be used to load only the information that it generally needed (i.e. the account owner id). If the remaining (unloaded) information is required it can be loaded later. This antipattern will in fact be removed if the application filter is pushed into the database tier as suggested in the antipat- tern solution for the application filter. However there may be situations where it may not be possible to easily remove the application filter and where lazy load- ing can be applied. Lazy loading can in fact be applied in any situation where only a small proportion of the entity bean fields are ever accessed. The PAD tool provided the following information when this antipattern was identified: The AccountDetails Object was created on average 1886 times across the 7 run-time paths with a maximum value per path of 2400 times and a minimum value of 1200 times. The object has 8 accessor methods. Each method is given below with the corresponding average number of times it was accessed per run-time path: getCustomerID’s 1886 times, getType 1.4 times, getDescription 3 times, getBal- ance 2.6 times, getAccountId 8.1 times, getCreditLine 2.3 times, getBeginBalance 12Precise Java Website, http://www.precisejava.com 147
- 160. 0 times andgetBeginBalanceCreditLine 0 times. From the PAD tool output it is evident that (if the application filter could not be removed) it would be benefi- cial from a performance perspective to apply lazy loading for the account bean and DTO in this application to all fields except for the CustomerIds field. • Incorrect Stateful Session Instance Cache Size (category 5) 13: The final an- tipattern added to the dukes bank application was a deployment antipattern. We redeployed the application and modified the size of the stateful session in- stance cache to 10 (from the default 1000000). To detect this antipattern we ran the tests in multi user mode. We loaded the application with 40 users (over 10 seconds) and monitored the system for a 5 minute period. We specified the max- imimum passivation level as 4 (10% of user load) in (the user configuration files associated with) our antipattern rules. The passivation level is the number of beans that are passivated at any one time. Since passivation can be an expensive task, it should ideally be kept to a minimum. The PAD tool detected the stateful session beans instance cache size antipattern and presented the following infor- mation to the tool user: The AccountControllerBean instance cache passivation levels exceeded the specified user threshold of 4. The average passivation level during the measured period was 15. In this situation increasing the size of the instance cache is recommended. Next we discuss the issue of false positives and negatives detected by the PAD tool when applied to Duke’s Bank. In the strictest sense no false positives were found dur- ing the tests i.e. the tool did not identify antipatterns instances that were not present in the system. However with performance related antipatterns we are more concerned with identifying antipatterns that have an impact on the overall system performance. It is likely that the fine grained remote calls antipattern instance would not have a significant impact on the system performance and thus might be considered a false positive in this instance. However, by modifying the user defined threshold asso- ciated with the rule to detect this antipattern we can filter out instances with a low performance impact. Our aim was to show that instances of this antipattern can be identified by our tool and thus we set the threshold value such that even insignificant instances (from a performance perspective) were also identified. Similarly the remote calls locally antipattern may not have a performance impact in application servers that can optimize remote calls that are made to local components. However, again our aim was to show that this antipattern can be identified using our tool. By studying the Duke’s Bank documentation 14 and source code we were confident that our tests did not produce false negatives i.e. there were no antipatterns in the application, which were defined in our antipattern library that we did not detect. 13Precise Java Website, http://www.precisejava.com 14Duke’s Bank Documentation, http://java.sun.com/javaee/5/docs/tutorial/doc/ 148
- 161. 7.4.2 Antipatterns Detectedin the IBM Workplace Application - Beta Version The second system tested was an early beta version of the IBM Workplace Applica- tion15. IBM Workplace is a collaborative enterprise application built on the JEE tech- nology. In total 76 EJBs were instrumented, 38 of these were only ever invoked during the test runs (17 entity beans with Container Managed Persistence and 21 Session beans). The test run consisted of invoking 25 different user actions. Monitoring was performed using the COMPAS JEEM tool (COMPAS BCI was unavailable at the time of testing and could not have been applied to the JVM version we were using). Object tracking was not performed and performance metrics were not collected during these tests due to our limited access to the system. All tests were carried out in single user mode. The IBM Workplace application was running on the IBM WebSphere applica- tion server (version 5.x). The database used was IBM’s DB2. Using the PAD tool we identified antipatterns from four of the different categories outlined in section 6.1: • Local and Remote Interfaces Simultaneously (category 6): This antipattern oc- curs when a bean exposes both local and remote interfaces. There are a number of issues associated with this antipattern including exception handling issues, security issues and performance problems. From the 76 beans instrumented the PAD tool identified 30 (session) beans that exposed both local and remote in- terfaces. Many of the beans identified were not invoked during the test run, however the required information for antipattern detection in this instance was available in the beans meta data. On identification of this antipattern we con- tacted the development team who acknowledged that this was indeed an an- tipattern that had been removed in a later release due to security concerns. • Unusual/Bulky Session-Entity Communication (category 3): The second an- tipattern type identified by the PAD tool was in relation to database commu- nication. The tool identified unusually high communications between session and (persistent) entity beans (see rule in figure 6.3 on page 117). In fact as a result of the 25 user actions, a frequent sequence of entity bean calls (from a ses- sion bean) of size 24 occurred 58 times. In fact in one particular run-time path this sequence occurred 33 times. The 24 calls in the sequence were all to the same method which suggested that the sequence was in fact a loop of size 24. From this data it seemed that there was potentially an application filter causing this issue. Again we contacted the development team responsible for this code. The development team had identified this antipattern in a later release and had rectified it by pushing the filtering into the database tier as suggested by the application filter antipattern solution. • Transactions-A-Plenty/Incorrect Transaction size [61](category 1): Another an- 15IBM Workplace, http://www.ibm.com/software/workplace 149
- 162. tipattern detected bythe PAD tool was the transactions-a-plenty antipattern. The tool identified that for one particular use case a high number of transac- tions were being created. In fact the tool identified that 131 transactions were created in a single run-time path. In fact, for every session bean method call, a transaction was being initiated. On further inspection we discovered that the issue was that the session beans methods’ transactional settings were being set to ”Requires New” by default. The development team addressed this issue by editing the transactional settings for the beans in the deployment descriptors to initiate transactions only where new transactions were actually required. • Bloated Session Bean Antipattern [61](category 2): The final antipattern type detected was the bloated session bean antipattern. This antipattern is similar to the well known God class antipattern [36]. It describes a situation in EJB sys- tems where a session bean has become too bulky. Such session beans generally implement methods that operate against a large number of abstractions. For example one method may check the balance of an account while another may approve an order, and yet another may apply a payment to an invoice. Such sessions can create memory and latency issues since their creation and manage- ment can come with much overhead due to their bulky nature. The PAD tool identified two potential bloated session beans in the IBM Workplace applica- tion. The first potential instance of this antipattern was a session bean which had 8 entity bean relationships, 6 session bean relationships and was invoked in 11 of the 25 user actions executed. It also contained a high number of business methods (47). The second instance of this antipattern was a session bean which had 7 entity relationships 1 session relationship and was invoked in 6 of the 25 user actions. This session had 14 business methods. In general the rule of thumb is that there should be a 1 to 1 relationship between session and entity beans. From the information (above), presented by the PAD tool upon identification of these potential antipattern instances, it seemed that these session beans were in fact bloated session beans. Unfortunately we were unable to contact the devel- opers originally responsible for this code. However we did discover that this code was removed from later releases of the application which indicated that it was indeed problematic. No false positives were identified when we applied the PAD tool to the IBM work- place application and in fact all antipatterns identified were addressed in the sub- sequent versions of the application which suggested they were indeed problematic pieces of code. Unfortunately we could not assess whether false negatives were iden- tified in the application as we did not have access to the complete system source code or documentation. 150
- 163. 7.5 Validation In thissection we revisit the validation criteria outlined in section 3.1.1 as well as the contributions which we have claimed. We discuss how the contributions have been validated in respect to these criteria either through the tests that have been carried out in this chapter or by other means. 7.5.1 Overview of Contributions and Evaluation Criteria We have claimed the following contributions in this thesis: • An approach for the automatic detection of design and deployment antipatterns in systems built using component based enterprise frameworks. • A portable, low overhead, non-intrusive, end-to-end run-time path tracer for JEE systems. • A technique for the identification of interesting component communication pat- terns in a collection of run-time paths. In chapter 3 we outline a number of validation criteria to evaluate our work against. These criteria are attributes which we believe are required in order for our approach to be considered useful. The criteria are as follows: • The monitoring approach should be portable. • The monitoring approach should be non-intrusive. • The monitoring approach should exert a low-overhead on the system under test. • The monitoring approach should be end-to-end. • The monitoring approach should be applicable in distributed environments. • Analysis and detection should be efficient. • The number of false positives detected should be low. • The approach should be user configurable and extensible. • The approach should scale when applied to large real world enterprise applica- tions. The above criteria are discussed in more detail in chapter 3. Next we discuss how these criteria have been met. 151
- 164. 7.5.2 Validation ofContributions The first contribution outlined above has been realised in the form of the PAD tool. We have evaluated the tool by applying it to two JEE applications (see section 7.4). Our results show how it has been used to correctly identify a number of performance design and deployment antipatterns in these systems. We have shown how the num- ber of false positives was low when applied to both applications. The fact that this tool has been applied to identify real problems, in a real large scale JEE application, shows that our approach scales when applied to real systems. Applying our tool to real systems also shows that the problems we try to detect do indeed exist in reality and that they can be automatically detected using this approach. The PAD tool is ex- tensible and user configurable, since it allows for custom antipatterns to be added and user specified thresholds to be set. Our tests on the analysis module (see section 7.3) show that the approach is efficient in terms of the time taken for data analysis. The most time consuming component of the analysis is the extraction of communication patterns (the third contribution from above). Our test results show that this can be performed in a timely manner using our preprocessing techniques (see section 7.3.1). While the extraction of communi- cation patterns is currently semi-manual (using CIR), this process is not particularly time consuming or arduous. We also believe it can be automated in the future. This approach has been shown to scale well in relation to data from real enterprise systems in terms of both the output produced and its run-time. We have also shown in our tests that the amount of data output has been significantly reduced using our non- overlapping support counting technique, in comparison to weighted support count- ing (see section 7.3.1). Finally the usefulness of extracting component communication patterns has also been highlighted. Our results show the output from FSM applied to run-time paths can be used to (manually) identify serious performance design flaws in sample and real JEE applications (see section 7.3.2). The second contribution outlined above has also been evaluated on a number of lev- els. Firstly our results show that it is truly portable across different middleware im- plementations (see section 7.2.2). Secondly it has been shown to be a low-overhead monitoring approach (see section 7.2.3). Performance tests show that it has no signif- icant impact on a loaded system running under normal conditions. The monitoring tool has been applied to all server side tiers of a number of JEE applications and is completely end-to-end. A solution for the tracking of physically distributed calls has also been presented in section 4.2.5.3. Finally the output of the monitoring tool has been shown to be particularly effective for reverse engineering purposes and has been applied to reserve engineer a number of JEE applications such that design flaws can be (manually) identified (see section 7.2.1). 152
- 165. CHAPTER EIGHT Conclusions In this chapterwe give our conclusions. We also outline the limitations of this ap- proach. Finally we give some ideas on directions for future work in this area. 153
- 166. 8.1 Thesis Conclusions Thisthesis aims to address an issue in relation to today’s enterprise applications: de- velopers very often suffer from a lack of understanding in relation to these systems and as a result make suboptimal design choices which can lead to major performance issues. To address this issue we present an approach to automatically detect performance design and deployment antipatterns in component based enterprise applications. The thesis outlines the various components that are required in order to be able to achieve per- formance antipattern detection in such systems. Namely, a monitoring module, an analysis module and a detection module. The thesis outlines why dynamic analysis is required for detecting performance re- lated antipatterns in such systems and how it can be achieved in a portable, non- intrusive and low overhead manner. We discuss the various techniques that can be used to collect information from the system such that a complete representation of the system’s run-time design can be reconstructed. In particular we present an approach and implementation for portable, low-overhead and non-intrusive tracing of run-time paths in JEE applications. We also show how information, such as server resource information and component meta-data, can be obtained. The thesis further outlines how, from the monitoring data, the run-time design of the application can be automatically extracted. A reconstructed design model of the sys- tem is required such that the system design can be further analysed for antipattern detection. We show how to extract information from the monitoring data, that forms the basis of the run-time design model. In particular we have shown how interesting com- ponent communication patterns can be identified in run-time paths using frequent sequence mining. The design model is constructed by extracting the following information and adding it to the model: component structural and functional information, component relationships, component communication patterns and component object usage pat- terns. Reconstructed run-time container services that are utilised by the components are also added. Furthermore we show how large volumes of data recorded at run-time can be summarised and how this summarised data can be used to enhance the design model. In addition, this thesis shows how the design model can be further analysed such that performance design and deployment antipatterns can be detected within it. Detec- tion can be performed using a rule-engine based approach. An antipattern library is provided where by antipatterns are described in the form of rule-engine specific rules. We have outlined a number of validation criteria that we have used to evaluate our work against. We have shown through empirical studies (i.e. through case studies and experiments) how our work meets these criteria. 154
- 167. 8.2 Limitations In thissection we detail the limitations of this approach and how we feel they can be addressed: • Dynamic Analysis: Since the approach outlined in this thesis relies heavily on dynamic information it is not possible to identify antipatterns in the parts of the system that are never actually executed. Thus we assume a realistic testing scenario is available. • Run-Time Path Tracing Tool: we found during our tests that the main limitation of this approach was the fact that it the instrumentation process requires the sys- tem to be redeployed. During our initial tests on smaller sample applications, which were usually packaged as a single archive file, redeployment was a very simple task. On real larger systems however (e.g. IBM Workplace) the system was made up of a large number of different components and deployment was more complex. Thus, instrumenting the application was more time consuming than had been initially imagined. We believe this can be solved by modifying the monitoring approach used by the run-time path tracing tool. The moni- toring approach can be modified to make use of JVMTI and dynamic bytecode instrumentation to allow for instrumentation without redeployment. However this technique will only be applicable to newer JVMs (1.5+) and thus could not have been applied to the applications we have tested in this work. An initiative is already underway in this regard, i.e. COMPAS BCI (see section 4.2.5.8). • Applying FSM to Run-time Paths: The main limitation to this approach is fact that it is currently semi-manual. That is, a user of the approach is required to identify consecutive items and to replace them. We found during our tests that this can be easily performed, making use of the results from the initial FSM run and by using simple search and replace functions of a text editor. The task is not particularly time consuming or difficult. However we believe that the task could be easily automated to allow for complete automation. In fact after the initial FSM run, the results could be easily analysed and any consec- utive items in the database could be automatically replaced. A record of the changes/replacements would be maintained and a reversal process could be carried out after the CIR process has finished. This work will form part of our plans for future work. Another issue with this approach is that it can only be ap- plied when the output from the initial FSM run contains consecutive sequences. While, from our results, its seems that this will be the case for typical JEE sys- tems, if a systems is highly dynamic in terms of its run-time bindings this may not necessarily occur. An alternative approach in this situation might be to use parental splitting of the transactions (see section 5.4.4.1). However, as shown in our results section (7.3.1), this approach may not be effective for long trans- 155
- 168. actions. Further researchis required in this area to identify how FSM can be applied to highly dynamic enterprise systems. • PAD Tool Output: The PAD tool currently presents all antipatterns identified to the tool user with corresponding contextual information. The information presented (e.g. antipattern description, performance metrics) can be used by the tool user to easily determine the impact of the antipatterns detected. However it would be more desirable that the performance impact assessment would be automated. This would allow for the PAD tool to rank antipatterns instances of the same type according to their performance impact and developers could concentrate their efforts on addressing the most significant issues first. This will form part of our ideas for future work. • PAD Tool: There are a number of steps in the PAD tool detection process that have not been fully automated and which require human intervention. A fully automated tool is currently being developed. • Message Driven Beans: In this work we do not address issues related to message driven beans. We consider only session and entity beans. A number of antipat- terns however exist for message driven beans 1 [161] [61] and research in this area could form part of future work. 8.3 Future Work In this section we detail interesting areas for future work in this area: • Monitoring: We plan to address the limitations detailed above in relation to our monitoring module. In particular we plan to remove the need for system re- deployment during the instrumentation process. This can be achieved through dynamic bytecode instrumentation. Dynamic bytecode instrumentation can be performed for newer versions of the JVM (1.5+) through the JVMTI. Work in this area is already underway and forms part of the COMPAS BCI tool. Another limitation of our monitoring approach is that it has not been applied to asyn- chronous components (such as message driven beans). We plan to investigate how the run-time path tracing approach in particular can be applied to message driven beans. This could most likely be achieved by modifying our proxy com- ponents such that they intercept calls to the onMessage method of the message driven beans. • Integration with Real Time Correlation Engine: A proof of concept, real-time correlation engine has recently been implemented with the IBM Dublin Soft- 1Precise Java, http://www.precisejava.com/ 156
- 169. ware Lab [46].The tool correlates server logs from the different servers on a dis- tributed system such that a single log is available to system testers at run-time. The solves the problem of the system testers having to correlate a range of logs from different machines in order to understand the results of their tests. The tool can allow for correlation of log files by various criteria, (e.g. time stamp, user id, error type) and allows for easier and quicker problem determination. Currently an industry research project is underway to integrating this tool with COMPAS JEEM, such that system testers can analyse both system level information (e.g. the server logs) and application level information (e.g. run-time paths). The aim is to give the system tester a complete picture of the system under test, i.e. both at the system level and application level. • Analysis: Future work can be carried out to address the limitations of the FSM CIR approach by fully automating the replacement approach. The results from applying FSM to run-time paths show that this information is indeed useful once it can be extracted. An interesting area for future work would be to investigate sliding window approaches for finding episodes in sequences [112] to assess whether they can be modified such that they can be applied to a transactional database (e.g. a database of run-time paths). To date this approach has generally been applied to event sequence data. The benefit of this approach over the FSM approach using weighted support is that it does not suffer from combinatorial explosion when transactions grow in length and can not be reduced in size using preprocessing techniques. • Antipattern documentation: Technology specific antipatterns have been well documented for the Java technology covering pitfalls for both base java and the java enterprise technology. Enterprise applications are often built using other technologies/enterprise frameworks (e.g. .Net, CCM, Spring 2). However, we are unaware of any literature that details instances of antipatterns in the other common enterprise technologies. We firmly believe that similar problems con- sistently occur in the other enterprise technologies. Documenting these prob- lems in the form of antipatterns would bring understanding to developers build- ing systems using these technologies. • Antipattern detection for other technologies: The creation of a catalog of antipat- terns for alternative enterprise technologies would allow for the automatic an- tipattern detection using the approach we have outlined in this thesis. Research would be required to develop or identify a monitoring framework that could perform call path tracing. Much of the call path tracing logic and mechanisms we present could be reused. The analysis and detection modules are portable across different technologies. 2The Spring Framework, http://www.springframework.org/ 157
- 170. • Automatic AntipatternAssessment: Assessment of detected antipatterns would allow for automatic ranking of antipatterns in terms of the antipattern perfor- mance impact. This would allow developers to address the most significant antipatterns first. This can be achieved using predefined cost functions. Cost functions can be potentially created for each antipattern described in the an- tipattern library. Cost functions could be created to rank instances of the same antipattern. Cost functions describe antipatterns performance characteristics, and using the performance metrics collected by the monitoring module would allow for automatic assessment of the antipatterns performance impact. 158
- 171. REFERENCES [1] M. K.Agarwal, M. Gupta , G. Kar, A. Neogi and A. Sailer, Mining Activity Data for Dynamic Dependency Discovery in e-Business Systems, IEEE eTransac- tionson Network and Service Management Journal, Vol.1 No.2, September, 2004 [2] R. Agrawal, H. Mannila, R. Srikant, H. Toivonen and A.I. Verkamo. Fast discovery of association rules. In Advances in Knowledge Discovery and Data Mining, 1996. [3] R. Agrawal and R. Srikant. Mining sequential patterns. In P. S. Yu and A. L. P. Chen, editors, In Proceedings 11th International Confonference on Data Engineering, 1995 [4] R. Agrawal, T. Imielinski, and A.N. Swami. Mining association rules between sets of items in large databases. In P. Buneman and S. Jajodia, editors, In Pro- ceedings of the 1993 ACM SIGMOD International Conference on Manage- ment of Data, volume 22(2) of SIGMOD Record, 1993. [5] H. Albin-Amiot, P. Cointe, Y. G. Gueheneuc, N. Jussien. Instantiating and Detecting Design Patterns: Putting Bits and Pieces Together, 16th IEEE Interna- tional Conference on Automated Software Engineering (ASE’01), 2001. [6] C. Alexander. A Timless Way of Building. Oxford University Press, 1979. [7] D. Alur, J. Crupi and D. Malks. Core J2EE Patterns: Best Practices and Design Strategies. Prentice Hall, Sun Microsystems Press, 2001 [8] G. Ammons, T. Ball and J.R. Larus, Exploiting hardware performance counters with flow and context sensitive profiling. ACM Conference on Programming Language Design and Implementation. ACM Press, New York, 1997. [9] G. Ammons, J.D. Choi, M. Gupta, and N. Swamy. Finding and Removing Performance Bottlenecks in Large Systems., In Proceedings of ECOOP, 2004. [10] J.M. Anderson, L.M. Berc, J. Dean, S. Ghemawat, M.R. Henzinger, S.A. Le- ung, R.L. Sites, M.T. Vandervoorde, C.A. Waldspurger and W.E. Weihl. Con- tinuous profiling: Where have all the cycles gone? Proceedings of the Sympo- sium on Operating Systems Principles. ACM Press, New York, 1997. [11] G. Antoniol, G. Casazza, M. Di Penta, and R. Fiutem. Object-oriented design patterns recovery. Journal of Systems and Software, 2001. 159
- 172. [12] M. Arnoldand D. Grove. Collecting and Exploiting High-Accuracy Call Graph Profiles in Virtual Machines. International Symposium on Code Generation and Optimization, San Jose, California, March, 2005. [13] Z. Balanyi, R. Ferenc. Mining Design Patterns from C++ Source Code. 19th IEEE International Conference on Software Maintenance, 2003. [14] T. Ball and S. K. Rajamani. The SLAM project: debugging system software via static analysis, In Proceedings of the 29th POPL, January 2002. [15] S. Balsamo, A. Di Marco, P. Inverardi, M. Simeoni. Model-Based Performance Prediction in Software Development: A Survey. IEEE Transactions on Software Engineering, , pp. 295-310, Vol. 30, No. 5, MAY 2004. [16] J. Bansiya. Automating Design-Pattern Identification. Dr. Dobb’s Journal, vol. 23, no. 6, June, 1998. [17] P. Barford and M. Crovella, Generating representative web workloads for network and server performance evaluation. In ACM SIGMETRICS, 1998. [18] P. Barham, A. Donnelly, R. Isaacs and R. Mortier. Using Magpie for request extraction and workload modelling, Symposium on Operating Systems Design and Implementation, pp 259–272, San Francisco, CA, USA, December 2004. [19] P. Barham, R. Isaacs, R. Mortier and D. Narayanan. Magpie: online modelling and performance-aware systems. In the 9th Workshop on Hot Topics in Oper- ating Systems, May, 2003. [20] V.R. Basili. The Experimental Paradigm in Software Engineering. Experimen- tal Software Engineering Issues: Cirtical Assessment and Future Directives, Springer-Verlang, Lecture Notes in Computer Science, 1993. [21] L. Bass, P. Clements and R Kazman. Software Architecture in Practice (2nd Edition). Addison-Wesley, 2003. [22] I. D. Baxter, C. Pidgeon, M. Mehlich. DMS: Program transformations for prac- tical scalable software evolution. Proceedings of the IEEE International Confer- ence on Software Engineering. IEEE Computer Society Press: Los Alamitos, CA, 2004. [23] J. Beck, D. Eichmann, Program and Interface Slicing for Reverse Engineering. In Proceedings of the International Conference on Software Engineering, 1993. [24] K. Beck. Extreme Programming Explained: Embrace Change. Addison-Wesley, 1999. [25] K. Beck. Test Driven-Development by Example, Addison-Wesley, Boston, MA, 2003. [26] K. Beck and W. Cunningham. Using Pattern Languages for Object-Oriented Programs, Technical Report No. CR-87-43 submitted to OOPSLA-87 work- shop on the Specification and Design for Object Oriented Programming, September, 1987. 160
- 173. [27] F. Bergentiand A. Poggi. Improving UML Designs Using Automatic Design Pattern Detection. Proceedings of the 12th International Conference on Soft- ware Enginneering and Knowledge Engineering, July, 2000. [28] P. Berkhin. Survey of clustering data mining techniques. Technical report, Ac- crue Software, San Jose, CA, 2002. [29] D. Beyer, A. Noack and C. Lewerentz. Efficient Relational Calculation for Soft- ware Analysis. IEEE Transactions on Software Engineering, vol. 31, no. 2, February, 2005. [30] W. Binder, Portable and accurate sampling profiling for Java. Software Practice and Experience, Vol. 36, Issue 6, John Wiley and Sons, Inc., New York, NY, USA, February, 2006. [31] W. Bischofberger, J. Kuhl and S. Loffler. Sotogrpah - a Pragmatic Approach to Source Code Architecture Conformance Checking. Proceedings First European Workshop on Software Architecture, Lecture Notes in Computer Science 3047, Springer, 2004. [32] F. Bodon, A Trie-based APRIORI Implementation for Mining Frequent Item Se- quences. ACM SIGKDD Workshop on Open Source Data Mining Workshop (OSDM’05), pages 56 - 65, Chicago, IL, USA. 2005 [33] M. D. Bond and K. S. McKinley. Continuous Path and Edge Profiling. 38th In- ternational Symposium on Microarchitecture, Barcelona, November, 2005. [34] L.C. Briand, Y. Labiche, and J. Leduc, Towards the Reverse Engineering of UML Sequence Diagrams for Distributed, Multithreaded Java Software. Technical Re- port SCE-04-04, Carleton Univ., http://www.sce.carleton.ca/Squall, Sept. 2004. [35] L.C. Briand, Y. Labiche and J. Leduc. Toward the Reverse Engineering of UML Sequence Diagrams for Distributed Java Software. IEEE Transactions on Soft- ware Engineering, vol. 32, no. 9, September, 2006. [36] W. J. Brown, R. C. Malveau, and T. J. Mowbray. AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis. Wiley, 1998. [37] E. Bruneton, T. Coupaye, M. Leclercq, V. Qu´ema, and J.-B. Stefani. The Frac- tal Component Model and Its Support in Java. Software Practice and Experi- ence, special issue on Experiences with Auto-adaptive and Reconfigurable Systems. 2006. [38] A. Buble, L. Bulej and P. Tuma. CORBA Benchmarking: A Course with Hidden Obstacles. Proceedings of IPDPS Workshop on Performance Modeling, Eval- uation and Optimization of Parallel and Distributed Systems, Nice, France, 2003. [39] F. Buchli. Detecting Software Patterns using Formal Concept Analysis, Diploma Thesis, University of Bern, 2003. [40] M. Burch, S. Diehl and P. Weissgerber. Visual data mining in software archives. Proceedings of the 2005 ACM symposium on Software visualization, St. Louis, Missouri, 2005. 161
- 174. [41] D. Burdick,M. Calimlim, J. Flannick, J. Gehrke and T. Yiu, MAFIA: A Max- imal Frequent Itemset Algorithm. IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 11, pp. 1490-1504, Nov., 2005. [42] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad and M. Stal. Pattern- Oriented Software Architecture, A System of Patterns, Volume 1. Wiley, 1996. [43] E. Cecchet, J. Marguerite and W. Zwaenepoel. Performance and Scalability of EJB Applications. Proc. 17th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, pp 246-261, Seattle, Washington, USA, 2002. [44] M. Chen, E. Kiciman, A. Accardi, A. Fox and E. Brewer. Using runtime paths for macro analysis. In Proceedings 9th Workshop on Hot Topics in Operating Systems, Lihue, HI, USA, May 2003 [45] M. Chen, E. Kiciman, E. Fratkin, A. Fox and E. Brewer: Pinpoint: Problem De- termination in Large, Dynamic, Internet Services. In Proceedings International Conference on Dependable Systems and Networks (IPDS Track), Washing- ton, D.C., June 2002 [46] L. Chen, P. O’Sullivan, L. P. Bergman, V. Castelli, E. Labadie, P. Sohn, T. Par- sons. Problem Determination in Large Enterprise Systems. At the Software En- gineering Tools For Tomorrow (SWEFT) 2006 conference, T.J. Watson, New York, Oct 17 - 19, 2006. (Abstract only available) [47] H. Chen and D. Wagner. MOPS: an infrastructure for examining security prop- erties of software. In Proceedings of the Ninth ACM Conference on Computer and Communications Security, Washington, DC, November 2002. [48] E. J. Chikofsky and J. H. Cross II. Reverse Engineering and Design Recovery: A Taxonomy. IEEE Software, 7(1):13– 17, January/February 1990. [49] O. Ciupke. Automatic detection of design problems in object-oriented reengineer- ing, In proceedings Technology of Object-Oriented Languages and Systems, 1999. [50] P. Clements. A Survey of Architecture Description Languages. In Proceedings of the 8th International Workshop on Software Specification and Design, 1996. [51] P. Clements, F. Bachmann, L. Bass, D. Garlan, J. Ivers, R. Little, R. Nord and J. Stafford. Documenting Software Architectures: Views and Beyond. Boston: Addison-Wesley, 2003. [52] P. Clemeents, R. Kazman and M. Klein. Evaluating Software Architectures, Methods and Case Studies Addison-Wesley, 2001. [53] J. Coplien. Advanced C++ Programming Styles and Idioms. Addison-Wesley, 1991. [54] J. Coplien and N. Harrison. Organizational Patterns of Agile Software Develop- ment. Prentice-Hall, 2004. [55] J. Corbett, M. Dwyer, J. Hatcliff, C. Pasareanu, Robby, S. Laubach, H. Zheng. Bandera: Extracting Finite-state Models from Java Source Code, In Proc. of ICSE, June 2000. 162
- 175. [56] G. Costagliola,A. De Lucia, V. Deufemia, C. Gravino, and M. Risi. Design Pattern Recovery by Visual Language Parsing, Proceedings of the Ninth Eu- ropean Conference on Software Maintainance and Reengineering, March, 2005. [57] M. Dahm. Byte code engineering with the BCEL API. Technical Report B- 17-98, Freie Universit¨at Berlin, 2001. [58] G. Denaro, A. Polini, and W. Emmerich, 2004. Early performance testing of distributed software applications. In Proceedings of the 4th international Work- shop on Software and Performance, Redwood Shores, California, 2004. [59] M. Dmitriev, Profiling Java applications using code hotswapping and dynamic call graph revelation. In Proceedings of the 4th international workshop on Software and performance, Redwood Shores, California, 2004. [60] M. Dodani: Patterns of Anti-Patterns, in Journal of Object Technology, vol. 5, no. 6, pp. 29-33, July - August 2006. [61] B. Dudney, S. Asbury, J. K. Krozak and K. Wittkopf. J2EE Antipatterns, Wiley, 2003. [62] A. H. Eden and R. Kazman. Architecture, Design, Implementation. In pro- ceesings of the International Conference on Software Engineering, Portland, USA, May, 2003. [63] M. El-Attar and J. Miller. Matching Antipatterns to Improve the Quality of Use Case Models. 14th IEEE International Requirements Engineering Conference, Minnesota, USA, 2006. [64] S. Elbaum and M. Diep,Profiling Deployed Software: Assessing Strategies and Testing Opportunities. IEEE Transactions on Software Engineering, Vol. 31, No. 4, April 2005. [65] G. Florijn, M. Meijers, and P. van Winsen. Tool support for object-oriented pat- terns. In Ecoop 1997: Object-Oriented Programming, volume 1241 of Lecture Notes in Computer Science, 1997. [66] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. The KDD process for extract- ing useful knowledge from volumes of data. Communications of the ACM, vol- ume 39, issue 11, pp27-34, November 1996 [67] M. Fowler. Patterns of Enterprise Application Architecture. Addison-Wesley Professional, 2002 [68] M. Fowler. Refactoring: improving the design of existing code. Addison-Wesley, 1999. [69] E. Friedman-Hill. Jess in Action. Manning Publications, July, 2003. [70] L. Fulop, T. Gyovai, and R. Ferenc. Evaluating C++ Design Pattern Miner Tools. In Sixth IEEE International Workshop on Source Code Analysis and Manipulation, Sep. 2006. 163
- 176. [71] E. Gamma, R. Helm, R. Johnson and J. Vlissides. Design Patterns: Abstrac- tion and Reuse of Object-Oriented Design. European Conference on Object Ori- ented Programming, 1993 [72] E. Gamma , R. Helm, R. Johnson and J. Vlissides: Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley, 1995 [73] D. Garlan and M. Shaw. An introduction to software architecture. In Advances in Software Engineering and Knowledge Engineering. Volume 1. World Scientific Publishing Co., 1993. [74] R. Glass. The Software Research Crisis. IEEE Software, November, 1994. [75] B. Goethals, and M. J. Zaki , ”Advances in Frequent Itemset Mining Implemen- tations: Report of FIMI 2003”, ACM SIGKDD Explorations; Vol. 6, nr. 1, 2003. [76] I. Gorton and A. Liu, Software Component Quality Assessment in Practice: Suc- cesses and Practical Impediments, the Proceeding of the International Confer- ence on Software Engineering, U.S., May 2002 [77] I. Gorton and L. Zhu. Tool Support for Just-In-Time Architecture Reconstruction and Evaluation: An Experience Report, Proc. 27th International Conference on Software Engineering, St. Louis, Mousiri, USA, 2005 [78] K. Govindraj, S. Narayanan, B. Thomas, P Nair, and S. Peeru. On Using AOP for Application Performance Management. In proceedings Fifth International Conference on Aspect-Oriented Software Development (Industry Track), Bonn, Germany, March 2006. [79] S. Graham and M. McKusick, Gprof: A Call Graph Execution Profiler. ACM SIGPLAN, vol. 17, no. 6, pp. 120-126, June 1982. [80] M. Grand. Patterns in Java. vol. 2. New York, New York: Wiley: 1999. [81] J. Grass. Object-oriented design archeology with CIA. Computing Systems, 1992. [82] T. Gschwind, K. Eshghi, P. K. Garg and K. Wurster. WebMon: A Perfor- mance Profiler for Web Transactions, Proc. Fourth IEEE International Work- shop on Advanced Issues of E-Commerce and Web-Based Information Sys- tems, Newport Beach, California, USA, June, 2002 [83] Y.G. Gueheneuc, H. Albin-Amiot. Using Design Patterns and Constraints to Automate the Detection and Correction of Inter-Class Design Defects. 39th Inter- national Conference and Exhibition on Technology of Object-Oriented Lan- guages and Systems (TOOLS39), 2001. [84] R.J. Hall. Call Path Refinement Profiles. IEEE Transactions on Software Engi- neering, vol. 21, no. 6, June 1995. [85] R.J. Hall. CPPROFJ: Aspect-capable Call Path Profiling of Multi-threaded Java Applications. Proceedings of the 17 th IEEE International Conference on Au- tomated Software Engineering, 2002. 164
- 177. [86] H. H.Hallal, E. Alikacem, W. P. Tunney, S. Boroday, and A. Petrenko. Antipattern-Based Detection of Deficiencies in Java Multithreaded Software, Pro- ceedings of the Quality Software, Fourth International Conference on (QSIC’04), IEEE Computer Society, Washington, DC, USA, 2004. [87] D. Hand, H. Mannila, P. Smyth. Principles of Data Mining, MIT Press, 2001. [88] M. Hauswirth, P.F. Sweeney, A. Diwan and M. Hind. Vertical profiling: Un- derstanding the behavior of object-oriented applications. Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-oriented Programming, Sys- tems, Languages, and Applications, 2004. [89] D. Heuzeroth, T. Holl, G. Hogstrom, and W. Lowe. Automatic design pat- tern detection. In 11th International Workshop on Program Comprehension (IWPC), USA, May 2003. IEEE. [90] D. Heuzeroth, T. Holl, and W. Lowe. Combining static and dynamic analyses to detect interaction patterns. In Sixth International Conference on Integrated Design and Process Technology (IDPT), Jun 2002. [91] D. Heuzeroth, S. Mandel, and W. Lowe. Generating design pattern detectors from pattern specifications. In 18th International Conference on Automated Software Engineering (ASE). IEEE, Oct. 2003. [92] C.W. Ho, M.J. Johnson, L. Williams and E.M. Maximilien. On Agile Perfor- mance Requirements Specification and Testing. Proceedings of AGILE 2006. [93] G. Hohpe, and B. Woolf: Enterprise Integration Patterns. Addison-Wesley Professional, 2003 [94] D.F. Jerding, J.T. Stasko and T. Ball. Visualizing Interactions in Program Execu- tions. In the proceedings of the International Conference on Software Engi- neering, 1997. [95] S.C. Johnson. Lint, a C program checker. Unix Programmer’s Manual, 4.2 Berkeley Software Distribution Supplementary Docs; U.C. Berkeley, 1984. [96] M.J. Johnson, E.M. Maximilien, C.W. Ho and L. Williams. Incorporating Per- formance Testing in Test-Driven Development. IEEE Software, May/June, 2007. [97] S. Kamin and D. Hyatt. A Special Purpose Language for Picture Drawing, In Proceedings of the Conference of Domain Specific Languages, Santa Bar- bara, CA, USA, 1997 [98] R.K. Keller, R. Schauer, S. Robitaille, and P. Pag´e. Pattern-based reverse engi- neering of design components. In Software Engineering, 1999. Proceedings of the 1999 International Conference on, 1999. [99] M. Kis. Information Security Antipatterns in Software Requirements Engineering. In proceedings 9th Conference on Pattern language of Programs, 2002. [100] B. Kitchenham, L. Pickard and S.L. Pfleeger. Case Studies for Method and Tool Evaluation. IEEE Software, July, 1995. [101] A. Koenig. Patterns and Antipatterns, Journal of Object Oriented Program- ming, volume 8, number 1, 1995. 165
- 178. [102] P. Kovari,D. Cerecedo Diaz ,F. C. H. Fernandes ,D. Hassan , K. Kawamura , D. Leigh, N. Lin, D. Masicand, G. Wadley and P. Xu. WebSphere Application Server Enterprise V5 and Programming Model Extensions WebSphere Handbook Series, September, August, 2003 [103] C. Kramer and L. Prechelt. Design recovery by automated search for structural design patterns in object-oriented software. In Reverse Engineering, 1996, Pro- ceedings of the Third Working Conference on, 1996. [104] P. Kruchten. The 4+1 View Model of Architecture. IEEE Software, volume 12, issue 6, November, 1995. [105] M. Kunnumpurath. JBoss Administration and Development Third Edition (3.2.x Series, Apress, October 2003 [106] J. R. Larus and T. Ball. Rewriting executable files to measure program behavior. Technical Report Computer Sciences Technical Report 1083, University of Wisconsin-Madison, 1992. [107] C.H. Lee, J.C. Ou and M.S. Chen. Progressive Weighted Miner: An Efficient Method for Time-Constraint Mining. PAKDD, 2003. [108] M. Leleu, C. Rigotti , J.F. Boulicaut and G. Euvrard. GO-SPADE: Mining Se- quential Patterns over Datasets with Consecutive Repetitions. Proc. 3rd Int. Conf. Machine Learning and Data Mining in Pattern Recognition, Leipzig, Ger- many, 2003. [109] S. Lo. Binary Prediction Based on Weighted Sequential Mining Method. Proc. Int. Conf. on Web Intelligence, 2005. [110] M. Mahemoff. Ajax Design Patterns. Sebastopol, California: O’Reilly, 2006. [111] H. Mannila, H. Toivonen and A.I. Verkamo. Discovering Frequent Episodes in Sequences, Proc. 1st Int. Conf. on Knowledge Discovery and Data Mining, Montreal, Canada, 1995 [112] H. Mannila, H. Toivonen and A.I. Verkamo. ”Discovery of frequent episodes in event sequences”, Data Mining and Knowledge Discovery 1(3): 259 - 289, November 1997. [113] F. Marinescu. EJB Design Patterns. Wiley, 2002. [114] R. Marinescu. Measurement and Quality in Object-Oriented Design. PhD thesis, Politehnica University of Timisoara, 2002. [115] N. Medvidovic and R. N. Taylor. A Classification and Comparison Framework for Software Architecture Description Languages. IEEE Transactions on Soft- ware Engineering, vol. 26, no. 1, January, 2000. [116] N. Moha and Y.G. Gu´eh´eneuc. On the Automatic Detection and Correction of Design Defects. In Serge Demeyer, Kim Mens, Roel Wuyts, and St´ephane Ducasse, editors, Proceedings of the 6th ECOOP workshop on Object- Oriented Reengineering, July 2005. 166
- 179. [117] N. Moha,J. Rezgui, Y.G. Gueheneuc, P. Valtchev and G. El Boussaidi. Using FCA to Suggest Refactorings to Correct Design Defects. Proceedings of the 4th International Conference On Concept Lattices and Their Applications (CLA 2006),In S. Ben Yahia and E. Mephu Nguifo Eds, Hammamet, Tunisia, 2006. [118] A. Mos. A Framework for Adaptive Monitoring and Performance Management of Component-Based Enterprise Applications. PhD thesis, Dublin City University, Ireland, 2004. [119] A. Mos and J. Murphy. COMPAS: Adaptive Performance Monitoring of Component-Based Systems. Proc. Workshop on Remote Analysis and Mea- surement of Software Systems at 26th International Conference on Software Engineering, Edinburgh, Scotland, May 2004. [120] A. Mos and J. Murphy. Performance Management in Component-Oriented Sys- tems using a Model Driven Architecture Approach, Proc. 6th IEEE International Enterprise Distributed Object Computing Conference, Lausanne, Switzer- land, September 2002. [121] M.J. Munro. Product metrics for automatic identifcation of bad smell design prob- lems in java source-code. In Lanubile, F., Seaman, C., eds.: proceedings of the 11th International Software Metrics Symposium, IEEE Computer Society Press, 2005. [122] G.C. Murphy, D. Notkin, and K. Sullivan. Software Reflexion Models: Bridging the Gap between Source and High-Level Models. Proc. SIGSOFT Symp. Founda- tions of Software Eng., ACM Press, New York, 1995. [123] J. Niere, J. P. Wedsack, and L. Wendehals. Handling Large Search Space in Pattern-based Reverse Engineering. In 11th IEEE International Workshop on Program Comprehension, May 2003. [124] G. Papadimitriou, A. Vakali, G. Pallis, S. Petridou and A. Pomportsis, Sim- ulation in Web data management, Applied system simulation: methodologies and applications, Kluwer Academic Press, Norwell, MA, USA, 2003. [125] T. Parsons and J. Murphy. Detecting Performance Antipatterns in Component Based Enterprise Systems. Accepted for Publication in the Journal of Object Technology. [126] T. Parsons, J. Murphy, P. O’Sullivan, Applying Frequent Sequence Mining to Identify Design Flaws in Enterprise Software Systems. In Proceedings 5th Intl. Conference on Machine Learning and Data Mining, Leipzig, Germany, July 18-20, 2007. [127] T. Parsons, J. Murphy, S. Pizzoli, P. O’Sullivan, A. Mos, Reverse Engineering Distributed Enterprise Applications to Identify Common Design Flaws. At the Software Engineering Tools For Tomorrow (SWEFT) 2006 Conference, T.J. Watson, New York, Oct 17 - 19, 2006. [128] T. Parsons, A. Mos and J. Murphy. Non-Intrusive End to End Run-time Path Tracing for J2EE Systems. IEE Proc. Software, August, 2006. 167
- 180. [129] T. Parsonsand J. Murphy.. The 2nd International Middleware Doctoral Sympo- sium: Detecting Performance Antipatterns in Component-Based Enterprise Sys- tems. IEEE Distributed Systems Online, vol. 7, no. 3, March, 2006. [130] T. Parsons. A Framework for Detecting Performance Design and Deployment An- tipatterns in Component Based Enterprise Systems. Proc. 2nd Int’l Middleware Doctoral Symp., ACM Press, art. no. 7, Grenoble, France, 2005. [131] T. Parsons. A Framework for Detecting, Assessing and Visualizing Performance Antipatterns in Component Based Systems. First Place at ACM SIGPLAN Stu- dent Research Competition Graduate Division, In OOPSLA’04: Companion to the 19th annual ACM SIGPLAN conference on Object-oriented program- ming systems, languages, and applications, Vancouver, BC, Canada, 2004. [132] T. Parsons and J. Murphy.. A Framework for Automatically Detecting and As- sessing Performance Antipatterns in Component Based Systems using Run-Time Analysis. The 9th International Workshop on Component Oriented Pro- gramming, part of the 18th European Conference on Object Oriented Pro- gramming. Oslo, Norway, June, 2004. [133] T. Parsons and J. Murphy. Data Mining for Performance Antipatterns in Compo- nent Based Systems Using Run-Time and Static Analysis. Transactions on Au- tomatic Control and Control Science, Vol. 49 (63), No. 3, May 2004. [134] D.J. Pearce, P. H. J. Kelly, T. Field, U. Harder. Gilk: A dynamic instrumentation tool for the Linux kernel. Proceedings of the International TOOLS Conference. Springer: Berlin, 2002. [135] D. J. Pearce, M. Webster, R. Berry and Paul H. J. Kelly, Profiling with AspectJ. Software Practice and Experience Vol. 37, Issue 7, John Wiley and Sons, Inc. New York, NY, USA, June 2007. [136] R. Pena-Ortiz, J. Sahuquillo, A. Pont and Jose A. Gil, Modeling Continuous Changes of the User’s Dynamic Behavior in the WWW, Proceedings of the 5th international workshop on Software and performance, Palma, Illes Balears, Spain, 2005. [137] N. Pettersson. Measuring precision for static and dynamic design pattern recog- nition as a function of coverage. In ICSE Workshop on Dynamic Analysis, St. Louis, USA, May 2005. [138] N. Pettersson and W. L¨owe. Efficient and Accurate Software Pattern Detec- tion. 13th Asia Pacific Software Engineering Conference (APSEC), Decem- ber 2006. [139] N. Pettersson, W. L¨owe and J. Nivre. On Evaluation of Accuracy in Pattern Detection. First International Workshop on Design Pattern Detection for Re- verse Engineering, October 2006. [140] F. Pl´asil, D. B´alek and R. Janecek. SOFA/DCUP: Architecture for Compo- nent Trading and Dynamic Updating. In proceedings of ICCDS.98, Annapolis, Maryland, USA, IEEE CS Press, May 1998 [141] C. Ponder and R.J. Fateman. Inaccuracies in Program Profilers. Software Prac- tice and Experience, May, 1998. 168
- 181. [142] M. Pont.Patterns for Time-Triggered Embedded Systems. Harlow, England: Addison-Wesley: 2001. [143] A. Postma: A method for module architecture verification and its applica- tion on a large component based system, Information and Software Tech- nology 45(4), 2003, 171-194 [144] C. Potts. Software Engineering Research Revisited. IEEE Software, September, 1993. [145] D. Reimer et al. SABER: Smart Analysis Based Error Reduction, Proceedings of the ACM SIGSOFT international symposium on Software testing and anal- ysis, 2004. [146] T. Richner and S. Ducasse. Recovering High-Level Views of Object-Oriented Applications from Static and Dynamic Information. Proceedings International Conference on Software Maintenance, 1999. [147] H. Ritch and H. M. Sneed. Reverse engineering programs via dynamic analysis. In Proceedings of WCRE 1993. IEEE, May 1993. [148] E. Roman, Scott W. Ambler and Tyler Jewell, Mastering Enterprise JavaBeans, second edition, J.Wiley and Sons, USA and Canada, 2002. [149] B. Schmerl, J. Aldrich, D. Garlan, R. Kazman, and H. Yan. Discovering Archi- tectures from Running Systems. IEEE Transactions on Software Engineering, July, 2006. [150] M. Schumacher, E. Fernandez-Buglioni, D. Hypertson, F. Buschmann, and P. Sommerlad, Security Patterns. Chichester, England: Wiley, 2006. [151] J. Seemann and J. W. von Gudenberg. Pattern-based design recovery of Java software. SIGSOFT 1998/FSE-6: Proceedings of the 6th ACM SIGSOFT in- ternational symposium on Foundations of software engineering, 1998 [152] M. Shaw and D. Garlan. Software Architecture, Perspectives on an Emerging Discipline. Prentice Hall, 1996. [153] N. Shi and R. A. Olsson. Reverse Engineering of Design Patterns from Java Source Code. ASE ’06: Proceedings of the 21st IEEE International Conference on Automated Software Engineering, 2006. [154] C. U. Smith, and L. Williams. Performance Solutions. Addison Wesley, 2002. [155] A. Srivastava and A. Eustace. ATOM: A system for building customized pro- gram analysis tools. Proceedings of the ACM Conference on Programming Language Design and Implementation. ACM Press: New York, 1994. [156] P.F. Sweeney, M. Hauswirth, B. Cahoon, P. Cheng, A. Diwan, D. Grove and M. Hind. Using hardware performance monitors to understand the behavior of Java applications. Virtual Machine Research and Technology Symposium. USENIX Association: Berkeley, CA, USA, 2004. [157] C. Szyperski. Component Software - Beyond Object-Oriented Programming, Addison-Wesley, 1999 169
- 182. [158] TakeFive SoftwareGmbH. SNiFF+, 1996. [159] F. Tao, F. Murtagh and M. Farid. Weighted Association Rule Mining using weighted support and significance framework. KDD, Washington, D.C., 2003. [160] B. Tate. Bitter Java. Manning Publications Co., Greenwich, CT, USA, 2002. [161] B. Tate, M. Clarke, B. Lee and P. Linskey. Bitter EJB. Manning, 2003 [162] W.F. Tichy. Should Computer Scientists Experiment More?. IEEE Computer, 1998. [163] P. Tonella, M. Torchiano, B. Du Bois and T Systa. Empirical studies in reverse engineering: state of the art and future trends. Journal on Empirical Software Engineering, 2007. [164] P. Tonella and G. Antoniol. Object Oriented Design Pattern Inference. Proceed- ings of IEEE Conference on Software Maintenance, 1999. [165] Call Graph-Directed Boundary Condition Analysis in Contextual Composi- tion Frameworks. PhD Thesis, University College Dublin, April, 2007. [166] M. Trofin and J. Murphy. Removing Redundant Boundary Checks in Contextual Composition Frameworks. Journal of Object Technology, vol. 5, no. 6, July- August, 2006. [167] N. Tsantalis, A. Chatzigeorgiou, G. Stephanides, and S. T. Halkidis. Design Pattern Detection Using Similarity Scoring. IEEE Transactions on Software En- gineering, November 2006. [168] A. Ufimtsev and L. Murphy. Performance modeling of a JavaEE component application using layered queuing networks: revised approach and a case study. In Proceedings of the 2006 Conference on Specification and Verification of Component-Based Systems, Portland, Oregon, 2006. [169] D. Viswanathan and S. Liang, Java Virtual Machine Profiler Interface. IBM Sys- tems Journal, Java Performance ,vol. 39, no. 1, 2000. [170] E. Visser. Stratego: A language for program transformation based on rewriting strategies. Proceedings of the International Conference on Rewriting Tech- niques and Applications. Springer: Berlin, 2001. [171] J. M. Vlissides, J. O. Coplien and N. L. Kerth. Pattern Languages of Program Design 2 (Software Patterns Series). Addison-Wesley Professional, 1996. [172] M. Vok´ac: An efficient tool for recovering Design Patterns from C++ Code. Jour- nal of Object Technology, vol. 5, no. 1, 2006. [173] J. Wang and J. Han, BIDE: efficient mining of frequent closed sequences. Proc. 20th Int. Conf. on Data Engineering, 2004 [174] L. Wendehals. Improving design pattern instance recognition by dynamic analy- sis. In ICSE 2003 Workshop on Dynamic Analysis (WODA), Portland, USA, May 2003. [175] L.Wendehals and A. Orso. Recognizing Behavioral Patterns at Runtime using Finite Automata. In Workshop on Dynamic Analysis (WODA), May 2006. 170
- 183. [176] E. Weyukerand F. Vokolos. Experience with performance testring of software systems: issues, an approach, and case study. IEEE Transactions on Software Engineering, 2000. [177] N. Wilde and M. C. Scully. Software reconnaisance: Mapping program features to code. Software Maintenance: Research and Practice, 1992. [178] C. Wohlin, P. Runeson, M. Host, M. C. Ohlsson, B. Regnell and A. Wesslen. Experimentation in Software Engineering, An Introduction. Kluwer Academic Publishers, 2000. [179] E. Wohlstadter, S. Jackson and P. Devanbu, DADO: Enhancing Middleware to Support Crosscutting Features in Distributed, Heterogeneous Systems, Proceed- ings of the 25th International Conference on Software Engineering, May, 2003 [180] X. Yan, J. Han, and R. Afshar, CloSpan: Mining Closed Sequential Patterns in Large Databases. In SIAM Int. Conf. on Data Mining , San Francisco, CA, May 2003. [181] M.J. Zaki and C.-J. Hsiao. CHARM: An efficient algorithm for closed associa- tion rule mining. Technical Report 99-10, Department of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA, 1999. [182] M. J. Zaki and C. Hsiao. Charm: An efficient algorithm for closed itemset mining. In 2nd SIAM Int’l Conf. on Data Mining, 2002. 171
- 184. APPENDIX A Antipattern Rule Library Thisappendix chapter details the different rules that are contained in the PAD tool rule library. Examples of rules are given in section 6.3 which show how rules are structured in a Lisp like syntax. This section gives all the rules currently provided by the PAD tool. A.1 Rules In section we list the rules currently available as part of the PAD Tool rule library. Rules are documented in their respective categories as outlined in section 6. The rules that form the antipattern library also make use of a number of user defined functions and configuration settings which are discussed in sections A.2 and A.3 respectively. Full descriptions of these antipatterns are available in the different JEE antipattern literature1 2 3. A.1.1 Category1: Antipatterns Across or Within Run-Time Paths Transactions-A-Plenty (defrule detect-Transactions-A-Plenty-Antipattern (?rtp<-(runTimePath (numTrans ?nt))) (test (checkRTSLevel "Transactions" ?nt "THRESHOLD")) => (printAPSolution "Transactions-A-Plenty" ?rtp)) Figure A.1: The Transactions-A-Plenty Rule 1B. Tate, M. Clarke, B. Lee and P. Linskey. Bitter EJB. Manning, 2003 2B. Dudney, S. Asbury, J. K. Krozak and K. Wittkopf. J2EE Antipatterns, Wiley, 2003 3Precise Java, http://www.precisejava.com 172
- 185. This rule infigure A.1 identifies a situation where the number of transactions in a particular run-time path are above a user defined threshold value. The rule makes use of the checkRTSLevel user defined function which checks if number of transactions is this run-time path is too high, i.e. above a user defined threshold. If this rule fires the printAPSolution function is called. This function prints out the antipattern description and solution and related contextual information. Threshold values are specified in a configuration file. Conversational-Baggage (defrule detect-Conversational-Baggage-Antipattern (?rtp<-(runTimePath (compQuickList ?cql)(componentTypeList ?ctl))) (test (checkForComponent ?cql ?ctl "Stateful" "THRESHOLD")) => (printAPSolution "Conversational-Baggage" ?rtp)) Figure A.2: The Conversational-Baggage Rule This rule in figure A.2 identifies a situation where the number of stateful sessions in a particular run-time path are above a user defined threshold value. The user defined function checkForComponent is used by the rule to check the component types that exist in the run-time path and to see if the number of stateful beans is above the threshold level. Sessions-A-Plenty (defrule detect-Sessions-A-Plenty-Antipattern ?rtp<-(runTimePath (ID ?id)(compQuickList ?cql) (componentTypeList ?ctl)) (test (checkForComponent ?cql ?ctl "Stateless-Entity-ratio" "THRESHOLD")) => (printAPSolution "Sessions-A-Plenty" ?rtp)) Figure A.3: The Sessions-A-Plenty Rule This rule in figure A.3 identifies a situation where the number of stateless sessions in a particular run-time path are above a user defined threshold value or if the ratio of sessions to entities is too high. The user defined function checkForComponent is used to check the session/entity ratio and to see if it is above the threshold level for a particular run-time path. Similarly this rule can also use the function to check if the number of stateless sessions in the run-time path is above the threshold level. If either of these facts match the rule can fire. In this example rule we check the session/entity 173
- 186. ratio only. A.1.2 Category2:Inter-Component Relationship Antipatterns Needless-Session (defrule detect-Needless-Session-Antipattern ?C1<-(component(name ?N)(type ?T&:(eq ?T "Session")) (callees ?Callees)(callers ?Callers)) (component(name ?N2)(type ?T2&:(eq ?T2 "Entity")|?T2&:(eq ?T2 "DB"))) (not (test (existsInList ?N2 ?Callees))) (not (test (existsInList ?N2 ?Callers))) => (printAPSolution "Needless-Session" ?C1)) Figure A.4: The Needless-Session Rule The rule in figure A.4 identifies a situation where a session bean does not have any relationships with either the database or any entity beans. The rule makes use of the existsInList user defined function which is used to check the components list of caller and callee components, for entity or database components. Further checks could be made in this rule to see if the session bean is making use of the container services (see section 6.3). Remote-Calls-Locally The rule in figure A.5 identifies a situation where two components are running within the same JVM, and one of the components calls the other component through its re- mote interface. The rule use the sameJVM function to compare the JVM details. It also uses the existsInList function to check the component’s (C1’s) list of callers for the component C2. Accessing-Entities-Directly The rule in figure A.6 identifies a situation where a web component directly accesses an entity or database component. The existsInList function is used to identify the entity or DB components in the web components callee list. Bloated-Session The rule in figure A.7 identifies a situation where the number of relationships a ses- sion bean has with other beans is above a user specified threshold. The checkRelation- shipLevel function is used to check whether the number of relationships is above the threshold level. 174
- 187. (defrule detect-Remote-Calls-Locally-Antipattern ?C1<-(component (isLocal?L&:(eq ?L "false")) (isRemote ?R&:(eq ?R "true")) (JVMDetails ?jvm1) (Callers ?callers)) ?C2<-(component (name ?N2) (JVMDetails ?jvm2)) (test (sameJVM ?jvm1 ?jvm2)) (test (existsInList ?N2 ?Callers)) => (printAPSolution "Remote-Calls-Local" ?C1 ?C2)) Figure A.5: The Remote-Calls-Locally Rule (defrule detect-Accessing-Entities-DB-Directly-Antipattern ?C1<-(component (name ?N) (type ?T&:(eq ?T "Web")) (callees ?Callees)) ?C2<-(component (name ?N2) (type ?T2&:(eq ?T2 "Entity"))|?T2&:(eq ?T2 "DB")) (test (existsInList ?N2 ?Callees)) => (printAPSolution "Accessing-Entities-DB-Directly" ?C1 ?C2)) Figure A.6: The Accessing-Entities-Directly Rule (defrule detect-Bloated-Session-Antipattern ?C1<-(component (name ?N) (type ?T&:(eq ?T "Session")) (callees ?Callees) (callers ?Callers)) test(checkRelationshipLevel ?Callees ?Callers "THRESHOLD") => (printAPSolution "Bloated-Session" ?C1)) Figure A.7: The Bloated-Session Rule 175
- 188. A.1.3 Category3: AntipatternsRelated to Component Communication Patterns Unusual-or-Bulky-Session-Entity-Communication (defrule detect-Bulky-Session-Entity-Comms-Antipattern (?C1<-(component (name ?N1)(type ?T1&:(eq ?T1 "Session")) (callees ?Callees))) (?C2<-(component (name ?N2)(type ?T2&:(eq ?T2 "Entity")))) (?FS<-(frequentSequence(children ?Ch) (parents ?Ps) (methodIdLists ?Ms) (support ?S))) (test(existsInList ?N1 ?Ch)) (test(existsInList ?N2 ?Ps)) (test(flagHighResourceConsumption ?Ms ?S "THRESHOLD")) => (printAPSolution "Bulky-Session-Entity-Comms" ?FS ?C1 ?C2)) Figure A.8: The Unusual-or-Bulky-Session-Entity-Communication Rule The rule in figure A.8 identifies a situation where a frequent sequence exists that has a session as parent and an entity component as its child. For the rule to match the frequent sequence must also have a resource consumption (or frequency) level above a user defined threshold. The flagHighResourceConsumption function is used to identify if the threshold has been surpassed. Fine-Grained-Remote-Calls (defrule detect-Fine-Grained-Remote-Calls-Antipattern (?FS<-(frequentSequence (methodList ?Ms) (support ?S) (conf ?C))) (test (confidenceAboveThreshold ?C)) (test (supportAboveTheshold ?S)) (test (remoteMethods ?Ms)) => (printAPSolution "Fine-Grained-Remote-Call" ?FS)) Figure A.9: The Fine-Grained-Remote-Calls Rule 176
- 189. The rule infigure A.9 identifies a situation where a frequent sequence exists that has a support level above the user defined threshold (supportAboveTheshold), a rule confi- dence above the threshold level (confidenceAboveThreshold) and which contains two or more remote method calls. A.1.4 Category4: Data Tracking Antipatterns Unused-Data-Object (defrule detect-Unused-Data-Object-Antipattern ?TO<-(TrackedObject (name ?Name) (stats ?Stats) (type ?Type)) (test (lazyLoadable ?Stats ?Type)) => (printAPSolution "Unused-Data-Object" ?TO)) Figure A.10: The Unused-Data-Object Rule The rule in figure A.10 identifies a situation where the data in a tracked object is not being accessed more than a user specified percentage level. The lazyLoadable user de- fined function determines if a particular tracked object is suitable for lazy loading using the tracked object statistics and user defined thresholds. A.1.5 Category5: Pooling Antipatterns Incorrect-Pool-Size (defrule Incorrect-Pool-Size ?P<-(Pool (name ?N)(stats ?S) (type ?T)) test (poolThresholdExceeded ?S ?T) => (printAPSolution "Incorrect-Pool-Size" ?P)) Figure A.11: The Incorrect-Pool-Size Rule The rule in figure A.11 identifies a situation where the pool queues have exceeded a user specified threshold. The poolThresholdExceeded function looks up the user speci- fied threshold and compares it to the pool statistics. 177
- 190. A.1.6 Category6: Intra-ComponentAntipatterns Local-and-Remote-Simultaneously (defrule detect-Local-and-Remote-Intrfs-Simultaneously-Antipattern (?C<-(component (has_local_interface ?LI&:(eq? LI"true")) (has_remote_interface ?RI&:(eq ?RI "true")))) => (printAPSolution "Local-and-Remote-Intrfs-Simultaneously" ?C)) Figure A.12: The Local-and-Remote-Intefaces-Simultaneously Rule The rule in figure A.12 identifies a situation where a component exposes both local and remote interfaces. A.1.7 Adding Rules to The Rule Library The PAD tool’s rules are stored in a single rules.clp file. Adding rules to the library can be achieved by simply editing and appending to the rule library. Rules can also be easily modified by editing this file. A.2 Jess User Defined Functions provided by the PAD Tool The PAD tool library makes use of a number of Jess user defined functions. These functions are packaged as part of the PAD tool and can assist in representing rules in a more concise manner. Rules that use the functions can be less complicated and are thus easier to both write and read. By using these functions users can easily extend the antipattern library with their own antipattern rules. Jess has a number of built in functions that can be used when writing Jess rules. In some cases however Jess does not provide the capabilities to perform the tasks re- quired. To overcome this issue Jess provides a way to extend the language through the Userfunction interface4. The Userfunction interface can be implemented by a Java class. The PAD tool has made use of this capability to provide a number of functions that can be used when writing antipattern detection rules. Next we give an overview of the functions that have been developed as part of the PAD tool. printAPSolution: This function takes the antipattern name and any relevant data as arguments. The function then prints out the antipattern description, corresponding solution and contextual information that can be used by the tool user to reason about the problem identified. existsInList: This function takes a list and a string as input. It searches the list for the particular string and returns true if it exists. 4Ernest Friedman-Hill, Jess In Action, Manning, 2003. 178
- 191. sameJVM: This functiontakes two values (JVMDetails) as input and compares them. It returns true if they are the same. poolThresholdExcceded: This function takes the pool statistics and type as input and evaluates (i.e. returns true) if the user defined threshold has been exceeded. The user defined threshold is modifiable by editing a threshold configuration file. remoteMethods: This function takes a list of methods as input and returns true if two or more of the methods are remote calls. lazyLoadable: This function take the statistics associated with a tracked object and the tracked object type as input and evaluates (i.e. returns true) if the object is suitable for lazy loading using user defined thresholds. supportAboveThreshold: This function takes the support of a frequent sequence as input and returns true if it is above a user defined threshold. confidenceAboveThreshold: This function takes the confidence of a frequent se- quence as input and returns true if it is above a user defined threshold. flagHighResourceConsumption: This function takes the frequent sequence method list and the support (which can represent frequency or resources used) of a frequent sequence. It calculates if the resource consumption of the sequence is above a user defined threshold. If resource information is not available frequency is used. checkRelationshipLevel: This function take a list of a components callers and a list of component callees as input. The function evaluates if the number of bean relationships is above a user defined threshold. checkForComponents: This function takes a list of components as input and a com- ponent type. It checks if the number, of a particular component type is above a user defined threshold. It can also check for component type ratios (e.g. entity to sesseion). checkRTSLevel: This function takes a runtime service type as input, and the num- ber of run-time services. If the number of run-time services exceeds a user defined threshold level it returns true. A.3 Configuration Settings The PAD tool user can set threshold levels for a number of the antipatterns outlined above. The threshold values can be made by modifying a single configuration file. The user functions outlined above make use of this configuration file to determine if the specified user thresholds have been exceeded. 179
- 192. APPENDIX B JEEM and FSMImplementation Source Code The COMPAS JEEM source code and the FSM algorthim implementation source code have both been made freely available. COMPAS JEEM is part of the COMPAS open source project and is available to down- load from the COMPAS web site1. The FSM implementation used in this work is a modified version of Ferenc Bodon’s TRIE based FSM implementation. Bodon’s version is part of the open source Fre- quent Itemset Mining Template Library2. Our code has been made available3 and is currently being reviewed such that it can be added to the Frequent Itemset Mining Template Library. The data sets used as part of the tests in this work are also freely available3 4. 1COMPAS Open Source Project, http://compas.sourceforge.net/ 2Frequent Itemset Mining Template Library, http://www.cs.bme.hu/ bodon/en/index.html 3Trevor Parsons FSM implementation, http://pel.ucd.ie/tparsons/fsm 4Open Source Data Mining repository, http://www.ilab.sztaki.hu/∼bodon/osdm/ 180