A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
GridMate - End to end testing is a critical piece to ensure quality and avoid...ThomasParaiso2
End to end testing is a critical piece to ensure quality and avoid regressions. In this session, we share our journey building an E2E testing pipeline for GridMate components (LWC and Aura) using Cypress, JSForce, FakerJS…
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
UiPath Test Automation using UiPath Test Suite series, part 5
Managing an soa environment with tivoli redp4318
1. Front cover
Managing an SOA
Environment with
Tivoli
Discusses SOA performance and
availability management
Describes mediation scenarios
with ESB and message broker
Integrates ITCAM and
other Tivoli solutions
Budi Darmawan
Pradeep Nambiar
Prem Lall
Ravinder Gummadavelli
ibm.com/redbooks Redpaper
10. Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
Redbooks (logo) ® DB2® OS/400®
z/OS® ETE™ Rational®
Alerts® IBM® Redbooks®
AIX® IMS™ RACF®
Cloudscape® MQSeries® Tivoli Enterprise Console®
CICS® MVS™ Tivoli®
DataPower® OMEGAMON® WebSphere®
The following terms are trademarks of other companies:
SAP NetWeaver, SAP, and SAP logos are trademarks or registered trademarks of SAP AG in Germany and
in several other countries.
Oracle, JD Edwards, PeopleSoft, Siebel, and TopLink are registered trademarks of Oracle Corporation
and/or its affiliates.
ITIL is a registered trademark, and a registered community trademark of the Office of Government
Commerce, and is registered in the U.S. Patent and Trademark Office.
Enterprise JavaBeans, EJB, Java, JavaBeans, JDBC, JMX, JNI, JRE, JVM, J2EE, Solaris, Sun, Sun Java,
and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other
countries, or both.
Internet Explorer, Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the
United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
viii Managing an SOA Environment with Tivoli
12. Budi Darmawan is a project leader at the International Technical Support
Organization, Austin Center. He writes extensively and teaches IBM classes
worldwide on all areas of Tivoli and systems management. Before joining the
ITSO eight years ago, Budi worked in IBM Indonesia services as a technical lead
and solution architect. His current interests are Java™ programming, availability
management, business service management, and z/OS® management.
Pradeep Nambiar is a Worldwide Technical Evangelist in the IBM Tivoli
Business Automation Sales Enablement group. He has over 19 years experience
in the IT industry in various areas ranging from graphics systems, networked
graphics, IBM Component Broker/WebSphere Application Server system
management, business application architecture, design, and development. He is
an IBM Certified SOA Solution Designer, IBM Certified WebSphere Enterprise
Developer, and IBM Certified Solution Developer in XML and Related
Technologies. His current focus is on application management and the
automation family of products, including SOA management from IBM Tivoli.
He is based in Austin, TX.
Prem Lall is a Software Engineer currently assigned to the ITCAM for SOA
project where he specializes in the field of Web Services management. He has
had over 15 years experience in the IT field. During his 11 years at IBM, he has
helped design and implement a variety of software products. He has expertise in
front-end, middleware, and back-end development with an emphasis on
e-commerce. Among other things, he created end-to-end online banking
solutions for IBM clients in the Integrion consortium, he has been part of the
WebSphere Application Server development team, and helped create an
extensive SOA-based e-File application for the IRS that is currently used by
numerous businesses across the country. He holds a Masters of Science Degree
in Pure and Applied Mathematics from California State University, Northridge,
CA. He also worked as an Actuary, and he has worked in the Atmospheric
Physics Division of the NASA Goddard Space Flight Center.
Ravinder Gummadavelli is a Software Engineer with IBM Systems Technology
Group, in the USA. He has over 10 years of experience in the IT Systems Design
and Development field. He holds a Masters in Technology degree in Electrical
Engineering from REC, Warangal, India, and a Masters of Science degree in
Electrical Engineering from Auburn University, Auburn, AL. His areas of
expertise include Systems Design, Development, and SOA. His current interests
include SOA and IBM Virtualization offerings.
Thanks to the following people for their contributions to this project:
Karen Durston, Mark Anderson, Jayne Regan
IBM Software Group
x Managing an SOA Environment with Tivoli
13. Become a published author
Join us for a two- to six-week residency program! Help write a book dealing with
specific products or solutions, while getting hands-on experience with
leading-edge technologies. You will have the opportunity to team with IBM
technical professionals, IBM Business Partners, and Clients.
Your efforts will help increase product acceptance and customer satisfaction. As
a bonus, you will develop a network of contacts in IBM development labs and
increase your productivity and marketability.
Find out more about the residency program, browse the residency index, and
apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our papers to be as helpful as possible. Send us your comments about
this paper or other IBM Redbooks® publications in one of the following ways:
Use the online Contact us review IBM Redbooks publications form found at:
ibm.com/redbooks
Send your comments in an e-mail to:
redbooks@us.ibm.com
Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400
Preface xi
16. 1.1 Introduction to SOA
A service-oriented architecture (SOA) is an architecture that describes a system
that is composed of discrete services that interact with clients and accomplish
various tasks. Each service contains operations that perform a small unit of work
that corresponds to a high level definition of a given task. These services can
perform simple tasks, such as accessing data from a database and returning
data to a client, or can be part of a workflow that represents a more complex
task. A service usually either provides information or facilitates a way to modify it
in a certain way.
Although Web Services technology is commonly used to implement an SOA, it is
not required; in other words, a service does not have to use Web Services
constructs or technology. However, most SOAs rely on the standards, practices,
and tools that are available with Web Services.
Regard SOA as an approach to build distributed systems that deliver application
functionality as Web Services to user applications. Properly used, SOA principles
can provide a framework for matching business needs with realistic solutions.
Web Services is the programmatical interface to a capability that complies with
standard protocols, providing the interface technology and delivering platform
independency and loose coupling of the transport. SOA is potentially wider in its
scope of governing the policies, rules, and common services that enable logical
service bus structure for use by authorized consumers internal and external to
the enterprise regardless of implementation technology. Also, SOA enables the
design and quality of service that can be reused and that conforms to functional
and nonfunctional service level agreements (SLAs). Web Services are the
foundational interoperability technology, and SOA is the application of the
interoperability that implies considerable change in business and IT practices.
This magnitude of business and technology change requires a certain level of
management in order to successfully reap the benefits from the investment that
has been made for the change.
In this chapter, we look into the SOA structure and the management
requirements that lead to the need for this paper to define the SOA system
management environment.
1.2 SOA application principles
You can deploy an SOA-based application incrementally and slowly integrate it
into an existing environment. Developers have designed SOA scenarios to be
2 Managing an SOA Environment with Tivoli
17. used together or adopted incrementally. There are several well-known SOA
patterns, including:
Service creation
Service connectivity
Interaction and collaboration services
Business Process Management
Information as a service
You can apply the SOA design, governance, security, and management across
all of these scenarios.
Applications in a typical silo architecture build their functionality on top of an
existing application stack. In an SOA-based environment, the logical boundary
between Web Services consumer and provider is defined by a business function,
not by application boundaries (typical for silo architectures):
Separation of Web Services interface from its implementation
One of the most important aspects of SOA is that it separates a Web
Service’s implementation from its interface. A meta-language, such as Web
Services Definition Language (WSDL), can be very helpful, because it
describes a business interface that a Web Services provider can expose to a
client application and other Web Services while concealing the Web Service’s
implementation. A WSDL document acts as a server-side descriptor that
defines the Web Service’s operations and the messages they use so the
client will know how to invoke the Web Service. Web Services consumers
view a Web Services simply as an endpoint that supports a particular request
format or contract. Web Services consumers are not concerned with how the
Web Services executes their requests; they only expect that it will do so
according to the defined interface. Implementations can use anything from
Java 2 Platform, Enterprise Edition (J2EE™) entities to older code running in
a mainframe environment.
Loose coupling
The exposed interface is purposely “loosely coupled” from the Web Services
provider so that implementations can be modified or even replaced (swapped
in and out as desired in a plug and play manner), which increases the ability
to reuse existing function. Reuse promotes increased performance, reliability,
and Quality of Service (QoS), because a common interface can be exposed
to many clients regardless of what is underneath. Reusable components do
not have to be retested as often as well. Individual Web Services are also
loosely coupled, having little or no dependencies upon each other. Web
Services must also be stateless (information or state are not preserved from
one request to another).
Chapter 1. Introduction to SOA management 3
18. Business impact
Client applications in an SOA do not contain business logic; they consume
Web Services instead. Therefore, they become much smaller and easier to
implement. A shorter time-to-market for new products is an important result
for business development. SOA principles in general can help form a stronger
connection between the business needs that define the architecture and the
software and information systems that implement it. Web Services can be
created according to an abstract model that conforms to the needs of the
business. If done correctly, an abstraction can drive the concrete very
effectively.
1.3 SOA constructs and components
Standards have been developed to ensure Web Services technologies conform
to common principles. Support for Web Services interoperability, Web Services
security, sending attachments using Web Services, and Web Services
Management have been defined by organizations, such as W3C, OASIS, and so
on. Standards help ensure that Web Services products sold by different software
vendors conform to common agreed upon expectations. For example, a user
must be able to expect that a Web Services client written using Microsoft® .Net
can invoke a Web Services written using Apache Axis deployed on the
WebSphere Application Server without any complications if these tools conform
to Web Services standards.
With the advent of application server technology, Web Services can be
distributed across many machines and environments, making it easier to perform
scalability, clustering, and load balancing across your enterprise (SOA can ease
the transition away from existing systems towards modern application servers
and middleware). For example, COBOL and C++ applications that use
messaging software, such as IBM MQSeries®, can be phased out in favor of
Java applications that use SOAP/Java Message Service (JMS) and
message-driven beans (MDBs).
SOA-related constructs, such as enterprise service buses (ESBs), business
processes, and so on, can add further structure and flexibility to your architecture
by providing routing, mediation, and flow management functions. Just as you can
hide implementations from a client, transport/protocol layers and message
formats can be similarly concealed by the inclusion of an enterprise service bus.
SOAP/Java Message Service (JMS), SOAP/HTTP, Remote Method Invocation
(RMI)/Internet Inter-ORB Protocol (IIOP), XML/MQSeries, local Java calls, and
so on are all supported and transparent to the user. Often, people implement an
enterprise service bus by technologies that are found in certain
4 Managing an SOA Environment with Tivoli
19. middleware/infrastructure products, for example, WebSphere Enterprise Service
Bus, DataPower®, WebSphere Message Broker, and so on.
You can combine discrete Web Services into composite business processes to
accomplish more complex business objectives. As part of a business process,
individual Web Services can be viewed as activities within a workflow. This
facilitates the creation of a business process that can be described by a
meta-language called Business Process Execution Language (BPEL). Business
Process Execution Language is similar to Web Services Definition Language
(WSDL) in that it provides a static definition of a Business Process. Just as with
Web Services, the potential for reuse is quite high after these Business Process
definitions are created.
In recent years, a new method has been introduced to model Web Services in an
SOA called the Service Component Architecture (SCA). SCA components are
designed to separate implementation details from any business logic so that you
can put together an integrated application without knowing its implementation
details and make each component interoperate with any other SCA component.
Issues, such as security, transactions, and so forth, are resolved in a seamless
manner across SCA components. SCA components are often used in business
process modeling, because they provide a great deal of flexibility.
A mediation is a special type of SCA module. You can insert mediations between
loosely coupled Web Services. Introducing mediations between Web Services
provides added function for processing messages that are being passed
between these Web Services. Mediations intercept and modify messages that
are passed between existing Web Services and clients that want to use those
Web Services. Mediations are ideal for deployment in an environment that
contains an enterprise service bus, because they can reroute and examine Web
Services traffic.
1.4 SOA governance
It is important to consider both governance and management requirements when
planning an SOA.
SOA governance is an extension of IT governance that focuses on the life cycle
of Web Services and composite applications in an organization’s SOA. SOA
governance is related to establishing policies within the context of the activities
and constructs associated with SOA that are similar to those that exist for
managing and controlling other aspects of IT.
Chapter 1. Introduction to SOA management 5
20. Initially, the concept of SOA governance was applied narrowly to the
development and use of Web Services (validating Web Services adherence with
specific standards or managing Web Services in the SOA runtime environment).
Today, SOA governance spans SOA architecture, as well as the governance of
Web Services, across the entire implementation life cycle. Architecture
governance and Web Services-level life cycle governance are the two of the
main components of SOA governance. For the purposes of this discussion, we
will concentrate mainly on the latter.
1.4.1 Web Services life cycle governance
The goal of Web Services life cycle governance is to define:
Decision rights for the development, deployment, and management of new
Web Services
Monitoring and reporting processes for capturing and communicating
governance results
There are four phases of the Web Services life cycle:
Modeling: This phase involves incorporating business requirements and
objectives into your business design so that the design becomes a
specification of business processes that achieve goals and consider
assumptions.
Assembly: This phase centers around the information systems that will be
used to assemble the business design. Typically, an enterprise architect
working with a business analyst can convert the business design into a set
of business process definitions that are composed of activities. The required
Web Services are derived from the activity definitions and business
processes from the business process definitions.
Deployment: This phase involves creating the environment that will host
composite Web Service-based applications and then deploying those
applications there. This phase includes identifying resource dependencies
for the application, as well as operational conditions, requirements, and
constraints that impact the successful deployment and running of the
applications.
Management: This phase addresses the maintenance of the operational
environment and the policies that govern the deployed applications. This
phase includes monitoring the performance of Web Services requests and
responses and developing a recovery strategy for failures (detecting and
quarantining failures and logging them, rerouting traffic around failures and
recovering work affected by them, correcting problems and restoring the state
6 Managing an SOA Environment with Tivoli
21. of the system). The Management phase also includes tuning the operational
environment to conform to changes in the business design.
Because SOA applications are so loosely coupled, they introduce new
governance challenges. But with the proper standards, practices, and processes
in place, businesses can reap the full benefit of service orientation. Effective SOA
governance helps business and IT teams better identify how to achieve most
business goals. It also empowers employees by clearly defining their roles and
responsibilities.
1.4.2 Web Services life cycle management
After you implement the SOA governance framework, you use it in the model,
assemble, deploy, and manage phases within the SOA life cycle. With respect to
the operational aspects of implementing SOA governance, Web Services life
cycle management addresses how Web Services will be developed, deployed,
and managed.
Web Services life cycle management focuses on the development and
deployment of Web Services, while SOA governance supplies the decision
rights, processes, and policies for those activities. After a Web Services is
deployed, there must be management strategies in place to control and monitor
the Web Service. Web Services life cycle management is subject to the business
design created within the governance stage that ensures that reuse and cost
reduction are achieved.
1.5 SOA management considerations
Implementing SOA-based applications introduces new IT management
challenges. Because Web Services development tools make it easy to create
services within the SOA framework, there is a danger that services can
proliferate and become uncontrollable within the SOA enterprise, if the growth is
not managed properly. When systems are composed of multiple independent
business processes, the relationships between these processes and the
applications executing in the IT layer are not always obvious. For example,
consider verifying the correctness of a workflow in a system or locating a
performance bottleneck. While these actions are difficult in smaller systems and
might become impractical to manage as the size and complexity increases in
larger systems.
Simply stated, SOA management includes solutions and software for managing
and monitoring composite applications and the infrastructure that supports it
across the entire architecture.
Chapter 1. Introduction to SOA management 7
22. Considerations associated with managing an SOA environment include:
SOA management is needed on the following scenarios:
– Defining your Web Services in a way that they can be easily managed as
resources (Service Creation)
– Defining how Web Services relate to each other (Service Connectivity and
Interaction)
– How to manage your Web Services within their deployment environment
(Governance/Management)
The information that is needed from the management system:
– Capturing data that can be used to evaluate whether nonfunctional and
quality of service requirements as defined by business needs are
supported (reliability, scalability, cost, and so on)
– Defining which Web Services/Activities to group into Business
Processes/Workflows to accomplish a business goal (Collaboration
Services)
– Define service level agreements based on data that can be easily
monitored (average response time, maximum message size, and so on)
– Using standards whose enforcement can be monitored
– Evaluating whether your Web Services are secure (Security)
The management environment itself must be properly evaluated.
Considerations about the management technology are:
– How to display Monitored and Registered information to users (using a
console, such as the Tivoli Enterprise Portal (TEP) or the Tivoli Enterprise
Console® (TEC)
– What technology and products to use to monitor various resources
(examples include Simple Network Management Protocol (SNMP), Java
Management Extensions (JMX™), Java API for XML-Based RPC
(JAX-RPC), Java API for XML-Based Web Services (JAX-WS), and so on)
– How can you manage infrastructure that employs tiers and clustering.
Ensuring that both Web Services providers and consumers can be
monitored. Also, resolving how to display interactions if only certain things
are managed (how do you display unmanaged clients and Web Services
that are part of the same flow)
– What are some of the various user types that you will need monitor and
administer your applications (administrators, operators, and so on)
There are number of metrics that you must monitor for a holistic application
health view in an SOA environment.
8 Managing an SOA Environment with Tivoli
23. These metrics include:
Service response time
Service request message size
Service faults
Real-time service performance metrics
Application server health where the service is deployed
Health of dependent components, such as database, messaging resources,
and so on
Service performance metrics for historical purposes
Service request and response message data
These metrics help measure service level agreements (SLAs) for applications.
You can use these metrics to debug performance bottlenecks in applications.
You can also use these metrics to automatically take corrective actions or initiate
failover steps to maintain service availability when the primary service provider
goes down or is not performing to meet the SLAs.
IBM Tivoli Composite Application Manager (ITCAM) for SOA and other ITCAM
offerings provide all the necessary metrics to keep the applications in an SOA
environment healthy and resilient.
1.6 Users of SOA management
There are several types of users that use SOA management and need to interact
with the SOA-based environment. In this section, we discuss the access patterns
and the needs of those users in relation to SOA management.
With any management software, a number of various personnel can be potential
users. You can have multiple user roles, and each role can be defined in part by
the user’s current job responsibilities. For example, an application developer and
a performance tester most likely are interested in viewing varying pieces of
information at different times and thus might use the same console to view
different information. Depending on their responsibilities, they might even have
different levels of access to various views. Ultimately, the roles and
responsibilities that are defined for various users will depend greatly on how a
company defines their architecture and how they want to manage it.
Chapter 1. Introduction to SOA management 9
24. These are possible user roles for SOA management. The primary users are likely
to be common to most organizations. There can also be secondary and
provisional users as well:
Note: We consider the user roles for managing the performance and the
availability of an SOA-based application. We do not include security
management and its roles in this paper.
Primary users: These users are the most common in most organizations.
They have a critical need to use the SOA management tools for their
day-to-day jobs:
– Operator
– Middleware and application subject matter expert
Secondary users: These users have supportive roles that require occasional
access to the SOA management tools. Although the tools are not a necessity
for them to perform their work, the tools greatly enhance their productivity:
– Performance analyst
– System administrator
– Web Services application developer
Provisional users: These users also need access to SOA management only
as the need arises. These roles use SOA management rarely, and the tools
do not perform a critical role:
– Enterprise system management architect
– Business manager or executive
1.6.1 Operator
An operator or system operator is the first level of IT support personnel that might
detect system, application, or performance problems. The operator or system
operator job is directly related to ensuring the health of the IT environment and
attaining the appropriate service level for IT operation.
The primary interaction for an operator with the SOA management environment
relates to:
Monitoring for potential problems and correcting them
Ensuring that they adhere to the system SLA
Providing initial troubleshooting of a problem, and, if possible, fixing it
Escalating a problem to the next level of support or subject matter expert for
resolution if necessary
10 Managing an SOA Environment with Tivoli
25. To accomplish these objectives, an operator can perform monitoring by using
system management tools. The tools can provide views that the operator can
use to detect color-coded conditions or troublesome values in a measurement.
The tools can also provide automatic monitoring that generates events or alerts
for the operator to address. In most environments, the operator must rely on
alerts and events that happen on the environment instead of trying to navigate
the tools to uncover problems.
Using the Tivoli tools, IBM Tivoli Monitoring provides a facility for the operator to
get alerts and navigate views in order to problems in the Tivoli Enterprise Portal.
An alert might be sent to Tivoli Enterprise Portal to signal that a problem has
occurred. On other occasions, uncovering the problem might require
investigation by the operator. In either case, an operator must examine metric
data to see if the operator can make a preliminary diagnosis of the problem.
Event monitoring is represented by situations.
ITCAM provides situations that come predefined (ready to use) or that can be
modified by using the situation editor. Breaching an SLA can fire a situation to
alert the operator of the problem. The situation event console displays
information about these situations. Figure 1-1 shows the situation event console.
Figure 1-1 Situation event console
An operator can also look at the results of queries that are performed against
Tivoli Enterprise Monitoring Agent. These queries can be either predefined or
created by using the Tivoli Enterprise Portal query editor. The system or product
administrator loads the predefined queries into the Tivoli Enterprise Portal
Server. An administrator will often use the query editor to define and create
queries for display on the Tivoli Enterprise Portal.
1.6.2 Middleware or application subject matter expert
The subject matter experts on the application or the middleware perform the
in-depth problem determination. They might respond to problems that were
initially uncovered by an operator. For a composite application and specifically
for the SOA-based application, they must be able to see and trace the
Chapter 1. Introduction to SOA management 11
26. application beyond just the SOA interface and into the component level or a
breakdown of the transaction.
Subject matter experts on the application or the middleware perform these
in-depth diagnostics from several sources, such as:
Investigating the Web Services flow from one application server to another
Rerouting and modifying mediation and the Web Services flow
Collecting response time breakdowns using correlation tracking
Performing a method trace for the J2EE application
The subject matter experts on an application or the middleware perform these
functions by using a combination of ITCAM solutions, such as:
ITCAM for SOA
ITCAM for WebSphere and ITCAM for J2EE
ITCAM for Response Time Tracking
1.6.3 Performance analyst
A performance analyst inspects reports about availability and response time for
various applications, including applications deployed in an SOA environment. If
the metrics indicate that a threshold is close to being reached, the performance
analyst consults a trend analysis of how the application has been performing
over time.
The conclusion can indicate a sudden spike in activity or a increasing trend over
time, which might mean you need to expand your capacity. The performance
analyst collects this information from various data sources from the monitoring
system historical data.
The Tivoli solution in the IBM Tivoli Monitoring environment is based on Tivoli
Data Warehouse. The Tivoli Data Warehouse collects historical data from
various Tivoli Enterprise Monitoring Agents. This data can be reported and
analyzed by using the reporting tools that report into a DB2® database.
1.6.4 System administrator
The system administrator is a user with administrative privileges that performs
the day-to-day tasks of maintaining the management system. The system
administrator tasks include granting access to users, implementing a monitoring
solution, extending the monitoring solution by using a standard procedure, and
so on.
12 Managing an SOA Environment with Tivoli
27. The system administrator has the authority to modify the system, such as
installing a new feature or a patch to the monitoring system, thus, allowing
interaction with the management of the SOA solution.
The system administrator can install, configure, and maintain all the tools that a
subject matter expert needs. A system administrator can also configure, start,
and stop agent processes, or perhaps even reconfigure the entire monitoring
environment.
System administrator interactions include:
Managing situations, workspaces, and actions in Tivoli Enterprise Portal that
are related to the SOA application
Administering users, such as Tivoli Enterprise Portal users. Using the
Administer Users window for setting authorities to specific features,
specifying access to applications, and specifying access to Navigator views.
Selecting the features in Tivoli Enterprise Portal to provide access to each
user and to set the specific permissions granted to each user (see Figure 1-2
on page 14)
Chapter 1. Introduction to SOA management 13
28. Figure 1-2 Administer user dialog
1.6.5 Enterprise system management architect
The system management architect interacts with the business side of the
company and is familiar with the company’s business processes. An architect
must understand the model for the SOA-based application, including Web
Services, SCA, and business process choreography to model the architecture of
the business.
The architect observes as the model is built and then implemented into system
management. The architect must oversee the system management
implementation based on the business model and ensure that the management
model is kept up-to-date. The architect must review the rollout of the new
application and the change of the mediation rule to ensure that the monitoring
model is still current and meaningful.
14 Managing an SOA Environment with Tivoli
29. The architect utilizes the WebSphere Services Registry and Repository to
manage the Web Services life cycle and can use the discovery through Tivoli
Common Object Repository to monitor new services and the new model. You
can do this by displaying the ITCAM for SOA Services Overview view to see the
difference between which Web Services have been observed and which Web
Services are registered. If you notice that certain observed Web Services are not
registered, then you can update the registry.
1.6.6 Web Services application developer
The application developers are responsible for coding Web Services
applications. After the developer’s Web Services are deployed, the developers
must be available to fix problems. Depending on the severity of the problem, the
developers usually have a small amount of time in which to complete the fix, and
then they run tests to verify that the problem has been resolved. A typical
scenario is a new mediation that needs to be added to manage traffic for an
existing Web Service, because the system performance has been unexpectedly
affected.
The system management tools can assist the developer to identify the potential
bottlenecks and performance problems in test system. The developers need to
use the tools to test fixes for possible performance bottlenecks before the fixes
go into production. After the build with the fix is installed, the developers observe
whether the performance problem has been alleviated.
1.6.7 Business executives
The business executives care about their business processes and the
applications that support the business. The management solution must allow the
business executives to see the application health based on either the SOA
performance or another metric.
The business executives need an interface with minimum technical detail. The
business executives need a simple but meaningful view of the business process
health and possibly the SLA attainment.
Chapter 1. Introduction to SOA management 15
30. 1.7 Management needs for the SOA environment
Based on the user roles and the users’ needs in 1.6, “Users of SOA
management” on page 9, the management needs for the SOA environment are:
Understanding the current performance of the SOA application in real time in
order to proactively identify any potential outage and problem
Collecting historical trends of SOA performance
Building structure about the calling pattern of the Web Services
Understanding the structure and life cycle of Web Services
Performing “deep dives” and diagnoses on the SOA-based application
Modifying the routing and data analysis of the SOA Web Services calls
Showing the business impact of Web Services call performance problems or
outages
Managing security of the SOA-based application. We do not discuss security
management in this paper. See Understanding SOA Security Design and
Implementation, SG24-7310.
In a production environment, it is vital to have sufficient information for the
management system as long as collecting this information does not adversely
affect the managed environment.
To manage an SOA-based application, you need to have information about the
Web Services contained within it and the environment on which they are
deployed. You can observe data on Web Services by monitoring it or from static
definitions, such as WSDL documents that define a Web Service. Data on
Services, ports, and operations is exposed, as well as details about the server
(deployment environment) upon which the data runs (application servers, such
as WebSphere Application Server, WebLogic, JBoss, DataPower, and so on). As
with other software components, you must gather information throughout the
Web Service’s life cycle (see section on 1.4.2, “Web Services life cycle
management” on page 7).
1.7.1 Web Services metric data collection
You can collect metrics throughout the life of a Web Service. These metrics are
usually numeric information that you can use to indicate or calculate the health
and performance of Web Services. You can collect these metrics by using a
polling process or instrumenting the application to report the metrics.
Metrics that are collected at the Web Services operation level include average
response time, average message size, number of messages, number of faults,
16 Managing an SOA Environment with Tivoli
31. and so on. The metrics can provide valuable information about SOA and Web
Services. You need to be able to view real-time data, as well as historical data,
corresponding to a specific time interval.
You can collect metric data by using a management API, such as Java API for
XML-Remote Procedure Call (JAX-RPC) handlers, which make Web Services
into manageable resources. Data collection using the JAX-RPC handler happens
when when either of the following events occurs:
A client application invokes a Web service, which is referred to as a
client-side interception.
The Web Services request is received by the hosting application server,
which is referred to as a server-side interception.
The data can be stored in log files for later analysis, sent to a management
server, or loaded into a repository.
1.7.2 Web Services troubleshooting
Troubleshooting Web Services includes monitoring the health of the
infrastructure that supports the Web Services, such as the underlying
middleware. Think of Web Services as manageable resources in the system
management environment.
In this way, you can determine whether policies are being enforced in SOA and
whether SLAs are achieved or not. Data on throughput, availability, workload,
transactions, and so on can be gathered to help you determine if your current
topology is optimal, given the demands placed on your enterprise. Business
processes are managed indirectly in that the Web Services that make up the
business processes are managed resources.
You might also consider a SOA management standard, such as Web Services
Distributed Management (WSDM) from Oasis. See:
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsdm
1.7.3 Displaying data
You must be able to display Web Services entities, such as the deployed server,
operation, namespace, relationship, and other attributes, to users. You can
present this information in a tabular or graphical form, and you need to provide
drill-down capability for the user to see details about the Web Service.
A user must also be able to define events and alerts to monitor an SOA
application. These alerts allows users to be notified when a certain condition
Chapter 1. Introduction to SOA management 17
32. occurs. Alerts® allows the user to not have to monitor a particular view all the
time.
1.7.4 Mediation management
Mediation is a special type of SCA module. You can insert mediations between
loosely coupled Web Services. Introducing mediations between Web Services
provides added function for processing messages that are being passed
between these Web Services. Mediations intercept and modify messages that
are passed between existing Web Services and clients that want to use those
Web Services. Managing mediations includes the ability to filter mediations or
reroute Web Services calls based on monitoring metrics. You can perform this
function on a mediation platform, such as WebSphere Enterprise Service Bus or
WebSphere Message Broker.
18 Managing an SOA Environment with Tivoli
34. 2.1 ITCAM for SOA
in this section, we describe IBM Tivoli Composite Application Manager (ITCAM)
for SOA V6.1. We discuss:
“Product features” on page 20
“Product components” on page 21
2.1.1 Product features
ITCAM for SOA manages service-oriented architecture (SOA). It can monitor,
manage, and control the Web Services layer of the IT architecture, while drilling
down to the application or resource layer to identify the source of bottlenecks or
failures and to pinpoint services that take the most time or use the most
resources. ITCAM for SOA:
Provides service monitoring views in Tivoli Enterprise Portal. ITCAM for SOA
workspaces consist of data collector-based workspaces:
– Performance Summary: Shows the response time information for Web
Services calls as viewed from the client or the server
– Message Summary: Shows the message statistics, including the volume
and size of message information
– Fault Summary: Shows failure analysis for Web Services calls
Other workspaces for each agent are:
– Service Management Agent Environment: Provides a summary of the
Web Services metrics for all data collectors
– Service Management Agent: Shows the agent configuration summary,
data collectors, monitoring profiles, and filters
– Mediation Configuration: Shows configuration entries for mediation on
Service Component Architecture (SCA)
– Message arrival: Shows the message arrival rate and events based on the
message arrival critical situation
Leverages Tivoli Enterprise Portal situations to check thresholds. ITCAM for
SOA provides predefined situations that you need to tailor. The predefined
situations concern:
– Number of messages received by a service within a time window
– Size of the messages
Provides basic mediation support with the ability to filter or reject Web
Services call messages from a particular client or service. It can log request
and response messages for analysis.
20 Managing an SOA Environment with Tivoli
35. Offers heterogeneous platform coverage:
– Support for IBM WebSphere Application Server, CICS® Transaction
Server, Microsoft .NET, JBoss, BEA WebLogic, and other SOA clients and
servers
– Target IBM Enterprise Service Bus platforms: WebSphere Application
Server Versions 5.x and 6.x and WebSphere Business Integration Server
Foundation V5.1.1
Displays a list of services and operations that are monitored in the
environment
Leverages Tivoli Enterprise Portal workflow and policy editor for
threshold-triggered action sequences
Offers the ability to include services-layer views in Tivoli Enterprise Portal
The context-rich views and inter-workspace linkages in Tivoli Enterprise Portal
enables users to drill down to IT resources to identify Web Services bottlenecks
and failures. By providing built-in and extensible alerts, situations, and workflows,
users can create powerful automated mediation scenarios using the Tivoli
Enterprise Portal.
The service metrics, alerts, and automation workflows that are provided by
ITCAM for SOA and other Tivoli products can be displayed in Tivoli Enterprise
Portal with the cross-workspace linkages to provide a rich and multilayered
source of information. This information can help to reduce the time and skills that
are required for problem root-cause analysis and resolution.
ITCAM for SOA includes the Web Services Navigator, a plug-in to IBM Rational®
Application Development and other Eclipse-based tools. It provides a deep
understanding of the service flow, patterns, and relationships for developers and
architects. The Web Services Navigator uses data from the IBM Tivoli
Monitoring V6.1 Tivoli Data Warehouse or from the ITCAM for SOA log files
using the Log Assembler tool.
In Version 6.1, IBM Tivoli Composite Application Manager for SOA contains a
new component for mediation service management based on SCA. It enables
you to modify several of the mediation service settings dynamically. Mediation is
a facility that sits between Web Services requester and Web Services provider
that allows manipulation of Web Services messages, includes format translation,
filtering, and enrichment.
2.1.2 Product components
ITCAM for SOA manages Web Services. Web Services can be viewed as a
remote processing facility that is defined through the use of Web Services
Chapter 2. Tivoli application management products 21
36. Definition Language (WSDL). Usual access uses SOAP over HTTP. Internally,
Web Services are implemented using the Java API for XML-based Remote
Procedure Call (JAX-RPC). ITCAM for SOA installs itself as the JAX-RPC
handler to capture and manage Web Services calls.
ITCAM for SOA consists of these logical components:
Web Services data collector that acts as the JAX-RPC handler and intercepts
Web Services calls to collect statistical information and write to a log file.
Tivoli Enterprise Monitoring Agent that collects information from all of the data
collectors on a machine and forwards them to Tivoli Enterprise Monitoring
Server. We discuss the data collectors and Tivoli Enterprise Monitoring Agent
in “Monitoring agent data collector” on page 22.
An Eclipsed-based viewer that processes log files that are generated by the
Web Services data collector. It generates visual representations of various
characteristics of monitored Web Services. See “IBM Web Services
Navigator” on page 24.
Mediation SCA tools that enable partial monitoring of SCA within WebSphere
Enterprise Service Bus. See “Managing SCA mediation” on page 25.
Monitoring agent data collector
ITCAM for SOA works with several application server environments:
IBM WebSphere Application Server V5.1.0.5 with PQ89492, V6.0, and V6.1
IBM WebSphere Business Integration V5.1.1.1
IBM WebSphere Process Server V6.0.1
IBM WebSphere Enterprise Service Bus V6.0.1
IBM CICS Transaction Server V3.1 and later
BEA WebLogic Server V8.1.4
Microsoft .NET V1.1 with Service Pack 1 and V2.0
JBoss V4.03
WebSphere Community Edition V1.0 and its service packs
SAP® NetWeaver V6.40 with Service Pack 9 or later service packs
IBM WebSphere DataPower SOA Appliance Firmware V3.5.0.5 or later
Figure 2-1 on page 23 shows the ITCAM for SOA data collection conceptual
architecture.
22 Managing an SOA Environment with Tivoli
37. ITCAM for SOA
Monitoring agent
Application Server configuration
Web Services
handler or Data Data collector Tivoli Enterprise
extension collector adapter Monitoring Agent
log
Tivoli Enterprise
Management Server
Tivoli Enterprise Portal
Server
Figure 2-1 ITCAM for SOA structure
The monitoring agent data collector is implemented as a JAX-RPC handler or
service extension that is installed into the application servers that host the
monitored Web Services. The handler is given control when either of the
following events occurs:
A client application invokes a Web service, which is referred to as a
client-side interception.
The Web Services request is received by the hosting application server,
which is referred to as a server-side interception.
The monitoring agent records and collects monitored information into one or
more local log files. The information is then transferred to Tivoli Enterprise
Monitoring Server and can be archived into a historical database for later
retrieval with IBM Web Services Navigator.
ITCAM for SOA V6.1 focuses on the SOAP engine of IBM WebSphere
Application Server, WebSphere Service Integration Bus, Microsoft .NET
Framework, and BEA WebLogic.
The Web Services data collector supports both Java 2 Platform, Enterprise
Edition (J2EE) application client and server container environments, because
JAX-RPC handlers are supported only by these environments. The Web
Services must be compliant with JSR-109 specifications.
To ensure the proper operation of the JAX-RPC handler, verify that the client
applications are written according to the conventions at the following location:
http://www.jcp.org/aboutJava/communityprocess/final/jsr109/
Chapter 2. Tivoli application management products 23
38. IBM Web Services Navigator
IBM Web Services Navigator is an Eclipsed-based tool that is used to visualize
Web Services in an SOA environment. It provides a graphical display of:
Web Services transaction flows
Service topology
Flow patterns
Figure 2-2 illustrates Web Services Navigator concepts.
Metric
log
Data
collector TDW
warehouse
Metric
Tivoli Enterprise
log
Data Monitoring Agent
collector
Web Services
Navigator
Metric
log
Data
collector
Log Assembler
Combined
metric log
Metric
log
Data
collector
Figure 2-2 Web Services Navigator
The Web Services Navigator is a log-browsing tool intended for offline analysis of
SOA Web Services. The Web Services Navigator provides four primary views:
Statistic tables:
– Message statistics
Per-message statistics, including requestor, provider, send/receive time,
and message size
– Invocation statistics
Response time, network delay, message size, and more for each Web
Services invocation
– Transaction statistics
Statistics for aggregated transactions, including elapsed time, number of
faults, number of machines that this transaction involves, and number of
invocations comprising this transaction
24 Managing an SOA Environment with Tivoli
39. – Pattern invocation statistics
Statistics for discovered patterns, including operation names, number of
occurrences, response times, and message sizes
Note: To see the message content from the ITCAM for SOA metric log:
1. Set a monitor control higher than “none” for any or all of the Web
Services being monitored.
2. Include the subsequent xxxx.content.log when running Log Assembler.
Service topology view
This view is a graphical representation of the monitored Web Services that
displays aggregated information and details about the relationships between
Web Services.
Transaction flows view
The transaction flows view displays Universal Markup Language (UML) style
sequence diagrams. The transaction flow shows a chronological view of each
transaction, the flow between the various Web Services over time, and the
topology and statistics for each transaction. You can zoom in on the view to
see the details of individual transactions.
Flow pattern view
The flow pattern view is a visual representation of the aggregated pattern of
transactions represented in the log file. The view also represents each pattern
as a distinct sequence of Web Services calls and displays the frequency of
each pattern.
Managing SCA mediation
WebSphere Process Server and WebSphere Enterprise Service Bus introduce a
new way to model services in an SOA, which is called the Service Component
Architecture (SCA). SCA is designed to separate business logic from its
implementation so that you can focus on assembling an integrated application
without knowing implementation details.
There is a special type of SCA component, which is called a mediation. In an
SOA, where services are loosely coupled rather than connected directly to each
other, mediations can be inserted between the services, where they can
intercept and process messages that are passed between the services.
Mediations can process these messages and take appropriate actions, such as
reroute, log, or transform a message, or create a notification or an event.
IBM Tivoli Composite Application Manager for SOA provides the ability to
dynamically enable and disable the deployed mediation functions. This facility is
Chapter 2. Tivoli application management products 25
40. available for applications in the WebSphere Enterprise Service Bus or
WebSphere Process Server runtime environment. The invocation is provided in a
new workspace in Tivoli Enterprise Portal called the Mediation Configuration
workspace. The actions are:
ConfigureMediation_610
DeletePrimitiveProperty_610
2.2 ITCAM for WebSphere and ITCAM for J2EE
The IBM Tivoli application management solution for J2EE application servers
comes in the form of ITCAM for WebSphere and ITCAM for J2EE. These two
products share the same managing server. ITCAM for WebSphere and ITCAM
for J2EE observe and report on the health of J2EE-based applications. They
track the progress of applications as they traverse through J2EE application
servers, middleware adapters and transports, and database calls, and on to
back-end systems, such as CICS or IMS™, to extract business data or to invoke
mainframe business processes.
Tracking applications produces request traces, where the events in a request’s
life are recorded and stored in a monitoring repository database. ITCAM for
WebSphere and ITCAM for J2EE capture the CPU and the elapsed internal
times when events are called and when they are exited, measuring as far down
as the CPU times consumed and the elapsed internal times charged to individual
methods in J2EE classes. The methods or events taking the most time are
marked as an application’s parts that deserve attention for runtime improvement
studies and code optimizations.
ITCAM for WebSphere manages and monitors WebSphere-based application
servers, while ITCAM for J2EE manages and monitors the following J2EE
containers:
JBoss
Tomcat
SAP NetWeaver®
BEA WebLogic Server
Oracle® Application Server
Apache Web Server
Sun™ Java™ System Web Server
Microsoft IIS
WebSphere Application Server CE
ITCAM for WebSphere and ITCAM for J2EE do not need modification of any
J2EE or mainframe application code. The data collectors use the following
principal data sources: Java Virtual Machine Tool Interface (JVMTI) interfaces
26 Managing an SOA Environment with Tivoli
41. and primitives. ITCAM for WebSphere also uses WebSphere Performance
Management Interface (PMI) and z/OS System Measurement Facility (SMF)
120 records. The monitoring data is collected and analyzed to offer a wealth
of information about the health of J2EE applications and their servers.
These products collect and report many system-level performance metrics about
J2EE application servers. The status of the servers and their resources
(particularly at vital checkpoints, such as CPU utilization), memory usage, and
the status of internal components, such as database connection pools, JVM™
thread pools, EJB™ usage, and request processing statistics, can be extremely
important in locating real-time problems with J2EE applications. ITCAM for
WebSphere and ITCAM for J2EE bring attention to these critical indicators with
real-time, graphical displays of their values and their trends over spans of time.
2.2.1 Architecture and interconnection
ITCAM for WebSphere and ITCAM for J2EE are distributed performance
monitoring applications for application servers. The components are connected
through TCP/IP communication. The central component of ITCAM for
WebSphere and ITCAM for J2EE, the managing server, is its heart and brain. It
collects and displays various performance information from application servers.
Chapter 2. Tivoli application management products 27
42. The application servers run the data collector, which is a collecting agent that
runs in the application server and sends monitoring information to the managing
server. These data collectors operate independently of each other. Figure 2-3
shows the overall architecture of ITCAM for WebSphere and ITCAM for J2EE.
Browser interface
ITCAM for WebSphere
ITCAM for J2EE
Managing Server
I
Tivoli Enterprise
Monitoring Agent
e
ris nt
rp
te ge Tivoli Enterprise
En ng A
li i
Application servers with vo or
Ti onit
Management Server
Data collectors M and
Tivoli Enterprise
Portal Server
Web Servers
Figure 2-3 ITCAM for WebSphere and ITCAM for J2EE architecture
The application monitor consists of two major parts: the managing server and the
data collectors. A data collector agent runs on each monitored application server,
whether J2EE, CICS, or IMS, and communicates essential operational data to
the managing server. Unique sampling algorithms maintain low CPU and
network processing while providing application-specific performance information.
The managing server consists of several Java-based components that provide
the environment to collect and present management data.
2.2.2 The managing server
ITCAM for WebSphere and ITCAM for J2EE use one common managing server
that controls and coordinates data collectors for J2EE, CICS, and IMS servers
28 Managing an SOA Environment with Tivoli
43. that run applications. The difference between ITCAM for J2EE and ITCAM for
WebSphere is the platform support for the data collectors. These data collectors
can run independently.
The managing server uses the following software:
Managing server database (DB2 UDB or Oracle on Sun Solaris™) for the
relational data repository
WebSphere Application Server to run the visualization engine Web console
application
An optional Web server, such as IBM HTTP Server
The managing server overseer components, which are a set of Java-based
processes
The overseer components are the controlling logic for the managing server. For
the overseer components:
Kernels control the managing server. There are always two copies of the
kernels running on an ITCAM for WebSphere and ITCAM for J2EE managing
server for redundancy and failover. The kernels register components as they
join the managing server, periodically renew connections and registrations
with components and data collectors, and collect server and component
availability information.
Publishing servers receive application and system event data from the data
collectors, gather and compute request-level information about performance
metrics such as response times, and implement the trap monitoring and alerts
features.
Archive agents receive monitoring data from the publish servers and store the
monitoring data in the repositories of ITCAM for WebSphere and ITCAM for
J2EE.
The global publishing server collects information from the publish servers
and correlates all parts and pieces of multiserver requests, such as requests
from J2EE servers to execute CICS or IMS programs.
The message dispatcher is a conduit for messages from ITCAM for
WebSphere and ITCAM for J2EE using e-mail and Simple Network
Management Protocol (SNMP) facilities.
Chapter 2. Tivoli application management products 29
44. The visualization engine is a Web-based GUI with access to graphics, ITCAM
for WebSphere and ITCAM for J2EE performance reports, real-time views of
different slices of monitoring data, and ITCAM for WebSphere and ITCAM for
J2EE internal commands and event-driven functions. The visualization
engine runs on a WebSphere Application Server.
Figure 2-4 shows the conceptual relationship among the components.
Snapshot traffic
Publish traffic
Global Publish
Server (SAM)
Publish Server (PS)
Kernel (KL) Visualization Engine
Provide services on: Provide services on:
Message Dispatcher - Lookup -Administration
(MD) - Registration -Availability
- Recovery -Problem Determination
Archive Agent (AA) - Configuration -Performance Management
OCTIGATE
database
Figure 2-4 Kernel components
At the managing server, monitoring data is prepared for real-time displays within
the monitoring console and is inserted into the OCTIGATE data repository.
These are extremely resource-intensive operations. Having this processing in
the managing server isolates this from other the application servers, thus
reducing the footprints of ITCAM for WebSphere and ITCAM for J2EE in the
monitored systems. This design also helps keep the data collectors’ processing
at levels low enough for 24x7 production system monitoring.
Data from the data collectors is collected by the publishing server and then
stored in the OCTIGATE database by the archive agent. The visualization engine
reads the database to present data through the Web console, and snapshot
information, such as lock analysis and in-flight transactions, is retrieved directly
from the data collectors.
2.2.3 J2EE and WebSphere data collectors
The data collectors run inside the application servers. They use native system
services, and they are tailored for the particular environments where they
30 Managing an SOA Environment with Tivoli
45. execute. The data collectors for z/OS systems are written to take advantage of
services on z/OS, such as MVS™ Cross-Memory Services and address space
fencing, which are not available on distributed systems.
Data collectors are configured as a multithreaded process. They consist of the
following agents:
Command agent
The command agent collects requests from other components for information
about Enterprise Java Bean (EJB) invocations, database connection pools,
thread pools, stack traces, memory analyses, and heap dumps.
Event agent
The event agent provides data to the publish servers according to polling
frequencies. This data includes system initialization data, application
request-level data, and application method-level data.
Secondary collector
The optional secondary collector provides support for displaying data in Tivoli
Enterprise Portal for collecting WebSphere Application Server and other
J2EE application server performance metrics. This component communicates
with Tivoli Enterprise Monitoring Agent using a TCP/IP port.
Collectively, these agents and other data collector routines unleash the probes,
package the monitoring data into Java formats for the managing server, and
deliver the data to the managing server.
The data collectors send the probes into the application servers to analyze the
applications’ performance. The probes collect monitoring data and feed it to
transport routines that in turn route the data to the managing server. The
managing server processes it for display in the Web console and for storage in
the OCTIGATE repository, which relieves the processing burden of ITCAM for
WebSphere and ITCAM for J2EE from the application servers as much as
possible. The data collectors and probes are not designed to analyze or interpret
data, but to collect it and route it as quickly as possible to the managing server
where the analysis is performed.
The data sources that are employed by ITCAM for WebSphere and ITCAM for
J2EE are:
Java Virtual Machine Tools Interface (JVMTI) garbage collection data,
method trace, stack trace, CPU time, and heap dump
Java Management Extensions (JMX) system resources
System Management Facilities (SMF) system resources (z/OS only)
PMI system resources (WebSphere only)
Chapter 2. Tivoli application management products 31