This document discusses Business Process Insight (BPI), an approach and platform for discovering and analyzing end-to-end business processes. It presents the BPI lifecycle, architecture, and addresses key research challenges. The architecture uses a cloud-based data storage and includes modules for data integration, correlation, process mining, comparison and predictive analytics. It aims to provide process intelligence through analytics on both historical and real-time data to improve business operations and manage risks. Future work areas include balancing data scale and query capabilities and parallelizing algorithms.

1. Szabolcs Rozsnyai
July 2012
Business Process Insight
An Approach and Platform for the Discovery and Analysis of End-to-End Business
Processes
Szabolcs Rozsnyai, Geetika T. Lakshmanan, Vinod Muthusamy, Rania Khalaf and
Matthew J. Duftler
© 2009 IBM Corporation

2. IBM Presentation Template Full Version
Agenda
 Introduction and Motivation
 BPI Life-Cycle
 Architecture
 Research Challenges
 Conclusion and Future Work
Source: If applicable, describe source origin
2 © 2009 IBM Corporation

3. Introduction and Motivation 1/2
Understanding, managing and improving business processes in complex environments proves
to be a significant challenge and has a severe impact on the organizations process maturity
Organizational Challenges Technical Challenges
•Business processes • Processes are not coordinated by one entity
• can stretch across complex organizational silos and in • Systems are loosely coupled, heterogeneous and distributed
many cases even extends to customers • Business Process artifacts range from simple record entries to
• are not necessarily complete or accurate complex events at various granularity levels
• are heavily human-driven, require a lot of knowledge • Business Process activities might be represented through
and have a large number of exceptions multiple events
• Are simplified to preserve a high degree of freedom • Sometimes workflow engines emit events to mark the
• Are often in the heads of individuals, groups or buried start and end of an activity
in application logic
3 © 2009 IBM Corporation

4. Introduction and Motivation 2/2
 We propose a system to enable Process Intelligence from two perspectives
– Analytics on historical data
• to understand what, how, who and why aspects of end-to-end business process based on real-time and
historical data
• identify root causes of problems,
• understand process deficiencies and
• provides means to improve process performance
– Analytics on real-time data
• to increase the effectiveness of business operations, and managing operational risk
• to identify and predict situations in order to react on them
BPI platform is a software as a service (SaaS) enabled, collaborative system that realizes the end-to-end
BPI life-cycle.
Process Intelligence
BI BAM CEP BPM
Process Mining
The platform allows users to manage a variety of data at different levels of granularity including raw captured events,
correlated instance traces, mined process models, and prediction alerts.
4 © 2009 IBM Corporation

5. BPI Life-Cycle
5 © 2009 IBM Corporation

6. Architecture Overview
6 © 2009 IBM Corporation

7. Architecture – Data Management
• Volume and the complexity makes tracking and processing a
difficult and resource intensive task
• As data grows at a very high rate, tracking arbitrary artifacts for
provenance purposes within large organizations is very costly
•
Storing, organizing, retrieving and analyzing the artifacts
necessitate allocating large amount of computing resources
•
RDBMS requires trade-offs need to be made between the
amount of captured data and the granularity levels
• Aggregation vs. leaving out data
 both impact the potential for analytics
7 © 2009 IBM Corporation

8. Architecture – Data Management
• Cloud-based elastic storage (Hadoop/HBase)
• Distributed column-oriented key-value storage
• NoSQL but BPI API supports
• a limited set of queries
• Joins with constraint that has high selectivity
• Secondary indexing
• Allows to compose annotated graphs of relationships
8 © 2009 IBM Corporation

9. Architecture – Data Integration
• Schema-less structure easily allows
• to “dump” everything into data storage
• following a LET (Load Extract Transform) paradigm in
contrast to classical ETL approaches
• RAW data is preserved
• Attributes of interest are extracted based on deployed
and configured transformers
• Integration options:
• Using ESB (especially for real-time processing)
• Loading files that are following a defined XML
schema
9 © 2009 IBM Corporation

10. Architecture – Correlation Module
• Correlation Discovery
• Determines correlation rules that express how certain
events are related to each other by combining a unique
combination of statistics on event attributes
• Applies graph reduction algorithms to reduce the
number of correlation rules
OrderToShipment :
OrderReceived.OrderId = ShipmentCreated.OrderId,
ShipmentCreated.ShipmentId = TransportStarted.ShipmentId,
TransportStarted.TransportId = TransportEnded.TransportId
How can I reduce the complexity for rules?
10 © 2009 IBM Corporation

11. Architecture – Correlation Engine
• Higher level aggregations can be created that include
several lower level aggregation nodes using representation
of correlations.
• Statistics can be calculated over correlated events and
updated every time new events enter a correlation
• User can place queries for aggregates and drill-down based
on his interests
© 2009 IBM Corporation

12. Architecture – Process Aware Analytics
• Pluggable analytics module for
• Process mining
• Process comparison
• Predictive analytics
• Process Mining
• Algorithms can be plugged in (Alpha, Heuristics,
Biased, …)
• Results are transformed to a BPMN representation
• Queries can be applied to mine subsets of traces to
observe variations in the behavior
© 2009 IBM Corporation

13. Architecture – Process Aware Analytics
• Process Comparison
• Tree-Based comparison returns a detailed diff-list of
the process model
• Visual Overlay returns a visual representation of
how process models differ from each other
© 2009 IBM Corporation

14. Architecture – Process Aware Analytics
• Predictive Analytics
• algorithms in BPI currently include decision trees and
an instance-specific probabilistic process model
© 2009 IBM Corporation

15. Research Challenges
BPI addresses several key challenges defined by the process mining manifesto *)
C1 Finding, Merging, and Cleaning Event Data C4 - Dealing with Concept Drift
When extracting event data suitable for process mining The process may be changing while being analyzed.
several challenges need to be addressed: Understanding such phenomena is of prime importance
for the management of processes.
• data may be distributed over a variety of sources,
• event data may be incomplete,
• an event log may contain outliers and
• events at different level of granularity.
C2 Dealing with Complex Event Logs Having Diverse
C7 - Cross-Organizational Mining
Characteristics
Event logs may be extremely large making them difficult to Some organizations work together to handle process
handle whereas other event logs are so small that not enough instances (e.g., supply chain partners) or organizations
data is available to make reliable conclusions. are executing essentially the same process while sharing
experiences, knowledge, or a common infrastructure. The
analysis of event logs originating from multiple
organizations provides several challenges.
C8 - Providing Operational Support
Process mining is not restricted to off-line analysis and
can also be used for online operational support. Three
operational support activities can be identified: detect,
predict, and recommend..
15 © 2009 IBM Corporation

16. Future Work
 Scale vs. Query Expressiveness
– Data management scales out on cost of query expressiveness
• Experiments with relational-cloud hybrid models
 Parallelizing algorithms to scale-out
16 © 2009 IBM Corporation

17. Thank You
17 © 2009 IBM Corporation