BPM in the Cloud: Early Research Works and Challenges
Upcoming SlideShare
Loading in...5
×
 

BPM in the Cloud: Early Research Works and Challenges

on

  • 347 views

With an estimated market volume of US$ 130 billion and double-figure growth rates, cloud computing is on the path to change many aspects of our digital lives. In this talk I consider the question of ...

With an estimated market volume of US$ 130 billion and double-figure growth rates, cloud computing is on the path to change many aspects of our digital lives. In this talk I consider the question of potential benefits and open challenges when bringing BPM research into the cloud, such as (i) how can BPM benefit from the cloud, (ii) what should BPM in the cloud look like, and (iii) what can BPM bring to cloud computing practices? In this talk I will present three works around those questions, namely (a.) end-user-friendly process modelling in the cloud, (b.) process-aware auto-scaling of cloud resources, and (c.) process mining for error diagnostics in cloud management processes.

Statistics

Views

Total Views
347
Views on SlideShare
308
Embed Views
39

Actions

Likes
2
Downloads
9
Comments
0

3 Embeds 39

http://www.businessprocessincubator.com 22
http://pc040.trisotech.com 16
https://www.businessprocessincubator.com 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

BPM in the Cloud: Early Research Works and Challenges BPM in the Cloud: Early Research Works and Challenges Presentation Transcript

  • NICTA Copyright 2012 From imagination to impact BPM in the Cloud: Early Research Works and Challenges Dr. Ingo Weber Senior Researcher NICTA, Sydney (ATP) September 2013
  • NICTA Copyright 2012 From imagination to impact Cloud Service Market Growth vs. BPM Market Gartner (2013) • The cloud is big, and here to stay
  • NICTA Copyright 2012 From imagination to impact Cloud Service Market Growth vs. BPM Market Gartner (2013)BPM market estimated at $2.6B in 2013, $7B in 2018 (PRWeb 16/6/2013) • The cloud is big, and here to stay • By comparison, BPM is a dwarf
  • NICTA Copyright 2012 From imagination to impact Why BPM in the Cloud? • BPM as a standalone system may not take off… – But there are opportunities to use BPM thinking, techniques, and tools in other domains / applications • BPM in the cloud will happen – If something provides value, the industry will build it – This is an opportunity for the BPM research community to have practical impact on a promising area • Many avenues to explore… – This talk aims to provide a short overview of where we are at, what is needed, and some of my works. 4
  • NICTA Copyright 2012 From imagination to impact Cloud Computing Overview 5 Service Model SaaS Software as a Service Examples: CRM, email, online games, vitual desktops, etc. PaaS Platform as a Service Examples: web server, database, execution runtime, development tools, etc. IaaS Infrastructure as a Service Examples: virtual machines, storage, load balancers, network, etc. Examples in BPM In-browser modelling, e.g., Signavio, Oryx Specialized environments, e.g., IFTTT, Effektif Complete environments, e.g., Intalio|Create, Cordys (now OpenText) BPM execution platforms, e.g., Intalio Workflow tools, e.g., AWS Simple Workflow Service - Left-hand side based on Wikipedia article Cloud Computing, 09/2013
  • NICTA Copyright 2012 From imagination to impact SaaS-level BPM for end-users? • Characteristics needed – Limit functionality to something small but useful – User experience – Simplicity – Social aspects for easy connection to colleagues/friends – Short learning curve: people need to understand the tool very quickly • Are we there yet? – ...or should we try and re-think what BPM in the cloud can be? 6 Top part inspired by T. Baeyen„s talk at BPM, 08/2013
  • NICTA Copyright 2012 From imagination to impact Example: IFTTT does this quite well 7 www.ifttt.com, 09/2013
  • NICTA Copyright 2012 From imagination to impact Example recipes in IFTTT 8 www.ifttt.com, 09/2013 Whenever a game of the Boston Red Sox ends, send me an email
  • NICTA Copyright 2012 From imagination to impact Example recipes in IFTTT 9 www.ifttt.com, 09/2013 Whenever a game of the Boston Red Sox ends, send me an email Whenever someone tags me on a facebook photo, download that photo to my dropbox account
  • NICTA Copyright 2012 From imagination to impact Example: IFTTT 10 www.ifttt.com, 09/2013 If there is a birthday of a friend in my Google calendar for today, post „happy birthday“ on their wall
  • NICTA Copyright 2012 From imagination to impact Are we there yet? • Questions to drive research directions – How can BPM benefit from the cloud? – What should BPM in the cloud look like? – What can BPM bring to cloud computing practices? • Next: some of my early works 11
  • NICTA Copyright 2012 From imagination to impact How can BPM benefit from the cloud? • SaaS – No installation, very little burden for people to get started – if the tool fulfils the characteristics while being useful – My work: FormSys Process Designer [1,2] • PaaS / IaaS – Use cloud resources to enable process execution – Benefit from elasticity to stay within SLAs, but not over-pay for over-provisioning – My work: process-aware auto-scaling of cloud resources (on the IaaS level) [3] 12
  • NICTA Copyright 2012 From imagination to impact Process-aware auto-scaling of cloud resources • Assumption – Executing process model that calls Web services (WSs) – WSs are fully automated and perform compute-intensive tasks – Horizontal scaling of the VMs running the WSs is possible – Complexity of a task can depend on the input • Example – Integration logic around proprietary systems (e.g., in COBOL) – Translation, transformation, aggregation can create high utilization of resources (CPU, disk I/O, etc.) and slow down process execution • Problem – Say we have external SLAs for the whole process, e.g., to complete 95% of instances within 300s – How many resources to schedule for each task, when the requests change (complexity, frequency)? 13
  • NICTA Copyright 2012 From imagination to impact • Setup of our experiment 14 Process-aware auto-scaling of cloud resources Elastic Load Balancer 1 Auto Scaling Group 1 Instances for Service 1 Elastic Load Balancer 2 Auto Scaling Group 2 Instances for Service 2 Elastic Load Balancer 3 Auto Scaling Group 3 Instances for Service 3 ServiceProvider Service 1 Service 2 Service 3
  • NICTA Copyright 2012 From imagination to impact • Early work: use a fuzzy controller for process-aware auto-scaling • Process backlog on cloud IaaS: process-aware vs. standard auto-scaling Process-aware auto-scaling of cloud resources Load profile (different scale) Complexity of the requests increases, plateaus, then decreases, and finally no new instances are created
  • NICTA Copyright 2012 From imagination to impact • Early work: use a fuzzy controller for process-aware auto-scaling • Process backlog on cloud IaaS: process-aware vs. standard auto-scaling Process-aware auto-scaling of cloud resources Backlog: in a given minute, how many more process instances are started than completed? Load profile (different scale)
  • NICTA Copyright 2012 From imagination to impact • Early work: use a fuzzy controller for process-aware auto-scaling • Process backlog on cloud IaaS: process-aware vs. standard auto-scaling Process-aware auto-scaling of cloud resources Load profile (different scale) “Cut-offs” show where time-outs happened, caused by the ELBs.
  • NICTA Copyright 2012 From imagination to impact What should BPM in the cloud look like? • SaaS – My work: FormSys Process Designer (2010-2011) [1,2] – Metaphor: forms – ideally the ones known by users – Approach: scripting (not Programming by Demonstration), but made easy – For execution & to enable technical editing of end-user-created process models: translation to BPEL – Features: • user-editable forms are linked to Web services • restricted, yet powerful generic composition language • immediate automated process verification • process simulation in the design phase • automatic code generation: parallelized orchestrations – Evaluated in a user study 18
  • NICTA Copyright 2012 From imagination to impact FormSys Process Designer - core ideas • Service: name + icon – Visualize Data Service • Messages to / from services – Example: Sirca, computing GARCH Daily Volatility 19 Form through which the user usually controls a service
  • NICTA Copyright 2012 From imagination to impact FormSys Process Designer - core ideas • Service: name + icon – Visualize Data Service • Messages to / from services – Example: Sirca, computing GARCH Daily Volatility 20 Form through which the user usually controls a service Fields in the form can be linked to parts of WSDL messages (by a developer)
  • NICTA Copyright 2012 From imagination to impact FormSys Process Designer - example • Financial data analysis process: 1. Find news data: e.g., news data on the company „BHP‟ 2. Find performance data: e.g., on company code „BHP.AX‟ 3. Aggregate the performance data, e.g., avg. hourly stock prices 4. Merge datasets: e.g., merge the result data sets from the first and third steps 5. Visualize dataset: e.g., to see influence of news on prices 21 Control flow modelling Data flow modelling, see [1,2] for details
  • NICTA Copyright 2012 From imagination to impact FormSys Process Designer - User study [1] • Ability to solve tasks in FormSys Process Designer – Tech.: users with technical expertise – Fin.: users with financial expertise – Gen.: users with expertise in both 22 0 1 2 3 4 5 6 7 8 Tech. Gen. Fin. Solved Not solved Not attempted
  • NICTA Copyright 2012 From imagination to impact • Times by expertise 23 0:00 0:14 0:28 0:43 0:57 1:12 1:26 Gen. Tech. Fin. Time(h:mm,successonly) Expertise FormSys Process Designer - User study [1]
  • NICTA Copyright 2012 From imagination to impact • Times by expertise 24 0:00 0:14 0:28 0:43 0:57 1:12 1:26 Gen. Tech. Fin. Time(h:mm,successonly) ExpertiseMean time using Intalio|bpms FormSys Process Designer - User study [1]
  • NICTA Copyright 2012 From imagination to impact What can BPM bring to cloud computing practices? • My work: using process mining in systems operations (a.k.a. cloud administration / management) – Disclaimer: started only 2 months ago • Core question: can a process view help in operations? 1. Understanding how a tool is running 2. Understanding what the best practice is (manual processes) 3. Detecting when process steps are supposedly complete, but failed 4. Detecting deviations from best-practice / usual path 5. Detecting unusual log messages 25
  • NICTA Copyright 2012 From imagination to impact What can BPM bring to cloud computing practices? • My work: using process mining in systems operations (a.k.a. cloud administration / management) – Disclaimer: started only 2 months ago • Core question: can a process view help in operations? 1. Understanding how a tool is running 2. Understanding what the best practice is (manual processes) 3. Detecting when process steps are supposedly complete, but failed 4. Detecting deviations from best-practice / usual path 5. Detecting unusual log messages 26 Process discovery Assertion checks Conformance checks Classification Approaches:
  • NICTA Copyright 2012 From imagination to impact Process Discovery for System Operations • Challenges: – How to get the right logs – How to correlate events from several logs • Especially in a large-scale distributed system • What is a process instance – How to handle concurrent sub-processes – How to deal with low-level log messages – How to analyze incomplete log messages – How to deal with 1000s of log lines per instance 27
  • NICTA Copyright 2012 From imagination to impact Process Discovery – Example: Asgard • Netflix Asgard is a cloud management tool – Open source – For Amazon Web Services (AWS) – Sunny day example: 1 log source, high quality logs, ... • Example process: rolling upgrade – “Rolling”: upgrade one virtual machine (VM) at a time – Auto-scaling group (ASG): set of VMs from the same Amazon machine image (AMI), usually behind an elastic load balancer (ELB) – Upgrade by replacing AMI for the ASG, then de- register a VM from the ELB, terminate the VM, wait for ASG to create a new VM from the new AMI 28
  • NICTA Copyright 2012 From imagination to impact Process Discovery – Asgard logs: 3 lines [2013-07-18 15:37:31,369] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Started on thread Task:Pushing ami-ad059597 into group hadoopcluster--firstASG for app hadoopcluster. [2013-07-18 15:37:31,996] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Updating launch from hadoopcluster--firstASG-20130718120001 with ami-ad059597 into hadoopcluster--firstASG-20130718153731 [2013-07-18 15:37:31,998] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Create Launch Configuration 'hadoopcluster--firstASG- 20130718153731' with image 'amiad059597' 29
  • NICTA Copyright 2012 From imagination to impact Process Discovery – Asgard logs: 3 lines [2013-07-18 15:37:31,369] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Started on thread Task:Pushing ami-ad059597 into group hadoopcluster--firstASG for app hadoopcluster. [2013-07-18 15:37:31,996] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Updating launch from hadoopcluster--firstASG-20130718120001 with ami-ad059597 into hadoopcluster--firstASG-20130718153731 [2013-07-18 15:37:31,998] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Create Launch Configuration 'hadoopcluster--firstASG- 20130718153731' with image 'amiad059597' 30 „Instance ID“
  • NICTA Copyright 2012 From imagination to impact Process Discovery – Asgard logs: 3 lines [2013-07-18 15:37:31,369] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Started on thread Task:Pushing ami-ad059597 into group hadoopcluster--firstASG for app hadoopcluster. [2013-07-18 15:37:31,996] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Updating launch from hadoopcluster--firstASG-20130718120001 with ami-ad059597 into hadoopcluster--firstASG-20130718153731 [2013-07-18 15:37:31,998] [Task:Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] com.netflix.asgard.Task 2013-07-18_15:37:31 135: {Ticket: null} {User: null} {Client: localhost 0:0:0:0:0:0:0:1%0} {Region: apsoutheast- 2} [Pushing ami-ad059597 into group hadoopcluster-- firstASG for app hadoopcluster] Create Launch Configuration 'hadoopcluster--firstASG- 20130718153731' with image 'amiad059597' 31 Payload AMI ID ASG ID
  • NICTA Copyright 2012 From imagination to impact Process Discovery – Example: Asgard 32
  • NICTA Copyright 2012 From imagination to impact Process Discovery – Approach 33
  • NICTA Copyright 2012 From imagination to impact Open Challenges 34 Analysis Design Implementation Enactment Monitoring Evaluation On-demand self service Broad network access Resource pooling Rapid elasticity Measured service SaaS PaaS IaaS Private cloud Community cloud Public cloud Hybrid cloud BPMlife-cycle (Mendling 2007) NISTDefinitionofCloudComputing EssentialCharacteristics Service Model Deployment Models • Some dimensions when thinking about solutions Disclaimer: likely incomplete and needs per-tool analysis
  • NICTA Copyright 2012 From imagination to impact Summary & Outlook • BPM in the cloud will happen – If something provides value, the industry will build it – This is an opportunity for the BPM research community to have practical impact on a promising area • Presented some early work – Process-aware auto-scaling of cloud resources [3] – Domain-user BPM [1,2] – Process view on system operations using mining • Contact us for more information – http://ssrg.nicta.com.au/people/?cn=Ingo+Weber 35
  • NICTA Copyright 2012 From imagination to impact References 1. Ingo Weber, Hye young Paik, and Boualem Benatallah. Forms- based web service composition for domain experts. ACM Transactions on the Web (TWEB), conditionally accepted, 2013. 2. Ingo Weber, Hye young Paik, Boualem Benatallah. Forms-based Service Composition. Accepted for publication in ICSOC'11: 9th International Conference on Service Oriented Computing, short paper, Paphos, Cyprus, December 2011. 3. Christian Janiesch, Ingo Weber, Michael Menzel, and Jörn Kuhlenkamp. Optimizing the performance of automated business processes executed on virtualized resources. In HICSS'14: Hawaiian International Conference on System Science, January 2014, to appear. Acknowledgements: Thanks to Christian Janiesch and Len Bass for their input to the slides. And thanks to many colleagues for the discussions and feedback. 36