PTV Group_impact_camunda_bpm_20140122

Uploaded on


More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Camunda Community Meeting USING BUSINESS PROCESS MANAGEMENT IN SPATIAL DATA PROCESSING Siegfried Klausmann, Dominik Eisenberg Karlsruhe, 22.01.2014
  • 2. AGENDA 1. 2. 3. 4. 5. 6. 7. 8. Motivation and goals Main requirements System Architecture Demo of current solution Process/Service concepts Experiences with BPM introduction Future plans Discussion / Best practices
  • 3. MOTIVATION – PLANNING AND OPTIMISATION SOFTWARE Planning and optimising the flow of people Planning and optimising the flow of goods I Page 4
  • 4. MOTIVATION – PRODUCTS NEED OPTIMIZED DATA Provider raw data Quality Assurance steps Validation, Harmonization Database Conversion to binary formats  Vision Suite - PTV Visum, PTV Vissim, PTV Viswalk: Traffic planning, traffic technology and simulation  PTV Map&Guide: Transport route planning  PTV Smartour: Transport planning and optimisation  PTV Map&Market: Geomarketing, sales, marketing and sales force planning  PTV xServer: Developer components with core algorithms I Page 5
  • 5. MOTIVATION – CURRENT DATA PROCESSING Globaltabellen bearbeiten PND 1 PremiumNetz bearbeiten eKarte PND Aufbereitung Truck POI TMC eKarten Aufbereitung Global Tabellen PremiumMap TINFO Diverse Scripte Premium Netz MapInfo Mautdaten AND Meerland bearbeiten ATF AGF2MapInfo Premium Orte ProDBIntegration Map2ATF MapBase Premium ATFAGF-ID Zuordnung Hamlet Filter LSN-Korrektur TauschNamen/RoutNr ON Priorisierung Map2MIF MapInfo Kontrollsch. Map2AGF MapBase GDF2Map StatistiX Statistik Files AGF Check Log-Files Meerland Truck2agf Projekt Importer MapInfo Kontrollsch. TA Logistic Projektdaten bearbeiten Projekt-/ Fremddaten AND MapInfo Personenfähren löschen AND2AGF ATFID2AGFID PremiumOrte bearbeiten GDF Projekt Mid MapAGF2AGF Projektdaten bearbeiten Ableitungs vorschriften POI-Icon Ableitungen. Maut Projektdaten bearbeiten Steuerungs Globaldateien Daten 2 DEU LKW Sperren Projektdaten bearbeiten MapAGF Tiler AGF 1 Groß/KleinWandlung Löschen env. Zonen Typ konvertierung Regionennachberarbeitung Meeressaum bearbeiten Teleporter integration AGF2GeoDB finale Teleporter Meeressaum GeoDB TBL neu Creator Höhenschicht Australien Vorverarbeitung Statistik Files 2 TBL neu Map2MIF Mautdaten Manueller Bearbeitungsschritt Mit GIS-Systemen (MapInfo) Manueller Bearbeitungsschritt mit UltraEdit, Scripten, SQL,... Automatischer Bearbeitungsschritt Teleporter nachbearbeitung Greenzones MapInfo Kontrollsch. Teleporter abgleich Maut bearbeiten Relevante Zwischenformate Overviews Teleporter Analysesch. Greenzones bearbeiten Fährenlayer ASCII-Daten (intern) MIF/MID-Daten (intern) tw. mit zusätzlicher *.ini) *.xls-Daten (intern) „Externe“ Daten TBL alt Datenbank Fähr bearbeitung Wurmloch xml-Daten (intern)) I Page 6
  • 6. MAIN GOALS OF THE NEW SOLUTION Business perspective  Reduce overall data processing times by increased productivity  Enable world wide coverage incl. new feature layers with the same data processing team  But do not invent everything again / Limit implementation costs I Page 7
  • 7. MAIN REQUIREMENTS How to achieve the goals?  Easy to use GUI (Graphical User Interface) with transparent business workflows/processes  Homogeneous behavior of services / processes (e.g. Logging, Error handling)  Robust processing (high availability, monitoring, fault tolerance)  Increase automation  Parallel processing (support scaling)  Re-use of existing concepts and software (preferred Open Source), already existing business logic I Page 8
  • 9. SYSTEM ARCHITECTURE Web GUI  Loose coupling of process control and workers (JavaServer Pages/jQuery/Spring MVC) Process Control (Tomcat container with BPM engine)  Load distribution by using multiple workers and multiple data DBs BPM engine DB (PostgreSQL) Messaging System (Apache Active MQ JMS Message Broker) Worker 1 Worker 2 Worker 2 (Apache Karaf OSGi container with Apache Camel based Services) (Apache Karaf OSGi container with Apache Camel based Services) (Apache Karaf OSGi container with Apache Camel based Services) Service A Service B Service B Service A Service B Service C  Separating data DB from process control DB  No map data in payload of BPM engine CDM (PostGIS database) CDM (PostGIS database) CDM (PostGIS database)  Main spatial data processing in DB I Page 10
  • 11. PROCESS/SERVICE CONCEPTS BPM – Workflow engine  We use an existing workflow engine which supports to • • • • • Persist process state Store durations of process executions for later analysis and optimization Profit of existing and new features (tasks, transitions, forks, joins) Profit of tooling for process modeling, process monitoring, persistence  thus allowing an efficient and transparent data processing  2 kinds of BPM processes for spatial data processing at PTV • Data Production Processes • Technical Processes I Page 12
  • 12. PROCESS/SERVICE CONCEPTS BPM – Kinds of processes  Data Production Processes (DPPs) • • • • • Combining the Human Workflows with automatic data preparation The only processes, directly started by users Configured for the data preparation of multiple countries Can trigger several technical processes as data preparation steps Can be recovered in case of errors I Page 13
  • 13. PROCESS/SERVICE CONCEPTS BPM – Kinds of processes  Technical Processes (TPs) • • • • • • Contain only service tasks for complete automatic data preparation Not directly started by users Started by DPPs Cover a single step of the automatic data preparation Configured to for data preparation of a single country Can not be recovered in case of error I Page 14
  • 14. PROCESS/SERVICE CONCEPTS BPM – Extensions of Service Tasks  MultiProcessExecutionTask in Data Production Processes I Page 15
  • 15. PROCESS/SERVICE CONCEPTS BPM – Why extending the ServiceTask for this?  ServiceTask in Technical Processes I Page 16
  • 16. EXPERIENCES BPM Engine  Re-Use: BPM engine for process management is a powerful tool, has many features which must not be implemented by us  Lightweight: Using the BPM Engine does not have too much impact on architecture or application development  Transparency: Easy API for implementing visualization and monitoring of process information, allowing a central overview of all processes (historic, current, planned) I Page 17
  • 17. EXPERIENCES Pitfalls for developers using the BPM Engine  OptimisticLockingException • Querying variables with RuntimeService increments revision rev_ • Exception occurs when quering the same variables in another thread • E.g. for updating the process monitoring page in GUI • Solution: • Usage of HistoryService where ever possible • Isolating concurrent variable updates by semaphor  HistoricVariableUpdateQuery • Results of type HistoricVariableUpdate can contain the exact same Date returning by getTime() • Problem to determine the last variable update • Solution: • Workaround: Using the id or revision, which are strongly incremented I Page 18
  • 18. EXPERIENCES Co-Work with business users        If BPM is new for the business users, give introduction / benefits First examples with small models (created by developers) Hide generic / recurring workflows (especially technical ones) Create an easy to use and attractive GUI Train users and let them work with it Ask for feedback / improvements In common sessions with users, add more and more steps to workflows where users see an added value (do not oversize things)  If possible: Integrate business users with dev skills into dev team  Do not give up, it takes time to experience the benefits I Page 19
  • 19. FUTURE PLANS Modeling  Add higher level processes representing the overall workflow (all phases of map production beginning from provider data checks and ending with data delivery) Transparency  Visualize process statistics to be able to find improvements/speed up data production  Statistics depend on configuration Technical  Supporting different process versions  „Real“ parallel processing I Page 20
  • 21.