Advertisement

A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

Research Scientist at SINTEF
Jul. 10, 2019
Advertisement

More Related Content

Similar to A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments(20)

Advertisement

A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

  1. A TAXONOMY FOR COMBINING ACTIVITY RECOGNITION AND PROCESS DISCOVERY IN INDUSTRIAL ENVIRONMENTS Felix Mannhardt, Riccardo Bovo, Manuel Fradinho Oliveira, Simon Julier 1SINTEF, Trondheim, Norway , 2University College London, UK This research has received funding from the European Unions H2020 research and innovation programme under grant agreement no. 723737 (HUMAN).
  2. What is Process Mining? 2 • Activity (What?) • Ordering (When?) • Case / Context • SAP, MES, … • Sensors (IoT) • … • Assembly • Logistics • … • Work instructions • Assembly sequence • …
  3. 3 Operators Sensor data discover Process Models capture Activities recognise analyse tacit knowledge deliver explicit knowledge Process Engineer Process Discovery in Industrial Environments
  4. What is Activity Recognition (ARC)? 4 Some input sensor signal segmented according to the ground truth label
  5. 5 Example: Supervised detection using a LSTM
  6. 6 Some further typical examples Ekaterina H. Spriggs*, Fernando De La Torre, and Martial Hebert, "Temporal segmentation and activity classification from first-person sensing," IEEE Workshop on Egocentric Vision, CVPR 2009, June, 2009. Michael S. Ryoo Greg Mori Kris Kitani. CVPR1 Tutorial: Paper 18: Human Activity Recognition. 2018
  7. Project Context – HUMAN • Human MANufacturing Project (H2020) • Sensorised manufacturing operator (Wearables etc.) • Stress and fatigue detection through sensor data • Support operators in their workflow through technology • Augmented reality (AR) • Exoskeletons • Use of Process Mining on collected data for long-term diagnostics and optimisation of work 7 Challenge: Automatically detect manual shop-floor activities to re-discover the actual work process
  8. Exploratory search for existing work 8 Google Scholar Scopus Search terms • Manufacturing • Industrial • Activity recognition • Sensors • Process discovery • Process elicitation • Process analysis • Events • Tacit knowledge
  9. Categorized 26 relevant papers 9
  10. Preliminary taxonomy for ARC & Process discovery 10 Time Predictive Online Post-mortem
  11. 11 Knoch, S., Ponpathirkoottam, S., Fettke, P., Loos, P.: Technology-enhanced process elicitation of sworker activities in manufacturing. In: Business Process Management Workshops, pp. 273–284. Springer International Publishing (2018) Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tr, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7(2), 42–50 (2008) Ambient vs. Wearable | Video vs. Motion
  12. Preliminary taxonomy for ARC & Process discovery 12 Data Capture Sensor type Vision-based Motion-based Sound-based Radiowave-based Sensor location Wearable Objects Ambient Storage Batch Streaming Processing Supervised Unsupervised
  13. Very little unsupervised learning 13 Boettcher, S., Scholl, P.M., Laerhoven, K.V.: Detecting process transitions from wearable sensors. In: iWOAR 2017. ACM Press (2017)
  14. Some semi-supervised works 14 Blanke, U., Schiele, B.: Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In: ISWC 2010. IEEE (2010)
  15. Preliminary taxonomy for ARC & Process discovery 15 Processcontext Activity type Granularity Coarse Fine Complexity Simple Complex Control-flow type Sequence of steps Sequential workflow Collaborative workflow Process size Small Large
  16. 16 Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tr, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7(2), 42–50 (2008) Complexity & Control-Flow Blanke, U., Schiele, B.: Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In: ISWC 2010. IEEE (2010)
  17. Preliminary taxonomy for ARC & Process discovery 17 Environment Setting Laboratory Factory Layout Flexible Structured RigidInterference Privacy
  18. 18 Voulodimos, A.S., Kosmopoulos, D.I., Doulamis, N.D., Varvarigou, T.A.: A top-down event driven approach for concurrent activity recognition. Multimedia Tools and Applications 69(2), 293–311 (2012) Environment Knoch, S., Ponpathirkoottam, S., Fettke, P., Loos, P.: Technology-enhanced process elicitation of sworker activities in manufacturing. In: Business Process Management Workshops, pp. 273–284. Springer International Publishing (2018)
  19. Conclusion • Contribution • A structured overview of the work on activity recognition relevant to process discovery in industrial environments • Preliminary taxonomy derived from our literature search • Challenges identified for process discovery in industrial environments • Future work • Rigerous description of the finalised taxonomy • Comprehensive literature review based on the completed taxonomy • Validation of the taxonomy? • Apply process discovery paired with activity recognition in real industrial environments 19
  20. Mannhardt F., Bovo R., Oliveira M.F., Julier S. (2018) A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments. In: Yin H., Camacho D., Novais P., Tallón-Ballesteros A. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2018. IDEAL 2018. Lecture Notes in Computer Science, vol 11315. Springer, Cham

Editor's Notes

  1. Bildet illustrerer:  ·        bredden  i SINTEFs ekspertise, fra havrom til verdensrom. ·        hvilke områder og bransjer vi jobber innen for å realisere visjonen Teknologi for et bedre samfunn.   Bildestilen er basert på stikkordene fremtidsrettet, teknologi og norsk natur (naturressurser). SINTEFs visuelle univers er utviklet for SINTEF av Headspin Productions AS.
Advertisement