Successfully reported this slideshow.

A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

0

Share

Loading in …3
×
1 of 20
1 of 20

A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

0

Share

Download to read offline

Despite the increasing automation levels in an Industry 4.0 scenario, the tacit knowledge of highly skilled manufacturing workers remains of strategic importance. Retaining this knowledge by formally capturing it is a challenge for industrial organisations. This paper explores research on automatically capturing this knowledge by using methods from activity recognition and process mining on data obtained from sensorised workers and environments. Activity recognition lifts the abstraction level of sensor data to recognizable activities and process mining methods discover models of process executions.
We classify the existing work, which largely neglects the possibility of applying process mining, and derive a taxonomy that identifies challenges and research gaps.

Despite the increasing automation levels in an Industry 4.0 scenario, the tacit knowledge of highly skilled manufacturing workers remains of strategic importance. Retaining this knowledge by formally capturing it is a challenge for industrial organisations. This paper explores research on automatically capturing this knowledge by using methods from activity recognition and process mining on data obtained from sensorised workers and environments. Activity recognition lifts the abstraction level of sensor data to recognizable activities and process mining methods discover models of process executions.
We classify the existing work, which largely neglects the possibility of applying process mining, and derive a taxonomy that identifies challenges and research gaps.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments

  1. 1. A TAXONOMY FOR COMBINING ACTIVITY RECOGNITION AND PROCESS DISCOVERY IN INDUSTRIAL ENVIRONMENTS Felix Mannhardt, Riccardo Bovo, Manuel Fradinho Oliveira, Simon Julier 1SINTEF, Trondheim, Norway , 2University College London, UK This research has received funding from the European Unions H2020 research and innovation programme under grant agreement no. 723737 (HUMAN).
  2. 2. What is Process Mining? 2 • Activity (What?) • Ordering (When?) • Case / Context • SAP, MES, … • Sensors (IoT) • … • Assembly • Logistics • … • Work instructions • Assembly sequence • …
  3. 3. 3 Operators Sensor data discover Process Models capture Activities recognise analyse tacit knowledge deliver explicit knowledge Process Engineer Process Discovery in Industrial Environments
  4. 4. What is Activity Recognition (ARC)? 4 Some input sensor signal segmented according to the ground truth label
  5. 5. 5 Example: Supervised detection using a LSTM
  6. 6. 6 Some further typical examples Ekaterina H. Spriggs*, Fernando De La Torre, and Martial Hebert, "Temporal segmentation and activity classification from first-person sensing," IEEE Workshop on Egocentric Vision, CVPR 2009, June, 2009. Michael S. Ryoo Greg Mori Kris Kitani. CVPR1 Tutorial: Paper 18: Human Activity Recognition. 2018
  7. 7. Project Context – HUMAN • Human MANufacturing Project (H2020) • Sensorised manufacturing operator (Wearables etc.) • Stress and fatigue detection through sensor data • Support operators in their workflow through technology • Augmented reality (AR) • Exoskeletons • Use of Process Mining on collected data for long-term diagnostics and optimisation of work 7 Challenge: Automatically detect manual shop-floor activities to re-discover the actual work process
  8. 8. Exploratory search for existing work 8 Google Scholar Scopus Search terms • Manufacturing • Industrial • Activity recognition • Sensors • Process discovery • Process elicitation • Process analysis • Events • Tacit knowledge
  9. 9. Categorized 26 relevant papers 9
  10. 10. Preliminary taxonomy for ARC & Process discovery 10 Time Predictive Online Post-mortem
  11. 11. 11 Knoch, S., Ponpathirkoottam, S., Fettke, P., Loos, P.: Technology-enhanced process elicitation of sworker activities in manufacturing. In: Business Process Management Workshops, pp. 273–284. Springer International Publishing (2018) Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tr, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7(2), 42–50 (2008) Ambient vs. Wearable | Video vs. Motion
  12. 12. Preliminary taxonomy for ARC & Process discovery 12 Data Capture Sensor type Vision-based Motion-based Sound-based Radiowave-based Sensor location Wearable Objects Ambient Storage Batch Streaming Processing Supervised Unsupervised
  13. 13. Very little unsupervised learning 13 Boettcher, S., Scholl, P.M., Laerhoven, K.V.: Detecting process transitions from wearable sensors. In: iWOAR 2017. ACM Press (2017)
  14. 14. Some semi-supervised works 14 Blanke, U., Schiele, B.: Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In: ISWC 2010. IEEE (2010)
  15. 15. Preliminary taxonomy for ARC & Process discovery 15 Processcontext Activity type Granularity Coarse Fine Complexity Simple Complex Control-flow type Sequence of steps Sequential workflow Collaborative workflow Process size Small Large
  16. 16. 16 Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tr, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Computing 7(2), 42–50 (2008) Complexity & Control-Flow Blanke, U., Schiele, B.: Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In: ISWC 2010. IEEE (2010)
  17. 17. Preliminary taxonomy for ARC & Process discovery 17 Environment Setting Laboratory Factory Layout Flexible Structured RigidInterference Privacy
  18. 18. 18 Voulodimos, A.S., Kosmopoulos, D.I., Doulamis, N.D., Varvarigou, T.A.: A top-down event driven approach for concurrent activity recognition. Multimedia Tools and Applications 69(2), 293–311 (2012) Environment Knoch, S., Ponpathirkoottam, S., Fettke, P., Loos, P.: Technology-enhanced process elicitation of sworker activities in manufacturing. In: Business Process Management Workshops, pp. 273–284. Springer International Publishing (2018)
  19. 19. Conclusion • Contribution • A structured overview of the work on activity recognition relevant to process discovery in industrial environments • Preliminary taxonomy derived from our literature search • Challenges identified for process discovery in industrial environments • Future work • Rigerous description of the finalised taxonomy • Comprehensive literature review based on the completed taxonomy • Validation of the taxonomy? • Apply process discovery paired with activity recognition in real industrial environments 19
  20. 20. Mannhardt F., Bovo R., Oliveira M.F., Julier S. (2018) A Taxonomy for Combining Activity Recognition and Process Discovery in Industrial Environments. In: Yin H., Camacho D., Novais P., Tallón-Ballesteros A. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2018. IDEAL 2018. Lecture Notes in Computer Science, vol 11315. Springer, Cham

Editor's Notes

  • Bildet illustrerer: 
    ·        bredden  i SINTEFs ekspertise, fra havrom til verdensrom.
    ·        hvilke områder og bransjer vi jobber innen for å realisere visjonen Teknologi for et bedre samfunn.
     
    Bildestilen er basert på stikkordene fremtidsrettet, teknologi og norsk natur (naturressurser). SINTEFs visuelle univers er utviklet for SINTEF av Headspin Productions AS.
  • ×