Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Activity Recognition using RGBD


Published on

Current state of art contains several methods to achieve intelligent tracking. Some methods are machine learning oriented. In these methods, activities are learnt from the context in an unsupervised or semi supervised manner. One other method is description based event recognition. In the heart of the method , describing scenarios wrt activities employed. For the description, a language is necessarily needed. There are mathematical languages in which logic is used to represent activities and their relations.Also some graphical languages such as hidden markov models, state machines, state charts are being used. Some textual languages proposed as well.

Published in: Engineering
  • Be the first to comment

Activity Recognition using RGBD

  1. 1. VIDEO ACTIVITY RECOGNITION VIA RGB-D Francois BREMOND Carlos fernando Crispim junior Nazli TEMUR 1
  2. 2. 1) General Project Description 2) Contribution 3) Conclusion 4) Proposal 5) Bibliography CONTENT & Project & Motivations & Challenges & Goals 2
  3. 3. PROJECT General Project Description 3
  4. 4. PROJECT ➤ Domain of the project is real-time, automatic video sequences understanding. ➤ In this work, RGBD sensor is used to acquire 3D images, detect people and recognize interesting activities. ➤ The SUP library, which is developed by STARS Team is used for detection, tracking and recognition of the people. ➤ For privacy purposes we have focused ONLY on the depth information of RGBD cameras 4
  5. 5. INPUT / DATASET ➤ Dataset : Home Care [Nursing Home] recording on depth data, videoclips. ➤ 1 day[24 hours] of video is chosen out of months of recording of unconstraint recording. ➤ Already annotated video is used after verification. ➤ Total Annotated Events 197 with the numbers of Enter_Restroom: 18 Exit_Restroom : 19 Leave_Area_In_Bedroom : 75 Enter_Area_In_Bedroom : 76 Sitting : 9 5
  6. 6. MOTIVATION General Project Description 6
  7. 7. MOTIVATION ➤ Population aging is a motivation to develop intelligent technologies to support the life of older people , especially via monitoring systems. ➤ Activity recognition is a key way for better assessment of monitoring. ➤ So that my motivation is to improve activity recognition for providing comfort to the elderly people’s daily life. 7
  8. 8. CHALLENGES General Project Description 8
  9. 9. CHALLENGES ➤ People Detection • Occlusions[Perspective challenge] 12 3 9
  10. 10. CHALLENGES ➤ People Detection • Noises 1 2 10
  11. 11. CHALLENGES ➤ People Tracking • Re-identification • Overcome Occlusions • Overcome Noise 1 2 3 11
  12. 12. CHALLENGES ➤ Event Recognition • How to model activities • How to recognition human activities in real-time settings, past- present approach for video analysis 12 3 12
  13. 13. CHALLENGES ➤ Event Recognition • How to model activities • How to recognition human activities in real-time settings, past-present approach for video analysis 34 5 13
  14. 14. CHALLENGES ➤ Event Recognition • How to estimate target destination • How to maintain different actors’ speed to not be in trouble with duration • How to infer actors interaction with ambiguous, non-observed parts of the scene A B C 14
  15. 15. GOAL Evaluate Current Event Recognition Pipeline for Activity Recognition in Unconstrained Environment Study the event modeling language to implement new models. Study the limitations of existing solutions. Propose more suitable event models. TO DO 15
  16. 16. CONTRIBUTIONS 16
  17. 17. CONTRIBUTIONS ➤ Proposition of a new approach to reason about actors that leave/enter the scene in the midst of noisy people observations. ➤ - Previously: Empty scene (hard constraint) ➤ - Proposition: Increase/Decrease in the number of actors Note: Empty Scene restricts the events to a single actor. Thanks to newly proposed scenarios the event recognition can be done when multiple actor/noise present on the scene. 17
  18. 18. CONTRIBUTIONS ➤ Proposition of a new layout for the spatial zones. 18
  19. 19. EXAMPLE SCENARIOS OF THINTH LANGUAGE CompositeEvent(EXIT_RESTROOM, PhysicalObjects((p1 : Person), (z1 : Zone), (sc : Scene)) Components((c1: CompositeEvent Disappear(p1,sc)) (c2: PrimitiveEvent IN_AREA_IN_RESTROOM(p1,z1))) Constraints((c1 justBefore c2) (duration(c2) > 0.5)) Alarm ((Level : URGENT)) ) CompositeEvent(Disappear, PhysicalObjects((p1 : Person), (sc : Scene)) Components((c1: PrimitiveState Person_exists(p1)) (c2: PrimitiveState Negative_Change_Actor_Number(sc))) Constraints( (c1 justBefore c2) (!(Exist(p1)))) Alarm ((Level : URGENT)) ) PrimitiveState(Negative_Change_Actor_Number, PhysicalObjects((sc : Scene)) Constraints ((AttributeChange(sc->NumberOfActors) = 2)) Alarm ((Level : URGENT)) ) ATTRIBUTES ➤ Composite Event ➤ Primitive Event ➤ Primitive State 19
  20. 20. CONCLUSION 20
  21. 21. CONCLUSION In this project, description based video activity recognition and evaluation is studied. Improvement proposals are suggested on Event Modeling and Trajectory Analysis Topics. Thanks to the newly proposed event models, it is possible to recognize events in a scene where multiple actors and noises are present. For the evaluation process, we benefited from several tool such as SUP Event Recognition Platform, Thinth Event Modeling Language ,Viseval ,Viper, KreateTool. While evaluating , we have used True Positive,False Positive and False Negative measurements; F1 score, precision, recall metrics.[TP,FP,FN] Evaluation on previously defined event models are completed. Currently, latest assessment on newly defined zones and event models is ongoing because of Viseval tool related error. 21
  22. 22. PROPOSALS 22
  23. 23. PROPOSALS ➤ Use of Trajectory is investigated. Trajectory usage is promising, but to be able to implement a precise solution, state of art for the trajectory need to be reviewed wrt occlusion and noise reduction. 23
  24. 24. BIBLIOGRAPHY 24
  25. 25. [1] C. Crispim-Junior, K. Avgerinakis, V. Buso, G. Meditskos, A. Briassouli, J. Benois-Pineau, Y. Kompatsiaris and F. Bremond. Semantic Event Fusion of Different Visual Modality Concepts for Activity Recognition, Transactions on Pattern Analysis and Machine Intelligence - PAMI to appear, 2016. [2] C. Crispim-Junior, V. Bathrinarayanan, B. Fosty, A. Konig, R. Romdhane, M. Thonnat and F. Bremond. Evaluation of a Monitoring System for Event Recognition of Older People. In the 10th IEEE International Conference on Advanced Video and Signal-Based Surveillance 2013, AVSS 2013, Krakow, Poland on August 27-30, 2013. [3] C. Crispim-Junior, B. Fosty, R. Romdhane, F. Bremond and M. Thonnat. Combining Multiple Sensors for Event Recognition of Older People. In the 1st ACM International Workshop on Multimedia Indexing and Information Retrieval for Healthcare, MIIRH 2013, Copyright 2013 ACM 978-1-4503-2398-7/13/10,, Barcelona, October 22, 2013. [4] Alberto Avanzi, Francois Bremond, Christophe Tornieri and Monique Thonnat, Design and Assessment of an Intelligent Activity Monitoring Platform, in EURASIP Journal on Applied Signal Processing, special issue in "Advances in Intelligent Vision Systems: Methods and Applications", 2005. [5] E. Corvee and F. Bremond. Haar like and LBP based features for face, head and people detection in video sequences. In the International Workshop on Behaviour Analysis, Behave 2011, Sophia Antipolis, France on the 23rd of September 2011. [6] C. Crispim-Junior and F. Bremond. Uncertainty Modeling Framework for Constraint-based Elementary Scenario Detection in Vision System. In the First International Workshop on Computer vision + ONTology Applied Cross-disciplinary Technologies in conjunction with ECCV 2014, CONTACT-2014, Zurich, Switzerland, September 7th, 2014. [7] A. König, C. Crispim, A. Covella, F. Bremond, A. Derreumaux, G. Bensadoum, R. David, F. Verhey, P. Aalten and P.H. Robert. Ecological Assessment of Autonomy in Instrumental Activities of Daily Living in Dementia Patients by the means of an Automatic Video Monitoring System, Frontiers in Aging Neuroscience - open access publication - 2015.00098, 02 June 2015 25
  26. 26. THANKS! 26
  27. 27. SHOWCASE 27