Your SlideShare is downloading. ×
0
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Ruleml2012  - Rule-based high-level situation recognition from incomplete tracking data
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Ruleml2012 - Rule-based high-level situation recognition from incomplete tracking data

261

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
261
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Rule-Based High-Level Situation Recognition from Incomplete Tracking Data David Münch1, Joris IJsselmuiden1, Ann-Kristin Grosselfinger1, Michael Arens1, and Rainer Stiefelhagen1,2 1 Fraunhofer IOSB, Germany, david.muench@iosb.fraunhofer.de 2 Karlsruhe Institute of Technology, Germany. The 6th International Symposium on Rules - Rule ML2012, Montpellier, France, August 27-29, 2012 1© Fraunhofer IOSB
  • 2. MotivationMotivation• Security systems: Persons and their behavior are the focus of attention: Person centric analysis• Threat detection• Visual surveillance• Activity logging• Video search• Driver Assistance Systems Input data is incomplete and noisy. 2© Fraunhofer IOSB
  • 3. Overview• Cognitive Vision System as a whole.• High-level knowledge and situation recognition.• Handling Incomplete Data.• Experiments. 3© Fraunhofer IOSB
  • 4. Cognitive Vision System 4© Fraunhofer IOSB
  • 5. Cognitive Vision System 5© Fraunhofer IOSB
  • 6. Cognitive Vision System 6© Fraunhofer IOSB
  • 7. Cognitive Vision System 7© Fraunhofer IOSB
  • 8. Cognitive Vision System 8© Fraunhofer IOSB
  • 9. Cognitive Vision System 9© Fraunhofer IOSB
  • 10. Conceptual Layer – Conceptual Primitives Level• Quantitative information (from Quantitative Layer) is transformed into primitive conceptual knowledge (Logic predicates).• Dictionary of basic rules.• Mainly domain independent.• Support of uncertainty and vagueness.• The rules in the CPL are mostly concerned with spatial relations and temporal relations on short time intervals. 10© Fraunhofer IOSB
  • 11. Dictionary of basic rules Dictionary of basic rules for every domain. Fuzzy Metric Temporal Logic (FMTL): Extension of first order logic by notions of fuzziness, time, and metrics on time. Inference engine: F-LIMETTE 11© Fraunhofer IOSB
  • 12. “Numbers” mapped to Concepts Fuzzy membership functions 𝜇 𝑠𝑝𝑒𝑒𝑑 𝑣𝑎𝑙𝑢𝑒 for the subset {zero, small, normal, high, very_high} of discrete conceptual speed values. Figure from: H.-H. Nagel, Steps toward a Cognitive Vision System, 2004, AI Mag. 12© Fraunhofer IOSB
  • 13. Conceptual Layer – Behavior Representation LevelHow to represent the expected Situations?• Knowledge represented in Situation Graph Trees. • Graphically editable • Easy to extend and edit • Interpretable (vs. black box)• Exhaustive situation recognition. 13© Fraunhofer IOSB
  • 14. Behavior Representation LevelBasic logic predicates from Conceptual Primitives Level are aggregated andstructured in Situation Graph Trees (SGT)  high-level conceptual situations.An SGT consists of situation schemes:• Can be start and/or end nodes.• Unique name.• State scheme (Precondition).• Action scheme (Postcondition).Specialize a situation scheme:• Prediction edges link to a possible subsequent situation scheme.• Specialization edges link to more specific situation graphs in a hierarchical structure. 14© Fraunhofer IOSB
  • 15. Behavior Representation Level 15© Fraunhofer IOSB
  • 16. Handling Incomplete Data – Low Level in Scene Domain Level Perfect input data. Truth value Time 16© Fraunhofer IOSB
  • 17. Handling Incomplete Data – Low Level in SDL Incomplete input data. Truth value Time 17© Fraunhofer IOSB
  • 18. Handling Incomplete Data – Low Level in SDLInterpolation of input data: Truth value Time 18© Fraunhofer IOSB
  • 19. Handling Incomplete Data – Low Level in SDLInterpolation of input data: Time 19© Fraunhofer IOSB
  • 20. Handling Incomplete Data – High Level in BRL 20© Fraunhofer IOSB
  • 21. Handling Incomplete Data – High Level in BRL What if is_open(trunk, Car) fails? 21© Fraunhofer IOSB
  • 22. Handling Incomplete Data – High Level in BRLHallucination (abduction) of missing evidence. What if is_open(trunk, Car) fails?  Hallucinate is_open(trunk, Car) and continue! 22© Fraunhofer IOSB
  • 23. VIRAT Video Dataset VIRAT Video Dataset Release 1.0 Input data: annotated ground truth: persons and their situations. Place: 0000: Place: 0002: Videos: 02 03 04 06 Videos: 00 06 23© Fraunhofer IOSB
  • 24. Situations1. Person loads object into car.2. Person unloads object from car.3. Person gets into car.4. Person gets out of car. VIRAT Video Dataset Release 1.0 24© Fraunhofer IOSB
  • 25. Evaluation• The annotated ground truth is regarded as complete information.• Randomly make gaps of a distinct length into the data.• Increase the amount of missing data in steps of 5%.• Repeat each experiment several times. 25© Fraunhofer IOSB
  • 26. Evaluation Original, F-Score unmodified Gap size 5 seconds. Precision RecallGap size Gap size5 seconds. 5 seconds. 26© Fraunhofer IOSB
  • 27. Evaluation Original, F-Score unmodified Gap size 5 seconds. Precision RecallGap size Gap size5 seconds. 5 seconds. 27© Fraunhofer IOSB
  • 28. Evaluation ROC-curves for video (d) with gap size 5 seconds. TPR TPR FPR FPR With interpolation and hallucination. Without interpolation and hallucination. 28© Fraunhofer IOSB
  • 29. Evaluation Video (d), F-Score Gap size 5 seconds.Gap size Gap size5 seconds. 5 seconds. 29© Fraunhofer IOSB
  • 30. Conclusion• low level: interpolation of data and its uncertainty.  ordinary incomplete data.• high level: extension of the situation recognition inference algorithm (hallucination, abduction).  high-level incomplete data such as occlusions.• Knowledge base for vehicle-centered situations.• Runs in real-time on off-the-shelf hardware. 30© Fraunhofer IOSB
  • 31. Finis. 31© Fraunhofer IOSB

×