This document provides an overview of visual sensor networks and smart cameras. It discusses sensor networks and how they use wireless communication between small battery-powered devices to collect and transmit sensor data over multiple hops. It then describes smart cameras, which combine sensing, processing, and communication to perform image analysis and collaboration. Key characteristics of visual sensor networks are discussed, including resource limitations, on-board processing, real-time operation, and autonomous camera collaboration. Several problems in visual sensor networks are also outlined, such as sensor placement, synchronization, and data distribution. Finally, examples of centralized and distributed configuration methods and a market-based approach to tracking handovers between cameras are presented.
Design of self powered embedded wireless smart camera using multimodal video ...eSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Modern Street Lightening System with Intensity Control using GSMpaperpublications3
Abstract: As the LED's lumen efficiency increases rapidly in recent years, many new LED illumination applications are emerging. LEDs have features such as long-life, small and low power consumption. Therefore, they are used in various occasion such as full color large sized LED displays, traffic lights, and etc. In this paper, an energy efficient street lighting system is proposed. The presented system consists of a LED lamp module, which can be controlled from remote location. The proposed remote-control system can optimize intensity and efficiency of street lighting systems. It uses GSM based wireless devices which enable more efficient street lamp-system management, thanks to an advanced interface and control architecture. It uses a sensor combination to control and guarantee the desired system parameters; the information is transferred point by point using GSM Module and is sent to a control terminal used to check the status of the street lamps and to take appropriate measures in case of failure.A developed prototype system will be presented in this paper and experiments will be performed to verify the correctness of the proposed system. According to the experimental results, the lighting efficiency is 85 % and the conversion efficiency is 90 %.
Keywords: WSN (Wireless Sensor Network), GSM (Global System for Mobile Communication), IR (Infrared) CEPT (Conference of European Posts and Telegraphs), IDEN (Integrated Digital Enhanced Network).
Title: Modern Street Lightening System with Intensity Control using GSM
Author: Kapil Aherkar, Pratik Dongrikar, Nikul Dengda, Sukrit Bhattacharya
ISSN 2349-7815
International Journal of Recent Research in Electrical and Electronics Engineering (IJRREEE)
Paper Publications
Mobile Radiation Measuring System using Small Linux box and GPS sensorIJMER
The effect of radiation and the measuring the radiation dose rate amount has become as
important research topics among the researcher, after the nuclear accident happened in Japan. Several
commercial equipments are available that can measure the dose amount but most of them have some
limitations. However designing mobile radiation measuring system using PC is generally expensive, and
complicated. It also requires electrical power. In this paper, we propose a new technique to combine this
type of equipments and GPS sensor using small Linux box. The power consumption and data access are also
discussed
Wireless Sensor networks are dense networks, which consist of small low cost
sensors having severely constrained computational and energy resources, which operate in
an adhoc environment. Sensor network combines the aspects of distributed sensing,
computing and communication. Despite the numerous applications of sensor networks in
various fields there are various issues which need to be explored and resolved such as
resource constraints, routing, coverage, security, information collection and gathering etc.
In this paper we aim to provide the detailed overview of the wireless sensor technologies and
issues related to them, such as advancement of sensor technology, architecture, applications,
issues and the work done in the field of routing, coverage and security.
01 elements of modern networking by nader elmansiNader Elmansi
Foundations of Modern Networking SDN, NFV, QoE, IoT, and Cloud
PART I MODERN NETWORKING
CHAPTER 1 Elements of Modern Networking
regenerated by Nader Elmansi
Data acquisition and storage in Wireless Sensor NetworkRutvik Pensionwar
1. Introduce to Wireless Sensor Network and various data retrieval techniques.
2. Present different algorithms used in Wireless Sensor Network to achieve efficiency and manage power effectively.
Design of self powered embedded wireless smart camera using multimodal video ...eSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Modern Street Lightening System with Intensity Control using GSMpaperpublications3
Abstract: As the LED's lumen efficiency increases rapidly in recent years, many new LED illumination applications are emerging. LEDs have features such as long-life, small and low power consumption. Therefore, they are used in various occasion such as full color large sized LED displays, traffic lights, and etc. In this paper, an energy efficient street lighting system is proposed. The presented system consists of a LED lamp module, which can be controlled from remote location. The proposed remote-control system can optimize intensity and efficiency of street lighting systems. It uses GSM based wireless devices which enable more efficient street lamp-system management, thanks to an advanced interface and control architecture. It uses a sensor combination to control and guarantee the desired system parameters; the information is transferred point by point using GSM Module and is sent to a control terminal used to check the status of the street lamps and to take appropriate measures in case of failure.A developed prototype system will be presented in this paper and experiments will be performed to verify the correctness of the proposed system. According to the experimental results, the lighting efficiency is 85 % and the conversion efficiency is 90 %.
Keywords: WSN (Wireless Sensor Network), GSM (Global System for Mobile Communication), IR (Infrared) CEPT (Conference of European Posts and Telegraphs), IDEN (Integrated Digital Enhanced Network).
Title: Modern Street Lightening System with Intensity Control using GSM
Author: Kapil Aherkar, Pratik Dongrikar, Nikul Dengda, Sukrit Bhattacharya
ISSN 2349-7815
International Journal of Recent Research in Electrical and Electronics Engineering (IJRREEE)
Paper Publications
Mobile Radiation Measuring System using Small Linux box and GPS sensorIJMER
The effect of radiation and the measuring the radiation dose rate amount has become as
important research topics among the researcher, after the nuclear accident happened in Japan. Several
commercial equipments are available that can measure the dose amount but most of them have some
limitations. However designing mobile radiation measuring system using PC is generally expensive, and
complicated. It also requires electrical power. In this paper, we propose a new technique to combine this
type of equipments and GPS sensor using small Linux box. The power consumption and data access are also
discussed
Wireless Sensor networks are dense networks, which consist of small low cost
sensors having severely constrained computational and energy resources, which operate in
an adhoc environment. Sensor network combines the aspects of distributed sensing,
computing and communication. Despite the numerous applications of sensor networks in
various fields there are various issues which need to be explored and resolved such as
resource constraints, routing, coverage, security, information collection and gathering etc.
In this paper we aim to provide the detailed overview of the wireless sensor technologies and
issues related to them, such as advancement of sensor technology, architecture, applications,
issues and the work done in the field of routing, coverage and security.
01 elements of modern networking by nader elmansiNader Elmansi
Foundations of Modern Networking SDN, NFV, QoE, IoT, and Cloud
PART I MODERN NETWORKING
CHAPTER 1 Elements of Modern Networking
regenerated by Nader Elmansi
Data acquisition and storage in Wireless Sensor NetworkRutvik Pensionwar
1. Introduce to Wireless Sensor Network and various data retrieval techniques.
2. Present different algorithms used in Wireless Sensor Network to achieve efficiency and manage power effectively.
International Journal of Engineering and Science Invention (IJESI)inventionjournals
International Journal of Engineering and Science Invention (IJESI) is an international journal intended for professionals and researchers in all fields of computer science and electronics. IJESI publishes research articles and reviews within the whole field Engineering Science and Technology, new teaching methods, assessment, validation and the impact of new technologies and it will continue to provide information on the latest trends and developments in this ever-expanding subject. The publications of papers are selected through double peer reviewed to ensure originality, relevance, and readability. The articles published in our journal can be accessed online.
TheThingsConference 2019 Slides of Alex RaimondiSchekeb Fateh
Next generation IoT sensors will perform high performance audio-visual processing supported by AI directly on the edge to enable the next generation of battery-powered sensing devices for use cases like predictive maintenance, people counting, feature recognition in images, etc.
EFFECT OF HASH FUNCTION ON PERFORMANCE OF LOW POWER WAKE UP RECEIVER FOR WIRE...IJNSA Journal
Next generation network will consist of different types of wireless networks like WSN, Wi-Fi, WiMAX, UMTS, LTE and etc. Wireless Sensor Network (WSN) finds unique and special application as compared to the said networks because sensors are deployed in a very secret, awkward and hostile environment like battle field etc. Various wireless sensor nodes are interconnected and form a Wireless Sensor Network. Sensor nodes once deployed in a region, can’t be repaired thus the power system deployed in the nodes becomes a major key issue i.e. how long its battery life can be utilised. Another major issue of WSN is to have a more secured network which is a function of hash keys. Increase usage of hash key means enhanced security but at the cost of power and area. Sensor systems must utilize the minimal possible energy while operating over secured and wide range of operating scenarios. In this paper, we have proposed a novel ID matching mechanism that uses a Bloom filter to realize wake-up wireless communication. Paper uses hash function for uniquely recognizing particular sensor- node- cluster among all clusters. Paper also shows the effect of number of hash functions on performance of wireless sensor node. The design and implementation of a wireless wake-up receiver module simulation reveals
that proposed model consume 724nW dynamic power and with bloom filter, the proposed model consumes dynamic power 85% less than the consumption cited in “Takiguchi” model[1]. Dynamic power is further reduced by 10% when parallel processing is implemented. Finally paper provides a novel approach to save the dynamic power and subsequently increases the battery life of wireless sensor node and network as a whole.
With the advancements in wireless technology and digital electronics, some tiny devices have started to be used in numerous areas in daily life. These devices are capable of sensing, computation and communicating. They are generally composed of low power radios, several smart sensors and embedded CPUs (Central Processing Units). These devices are used to form wireless sensor network (WSN) which is necessary to provide sensing services and to monitor environmental conditions. In parallel to WSNs, the idea of internet of things (IoT) is developed where IoT can be defined as an interconnection between identifiable devices within the internet connection in sensing and monitoring processes. This paper presents detailed overview of WSNs. It also assesses the technology and characteristics of WSNs. Moreover, it provides a review of WSN applications and IoT applications.
International Journal of Engineering and Science Invention (IJESI)inventionjournals
International Journal of Engineering and Science Invention (IJESI) is an international journal intended for professionals and researchers in all fields of computer science and electronics. IJESI publishes research articles and reviews within the whole field Engineering Science and Technology, new teaching methods, assessment, validation and the impact of new technologies and it will continue to provide information on the latest trends and developments in this ever-expanding subject. The publications of papers are selected through double peer reviewed to ensure originality, relevance, and readability. The articles published in our journal can be accessed online.
TheThingsConference 2019 Slides of Alex RaimondiSchekeb Fateh
Next generation IoT sensors will perform high performance audio-visual processing supported by AI directly on the edge to enable the next generation of battery-powered sensing devices for use cases like predictive maintenance, people counting, feature recognition in images, etc.
EFFECT OF HASH FUNCTION ON PERFORMANCE OF LOW POWER WAKE UP RECEIVER FOR WIRE...IJNSA Journal
Next generation network will consist of different types of wireless networks like WSN, Wi-Fi, WiMAX, UMTS, LTE and etc. Wireless Sensor Network (WSN) finds unique and special application as compared to the said networks because sensors are deployed in a very secret, awkward and hostile environment like battle field etc. Various wireless sensor nodes are interconnected and form a Wireless Sensor Network. Sensor nodes once deployed in a region, can’t be repaired thus the power system deployed in the nodes becomes a major key issue i.e. how long its battery life can be utilised. Another major issue of WSN is to have a more secured network which is a function of hash keys. Increase usage of hash key means enhanced security but at the cost of power and area. Sensor systems must utilize the minimal possible energy while operating over secured and wide range of operating scenarios. In this paper, we have proposed a novel ID matching mechanism that uses a Bloom filter to realize wake-up wireless communication. Paper uses hash function for uniquely recognizing particular sensor- node- cluster among all clusters. Paper also shows the effect of number of hash functions on performance of wireless sensor node. The design and implementation of a wireless wake-up receiver module simulation reveals
that proposed model consume 724nW dynamic power and with bloom filter, the proposed model consumes dynamic power 85% less than the consumption cited in “Takiguchi” model[1]. Dynamic power is further reduced by 10% when parallel processing is implemented. Finally paper provides a novel approach to save the dynamic power and subsequently increases the battery life of wireless sensor node and network as a whole.
With the advancements in wireless technology and digital electronics, some tiny devices have started to be used in numerous areas in daily life. These devices are capable of sensing, computation and communicating. They are generally composed of low power radios, several smart sensors and embedded CPUs (Central Processing Units). These devices are used to form wireless sensor network (WSN) which is necessary to provide sensing services and to monitor environmental conditions. In parallel to WSNs, the idea of internet of things (IoT) is developed where IoT can be defined as an interconnection between identifiable devices within the internet connection in sensing and monitoring processes. This paper presents detailed overview of WSNs. It also assesses the technology and characteristics of WSNs. Moreover, it provides a review of WSN applications and IoT applications.
it has a small description about how wireless sensor system network can be applied in various field. A application of leaksge detection is discussed in detail.
Many emerging applications require methods tailored towards high-speed data acquisition and filtering of streaming data followed by offline event reconstruction and analysis. In this case, the main objective is to relieve the immense pressure on the storage and communication resources within the experimental infrastructure. In other applications, ultra low latency real time analysis is required for autonomous experimental systems and anomaly detection in acquired scientific data in the absence of any prior data model for unknown events. At these data rates, traditional computing approaches cannot carry out even cursory analyses in a time frame necessary to guide experimentation. In this talk, Prof. Ogrenci will present some examples of AI hardware architectures. She will discuss the concept of co-design, which makes the unique needs of an application domain transparent to the hardware design process and present examples from three applications: (1) An in-pixel AI chip built using the HLS methodology; (2) A radiation hardened ASIC chip for quantum systems; (3) An FPGA-based edge computing controller for real-time control of a High Energy Physics experiment.
Lesson 00 slides for one day introductory course on wireless sensor networks and TinyOS, that took place at the University of Alcalá de Henares in Madrid Spain the 18th of September 2013. This course was jointly designed by the Electronics Department of the university and Advanticsys. Find source code for the lessons here: http://www.advanticsys.com/wiki/index.php?title=TinyOS%C2%AE_Course_at_UAH_18th_September_2013
Connectedness for enriching elderly care: Interactive Installation & System ...Jun Hu
Ageing has become a global topic with critical challenges for years. Currently, most attention of design and technological solutions for the ageing population is paid to physical health, mobility and safety, while in the field of social wellbeing and mental health, which are also important in ageing process, there is still much space to explore.
Closer to Nature: Interactive Systems for Seniors with Dementia in Long-term ...Jun Hu
People with dementia living in Long-term Care (LTC) are gradually experiencing diminished functional abilities caused by this brain disease. The declined cognitive functioning, decreased mobility, loss of memory and inner motivation provides inevitable challenges in engaging this group in activities. Lack of engagement are associated with disruptive behavioral and psychological symptoms of dementia (BPSD) such as agitation, wondering, apathy, passivity and depression. With no known cure in sight, developing and evaluating meaningful activities that foster and sustain engagement is critical for promoting quality of life for seniors with dementia in LTC.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Search and Society: Reimagining Information Access for Radical FuturesBhaskar Mitra
The field of Information retrieval (IR) is currently undergoing a transformative shift, at least partly due to the emerging applications of generative AI to information access. In this talk, we will deliberate on the sociotechnical implications of generative AI for information access. We will argue that there is both a critical necessity and an exciting opportunity for the IR community to re-center our research agendas on societal needs while dismantling the artificial separation between the work on fairness, accountability, transparency, and ethics in IR and the rest of IR research. Instead of adopting a reactionary strategy of trying to mitigate potential social harms from emerging technologies, the community should aim to proactively set the research agenda for the kinds of systems we should build inspired by diverse explicitly stated sociotechnical imaginaries. The sociotechnical imaginaries that underpin the design and development of information access technologies needs to be explicitly articulated, and we need to develop theories of change in context of these diverse perspectives. Our guiding future imaginaries must be informed by other academic fields, such as democratic theory and critical theory, and should be co-developed with social science scholars, legal scholars, civil rights and social justice activists, and artists, among others.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
5. B. Rinner 5
Sensor Node Platforms
• From research prototypes to commercial products
The Vision
„Smart Dust“ UC
Berkeley
late 1990‘s
Commercial Products
„Mote-on-a Chip“
Dust Networks, 2010
Research Prototypes
„Mica-2“
Crossbow 2004
6. B. Rinner 6
Some Applications of Sensor Networks
• Health
• Structural Monitoring
• Agriculture
• Environmental
[(c) University of Ghent][Kim et al. ACM SenSys, 2006
[AgriNet][M. Welsh, Harvard 2007]
7. B. Rinner 7
Communication is Key
• Wireless communication is an enabling technology
– Eases deployment
– Enables mobility
– Increases flexibility
– Reduces costs
• Communicate on demand (ad hoc, spontaneous) with dynamic
infrastructure
– Nodes organize themselves into network
– Data is transferred via multiple hops
source destination
9. B. Rinner 999
The “Energy Problem”
• Major sources of energy consumption
– Sensing, computing, communication
– High temporal variation
• Energy is the scarce resource for WSN. Several challenges
– What energy reservoirs to exploit?
Constraints: availability, max. power, size, …
– How to distribute power over the network?
Energy provider and consumer might be dislocated.
– How to control the distribution?
“The proper amount of energy in the right place at the right time”
• Sensor networks have always been a “green” technology!
10. B. Rinner 10
Alternative Energy Reservoirs
• Maybe micro-heat engines
– Exploit MEMS technology
to build „internal combustion engines“
– Expected power: 10-20 W
– Still in early research/development
phase
• Or harvest energy from environment
– Organic semiconductors for exploiting indoor ambient light
– Thin film batteries for storing energy
– EnHANTs : energy harvesting
networked active tags
[Handbook of Sensor Networks, Wiley]
[Columbia University, CLUE]
12. B. Rinner. 12
Principle of Smart Cameras
• Smart cameras combine
– sensing,
– processing and
– communication
in a single embedded device
• perform image and video analysis in real-time closely
located at the sensor and transfer only the results
• collaborate with other cameras in the network
TrustEYE.M4 prototype
on top of RaspberryPI
13. B. Rinner. 13
Differences to traditional Cameras
Traditional Camera
– Optics and sensor
– Electronics
– Interfaces
delivers data in form of
(encoded) images and videos,
respectively
Smart Camera
– Optics and sensor
– Onboard computer
– Interfaces
delivers abstracted image data and
is configurable and programmable
Sensor
Electronics
Image enhancement/
Compression
Image
Video
Sensor
Embedded
Computer
Image analysis
„Events“
Programming
Configuration
Light Light
15. B. Rinner 15
Be aware of scarce Resources
• Major resource limitations
– Processing power
– Communication bandwidth
– Onboard memory
– Energy
• Various Prototypes (with decreasing performance)
Sony XCISX100C/XP
x86 VIA Eden ULV @ 1 GHz
TrustEYE.M4
ARM Cortex@ 168MHz
SLR Engineering
Atom Z530@ 1.6 GHz
CITRIC
PXA 270@ 13-640MHz
[Rinner et al. The Evolution from Single to Pervasive Smart Cameras. Proc. ICDSC 2008]
17. B. Rinner 17
Video Surveillance Network
• 3rd generation
– all-digital systems
• 3+ generation
– smart cameras
– surveillance tasks run on-site on smart cameras, e.g.,
• video compression traffic statistics
• accident detection wrong-way drivers
• stationary vehicles (tunnels) vehicle tracking
• 1st and 2nd generation
– primarily analog frontends
– backend systems are digital
[Regazzoni, Ramesh, Foresti. Special Issue on Video Communications, Processing and Understanding
for Third Generation Surveillance Systems. Proceedings of the IEEE. October 2001]
18. B. Rinner 18
Video Surveillance Network (2)
• Even third generation networks rely on “heavy” infrastructure.
– Camera nodes: sensor, onboard processing (encryption)
– Network: hierarchically structured, wired, large bandwidth
– Energy: dedicated supply
• Surveillance networks typically consist of large number of
cameras
• Processing in network is fixed; (compressed) data is streamed
to control center
19. B. Rinner 19
Characteristics of VSN
• Visual sensor networks lie somewhere in between wireless
sensor networks (WSN) and multi-camera/surveillance
networks.
• VSN have unique characteristics (wrt. traditional WSN)
• Resource limitations
– Need to process and transfer large amounts of data
– Energy and bandwidth
• On-board processing (cp. Smart cameras)
– Challenging vision algorithms
– Adaptive behavior
[Soro et al. A Survey of Visual Sensor Networks. Advances in Multimedia 2009]
20. B. Rinner 20
Characteristics of VSN (2)
• Real-time operation
– Most applications require real-time analysis (camera to user)
• Location and orientation information (spatial calibration)
– Absolute or relative coordinates and orientations
– (Multi-)camera calibration
• Time Synchronization (temporal calibration)
• Data Storage
– Access to historic data necessary, eg., frame buffer, detected events
– Stored data may be discarded over time
• Autonomous Camera Collaboration
– cp. Distributed smart cameras (DSCs)
21. B. Rinner. 21
(Selected) VSN Problems
• Sensor Placement
– Eg., dynamic setting of PTZ parameters
• Clustering, cluster head election
– Eg., what cameras should “work together”, who is the “leader”
• Synchronization and calibration
– Eg., establish temporal and spatial correlation
• Data (and energy) distribution
– Eg., when and what data to exchange
• …
23. B. Rinner 23
Configuring Smart Camera Networks
• Smart camera networks process data onboard can modify their
functionality/execute actions during runtime to reflect changes
– to the state of the environment
– to the user criteria
• A configuration describes what is processed/executed where;
specified by
– Description of camera network (including the available actions/tasks)
– Specification of the objective
• We study configuration methods to use scarce resources in
these networks more efficiently
24. B. Rinner 24
Configuration Problem (example)
• Configuring a camera network
– Select a set of cameras to monitor an area of interest
– Set the sensor (frame rate, resolution, PTZ) to achieve QoS
– Assign monitoring functions to cameras
– Optimize wrt. multiple criteria
– Dependent on dynamics of environment
s1
s2 s3
s4
s5
t1
t2
t3
p1, p2
p3
p4, p5
25. B. Rinner 25
Configuration Design Space
Design space for configuration methods is given by:
• Dynamics of environment (static vs. mobile observation points)
• Configuration algorithm (centralized vs. distributed)
• Tasks and sensors (homogeneous/heterogeneous; static/mobile cameras)
• A priori knowledge (complete vs. no knowledge of environment/VSN)
• Various alternatives for solving this optimization problem, eg.
– Centralized configuration algorithms
– Distributed configuration algorithms
focus on resource-aware approaches
26. B. Rinner 26
Centralized Configuration with EA
• Approximation with evolutionary algorithm satisfying all
requirements along multiple criteria (eg., energy, data, QoS)
• Smart Camera Network
– Set of cameras at known position with fixed FoV
– Sensor configurations (frame rate, resolution)
• Observation Area
– Static set of observation points with monitoring activity a
at required QoS (pot, fps)
• Monitoring tasks
– Assign procedures for achieving
– Required resources for
},...{ 1 nSSS =
[Dieber, Micheloni, Rinner. Resource-Aware Coverage and Task Assignment in Visual Sensor Networks
IEEE Transactions on Circuits and Systems for Video Technology, Aug 2011]
},...,{ 1 ki ddD =
},...,{ 1 mttT =
},...,{; 1 apaa ppPAa =∈
),,(),( iiiii emcdPr →
27. B. Rinner 27
Self-aware Configuration
• Adopted from proprioceptive computing systems
– use proprioceptive sensors to monitor “one self”
(concept from psychology, robotics/prosthetics, …, fiction)
– reason about their behavior (self-awareness)
– effectively and autonomously adapt their behavior to
changing conditions (self-expression)
• Demonstrate autonomous multi-object tracking in camera
network
– Exploit single camera object detector & tracker
– Perform camera handover
– Learn camera topology
28. B. Rinner 28
Bid C4
Virtual Market-based Handover
• Initialize auctions for exchanging tracking responsibilities
– Cameras act as self-interested agents, i.e., maximize their own utility
– Selling camera (where object is leaving FOV) opens the auction
– Other cameras return bids with price corresponding to “tracking” confidence
– Camera with highest bid continues tracking;
trading based on Vickrey auction
Camera 1 Camera 2
Camera 3
Camera 4
Init
auction
Bid C3
Fully distributed approach
no a-priori topology knowledge required
29. B. Rinner 29
Market-based Tracking Handover
• Utility function (each camera) rpjvcOU
iOj
ijjii +−Φ⋅⋅= ∑∈
)]([)(
tracking decision
visibility
confidence
payments made
payments received
Simulation
green: tracking
yellow: shared FOV
red: trading (handover)
30. B. Rinner 30
Tracking Performance
• Tradeoff between utility and communication effort
Scenario 1 (5 cameras, few objects) Scenario 2 (15 cameras, many objects)
• Emerging Pareto front
[Esterle et al. Socio-Economic Vision Graph Generation and Handover in
Distributed Smart Camera Networks. ACM Trans. Sensor Networks. 2013]
31. B. Rinner 31
Learn Neighborhood Relationships
• Gaining knowledge about the network topology (vision graph) by
exploiting the trading activities
• Temporal evolution of the vision graph
32. B. Rinner 32
Learning Heterogeneous Strategies
• Heterogeneous strategies at cameras may improve Pareto front
• Adapt camera behaviour by online learning using bandit solvers
Homogeneous vs. heterogeneous
handover strategies (offline)
Online learning strategies with
different bandit solvers
[Lewis et al. Learning to be different: Heterogeneity and Efficiency in Distributed
Smart Camera Networks. In Proc. IEEE SASO. 2013]
34. B. Rinner 343434
#1 Trustworthy Cameras
• Smart cameras
– Highly capable embedded systems (on-board video analysis)
– Large software stacks
– Networked devices using closed (CCTV) and public networks
• Applications no longer only in public but also in private
areas (assisted living, home monitoring, …)
• Protection of sensitive image data
– Protection against manipulation (e.g., enforcement applications;
evidence at court)
– Privacy of monitored people
35. B. Rinner 353535
Goals and Assumptions
• We present a system level approach that addresses the following
security issues:
– Integrity: detect manipulation of image and video data
– Authenticity: provide evidence about the origin of image and videos
– Confidentiality: make sure that privacy sensitive image data cannot be
accessed by an unauthorized party
– Multi-level Access Control: support different abstraction levels and
enforce access control for confidential data
• Security and privacy protection as inherent features of the
camera
• Considered attack types: only software attacks
[Winkler, Rinner. Securing Embedded Smart Cameras with Trusted Computing.
EURASIP Journal on Wireless Communications and Networking, 2011 ]
36. Approach
• Bringing of Trusted Computing concepts into cameras
• Trusted Platform Modules (TPMs) are well defined, readily
available and cheap
• TC is an open industry standard
• TPMs are available from many manufacturers
B. Rinner 36
37. Hardware Security Anchor
37
• Trusted Platform Module (TPM) at a glance
– Secure storage for cryptographic keys
– Data encryption, digital signatures
– System status monitoring and reporting (measurement + attestation)
– Unique platform ID
Security Chip
(TPM)
Image Sensor CPU RAM
Bootloader
Operating System (e.g., Embedded Linux)
Software Libraries and Middleware
Image Processing and Analysis Communication…
Software
Hardware
B. Rinner
38. Implemented Security Features
38
• Trusted boot where camera software stack is “measured” and
the status is securely reported to operator
• Integrity and authenticity guarantees using non-migratable,
TPM-protected RSA keys
• Freshness/timestamping for outgoing images via TPM-
protected tick (counter) sessions
B. Rinner
39. B. Rinner 393939
Hardware Prototype
• TI OMAP 3530 CPU:
ARM @ 480MHz and
DSP @ 430MHz
• 256MB RAM,
SD-Card as mass storage
• VGA color image sensor
• wireless: 802.11b/g WiFi
and 802.15.4 (XBee)
• LAN via USB
(primarily used for debugging)
• Atmel hardware TPM
on I2C bus
40. Privacy Protection Approaches
40
• Protection as an inherent
feature of the camera
• Object-based protection:
Identification of sensitive
data (e.g., human faces)
• Data abstraction and
obfuscation
• Global protection techniques: Uniform protection of entire
frames (insensitive to misdetections of computer vision)
B. Rinner
41. Multi-Level Protection
41
• Video stream contains sub streams
• Every sub stream is encrypted
– Hardware-bound cryptographic keys
• Recovery of identities only via four eyes
principle
Video Stream
Smart
Camera
Sub Streams
B. Rinner
43. B. Rinner 434343
Privacy-aware Camera Networks
• What about users (i.e., monitored people)?
• users usually do not care much about integrity, authenticity of time
stamping
• users (hopefully!) care about confidentiality and privacy!
• Question 1: How can we increase privacy awareness?
• Question 2: How can we demonstrate that (our) cameras
protect the privacy of users?
44. B. Rinner 444444
Raising Privacy Awareness
• Let users know if there are cameras in their environment
• Use user's handheld (e.g., smart phone) for location-based
notifications
45. User Feedback
• Goal: Trustworthy feedback to monitored
persons about camera’s privacy protection
• Visual communication for authentication
– Direct line of sight
– Intuitive way to select intended camera
• Operator discloses applications to TrustCenter
T. Winkler and B. Rinner, “User Centric Privacy Awareness in Video Surveillance,”
Multimedia Systems Journal, vol. 18, no. 2, pp. 99–121, 2012.
B. Rinner
47. B. Rinner 47
#2 Aerial Cameras for Disaster Mgm.
• Develop autonomous multi-UAV system for aerial
reconnaissance
• Up-to-date aerial overview images are helpful in many
situations:
“Google Earth with up-to-date images in high resolution”
• Small-scale quadcopter platform with
onboard sensors and computation
• GPS receiver for autonomous
waypoint flights
• Generic framework not bound
to specific UAV
48. B. Rinner 48
Key Challenges
• Increase autonomy
– Control and coordination of multiple UAVs
– High-level interaction with user
• Provide prompt response to user
– Provide preliminary results fast and improve over time
• Deal with strong resource limitations
– Flight time, payload, computation and communication
– Limited sensing capabilities
50. B. Rinner 50
Key Questions
• How to generate and update movement routes for the UAVs?
– Achieve multiple optimization goals
– Deal with changes in the environment
• How to setup a wireless UAV network?
– Provide networking coverage
• How to generate the mosaic image?
– Apply incremental image stiching
– Combine RGB and thermal images
• System integration and demonstration
52. B. Rinner. SCVSN Tutorial (Chapter 3) 525252
Research Directions of
Visual Sensor Networks
53. B. Rinner. 53
#1: Architecture
• Low-power (high performance) camera nodes
– Dedicated platforms: vision processors, PCBs, systems
– Many examples: CITRIC, NXP
• Visual/Multimedia Sensor Networks
– Topology and (multi-tier) architecture
– Multi-radio communication
• Dynamic Power Management
– For sensing, processing and communication
How to design resource-aware nodes and networks
54. B. Rinner. 54
#2: Networking
• Ad hoc, p2p communication over wireless channels
– Providing RT and QoS
– Eventing and/or streaming
• Dynamic resource management
– (local) computation, compression, communication, etc.
– Degree of autonomy: dynamic, adaptive, self-organizing
– Fault tolerance, scalability
– Network-level software, middleware
How to process and transfer data in the network
55. B. Rinner 55
#3: Deployment, Operation, Maintenance
• Development support for applications
– Model/simulate the application (function, resources, QoS)
– Reuse/exchange of software/libraries
– Software updates, debugging etc.
• Autonomous calibration and scene adaption
– Avoid manual procedures
– Adapt to different scenes and settings
• Network configuration
Consider the entire life cycle of the camera network
56. B. Rinner. 56
#4: Distributed Sensing & Processing
• Sensor placement, calibration & selection
– Optimization problem
– Distributed approaches eg., consensus, game theory, multi-agent systems
• Compressive Sensing
• Collaborative data analysis
– Multi-view, multi-temporal, multi-modal
– Sensor fusion
• Online/real-time processing
– Can not effort to store large amounts of data
Where to place sensors and analyze the data
57. B. Rinner 57
#5: Mobility
• Mobile cameras are ubiquitous
– PTZ, vehicles, robotics etc.
– Mobile phones
• Advanced vision algorithms
– Ego motion, online calibration
– Closed-loop control, active vision
How to exploit networks of mobile cameras
58. B. Rinner 58
#6: Usability
• Ease of deployment, maintenance
– Self-* functionality
– “Smart cameras for dumb people”
• Privacy and Security
– Trust of the user
– Control the privacy setting
• Interaction with the camera network
How to provide useful services to people
59. B. Rinner 59
#7: Applications
• Demonstrations
– Large scale networks eg., for surveillance
– Small scale networks eg., for entertainment, home environments
– Only single camera application?
• Market opportunities
• Killer Application
What applications can (only) be solved by DSC
60. B. Rinner. SCVSN Tutorial (Chapter 3) 60
Summary
• VSNs exploit various advantages of distributed camera sensors
such as increased coverage, redundancy and 3D information.
• Distributed cameras impose various challenges such as huge
amount of data, required infrastructure and (network)
topology.
• VSN have unique characteristics (wireless sensor networks vs.
surveillance camera networks)
• Current research addresses signal processing,
communications, architecture and middleware issues.
61. B. Rinner 61
Acknowledgements & Further Info
Pervasive Computing @ AAU UAV Research
http://pervasive.aau.at http://uav.aau.at
• Tutorial site
Most recent course material is available at
http://pervasive.aau.at/S5-tutorial