codename: imaGInation KIeve

FYP Presentation – the Finale! 24.04.08 Project Team : Syed Waqas A Burney, 2004185, FCSE (Team Lead) Mutahira Ikram Khan, 2004136, FCSE Project Advisor : Mr. Badre Munir, FCSE

FYP Presentation – the Finale!

24.04.08

Project Team :

Syed Waqas A Burney, 2004185, FCSE (Team Lead)

Mutahira Ikram Khan, 2004136, FCSE

Project Advisor :

Mr. Badre Munir, FCSE

Agenda : Core Progeny Project Objective Project Modularization The System Overview Architecture Module 1 – Motion Detection & Human Activity Monitoring Module 2 – The Decision Making Software Engine Module 3 – Appliance Control Mechanism & Prototype Modeling Project Status Envisioning Ahead v Smart Demo

Agenda :

Core Progeny

Project Objective

Project Modularization

The System

Overview

Architecture

Module 1 – Motion Detection & Human Activity Monitoring

Module 2 – The Decision Making Software Engine

Module 3 – Appliance Control Mechanism & Prototype Modeling

Project Status

Envisioning Ahead

v Smart Demo

Core Progeny Computer Vision The branch of applied computing that concerns modeling and developing artificial systems that attain information from images and thereby provide useful functionality and results. Domotics Study of specific automation requirements for homes and application of automation techniques for the comfort, convenience and security of the residents Development Platform : .NET 2.0 Framework (C#) Visual Studio 2005 MS SQL Server 2005

Core Progeny

Computer Vision

The branch of applied computing that concerns modeling and developing artificial systems that attain information from images and thereby provide useful functionality and results.

Domotics

Study of specific automation requirements for homes and application of automation techniques for the comfort, convenience and security of the residents

Development Platform :

.NET 2.0 Framework (C#) Visual Studio 2005 MS SQL Server 2005

Objective Prototyping an integrated system for the smart control of domestic electronic appliances in local environments, based on monitored user presence and activity ; thereby achieving multidimensional user-convenience and using modern technology to endow a sustainable environment in our everyday lives. Applications : User Convenience and Energy Saving Offices (lunch hours, after-office timings, etc) Houses (the ‘Digital Home’ concept)

Objective

Prototyping an integrated system for the smart control of domestic electronic appliances in local environments, based on monitored user presence and activity ; thereby achieving multidimensional user-convenience and using modern technology to endow a sustainable environment in our everyday lives.

Applications :

User Convenience and Energy Saving

Offices (lunch hours, after-office timings, etc)

Houses (the ‘Digital Home’ concept)

Modularization (and timeline) Problem Statement Ideation through innovative techniques/lateral thinking/ brain storming Engineering solutions, and thereby outlining the very FYP definition, field & scope Adopting a research based/experimental approach System design Motion detection and respective human activity monitoring Prototype modeling Serially interfaced appliance control mechanism Decision taking software engine– a multi-threaded expert system Optimization and deployment Completion Nov Jun Dec Jan Mar Apr codename: imaGInation KIeve

Modularization (and timeline)

Problem Statement

Ideation through innovative techniques/lateral thinking/ brain storming

Engineering solutions, and thereby outlining the very FYP definition, field & scope

Adopting a research based/experimental approach

System design

Motion detection and respective human activity monitoring

Prototype modeling

Serially interfaced appliance control mechanism

Decision taking software engine– a multi-threaded expert system

Optimization and deployment

Completion

System Overview Module 1 Human Activity Monitoring ( Input ) Module 2 The Software Engine ( Processing ) Module 3 Appliance Control ( Output ) CMOS Webcams Local Environment

System Overview

Module 1

Human Activity Monitoring ( Input )

Module 2

The Software Engine

( Processing )

Module 3

Appliance Control ( Output )

CMOS Webcams

Local Environment

System Architecture

System Architecture

System Architecture MODULE 1 MODULE 1

System Architecture

Module 1 Motion detection and respective human activity monitoring System Deployment Multi-Camera Registration & Deployment HotSpot Marking Utility System GUI System Initialization Motion Detection MER Formation Multi-Threading

Module 1

Motion detection and respective human activity monitoring

System Deployment

Multi-Camera Registration & Deployment

HotSpot Marking Utility

System GUI

System Initialization

Motion Detection

MER Formation

Multi-Threading

Module 1 (cntd..) Motion detection and respective human activity monitoring Critical Difficulties Encountered– Camera Zoom [View Angle , Shadows] Camera Mounted in Room Camera Mounted in the Model

Module 1 (cntd..)

Motion detection and respective human activity monitoring

Critical Difficulties Encountered– Camera Zoom [View Angle , Shadows]

Camera Mounted in Room

Camera Mounted in the Model

Module 1 (cntd..) Motion detection and respective human activity monitoring Prototype Environments : Room (Hostel); Model House Image Capture Devices : CMOS Webcam; 640 x 480 pixels; Frame Rate: 30 fps; View Angle: 54 degrees Platform : .NET 2.0 based Aforge C# Framework (open source) Using the AForge.Imaging & AForge.Vision libraries [ http://code.google.com/p/aforge/ ]

Module 1 (cntd..)

Motion detection and respective human activity monitoring

Prototype Environments :

Room (Hostel); Model House

Image Capture Devices :

CMOS Webcam; 640 x 480 pixels; Frame Rate: 30 fps; View Angle: 54 degrees

Platform :

.NET 2.0 based Aforge C# Framework (open source)

Using the AForge.Imaging & AForge.Vision libraries

[ http://code.google.com/p/aforge/ ]

Module 1 (cntd..) Motion detection and respective human activity monitoring User Detection Approach : Current–Previous, Current–First Algorithm Blob Motion Algorithm Capturing Results : MER ( Minimum Enclosed Rectangle) Drawing (.NET Imaging Library) Weighted Average, K-Means/K-Means ++ Algorithm Activity Monitoring : Initial Hotspot outline by the User on the S/W (.NET Imaging Library) Hotspot & MER Overlap Detection using Crossing Number Method

Module 1 (cntd..)

Motion detection and respective human activity monitoring

User Detection Approach :

Current–Previous, Current–First Algorithm

Blob Motion Algorithm

Capturing Results :

MER ( Minimum Enclosed Rectangle) Drawing (.NET Imaging Library)

Weighted Average, K-Means/K-Means ++ Algorithm

Activity Monitoring :

Initial Hotspot outline by the User on the S/W (.NET Imaging Library)

Hotspot & MER Overlap Detection using Crossing Number Method

Module 1 (cntd..) Motion detection and respective human activity monitoring MER formation for Motion Detection Hotspot marking by user

Module 1 (cntd..)

Motion detection and respective human activity monitoring

MER formation for Motion Detection

Hotspot marking by user

Module 1 (cntd..) Motion detection and respective human activity monitoring Multi-Cam Supportive GUI User-friendly Camera Registration Spawning Threads for Concurrent Multi-User Support

Module 1 (cntd..)

Motion detection and respective human activity monitoring

Multi-Cam Supportive GUI

User-friendly Camera Registration

Spawning Threads for Concurrent Multi-User Support

System Architecture MODULE 2 MODULE 2

System Architecture

Module 2 Decision taking software engine – an expert system Knowledge Base Design Action Triggering Overlap Calculations Other Parameter Calculations Reaction Generation Serial Transmission

Module 2

Decision taking software engine – an expert system

Knowledge Base Design

Action Triggering

Overlap Calculations

Other Parameter Calculations

Reaction Generation

Serial Transmission

Module 2 Decision taking software engine – an expert system The brain of the system– responsible for the actual “monitoring” Decides when to turn off what Expert System : Defining classes for rules Having defined facts in a relational database Using inference techniques to judge which rules, based on user activity, stand, and hence which devices are to be respectively switched off Rules and Facts subject to changes and modifications based on user behavior Overview : Small scale expert system with a dynamically adjusting knowledge base

Module 2

Decision taking software engine – an expert system

The brain of the system– responsible for the actual “monitoring”

Decides when to turn off what

Expert System :

Defining classes for rules

Having defined facts in a relational database

Using inference techniques to judge which rules, based on user activity, stand, and hence which devices are to be respectively switched off

Rules and Facts subject to changes and modifications based on user behavior

Overview :

Small scale expert system with a dynamically adjusting knowledge base

System Architecture MODULE 3 MODULE 3

System Architecture

Module 3 Serially interfaced appliance control mechanism Connection to Devices : Serial port communication via RS232 Microcontroller Interface circuitry Device Model House. Microcontroller Interface Electronic Devices Computer RS232

Module 3

Serially interfaced appliance control mechanism

Connection to Devices :

Serial port communication via RS232

Microcontroller

Interface circuitry

Device Model House.

Microcontroller

Interface

Electronic Devices

Computer

RS232

Module 3 Serially interfaced appliance control mechanism Trainer Board Circuitry Vero Board Circuitry

Module 3

Serially interfaced appliance control mechanism

Trainer Board Circuitry

Vero Board Circuitry

Module 3 Serially interfaced appliance control mechanism

Module 3

Serially interfaced appliance control mechanism

Module 3 Serially interfaced appliance control mechanism Serial Communication Data is transmitted to the microcontroller via RS-232 standard DB- 9 connector is used to connect to the microcontroller Hardware circuitry ATMEL 89C51 MAX 233 2n222 Transistors Diodes 6 volt relays Capacitors Resistors

Module 3

Serially interfaced appliance control mechanism

Serial Communication

Data is transmitted to the microcontroller via RS-232 standard

DB- 9 connector is used to connect to the microcontroller

Hardware circuitry

ATMEL 89C51

MAX 233

2n222 Transistors

Diodes

6 volt relays

Capacitors

Resistors

Module 3 Serially interfaced appliance control mechanism Circuit : Part I

Module 3

Serially interfaced appliance control mechanism

Circuit : Part I

Module 3 Serially interfaced appliance control mechanism Circuit : Part II (implemented on Veroboard)

Module 3

Serially interfaced appliance control mechanism

Circuit : Part II (implemented on Veroboard)

Module 3 (cntd..) Serially interfaced appliance control mechanism Microcontroller ( ATMEL 89C51): Receives data serially from computer port Processes the data Sends data to its port to which devices are connected Turning relays on and off MAX 233 RS-232 not compatible with today’s microcontrollers Line driver to convert RS-232 signal to TTL logic levels Interface Circuit : BJT based Relay-Driver (2n222) : It provides sufficient current to drive the relay Diode : When the relay is switched off, magnetic field generated inside produces a high voltage, which can damage circuit Diode prevents short circuiting when the relay is switched off

Module 3 (cntd..)

Serially interfaced appliance control mechanism

Microcontroller ( ATMEL 89C51):

Receives data serially from computer port

Processes the data

Sends data to its port to which devices are connected

Turning relays on and off

MAX 233

RS-232 not compatible with today’s microcontrollers

Line driver to convert RS-232 signal to TTL logic levels

Interface Circuit :

BJT based Relay-Driver (2n222) :

It provides sufficient current to drive the relay

Diode :

When the relay is switched off, magnetic field generated inside produces a high voltage, which can damage circuit

Diode prevents short circuiting when the relay is switched off

Module 3 (cntd..) Serially interfaced appliance control mechanism Interface Circuit (cntd..) : Resistor : It prevents flow of high current from the transistor to the ground on the application of VCC Relay : Mechanical relay is used to switch device off

Module 3 (cntd..)

Serially interfaced appliance control mechanism

Interface Circuit (cntd..) :

Resistor :

It prevents flow of high current from the transistor to the ground on the application of VCC

Relay :

Mechanical relay is used to switch device off

Modeling the Prototype The Model House ( realistic cut-scale ) Model Specifications: 3.5‘ x 3.5‘ x (equivalent height) Two bedroom house (1.5‘ x 1.5') Double-walled structure with the outer walls removable The house is raised 1 feet in height, by supports, for magnet movements Test Men with magnets are used to depict movements in the household Outer Slide-Up-and-Removable Walls Normal Walls Wiring concealed in between by the removable Outer Wall ROOM

Modeling the Prototype

The Model House ( realistic cut-scale )

Model Specifications:

3.5‘ x 3.5‘ x (equivalent height)

Two bedroom house (1.5‘ x 1.5')

Double-walled structure with the outer walls removable

The house is raised 1 feet in height, by supports, for magnet movements

Test Men with magnets are used to depict movements in the household

Modeling the Prototype (cntd..) The Model House House Dimensions: 3.5’ x 3.5’ x 2’ Room Dimensions: 1.5’ x 1.5’ x 8”

Modeling the Prototype (cntd..)

The Model House

House Dimensions: 3.5’ x 3.5’ x 2’ Room Dimensions: 1.5’ x 1.5’ x 8”

Modeling the Prototype (cntd..) The Model House with Domestic Appliances Installed Installed Appliances: Room Lights, Room Fan, Bed-side Lamp & Study-table Lamp

Modeling the Prototype (cntd..)

The Model House with Domestic Appliances Installed

Installed Appliances: Room Lights, Room Fan, Bed-side Lamp & Study-table Lamp

(Work-based) Modular Completion Status Module 1 : Motion detection and respective human activity monitoring System Deployment Multi-Camera Registration & Deployment HotSpot Marking Utility System GUI System Initialization Motion Detection MER Formation Multi-Threading ( debugging issues! ) Module 2 : Decision taking software engine – an expert system Action Triggering Overlap Calculations Other Parameter Calculations Knowledge Base Design Reaction Generation Serial Transmission Module 3 : Serially interfaced appliance control mechanism Trainer/Vero-board Circuit Designing Microcontroller Interpretation Electrical Interfacing of Appliance System Optimization

(Work-based) Modular Completion Status

Module 1 : Motion detection and respective human activity monitoring

System Deployment

Multi-Camera Registration & Deployment

HotSpot Marking Utility

System GUI

System Initialization

Motion Detection

MER Formation

Multi-Threading ( debugging issues! )

Module 2 : Decision taking software engine – an expert system

Action Triggering

Overlap Calculations

Other Parameter Calculations

Knowledge Base Design

Reaction Generation

Serial Transmission

Module 3 : Serially interfaced appliance control mechanism

Trainer/Vero-board Circuit Designing

Microcontroller Interpretation

Electrical Interfacing of Appliance

System Optimization

Envisioning Ahead Microsoft Imagine Cup 2008 GIKI Participation @ Different Technology Competitions ~ Softec ’08 Submissions on Various Research-based Platforms for Domotics Video Documentation

Envisioning Ahead

Microsoft Imagine Cup 2008

GIKI Participation @ Different Technology Competitions ~ Softec ’08

Submissions on Various Research-based Platforms for Domotics

Video Documentation