Internet of Things, IoT

1. 1
Personal and Community Context
Discovery
17 May 2015
Arpan Pal
Principal Scientist and Research Head
Innovation Lab, Cyber physical Systems
Tata Consultancy Services (TCS)

2. 2
Tata Consultancy Services Ltd. (TCS)
 Pioneer & Leader in Indian IT
TCS was established in 1968
 One of the top ranked global software service provider
 Largest Software service provider in Asia
 250,000+ associates
 USD 10B + annual revenue
 Global presence
 First Software R&D Center in India
- 2 -

3. 3
The Heart of Innovation – TCS Innovation Labs
Bangalore, India1
TCS Innovation Labs - Bangalore
Chennai, India2
TCS Innovation Labs - Chennai
TCS Innovation Labs - Retail
TCS Innovation Labs - Travel & Hospitality
TCS Innovation Labs - Insurance
TCS Innovation Labs - Web 2.0
TCS Innovation Labs - Telecom
Cincinnati, USA3
TCS Innovation Labs - Cincinnati
Delhi, India4
TCS Innovation Labs - Delhi
Hyderabad, India5
TCS Innovation Labs - Hyderabad
TCS Innovation Labs - CMC
Kolkata, India6
TCS Innovation Labs - Kolkata
Mumbai, India7
TCS Innovation Labs - Mumbai
TCS Innovation Labs - Performance Engineering
Peterborough, UK8
TCS Innovation Labs - Peterborough
Pune, India9
TCS Innovation Labs - TRDDC - Process Engineering
TCS Innovation Labs - TRDDC - Software Engineering
TCS Innovation Labs - TRDDC - Systems Research
TCS Innovation Labs - Engineering & Industrial Services
1 2
3
4
5
97
6
8
2000
+
Associates in Research, Development and Asset
Creation
19 Innovation Labs

4. 4
Integrated Platform for Intelligent Infrastructure
People Feedback & Emotions
Social Media
Integrated Services
Sensors & IoT
Platform
Traditional Monitoring & Control
Systems
Citizen Data
Smart Integration Platform
Transportation Healthcare Electricity
WaterPublic Safety Tourism
Smart Integrated Services
Sense
Analyze
Extract
Respond
Intelligence
Smart Domain Services
Community
etc.
Sense: People Activity, Appliances, Vehicles , Road, Home/Bldg, Utility
Infrastructure
Detect gas leakage/water contamination :
mobilize rescue team, suggest optimum route
Divert Road Traffic in case of
Water Pipeline Burst
Correlate Electricity/Water
/Gas consumption patterns
Intelligent Integration Platform
Integrated Intelligent Services

5. Outline
What we mean by Context
Example Use Cases
Proposed System
Conclusion

6. 6
Personal and Community Context Discovery
Context - patterns of individual, group and societal behaviours.
Broadly classified into three categories –
 Personal Physical Network Discovery
 Who is interacting with whom? What is the level of interaction? Who all are
part of similar-interest networks?
 Individual Context Discovery
 Who is doing what?
 Community Context Discovery
 Can we discover how a community / group behaves as a whole?

7. 7
Example Use Case - Campus
Source: Zhang et. al., “Extracting Social and Community Intelligence from Digital Footprints: An Emerging
Research Area”, UIC 2010, LNCS 6406, pp. 4–18, 2010. © Springer-Verlag Berlin Heidelberg 2010

8. 8
Other Example Use Cases
 Organizational Behavior Analysis
 Team Efficiency Study
 Best Practice Study
 Workspace Ergonomics Study
 Customer Behavior Study in Retail Stores
 Customer movement pattern
 Customer interaction pattern with shelves / merchandize
 Crowdedness measure in public places
 Efficient scheduling of public transport
 Crowd Behavior analysis
 Evacuation planning during disaster
Ref. - Alex Pentland et. al., MIT media Lab

9. 9
What do we need to Sense
 Location
 Proximity
 Activity
 Identity
Provide Context discovery as a
Service

10. 10
How to Sense
 Needs to be Ubiquitous and Unobtrusive
 There should not be any new hardware / device to carry
for an individual
 Proposal
o Use smartphone-based sensors (GPS, accelerometer,
compass, microphone)
o Use 3D surveillance cameras (like Kinect)
o Augment with social network data and email data analytics
o Multimodal fusion of all the above
 Privacy can be an issue – needs to be handled on an use
case-by-use case basis
o Privacy vs. Utility

11. 11
Proposed Architecture
Platform Service
c
Sensors
(location,
proximity,
activity)
Camera
Web-based
soft
sensing
Personal Physical
Network Discovery
Individual Context
Discovery
Microphone
Community Context
Discovery
Behavioral Analytics
Applications
Context Discovery Service
Mobile Phones, Kinect, Email, Social Network
cMultimodal Fusion

12. 12
Mobile Phone Based Sensing
 Proximity / presence
– Using Bluetooth for finding nearby mobiles
– Using Wi-Fi to discover other mobiles nearby
 Location
– Using ultrasound beacon
– Using GPS (outdoors)
– Using Accelerometer / compass
 Activity
– Using Accelerometer
 Interaction Level
– Using Microphone Audio
 Identity
– From Network ID
On-board sensors
Accelerometer, GPS,
Compass
Camera, Microphone
Network
Bluetooth, WiFi, 2G/GPRS, 3G
Network
2G/GPRS, Bluetooth
On-board sensors
Microphone, Camera

13. 13
Sensor Penetration and power consumption in
Mobile Phones
0 20 40 60 80 100
Bluetooth
USB
Edge
GPRS
Wifi
3G
Camera
GPS
Accelarometer
Digital Compass
Consolidated Market Penetration
Source: Nericell: Rich Monitoring of of Road and Traffic
Conditions using Mobile Smartphones, Prashant Mohan et.
al., Microsoft Research, SenSys 2008, North Carolina, USA

14. 14
Kinect Based Sensing
 Human Identification
– Skeleton Model Based
– External Stimulus based refinement
 Network Discovery
– Network discovery through proximity
– Level of Interaction through Audio
• 2D Camera with
IR depth sensor
• Excitation by IR
light pattern
• Directional Mic.

15. 15
Kinect Based Sensing (contd …)
Working on a public Kinect dataset
• People Discussion
• Give/Put/Take an object
• Enter/leave a room
• Leave baggage unattended
• Handshaking
• Typing on a keyboard
• Telephone conversation (Mobile, landline)
Image and the corresponding 3D cloud point
 Human Interaction
– Activity Detection on 3D Point Cloud
– Physical object Identification
– Interaction with Objects
Human activities recognition and localization competition (HARL), ICPR 2012

16. 16
Soft sensing from Web
Unstructured Data
• Social network posts
such as tweets, facebook
• Blog posts
• Email bodies
Structured Data
• Social network profiles
and network information
• Email headers
• Tweet Attributes
Personal
Network

17. 17
IoT Platform from TCS
Internet
End Users
Administrators
Device Integration & Management Services
Analytics Services
Application Services
Storage
Messaging & Event Distribution Services
ApplicationServices
Presentation Services
Application Support Services
Middleware
Edge Gateway
Sensors
Internet
Back-end on Cloud
RIPSAC – Real-time Integrated Platform for Services & AnalytiCs
Traditional
Internet
 Service Delivery
Platform & App
Development
Platform
 Security/Privacy
Framework
 Lightweight M2M
Protocols
 Analytics-as-a-
Service
 Social Network
Integration
 SDKs and APIs for
App developer

18. 18
Summary
 Introduced the concept of personal and community context
discovery as a service with help of example use cases
 Proposed an unobtrusive and ubiquitous way to gather the
context through mobile phone based sensors, 3D camera
and web data
 Each method has its own limitation both from application
and technology perspective
 Need for a multimodal fusion for improved accuracy
 A generic IoT platform to implement and deploy the services
for application developers

19. Thank You
arpan.pal@tcs.com