Panacea - Augmented Reality

PANACEA
TWANACEA
* Possible names for the application

The Team
Navneet
Nikhil
Manohar
Ritesh

For people who are on webex
Please visit this URL for all the videos that we play.
http://tinyurl.com/twdemo

We built
“
An augmented reality application to be used inside an Enterprise to manage contacts and location services.
“

Augmented Reality?
Augmented reality (AR) is a term for a live direct or an indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input, such as sound or graphics
Simple Defn : overlaying of digital data on the real world.
* Courtesy – wikipedia and howstuffworks.com

Contacts
Problems

Contacts
Problems
Ritesh M NayakDeveloper
ThoughtWorks Technologies (India) Pvt Ltd2nd Floor, Tower C, Corporate Block, Diamond District Airport Road, Bangalore - 560 008, Indiatel : +91 80 4064 9570, fax : +91 80 2508 9584
riteshm@thoughtworks.com
… passé

Contacts
Problems
My next visiting card will look like this
Ritesh M Nayak
With Best Wishes
ThoughtWorks Technologies (India) Pvt Ltd2nd Floor, Tower C, Corporate Block, Diamond District Airport Road, Bangalore - 560 008, Indiatel : +91 80 4064 9570, fax : +91 80 2508 9584
riteshm@thoughtworks.com
. . .You know why?

Contacts
Problems
My only use for it now is to attach it to gifts that I give.

A radical new way of exchanging contacts inside the enterprise
vCards 2.0

Contacts Management
Video Demo – Play Video 1 on the supporting website

You meet these guys at X-Conf. How long do you think it will take you exchange your contact information?
We say about 10 seconds!

How we do this.
Tech

Application Walkthrough

- 3 step process
- Core of the idea is a ML
Algorithm for face recognition
As with any ML algorithm, the larger the training set, the better the quality of the algorithm.
Face Recognition
Status flags convey what thread is currently active and processing.

Step 1
Detection /Scanning
No faces in this image
Face Detection –algorithm to detect if there is a face present on the given image.

Step 2
Recognition
Point out the faces that were detected
Face Recognition – Once a face is detected, the face has to be processed by an algorithm which will associate a unique id from the training set.

Step 3
Data/Action Fetch
Augement real time image with info bubble
PrateekshaUday
nono@hotmail.com
9889090123
Once the algorithm returns a unique identifier based on the processed image, we fetch the data corresponding to that id. In our case, the details from Active Directory. (actually, not just AD)

Implementation and Challenges
Uh Oh!

Face Recognition

First Approach
Training Set
Manual Training
Face Detection
LDAP
Face Recognition
Service Calls
App
Massive Database of Photos

Problems with this approach 
Too Painful
Training Set
Manual Training
Face Detection
Face Recognition
Service Calls
App
Not Real time – Not scalable

We found Open Source Computer Vision. A free library that does object detection, recognition and 500 other image processing algorithms.
Yay!
OpenCV is a C++ library 

Second Approach
Gives
Compile Using
opencv.so
C++ Libraries
Android NDK
opencv.so
Java Native Interface (JNI)
App

Pros & Cons - Second Approach
opencv.so
Java Native Interface (JNI)
App
Not Accurate
Near Real time
No Service Calls
Poor Training data
Totally in Phone

Hybrid Approach
Face Detection
Scanning Mode
Face?
Yes
Face Recognition
Training Mode

A new way of putting up Assistance and event posters.
Posters

Contacts
Problems
Wait! Is there a name of a person as well? So, now I know the person as well his/her phone number.
What happens if the Phone number changes?
Exposed Phone numbers are dangerous. Can be abused by telemarketers

A new way of putting up assistance posters. No numbers, no hassles.
For all Travel related Queries

X-Conf 4 – March 5th
Event Posters - Revisited
//TODO: Add functionality that will add this event to your calendar and notify you 10 mins before the event
Title: Don't do that, then! (Doctor, it hurts when I rewrite legacy applications)
* Photo courtesy thoughtworker.com

Presence
* VISION
@starkcoffee - hopefully my.thoughtworks will be a completely fresh experience to TWers !

* Almost there
Dinesh Tantri – Geek Lunch
Talk: Introducing my.thoughtworks
Give Feedback , ask questions – use panacea
Event / Conference Feedback – 1

Event / Conference Feedback - 2
* Almost there
Talk: Introducing my.thoughtworks!
Comments
Feedback

* VISION
Visual URL’s and Bookmarking
Use OCR to save this URL on a poster or on a hoarding.

Wait, does this mean we need to train photos of every employee?Damn, that’s painful.Luckily . . . We have a solution

Distributed Dataset Training

Possibilities …
* Stock photo from stck.xchng

Do you know these applications?
* Logos courtesy their corresponding websites

Location
Problems
Only 13% of the smart phones sold across the world are Location Aware
Less than 4% of all mobile phones sold last year were Location aware.
* Stats courtesy: http://textopiablog.wordpress.com/2010/02/22/how-many-people-own-a-location-aware-device/

Location
Problems
Too many places. Search doesn’t solve the problem either.
*screen shot courtesy google.com

Location
Problems
Roof top Restaurant
Roof top - Bar
Pizzeria
Chinese Restaurant
Cafe
Multiple Avenues – Same GPS Location

Creating and using GPS agnostic Hyper local communities
Location

But, how?
Most smartphones come with a good camera. Given that we already had the technology to recognize faces, we just appropriated it to make a location service just like FourSquare or Gowalla.
(our algorithms can be trained on other objects as well, like boards, toys etc)

Introducing the PlaceMark!
Join my community and share your thoughts

HyperLocal Communities

Enterprise Network
Chicago
Bangalore
Pune
Chennai
Melbourne

Implementation
Face Detection
Scanning Mode
Face?
Yes
Face Recognition
Location?

Genesis, Ideation and Development
Prototype

Mockups
* We used http://iphonemockup.lkmc.ch/ for smart phone mockups

Application Architecture
Main UI Surface
Overlays
Camera
Info Overlay
status Overlay
Processing
Android native detection
Recognition service
Gab server
Location server
PANACEA

http://riteshnayak.com	319
url_unknown	27
https://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com	15
http://feeds.feedburner.com	1

Panacea - Augmented Reality

by itsmeritesh on Mar 11, 2011

Accessibility

Categories

Upload Details

Usage Rights

4 Embeds 362

Statistics

Panacea - Augmented Reality — Presentation Transcript