SlideShare a Scribd company logo
Pocket Data Mining
The Next Generation in
Predictive Analytics
Dr Mohamed Medhat Gaber
School of Computing
University of Portsmouth
E-mail: mohamed.gaber@port.ac.uk
Research Timeline
• 2003 – 2006
– Adaptive resource-aware data stream
mining approach and techniques
– Algorithm Granularity (AG)
• Algorithm Output Granularity (AOG)
• Algorithm Input Granularity (AIG)
• Algorithm Processing Granularity (APG)
• 2007 – 2010
– Situation-aware data stream mining
– Fuzzy Situation Inference (FSI)
• 2008 – 2011
– Clutter-aware visualisation
– Adaptive Clutter Reduction (ACR)
• 2010 – 2012
– Distributed and collaborative mobile data
stream mining
– Pocket Data Mining (PDM)
2
Agenda
• Introduction to Data Streams
• Earlier work
– Granularity-based Approach
– Situation-aware Data Stream Mining
– Clutter-aware Visualisation
• Pocket Data Mining
– Background on Mobile Software Agents
– PDM Architecture and Procedure
– Hoeffding Tree Agent Miner
– Naïve Bayes Agent Miner
– Experimental Results
• Summary
3
Introduction to Data Streams
• The advances in data acquisition hardware and the
emergence of applications that process continuous
flow of data records have led to the data stream
phenomenon.
• A data stream is a continuous, rapid flow of data
that challenge our state-of-the-art processing and
communication infrastructure.
• The general features of data streams are:
– Very high rate input data
– Read only once by an algorithm
– Real time processing demand
– Unbounded
– Time varying. 4
Data Stream Processing in Resource-
constrained Environments
• A wide range of data
streams are generated in or
sent to resource-
constrained computing
environments.
– Spacecrafts
– Wireless sensor networks
– PDAs and smartphones
5
Source: www.freeimages.co.uk
Research Issues
• Limited computational resources
• Limited bandwidth
• Limited screen realestate
• Change of the user’s context
6
Our Approach
• Adaptability with regard to:
– Computational resources
– User’s situation
– Visual clutter
7
Granularity-based Approach
• Combining the three
possible granularity-based
adaptation, namely:
– AIG: Algorithm Input
Granularity
– AOG: Algorithm Output
Granularity
– APG: Algorithm
Processing Granularity
8
9
Situation-Aware Adaptation
9
Situations
Context
Sensory-originated
data
Situation Inferencing
• Capture Application’s
“Situation”
• Fuzzy Context Spaces
• Enhance probabilistic situation
inferencing with fuzziness
• Cope with changing situations
• Cope with unknown situations
10
Adaptive Clutter Reduction
• Similar to resource-awareness and situation-awareness, we have developed a
novel way to automatically reduce the clutter
• The new approach has many important applications (especially in disaster
management)
11
12
Adaptive Clutter Reduction
50% Coverage and 80% Overlap
13
Open Mobile Miner - OMM
14
PDM: Pocket Data Mining
• Pocket Data Mining (PDM) is our
new term describing collaborative
mining of streaming data in mobile
and distributed computing
environments.
• With continuous advances in
computational power and
communication abilities for
smartphones and tablet computers;
and
• The sheer amounts of data streams
that we subscribe to or acquire using
the onboard sensing capabilities
• There is an unprecedented opportunity
to perform complex data analysis tasks
that can benefit mobile users
15
Source: Lane, N.D.; Miluzzo, E.; Hong Lu; Peebles, D.;
Choudhury, T.; Campbell, A.T.; , "A survey of mobile
phone sensing," Communications Magazine, IEEE ,
vol.48, no.9, pp.140-150, Sept. 2010
Technology Enablers
• This can be realised with the help of several established areas
of study including:
– data stream mining;
– mobile software agents; and
– programming for small devices.
16
What is a Mobile Agent ?
17
 A software program
 Moves from machine to machine under
its own control
 Suspends execution at any point in
time, transport itself to a new machine
and resume execution
 Once created, a mobile agent
autonomously decides which locations
to visit and what instructions to
perform
 Continuous interaction with the agent’s
originating source is not required
 HOW?
 Implicitly specified through the
agent code
 Specified through a run-time
modifiable itinerary
JADE Architecture
PDM Architecture and Procedure
18
PDM Agents
• (Mobile) agent miners (AM): these
agents are either distributed over
the network when the mining task
is initiated or are already located
on the mobile device.
• Mobile data stream mining
• Mobile agent resource discoverers
(MRD): these agents are used to
explore the available
computational resources,
processing techniques, and data
sources.
 Mobile cloud
• Mobile agent decision makers
(MADM): these agents roam the
network consulting the mobile
agent miners to collaborate in
reaching the final decision.
• Ensemble learning
19
Source: http://www.datacenterknowledge.com
Source: Polikar 2008
Agent Miners (AMs)
• We have used two stream classifiers, namely:
– Hoeffding trees
• Known for its statistically guaranteed accuracy
– Incremental Naïve Bayes
• Known for its computational efficiency and
simplicity
20
Simple Weighted Majority Voting of the MADM
21
Y = 1.75 (0.55+0.65+0.55)
X = 1 .80 (0.95+0.85)
Experimental Study
• Datasets
• Each AM has access to 20%, 30%, or 40% of
the features (random vertical partitioning).
22
PDM with Hoeffding Trees
23
PDM with Naive Bayes
24
25
PDM Potential Applications
• Mobile ECG analysis
• Mobile social media analysis
• Mobile policing
26Source: YouTube videos
PDM Demonstration
YouTube video link:
http://www.youtube.com/watch?v=MOvlYxmttkE
27
Making News
28
Summary
• Pocket data mining has been the outcome of
earlier developments started in 2003.
• PDM is a mobile agent based framework for
distributed and mobile ad-hoc data stream
mining.
• PDM has proven its applicability experimentally
with Hoeffding trees and Naïve Bayes classifiers.
• Many potential applications can benefit from
PDM.
29
Main References
Gaber M. M., Data Stream Mining Using Granularity-based Approach, a book chapter in Foundations of
Computational Intelligence – Volume 6, (Eds.) Abraham A., Hassanien A., Carvalho A., and Snase V.,
Volume 206/2009, pp. 47-66, ISSN 1860-949X (Print) 1860-9503 (Online), ISBN 978-3-642-01090-3,
Springer Berlin/Heidelberg, Germany, 2009.
Gaber M. M., and Yu P. S., A Holistic Approach for Resource-aware Adaptive Data Stream Mining,
Journal of New Generation Computing, ISSN 0288-3635 (Print) 1882-7055 (Online), Volume 25, Number 1,
November, 2006, pp. 95-115, Ohmsha, Ltd., and Springer Verlag.
Haghighi P. D., Krishnaswamy S., Zaslavsky A., Gaber M. M., Reasoning About Context in Uncertain
Pervasive Computing Environments, Daniel Roggen, Clemens Lombriser, Gerhard Tröster, GerdKortuem,
Paul J. M. Havinga (Eds.): Smart Sensing and Context, Third European Conference, EuroSSC 2008, pp. 112-
125, Zurich, Switzerland, October 29-31, 2008. Proceedings. Lecture Notes in Computer Science 5279
Springer 2008, ISBN 978-3-540-88792-8.
Gaber M. M., Krishnaswamy S., Gillick B., Nicoloudis N., Liono J., AlTaiar H., Zaslavsky A., Adaptive
Clutter-Aware Visualization for Mobile Data Stream Mining, Proceedings of the IEEE 22nd
International Conference on Tools with Artificial Intelligence (ICTAI 2010), pp. 304-311,Arras, France, 27-29
October, 2010.
Stahl F., Gaber M. M., Aldridge P., May D., Liu H., Bramer M., and Yu P. S, Homogeneous and
Heterogeneous Distributed Classification for Pocket Data Mining, LNCS Transactions on Large-Scale
Data- and Knowledge-Centered Systems, Springer-Verlag, 2011.
More available at: http://gaberm.myweb.port.ac.uk/publications.htm
30
Our Books in the Area
31
Acknowledgement
• I would like to thank the following:
– Dr. Arkady Zaslavsky
– A/Prof.. Shonali Krishnaswamy
– Prof. Philip S. Yu
– Dr. Bernhard Scholz
– Dr. Uwe Roehm
– Dr. Suan Khai Chong
– Dr. Pari Delir Haghighi
– Duc Nhan Phung
– Iti Aggarwal
– Brett Gillick
– Osnat Horovitz
– Rahul Shah
– Dr Frederic Stahl
– Prof. Max Bramer
– Paul Aldridge
– Han Liu
– David May
– Oscar Campos
– Victor Mandujano
32
Thanks for listening
Dr Mohamed Medhat Gaber
School of Computing
Faculty of Technology
University of Portsmouth
E-mail: mohamed.gaber@port.ac.uk, mohamed.m.gaber@gmail.com
Web: http://gaberm.myweb.port.ac.uk/

More Related Content

What's hot

PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...
PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...
PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...
ijaia
 
IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016
tsysglobalsolutions
 
COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013
COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013 COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013
COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013
Charith Perera
 
AI for Science
AI for ScienceAI for Science
AI for Science
inside-BigData.com
 
6 g white paper on machine learning
6 g white paper on machine learning6 g white paper on machine learning
6 g white paper on machine learning
Mirza Baig
 
hopfield neural network-based fault location in wireless and optical networks...
hopfield neural network-based fault location in wireless and optical networks...hopfield neural network-based fault location in wireless and optical networks...
hopfield neural network-based fault location in wireless and optical networks...
Amir Shokri
 
A Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine PerceptionA Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine Perception
Cory Andrew Henson
 
IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...
IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...
IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...
ijcsit
 
2018 learning approach-digitaltrends
2018 learning approach-digitaltrends2018 learning approach-digitaltrends
2018 learning approach-digitaltrends
Abhilash Gopalakrishnan
 
WF-IOT-2014, Seoul, Korea, 06 March 2014
WF-IOT-2014, Seoul, Korea, 06 March 2014WF-IOT-2014, Seoul, Korea, 06 March 2014
WF-IOT-2014, Seoul, Korea, 06 March 2014
Charith Perera
 
Top downloaded article in academia 2020 - International Journal of Informatio...
Top downloaded article in academia 2020 - International Journal of Informatio...Top downloaded article in academia 2020 - International Journal of Informatio...
Top downloaded article in academia 2020 - International Journal of Informatio...
Zac Darcy
 
Multipleregression covidmobility and Covid-19 policy recommendation
Multipleregression covidmobility and Covid-19 policy recommendationMultipleregression covidmobility and Covid-19 policy recommendation
Multipleregression covidmobility and Covid-19 policy recommendation
Kan Yuenyong
 
IJET-V3I1P6
IJET-V3I1P6IJET-V3I1P6
PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...
PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...
PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...
AIRCC Publishing Corporation
 
A Knowledge-based Approach for Real-Time IoT Stream Annotation and Processing
A Knowledge-based Approach for Real-Time IoT Stream Annotation and ProcessingA Knowledge-based Approach for Real-Time IoT Stream Annotation and Processing
A Knowledge-based Approach for Real-Time IoT Stream Annotation and Processing
PayamBarnaghi
 
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Artificial Intelligence Institute at UofSC
 
How to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsHow to make data more usable on the Internet of Things
How to make data more usable on the Internet of Things
PayamBarnaghi
 
A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...
A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...
A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...
Paolo Missier
 

What's hot (18)

PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...
PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...
PREDICTIVE MAINTENANCE AND ENGINEERED PROCESSES IN MECHATRONIC INDUSTRY: AN I...
 
IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016
 
COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013
COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013 COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013
COLLABORATECOM-2013, Austin, Texas, United States, 20 October 2013
 
AI for Science
AI for ScienceAI for Science
AI for Science
 
6 g white paper on machine learning
6 g white paper on machine learning6 g white paper on machine learning
6 g white paper on machine learning
 
hopfield neural network-based fault location in wireless and optical networks...
hopfield neural network-based fault location in wireless and optical networks...hopfield neural network-based fault location in wireless and optical networks...
hopfield neural network-based fault location in wireless and optical networks...
 
A Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine PerceptionA Semantics-based Approach to Machine Perception
A Semantics-based Approach to Machine Perception
 
IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...
IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...
IMMERSIVE TECHNOLOGIES IN 5G-ENABLED APPLICATIONS: SOME TECHNICAL CHALLENGES ...
 
2018 learning approach-digitaltrends
2018 learning approach-digitaltrends2018 learning approach-digitaltrends
2018 learning approach-digitaltrends
 
WF-IOT-2014, Seoul, Korea, 06 March 2014
WF-IOT-2014, Seoul, Korea, 06 March 2014WF-IOT-2014, Seoul, Korea, 06 March 2014
WF-IOT-2014, Seoul, Korea, 06 March 2014
 
Top downloaded article in academia 2020 - International Journal of Informatio...
Top downloaded article in academia 2020 - International Journal of Informatio...Top downloaded article in academia 2020 - International Journal of Informatio...
Top downloaded article in academia 2020 - International Journal of Informatio...
 
Multipleregression covidmobility and Covid-19 policy recommendation
Multipleregression covidmobility and Covid-19 policy recommendationMultipleregression covidmobility and Covid-19 policy recommendation
Multipleregression covidmobility and Covid-19 policy recommendation
 
IJET-V3I1P6
IJET-V3I1P6IJET-V3I1P6
IJET-V3I1P6
 
PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...
PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...
PROVIDES AN APPROACH BASED ON ADAPTIVE FORWARDING AND LABEL SWITCHING TO IMPR...
 
A Knowledge-based Approach for Real-Time IoT Stream Annotation and Processing
A Knowledge-based Approach for Real-Time IoT Stream Annotation and ProcessingA Knowledge-based Approach for Real-Time IoT Stream Annotation and Processing
A Knowledge-based Approach for Real-Time IoT Stream Annotation and Processing
 
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
Data Processing and Semantics for Advanced Internet of Things (IoT) Applicati...
 
How to make data more usable on the Internet of Things
How to make data more usable on the Internet of ThingsHow to make data more usable on the Internet of Things
How to make data more usable on the Internet of Things
 
A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...
A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...
A Customisable Pipeline for Continuously Harvesting Socially-Minded Twitter U...
 

Similar to Pocket Data Mining: The Next Generation in Predictive Analytics

Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things
PayamBarnaghi
 
Big data analytics
Big data analyticsBig data analytics
Big data analytics
nitesh saxena
 
Requirements and Challenges of Smart Mobility for IoT
Requirements and Challenges of Smart Mobility for IoTRequirements and Challenges of Smart Mobility for IoT
Requirements and Challenges of Smart Mobility for IoT
Ana Aguiar
 
Big Data : Risks and Opportunities
Big Data : Risks and OpportunitiesBig Data : Risks and Opportunities
Big Data : Risks and Opportunities
Kenny Huang Ph.D.
 
System Support for Internet of Things
System Support for Internet of ThingsSystem Support for Internet of Things
System Support for Internet of Things
HarshitParkar6677
 
Arpan pal tac tics2012
Arpan pal tac tics2012Arpan pal tac tics2012
Arpan pal tac tics2012
Arpan Pal
 
Semantics in Sensor Networks
Semantics in Sensor NetworksSemantics in Sensor Networks
Semantics in Sensor Networks
Oscar Corcho
 
Cloud-based Data Stream Processing
Cloud-based Data Stream ProcessingCloud-based Data Stream Processing
Cloud-based Data Stream Processing
Zbigniew Jerzak
 
SPAR 2015 - Civil Maps Presentation by Sravan Puttagunta
SPAR 2015 - Civil Maps Presentation by Sravan PuttaguntaSPAR 2015 - Civil Maps Presentation by Sravan Puttagunta
SPAR 2015 - Civil Maps Presentation by Sravan Puttagunta
Sravan Puttagunta
 
Data Ingestion At Scale (CNECCS 2017)
Data Ingestion At Scale (CNECCS 2017)Data Ingestion At Scale (CNECCS 2017)
Data Ingestion At Scale (CNECCS 2017)
Jeffrey Sica
 
Unit 1 (Chapter-1) on data mining concepts.ppt
Unit 1 (Chapter-1) on data mining concepts.pptUnit 1 (Chapter-1) on data mining concepts.ppt
Unit 1 (Chapter-1) on data mining concepts.ppt
PadmajaLaksh
 
OpenDataSoft - Towards Cost-efficient Innovation with Data Open Platforms
OpenDataSoft - Towards Cost-efficient Innovation with Data Open PlatformsOpenDataSoft - Towards Cost-efficient Innovation with Data Open Platforms
OpenDataSoft - Towards Cost-efficient Innovation with Data Open Platforms
OpenDataSoft
 
New Trends and Directions in Data Science - MIT Information Quality Conferenc...
New Trends and Directions in Data Science - MIT Information Quality Conferenc...New Trends and Directions in Data Science - MIT Information Quality Conferenc...
New Trends and Directions in Data Science - MIT Information Quality Conferenc...
Mario Faria
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
PayamBarnaghi
 
Ukd2008 18-9-08 andrea
Ukd2008 18-9-08 andreaUkd2008 18-9-08 andrea
Ukd2008 18-9-08 andrea
Andrea Zaza
 
Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013
Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013
Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013
The Air Force Office of Scientific Research
 
LIDAR Magizine 2015: The Birth of 3D Mapping Artificial Intelligence
LIDAR Magizine 2015: The Birth of 3D Mapping Artificial IntelligenceLIDAR Magizine 2015: The Birth of 3D Mapping Artificial Intelligence
LIDAR Magizine 2015: The Birth of 3D Mapping Artificial Intelligence
Jason Creadore 🌐
 
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...
Edward Curry
 
Data Mining: Future Trends and Applications
Data Mining: Future Trends and ApplicationsData Mining: Future Trends and Applications
Data Mining: Future Trends and Applications
IJMER
 
Streaming Hypothesis Reasoning - William Smith, Jan 2016
Streaming Hypothesis Reasoning - William Smith, Jan 2016Streaming Hypothesis Reasoning - William Smith, Jan 2016
Streaming Hypothesis Reasoning - William Smith, Jan 2016
Seattle DAML meetup
 

Similar to Pocket Data Mining: The Next Generation in Predictive Analytics (20)

Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things Intelligent Data Processing for the Internet of Things
Intelligent Data Processing for the Internet of Things
 
Big data analytics
Big data analyticsBig data analytics
Big data analytics
 
Requirements and Challenges of Smart Mobility for IoT
Requirements and Challenges of Smart Mobility for IoTRequirements and Challenges of Smart Mobility for IoT
Requirements and Challenges of Smart Mobility for IoT
 
Big Data : Risks and Opportunities
Big Data : Risks and OpportunitiesBig Data : Risks and Opportunities
Big Data : Risks and Opportunities
 
System Support for Internet of Things
System Support for Internet of ThingsSystem Support for Internet of Things
System Support for Internet of Things
 
Arpan pal tac tics2012
Arpan pal tac tics2012Arpan pal tac tics2012
Arpan pal tac tics2012
 
Semantics in Sensor Networks
Semantics in Sensor NetworksSemantics in Sensor Networks
Semantics in Sensor Networks
 
Cloud-based Data Stream Processing
Cloud-based Data Stream ProcessingCloud-based Data Stream Processing
Cloud-based Data Stream Processing
 
SPAR 2015 - Civil Maps Presentation by Sravan Puttagunta
SPAR 2015 - Civil Maps Presentation by Sravan PuttaguntaSPAR 2015 - Civil Maps Presentation by Sravan Puttagunta
SPAR 2015 - Civil Maps Presentation by Sravan Puttagunta
 
Data Ingestion At Scale (CNECCS 2017)
Data Ingestion At Scale (CNECCS 2017)Data Ingestion At Scale (CNECCS 2017)
Data Ingestion At Scale (CNECCS 2017)
 
Unit 1 (Chapter-1) on data mining concepts.ppt
Unit 1 (Chapter-1) on data mining concepts.pptUnit 1 (Chapter-1) on data mining concepts.ppt
Unit 1 (Chapter-1) on data mining concepts.ppt
 
OpenDataSoft - Towards Cost-efficient Innovation with Data Open Platforms
OpenDataSoft - Towards Cost-efficient Innovation with Data Open PlatformsOpenDataSoft - Towards Cost-efficient Innovation with Data Open Platforms
OpenDataSoft - Towards Cost-efficient Innovation with Data Open Platforms
 
New Trends and Directions in Data Science - MIT Information Quality Conferenc...
New Trends and Directions in Data Science - MIT Information Quality Conferenc...New Trends and Directions in Data Science - MIT Information Quality Conferenc...
New Trends and Directions in Data Science - MIT Information Quality Conferenc...
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
 
Ukd2008 18-9-08 andrea
Ukd2008 18-9-08 andreaUkd2008 18-9-08 andrea
Ukd2008 18-9-08 andrea
 
Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013
Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013
Darema - Dynamic Data Driven Applications Systems (DDDAS) - Spring Review 2013
 
LIDAR Magizine 2015: The Birth of 3D Mapping Artificial Intelligence
LIDAR Magizine 2015: The Birth of 3D Mapping Artificial IntelligenceLIDAR Magizine 2015: The Birth of 3D Mapping Artificial Intelligence
LIDAR Magizine 2015: The Birth of 3D Mapping Artificial Intelligence
 
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...
From Data Platforms to Dataspaces: Enabling Data Ecosystems for Intelligent S...
 
Data Mining: Future Trends and Applications
Data Mining: Future Trends and ApplicationsData Mining: Future Trends and Applications
Data Mining: Future Trends and Applications
 
Streaming Hypothesis Reasoning - William Smith, Jan 2016
Streaming Hypothesis Reasoning - William Smith, Jan 2016Streaming Hypothesis Reasoning - William Smith, Jan 2016
Streaming Hypothesis Reasoning - William Smith, Jan 2016
 

Recently uploaded

一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Timothy Spann
 
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
dwreak4tg
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
Social Samosa
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
rwarrenll
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
v3tuleee
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
oz8q3jxlp
 
Influence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business PlanInfluence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business Plan
jerlynmaetalle
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
bopyb
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
g4dpvqap0
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
apvysm8
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
javier ramirez
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
g4dpvqap0
 
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
nuttdpt
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
Walaa Eldin Moustafa
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
Roger Valdez
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
sameer shah
 

Recently uploaded (20)

一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
 
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
一比一原版(BCU毕业证书)伯明翰城市大学毕业证如何办理
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
 
My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.My burning issue is homelessness K.C.M.O.
My burning issue is homelessness K.C.M.O.
 
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理一比一原版(UofS毕业证书)萨省大学毕业证如何办理
一比一原版(UofS毕业证书)萨省大学毕业证如何办理
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
 
Influence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business PlanInfluence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business Plan
 
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
一比一原版(GWU,GW文凭证书)乔治·华盛顿大学毕业证如何办理
 
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
一比一原版(Glasgow毕业证书)格拉斯哥大学毕业证如何办理
 
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
办(uts毕业证书)悉尼科技大学毕业证学历证书原版一模一样
 
The Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series DatabaseThe Building Blocks of QuestDB, a Time Series Database
The Building Blocks of QuestDB, a Time Series Database
 
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
一比一原版(爱大毕业证书)爱丁堡大学毕业证如何办理
 
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
一比一原版(UCSF文凭证书)旧金山分校毕业证如何办理
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
 

Pocket Data Mining: The Next Generation in Predictive Analytics

  • 1. Pocket Data Mining The Next Generation in Predictive Analytics Dr Mohamed Medhat Gaber School of Computing University of Portsmouth E-mail: mohamed.gaber@port.ac.uk
  • 2. Research Timeline • 2003 – 2006 – Adaptive resource-aware data stream mining approach and techniques – Algorithm Granularity (AG) • Algorithm Output Granularity (AOG) • Algorithm Input Granularity (AIG) • Algorithm Processing Granularity (APG) • 2007 – 2010 – Situation-aware data stream mining – Fuzzy Situation Inference (FSI) • 2008 – 2011 – Clutter-aware visualisation – Adaptive Clutter Reduction (ACR) • 2010 – 2012 – Distributed and collaborative mobile data stream mining – Pocket Data Mining (PDM) 2
  • 3. Agenda • Introduction to Data Streams • Earlier work – Granularity-based Approach – Situation-aware Data Stream Mining – Clutter-aware Visualisation • Pocket Data Mining – Background on Mobile Software Agents – PDM Architecture and Procedure – Hoeffding Tree Agent Miner – Naïve Bayes Agent Miner – Experimental Results • Summary 3
  • 4. Introduction to Data Streams • The advances in data acquisition hardware and the emergence of applications that process continuous flow of data records have led to the data stream phenomenon. • A data stream is a continuous, rapid flow of data that challenge our state-of-the-art processing and communication infrastructure. • The general features of data streams are: – Very high rate input data – Read only once by an algorithm – Real time processing demand – Unbounded – Time varying. 4
  • 5. Data Stream Processing in Resource- constrained Environments • A wide range of data streams are generated in or sent to resource- constrained computing environments. – Spacecrafts – Wireless sensor networks – PDAs and smartphones 5 Source: www.freeimages.co.uk
  • 6. Research Issues • Limited computational resources • Limited bandwidth • Limited screen realestate • Change of the user’s context 6
  • 7. Our Approach • Adaptability with regard to: – Computational resources – User’s situation – Visual clutter 7
  • 8. Granularity-based Approach • Combining the three possible granularity-based adaptation, namely: – AIG: Algorithm Input Granularity – AOG: Algorithm Output Granularity – APG: Algorithm Processing Granularity 8
  • 10. Situation Inferencing • Capture Application’s “Situation” • Fuzzy Context Spaces • Enhance probabilistic situation inferencing with fuzziness • Cope with changing situations • Cope with unknown situations 10
  • 11. Adaptive Clutter Reduction • Similar to resource-awareness and situation-awareness, we have developed a novel way to automatically reduce the clutter • The new approach has many important applications (especially in disaster management) 11
  • 12. 12 Adaptive Clutter Reduction 50% Coverage and 80% Overlap
  • 13. 13
  • 14. Open Mobile Miner - OMM 14
  • 15. PDM: Pocket Data Mining • Pocket Data Mining (PDM) is our new term describing collaborative mining of streaming data in mobile and distributed computing environments. • With continuous advances in computational power and communication abilities for smartphones and tablet computers; and • The sheer amounts of data streams that we subscribe to or acquire using the onboard sensing capabilities • There is an unprecedented opportunity to perform complex data analysis tasks that can benefit mobile users 15 Source: Lane, N.D.; Miluzzo, E.; Hong Lu; Peebles, D.; Choudhury, T.; Campbell, A.T.; , "A survey of mobile phone sensing," Communications Magazine, IEEE , vol.48, no.9, pp.140-150, Sept. 2010
  • 16. Technology Enablers • This can be realised with the help of several established areas of study including: – data stream mining; – mobile software agents; and – programming for small devices. 16
  • 17. What is a Mobile Agent ? 17  A software program  Moves from machine to machine under its own control  Suspends execution at any point in time, transport itself to a new machine and resume execution  Once created, a mobile agent autonomously decides which locations to visit and what instructions to perform  Continuous interaction with the agent’s originating source is not required  HOW?  Implicitly specified through the agent code  Specified through a run-time modifiable itinerary JADE Architecture
  • 18. PDM Architecture and Procedure 18
  • 19. PDM Agents • (Mobile) agent miners (AM): these agents are either distributed over the network when the mining task is initiated or are already located on the mobile device. • Mobile data stream mining • Mobile agent resource discoverers (MRD): these agents are used to explore the available computational resources, processing techniques, and data sources.  Mobile cloud • Mobile agent decision makers (MADM): these agents roam the network consulting the mobile agent miners to collaborate in reaching the final decision. • Ensemble learning 19 Source: http://www.datacenterknowledge.com Source: Polikar 2008
  • 20. Agent Miners (AMs) • We have used two stream classifiers, namely: – Hoeffding trees • Known for its statistically guaranteed accuracy – Incremental Naïve Bayes • Known for its computational efficiency and simplicity 20
  • 21. Simple Weighted Majority Voting of the MADM 21 Y = 1.75 (0.55+0.65+0.55) X = 1 .80 (0.95+0.85)
  • 22. Experimental Study • Datasets • Each AM has access to 20%, 30%, or 40% of the features (random vertical partitioning). 22
  • 23. PDM with Hoeffding Trees 23
  • 24. PDM with Naive Bayes 24
  • 25. 25
  • 26. PDM Potential Applications • Mobile ECG analysis • Mobile social media analysis • Mobile policing 26Source: YouTube videos
  • 27. PDM Demonstration YouTube video link: http://www.youtube.com/watch?v=MOvlYxmttkE 27
  • 29. Summary • Pocket data mining has been the outcome of earlier developments started in 2003. • PDM is a mobile agent based framework for distributed and mobile ad-hoc data stream mining. • PDM has proven its applicability experimentally with Hoeffding trees and Naïve Bayes classifiers. • Many potential applications can benefit from PDM. 29
  • 30. Main References Gaber M. M., Data Stream Mining Using Granularity-based Approach, a book chapter in Foundations of Computational Intelligence – Volume 6, (Eds.) Abraham A., Hassanien A., Carvalho A., and Snase V., Volume 206/2009, pp. 47-66, ISSN 1860-949X (Print) 1860-9503 (Online), ISBN 978-3-642-01090-3, Springer Berlin/Heidelberg, Germany, 2009. Gaber M. M., and Yu P. S., A Holistic Approach for Resource-aware Adaptive Data Stream Mining, Journal of New Generation Computing, ISSN 0288-3635 (Print) 1882-7055 (Online), Volume 25, Number 1, November, 2006, pp. 95-115, Ohmsha, Ltd., and Springer Verlag. Haghighi P. D., Krishnaswamy S., Zaslavsky A., Gaber M. M., Reasoning About Context in Uncertain Pervasive Computing Environments, Daniel Roggen, Clemens Lombriser, Gerhard Tröster, GerdKortuem, Paul J. M. Havinga (Eds.): Smart Sensing and Context, Third European Conference, EuroSSC 2008, pp. 112- 125, Zurich, Switzerland, October 29-31, 2008. Proceedings. Lecture Notes in Computer Science 5279 Springer 2008, ISBN 978-3-540-88792-8. Gaber M. M., Krishnaswamy S., Gillick B., Nicoloudis N., Liono J., AlTaiar H., Zaslavsky A., Adaptive Clutter-Aware Visualization for Mobile Data Stream Mining, Proceedings of the IEEE 22nd International Conference on Tools with Artificial Intelligence (ICTAI 2010), pp. 304-311,Arras, France, 27-29 October, 2010. Stahl F., Gaber M. M., Aldridge P., May D., Liu H., Bramer M., and Yu P. S, Homogeneous and Heterogeneous Distributed Classification for Pocket Data Mining, LNCS Transactions on Large-Scale Data- and Knowledge-Centered Systems, Springer-Verlag, 2011. More available at: http://gaberm.myweb.port.ac.uk/publications.htm 30
  • 31. Our Books in the Area 31
  • 32. Acknowledgement • I would like to thank the following: – Dr. Arkady Zaslavsky – A/Prof.. Shonali Krishnaswamy – Prof. Philip S. Yu – Dr. Bernhard Scholz – Dr. Uwe Roehm – Dr. Suan Khai Chong – Dr. Pari Delir Haghighi – Duc Nhan Phung – Iti Aggarwal – Brett Gillick – Osnat Horovitz – Rahul Shah – Dr Frederic Stahl – Prof. Max Bramer – Paul Aldridge – Han Liu – David May – Oscar Campos – Victor Mandujano 32
  • 33. Thanks for listening Dr Mohamed Medhat Gaber School of Computing Faculty of Technology University of Portsmouth E-mail: mohamed.gaber@port.ac.uk, mohamed.m.gaber@gmail.com Web: http://gaberm.myweb.port.ac.uk/