Keynote at the Workshop on Building Research Collaboration: Electricity Systems. Purdue University, West Lafayette, IN. Aug 28-29, 2013.
Abstract:
Big Data has captured much interest in research and industry, with anticipation of better decisions, efficient organizations, and many new jobs. Much of the emphasis is on technology that handles volume, including storage and computational techniques to support analysis (Hadoop, NoSQL, MapReduce, etc), and the challenges of the four Vs of Big Data: Volume, Variety, Velocity, and Veracity. However, the most important feature of data, the raison d'etre, is neither volume, variety, velocity, nor veracity -- but value. In this talk, I will emphasize the significance of Smart Data, and discuss how it is can be realized by extracting value from Big Data. Accomplishing this task requires organized ways to harness and overcome the original four V-challenges; and while the technologies currently touted may provide some necessary infrastructure-- they are far from sufficient. In particular, we will need to utilize metadata, employ semantics and intelligent processing, and leverage some of the extensive work that predates Big Data.
For achieving energy sustainability, Smart Grids are known to transform the way we generate, distribute, and consume power. Unprecedented amount of data is being collected from smart meters, smart devices, and sensors all throughout the power grid. I will discuss the central question of deriving Value from the entire smart grid data deluge by discussing novel algorithms and techniques such as Semantic Perception for dealing with Velocity, use of ontologies and vocabularies for dealing with Variety, and Continuous Semantics for dealing with Velocity. I will discuss scenarios that exemplify the process of deriving Value from Big Data in the context of Smart Grid.
Additional background is at: http://wiki.knoesis.org/index.php/Smart_Data
A previous version of this talk with more technical details but not focused on energy: http://j.mp/SmatData
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...Amit Sheth
Featured Keynote at Worldcomp'14, July 2014: http://www.world-academy-of-science.org/worldcomp14/ws/keynotes/keynote_sheth
Video of the talk at: http://youtu.be/2991W7OBLqU
Big Data has captured a lot of interest in industry, with the emphasis on the challenges of the four Vs of Big Data: Volume, Variety, Velocity, and Veracity, and their applications to drive value for businesses. Recently, there is rapid growth in situations where a big data challenge relates to making individually relevant decisions. A key example is human health, fitness, and well-being. Consider for instance, understanding the reasons for and avoiding an asthma attack based on Big Data in the form of personal health signals (e.g., physiological data measured by devices/sensors or Internet of Things around humans, on the humans, and inside/within the humans), public health signals (information coming from the healthcare system such as hospital admissions), and population health signals (such as Tweets by people related to asthma occurrences and allergens, Web services providing pollen and smog information, etc.). However, no individual has the ability to process all these data without the help of appropriate technology, and each human has different set of relevant data!
In this talk, I will forward the concept of Smart Data that is realized by extracting value from Big Data, to benefit not just large companies but each individual. If I am an asthma patient, for all the data relevant to me with the four V-challenges, what I care about is simply, “How is my current health, and what is the risk of having an asthma attack in my personal situation, especially if that risk has changed?” As I will show, Smart Data that gives such personalized and actionable information will need to utilize metadata, use domain specific knowledge, employ semantics and intelligent processing, and go beyond traditional reliance on ML and NLP.
For harnessing volume, I will discuss the concept of Semantic Perception, that is, how to convert massive amounts of data into information, meaning, and insight useful for human decision-making. For dealing with Variety, I will discuss experience in using agreement represented in the form of ontologies, domain models, or vocabularies, to support semantic interoperability and integration. For Velocity, I will discuss somewhat more recent work on Continuous Semantics, which seeks to use dynamically created models of new objects, concepts, and relationships, using them to better understand new cues in the data that capture rapidly evolving events and situations.
Smart Data applications in development at Kno.e.sis come from the domains of personalized health, energy, disaster response, and smart city. I will present examples from a couple of these.
Smart Data and real-world semantic web applications (2004)Amit Sheth
Probably the first recorded use of "smart data" for achieving the Semantic Web and for realizing productivity, efficiency, and effectiveness gains by using semantics to transform raw data into Smart Data.
2013 retake on this is discussed at: http://wiki.knoesis.org/index.php/Smart_Data
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...Amit Sheth
Keynote given at ICDE2014, April 2014. Details at: http://ieee-icde2014.eecs.northwestern.edu/keynotes.html
A video of a version of this talk is available here: http://youtu.be/8RhpFlfpJ-A
(download to see many hidden slides).
Two versions of this talk, targeted at Smart Energy and Personalized Digital Health domains/apps at: http://wiki.knoesis.org/index.php/Smart_Data
Previous (older) version replaced by this version: http://www.slideshare.net/apsheth/big-data-to-smart-data-keynote
Presentation at the AAAI 2013 Fall Symposium on Semantics for Big Data, Arlington, Virginia, November 15-17, 2013
Additional related material at: http://wiki.knoesis.org/index.php/Smart_Data
Related paper at: http://www.knoesis.org/library/resource.php?id=1903
Abstract: We discuss the nature of Big Data and address the role of semantics in analyzing and processing Big Data that arises in the context of Physical-Cyber-Social Systems. We organize our research around the five V's of Big Data, where four of the Vs are harnessed to produce the fifth V - value. To handle the challenge of Volume, we advocate semantic perception that can convert low-level observational data to higher-level abstractions more suitable for decision-making. To handle the challenge of Variety, we resort to the use of semantic models and annotations of data so that much of the intelligent processing can be done at a level independent of heterogeneity of data formats and media. To handle the challenge of Velocity, we seek to use continuous semantics capability to dynamically create event or situation specific models and recognize new concepts, entities and facts. To handle Veracity, we explore the formalization of trust models and approaches to glean trustworthiness. The above four Vs of Big Data are harnessed by the semantics-empowered analytics to derive Value for supporting practical applications transcending physical-cyber-social continuum.
Physical Cyber Social Computing: An early 21st century approach to Computing ...Amit Sheth
Keynote given at WiMS 2013 Conference, June 12-14 2013, Madrid, Spain. http://aida.ii.uam.es/wims13/keynotes.php
Video of this talk at: http://videolectures.net/wims2013_sheth_physical_cyber_social_computing/
More information at: More at: http://wiki.knoesis.org/index.php/PCS
and http://knoesis.org/projects/ssw/
Replacing earlier versions: http://www.slideshare.net/apsheth/physical-cyber-social-computing & http://www.slideshare.net/apsheth/semantics-empowered-physicalcybersocial-systems-for-earthcube
Abstract: The proper role of technology to improve human experience has been discussed by visionaries and scientists from the early days of computing and electronic communication. Technology now plays an increasingly important role in facilitating and improving personal and social activities and engagements, decision making, interaction with physical and social worlds, generating insights, and just about anything that an intelligent human seeks to do. I have used the term Computing for Human Experience (CHE) [1] to capture this essential role of technology in a human centric vision. CHE emphasizes the unobtrusive, supportive and assistive role of technology in improving human experience, so that technology “takes into account the human world and allows computers themselves to disappear in the background” (Mark Weiser [2]).
In this talk, I will portray physical-cyber-social (PCS) computing that takes ideas from, and goes significantly beyond, the current progress in cyber-physical systems, socio-technical systems and cyber-social systems to support CHE [3]. I will exemplify future PCS application scenarios in healthcare and traffic management that are supported by (a) a deeper and richer semantic interdependence and interplay between sensors and devices at physical layers, (b) rich technology mediated social interactions, and (c) the gathering and application of collective intelligence characterized by massive and contextually relevant background knowledge and advanced reasoning in order to bridge machine and human perceptions. I will share an example of PCS computing using semantic perception [4], which converts low-level, heterogeneous, multimodal and contextually relevant data into high-level abstractions that can provide insights and assist humans in making complex decisions. The key proposition is to explain that PCS computing will need to move away from traditional data processing to multi-tier computation along data-information-knowledge-wisdom dimension that supports reasoning to convert data into abstractions that humans are adept at using.
[1] A. Sheth, Computing for Human Experience
[2] M. Weiser, The Computer for 21st Century
[3] A. Sheth, Semantics empowered Cyber-Physical-Social Systems
[4] C. Henson, A. Sheth, K. Thirunarayan, Semantic Perception: Converting Sensory Observations to Abstractions
Semantics-empowered Smart City applications: today and tomorrowAmit Sheth
Citation:
Amit Sheth, "Semantics-empowered Smart City applications: today and tomorrow,” Keynote presented at the The 6th Workshop on Semantics for Smarter Cities (S4SC 2015), collocated with the 14th International Semantic Web Conference (ISWC2015), Bethlehem, PA, USA. Oct 11-12, 2015.
http://kat.ee.surrey.ac.uk/wssc/index.html
Abstract: There has been a massive growth in potentially relevant physical (sensor/IoT)- cyber (Web)- social data related to activities and operations of cities and citizens. As part of our participation in smart city projects, including the EU-funded CityPulse project, we have analyzed a large number of of use cases with inputs from city administrations and end users, and developed a few early applications. In this talk, I will present some exciting smart city applications possible today and venture to speculate on some future ones where Big Data technologies and semantic computing, including the use of domain knowledge, play a critical role.
Presented at SW2012 @ ISWC2012.
http://amitsheth.blogspot.com/2012/08/semantics-empowered-physical-cyber.html
This is an old version of this talk, for more recent information on this topic (eg talks, papers, events), see: http://wiki.knoesis.org/index.php/PCS
"Computing for Human Experience: Semantics empowered Cyber-Physical, Social and Ubiquitous Computing beyond the Web" Keynote at On the Move Federated Conferences, Crete, Greece, October 18, 2011.
http://www.onthemove-conferences.org/
Details: http://wiki.knoesis.org/index.php/Computi
Smart Data for you and me: Personalized and Actionable Physical Cyber Social ...Amit Sheth
Featured Keynote at Worldcomp'14, July 2014: http://www.world-academy-of-science.org/worldcomp14/ws/keynotes/keynote_sheth
Video of the talk at: http://youtu.be/2991W7OBLqU
Big Data has captured a lot of interest in industry, with the emphasis on the challenges of the four Vs of Big Data: Volume, Variety, Velocity, and Veracity, and their applications to drive value for businesses. Recently, there is rapid growth in situations where a big data challenge relates to making individually relevant decisions. A key example is human health, fitness, and well-being. Consider for instance, understanding the reasons for and avoiding an asthma attack based on Big Data in the form of personal health signals (e.g., physiological data measured by devices/sensors or Internet of Things around humans, on the humans, and inside/within the humans), public health signals (information coming from the healthcare system such as hospital admissions), and population health signals (such as Tweets by people related to asthma occurrences and allergens, Web services providing pollen and smog information, etc.). However, no individual has the ability to process all these data without the help of appropriate technology, and each human has different set of relevant data!
In this talk, I will forward the concept of Smart Data that is realized by extracting value from Big Data, to benefit not just large companies but each individual. If I am an asthma patient, for all the data relevant to me with the four V-challenges, what I care about is simply, “How is my current health, and what is the risk of having an asthma attack in my personal situation, especially if that risk has changed?” As I will show, Smart Data that gives such personalized and actionable information will need to utilize metadata, use domain specific knowledge, employ semantics and intelligent processing, and go beyond traditional reliance on ML and NLP.
For harnessing volume, I will discuss the concept of Semantic Perception, that is, how to convert massive amounts of data into information, meaning, and insight useful for human decision-making. For dealing with Variety, I will discuss experience in using agreement represented in the form of ontologies, domain models, or vocabularies, to support semantic interoperability and integration. For Velocity, I will discuss somewhat more recent work on Continuous Semantics, which seeks to use dynamically created models of new objects, concepts, and relationships, using them to better understand new cues in the data that capture rapidly evolving events and situations.
Smart Data applications in development at Kno.e.sis come from the domains of personalized health, energy, disaster response, and smart city. I will present examples from a couple of these.
Smart Data and real-world semantic web applications (2004)Amit Sheth
Probably the first recorded use of "smart data" for achieving the Semantic Web and for realizing productivity, efficiency, and effectiveness gains by using semantics to transform raw data into Smart Data.
2013 retake on this is discussed at: http://wiki.knoesis.org/index.php/Smart_Data
TRANSFORMING BIG DATA INTO SMART DATA: Deriving Value via Harnessing Volume, ...Amit Sheth
Keynote given at ICDE2014, April 2014. Details at: http://ieee-icde2014.eecs.northwestern.edu/keynotes.html
A video of a version of this talk is available here: http://youtu.be/8RhpFlfpJ-A
(download to see many hidden slides).
Two versions of this talk, targeted at Smart Energy and Personalized Digital Health domains/apps at: http://wiki.knoesis.org/index.php/Smart_Data
Previous (older) version replaced by this version: http://www.slideshare.net/apsheth/big-data-to-smart-data-keynote
Presentation at the AAAI 2013 Fall Symposium on Semantics for Big Data, Arlington, Virginia, November 15-17, 2013
Additional related material at: http://wiki.knoesis.org/index.php/Smart_Data
Related paper at: http://www.knoesis.org/library/resource.php?id=1903
Abstract: We discuss the nature of Big Data and address the role of semantics in analyzing and processing Big Data that arises in the context of Physical-Cyber-Social Systems. We organize our research around the five V's of Big Data, where four of the Vs are harnessed to produce the fifth V - value. To handle the challenge of Volume, we advocate semantic perception that can convert low-level observational data to higher-level abstractions more suitable for decision-making. To handle the challenge of Variety, we resort to the use of semantic models and annotations of data so that much of the intelligent processing can be done at a level independent of heterogeneity of data formats and media. To handle the challenge of Velocity, we seek to use continuous semantics capability to dynamically create event or situation specific models and recognize new concepts, entities and facts. To handle Veracity, we explore the formalization of trust models and approaches to glean trustworthiness. The above four Vs of Big Data are harnessed by the semantics-empowered analytics to derive Value for supporting practical applications transcending physical-cyber-social continuum.
Physical Cyber Social Computing: An early 21st century approach to Computing ...Amit Sheth
Keynote given at WiMS 2013 Conference, June 12-14 2013, Madrid, Spain. http://aida.ii.uam.es/wims13/keynotes.php
Video of this talk at: http://videolectures.net/wims2013_sheth_physical_cyber_social_computing/
More information at: More at: http://wiki.knoesis.org/index.php/PCS
and http://knoesis.org/projects/ssw/
Replacing earlier versions: http://www.slideshare.net/apsheth/physical-cyber-social-computing & http://www.slideshare.net/apsheth/semantics-empowered-physicalcybersocial-systems-for-earthcube
Abstract: The proper role of technology to improve human experience has been discussed by visionaries and scientists from the early days of computing and electronic communication. Technology now plays an increasingly important role in facilitating and improving personal and social activities and engagements, decision making, interaction with physical and social worlds, generating insights, and just about anything that an intelligent human seeks to do. I have used the term Computing for Human Experience (CHE) [1] to capture this essential role of technology in a human centric vision. CHE emphasizes the unobtrusive, supportive and assistive role of technology in improving human experience, so that technology “takes into account the human world and allows computers themselves to disappear in the background” (Mark Weiser [2]).
In this talk, I will portray physical-cyber-social (PCS) computing that takes ideas from, and goes significantly beyond, the current progress in cyber-physical systems, socio-technical systems and cyber-social systems to support CHE [3]. I will exemplify future PCS application scenarios in healthcare and traffic management that are supported by (a) a deeper and richer semantic interdependence and interplay between sensors and devices at physical layers, (b) rich technology mediated social interactions, and (c) the gathering and application of collective intelligence characterized by massive and contextually relevant background knowledge and advanced reasoning in order to bridge machine and human perceptions. I will share an example of PCS computing using semantic perception [4], which converts low-level, heterogeneous, multimodal and contextually relevant data into high-level abstractions that can provide insights and assist humans in making complex decisions. The key proposition is to explain that PCS computing will need to move away from traditional data processing to multi-tier computation along data-information-knowledge-wisdom dimension that supports reasoning to convert data into abstractions that humans are adept at using.
[1] A. Sheth, Computing for Human Experience
[2] M. Weiser, The Computer for 21st Century
[3] A. Sheth, Semantics empowered Cyber-Physical-Social Systems
[4] C. Henson, A. Sheth, K. Thirunarayan, Semantic Perception: Converting Sensory Observations to Abstractions
Semantics-empowered Smart City applications: today and tomorrowAmit Sheth
Citation:
Amit Sheth, "Semantics-empowered Smart City applications: today and tomorrow,” Keynote presented at the The 6th Workshop on Semantics for Smarter Cities (S4SC 2015), collocated with the 14th International Semantic Web Conference (ISWC2015), Bethlehem, PA, USA. Oct 11-12, 2015.
http://kat.ee.surrey.ac.uk/wssc/index.html
Abstract: There has been a massive growth in potentially relevant physical (sensor/IoT)- cyber (Web)- social data related to activities and operations of cities and citizens. As part of our participation in smart city projects, including the EU-funded CityPulse project, we have analyzed a large number of of use cases with inputs from city administrations and end users, and developed a few early applications. In this talk, I will present some exciting smart city applications possible today and venture to speculate on some future ones where Big Data technologies and semantic computing, including the use of domain knowledge, play a critical role.
Presented at SW2012 @ ISWC2012.
http://amitsheth.blogspot.com/2012/08/semantics-empowered-physical-cyber.html
This is an old version of this talk, for more recent information on this topic (eg talks, papers, events), see: http://wiki.knoesis.org/index.php/PCS
"Computing for Human Experience: Semantics empowered Cyber-Physical, Social and Ubiquitous Computing beyond the Web" Keynote at On the Move Federated Conferences, Crete, Greece, October 18, 2011.
http://www.onthemove-conferences.org/
Details: http://wiki.knoesis.org/index.php/Computi
Presented at the Panel on
Sensor, Data, Analytics and Integration in Advanced Manufacturing, at the Connected Manufacturing track of Bosch-USA organized "Leveraging Public-Private Partnerships for Regional Growth Summit". Panel statement: Sensors, data and analytics are the core of any smart manufacturing system. What are the main challenges to create actionable outputs, replicate systems and scale efficiency gains across industries?
Moderator: Thomas Stiedl, Bosch
Panelists:
1. Amit Sheth, Wright State University
2. Howie Choset, Carnegie Melon University
3. Nagi Gebraeel, Georgia Institute of Technology
4. Brian Anthony, Massachusetts Institute of Technology
5. Yarom Polosky, Oak Ridget National Laboratory
For in-depth look:
Smart IoT: IoT as a human agent, human extension, and human complement
http://amitsheth.blogspot.com/2015/03/smart-iot-iot-as-human-agent-human.html
Semantic Gateway: http://knoesis.org/library/resource.php?id=2154
SSN Ontology: http://knoesis.org/library/resource.php?id=1659
Applications of Multimodal Physical (IoT), Cyber and Social Data for Reliable and Actionable Insights: http://knoesis.org/library/resource.php?id=2018
Smart Data: Transforming Big Data into Smart Data...: http://wiki.knoesis.org/index.php/Smart_Data
Historic use of the term Smart Data (2004): http://www.scribd.com/doc/186588820
This tutorial presents tools and techniques for effectively utilizing the Internet of Things (IoT) for building advanced applications, including the Physical-Cyber-Social (PCS) systems. The issues and challenges related to IoT, semantic data modelling, annotation, knowledge representation (e.g. modelling for constrained environments, complexity issues and time/location dependency of data), integration, analy- sis, and reasoning will be discussed. The tutorial will de- scribe recent developments on creating annotation models and semantic description frameworks for IoT data (e.g. such as W3C Semantic Sensor Network ontology). A review of enabling technologies and common scenarios for IoT applications from the data and knowledge engineering point of view will be discussed. Information processing, reasoning, and knowledge extraction, along with existing solutions re- lated to these topics will be presented. The tutorial summarizes state-of-the-art research and developments on PCS systems, IoT related ontology development, linked data, do- main knowledge integration and management, querying large- scale IoT data, and AI applications for automated knowledge extraction from real world data.
Related: Semantic Sensor Web: http://knoesis.org/projects/ssw
Physical-Cyber-Social Computing: http://wiki.knoesis.org/index.php/PCS
Knowledge Will Propel Machine Understanding of Big DataAmit Sheth
Preview video: https://youtu.be/4e0dtV7CTWM
CCKS Keynote, August 2017: http://www.ccks2017.com/?page_id=358
SEAS Summer School, July 2017
https://sites.google.com/view/seasschool2017/talks
Related paper: http://knoesis.org/node/2835
CCKS Conf had over 500 attendees- some photos: https://photos.app.goo.gl/5CdlfAX1uYwvgqsQ2
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...Amit Sheth
Keynote at Web Intelligence 2017: http://webintelligence2017.com/program/keynotes/
Video: https://youtu.be/EIbhcqakgvA Paper: http://knoesis.org/node/2698
Abstract: While Bill Gates, Stephen Hawking, Elon Musk, Peter Thiel, and others engage in OpenAI discussions of whether or not AI, robots, and machines will replace humans, proponents of human-centric computing continue to extend work in which humans and machine partner in contextualized and personalized processing of multimodal data to derive actionable information.
In this talk, we discuss how maturing towards the emerging paradigms of semantic computing (SC), cognitive computing (CC), and perceptual computing (PC) provides a continuum through which to exploit the ever-increasing and growing diversity of data that could enhance people’s daily lives. SC and CC sift through raw data to personalize it according to context and individual users, creating abstractions that move the data closer to what humans can readily understand and apply in decision-making. PC, which interacts with the surrounding environment to collect data that is relevant and useful in understanding the outside world, is characterized by interpretative and exploratory activities that are supported by the use of prior/background knowledge. Using the examples of personalized digital health and a smart city, we will demonstrate how the trio of these computing paradigms form complementary capabilities that will enable the development of the next generation of intelligent systems. For background: http://bit.ly/PCSComputing
Smart Data - How you and I will exploit Big Data for personalized digital hea...Amit Sheth
Amit Sheth's keynote at IEEE BigData 2014, Oct 29, 2014.
Abstract from:
http://cci.drexel.edu/bigdata/bigdata2014/keynotespeech.htm
Big Data has captured a lot of interest in industry, with the emphasis on the challenges of the four Vs of Big Data: Volume, Variety, Velocity, and Veracity, and their applications to drive value for businesses. Recently, there is rapid growth in situations where a big data challenge relates to making individually relevant decisions. A key example is personalized digital health that related to taking better decisions about our health, fitness, and well-being. Consider for instance, understanding the reasons for and avoiding an asthma attack based on Big Data in the form of personal health signals (e.g., physiological data measured by devices/sensors or Internet of Things around humans, on the humans, and inside/within the humans), public health signals (e.g., information coming from the healthcare system such as hospital admissions), and population health signals (such as Tweets by people related to asthma occurrences and allergens, Web services providing pollen and smog information). However, no individual has the ability to process all these data without the help of appropriate technology, and each human has different set of relevant data!
In this talk, I will describe Smart Data that is realized by extracting value from Big Data, to benefit not just large companies but each individual. If my child is an asthma patient, for all the data relevant to my child with the four V-challenges, what I care about is simply, “How is her current health, and what are the risk of having an asthma attack in her current situation (now and today), especially if that risk has changed?” As I will show, Smart Data that gives such personalized and actionable information will need to utilize metadata, use domain specific knowledge, employ semantics and intelligent processing, and go beyond traditional reliance on ML and NLP. I will motivate the need for a synergistic combination of techniques similar to the close interworking of the top brain and the bottom brain in the cognitive models.
For harnessing volume, I will discuss the concept of Semantic Perception, that is, how to convert massive amounts of data into information, meaning, and insight useful for human decision-making. For dealing with Variety, I will discuss experience in using agreement represented in the form of ontologies, domain models, or vocabularies, to support semantic interoperability and integration. For Velocity, I will discuss somewhat more recent work on Continuous Semantics, which seeks to use dynamically created models of new objects, concepts, and relationships, using them to better understand new cues in the data that capture rapidly evolving events and situations.
Smart Data applications in development at Kno.e.sis come from the domains of personalized health, energy, disaster response, and smart city.
Data privacy and security in ICT4D - Meeting Report UN Global Pulse
On May 8th, 2015 UN Global Pulse hosted a workshop on data privacy and security in technology-enabled development projects and programmes, as part of a series of events about the Nine Principles for Digital Development. This report summarizes the presentations and discussions from the workshop. http://unglobalpulse.org/blog/improving-privacy-and-data-security-ict4d-projects
Crowdsourcing Approaches for Smart City Open Data ManagementEdward Curry
A wide-scale bottom-up approach to the creation and management of open data has been demonstrated by projects like Freebase, Wikipedia, and DBpedia. This talk explores how to involving a wide community of users in collaborative management of open data activities within a Smart City. The talk discusses how crowdsourcing techniques can be applied within a Smart City context using crowdsourcing and human computation platforms such as Amazon Mechanical Turk, Mobile Works, and Crowd Flower.
GIM encompasses the management, leadership, structures and practices required for the successful operation of GIS within an entity, nationally, regionally or globally.
Multipleregression covidmobility and Covid-19 policy recommendationKan Yuenyong
Multiple Regression Analysis and Covid-19 policy is the contemporary agenda. It demonstrates how to use Python to do data wrangler, to use R to do statistical analysis, and is enable to publish in standard academic journal. The model will explain whether lockdown policy is relevant to control Covid-19 outbreak? It cinc
A key contemporary trend emerging in big data science is the Quantified Self (QS) - individuals engaged in the deliberate self-tracking of any kind of biological, physical, behavioral, or transactional information as n=1 individuals or in groups. This is giving rise to interesting pools of individual data, group data, and big data which can be interlinked to create a new era of highly-targeted value-specific consumer applications. There are significant opportunities in big data to develop models to support QS data collection, integration, analysis, and use for personal lifestyle and consumption management. There are also opportunities to provide leadership in designing consumer-friendly standards and etiquette regarding the use of personal and collective data. Next-generation QS big data applications and services could include tools for rendering QS data meaningful in behavior change, establishing baselines and variability in objective metrics, applying new kinds of pattern recognition techniques, and aggregating multiple self-tracking data streams from wearable electronics, biosensors, mobile phones, genomic data, and cloud-based services. Potential limitations regarding QS activity need to be considered including consumer non-adoption, data privacy and sharing concerns, the digital divide, ease-of-use, and social acceptance.
Knowledge Graphs and their central role in big data processing: Past, Present...Amit Sheth
Keynote at CODS-COMAD 2020, Hyderabad, India, 06 Jan 2020: https://cods-comad.in/keynotes.html
Abstract : Early use of knowledge graphs, before the start of this century, related to building a knowledge graph manually or semi-automatically and applying them for semantic applications, such as search, browsing, personalization, and advertisement. Taalee/Semagix Semantic Search in 2000 had a KG that covered many domains and supported search with an equivalent of today’s infobox. Along with the growth of big data, machine learning became the preferred technique for searching, analyzing and deriving insights from such data. We observed the complementary nature of bottom-up (machine learning-driven) and top-down (semantic, knowledge graph and planning based) techniques. Recently we have seen growing efforts involving the shallow use of a knowledge graph to improve the semantic and conceptual processing of data. The future promises deeper and congruent incorporation or integration of the knowledge graphs in the learning techniques (which we call knowledge-infused learning), where knowledge graphs combining statistical AI (bottom-up) and symbolic AI learning techniques (top-down) play a critical role in hybrid and integrated intelligent systems. Throughout this talk, we will provide real-world examples, products, and applications where the knowledge graph played a pivotal role.
Big Data for Development and Humanitarian Action: Towards Responsible Governa...UN Global Pulse
This report presents a summary of the main topics discussed by the PAG in general, which were mainly summarized during the
2015 PAG meeting. It also describes some of the outcomes that came out of the PAG meeting of 23-24 October 2015.
Presented at the Panel on
Sensor, Data, Analytics and Integration in Advanced Manufacturing, at the Connected Manufacturing track of Bosch-USA organized "Leveraging Public-Private Partnerships for Regional Growth Summit". Panel statement: Sensors, data and analytics are the core of any smart manufacturing system. What are the main challenges to create actionable outputs, replicate systems and scale efficiency gains across industries?
Moderator: Thomas Stiedl, Bosch
Panelists:
1. Amit Sheth, Wright State University
2. Howie Choset, Carnegie Melon University
3. Nagi Gebraeel, Georgia Institute of Technology
4. Brian Anthony, Massachusetts Institute of Technology
5. Yarom Polosky, Oak Ridget National Laboratory
For in-depth look:
Smart IoT: IoT as a human agent, human extension, and human complement
http://amitsheth.blogspot.com/2015/03/smart-iot-iot-as-human-agent-human.html
Semantic Gateway: http://knoesis.org/library/resource.php?id=2154
SSN Ontology: http://knoesis.org/library/resource.php?id=1659
Applications of Multimodal Physical (IoT), Cyber and Social Data for Reliable and Actionable Insights: http://knoesis.org/library/resource.php?id=2018
Smart Data: Transforming Big Data into Smart Data...: http://wiki.knoesis.org/index.php/Smart_Data
Historic use of the term Smart Data (2004): http://www.scribd.com/doc/186588820
This tutorial presents tools and techniques for effectively utilizing the Internet of Things (IoT) for building advanced applications, including the Physical-Cyber-Social (PCS) systems. The issues and challenges related to IoT, semantic data modelling, annotation, knowledge representation (e.g. modelling for constrained environments, complexity issues and time/location dependency of data), integration, analy- sis, and reasoning will be discussed. The tutorial will de- scribe recent developments on creating annotation models and semantic description frameworks for IoT data (e.g. such as W3C Semantic Sensor Network ontology). A review of enabling technologies and common scenarios for IoT applications from the data and knowledge engineering point of view will be discussed. Information processing, reasoning, and knowledge extraction, along with existing solutions re- lated to these topics will be presented. The tutorial summarizes state-of-the-art research and developments on PCS systems, IoT related ontology development, linked data, do- main knowledge integration and management, querying large- scale IoT data, and AI applications for automated knowledge extraction from real world data.
Related: Semantic Sensor Web: http://knoesis.org/projects/ssw
Physical-Cyber-Social Computing: http://wiki.knoesis.org/index.php/PCS
Knowledge Will Propel Machine Understanding of Big DataAmit Sheth
Preview video: https://youtu.be/4e0dtV7CTWM
CCKS Keynote, August 2017: http://www.ccks2017.com/?page_id=358
SEAS Summer School, July 2017
https://sites.google.com/view/seasschool2017/talks
Related paper: http://knoesis.org/node/2835
CCKS Conf had over 500 attendees- some photos: https://photos.app.goo.gl/5CdlfAX1uYwvgqsQ2
Semantic, Cognitive, and Perceptual Computing – three intertwined strands of ...Amit Sheth
Keynote at Web Intelligence 2017: http://webintelligence2017.com/program/keynotes/
Video: https://youtu.be/EIbhcqakgvA Paper: http://knoesis.org/node/2698
Abstract: While Bill Gates, Stephen Hawking, Elon Musk, Peter Thiel, and others engage in OpenAI discussions of whether or not AI, robots, and machines will replace humans, proponents of human-centric computing continue to extend work in which humans and machine partner in contextualized and personalized processing of multimodal data to derive actionable information.
In this talk, we discuss how maturing towards the emerging paradigms of semantic computing (SC), cognitive computing (CC), and perceptual computing (PC) provides a continuum through which to exploit the ever-increasing and growing diversity of data that could enhance people’s daily lives. SC and CC sift through raw data to personalize it according to context and individual users, creating abstractions that move the data closer to what humans can readily understand and apply in decision-making. PC, which interacts with the surrounding environment to collect data that is relevant and useful in understanding the outside world, is characterized by interpretative and exploratory activities that are supported by the use of prior/background knowledge. Using the examples of personalized digital health and a smart city, we will demonstrate how the trio of these computing paradigms form complementary capabilities that will enable the development of the next generation of intelligent systems. For background: http://bit.ly/PCSComputing
Smart Data - How you and I will exploit Big Data for personalized digital hea...Amit Sheth
Amit Sheth's keynote at IEEE BigData 2014, Oct 29, 2014.
Abstract from:
http://cci.drexel.edu/bigdata/bigdata2014/keynotespeech.htm
Big Data has captured a lot of interest in industry, with the emphasis on the challenges of the four Vs of Big Data: Volume, Variety, Velocity, and Veracity, and their applications to drive value for businesses. Recently, there is rapid growth in situations where a big data challenge relates to making individually relevant decisions. A key example is personalized digital health that related to taking better decisions about our health, fitness, and well-being. Consider for instance, understanding the reasons for and avoiding an asthma attack based on Big Data in the form of personal health signals (e.g., physiological data measured by devices/sensors or Internet of Things around humans, on the humans, and inside/within the humans), public health signals (e.g., information coming from the healthcare system such as hospital admissions), and population health signals (such as Tweets by people related to asthma occurrences and allergens, Web services providing pollen and smog information). However, no individual has the ability to process all these data without the help of appropriate technology, and each human has different set of relevant data!
In this talk, I will describe Smart Data that is realized by extracting value from Big Data, to benefit not just large companies but each individual. If my child is an asthma patient, for all the data relevant to my child with the four V-challenges, what I care about is simply, “How is her current health, and what are the risk of having an asthma attack in her current situation (now and today), especially if that risk has changed?” As I will show, Smart Data that gives such personalized and actionable information will need to utilize metadata, use domain specific knowledge, employ semantics and intelligent processing, and go beyond traditional reliance on ML and NLP. I will motivate the need for a synergistic combination of techniques similar to the close interworking of the top brain and the bottom brain in the cognitive models.
For harnessing volume, I will discuss the concept of Semantic Perception, that is, how to convert massive amounts of data into information, meaning, and insight useful for human decision-making. For dealing with Variety, I will discuss experience in using agreement represented in the form of ontologies, domain models, or vocabularies, to support semantic interoperability and integration. For Velocity, I will discuss somewhat more recent work on Continuous Semantics, which seeks to use dynamically created models of new objects, concepts, and relationships, using them to better understand new cues in the data that capture rapidly evolving events and situations.
Smart Data applications in development at Kno.e.sis come from the domains of personalized health, energy, disaster response, and smart city.
Data privacy and security in ICT4D - Meeting Report UN Global Pulse
On May 8th, 2015 UN Global Pulse hosted a workshop on data privacy and security in technology-enabled development projects and programmes, as part of a series of events about the Nine Principles for Digital Development. This report summarizes the presentations and discussions from the workshop. http://unglobalpulse.org/blog/improving-privacy-and-data-security-ict4d-projects
Crowdsourcing Approaches for Smart City Open Data ManagementEdward Curry
A wide-scale bottom-up approach to the creation and management of open data has been demonstrated by projects like Freebase, Wikipedia, and DBpedia. This talk explores how to involving a wide community of users in collaborative management of open data activities within a Smart City. The talk discusses how crowdsourcing techniques can be applied within a Smart City context using crowdsourcing and human computation platforms such as Amazon Mechanical Turk, Mobile Works, and Crowd Flower.
GIM encompasses the management, leadership, structures and practices required for the successful operation of GIS within an entity, nationally, regionally or globally.
Multipleregression covidmobility and Covid-19 policy recommendationKan Yuenyong
Multiple Regression Analysis and Covid-19 policy is the contemporary agenda. It demonstrates how to use Python to do data wrangler, to use R to do statistical analysis, and is enable to publish in standard academic journal. The model will explain whether lockdown policy is relevant to control Covid-19 outbreak? It cinc
A key contemporary trend emerging in big data science is the Quantified Self (QS) - individuals engaged in the deliberate self-tracking of any kind of biological, physical, behavioral, or transactional information as n=1 individuals or in groups. This is giving rise to interesting pools of individual data, group data, and big data which can be interlinked to create a new era of highly-targeted value-specific consumer applications. There are significant opportunities in big data to develop models to support QS data collection, integration, analysis, and use for personal lifestyle and consumption management. There are also opportunities to provide leadership in designing consumer-friendly standards and etiquette regarding the use of personal and collective data. Next-generation QS big data applications and services could include tools for rendering QS data meaningful in behavior change, establishing baselines and variability in objective metrics, applying new kinds of pattern recognition techniques, and aggregating multiple self-tracking data streams from wearable electronics, biosensors, mobile phones, genomic data, and cloud-based services. Potential limitations regarding QS activity need to be considered including consumer non-adoption, data privacy and sharing concerns, the digital divide, ease-of-use, and social acceptance.
Knowledge Graphs and their central role in big data processing: Past, Present...Amit Sheth
Keynote at CODS-COMAD 2020, Hyderabad, India, 06 Jan 2020: https://cods-comad.in/keynotes.html
Abstract : Early use of knowledge graphs, before the start of this century, related to building a knowledge graph manually or semi-automatically and applying them for semantic applications, such as search, browsing, personalization, and advertisement. Taalee/Semagix Semantic Search in 2000 had a KG that covered many domains and supported search with an equivalent of today’s infobox. Along with the growth of big data, machine learning became the preferred technique for searching, analyzing and deriving insights from such data. We observed the complementary nature of bottom-up (machine learning-driven) and top-down (semantic, knowledge graph and planning based) techniques. Recently we have seen growing efforts involving the shallow use of a knowledge graph to improve the semantic and conceptual processing of data. The future promises deeper and congruent incorporation or integration of the knowledge graphs in the learning techniques (which we call knowledge-infused learning), where knowledge graphs combining statistical AI (bottom-up) and symbolic AI learning techniques (top-down) play a critical role in hybrid and integrated intelligent systems. Throughout this talk, we will provide real-world examples, products, and applications where the knowledge graph played a pivotal role.
Big Data for Development and Humanitarian Action: Towards Responsible Governa...UN Global Pulse
This report presents a summary of the main topics discussed by the PAG in general, which were mainly summarized during the
2015 PAG meeting. It also describes some of the outcomes that came out of the PAG meeting of 23-24 October 2015.
Data Science Innovations : Democratisation of Data and Data Science suresh sood
Data Science Innovations : Democratisation of Data and Data Science covers the opportunity of citizen data science lying at the convergence of natural language generation and discoveries in data made by the professions, not data scientists.
BIMCV, Banco de Imagen Medica de la Comunidad Valenciana. María de la IglesiaMaria de la Iglesia
Según Hal Varian (experto en microeconomía y economía de la información y, desde el año 2002, Chief Economist de Google) “En los próximos años, el trabajo más atractivo será el de los estadísticos: La capacidad de recoger datos, comprenderlos, procesarlos, extraer su valor, visualizarlos, comunicarlos serán todas habilidades importantes en las próximas décadas. Ahora disponemos de datos gratuitos y omnipresentes. Lo que aún falta es la capacidad de comprender estos datos“.
Big Data for the Social Sciences - David De Roure - Jisc Digital Festival 2014Jisc
The analysis of government data, data held by business, the web, social science survey data will support new research directions and findings. Big Data is one of David Willetts’ 8 great technologies, and in order to secure the UK’s competitive advantage new investments have been made by the Economic Social Science Research Council ( ESRC) in Big Data, for example the Business Datasafe and Understanding Populations investments. In this session the benefits of the use of Big Data in social science , and the ESRCs Big Data strategy will be explained by Professor David De Roure.of the Oxford e-Research Centre and advisor to the ESRC.
Learning Objective: Discuss the upcoming trends of information technology
This seminar looks at the forefront of technology trends in the community for technology leaders. As a technology professional, staying on top of trends is crucial. Below is a list of technology topics that this seminar will cover.
1. Emergence of the Mobile Cloud
Mobile distributed computing paradigm will lead to explosion of new services.
2. From Internet of Things to Web of Things
Need connectivity, internetworking to link physical and digital.
3. From Big Data to Extreme Data
Simpler analytics tools needed to leverage the data deluge.
4. The Revolution Will Be 3D
New tools; techniques bring 3D printing power to masses.
5. Supporting New Learning Styles
Online courses demand seamless, ubiquitous approach.
6. Next-generation mobile networks
Mobile infrastructure must catch up with user needs.
7. Balancing Identity and Privacy
Growing risks and concerns about social networks.
8. Smart and Connected Healthcare
Intelligent systems, assistive devices will improve health.
9. E-Government
Interoperability a big challenge to delivering information.
10. Scientific Cloud Computing
Key to solving grand challenges, pursuing breakthroughs.
At the end of this seminar, participants will be able to:
a. Explore the multiple uses of the internet.
b. Identify ways that technology can make our society more productive.
c. Examine what we give up when we advance technologically.
Keynote talk for NCRM Stream Analytics workshop, 19 January 2017, Manchester.
My talk is called "New and Emerging Forms of Data: Past, Present, and Future” and I will be giving a perspective from my role as one of the ESRC Strategic Advisers for Data Resources, in which I was responsible for new and emerging forms of data and realtime analytics. The talk also includes some of the current work in the Oxford e-Research Centre on Social Machines (the SOCIAM project) and an introduction to the PETRAS Internet of Things project.
The talk raises a number of important issues looking ahead, including massive scale of data that is already being supplied by Internet of Things, the implications of automation in our research, reproducibility and confidence in research results. I will also ask, how can the new forms of data and new research methods enable social scientists to work in new ways, and can we move on from the dependence on the traditional investment in longitudinal studies?
Over the past decade, cloud computing has acted as a disrupter in several areas of IT business. Soon, it will overhaul one area of technology that has been in rapid growth itself: Data Analytics. Nicky will focus on the recent study of IBM Institute of Business Value which shows that capabilities that enable an organization to consume data faster – to move from raw data to insight-driven actions – are now the key differentiator to creating value using data and analytics. He will also talk about the requirements for the underlying infrastructure as critical component allowing real-time crunching and analysis of high volume of data. Based on real cases like retailers and energy companies, we will look at five predictions in five years, based on:
Analytics, Big data, and Cloud coming together will energize the Speed Advantage.
The Internet of Things, or the IoT is a vision for a ubiquitous society wherein people and “Things” are connected in an immersively networked computing environment, with the connected “Things” providing utility to people/enterprises and their digital shadows, through intelligent social and commercial services. However, translating this idea to a conceivable reality is a work in progress for close to two decades; mostly, due to assumptions favoured more towards a “Things”-centric rather than a “Human”-centric approach coupled with the evolution/deployment ecosystem of IoT technologies.
Estimates on the spread and economic impact of IoT over the next few years are in the neighborhood of 50 billion or more connected “Things” with a market exceeding $350 billion through smarter cities and infrastructure, intelligent appliances, and healthier lifestyles. While many of these potential benefits from IoT are real and achievable, the road to accomplish these may need an rethink.
In the last few years, there has been a realization that an effective architecture for IoT (particularly, for emerging nations with limited technology penetration at the national scale) that is both affordable and sustainable should be based on tangible technology advances in the present, ubiquitous capabilities of the present/future, and practical application scenarios of social and entrepreneurial value. Hence, there is a revitalized interest to rethink the above assumptions, and this exercise has led to a more plausible set of scenarios wherein humans along with data, communication and devices play key roles.
In this presentation, an attempt is made to disaggregate these core problems; and offer a trajectory with a set of design paradigms for a renewed IoT ecosystem.
The Philosophy of Big Data is the branch of philosophy concerned with the foundations, methods, and implications of big data; the definitions, meaning, conceptualization, knowledge possibilities, truth standards, and practices in situations involving very-large data sets that are big in volume, velocity, variety, veracity, and variability
Similar to Transforming Big Data into Smart Data for Smart Energy: Deriving Value via harnessing Volume, Variety and Velocity (20)
Operation “Blue Star” is the only event in the history of Independent India where the state went into war with its own people. Even after about 40 years it is not clear if it was culmination of states anger over people of the region, a political game of power or start of dictatorial chapter in the democratic setup.
The people of Punjab felt alienated from main stream due to denial of their just demands during a long democratic struggle since independence. As it happen all over the word, it led to militant struggle with great loss of lives of military, police and civilian personnel. Killing of Indira Gandhi and massacre of innocent Sikhs in Delhi and other India cities was also associated with this movement.
Synthetic Fiber Construction in lab .pptxPavel ( NSTU)
Synthetic fiber production is a fascinating and complex field that blends chemistry, engineering, and environmental science. By understanding these aspects, students can gain a comprehensive view of synthetic fiber production, its impact on society and the environment, and the potential for future innovations. Synthetic fibers play a crucial role in modern society, impacting various aspects of daily life, industry, and the environment. ynthetic fibers are integral to modern life, offering a range of benefits from cost-effectiveness and versatility to innovative applications and performance characteristics. While they pose environmental challenges, ongoing research and development aim to create more sustainable and eco-friendly alternatives. Understanding the importance of synthetic fibers helps in appreciating their role in the economy, industry, and daily life, while also emphasizing the need for sustainable practices and innovation.
Honest Reviews of Tim Han LMA Course Program.pptxtimhan337
Personal development courses are widely available today, with each one promising life-changing outcomes. Tim Han’s Life Mastery Achievers (LMA) Course has drawn a lot of interest. In addition to offering my frank assessment of Success Insider’s LMA Course, this piece examines the course’s effects via a variety of Tim Han LMA course reviews and Success Insider comments.
Unit 8 - Information and Communication Technology (Paper I).pdfThiyagu K
This slides describes the basic concepts of ICT, basics of Email, Emerging Technology and Digital Initiatives in Education. This presentations aligns with the UGC Paper I syllabus.
Acetabularia Information For Class 9 .docxvaibhavrinwa19
Acetabularia acetabulum is a single-celled green alga that in its vegetative state is morphologically differentiated into a basal rhizoid and an axially elongated stalk, which bears whorls of branching hairs. The single diploid nucleus resides in the rhizoid.
Welcome to TechSoup New Member Orientation and Q&A (May 2024).pdfTechSoup
In this webinar you will learn how your organization can access TechSoup's wide variety of product discount and donation programs. From hardware to software, we'll give you a tour of the tools available to help your nonprofit with productivity, collaboration, financial management, donor tracking, security, and more.
June 3, 2024 Anti-Semitism Letter Sent to MIT President Kornbluth and MIT Cor...Levi Shapiro
Letter from the Congress of the United States regarding Anti-Semitism sent June 3rd to MIT President Sally Kornbluth, MIT Corp Chair, Mark Gorenberg
Dear Dr. Kornbluth and Mr. Gorenberg,
The US House of Representatives is deeply concerned by ongoing and pervasive acts of antisemitic
harassment and intimidation at the Massachusetts Institute of Technology (MIT). Failing to act decisively to ensure a safe learning environment for all students would be a grave dereliction of your responsibilities as President of MIT and Chair of the MIT Corporation.
This Congress will not stand idly by and allow an environment hostile to Jewish students to persist. The House believes that your institution is in violation of Title VI of the Civil Rights Act, and the inability or
unwillingness to rectify this violation through action requires accountability.
Postsecondary education is a unique opportunity for students to learn and have their ideas and beliefs challenged. However, universities receiving hundreds of millions of federal funds annually have denied
students that opportunity and have been hijacked to become venues for the promotion of terrorism, antisemitic harassment and intimidation, unlawful encampments, and in some cases, assaults and riots.
The House of Representatives will not countenance the use of federal funds to indoctrinate students into hateful, antisemitic, anti-American supporters of terrorism. Investigations into campus antisemitism by the Committee on Education and the Workforce and the Committee on Ways and Means have been expanded into a Congress-wide probe across all relevant jurisdictions to address this national crisis. The undersigned Committees will conduct oversight into the use of federal funds at MIT and its learning environment under authorities granted to each Committee.
• The Committee on Education and the Workforce has been investigating your institution since December 7, 2023. The Committee has broad jurisdiction over postsecondary education, including its compliance with Title VI of the Civil Rights Act, campus safety concerns over disruptions to the learning environment, and the awarding of federal student aid under the Higher Education Act.
• The Committee on Oversight and Accountability is investigating the sources of funding and other support flowing to groups espousing pro-Hamas propaganda and engaged in antisemitic harassment and intimidation of students. The Committee on Oversight and Accountability is the principal oversight committee of the US House of Representatives and has broad authority to investigate “any matter” at “any time” under House Rule X.
• The Committee on Ways and Means has been investigating several universities since November 15, 2023, when the Committee held a hearing entitled From Ivory Towers to Dark Corners: Investigating the Nexus Between Antisemitism, Tax-Exempt Universities, and Terror Financing. The Committee followed the hearing with letters to those institutions on January 10, 202
Francesca Gottschalk - How can education support child empowerment.pptxEduSkills OECD
Francesca Gottschalk from the OECD’s Centre for Educational Research and Innovation presents at the Ask an Expert Webinar: How can education support child empowerment?
The French Revolution, which began in 1789, was a period of radical social and political upheaval in France. It marked the decline of absolute monarchies, the rise of secular and democratic republics, and the eventual rise of Napoleon Bonaparte. This revolutionary period is crucial in understanding the transition from feudalism to modernity in Europe.
For more information, visit-www.vavaclasses.com
2. 2
Power Grids: A Historical Perspective on Complexity
Before Alternating Current (AC) After Alternating Current After/During Smart Grid
High System
Complexity!
Moderate System Complexity +
Low Data Complexity
High System + Data
Complexity!
Separate power lines
for different voltages.
AC as a boon for Electric
companies.
Smart Grid = high volume,
variety and velocity
http://en.wikipedia.org/wiki/Electric_power_transmission
Late 1800’s 1900’s Today
3. 3
Big Data in Smart Grid
One data point per month 96 million data points / day / million
consumers
Low instrumentation of the
power grid with sensors High instrumentation of the power
grid with sensors
Low number of energy sources
High proliferation of cleaner energy
sources like renewable energy
http://www.smartgridupdate.com/dataforutilities/pdf/DataManagementWhitePaper.pdf
4. 4
Sources of Big Data in Smart Grid
Velocity
Volume
Variety
Veracity
Original 3Vs: Doug Laney: http://goo.gl/wH3qG
From: http://www.smartgridupdate.com/dataforutilities/pdf/DataManagementWhitePaper.pdf
6. • What if your data volume gets so large and
varied you don't know how to deal with it?
• Do you store all your data?
• Do you analyze it all?
• How can you find out which data points are
really important?
• How can you use it to your best advantage?
6
Questions typically asked on Big Data
http://www.sas.com/big-data/
8. • Prediction of the spread of flu in real time during H1N1 2009
– Google tested a mammoth of 450 million different mathematical
models to test the search terms, comparing their predictions against
the actual flu cases; 45 important parameters were founds
– Model was tested when H1N1 crisis struck in 2009 and gave more
meaningful and valuable real time information than any public health
official system [Big Data, Viktor Mayer-Schonberger and Kenneth Cukier, 2013]
• FareCast: predict the direction of air fares over different
routes [Big Data, Viktor Mayer-Schonberger and Kenneth Cukier, 2013]
• NY city manholes problem [ICML Discussion, 2012]
8
Illustrative Big Data Applications
9. • Current focus mainly to serve business intelligence and targeted analytics
needs, not to serve complex individual and collective human needs (e.g.,
empower human in health, fitness and well-being; better disaster
coordination, smart energy consumption) that is highly
personalized/individualized/contextualized
– Incorporate real-world complexity: multi-modal and multi-sensory nature of real-
world and human perception
– Need deeper understanding of data and its role to information (e.g., skew,
coverage)
– Beyond correlation -> causation :: actionable info, decisions grounded on insights
• Human involvement and guidance: Leading to actionable information,
understanding and insight right in the context of human activities
– Bottom-up & Top-down processing: Infusion of models and background knowledge
(data + knowledge + reasoning)
9
What is missing?
11. Smart Data
Smart data makes sense out of Big data
It provides value from harnessing the
challenges posed by volume, velocity,
variety and veracity of big data, in-turn
providing actionable information and
improve decision making.
11
12. “OF human, BY human and FOR human”
Smart data is focused on the actionable
value achieved by human involvement in
data creation, processing and consumption
phases for improving
the human experience.
Another perspective on Smart Data
12
13. • Focus on verticals: advertising‚ social media‚ retail‚
financial services‚ telecom‚ and healthcare
– Aggregate data, focused on transactions, limited
integration (limited complexity), analytics to find
(simple) patterns
– Emphasis on technologies to handle volume/scale,
and to lesser extent velocity: Hadoop, NoSQL,MPP
warehouse ….
– Full faith in the power of data (no hypothesis),
bottom up analysis
13
Current Focus on Big Data
15. “OF human, BY human and FOR human”
Another perspective on Smart Data
15
16. Petabytes of Physical(sensory)-Cyber-Social Data everyday!
More on PCS Computing: http://wiki.knoesis.org/index.php/PCS 16
„OF human‟ : Relevant Real-time Data
Streams for Human Experience
17. “OF human, BY human and FOR human”
17
Another perspective on Smart Data
18. Use of Prior Human-created Knowledge Models
18
„BY human‟: Involving
Crowd Intelligence in data processing workflows
Crowdsourcing and Domain-expert guided
Machine Learning Modeling
19. “OF human, BY human and FOR human”
Another perspective on Smart Data
19
20. Electricity usage over a day, device at
work, power consumption, cost/kWh,
heat index, relative humidity, and public
events from social stream
Weather Application
Power Monitoring
Application
20
„FOR human‟ :
Improving Human Experience
Population Level Observations
Personal Level Observations
Action in the Physical World
Washing and drying has
resulted in significant cost
since it was done during peak
load period. Consider
changing this time to night.
21. 21
What matters?
Personal and Population
Level Observations
Actionable information for
optimized resource utilization
“The challenge for utilities in maximizing the benefits from smart grid data
analytics is the ability to turn the huge volume of smart grid data into value”
- Marianne Hedin, Senior Research Analyst,
Navigant Research
22. 22
Why do we care about Smart Data
rather than Big Data?
23. Transforming Big Data into Smart Data for Smart Energy:
Deriving Value via harnessing Volume, Variety and Velocity
using semantics and Semantic Web
Put Knoesis Banner
Keynote at Building Research Collaborations: Electricity Systems @ Purdue, August 28-29, 2013
Pavan
Kapanipathi
Pramod
Anantharam
Amit Sheth
Cory
Henson
Dr. T.K.
Prasad
Maryam
Panahiazar
Contributions by many, but Special Thanks to:
Hemant
Purohit
Special Thanks
The Ohio Center of Excellence in Knowledge-enabled Computing (Kno.e.sis), Wright State, USA
24. 24
10 Years Ago, August 14, 2003 Blackout!
http://www.npr.org/2013/08/14/210620446/10-years-after-the-blackout-how-has-the-power-grid-changed
Robert Giroux/Getty Images
25. 25
50 Million People without Power in 5 Northeastern States of US
http://www.npr.org/2013/08/14/210620446/10-years-after-the-blackout-how-has-the-power-grid-changed
Jonathan Fickies/Getty Images
26. 26
$6 Billion Lost Revenue
http://www.scientificamerican.com/article.cfm?id=2003-blackout-five-years-later
http://www.npr.org/2013/08/14/210620446/10-years-after-the-blackout-how-has-the-power-grid-changed
Julie Jacobson/AP
Julie Jacobson/AP
Utilities are hit with millions of dollars of fine when such blackouts
happen costing them on an average 1 million dollars a day!
27. 27
Cause of the Problem: Informal Investigation
Excessive summer heat (31° C or 88° F) caused consumers to draw excess
power for running air conditioners. Heating of power lines led to sagged
cables touching vegetation creating a fault.
FirstEnergy (FE) Corporation’s control room had a failed alarm system
further propagating the fault (cascading effect).
Lack of situational awareness by the control room is only one aspect of the
problem. The problem is deeply rooted in consumer awareness for making
informed decisions
http://www.npr.org/2013/08/14/210620446/10-years-after-the-blackout-how-has-the-power-grid-changed
28. 28
Cause of the Problem: Official Investigation
The U.S.-Canada Power System Outage Task Force reported four major causes
leading to the blackout:
1) "failed to assess and understand the inadequacies of FE's system, particularly with
respect to voltage instability and the vulnerability of the Cleveland-Akron area, and FE
did not operate its system with appropriate voltage criteria."
2) "did not recognize or understand the deteriorating condition of its system."
3) "failed to manage adequately tree growth in its transmission rights-of-way."
4) "failure of the interconnected grid's reliability organizations to provide effective real-time
diagnostic support."
http://en.wikipedia.org/wiki/Northeast_blackout_of_2003
29. 29
"We've done some things that will reduce the risks of the blackouts that happened last
time, but haven't done things that would prevent the next blackout”
-- Paul Hines, University of Vermont
Can we Prevent such Blackouts?
“we have new sensors installed in the grid, but utilities don't totally understand what to do
with all the data”
-- Paul Hines, University of Vermont
http://epaabuse.com/5159/news/after-coal-plants-close-where-does-america-get-cheap-electricity/
31. 31
Derive Insights from Smart Grid Data
"Big data .. for utility companies.. can turn the information from smart meter and smart grid
projects into meaningful operational insights and insights about their customer’s behavior."
- Big Data in Action, IBM
http://www.ecomagination.com/portfolio/ges-grid-iq-advanced-metering-infrastructureami-point-to-multipoint-p2mp-solution
http://gkenergyproject.blogspot.com/2010/07/smart-meter-diagram.html
32. 32
Power Grid Control Rooms are Complex!
Pacific Gas and Electric Company in California has collected over 70 terabytes of AMI
(Advanced Metering Infrastructure) data and this volume is increasing by 3 terabytes a
month
- Data Management And Analytics for Utilities, Smart Grid Update, 2013
http://www.rugeleypower.com/electricity-generation/producing-electricity.php
33. 33
Multimodal, Multisensory, and Real-time Observations
Synchrophaso
r data
Heat index,
relative humidity
Current Grid
Conditions
Renewable energy
generation forecast
What is the overall health of the Grid?
What are the vulnerabilities for today?
Power consumption
by consumers
http://www.rugeleypower.com/electricity-generation/producing-electricity.php
34. 34
Grid Health Score (diagnostic)
Semantic Perception and risk assessment algorithms can transform raw data (hard to
comprehend) to abstractions (e.g., Grid Health is 3 on a scale of 5) that is intuitively
understandable and valuable for decision makers.
Having health score for various parts of a grid will allow efficient utilization of
a decision maker’s precious attention
Risk assessment
model
Semantic
Perception
Synchrophaso
r data
Heat index,
relative humidity
Current Grid
Conditions
Renewable energy
generation forecast
Power consumption
by consumers
35. 35
Vulnerability Score (prognostic)
Vulnerability score (e.g., Today’s vulnerability score 4 on a scale of 5) is an abstraction
that uses current state of the grid (health score), power demand forecast, availability of
alternative energy sources, and historical consumer behavior
Vulnerability score will alleviate the data deluge problem of decision makers by
leveraging prior knowledge of the domain for creating risk assessment models
Risk assessment
model
Semantic
Perception
Synchrophaso
r data
Heat index,
relative humidity
Current Grid
Conditions
Renewable energy
generation forecast
Power consumption
by consumers
37. 37
“To make good on the promise of a truly “smart” grid, the industry must continue to
implement equipment that employs distributed intelligence, out to the edges of the
distribution system.“
-- Layered Intelligence Smart Grid Solutions, S&C Electric Company
“Intelligence at the Edges” of a Smart Grid
http://www.digikey.com/us/en/techzone/energy-harvesting/resources/articles/zigbees-smart-energy-20-profile.html
38. 38
Data Overload for Consumers
“They respond well to suggestions to do something.”
- Alex Laskey, President
and Founder of Opower
Personal
Schedule Smart Meters Power Consumption
Temperature,
relative humidity
Dynamic pricing
information
http://www.identika.com/2012/02/every-movie-made/
39. 39
Optimizing Cost, Benefit, and Preferences
Algorithms on the consumer side of the Smart Grid should should consider cost, benefit, and
preference of the user to devise an optimal strategy for power consumption
Which devices are contributing to higher power bill?
When should I operate the washer/dryer?
How much convenience I am willing to forego?
Semantic
Perception
Personalized
optimization
Personalized
recommendation
Img: http://marloncarvallovillae.blogspot.com/2011_02_01_archive.html
http://www.1800timeclocks.com/icon-time-systems/icon-time-upgrades/icon-time-advanced-pack-upgrade-sb100-pro/
Personal
Schedule
Smart Meters
Power Consumption
Temperature,
relative humidity
Dynamic pricing
information
40. 41
Big Data to Smart Data: A peek at some domains
Healthcare
Social Media &
Disaster Response
http://theshannoncompany.com.au/blog/?p=504
41. Sensing is a key enabler of the Internet of Things
BUT, how do we make sense of the resulting avalanche
of sensor data?
50 Billion Things by 2020 (Cisco)
44
42. … and do it efficiently and at scale
What if we could automate this
sense making ability?
45
44. People are good at making sense of sensory input
What can we learn from cognitive models of perception?
• The key ingredient is prior knowledge
47
45. * based on Neisser’s cognitive model of perception
Observe
Property
Perceive
Feature
Explanation
Discrimination
1
2
Perception Cycle*
Translating low-level signals
into high-level knowledge
Focusing attention on those
aspects of the environment that
provide useful information
Prior Knowledge
48
46. To enable machine perception,
Semantic Web technology is used to integrate
sensor data with prior knowledge on the Web
49
47. Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
50
48. Prior knowledge on the Web
W3C Semantic Sensor
Network (SSN) Ontology Bi-partite Graph
51
50. Explanation
Inference to the best explanation
• In general, explanation is an abductive problem; and
hard to compute
Finding the sweet spot between abduction and OWL
• Single-feature assumption* enables use of OWL-DL
deductive reasoner
* An explanation must be a single feature which accounts for
all observed properties
Explanation is the act of choosing the objects or events that best account for a set of
observations; often referred to as hypothesis building
53
51. Explanation
Explanatory Feature: a feature that explains the set of observed properties
ExplanatoryFeature ≡ ∃ssn:isPropertyOf—.{p1} ⊓ … ⊓ ∃ssn:isPropertyOf—.{pn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Observed Property Explanatory Feature
54
52. Discrimination is the act of finding those properties that, if observed, would help distinguish
between multiple explanatory features
Observe
Property
Perceive
Feature
Explanation
Discrimination
2
Focusing attention on those
aspects of the environment that
provide useful information
Discrimination
55
53. Discrimination
Expected Property: would be explained by every explanatory feature
ExpectedProperty ≡ ∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ∃ssn:isPropertyOf.{fn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Expected Property Explanatory Feature
56
54. Discrimination
Not Applicable Property: would not be explained by any explanatory feature
NotApplicableProperty ≡ ¬∃ssn:isPropertyOf.{f1} ⊓ … ⊓ ¬∃ssn:isPropertyOf.{fn}
elevated blood pressure
clammy skin
palpitations
Hypertension
Hyperthyroidism
Pulmonary Edema
Not Applicable Property Explanatory Feature
57
56. Through physical monitoring and
analysis, our cellphones could act as
an early warning system to detect
serious health conditions, and
provide actionable information
canary in a coal mine
Our Motivation
kHealth: knowledge-enabled healthcare
59
57. How do we implement machine perception efficiently on a
resource-constrained device?
Use of OWL reasoner is resource intensive
(especially on resource-constrained devices),
in terms of both memory and time
• Runs out of resources with prior knowledge >> 15 nodes
• Asymptotic complexity: O(n3)
60
58. intelligence at the edge
Approach 1: Send all sensor observations
to the cloud for processing
Approach 2: downscale semantic
processing so that each device is capable
of machine perception
61
Henson et al. 'An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices,
ISWC 2012.
59. Efficient execution of machine perception
Use bit vector encodings and their operations to encode prior knowledge and
execute semantic reasoning
010110001101
0011110010101
1000110110110
101100011010
0111100101011
000110101100
0110100111
62
60. O(n3) < x < O(n4) O(n)
Efficiency Improvement
• Problem size increased from 10’s to 1000’s of nodes
• Time reduced from minutes to milliseconds
• Complexity growth reduced from polynomial to linear
Evaluation on a mobile device
63
61. 2 Prior knowledge is the key to perception
Using SW technologies, machine perception can be formalized and
integrated with prior knowledge on the Web
3 Intelligence at the edge
By downscaling semantic inference, machine perception can
execute efficiently on resource-constrained devices
Semantic Perception for smarter analytics: 3 ideas to takeaway
1 Translate low-level data to high-level knowledge
Machine perception can be used to convert low-level sensory
signals into high-level knowledge useful for decision making
64
62. Qualities
-High BP
-Increased Weight
Entities
-Hypertension
-Hypothyroidism
kHealth
Machine Sensors
Personal Input
EMR/PHR
Comorbidity risk score
e.g., Charlson Index
Longitudinal studies of
cardiovascular risks
- Find correlations
- Validation
- domain knowledge
- domain expert
Parameterize the
model
Risk Assessment Model
Current Observations
-Physical
-Physiological
-History
Risk Score
(Actionable Information)
Model CreationValidate correlations
Historical observations
of each patient
Risk Score: from Data to Abstraction and Actionable Information
65
63. 66
1 http://www.pdf.org/en/parkinson_statistics
10
million 60,000
$25
billion
$100,00
0
1 million
People worldwide are
living with Parkinson's
disease1.
Americans are
diagnosed with
Parkinson's disease
each year1.
Spent on Parkinson’s
alone in a year in the
US1
Therapeutic surgery
can cost up to $100,000
dollars per patient1.
Americans live with
Parkinson’s Disease1
Parkinson‟s Disease (PD)
64. Parkinson’s disease (PD) data from The Michael J. Fox Foundation
for Parkinson’s Research.
67
1https://www.kaggle.com/c/predicting-parkinson-s-disease-progression-with-smartphone-data
8 weeks of data from 5 sensors on a smart phone, collected for 16 patients
resulting in ~12 GB (with lot of missing data).
Variety Volume
VeracityVelocity
Value
Can we detect the onset of Parkinson’s disease?
Can we characterize the disease progression?
Can we provide actionable information to the patient?
semantics
Representing prior knowledge of PD
led to a focused exploration of this
massive dataset
WHY Big Data to Smart Data: Healthcare example
65. 68
Big Data to Smart Data Using a Knowledge Based Approach
ParkinsonMild(person) = Tremor(person) ∧ PoorBalance(person)
ParkinsonModerate(person) = MoveSlow(person) ∧ PoorSleep(person) ∧ MonotoneSpeech(person)
ParkinsonAdvanced(person) = Fall(person)
Control Group PD Patients
Movements of an active
person has a good
distribution over X, Y, and
Z axis
Restricted movements by
a PD patient can be seen
in the acceleration
readings
Audio is well modulated
with good variations in
the energy of the voice
Audio is not well
modulated represented a
monotone speech
Declarative Knowledge of
Parkinson’s Disease used to focus
our attention on symptom
manifestations in sensor
observations
67. Asthma is a multifactorial disease with health signals spanning personal,
public health, and population levels.
70
Real-time health signals from personal level (e.g., Wheezometer, NO in breath,
accelerometer, microphone), public health (e.g., CDC, Hospital EMR), and
population level (e.g., pollen level, CO2) arriving continuously in fine grained
samples potentially with missing information and uneven sampling frequencies.
Variety Volume
VeracityVelocity
Value
Can we detect the asthma severity level?
Can we characterize asthma control level?
What risk factors influence asthma control?
What is the contribution of each risk factor?semantics
Understanding relationships between
health signals and asthma attacks
for providing actionable information
WHY Big Data to Smart Data: Healthcare example
68. 71
Population Level
Personal
Public Health
Variety: Health signals span heterogeneous sources
Volume: Health signals are fine grained
Velocity: Real-time change in situations
Veracity: Reliability of health signals may be compromised
Value: Can I reduce my asthma attacks at night?
Decision support to doctors
by providing them with
deeper insights into patient
asthma care
Asthma: Demonstration of Value
69. 72
Sensordrone – for monitoring
environmental air quality
Wheezometer – for monitoring
wheezing sounds
Can I reduce my asthma attacks at night?
What are the triggers?
What is the wheezing level?
What is the propensity toward asthma?
What is the exposure level over a day?
What is the air quality indoors?
Commute to Work
Personal
Public Health
Population Level
Closing the window at home
in the morning and taking an
alternate route to office may
lead to reduced asthma attacks
Actionable
Information
Asthma: Actionable Information for Asthma Patients
70. Personal, Public Health, and Population Level Signals for Monitoring Asthma
Asthma Control => Daily Medication
Choices for starting
therapy
Not Well Controlled Poor Controlled
Severity Level of
Asthma
(Recommended Action) (Recommended Action) (Recommended Action)
Intermittent Asthma SABA prn - -
Mild Persistent Asthma Low dose ICS Medium ICS Medium ICS
Moderate Persistent
Asthma
Medium dose ICS alone
Or with
LABA/montelukast
Medium ICS +
LABA/Montelukast
Or High dose ICS
Medium ICS +
LABA/Montelukast
Or High dose ICS*
Severe Persistent Asthma High dose ICS with
LABA/montelukast
Needs specialist care Needs specialist care
ICS= inhaled corticosteroid, LABA = inhaled long-acting beta2-agonist, SABA= inhaled short-acting beta2-agonist ;
*consider referral to specialist
Asthma Control
and Actionable Information
Sensors and their observations
for understanding asthma
73
71. 74
Personal
Level Signals
Societal Level
Signals
(Personal Level Signals)
(Personalized
Societal Level Signal)
(Societal Level Signals)
Societal Level Signals
Relevant to the
Personal Level
Personal Level Sensors
(kHealth**) (EventShop*)
Qualify Quantify
Action
Recommendation
What are the features influencing my asthma?
What is the contribution of each of these features?
How controlled is my asthma? (risk score)
What will be my action plan to manage asthma?
Storage
Societal Level Sensors
Asthma Early Warning Model (AEWM)
Query AEWM
Verify & augment
domain knowledge
Recommended
Action
Action
Justification
Asthma Early Warning Model
*http://www.slideshare.net/jain49/eventshop-120721, ** http://www.youtube.com/watch?v=btnRi64hJp4
72. 75
Population Level
Personal
Wheeze – Yes
Do you have tightness of chest? –Yes
ObservationsPhysical-Cyber-Social System Health Signal Extraction Health Signal Understanding
<Wheezing=Yes, time, location>
<ChectTightness=Yes, time, location>
<PollenLevel=Medium, time, location>
<Pollution=Yes, time, location>
<Activity=High, time, location>
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
Wheezing
ChectTightness
PollenLevel
Pollution
Activity
RiskCategory
<PollenLevel, ChectTightness, Pollution,
Activity, Wheezing, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
<2, 1, 1,3, 1, RiskCategory>
.
.
.
Expert
Knowledge
Background
Knowledge
tweet reporting pollution level
and asthma attacks
Acceleration readings from
on-phone sensors
Sensor and personal
observations
Signals from personal, personal
spaces, and community spaces
Risk Category assigned by
doctors
Qualify
Quantify
Enrich
Outdoor pollen and pollution
Public Health
Health Signal Extraction to Understanding
Well Controlled - continue
Not Well Controlled – contact nurse
Poor Controlled – contact doctor
73. • Real Time Feature Streams:
http://www.youtube.com/watch?v=_ews4w_eCpg
• kHealth: http://www.youtube.com/watch?v=btnRi64hJp4
76
Demos
74. 77
Smart Data in Social Media & Disaster Response
To Understand
critical information
dynamics in real
world events
75. Twitris‟ Dimensions of Integrated Semantic Analysis
78Sheth et al. Twitris- a System for Collective Social Intelligence, ESNAM-2013
76. What is Smart Data in the context of
Disaster Management
ACTIONABLE: Timely delivery of
right resources and information to
the right people at right location!
79
77. Join us for the Social
Good!
http://twitris.knoesis.org
RT @OpOKRelief:
Southgate Baptist Church
on 4th Street in Moore
has food, water, clothes,
diapers, toys, and more.
If you can't go,call 794
Text "FOOD" to
32333, REDCROSS to
90999, or STORM to
80888 to donate $10
in storm relief.
#moore #oklahoma
#disasterrelief
#donate
Want to help animals in
#Oklahoma? @ASPCA tells
how you can help:
http://t.co/mt8l9PwzmO
CITIZEN SENSORS
RESPONSE TEAMS
(including humanitarian
org. and ‘pseudo’ responders)
VICTIM SITE
Coordination of
emerging needs
after a disaster
Does anyone
know where to
send a check to
donate to the
tornado
victims?
Where do I go
to help out for
volunteer work
around Moore?
Anyone know?
Anyone know
where to donate
to help the
animals from the
Oklahoma
disaster? #oklah
oma #dogs
Matched
Matched
Matched
Serving the need!
If you would like to volunteer
today, help is desperately
needed in Shawnee. Call
273-5331 for more info
http://www.slideshare.net/hemant_knoesis/cscw-2012-hemantpurohit-11531612
80
Purohit et al. Framework to Analyze Coordination in Crisis Response, 2012. Int’l Collaboration in-progress:
78. Smart Data from Twitris system for
Disaster Response Coordination
Which are the primary locations of
power failure?
Who are all the people to engage
with for better information
diffusion?Where are the charging stations to
sustain communication?
Smart data provides actionable information and improve decision making through
semantic analysis of Big Data.
Who are the resource seekers and
suppliers?
81
79. Disaster Response Coordination:
Twitris Summary for Actionable Nuggets
83
Important tags to
summarize Big Data flow
Related to Oklahoma
tornado
Images and Videos Related
to Oklahoma tornado
80. 84
Disaster Response Coordination:
Twitris Real-time information for needs
Incoming Tweets with need
types to give quick idea of
what is needed and where
currently #OKC
Legends for Different
needs #OKC
(It is real-time widget for monitoring of needs, so will not be active after the event has passed)
http://twitris.knoesis.org/oklahomatornado
82. Really sparse Signal to Noise:
• 2M tweets during the first week after #Oklahoma-tornado-2013
- 1.3% as the highly precise donation requests to help
- 0.02% as the highly precise donation offers to help
86
• Anyone know how to get involved to
help the tornado victims in
Oklahoma??#tornado #oklahomacity
(OFFER)
• I want to donate to the Oklahoma cause
shoes clothes even food if I can (OFFER)
Disaster Response Coordination:
Finding Actionable Nuggets for Responders to act
• Text REDCROSS to 909-99 to donate to
those impacted by the Moore tornado!
http://t.co/oQMljkicPs (REQUEST)
• Please donate to Oklahoma disaster
relief efforts.: http://t.co/crRvLAaHtk
(REQUEST)
For responders, most important information is the scarcity and
availability of resources, can we mine it via Social Media?
83. 87
Disaster Response Coordination:
Engagement Interface for responders
What-Where-How-Who-Why
Coordination
Influential users to engage
with and resources for
seekers/supplies at a location,
at a timestamp
Contextual
Information for a
chosen topical tags
84. • Illustrious scenario: #Oklahoma-tornado 2013
88
Disaster Response Coordination:
Anecdote for the value of Smart Data
FEMA asked us to quickly filter
out gas-leak related data
Mining the data for smart nuggets
to inform FEMA (Timely needs)
Engaged with the author of this
information to confirm (Veracity)
e.g., All gas leaks in #moore were capped and stopped by
11:30 last night (at 5/22/2013 1:41:37)
Lot of tweets for ‘how to/where to’ assist (‘pseudo’ responders)
e.g., I want to go to Oklahoma this weekend & do what i can to help those people with
food,cloths & supplies,im in the feel of wanting to help ! :)
85. 89
Current Grid Conditions
Renewable energy
generation forecast
Synchrophasor data
Heat index,
relative humidity
Power consumption
by consumers
Big Data from Smart Grid Smart Data from Smart Grid
What is the overall health of the Grid?
What are the vulnerabilities for today?
Red, yellow, and green indicate high,
medium, and low risk allowing decision
makers to focus on red & yellow lines
Big Data vs. Smart Data in Smart Grids (Utilities perspective)
86. 90
Personal Schedule
Big Data from Smart Grid
& Smart Meters
Smart Data from Smart
Grid & Smart Meters
Smart Meters
Power Consumption
Temperature, relative humidity
Dynamic pricing information
http://www.digikey.com/us/en/techzone/energy-harvesting/resources/articles/zigbees-smart-energy-20-profile.html
Which devices are contributing to higher power bill?
When should I operate the washer/dryer?
Red, yellow, and green
indicating high, medium, and
low power consumption
Recommendation algorithms
will analyze these abstractions
with domain knowledge
Actions to optimize power bill
will be recommended
Big Data vs. Smart Data in Smart Grids (Consumer perspective)
87. Take Away
• Data processing for Smart Grids/Utilities and Consumers is
lot more than a Big Data processing problem
• It is all about the human – not computing, not device: help
them make better decisions, give actionable information
– Computing for human experience
• Whatever we do in Smart Data, focus on human-in-the-loop
(empowering machine computing!):
– Of Human, By Human, For Human
– But in serving human needs, there is a lot more than what
current big data analytics handle – variety, contextual,
personalized, subjective, spanning data and knowledge across P-
C-S dimensions
91
88. Acknowledgements
• Kno.e.sis team
• Funds: NSF, NIH, AFRL, Industry…
• Note:
• For images and sources, if not on slides, please see slide notes
• Some images were taken from the Web Search results and all such images belong
to their respective owners, we are grateful to the owners for usefulness of these
images in our context.
92
89. • OpenSource: http://knoesis.org/opensource
• Showcase: http://knoesis.org/showcase
• Vision: http://knoesis.org/node/266
• Publications: http://knoesis.org/library
93
References and Further Readings
90. Amit Sheth’s
PHD students
Ashutosh Jadhav
Hemant
Purohit
Vinh
Nguyen
Lu Chen
Pavan
Kapanipathi
Pramod
Anantharam
Sujan
Perera
Alan Smith
Pramod Koneru
Maryam Panahiazar
Sarasi Lalithsena
Cory Henson
Kalpa
Gunaratna
Delroy
Cameron
Sanjaya
Wijeratne
Wenbo
Wang
Kno.e.sis in 2012 = ~100 researchers (15 faculty, ~50 PhD students)
91. 95
thank you, and please visit us at
http://knoesis.org
Kno.e.sis – Ohio Center of Excellence in Knowledge-enabled Computing
Wright State University, Dayton, Ohio, USA
Smart Data
Editor's Notes
Major historical events are outlined here.Before Alternating Current (AC) was discovered, each device operating at different voltage required a separate power line! Also, the power generation has to be very close to the load. The power grid was visualized more like a distributed generators all throughout the grid.After AC, the power could be transmitted over long distances and a single voltage could be adapted to various devices operating at different voltages.The power grids are getting complex with many alternate sources of energy, varying consumer demand, increasing area of coverage, and increased loads due to electric vehicles. Smart Grid will generate lot of data and interpretation of this data is a challenging.
http://radhakrishna.typepad.com/rks_musings/2013/04/big-data-review.htmlGoogle predicted the spread of flu in real time - after analyzing two datasets, a.) 50 million most common terms that Americans type, b.) data on the spread of seasonal flu from public health agency- tested a mammoth of 450 million different mathematical models to test the search terms, comparing their predictions against the actual flu cases- model was tested when H1N1 crisis struck in 2009 and gave more meaningful and valuable real time information than any public health official system (Big Data, Viktor Mayer-Schonberger and Kenneth Cukier, 2013)
Better Algorithms Beat More Data — And Here’s Whyhttp://allthingsd.com/20121128/better-algorithms-beat-more-data-and-heres-why/Big Data Cannot Replace Human Judgmenthttp://www.matchcite.com/blog/blog/2012/july/big-data-cannot-replace-human-judgment.aspx**Comments about the articles
Smart data makes sense out of big data – it provides value from harnessing the challenges posed by volume, velocity, variety and veracity of big data, to provide actionable information and improve decision making.
- HUMAN CENTRIC!!
Information is CREATED by human with the Machinery available – Wikipedia tool, sensors and social networksInformation is STORED in Man+Machine readable format, LODInformation is PROCESSED using the LOD and Human assisted Knowledge-basedHigher level abstraction on info is now consumed in many mechanistic ways (including GIS) to provide EXPERIENCE for humans
All the data related to human activity, existence and experiencesMore on PCS Computing: http://wiki.knoesis.org/index.php/PCS
Information is CREATED by human with the Machinery available – Wikipedia tool, sensors and social networksInformation is STORED in Man+Machine readable format, LODInformation is PROCESSED using the LOD and Human assisted Knowledge-basedHigher level abstraction on info is now consumed in many mechanistic ways (including GIS) to provide EXPERIENCE for humans Example of a human guided modeling and improved performancehttp://research.microsoft.com/en-us/um/people/akapoor/papers/IJCAI%202011a.pdf
Also, we have weather application which performs abstraction on weather sensory observations to identify blizzard conditions (food for actions!!) :--20,000 weather stations (with ~5 sensors per station)-- Real-Time Feature Streams - live demo: http://knoesis1.wright.edu/EventStreams/ - video demo: https://skydrive.live.com/?cid=77950e284187e848&sc=photos&id=77950E284187E848%21276
Lets find it..
Starting slide Various Big data problems – Traditional examples vs what we are doing examples. Variety and Velocity than Volume. kHealth problem. People will be interested in Smart Data.Traditional ML techniques, High Performance Computing, Statistics. Human level of Abstraction is Smart data.
Such a blackout would cause billions of dollars in lost revenue. This particular blackout resulted in 6 billion dollar loss.Not only the consumers lose revenue even the power utility companies are fined almost a million dollars a day.
Two of the problems are problems in “understanding” the system and the available data/observations. Lack of experience/training in deciphering this data had serious implications.The items 1 and 2 related to understanding real-world complexity by incorporating multi-modal and multi-sensory observations. Algorithms that can provide abstractions to decision makers for better comprehension of the situation.Providing abstractions in the context of the grid state continuously would lead to actionable information to decision makers.
“Syncrophasors are like traffic cameras on a road traffic monitoring system”It provides high frequency (30 samples per second) updates on voltage, power flow in an electric network, phase difference, and many more electric quantities.https://www.selinc.com/SELUniversity/Courses/SYS/310/ http://green.blogs.nytimes.com/2010/04/01/for-the-smart-grid-a-synchophasor/?_r=0
All the dollar amount here is per year.On an average, each household can save $369 / year which is $4 billion / year of reduced energy bills in the US=> Not all this is by doing nothing but just monitoring the usage and provide near real-time power consumption and electricity bill using smart meters
Larry Smarr is a professor at the University of California, San DiegoAnd he was diagnosed with Crones DiseaseWhat’s interesting about this case is that Larry diagnosed himselfHe is a pioneer in the area of Quantified-Self, which uses sensors to monitor physiological symptomsThrough this process he discovered inflammation, which led him to discovery of Crones DiseaseThis type of self-tracking is becoming more and more common
- what if we could automate this sense making ability?- and what if we could do this at scale?
sense making based on human cognitive models
perception cycle contains two primary phasesexplanationtranslating low-level signals into high-level abstractions inference to the best explanationdiscriminationfocusing attention on those properties that will help distinguish between multiple possible explanationsused to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
perception cycle contains two primary phasesexplanationtranslating low-level signals into high-level abstractions inference to the best explanationdiscriminationfocusing attention on those properties that will help distinguish between multiple possible explanationsused to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
A single-feature (disease) assumption means that all the observed properties (symptoms) must be explained by a single feature.i.e., this framework is not expressive enough to model comorbidity where there may be more than one feature (disease) co-existing For example, if there are two diseases causing disjoint symptoms, and all the symptoms of both the diseases are observed, then this framework will not be able to find the coverage and returns no diseases.
perception cycle contains two primary phasesexplanationtranslating low-level signals into high-level abstractions inference to the best explanationdiscriminationfocusing attention on those properties that will help distinguish between multiple possible explanationsused to intelligently task sensors and collect additional observations (rather than brute force approach of blindly collecting all observations)
- With this ability,many problems could be solved- For example: we could help solve health problems (before they become serious health problems) through monitoring symptoms and real-time sense making, acting as an early warning system to detect problematic health conditions
Intelligence distributed at the edge of the networkRequires resource-constrained devices (mobile phones, gateway notes, etc.) to be able to utilize SW technologies
Intelligence distributed at the edge of the networkRequires resource-constrained devices (mobile phones, gateway notes, etc.) to be able to utilize SW technologiesHenson et al. 'An Efficient Bit Vector Approach to Semantics-based Machine Perception in Resource-Constrained Devices, ISWC 2012.
compute machine perception inferences -- i.e., explanation and discrimination -- of high-complexity on a resource-constrained devices in milisecondsDifference between the other systems and what this system provides
Intelligence at the age. Shipping computation and domain models to the edge (Distributed)
Massive amount of data will be collected by sensors and mobile devices yet patients and doctors care about “actionable” information.This data has all the four Vs of big data and we used knowledge enabled techniques to transform it into valueIn the context of PD, we analyzed massive amount of sensor data collected by sensors on a smartphones to understand detection and characterization of PD severity.
Main idea: Prior knowledge of PD was used to facilitate its detection from massive sensor data by reducing the search spaceDetails:Declarative knowledge of PD includes PD severity and their symptoms as shown in the logical rule aboveEach PD severity level is a conjunction of a set of PD symptomsEach symptom was mapped to its manifestation in sensor observationsThe availability of declarative knowledge significantly improved the analytics by aiding feature selection processThe graphs above contrasts the physical movements and voice of two control group members and two PD patients
Much of the early work in Big data is being done with focusing on uni-directional among XYZ.
Categorization of severity based on weather conditions. Actionable information is contextually dependent.
Power Grid Context:Power blackout will result in critical needs such as food, water, charging stations, fuel, etc. which need to be addressed by the on-ground responders and remote responders. - 1 (+half) minuteAlright, so let’s motivate by this situation during emergency - Various actors: resource seekers, responder teams, resource providers at remote siteAnd - each of these actor groups have questions --- - needs - providers - responders: wondering!Here we have social network to connect these actors and bridge the gap for communication platformBut it’s potential use is yet to be realized for effective helpBecause.. (next slide)
Talk about what kind of smart data we provide that helps the actions of crisis response coordination.
Source: Purohit et. al 2013 (https://docs.google.com/a/knoesis.org/document/d/1aBJ2egHICUwaWxR8jOoTIUfEYj1QAnUt0q7haIKoYGY/edit# , http://www.knoesis.org/library/resource.php?id=1865)
http://twitris.knoesis.org/oklahomatornado
(It is real-time widget for monitoring of needs, so will not be active after the event has passed) http://twitris.knoesis.org/oklahomatornado
Highly rich interface for response team
Imagine if this is a power failure, we can get actionable insights in a timely manner
More at: http://wiki.knoesis.org/index.php/PCSAnd http://knoesis.org/projects/ssw/