The document discusses the concept of citizen sensing in the context of mining social signals and perceptions, emphasizing the role of semantics in enriching human experience and enabling effective data analysis. It covers various applications, tools, and frameworks developed over the years, including 'Twitris' for analyzing social media data, and highlights the significance of semantic models in processing user-generated content. The findings denote the explosive growth of data and the necessity for semantic scalability to derive meaningful insights for human activities and decision-making.

2. Citizen SensingOpportunities and Challenges in Mining Social Signals and PerceptionsAmit P. Shethamit@knoesis.orgLexisNexis Ohio Eminent ScholarOhio Center of Excellence in Knowledge enabled Computing (Kno.e.sis)Wright State University, Dayton, OHhttp://knoesis.orgThanks: Kno.e.sis team, esp. Wenbo Wang, Chen Lu, Cory, Hemant, Pavan

3. Ohio Center of Excellence in Knowledge-enabled ComputingSemantics as core enabler, enhancer @ Kno.e.sisone of the two largest academic groups in Semantic Web; multidisciplinary

4. Semantics & Semantic Web in 1999-2002

5. ATTRIBUTE & KEYWORDQUERYINGSEMANTIC BROWSINGuniform view of worldwide distributed assets of similar typeTargeted e-shopping/e-commerceassets accessTaalee Semantic/Faceted Search & Browsing (1999-2001)BLENDED BROWSING & QUERYINGTaalee Semantic Search ….

6. Links to news on companies that compete against Commerce OneLinks to news on companies Commerce One competes against(To view news on Ariba, click on the link for Ariba)Crucial news on Commerce One’s competitors (Ariba) can be accessed easily and automaticallySearch for company ‘Commerce One’Semantic Search/Browsing/Directory (2001-….)

7. Semantic Search/Browsing/Directory (2001-….)System recognizes ENTITY & CATEGORYRelevant portionof the Directory is automatically presented.

8. Semantic Search/Browsing/Directory (2001-….)Users can exploreSemantically relatedInformation.

9. Extracting Semantic Metadata from Semistructured and Structured Sources (1999 – 2002)Semagix Freedom for building ontology-driven information systemManaging Semantic Content on the Web

10. Fast forward to 2010-2011

11. SearchIntegrationAnalysisDiscoveryQuestion AnsweringSituational AwarenessDomain ModelsPatterns / Inference / ReasoningRDBRelationship WebMetadata ExtractionMeta data / Semantic AnnotationsText(formal/Informal)Structured and Semi-structured dataSensor DataMultimedia Content and Web data

12. Let Us Start with Social Data

13. Jan. 2011Egypt ProtestImage:http://bit.ly/g4yPXSImage:http://bit.ly/qHI7wIImage:http://bit.ly/qmDocA

14. http://bit.ly/nP1E4qImage:http://bit.ly/fl4gEJMar. 2011 http://cnet.co/jdQgMEJapan Earthquake and Tsunamihttp://bit.ly/gWboibRecently funded NSF proposal: Social Media Enhanced Organizational Sensemakingin Emergency Response

15. Image:http://bit.ly/nqq6WjJul. 2011I-75 Traffic Jam in US

16. Citizen SensingWho?An interconnected network of peopleWhat?Observe, report, collect, analyze, and disseminate informationHow?Via text, audio, video and built in device sensor (and smart devices)Image: http://bit.ly/nvm2iP "Citizen Sensing, Social Signals, and Enriching Human Experience”, Internet Computing, July-Aug. 2009

17. Enablers: Mobile Devices & Ubiquitous ConnectivityMobile Platforms Hit Critical Mass, Over 5 billions users 1+B with internet connected mobile devices (2010)Smartphones > PCs + Notebooks > Notebooks + Netbooks (2010E)500K+ mobile phone applications74% of mobile phone users (2.4B)worldwide used SMS (2007)Mobile is Global; Ubiquitous; 24x7Built in sensorsenvironmental, biometric/biomedical,...Image: http://bit.ly/mYqcPF

18. Enablers: Web 2.0 & Social MediaA huge number of users750M+ active Facebook Users1+B tweets/wk; 175M+ Twitter usersInternet Users:2BlnImage: http://bit.ly/euLETTImage: http://bit.ly/euLETTImage: http://bit.ly/euLETTImage: http://bit.ly/euLETTImage: http://bit.ly/euLETT

19. Role of Semantics in Citizen SensingKey of citizen sensing: extract metadata/annotate different types of metadata (depend on application need)Spatial, temporal, thematic: key phrase, named entity, relationship, topic/category, event descriptors, sentiment…People, network, contentSemantics: provide the meaning of datavarious forms of semantic models: core vocabularies/nomenclatures,community created dictionaries/folksonomies/reference databases, automatically extracted domain models, manually created taxonomies, formal ontologiesdeal with complexities of user generated data; supplement well-known statistical and natural language processing (NLP) techniques

20. Research Application: Twitris

21. Twitris - MotivationImage: http://bit.ly/etFezlWhat were people in U.S.A. saying about Bin Laden’s death?How about people in Egypt?How about people in India?TOO MANY tweets to be read each day!!!TwitrisNow: WHEN, WHERE, People are talking WHATFuture: socio, cultural, behavioral studies‘Spatio-Temporal-Thematic Analysis of Citizen-Sensor Data – Challenges and Experiences’, 2009

22. Twitris: Semantic Social Web Mash-upSelect topicSelect dateN-gram summariesTopic treeSpatial MarkerTweet trafficSentiment AnalysisImages & VideosReference newsRelated tweetsWikipedia articlesMore: TWITRIS

23. Analyzing Events from Temporal PerspectiveHow did tweets in United States on the death of Bin Laden evolve over time?May 2ndMay 4th“RT @TWlTTERWHALE: Please do not click on any links saying Osama Bin Laden EXECUTION Video! This is a virus that hacks accounts. ”RT @ReallyVirtual: Here's a picture of OBL'shideout in Abbottabad, as shared by a friend @Rahat http://yfrog.com/h7w4izmjImg:http://www.twitpic.com/4t1mt0

24. Analyzing Events from Spatial PerspectiveTweets (Death of Bin Laden) in Egypt VS tweets in IndiaMay 2ndEgyptMay 2ndIndia“RT @mvatlarge: U.S. has given Pakistani military nearly $20 billion since 9/11 for the privilege of housing bin Laden: http://is.gd/xegnFm”“#Egypt foreign minister: Egygov't has no official comment but we condemn all forms of violence in international relations. #osama #obl”

25. A sample of current research @ Kno.e.sis demonstrating role of semantics in Citizen Sensing & Social Media Analysis

26. User-communityEngagement Analysis

27. User-community EngagementHow do we understand the phenomenon of user participation (engagement) in topic discussions？How communities form during the product launch?What factors can attract users to engage in these communities, therefore further spreading the message?How quickly we can disseminate information between resource providers and people in need of resources in case of emergency?Image: http://itcilo.wordpress.com

28. Analysis Framework:People-Content-network Analysis (PCNA)28Understanding User-Community Engagement by Multi-faceted Features: A Case Study on Twitter. SoME @ WWW2011.

29. Three Sets of FeaturesWhich set is more important?Content features[Characteristics of tweets posted by active friends of U]:keywords: number of event-relevant keywordshashtags: number of event-relevant hashtagsretweet: number of retweetsmention: number of mentionsurl: number of relevancy-adjust hyperlinksIrrelevant hyperlink is given number -1subjectivity: Subjectivity scores for words and emoticonsLinguistic Cues (LIWC1 analysis): Features for the language usage. Top-3 transformed features using Principle Component Analysis (PCA) extractedCommunity features: [Characteristics of the active community/network under consideration]wccSize: size of the weakly-connected component (WCC) which U’s friends belongs to in the active network.wccPercent: ratio of wccSizeto the size of the active network.connectivity: number of active friends (i.e. followees) in the community.communitySize: size of the active community.Author features [Characteristics of friends that U is following]:Only friends in the active community are considered.logFollower: logarithm of follower countlogFollowee: logarithm of followee countKlout[1]: a integrated measure of user influence and popularityOther profile information and activity history[2].

30. Experiments: ResultsContent is the key to understand user community engagementHighLowPerformanceNetworkContentAllPeopleEvent-TypeSummary of Prediction Accuracy (%)Statistical significant results are in bold30

31. Background Knowledge to improve Social Data AnalysisEvent oriented CommunityExternal Knowledge basesDynamic Domain Model for the eventSEMANTIC ASSOCIATION TO UNDERSTAND ENGAGEMENT LEVEL &IMPROVE IESocial NetworkUser ProfilesMined User Interests and User Types

32. Analysing the Content can be Hard…Is Merry Christmas a song? If it is, which ‘Merry Christmas’since there are 60 songs of the same name.‘So Good’ is also a song!Using a domain model (E.g., MusicBrainz)Using context cues from the contente.g. new Merry Christmas tuneReduce potential entity spot size (with restrictions) e.g. new albums/songs ‘Multimodal Social Intelligence in a Real-Time Dashboard System’ VLDB Journal 2010

33. Real Time Social Media Data Analysis

34. MotivationPeople can’t wait for InformationDisaster ManagementUshahidi (www.ushahidi.org)Real-Time MarketsRealTimeMarkets (http://www.realtimemarkets.com/) Brand TrackingTwarql (http://wiki.knoesis.org/index.php/Twarql)Movie reviewsFlicktweets (www.flicktweets.com)Journalism

35. ScenariosBrand TrackingGive me a stream of locations whereKinectis being mentioned right nowGive me all people that have said negative things aboutKinectHow can we do this?

36. Twarql (Twitter Feeds through SPARQL)Semantically annotate tweets with entities, hashtags, URLs, sentiments, etc.Encode content in a structured format (RDF) using shared vocabularies (FOAT, SIOC, MOAT, etc.)Structured querying of tweetsSubscribe to a stream of tweets that match a given queryReal-time delivery of streaming data.More: TWARQL

37. Twarql Architecture

38. Back to the ScenarioGive me a stream of locations where Kinect is being mentioned nowGive me all people that have said negative things about Kinect

39. Dynamic Domain Models for Semantic Analysis of Real-Time DataakaContinuous Semantics

40. MotivationSemantic processing using a model of the domainBut it is difficult to model dynamic domains on social webspontaneous (arising suddenly)real-time data requiring continuous searching and analysisdistributed participants with fragmented and opinionated informationdiverse viewpoints involving topical or contentious subjectsfeature context colored by local knowledge as well as perceptions based on different observations and their socio-cultural background.More: Continuous Semantics

41. Dynamic Model CreationHeliopolis is a suburb of Cairo.

42. Dynamic Evolving Models to underpin Semantics“Reports from Azadi Square - 4 people killed by police, people killed police who shot. More shots being fired #iranelections”“Both Ahmadinejad& Mousavideclare victory in Iranian Elections.”“situation in tehran University is so worrisome. police have attacked to girls dormitory #tehran #iranelection”EventsJune 13 2009June 15 2009June 12 2009Ahmadinejad & Mousavi are politicians in IranTehran University is a University in IranAzadi Square is a city square in TehranKey phrasesModels

43. Sentiment/Opinion Extraction

44. ChallengesDomain/Topic-dependency: spotting the target of the sentiment is as important as finding sentiment itselfE.g., “longriver” (no sentiment), “long battery life” (positive), or “long time for downloading” (negative).Context-awareness : encoding the context information into the extracted sentimentE.g., “mustwatch a movie today” (no sentiment) and “this movie is a must see” (positive). Informal language (abbreviations, misspelling, slang...): using Urban Dictionary

45. The Usage of Background Knowledge

46. Real Time Feature Stream

47. Real-Time Sensor, Social, Multi-media data2010’sDynamic User Generated Content2000’s1990’sStatic Document and files Web DATA evolved over time

48. Semantic AbstractionDoes -1,-2,-4,-4 make any sense to you?Overwhelming amount of raw sensor data doesn’t make much sense to decision makersFreezing temperatureRainWhat if the data is from sensors on a highway?Freezing temperature + rain => icy roadClose the highway? OR Spread salt on the road to melt ice?So what?Semantic Sensor Web Demo

49. A cross-country flight from New York to Los Angeles on a Boeing 737 plane generates a massive 240 terabytes of data- GigaOmni MediaBut a pilot or a ground engineer at the destination is interested in very smallnumber of events and associated observational data that are relevant to their work.Higginbotham, S. (2010, September). Sensor Networks Top Social Networks for Big Data.Gigaom.com. http://gigaom.com/cloud/sensor-networks-top-social-networks-for-big-data-2/.49

50. Background KnowledgeBlizzardRain StormABSTRACTIONHuge amount of Raw Sensor DataFeatures representing Real-World eventsAbstractionMore: Semantic Sensor Web

51. Weather Alert ApplicationDetection of events, such as blizzards, from weather station observations

52. EvaluationData Used: Nevada Blizzard (April 1st – April 6th)Show me which places had blizzard in past 24 hrs70% Data clear30% Feature Observed

53. Evaluation (cont.)Abstraction can:Make sense out of raw dataGreatly reduce the size of dataData Used: Nevada Blizzard (April 1st – April 6th)Amount of Raw Sensor DataAmount of Abstraction DataSemantic Scalability

54. Traffic ApplicationReasons can be found via other types data: tweets, news papers, etc.Sensor data: 10 passing cars per minuteWhat might be the reason?

55. Integration and Abstraction of Traffic DataStratified explanationKnowledge Model Empowered AbstractionKnowledge ModelsDifferent types of Data

56. Take Home MessageAmount of citizen sensing (and machine sensing) data is huge, varied, and growing rapidly. Search and Sift won’t work.

57. Take Home Message (Cont.)Semantics play a key role in refering "meaning" behind the data. Requires progress from keywords -> entities -> relationships -> events, from raw data to human-centric abstractions.

58. Take Home Message (Cont.)Wide variety of semantic models and KBs(vocabularies, social dictionaries, community created semi-structured knowledge, domain-specific datasets, ontologies)empower semantic solutions. This can lead to Semantic Scalability – scalability that is meaningful to human activities and decision making.

59. Interested in more?Kno.e.sis Wiki for the following and more:Computing for Human ExperienceContinuous Semantics to Analyze Real-Time DataSemantic Modeling for Cloud ComputingCitizen Sensing, Social Signals, and Enriching Human ExperienceSemantics-Empowered Social ComputingSemantic Sensor Web Traveling the Semantic Web through Space, Theme and Time Relationship Web: Blazing Semantic Trails between Web Resources SA-REST: Semantically Interoperable and Easier-to-Use Services and MashupsSemantically Annotating a Web ServiceTutorial: Citizen Sensor Data Mining, Social Media Analytics and Development Centric Web Applications (WWW2011)Partial Funding: NSF (Semantic Discovery: IIS: 071441, Spatio Temporal Thematic: IIS-0842129), AFRL and DAGSI (Semantic Sensor Web), Microsoft Research (Semantic Search) and IBM Research (Analysis of Social Media Content),and HP Researh(Knowledge Extraction from Community-Generated Content).