Sentiment mining- The Design and Implementation of an Internet PublicOpinion Monitoring and Analysing System
THE DESIGN AND IMPLEMENTATION OF AN INTERNET PUBLIC
OPINION MONITORING AND ANALYSING SYSTEM
The Curious case
THE GREEK GOD WHO LOVES TO TRAVEL HAS BEEN
REINCARNATED IN INDIA.
A complete framework of an Internet Public Opinion Monitoring and Analyzing System (IOPMAS)
provided in paper
System composed of (4):
◦ Web crawler
◦ Information processing.
◦ Public opinion information analyzing and mining module.
◦ Information services module.
System can collect web pages from the entire web space including news groups, portal websites,
forums, BBS, blogs, microblogging websites etc.
System gives public opinion information analysis results thorough processing and analyzing collected
This system can help supervisors to timely monitor the concerned public opinion and ‘’guide them’’.
Introduction- Public Opinion
◦ society and political attitude toward the social administration in certain social space.
250 Mn users are online in India alone, half of them are teens or younger.
Exponentially increasing use of social media to voice opinions
5 characteristics of internet public opinion:
◦ 1. Diversity of Information Sources
◦ 2. Massive amount of information opinion
◦ 3. Burstiness of internet public opinion
◦ 4. Public opinion in response to emergencies
◦ 5. Set of massive structured, semi structured and unstructured information
EVOLUTION & HISTORY OF SENTIMENT
Biber, 1998- Classifying document according to source or source style, with statistically detected
stylistic variation as important cue.
Karlgren and cutting, 1994- Determining the genre of texts, like subjective or ‘editorial’ texts.
Wiebe, et al., 2001- Explicitly finding features that less subjective knowledge is being used.
Turnkey and Littman, 2002- Classifying the semantic orientation of words or phrases using
◦ Applied specific unsupervised learning technique based on mutual information between document
phrases and the words ‘’excellent’’ and ‘’poor’’
◦ This mutual information computed by using statistics gathered by search engines (crawlers).
BACKGROUND OF IPOMAS
Past method based on:
◦ Traditional and manual search engine.
◦ Demand unmanaged, such as full coverage, unstructured information and rapid detection.
◦ Hard to process the unstructured information.
Internet public opinion monitoring and analysing system:
◦ Full collection and timely detection.
◦ Effective processing of semi structured and structured information.
◦ Displayed effectively.
In this paper, an internet public opinion monitoring and analyzing system are proposed and the
corresponding system is designed and implemented.
The Framework of the Monitoring and Analyzing System
◦ Internet public opinion information collection
◦ Pre-processing of information
◦ Information processing and mining
◦ Public opinion information service
Data files: Web database file, Series of
segment file and Index file.
Crawler: an Internet bot that
systematically browses the World Wide
Web, typically for the purpose of Web
Web parsing- Extracting information
from websites by simulating human
exploration of world wide web.
Reduplicate Web Elimination
Web purification- Noise reduction
Speech text attributes analysis
Tokenization- Segmenting sentences into
Part-of-speech tagging- Speech to text
conversion and indexing the information
Key and useful information
from huge amount of data
TDT-Topic Detection and
Named entity identification
Text classification algorithm
Text clustering algorithm
Emotional tendency analysis
Architecture of IOPMAS
DPSB (Data Processing Service Bus)
. . . . . .
Data Accessing Bus (DAB)
RESULTS AND DISCUSSIONS:
IPOMS based on SOA (service-oriented-architecture) and ESB (enterprise-
Leading technology, functional and practical, good scalability.
The SOA realize a loose coupling between components by defining good
interfaces, ensures good scalability, reusability and maintainability.
A functional framework and the overall software architecture of the internet
public opinion monitoring and analyzing system.
Main technology in the system.
Public Emergencies Security Alert and Better Control of Situation
Real-time Feedback on Administration & Policies
Opinion study based on polarity of the
Opinion Mining for Market
Development and New Product
Sentiment Analysis using fuzzy logic
Alexandra, B.,MONTOYO, A.: Feature Dependent Method for Opinion Mining and Classification.
Natural Language Processing and Knowledge Engineering (2008)
Yee,W., Vidyasagar,P.:A Review of Opinion Mining and Sentiment Classification Framework in Social
Networks.Digital Ecosystems and Technologies (2009)
Shaidah,J., Hejab, M.:Applying Fuzzy Sets for Opinion Mining.Computer Applications Technology
Li, Xiu and Gao Liping, The Design and Implementation of an Internet Public Opinion Monitoring and
Analyzing System, 2013 International Conference on Service Science.
Sethi, Pranay, Public Opinion Aggregation by Annotation and Tagging
of Online News Stories, iConference 2013