Online Forums vs. Social Networks: Two Case Studies to support eGovernment with Topic Opinion Analysis

  • 325 views
Uploaded on

 

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
325
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Figure 2 shows the applied process model how stakeholders were engaged, both to determine requirements and to evaluate the toolbox. The idea behind this approach is to identify potential use cases that are in the end users’ daily working lives. These cases are therefore of value to the end user, and can be used for validating the toolbox and its analysis results. Figure 1 shows two examples of such use cases (“HeadsUp” and “social networks”), and these are discussed throughout the paper to illustrate how the topic opinion analysis can be applied in everyday politics.The top row in Figure 2 shows users on the Internet (the digital society) – for instance users of online forums or social networks. The second row shows stakeholders, and how they interact with the users on the Internet. In the use cases, the stakeholders already perform (often manual) analyses on the data they get from citizens on social networks. The results of their existing analyses are shown in the bottom row – here we call these data the “control group”. The control group is compared with the toolbox’s analyses of the same data. For instance the operator of the HeadsUp discussion forum (cp. left) analyses the forum discussions manually to get an insight on the debate. Another example is the policy maker (cp. right) who extracts topics from social networks to get insight into the discussion.

Transcript

  • 1. Online Forums vs. Social Networks: Two Case Studies to support eGovernment with Topic Opinion Analysis Timo Wandhöfer, Beccy Allen, Steve Taylor, Paul Walland, Sergej Sizov IFIP EGOV13, University of Koblenz
  • 2. Contents • Background & Context • Applied Process Model • Online Forums: The HeadsUp Case • Social Networks: The Facebook/Twitter Case • General Findings • Conclusions & Further Work
  • 3. Background
  • 4. Context: WeGov EU-Project • Allow policy makers to interact directly with citizens using Social Network Sites – Use the tools the citizens already use • Find and understand people’s opinions • Become part of the discussion • Open dialog • Respect privacy • Encourage trust
  • 5. Problem Statement • In many cases, discussion tracks in social media become long and complex • Stakeholders (WeGov) are often interested in gaining a quick overview of a discussion • However, completely reading hundreds of posts is too time-consuming to be practical • There is thus a huge need to summarize the discussion tracks…
  • 6. WeGov Toolbox: Topic Opinion Analysis
  • 7. Is this really useful within everyday life?
  • 8. Applied Process Model
  • 9. Online Forums: The HeadsUp Case
  • 10. Online Forums • Civil society groups run forums and blogs to connect with their members and supporters • Analysing the themes of discussions is often beyond the organizations resources
  • 11. HeadsUp • Online forums for connecting young people with politics and politicians • Forums ran for a decade (2003-2013) by the Hansard Society • They racked up nearly 10,000 comments from students in 50 forums • 330 parliamentarians & decision-makers have taken part in debates with 11-18 year olds • HeadsUp users' views have been mentioned in debates in both the House of Commons and the House of Lords
  • 12. Methodology • Evaluating the usefulness (accuracy, reliability) with regard to forum data using a real world data set • Independent data (created beforehand) • Each forum was analyzed manually within a report • Three forums of different sizes: – Sex Education – Do you get enough? (36 posts) – Youth Citizenship Commission: are young peopled allergic to politics? (317 posts) – How equal is Britain? (1186 posts) • Toolbox output was compared to the forum reports – Themes of the debate, sentiment, user interface
  • 13. Findings • The toolbox is best at dealing with large quantities of data, amounts that could not be analyzed effectively by a human. • The toolkit performs well on relatively in-depth data - this lends itself to blogs and forums that encourage more considered and less immediate responses. • The toolbox also performed well in showing the nuances between different elements of a wider debate. E.g. the subtopics for “How equal is Britain?” • The toolkit works best when analyzing medium length comments that focus on one issue and when spelling and vocabulary are good.
  • 14. Social Networks: Facebook/Twitter Case
  • 15. Methodology: Intention • The intention of this case study was the validation of usefulness of topic opinion analysis of social media for politics. • Therefore we designed use cases how this technology may support politicians’ everyday life.
  • 16. Methodology: Real Life Applications 1. Local Facebook topics: – Monitoring a sample of at least ten Facebook pages represent a geographical area like an MP’s constituency. – Apply topic opinion analysis to extract the topics that people discuss on the pages. – Each topic is a combination of words that represents a theme of the discussion, and comes with key users, and key comments. 2. Monitoring topics on Twitter: – Identify subtopics e.g. the general debate on climate change covers subtopics like green energy, new kinds of technologies. – Because the results are already filtered by the search, the analysis produce subtopics.
  • 17. Methodology: Stakeholders • Conducted with a number of governmental representatives as end-user stakeholders: – two members of the German Bundestag – four employees that work directly for a member of the German Bundestag – two members of the State Parliament North Rhine- Westphalia – one small German city (Kempten) – one big German city (Cologne) – and with a German state chancellery (Saarland)
  • 18. Methodology: Qualitative Workflow • Input: Crawling of individual data (themes and constituency) • Analysis: Topic Opinion Analysis • Report: Reports covering an individual analysis • Questionnaire: Using individual examples from the report • Follow-up interview: Conducted to receive more in-depth assessments about the results
  • 19. Findings • Sensible and expected topics – All topics from local Facebook pages that were assessed as understandable were known beforehand; the reasons being: WeGov stakeholders are ‘aware of the (online) public area’ – Equal with Twitter • Quality of topics – Better results with Twitter – Less clear topics with local Facebook pages. The reason why 42% of the 110 topics were assessed as understandable topics is due to the fact that policy makers know what’s happening in the area of their electorate • Different meanings for topics – the combination of five words for one topic could have multiple meanings
  • 20. General Findings
  • 21. Potential end-user groups • The toolbox could play an important role in helping to understand feedback across a range of communication channels – small not-for-profit organizations – larger media organizations – as well as politicians and policy (parties, parliaments, bigger city administrations) • Anyway: Experiences with social media and as well a social media strategy in general are necessary!
  • 22. Improvements • A plain explanation of how the algorithm understands and processes data -> ensure users trust the results. An explanation of irregularities. E.g. – Why the same data sometimes yields different results? – Why keywords appear in the order and frequency that they do? • Showing the hidden workings of the toolkit. E.g. – The relative influence of a greater number of key words e.g. via a tag cloud. – Highlighting positive/negative words that contribute to sentiment scores. • Implementing more options for the users to refine the data and customize it to their situation and needs. E.g. – The ability to exclude certain posts or words from analysis. – Splitting up long posts into sections that can be analyzed separately to avoid the conflicting analysis of longer posts.
  • 23. Conclusions • Two case studies how to apply and validate topic opinion analysis – HeadsUp focused on accuracy/reliability – Social network case focused on the usability within the decision- makers’ everyday life • Follow the same process model and show added value / boarder- lines • Within both cases there was already an understanding of what the data were about – we call it the „control group“ • Toolbox performed well in showing the nuances between different elements of a wider debate • With local Facebook pages the quality of results worsen due to the fact of the quality of input data and less political conversations • Qualitative validating process was very effective but time consuming • The Cases protect the rights and privacy of citizens