Zinayida Petrushyna, Ralf Klamma RWTH Aachen University Workshop “Digital social networks”, Munich September 12, 2008 The Troll under the Bridge: Data Management for Huge Web Science Mediabases

Agenda Motivation & Problem definition Data Management for Web Science Crawling: Watchers Analysis: Patterns Visualization: Graphs Conclusion Outlook

Motivation & Problem definition

Data Management for Web Science

Crawling: Watchers

Analysis: Patterns

Visualization: Graphs

Conclusion

Outlook

Data Management issues in Web Science Interoperable formats XML based – Wikis , RSS Feeds, Microformat SQL based – Deep Web Text based – Websites, Forums Non-continuous analysis Crawling vs. Dumps Special purpose vs. General purpose Aggregation level is not possible to achieve Data warehouses Theoretical considerations of agency – Actor network theory

Interoperable formats

XML based – Wikis , RSS Feeds, Microformat

SQL based – Deep Web

Text based – Websites, Forums

Non-continuous analysis

Crawling vs. Dumps

Special purpose vs. General purpose

Aggregation level is not possible to achieve

Data warehouses

Theoretical considerations of agency – Actor network theory

Data Model for the Web 2.0 Latour: On Recalling ANT , 1999

Mediabase A Mediabase is a six-tuple graph

A Mediabase is a six-tuple graph

Actors in the Mediabase

Crawling Technologies Mix of dumps (Wikis) and special purpose crawlers:

Trolls under the Bridge What is a disturbance, e.g. a troll? Sensing an incompatibility between theories exposed and theories-in-use Disturbances are starting points of learning processes Disturbances disturb, prevent … but they are creating reflection Disturbances are hard to detect or to forecast

What is a disturbance, e.g. a troll?

Sensing an incompatibility between theories exposed and theories-in-use

Disturbances are starting points of learning processes

Disturbances disturb, prevent … but they are creating reflection

Disturbances are hard to detect or to forecast

Complex Troll Pattern in Basic Notation

Complex Troll Pattern in Basic Notation 10 )

Pattern Language Variables – simple variables ( troll, thread ), properties ( thread.author ) and set variables ( v 1 ,…,v n ). Operations Arithmetic (+, -, *, / ) Aggregate ( SUM , COUNT , AVERAGE ) Logical (&, |, ~, FORALL and EXISTS ) Comparison ( = , != , > , < ). Rules for variable binding Simple variables – pattern parameters, actors or set variables Properties – actor properties or relations Set variables – actors Interpreted by a finite state automaton

Variables – simple variables ( troll, thread ), properties ( thread.author ) and set variables ( v 1 ,…,v n ).

Operations

Arithmetic (+, -, *, / )

Aggregate ( SUM , COUNT , AVERAGE )

Logical (&, |, ~, FORALL and EXISTS )

Comparison ( = , != , > , < ).

Rules for variable binding

Simple variables – pattern parameters, actors or set variables

Properties – actor properties or relations

Set variables – actors

Interpreted by a finite state automaton

Pattern Language for PALADIN : Example Troll Troll Pattern : This pattern tries to discover the cases when a troll exists in a digital social network. A troll in the network is considered a disturbance. Disturbance : (EXISTS [medium | medium.affordance = threadArtefact]) & (EXISTS [troll |(EXISTS [thread | (thread.author = troll) & (COUNT [message | (message.author = troll) & (message.posted = thread)]) > minPosts]) & (~EXISTS[ thread 1 , message 1 | (thread 1 .author 1 != troll) & (message 1 .author = troll & message 1 .posted = thread 1 ]))])]) Forces : medium; troll; network; member; thread; message; url Force Relations : neighbour(troll, member); own thread(troll, thread) Solution : No attention must be paid to the discussions started by the troll . Rationale : The troll needs attention to continue its activities. If no attention is paid, he/she will stop participating in the discussions. Pattern Relations : Associates Spammer pattern.

Troll Pattern : This pattern tries to discover the cases when a troll exists in a digital social network. A troll in the network is considered a disturbance.

Disturbance :

(EXISTS [medium | medium.affordance = threadArtefact]) &

(EXISTS [troll |(EXISTS [thread | (thread.author = troll) &

(COUNT [message | (message.author = troll) &

(message.posted = thread)]) > minPosts]) &

(~EXISTS[ thread 1 , message 1 | (thread 1 .author 1 != troll) &

(message 1 .author = troll & message 1 .posted = thread 1 ]))])])

Forces : medium; troll; network; member; thread; message; url

Force Relations : neighbour(troll, member); own thread(troll, thread)

Solution : No attention must be paid to the discussions started by the troll .

Rationale : The troll needs attention to continue its activities. If no attention is paid, he/she will stop participating in the discussions.

Pattern Relations : Associates Spammer pattern.

Pattern Discovery Process Digital Social Network 1. Set pattern parameters 2. Instantiate disturbances 3. Evaluate disturbances 4a. Change Pattern Parameters 4b. Apply Pattern Solution Pattern Disturbance Variables Pattern Template Disturbance Variables Pattern Parameters Pattern Template Instance Pattern Instance Disturbance Variables Pattern Parameters Forces Force Relations Rationale Dependencies Description Solution Pattern Relations Disturbance Instances Variables Pattern Parameters

Visualization

Conclusions and Outlook Homogeneous data management Pattern language for disturbance analysis Graph-based visualization Data uncertainty and inconsistent data Goals and intentions of analysts Dynamic Mediabase visualization

Homogeneous data management

Pattern language for disturbance analysis

Graph-based visualization

Data uncertainty and inconsistent data

Goals and intentions of analysts

Dynamic Mediabase visualization