Facets of applied smw
Upcoming SlideShare
Loading in...5
×
 

Facets of applied smw

on

  • 4,498 views

Smwcon fall 2011 tutorial #4...

Smwcon fall 2011 tutorial #4
The Facets of Applied Semantic MediaWiki
It covers jumpstart wiki with bundles, packages, deployment, customization, extensions, visualization, data i/o, tips and tricks, integration, workflow, project management and knowledge processing examples.

Statistics

Views

Total Views
4,498
Views on SlideShare
4,495
Embed Views
3

Actions

Likes
4
Downloads
26
Comments
0

3 Embeds 3

https://twitter.com 1
http://www.linkedin.com 1
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • I’m Jesse Wang. Studied Computer Science in universities, worked as a software engineer before (in Microsoft and Vulcan).Started working on Semantic MediaWiki in 2008.Semantic MediaWiki ++ on application side of Semantic MediaWiki Project Halo sponsors Semantic MediaWiki development.Dr. Mark Greaves directs the whole program in Vulcan Inc.Without it, Semantic MediaWiki wouldn’t be what it is today. I wouldn’t be here having the privilege talking to you.Paul Allen has the vision. ----- Meeting Notes (9/21/11 13:06) -----I guess you don't really need a celebrity's endorsement.but having a succesfully technology legend's support is definitely something nicer to have.
  • A set of author-selected extensionsA fixed set of extensions, few customization optionsSimplified download and configuration processStill need manual configurationUsually up-to-date and work well togetherCommunity tested and supported
  • Features:A set of (fixed) company selected extensionsProfessionally documented and testedUsually weeks after major release of latest SMWSimplified installation and configurationWindows installer and VM images are greatYou may be curious about what it does exactly…
  • Deployment Framework is growing fast. At this moment, your complete needs may or may not be satisfied by Deployment Framework, but please keep an eye on it, because if it is growing this fast, in the not-so-far-away future, you’ll get all you want from it.
  • A few times, you found what you want basicallyMost times, you settle or work around on what you haveEvery time, you wish it is easier
  • Counter information overload with visualizations
  • WikiTags is here to bridge semantic wikis with more potential users, such as users of Microsoft Word, Outlook and Excel, with Microsoft SmartTag technology.

Facets of applied smw Facets of applied smw Presentation Transcript

  • Facets of Applied Semantic MediaWiki ++
    SMWCon Fall 2011 Tutorial #4
    2011.09.21
  • About
    This tutorial is intended to be used for people who already had the basic knowledge about Semantic MediaWiki.
    Me: http://semantic-mediawiki.org/wiki/User:Jesse
    SMW++ : Ontoprise, TeamMersion, Vulcan, etc.
    Project Halo: http://projecthalo.com/
    Dr. Mark Greaves
    Vulcan Inc.
    Paul Allen, Idea Man
  • Agenda
    A high-level broad view
  • Jumpstart
  • Using a Bundle
    Semantic Bundle
    http://www.mediawiki.org/wiki/Semantic_Bundle
    Features: (Pros and Cons)
    A set of author-selected extensions
    A fixed set of extensions, few customization options
    Simplified download and configuration process
    Still need manual configuration
    Usually up-to-date and work well together
    Community tested and supported
  • Installation Package
    SMW+ Package
    http://smwforum.ontoprise.com/smwforum/index.php/Help:SMW%2B
    Features:
    A set of (fixed) company selected extensions
    Professionally documented and tested
    Usually weeks after major release of latest SMW
    Simplified installation and configuration
    Windows installer and VM images are great
    You may be curious about what it does exactly…
  • Deployment Framework
    Really easy to use and flexible
    Functionality (extensions) growing fast
    Saves lots of time of maintenance
    Clicks rather than lots of commands
    With the benefit of efficiency, quality and consistency
    Limited by the extensions and packages in it
    Supported extensions and packages are growing fast
  • Developer Deployment
    When you want a set of custom extensions, you want
    Your
    OWN
    deployment
    mechanism!
  • Deployment Practices
    Your own codebase, with checkpoints
    SVN or Git to get the base, and then
    Customize the wiki
    Script or Manual or Mixed
    A third-party deployment framework and repository
    Ontoprise: http://smwforum.ontoprise.com/smwforum/index.php/Help:Deployment_Framework
    RPI: http://code.google.com/p/smwbp/wiki/setup_wiki
    Referata: http://smw.referata.com/wiki/Category:Packages
  • Build System
    You own it!
    Your own selection of extensions
    And versions of them
    Own patches, scripts and templates
    Vulcan: Internal GIT repositories of completed build
    Versioning and branches between projects
    Other examples:
    RPI Best Practice (Google code)
    Hudson build (PNNL)
  • Future Trend
    A public wiki installation repository
    with lots of packages
    Including necessary data
    Semantically marked up
    Well documented (or even reviewed)
    Easy to find and use
    Federated wiki package store (Wiki AppStore)
    Call for Participation
  • Customization
  • What to Customize
    Skins
    Extensions
    Schema and Data
    Templates and forms
    Anything else you can code… 
  • Skins
    How to customize a skin
    Create your skin file in /skins folder within your wiki directory, like Ontoskin.php
    Finish this skin files following Ontoskin.php skin files
    There are two classes in this skin files:
    Inherit from SkinTemplate, set the CSS and template filter.
    Inherit from QuickTemplate, set the UI style
    Create skin folder with your skin name in /skins
    Copy image and CSS files to your skin folder
    Use your own skin
    In LocalSettings.php set
    $wgDefaultSkin = 'wiking_skin'
  • Seahawks Video Skin
  • Skin Customization Example
    Vulcan Development Wiki and Seahawk wiki.
    We have customized these two skins for our projects and the key is the method of QuickTemplate. In this method, you can decide what to show and how it shows on your wiki
    You can customize wiki site like menu links, page titles, CSS and JavaScript files in the data properties of QuickTemplate
    For example, show wiki side bar in your skin
    <?phpforeach ($this->data['sidebar'] as $bar => $cont) { ?>
    <li>
    <a href="#"><h5><?php $out = wfMsg( $bar );></h5></a>
    <ul>
    <?phpforeach($cont as $key => $val) { ?>
    <li id="<?php echo $val['id'] ?>" >
    <a href="<?php echo $val['href'] ?>"><?php echo $val['text'] ?></a>
    </li>
    <?php } ?>
    </ul>
    </li>
    <?php } ?>
  • Personalized Skin
    Skin supports customization via __USERNAME__
    User can have personalized styles within a skin
    http://www.mediawiki.org/wiki/Manual:Skin_configuration#User_CSS
    Tip: A way to turn off Wikipedia banner ad:
    http://en.wikipedia.org/wiki/User:Jesseone/vector.css
  • Schema Design
  • “Just Do It!”
    Easy to start
    Flat, large ontology, at beginning
    Many properties only valid in certain cases
    May have similar/duplicate items
    But it’ll get better over time!
    Seahawks Video Wiki
  • Designer Ontology
    Professionally generated after rounds of discussions
    Time consuming
    More comprehensive and thoroughly designed
    More data (restriction) than required
    Data (ontology) importer isn’t powerful to fully support all features in the designing tool (Protégé)
    Biology Ontology Wiki
  • The Middle Way
    Practical schema design
    Plan and Explore
    Think, adapt, and evolve
    Our Agile Project Management Wiki
  • N-ary Relations
    Type:Record in SMW
    Semantic Internal Object extension
    Adding an object relation in the middle (Reification)
    Using parameterized properties (Example: the third turn wiki)
    Using a triple store (external)
  • Extensions
  • Find Extensions
    Where to find extensions
    http://www.mediawiki.org/wiki/Extension_Matrix
    By supported version (1.16, 1.17, …)
    By status (beta, stable, …)
    By type (API, AJAX, hook, user rights…)
    By new-ness (sort by most recently created time)
    By talk page new-ness…
    By most updated versions…
    And of course using search tools from G.Y.M…
  • Get the Extensions!
    Some essential MediaWiki extensions!
    http://semantic-mediawiki.org/wiki/Help:MediaWiki_extensions
    And quite a few Semantic MediaWiki extensions:
    http://semantic-mediawiki.org/wiki/Help:SMW_extensions
    Find what you want
    Not exactly? Then patch it!
    Nothing close? Then build it!
    Or find people to build it 
  • Our Experiences
    We built a few semantic wikis (2008-2011)
    Sci-Fi movie demo
    Semantic Football (Entertainment)
    NGT – a work-order handling system
    Ultrapedia – an Analytical Encylopedia
    Wiking agile project management
    Biology ontology
    Stanford class wiki
    Neurowiki …
  • Patches Often Needed
    Many features are sort of there, some still not.
    Popular extensions (e.g. SF, SRF, etc.) often need patches or tuning
    Example: SF Auto-completion (over the years)
    Fixed set of allowed values
    All values in a category
    All values having a specific property
    All values from a custom query
  • New Features Added
    Real-time notification (Semantic NotifyMe, 2008)
    Aggregation (average, max, sum, etc.) (2009)
    Video merging 
    Open Flash Charts (OFC) SRF (2009-2010)
    Strict > and < comparison (2010)
    Exhibit enhancements (webit) (2009)
    New formats: Runway, Time Plot, etc.
    Mapping: unique ID requirements
    Geocoding
    Semantic WikiTags (2009)
    Semantic Connector (2010)
  • Two of Our Latest Extensions
    Wiki Object Model
    Enable third parties to consume wiki data via APIs
    APIs (and URLs) for easy and flexible manipulation of wiki page (and data)
    At a finer granularity
    Widget Designer / Page Creator (prototype)
    To lower structured page creation dramatically
    To enable data-driven application building
    Come to my Semantic Page Creator talk on Friday
  • Why Wiki Object Model
    Strong needs for access the structured data and unstructured data in the wiki as a data store
    From internal and external applications’ point of view
    A DOM-like approach is good choice
    Xpath is a reasonable tool and standard
    Granularity offers flexibility and power
    Easier to operate on known objects and calling REST APIs rather than parsing and handling wiki texts
    Common gateway for both internal and external apps
  • Wiki Object Model Objects
    Category
    Link
    Property
    Text
    Magic word
    HTML tag
    Sentence
    Word*
    Image
    31
    Page
    Section
    Parameter
    Parameter value
    Template
    Template field
    Parser function
    List item
    Table
    Table cell
    * Not yet implemented as of September 2011
  • Demo of WOM
    • Demo at http://wiking.vulcan.com/dev_sandbox/
    • Demo 1: APIs
    • The API is part of MediaWiki API, with our additions to allow get and set (read/write) the content of a wiki page
    • http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Apis
    • Demo 2: Inline Editor
    • A simple inline editor (AJAX styled) to let user modify a specific portion of the wiki page
    • Based on http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Functions
    32
  • Security Extensions
    Many choices = nothing is perfect
    Options: from very basic to very advanced:
    No Security
    MediaWiki Built-in Groups
    Simple Security (Page Security)
    HaloACL
    Features
    Restrictions
  • A Short Case Study
    Proposal Management System (Wiki)
    Users (including privileged users) submit proposals
    Reviewers (privileged users) rate and review them
    Reviewers discuss and make decision and assign
    What security extension should it employ?
    HaloACL? – the most powerful ACL out there
    SimpleSecurity? – simple and effective
    Or MediaWiki built-in groups? – simplest
    MediaWiki Groups
  • Visualizations
    Semantic Result Formatter
    Format = ???
    {Param} = ???
  • Visualization: Our Path
    2007: SRF used to only have few options
    2008-2009: Exhibit seemed cool
    Ported webit
    Fixed some bugs and enhanced features
    Runway, TimePlot, multiple rows in Map view etc.
    2009-2010: Open Flash Chart
    More bars, pies, lines, scatter plot…
    Interaction
    Richer information display
    2010-11: TabbedHighChart, OFC/ScatterPlot
  • Comparing SRF Options
    Google Charts (pie and bar)
    Google does it! (G gets your data, no offline choice…)
    Static bar and pie, no color options…
    Exhibit
    More choices (timeline, map, facet, runway…)
    Size limit; Browser compatibility issues
    Flash-based: OFC (Open Flash Chart), High Chart
    Many bars, charts, plots, even scattered plots
    Flash required
  • Sidebar Customization
    Basic: MediaWiki:Sidebar
    Enhanced: semantic queries, and tree views
    Advanced: use variables together with queries
    Custom: Using an extension: DynamicSidebar
    Customize your own sidebar at User:<username>/Sidebar
    http://www.mediawiki.org/wiki/Extension:DynamicSidebar
    Example: http://wiking.vulcan.com/dev/
  • Current Sprint in Sidebar
    *Current sprint
    {{#ask: [[Category:Project sprints]]
    [[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]
    [[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]
    |format=template
    |template=Sidebar query template
    |link=none
    |}}
  • My Active Sprint Tasks in Sidebar
    *My tasks
    {{#ask: [[Category:Project tasks]]
    [[Project task owner::<q>[[User alias::{{CURRENTUSER}}]]</q>]]
    [[Project story::<q>
    [[Project sprint::<q>[[Category:Project sprints]]
    [[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]
    [[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]</q>]]
    </q>]]
    [[Project task status::In progress||Not started]]
    |format=template
    |template=Sidebar query template
    |link=none
    |}}
  • More Extension Examples
    Twitter Feed
    http://www.mediawikiwidgets.org/Twitter_Search (example)
    Flickr Feed
    http://www.mediawikiwidgets.org/Flickr
    Amazon Carousel
    https://widgets.amazon.com/Amazon-Carousel-Widget/
    Google Analytics
    http://www.mediawiki.org/wiki/Extension:Google_Analytics
    Optify Analytics
    http://www.optify.net/
  • Templates and Forms
    MediaWiki template is essential for its structured data handling
    Editing, especially Template editing, is not trivial
    Semantic Forms extension greatly lowers the bar
    Customizing templates and forms can provide very visually appealing pages
  • Form with a Style
    http://www.thethirdturn.com/w/index.php?title=Form:Driver&action=edit
  • Forms with autocompletion
    Advanced Auto-Completion on Customized Query Results
    Basic Auto-Completion on Category Values
  • The Sky is the Limit
  • Integration
  • Steps
    Know what to build
    Customers, customers, customers!
    Get your collection of extensions…
    Customize your wiki (template, forms)
    Handle the data (schema, query, API, …)
  • Case 1. Proper Wiki
    Focus on Skin, Form and Template
    Steps:
    Pick a (default) skin
    Design some forms
    And templates (queries) behind the forms
    Build starter pages and Go!
  • Biology Ontology Wiki
    People need discuss about the details related to construct a biology ontology
    Classes (categories)
    Properties
    Relations
    Textbook analysis
    Discussion
    Forms necessary to keep content tidy
  • Ex: Property Form
  • Forms on Namespace
    http://www.mediawiki.org/wiki/Extension:Semantic_Forms#Based_on_namespace
  • Case 2: Simple Workflow App
    Idea: go from one form to another form
    Need form links in the template
    May need user rights management
  • Proposal Review Workflow Example
    User submits a proposal via a form
    Fill in basic information about the proposal
    Set a field in the form to be, say, “New”
    User doesn’t see further information about “Reviewer” or “Owner”, etc.
    Semantic notification (email and RSS feed) facilitates communication
    Reviewer finds the form in a canned query
    Reviewer changes the status to, say, “Approved”
    Reviewer field (say “Reviewed by”) is (automatically) set
    Then reviewer assigns the proposal to a owner, say, “Alice”
    The owner (“Alice”) now sees it and can start work on it
    Changes the status to “Active”
    Adds a start date or maybe an estimated end date too
  • Ways to Handle Workflow
    Add a link to open another form
    Force edit “review” using another form (e.g. reviewer form)
    When saving the new form, the category is changed
    Now we have a different category  a different default form
    Change a value to include or exclude to another template
    May need #if (or #switch) statement in template to change (or include/exclude) template values
    Use “Page has default form” property (Semantic Forms extension)
  • More Workflow
    Good summary by YaronKoren
    SMWCon Spring 2011 TalkSMW and the Workflow Puzzle
    My Semantic Page Creator Talk on Friday 9/23
    Example: A simple HelpDesk wiki
  • Case 3: WikingDev Wiki
    Our project management wiki
    We use it for *all* things it can do:
    Proposals, ideas, email messages, features, bugs
    Agile development: milestones, sprints, stories, tasks
    Code/Feature association: SVN mapped to tasks/bugs
    Progress analysis: burn-down charts, work calendar
    Collaboration, Documentation, Demonstration
  • Examples of Data I/O
    Wiking Development Wiki
    An Imported email
    http://wiking.vulcan.com/dev/index.php/Demo_scenarios
    An email imported as a Project Bug
    http://wiking.vulcan.com/dev/index.php/Issue_11886_Insert_a_single_property
    A task uploaded via Outlook with multiple commits
    http://wiking.vulcan.com/dev/index.php/WikiTags_Release_Structure
  • Microsoft Office Connector
    Leverage Microsoft Office applications and technology
    Bring SMW info to Office applications on-demand
    API for data I/O: add and modify wiki data within Microsoft Office
    Utilize semantics to improve relevance
    Smart actions for semantic properties
    API
    API
    Connections
    Smart
    Jesse Wang | SemTech 2010
  • Our Agile Project Management (Scrum)
    60
    Wiking
    Jesse Wang | SMWCon Amsterdam 2010
  • Wiki Data I/O
    Wiking development wiki has two external applications
    Subversion integration
    Microsoft Office integration
    Extensions to support it
    Semantic Wikitags
    Semantic Connector
  • Data I/O Extensions
    MediaWiki API – very basic read/write
    Page Object Model
    First Data I/O extension to allow access some wiki structure data (links, title, basic template field)
    SMWWriter
    Based on POM, support annotations (semantic properties)
    Data API extension in SMW+
    Semantic WikiTags and Semantic Connector
    Wiki Object Model
    A powerful and comprehensive object model (Data API)
    Read/write at finer object level (sentences, template parameters)
  • Case Study: Battle-space Luminary System
    Discover when New Information represents a change in understanding of entities
    Discovery of explicit entity links, implicit relationships
    Large Volumes of Data in various formats
    Unstructured news articles
    Tactical Reports, Field Intelligence
    Structured Database Information
    Use Wiki Pages to represent current knowledge about an entity – “what we know”
    Domain Ontology to represent domain of information – “what we want to know”
    Issue Alerts when Significant Events occur
    New information according to category
    Changing information on topics of interest
    Need to send information to various devices – cell phones, email, etc.
    63
  • System Design
    Wiki Configuration
    Semantic MediaWiki: Large developer community, active development, open source. Wikipedia uses MediaWiki, so scalability and performance are important.
    Semantic Results Format: Provides various rich media displays of semantic information, including graphs, timelines, maps
    Semantic Forms: Provides convenient user interface for entering semantic data into wiki, avoiding cumbersome wikitext
    Semantic Notifications: Enables sending of notifications when results of semantic query change.
    Domain Ontology
    Created OWL Ontology for Terrorism
    Semantic Parsing, Extraction, Reasoning
    Java Process using various Open-Source Toolkits
    Rapid plugin of new technologies
    Multiple Data Sources supported
    64
  • Sample Content Page
    65
  • Wiki Content Design
    Use Templates to Ensure Consistent Look-and-Feel
    Templates Correspond to Ontology Classes
    Fields within Templates correspond to Properties within Ontology
    Rich Content Visualizations derived in consistent way
    Hierarchical Categories match Class Hierarchy within Ontology
    Ensures Validity for Properties
    Category included on each Template page to ensure consistency
    FormsProvide ability for users to enter data directly into wiki without knowing Wiki Text
    Each form corresponds to a Template
    Fields within forms correspond to the fields/properties within the Template
    GUI can include auto-completion
    Created Page immediately linked semantically to rest of Wiki
    66
  • Sample Visualizations
    67
    UI enables notifications based on results of query – message sent when visualization changes
    Visualizations automatically created w/o user edit
    (tables, timelines, maps, social networks…)
  • Optimize
  • Usability
    MediaWiki Usability Initiative
    Semantic MediaWiki Usability
    Fact Box, Special pages
    Extensions
    Semantic Forms
    One Click Way
    Auto-complete
    SRF
    User Interface
    Example: Wiking Project Task
    http://wiking.vulcan.com/dev/index.php/Rehearsal_of_WE_Demo
  • Performance
    Fundamentals
    http://smwforum.ontoprise.com/smwforum/index.php/Help:Optimizing_response_times_1.5.6
    Caches: squid, apache, parser, query, browser, …
    Global Load Balancing Service
    Batch Process
    RefreshData.php
    Other services:
    memcache
    Triple Store
    Solr
  • Search Engine Optimization
    General SEO rules (Official Google Guide 32pp)
    MediaWiki
    Robot.txt : edit, talk, websvn, …
    Metadata: title, description, media, alt tags…
    Semantic MediaWiki
    RDFa/Rich Snippets/Search Monkey
    For more info:
    Wil Smith: Search Engine Optimization for SMW
  • Summary
    Jumpstart
    Deployment of your bundle/suite/package
    Code, pseudo-code/meta-data, and data
    Customize
    Skins, Styles, Extensions
    Data, Code, Code, Data
    Integrate
    Know the requirements and data
    Put everything together and evolve
  • Acknowledgement
    Jesse Wang | SMWCon Amsterdam 2010
    73
  • Thank you
    Questions and/or comments?
    Slide uploaded at http://www.slideshare.net/jiaxinwang/
    Email me at jessew @ vulcan.comor @aiwang