Facets of applied smw


Published on

Smwcon fall 2011 tutorial #4
The Facets of Applied Semantic MediaWiki
It covers jumpstart wiki with bundles, packages, deployment, customization, extensions, visualization, data i/o, tips and tricks, integration, workflow, project management and knowledge processing examples.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • I’m Jesse Wang. Studied Computer Science in universities, worked as a software engineer before (in Microsoft and Vulcan).Started working on Semantic MediaWiki in 2008.Semantic MediaWiki ++ on application side of Semantic MediaWiki Project Halo sponsors Semantic MediaWiki development.Dr. Mark Greaves directs the whole program in Vulcan Inc.Without it, Semantic MediaWiki wouldn’t be what it is today. I wouldn’t be here having the privilege talking to you.Paul Allen has the vision. ----- Meeting Notes (9/21/11 13:06) -----I guess you don't really need a celebrity's endorsement.but having a succesfully technology legend's support is definitely something nicer to have.
  • A set of author-selected extensionsA fixed set of extensions, few customization optionsSimplified download and configuration processStill need manual configurationUsually up-to-date and work well togetherCommunity tested and supported
  • Features:A set of (fixed) company selected extensionsProfessionally documented and testedUsually weeks after major release of latest SMWSimplified installation and configurationWindows installer and VM images are greatYou may be curious about what it does exactly…
  • Deployment Framework is growing fast. At this moment, your complete needs may or may not be satisfied by Deployment Framework, but please keep an eye on it, because if it is growing this fast, in the not-so-far-away future, you’ll get all you want from it.
  • A few times, you found what you want basicallyMost times, you settle or work around on what you haveEvery time, you wish it is easier
  • Counter information overload with visualizations
  • WikiTags is here to bridge semantic wikis with more potential users, such as users of Microsoft Word, Outlook and Excel, with Microsoft SmartTag technology.
  • Facets of applied smw

    1. 1. Facets of Applied Semantic MediaWiki ++<br />SMWCon Fall 2011 Tutorial #4<br />2011.09.21<br />
    2. 2. About<br />This tutorial is intended to be used for people who already had the basic knowledge about Semantic MediaWiki.<br />Me: http://semantic-mediawiki.org/wiki/User:Jesse<br />SMW++ : Ontoprise, TeamMersion, Vulcan, etc.<br />Project Halo: http://projecthalo.com/<br />Dr. Mark Greaves<br />Vulcan Inc.<br />Paul Allen, Idea Man<br />
    3. 3. Agenda<br />A high-level broad view<br />
    4. 4. Jumpstart<br />
    5. 5. Using a Bundle<br />Semantic Bundle <br />http://www.mediawiki.org/wiki/Semantic_Bundle<br />Features: (Pros and Cons)<br />A set of author-selected extensions<br />A fixed set of extensions, few customization options<br />Simplified download and configuration process<br />Still need manual configuration<br />Usually up-to-date and work well together<br />Community tested and supported<br />
    6. 6. Installation Package<br />SMW+ Package <br />http://smwforum.ontoprise.com/smwforum/index.php/Help:SMW%2B<br />Features:<br />A set of (fixed) company selected extensions<br />Professionally documented and tested<br />Usually weeks after major release of latest SMW<br />Simplified installation and configuration<br />Windows installer and VM images are great<br />You may be curious about what it does exactly…<br />
    7. 7. Deployment Framework<br />Really easy to use and flexible<br />Functionality (extensions) growing fast<br />Saves lots of time of maintenance<br />Clicks rather than lots of commands<br />With the benefit of efficiency, quality and consistency<br />Limited by the extensions and packages in it<br />Supported extensions and packages are growing fast<br />
    8. 8. Developer Deployment<br />When you want a set of custom extensions, you want<br />Your <br />OWN <br />deployment <br />mechanism!<br />
    9. 9. Deployment Practices<br />Your own codebase, with checkpoints<br />SVN or Git to get the base, and then<br />Customize the wiki<br />Script or Manual or Mixed<br />A third-party deployment framework and repository<br />Ontoprise: http://smwforum.ontoprise.com/smwforum/index.php/Help:Deployment_Framework<br />RPI: http://code.google.com/p/smwbp/wiki/setup_wiki<br />Referata: http://smw.referata.com/wiki/Category:Packages<br />
    10. 10. Build System<br />You own it!<br />Your own selection of extensions<br />And versions of them<br />Own patches, scripts and templates<br />Vulcan: Internal GIT repositories of completed build<br />Versioning and branches between projects<br />Other examples:<br />RPI Best Practice (Google code)<br />Hudson build (PNNL)<br />
    11. 11. Future Trend<br />A public wiki installation repository<br />with lots of packages<br />Including necessary data<br />Semantically marked up<br />Well documented (or even reviewed)<br />Easy to find and use<br />Federated wiki package store (Wiki AppStore)<br />Call for Participation<br />
    12. 12. Customization<br />
    13. 13. What to Customize<br />Skins<br />Extensions<br />Schema and Data<br />Templates and forms<br />Anything else you can code… <br />
    14. 14. Skins<br />How to customize a skin<br />Create your skin file in /skins folder within your wiki directory, like Ontoskin.php<br />Finish this skin files following Ontoskin.php skin files<br />There are two classes in this skin files:<br />Inherit from SkinTemplate, set the CSS and template filter.<br />Inherit from QuickTemplate, set the UI style<br />Create skin folder with your skin name in /skins<br />Copy image and CSS files to your skin folder<br />Use your own skin<br />In LocalSettings.php set<br />$wgDefaultSkin = 'wiking_skin'<br />
    15. 15. Seahawks Video Skin<br />
    16. 16. Skin Customization Example<br />Vulcan Development Wiki and Seahawk wiki. <br />We have customized these two skins for our projects and the key is the method of QuickTemplate. In this method, you can decide what to show and how it shows on your wiki<br />You can customize wiki site like menu links, page titles, CSS and JavaScript files in the data properties of QuickTemplate<br />For example, show wiki side bar in your skin<br /><?phpforeach ($this->data['sidebar'] as $bar => $cont) { ?><br /> <li><br /> <a href="#"><h5><?php $out = wfMsg( $bar );></h5></a><br /> <ul><br /> <?phpforeach($cont as $key => $val) { ?><br /> <li id="<?php echo $val['id'] ?>" ><br /> <a href="<?php echo $val['href'] ?>"><?php echo $val['text'] ?></a><br /> </li><br /> <?php } ?><br /> </ul><br /> </li><br /><?php } ?><br />
    17. 17. Personalized Skin<br />Skin supports customization via __USERNAME__<br />User can have personalized styles within a skin<br />http://www.mediawiki.org/wiki/Manual:Skin_configuration#User_CSS<br />Tip: A way to turn off Wikipedia banner ad:<br />http://en.wikipedia.org/wiki/User:Jesseone/vector.css<br />
    18. 18. Schema Design<br />
    19. 19. “Just Do It!”<br />Easy to start<br />Flat, large ontology, at beginning<br />Many properties only valid in certain cases<br />May have similar/duplicate items<br />But it’ll get better over time!<br />Seahawks Video Wiki<br />
    20. 20. Designer Ontology<br />Professionally generated after rounds of discussions<br />Time consuming<br />More comprehensive and thoroughly designed<br />More data (restriction) than required<br />Data (ontology) importer isn’t powerful to fully support all features in the designing tool (Protégé)<br />Biology Ontology Wiki<br />
    21. 21. The Middle Way<br />Practical schema design <br />Plan and Explore<br />Think, adapt, and evolve<br />Our Agile Project Management Wiki<br />
    22. 22. N-ary Relations<br />Type:Record in SMW<br />Semantic Internal Object extension<br />Adding an object relation in the middle (Reification)<br />Using parameterized properties (Example: the third turn wiki)<br />Using a triple store (external)<br />
    23. 23. Extensions<br />
    24. 24. Find Extensions<br />Where to find extensions<br />http://www.mediawiki.org/wiki/Extension_Matrix<br />By supported version (1.16, 1.17, …)<br />By status (beta, stable, …)<br />By type (API, AJAX, hook, user rights…)<br />By new-ness (sort by most recently created time)<br />By talk page new-ness…<br />By most updated versions…<br />And of course using search tools from G.Y.M…<br />
    25. 25. Get the Extensions!<br />Some essential MediaWiki extensions!<br />http://semantic-mediawiki.org/wiki/Help:MediaWiki_extensions<br />And quite a few Semantic MediaWiki extensions:<br />http://semantic-mediawiki.org/wiki/Help:SMW_extensions<br />Find what you want <br />Not exactly? Then patch it!<br />Nothing close? Then build it!<br />Or find people to build it <br />
    26. 26. Our Experiences<br />We built a few semantic wikis (2008-2011)<br />Sci-Fi movie demo<br />Semantic Football (Entertainment) <br />NGT – a work-order handling system<br />Ultrapedia – an Analytical Encylopedia<br />Wiking agile project management<br />Biology ontology<br />Stanford class wiki<br />Neurowiki …<br />
    27. 27. Patches Often Needed<br />Many features are sort of there, some still not.<br />Popular extensions (e.g. SF, SRF, etc.) often need patches or tuning<br />Example: SF Auto-completion (over the years)<br />Fixed set of allowed values<br />All values in a category<br />All values having a specific property<br />All values from a custom query<br />
    28. 28. New Features Added<br />Real-time notification (Semantic NotifyMe, 2008)<br />Aggregation (average, max, sum, etc.) (2009)<br />Video merging <br />Open Flash Charts (OFC) SRF (2009-2010)<br />Strict > and < comparison (2010)<br />Exhibit enhancements (webit) (2009)<br />New formats: Runway, Time Plot, etc.<br />Mapping: unique ID requirements<br />Geocoding<br />Semantic WikiTags (2009)<br />Semantic Connector (2010)<br />
    29. 29. Two of Our Latest Extensions<br />Wiki Object Model<br />Enable third parties to consume wiki data via APIs<br />APIs (and URLs) for easy and flexible manipulation of wiki page (and data) <br />At a finer granularity<br />Widget Designer / Page Creator (prototype)<br />To lower structured page creation dramatically<br />To enable data-driven application building<br />Come to my Semantic Page Creator talk on Friday<br />
    30. 30. Why Wiki Object Model<br />Strong needs for access the structured data and unstructured data in the wiki as a data store<br />From internal and external applications’ point of view <br />A DOM-like approach is good choice<br />Xpath is a reasonable tool and standard<br />Granularity offers flexibility and power<br />Easier to operate on known objects and calling REST APIs rather than parsing and handling wiki texts<br />Common gateway for both internal and external apps<br />
    31. 31. Wiki Object Model Objects<br />Category<br />Link<br />Property<br />Text<br />Magic word<br />HTML tag<br />Sentence<br />Word*<br />Image<br />31<br />Page<br />Section<br />Parameter<br />Parameter value<br />Template<br />Template field<br />Parser function<br />List item<br />Table<br />Table cell<br />* Not yet implemented as of September 2011 <br />
    32. 32. Demo of WOM<br /><ul><li>Demo at http://wiking.vulcan.com/dev_sandbox/
    33. 33. Demo 1: APIs
    34. 34. The API is part of MediaWiki API, with our additions to allow get and set (read/write) the content of a wiki page
    35. 35. http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Apis
    36. 36. Demo 2: Inline Editor
    37. 37. A simple inline editor (AJAX styled) to let user modify a specific portion of the wiki page
    38. 38. Based on http://wiking.vulcan.com/dev/index.php/Extension:Wiki_Object_Model/Functions</li></ul>32<br />
    39. 39. Security Extensions<br />Many choices = nothing is perfect<br />Options: from very basic to very advanced:<br />No Security<br />MediaWiki Built-in Groups<br />Simple Security (Page Security)<br />HaloACL<br />Features<br />Restrictions<br />
    40. 40. A Short Case Study<br />Proposal Management System (Wiki)<br />Users (including privileged users) submit proposals<br />Reviewers (privileged users) rate and review them<br />Reviewers discuss and make decision and assign<br />What security extension should it employ?<br />HaloACL? – the most powerful ACL out there<br />SimpleSecurity? – simple and effective<br />Or MediaWiki built-in groups? – simplest<br />MediaWiki Groups<br />
    41. 41. Visualizations<br />Semantic Result Formatter<br />Format = ???<br />{Param} = ???<br />
    42. 42. Visualization: Our Path<br />2007: SRF used to only have few options <br />2008-2009: Exhibit seemed cool<br />Ported webit<br />Fixed some bugs and enhanced features <br />Runway, TimePlot, multiple rows in Map view etc.<br />2009-2010: Open Flash Chart<br />More bars, pies, lines, scatter plot…<br />Interaction<br />Richer information display<br />2010-11: TabbedHighChart, OFC/ScatterPlot<br />
    43. 43. Comparing SRF Options<br />Google Charts (pie and bar)<br />Google does it! (G gets your data, no offline choice…)<br />Static bar and pie, no color options…<br />Exhibit<br />More choices (timeline, map, facet, runway…)<br />Size limit; Browser compatibility issues<br />Flash-based: OFC (Open Flash Chart), High Chart<br />Many bars, charts, plots, even scattered plots<br />Flash required<br />
    44. 44.
    45. 45. Sidebar Customization<br />Basic: MediaWiki:Sidebar<br />Enhanced: semantic queries, and tree views<br />Advanced: use variables together with queries<br />Custom: Using an extension: DynamicSidebar<br />Customize your own sidebar at User:<username>/Sidebar<br />http://www.mediawiki.org/wiki/Extension:DynamicSidebar<br />Example: http://wiking.vulcan.com/dev/<br />
    46. 46. Current Sprint in Sidebar<br />*Current sprint<br />{{#ask: [[Category:Project sprints]]<br />[[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]<br />[[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]<br />|format=template<br />|template=Sidebar query template<br />|link=none<br />|}}<br />
    47. 47. My Active Sprint Tasks in Sidebar<br />*My tasks<br />{{#ask: [[Category:Project tasks]]<br />[[Project task owner::<q>[[User alias::{{CURRENTUSER}}]]</q>]]<br />[[Project story::<q><br />[[Project sprint::<q>[[Category:Project sprints]]<br />[[Sprint start date::<{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]<br />[[Sprint end date::>{{CURRENTYEAR}}/{{CURRENTMONTH}}/{{CURRENTDAY}}]]</q>]]<br /></q>]]<br />[[Project task status::In progress||Not started]]<br />|format=template<br />|template=Sidebar query template<br />|link=none<br />|}}<br />
    48. 48. More Extension Examples<br />Twitter Feed<br />http://www.mediawikiwidgets.org/Twitter_Search (example)<br />Flickr Feed<br />http://www.mediawikiwidgets.org/Flickr<br />Amazon Carousel<br />https://widgets.amazon.com/Amazon-Carousel-Widget/<br />Google Analytics<br />http://www.mediawiki.org/wiki/Extension:Google_Analytics<br />Optify Analytics<br />http://www.optify.net/<br />
    49. 49. Templates and Forms<br />MediaWiki template is essential for its structured data handling<br />Editing, especially Template editing, is not trivial<br />Semantic Forms extension greatly lowers the bar<br />Customizing templates and forms can provide very visually appealing pages<br />
    50. 50. Form with a Style<br />http://www.thethirdturn.com/w/index.php?title=Form:Driver&action=edit<br />
    51. 51. Forms with autocompletion<br />Advanced Auto-Completion on Customized Query Results<br />Basic Auto-Completion on Category Values<br />
    52. 52. The Sky is the Limit<br />
    53. 53. Integration<br />
    54. 54. Steps<br />Know what to build<br />Customers, customers, customers!<br />Get your collection of extensions…<br />Customize your wiki (template, forms)<br />Handle the data (schema, query, API, …)<br />
    55. 55. Case 1. Proper Wiki<br />Focus on Skin, Form and Template<br />Steps:<br />Pick a (default) skin<br />Design some forms<br />And templates (queries) behind the forms<br />Build starter pages and Go!<br />
    56. 56. Biology Ontology Wiki<br />People need discuss about the details related to construct a biology ontology<br />Classes (categories)<br />Properties<br />Relations<br />Textbook analysis<br />Discussion<br />Forms necessary to keep content tidy<br />
    57. 57. Ex: Property Form<br />
    58. 58. Forms on Namespace<br />http://www.mediawiki.org/wiki/Extension:Semantic_Forms#Based_on_namespace<br />
    59. 59. Case 2: Simple Workflow App<br />Idea: go from one form to another form<br />Need form links in the template<br />May need user rights management<br />
    60. 60. Proposal Review Workflow Example<br />User submits a proposal via a form<br />Fill in basic information about the proposal<br />Set a field in the form to be, say, “New”<br />User doesn’t see further information about “Reviewer” or “Owner”, etc.<br />Semantic notification (email and RSS feed) facilitates communication<br />Reviewer finds the form in a canned query<br />Reviewer changes the status to, say, “Approved”<br />Reviewer field (say “Reviewed by”) is (automatically) set<br />Then reviewer assigns the proposal to a owner, say, “Alice”<br />The owner (“Alice”) now sees it and can start work on it<br />Changes the status to “Active”<br />Adds a start date or maybe an estimated end date too<br />
    61. 61. Ways to Handle Workflow<br />Add a link to open another form<br />Force edit “review” using another form (e.g. reviewer form) <br />When saving the new form, the category is changed<br />Now we have a different category  a different default form<br />Change a value to include or exclude to another template<br />May need #if (or #switch) statement in template to change (or include/exclude) template values<br />Use “Page has default form” property (Semantic Forms extension)<br />
    62. 62. More Workflow<br />Good summary by YaronKoren<br />SMWCon Spring 2011 TalkSMW and the Workflow Puzzle<br />My Semantic Page Creator Talk on Friday 9/23<br />Example: A simple HelpDesk wiki<br />
    63. 63. Case 3: WikingDev Wiki<br />Our project management wiki<br />We use it for *all* things it can do:<br />Proposals, ideas, email messages, features, bugs<br />Agile development: milestones, sprints, stories, tasks<br />Code/Feature association: SVN mapped to tasks/bugs<br />Progress analysis: burn-down charts, work calendar<br />Collaboration, Documentation, Demonstration<br />
    64. 64. Examples of Data I/O<br />Wiking Development Wiki <br />An Imported email<br />http://wiking.vulcan.com/dev/index.php/Demo_scenarios<br />An email imported as a Project Bug<br />http://wiking.vulcan.com/dev/index.php/Issue_11886_Insert_a_single_property<br />A task uploaded via Outlook with multiple commits<br />http://wiking.vulcan.com/dev/index.php/WikiTags_Release_Structure<br />
    65. 65. Microsoft Office Connector<br />Leverage Microsoft Office applications and technology<br />Bring SMW info to Office applications on-demand<br />API for data I/O: add and modify wiki data within Microsoft Office <br />Utilize semantics to improve relevance<br />Smart actions for semantic properties<br />API<br />API<br />Connections<br />Smart<br />Jesse Wang | SemTech 2010<br />
    66. 66. Our Agile Project Management (Scrum)<br />60<br />Wiking<br />Jesse Wang | SMWCon Amsterdam 2010<br />
    67. 67. Wiki Data I/O<br />Wiking development wiki has two external applications <br />Subversion integration<br />Microsoft Office integration<br />Extensions to support it<br />Semantic Wikitags<br />Semantic Connector<br />
    68. 68. Data I/O Extensions<br />MediaWiki API – very basic read/write<br />Page Object Model<br />First Data I/O extension to allow access some wiki structure data (links, title, basic template field)<br />SMWWriter<br />Based on POM, support annotations (semantic properties)<br />Data API extension in SMW+<br />Semantic WikiTags and Semantic Connector<br />Wiki Object Model<br />A powerful and comprehensive object model (Data API)<br />Read/write at finer object level (sentences, template parameters)<br />
    69. 69. Case Study: Battle-space Luminary System <br />Discover when New Information represents a change in understanding of entities<br />Discovery of explicit entity links, implicit relationships<br />Large Volumes of Data in various formats<br />Unstructured news articles<br />Tactical Reports, Field Intelligence<br />Structured Database Information<br />Use Wiki Pages to represent current knowledge about an entity – “what we know”<br />Domain Ontology to represent domain of information – “what we want to know”<br />Issue Alerts when Significant Events occur<br />New information according to category<br />Changing information on topics of interest<br />Need to send information to various devices – cell phones, email, etc.<br />63<br />
    70. 70. System Design<br />Wiki Configuration<br />Semantic MediaWiki: Large developer community, active development, open source. Wikipedia uses MediaWiki, so scalability and performance are important.<br />Semantic Results Format: Provides various rich media displays of semantic information, including graphs, timelines, maps<br />Semantic Forms: Provides convenient user interface for entering semantic data into wiki, avoiding cumbersome wikitext<br />Semantic Notifications: Enables sending of notifications when results of semantic query change.<br />Domain Ontology<br />Created OWL Ontology for Terrorism<br />Semantic Parsing, Extraction, Reasoning<br />Java Process using various Open-Source Toolkits<br />Rapid plugin of new technologies<br />Multiple Data Sources supported<br />64<br />
    71. 71. Sample Content Page<br />65<br />
    72. 72. Wiki Content Design<br />Use Templates to Ensure Consistent Look-and-Feel<br />Templates Correspond to Ontology Classes<br />Fields within Templates correspond to Properties within Ontology<br />Rich Content Visualizations derived in consistent way<br />Hierarchical Categories match Class Hierarchy within Ontology<br />Ensures Validity for Properties<br />Category included on each Template page to ensure consistency<br />FormsProvide ability for users to enter data directly into wiki without knowing Wiki Text<br />Each form corresponds to a Template<br />Fields within forms correspond to the fields/properties within the Template<br />GUI can include auto-completion<br />Created Page immediately linked semantically to rest of Wiki<br />66<br />
    73. 73. Sample Visualizations<br />67<br />UI enables notifications based on results of query – message sent when visualization changes<br />Visualizations automatically created w/o user edit<br />(tables, timelines, maps, social networks…)<br />
    74. 74. Optimize<br />
    75. 75. Usability<br />MediaWiki Usability Initiative<br />Semantic MediaWiki Usability<br />Fact Box, Special pages<br />Extensions<br />Semantic Forms<br />One Click Way<br />Auto-complete<br />SRF<br />User Interface<br />Example: Wiking Project Task<br />http://wiking.vulcan.com/dev/index.php/Rehearsal_of_WE_Demo<br />
    76. 76. Performance<br />Fundamentals<br />http://smwforum.ontoprise.com/smwforum/index.php/Help:Optimizing_response_times_1.5.6<br />Caches: squid, apache, parser, query, browser, …<br />Global Load Balancing Service<br />Batch Process<br />RefreshData.php<br />Other services:<br />memcache<br />Triple Store<br />Solr<br />
    77. 77. Search Engine Optimization<br />General SEO rules (Official Google Guide 32pp)<br />MediaWiki <br />Robot.txt : edit, talk, websvn, …<br />Metadata: title, description, media, alt tags… <br />Semantic MediaWiki<br />RDFa/Rich Snippets/Search Monkey<br />For more info:<br />Wil Smith: Search Engine Optimization for SMW<br />
    78. 78. Summary<br />Jumpstart<br />Deployment of your bundle/suite/package<br />Code, pseudo-code/meta-data, and data<br />Customize<br />Skins, Styles, Extensions<br />Data, Code, Code, Data<br />Integrate <br />Know the requirements and data<br />Put everything together and evolve<br />
    79. 79. Acknowledgement<br />Jesse Wang | SMWCon Amsterdam 2010<br />73<br />
    80. 80. Thank you<br />Questions and/or comments?<br />Slide uploaded at http://www.slideshare.net/jiaxinwang/<br />Email me at jessew @ vulcan.comor @aiwang<br />