SlideShare a Scribd company logo
wikilytics




DATA SUMMIT
      Feb 4, 2011
   Diederik van Liere
“ Live Demo”
answers
           wikilytics   platform




raw data
Wikilytics
 config | download | extract | sort | store | transform

      raw data                                   answers
configure | wikilytics




config | download | extract | sort | store | transform
language support | wikilytics




config | download | extract | sort | store | transform
download dump | wikilytics




config | download | extract | sort | store | transform
extract variables | wikilytics




config | download | extract | sort | store | transform
mergesort edits by editor | wikilytics




config | download | extract | sort | store | transform
store editors in db | wikilytics




config | download | extract | sort | store | transform
precompute variables | wikilytics




config | download | extract | sort | store | transform
config | download | extract | sort | store | transform

     00:00:00:000                          01:05:29:352
                                               (Polish wiki)
“Command lines are too hard...”
Launch the entire data-processing from
            your browser
The entire data-processing chain is
             running...
wikilytics              platform




                Plugins (Python)         Map / Reduce (JS)
Analysis



               BSON Document in          New Collection in
Storage           MongoDB                   MongoDB




Output
Each analysis is a plugin...




      def new_editor_count(var, editor, **kwargs):
          '''
          Summary: This function generates an overview of the number of
          new_wikipedians for a given year / month combination.
          Purpose: This data can be used to compare with Erik Zachte's
          stats.download.org to make sure that we are using the same numbers.
          '''
      #   headers = ['year', 'month', 'count']
          new_wikipedian = editor['new_wikipedian']
          var.add(new_wikipedian, {0:1})
          return var
...and available in your browser
Text
Resources
•strategy.wikimedia.org/wiki/Editor_Trends_Study
•strategy.wikimedia.org/wiki/Editor_Trends_Study/Software
•svn.wikimedia.org/svnroot/mediawiki/trunk/tools/editor_trends/

More Related Content

Similar to Wikilytics

Oscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to ProductionOscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to Production
Patrick Chanezon
 
Drivetribe: A Social Network on Streams
Drivetribe: A Social Network on StreamsDrivetribe: A Social Network on Streams
Drivetribe: A Social Network on Streams
C4Media
 
Getting Started with Docker - Nick Stinemates
Getting Started with Docker - Nick StinematesGetting Started with Docker - Nick Stinemates
Getting Started with Docker - Nick Stinemates
Atlassian
 
Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?
John Rofrano
 
JavaEdge 2008: Your next version control system
JavaEdge 2008: Your next version control systemJavaEdge 2008: Your next version control system
JavaEdge 2008: Your next version control system
Gilad Garon
 
Docker Training - June 2015
Docker Training - June 2015Docker Training - June 2015
Docker Training - June 2015
{code}
 
Docker and Containers overview - Docker Workshop
Docker and Containers overview - Docker WorkshopDocker and Containers overview - Docker Workshop
Docker and Containers overview - Docker Workshop
Jonas Rosland
 
Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015
Jonas Rosland
 
Are VMs Passé?
Are VMs Passé?Are VMs Passé?
Are VMs Passé?
Docker, Inc.
 
Are VM Passé?
Are VM Passé? Are VM Passé?
Are VM Passé?
dotCloud
 
SwissJUG_Bringing the cloud back down to earth.pptx
SwissJUG_Bringing the cloud back down to earth.pptxSwissJUG_Bringing the cloud back down to earth.pptx
SwissJUG_Bringing the cloud back down to earth.pptx
Grace Jansen
 
meetup devops 2023/04/04 - Astonish April
meetup devops 2023/04/04 - Astonish Aprilmeetup devops 2023/04/04 - Astonish April
meetup devops 2023/04/04 - Astonish April
Frederic Leger
 
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB
 
Datasheet weblogicpluginforrd
Datasheet weblogicpluginforrdDatasheet weblogicpluginforrd
Datasheet weblogicpluginforrd
MidVision
 
Hello Cotrix
Hello CotrixHello Cotrix
Hello Cotrix
Fabio Simeoni
 
Is OSGi Modularity Always Worth It? - Glyn Normington
Is OSGi Modularity Always Worth It? - Glyn NormingtonIs OSGi Modularity Always Worth It? - Glyn Normington
Is OSGi Modularity Always Worth It? - Glyn Normington
mfrancis
 
SofiaDev L9 Source Control Management
SofiaDev L9 Source Control ManagementSofiaDev L9 Source Control Management
SofiaDev L9 Source Control Management
Martin Kulov
 
Continuous Delivery Applied
Continuous Delivery AppliedContinuous Delivery Applied
Continuous Delivery Applied
Excella
 
Continuous Delivery Applied (Agile Richmond)
Continuous Delivery Applied (Agile Richmond)Continuous Delivery Applied (Agile Richmond)
Continuous Delivery Applied (Agile Richmond)
Mike McGarr
 
Continuous Delivery Applied
Continuous Delivery AppliedContinuous Delivery Applied
Continuous Delivery Applied
DC Agile Engineering Conference
 

Similar to Wikilytics (20)

Oscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to ProductionOscon London 2016 - Docker from Development to Production
Oscon London 2016 - Docker from Development to Production
 
Drivetribe: A Social Network on Streams
Drivetribe: A Social Network on StreamsDrivetribe: A Social Network on Streams
Drivetribe: A Social Network on Streams
 
Getting Started with Docker - Nick Stinemates
Getting Started with Docker - Nick StinematesGetting Started with Docker - Nick Stinemates
Getting Started with Docker - Nick Stinemates
 
Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?Microservices: How loose is loosely coupled?
Microservices: How loose is loosely coupled?
 
JavaEdge 2008: Your next version control system
JavaEdge 2008: Your next version control systemJavaEdge 2008: Your next version control system
JavaEdge 2008: Your next version control system
 
Docker Training - June 2015
Docker Training - June 2015Docker Training - June 2015
Docker Training - June 2015
 
Docker and Containers overview - Docker Workshop
Docker and Containers overview - Docker WorkshopDocker and Containers overview - Docker Workshop
Docker and Containers overview - Docker Workshop
 
Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015Docker and containers - For Boston Docker Meetup Workshop in March 2015
Docker and containers - For Boston Docker Meetup Workshop in March 2015
 
Are VMs Passé?
Are VMs Passé?Are VMs Passé?
Are VMs Passé?
 
Are VM Passé?
Are VM Passé? Are VM Passé?
Are VM Passé?
 
SwissJUG_Bringing the cloud back down to earth.pptx
SwissJUG_Bringing the cloud back down to earth.pptxSwissJUG_Bringing the cloud back down to earth.pptx
SwissJUG_Bringing the cloud back down to earth.pptx
 
meetup devops 2023/04/04 - Astonish April
meetup devops 2023/04/04 - Astonish Aprilmeetup devops 2023/04/04 - Astonish April
meetup devops 2023/04/04 - Astonish April
 
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
MongoDB World 2019: Using the MongoDB Enterprise Kubernetes Operator to Scale...
 
Datasheet weblogicpluginforrd
Datasheet weblogicpluginforrdDatasheet weblogicpluginforrd
Datasheet weblogicpluginforrd
 
Hello Cotrix
Hello CotrixHello Cotrix
Hello Cotrix
 
Is OSGi Modularity Always Worth It? - Glyn Normington
Is OSGi Modularity Always Worth It? - Glyn NormingtonIs OSGi Modularity Always Worth It? - Glyn Normington
Is OSGi Modularity Always Worth It? - Glyn Normington
 
SofiaDev L9 Source Control Management
SofiaDev L9 Source Control ManagementSofiaDev L9 Source Control Management
SofiaDev L9 Source Control Management
 
Continuous Delivery Applied
Continuous Delivery AppliedContinuous Delivery Applied
Continuous Delivery Applied
 
Continuous Delivery Applied (Agile Richmond)
Continuous Delivery Applied (Agile Richmond)Continuous Delivery Applied (Agile Richmond)
Continuous Delivery Applied (Agile Richmond)
 
Continuous Delivery Applied
Continuous Delivery AppliedContinuous Delivery Applied
Continuous Delivery Applied
 

Recently uploaded

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Jeffrey Haguewood
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
David Brossard
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Edge AI and Vision Alliance
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Project Management Semester Long Project - Acuity
Project Management Semester Long Project - AcuityProject Management Semester Long Project - Acuity
Project Management Semester Long Project - Acuity
jpupo2018
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Wask
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
MichaelKnudsen27
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
Webinar: Designing a schema for a Data Warehouse
Webinar: Designing a schema for a Data WarehouseWebinar: Designing a schema for a Data Warehouse
Webinar: Designing a schema for a Data Warehouse
Federico Razzoli
 

Recently uploaded (20)

Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
Salesforce Integration for Bonterra Impact Management (fka Social Solutions A...
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
OpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - AuthorizationOpenID AuthZEN Interop Read Out - Authorization
OpenID AuthZEN Interop Read Out - Authorization
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Project Management Semester Long Project - Acuity
Project Management Semester Long Project - AcuityProject Management Semester Long Project - Acuity
Project Management Semester Long Project - Acuity
 
Digital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying AheadDigital Marketing Trends in 2024 | Guide for Staying Ahead
Digital Marketing Trends in 2024 | Guide for Staying Ahead
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Nordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptxNordic Marketo Engage User Group_June 13_ 2024.pptx
Nordic Marketo Engage User Group_June 13_ 2024.pptx
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
Webinar: Designing a schema for a Data Warehouse
Webinar: Designing a schema for a Data WarehouseWebinar: Designing a schema for a Data Warehouse
Webinar: Designing a schema for a Data Warehouse
 

Wikilytics

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n