SlideShare a Scribd company logo
Visitors in Focus –
How Data
Analysis Can
Optimize the
Museum
Experience
Bachelor Project 2019/20
Internal Presentation · Revised for publication
Laura Holz, Selina Reinhard, Leon Schmidt,
Georg Tennigkeit, Christoph Thiede, Tom Wollnik
These slides were originally presented at HPI
on 2019-07-21 to the chair of Information
Systems headed by Prof. Dr. Felix Naumann.
Data Collection Integration Visualization
Social Media
Tickets,
Bookings
Reviews
Project Structure
These slides were originally presented at HPI
on 2019-07-21 to the chair of Information
Systems headed by Prof. Dr. Felix Naumann.
Example Power BI dashboard for Instagram
posts published by the museum.
Example Power BI dashboard for Instagram
posts published by the museum.
Inter alia, it allows to compare the
engagements and impressions of posts
depending on the hashtag they were
attributed with.
System Architecture
Christoph Thiede
2020-07-21 5
For more information on the system
architecture, see our GitHub repository:
https://github.com/Museum-Barberini-
gGmbH/Barberini-Analytics
Data Sources
2020-07-21 6
MUSEUM SYSTEM SOCIAL MEDIA RATING PLATFORMS
All supported data sources:
DOCUMENTATION.md#data-sources.
Data Source: go~mus
REST API
Excel Reports
Web Scraper
2020-07-21 7
CUSTOMER
S
BOOKINGS
EXHIBITIONS
ORDERS
All go~mus sources: DOCUMENTATION.md
Tech Stack
2020-07-21 8
Ubuntu VM
Power BI Online
Backend
Frontend
Python
PostgreSQL
Docker
GitLab Enterprise
requests
Google
API
Client
beautiful
soup
lxml
⋯
pandas
psycopg2
GitLab CI Runner
GitLab
Runner
for Windows
Power BI
Desktop
Image
Magick
.NET
Standard
Windows 10 VM
Encapsulate + automate all dependencies.
Complete system architecture:
DOCUMENTATION.md
Pipeline Orchestration
2020-07-21 9
Task
output(): Target
requires(): Task[]
run()
CsvToDb
copy()
rows()
read_csv(file)
DataPreparationTask
ensure_foreign_keys()
condense_time_series()
DailyEntriesToDb
ExtractDailyEntries
FetchGomusReport
gomus_report.xlsx gomus.daily_entries
daily_entries.csv
requires
requires
Pipeline documentation:
DOCUMENTATION.md#
data-mining-pipeline
Pipeline Orchestration – Example (Success)
2020-07-21 10
PostsToDb
Pipeline documentation:
DOCUMENTATION.md#
data-mining-pipeline
Pipeline Orchestration – Example (Failure)
2020-07-21 11
PostsToDb
FetchTwitter
Pipeline documentation:
DOCUMENTATION.md#
data-mining-pipeline
Database: Post Schema
post
social_media_post
fb_post
fb_comment
ig_post
tweet
museum_review
google_maps_review
app_review
appstore_review
gplay_review
2020-07-21 12
Complete database schema is defined in the
repository (scripts/migrations).
Database: Migration System
2020-07-21 13
migration_001.sql
migration_002.py
migration_000.sql
migration_003.sql
migration_004.sh
migration_005.sql
$MIGRATION_VERSION=005
$ ./migrate.sh
Applying 'migration_005.sql' ...
Done.
ALTER TABLE tweet
ADD COLUMN post_date TIMESTAMP;
Migration system documentation:
DOCUMENTATION.md#migration-system
Continuous
Integration
2020-07-21 14
Build
Unit tests
Minimal mining pipeline
Visualization tests
Code Analysis
CI details:
DOCUMENTATION.md#
continuous-integration
References
• Repository: https://github.com/Museum-Barberini-
gGmbH/Barberini-Analytics
• go~mus API documentation: https://giantmonkey.github.io/gomus-
api-doc/public_api.html
• Luigi framework documentation: https://luigi.readthedocs.io/
• Pandas framework: https://pandas.pydata.org/
• Power BI website: https://powerbi.microsoft.com/en-us/
• PostgreSQL: https://www.postgresql.org/
2020-07-21 15

More Related Content

Similar to Barberini Analytics - System Architecture

Pixelache 110311-Hintikka-Kari-A-Open-data-Network-esthetics
Pixelache 110311-Hintikka-Kari-A-Open-data-Network-estheticsPixelache 110311-Hintikka-Kari-A-Open-data-Network-esthetics
Pixelache 110311-Hintikka-Kari-A-Open-data-Network-esthetics
Kari A. Hintikka
 
Alexia Meyermann: Building a research infrastructure for educational studies ...
Alexia Meyermann: Building a research infrastructure for educational studies ...Alexia Meyermann: Building a research infrastructure for educational studies ...
Alexia Meyermann: Building a research infrastructure for educational studies ...
DIPF | Leibniz-Institut für Bildungsforschung und Bildungsinformation
 
10 years of IBM Connections
10 years of IBM Connections10 years of IBM Connections
10 years of IBM Connections
LetsConnect
 
Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020
Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020
Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020
Beat Estermann
 
Value creation, value flows and liability over virtualised resources
Value creation, value flows and liability over virtualised resourcesValue creation, value flows and liability over virtualised resources
Value creation, value flows and liability over virtualised resources
ictseserv
 
Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...
Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...
Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...
Bernhard Rieder
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
David Newbury
 
Observations on Annotations – From Computational Linguistics and the World Wi...
Observations on Annotations – From Computational Linguistics and the World Wi...Observations on Annotations – From Computational Linguistics and the World Wi...
Observations on Annotations – From Computational Linguistics and the World Wi...
Georg Rehm
 
WP8 Dissemination and Exploitation
WP8 Dissemination and ExploitationWP8 Dissemination and Exploitation
WP8 Dissemination and Exploitation
INSEMTIVES project
 
Data Management Plans for SNSF Grants
Data Management Plans for SNSF GrantsData Management Plans for SNSF Grants
Data Management Plans for SNSF Grants
ETH-Bibliothek
 
Robust Expert Finding in Web-Based Community Information Systems
Robust Expert Finding in Web-Based Community Information SystemsRobust Expert Finding in Web-Based Community Information Systems
Robust Expert Finding in Web-Based Community Information Systems
Ralf Klamma
 
Holographic Data Visualization - M. Valoriani & A. Musone
Holographic Data Visualization - M. Valoriani & A. MusoneHolographic Data Visualization - M. Valoriani & A. Musone
Holographic Data Visualization - M. Valoriani & A. Musone
Data Driven Innovation
 
Innovation through data capitalisation
Innovation through data capitalisationInnovation through data capitalisation
Innovation through data capitalisation
Joanne Jacobs
 
HADOOP based Recommendation Algorithm for Micro-video URL
HADOOP based Recommendation Algorithm for Micro-video URLHADOOP based Recommendation Algorithm for Micro-video URL
HADOOP based Recommendation Algorithm for Micro-video URL
dbpublications
 
Layerized and temporally - Digital+humanities 2010 @MIT Medialab
Layerized and temporally - Digital+humanities 2010 @MIT MedialabLayerized and temporally - Digital+humanities 2010 @MIT Medialab
Layerized and temporally - Digital+humanities 2010 @MIT Medialab
Samuel Huron
 
open data for an open future.pptx
open data for an open future.pptxopen data for an open future.pptx
open data for an open future.pptx
Fach- und Koordinationsstelle OGD, Kanton Zürich
 
Understanding everyday users’ perception of socio-technical issues through s...
Understanding everyday users’ perception of  socio-technical issues through s...Understanding everyday users’ perception of  socio-technical issues through s...
Understanding everyday users’ perception of socio-technical issues through s...
Ahreum lee
 
KiWi – A flexible platform for semantic social media applications
KiWi – A flexible platform for semantic social media applicationsKiWi – A flexible platform for semantic social media applications
KiWi – A flexible platform for semantic social media applications
Kiwi Community
 
2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».
2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».
2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».
eMadrid network
 
Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)
Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)
Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)
Beat Signer
 

Similar to Barberini Analytics - System Architecture (20)

Pixelache 110311-Hintikka-Kari-A-Open-data-Network-esthetics
Pixelache 110311-Hintikka-Kari-A-Open-data-Network-estheticsPixelache 110311-Hintikka-Kari-A-Open-data-Network-esthetics
Pixelache 110311-Hintikka-Kari-A-Open-data-Network-esthetics
 
Alexia Meyermann: Building a research infrastructure for educational studies ...
Alexia Meyermann: Building a research infrastructure for educational studies ...Alexia Meyermann: Building a research infrastructure for educational studies ...
Alexia Meyermann: Building a research infrastructure for educational studies ...
 
10 years of IBM Connections
10 years of IBM Connections10 years of IBM Connections
10 years of IBM Connections
 
Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020
Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020
Estermann Linked Data Ecosystem for Heritage Data - 29 Feb 2020
 
Value creation, value flows and liability over virtualised resources
Value creation, value flows and liability over virtualised resourcesValue creation, value flows and liability over virtualised resources
Value creation, value flows and liability over virtualised resources
 
Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...
Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...
Analyzing Social Media with Digital Methods. Possibilities, Requirements, and...
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
Observations on Annotations – From Computational Linguistics and the World Wi...
Observations on Annotations – From Computational Linguistics and the World Wi...Observations on Annotations – From Computational Linguistics and the World Wi...
Observations on Annotations – From Computational Linguistics and the World Wi...
 
WP8 Dissemination and Exploitation
WP8 Dissemination and ExploitationWP8 Dissemination and Exploitation
WP8 Dissemination and Exploitation
 
Data Management Plans for SNSF Grants
Data Management Plans for SNSF GrantsData Management Plans for SNSF Grants
Data Management Plans for SNSF Grants
 
Robust Expert Finding in Web-Based Community Information Systems
Robust Expert Finding in Web-Based Community Information SystemsRobust Expert Finding in Web-Based Community Information Systems
Robust Expert Finding in Web-Based Community Information Systems
 
Holographic Data Visualization - M. Valoriani & A. Musone
Holographic Data Visualization - M. Valoriani & A. MusoneHolographic Data Visualization - M. Valoriani & A. Musone
Holographic Data Visualization - M. Valoriani & A. Musone
 
Innovation through data capitalisation
Innovation through data capitalisationInnovation through data capitalisation
Innovation through data capitalisation
 
HADOOP based Recommendation Algorithm for Micro-video URL
HADOOP based Recommendation Algorithm for Micro-video URLHADOOP based Recommendation Algorithm for Micro-video URL
HADOOP based Recommendation Algorithm for Micro-video URL
 
Layerized and temporally - Digital+humanities 2010 @MIT Medialab
Layerized and temporally - Digital+humanities 2010 @MIT MedialabLayerized and temporally - Digital+humanities 2010 @MIT Medialab
Layerized and temporally - Digital+humanities 2010 @MIT Medialab
 
open data for an open future.pptx
open data for an open future.pptxopen data for an open future.pptx
open data for an open future.pptx
 
Understanding everyday users’ perception of socio-technical issues through s...
Understanding everyday users’ perception of  socio-technical issues through s...Understanding everyday users’ perception of  socio-technical issues through s...
Understanding everyday users’ perception of socio-technical issues through s...
 
KiWi – A flexible platform for semantic social media applications
KiWi – A flexible platform for semantic social media applicationsKiWi – A flexible platform for semantic social media applications
KiWi – A flexible platform for semantic social media applications
 
2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».
2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».
2021_07_01 «Learning Informatics as Inspiration for Learning Analytics».
 
Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)
Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)
Introduction - Lecture 1 - Advanced Topics in Information Systems (4016792ENR)
 

Recently uploaded

Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
Baishakhi Ray
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
Jimmy Lai
 
kk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdfkk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdf
KIRAN KV
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
Retrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with RagasRetrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with Ragas
Zilliz
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
ssuser1915fe1
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
FIDO Alliance
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
Bhajan Mehta
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
Brian Pichman
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
DianaGray10
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
janagijoythi
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
The History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal EmbeddingsThe History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal Embeddings
Zilliz
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
softsuave
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
ZachWylie3
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
SynapseIndia
 

Recently uploaded (20)

Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
Semantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software DevelopmentSemantic-Aware Code Model: Elevating the Future of Software Development
Semantic-Aware Code Model: Elevating the Future of Software Development
 
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python CodebaseEuroPython 2024 - Streamlining Testing in a Large Python Codebase
EuroPython 2024 - Streamlining Testing in a Large Python Codebase
 
kk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdfkk vathada _digital transformation frameworks_2024.pdf
kk vathada _digital transformation frameworks_2024.pdf
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
Retrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with RagasRetrieval Augmented Generation Evaluation with Ragas
Retrieval Augmented Generation Evaluation with Ragas
 
Feature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptxFeature sql server terbaru performance.pptx
Feature sql server terbaru performance.pptx
 
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
UX Webinar Series: Drive Revenue and Decrease Costs with Passkeys for Consume...
 
Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17Mule Experience Hub and Release Channel with Java 17
Mule Experience Hub and Release Channel with Java 17
 
Uncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in LibrariesUncharted Together- Navigating AI's New Frontiers in Libraries
Uncharted Together- Navigating AI's New Frontiers in Libraries
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3Communications Mining Series - Zero to Hero - Session 3
Communications Mining Series - Zero to Hero - Session 3
 
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptxMAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
MAKE MONEY ONLINE Unlock Your Income Potential Today.pptx
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
The History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal EmbeddingsThe History of Embeddings & Multimodal Embeddings
The History of Embeddings & Multimodal Embeddings
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
 
Camunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptxCamunda Chapter NY Meetup July 2024.pptx
Camunda Chapter NY Meetup July 2024.pptx
 
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptxUse Cases & Benefits of RPA in Manufacturing in 2024.pptx
Use Cases & Benefits of RPA in Manufacturing in 2024.pptx
 

Barberini Analytics - System Architecture

  • 1. Visitors in Focus – How Data Analysis Can Optimize the Museum Experience Bachelor Project 2019/20 Internal Presentation · Revised for publication Laura Holz, Selina Reinhard, Leon Schmidt, Georg Tennigkeit, Christoph Thiede, Tom Wollnik These slides were originally presented at HPI on 2019-07-21 to the chair of Information Systems headed by Prof. Dr. Felix Naumann.
  • 2. Data Collection Integration Visualization Social Media Tickets, Bookings Reviews Project Structure These slides were originally presented at HPI on 2019-07-21 to the chair of Information Systems headed by Prof. Dr. Felix Naumann.
  • 3. Example Power BI dashboard for Instagram posts published by the museum.
  • 4. Example Power BI dashboard for Instagram posts published by the museum. Inter alia, it allows to compare the engagements and impressions of posts depending on the hashtag they were attributed with.
  • 5. System Architecture Christoph Thiede 2020-07-21 5 For more information on the system architecture, see our GitHub repository: https://github.com/Museum-Barberini- gGmbH/Barberini-Analytics
  • 6. Data Sources 2020-07-21 6 MUSEUM SYSTEM SOCIAL MEDIA RATING PLATFORMS All supported data sources: DOCUMENTATION.md#data-sources.
  • 7. Data Source: go~mus REST API Excel Reports Web Scraper 2020-07-21 7 CUSTOMER S BOOKINGS EXHIBITIONS ORDERS All go~mus sources: DOCUMENTATION.md
  • 8. Tech Stack 2020-07-21 8 Ubuntu VM Power BI Online Backend Frontend Python PostgreSQL Docker GitLab Enterprise requests Google API Client beautiful soup lxml ⋯ pandas psycopg2 GitLab CI Runner GitLab Runner for Windows Power BI Desktop Image Magick .NET Standard Windows 10 VM Encapsulate + automate all dependencies. Complete system architecture: DOCUMENTATION.md
  • 9. Pipeline Orchestration 2020-07-21 9 Task output(): Target requires(): Task[] run() CsvToDb copy() rows() read_csv(file) DataPreparationTask ensure_foreign_keys() condense_time_series() DailyEntriesToDb ExtractDailyEntries FetchGomusReport gomus_report.xlsx gomus.daily_entries daily_entries.csv requires requires Pipeline documentation: DOCUMENTATION.md# data-mining-pipeline
  • 10. Pipeline Orchestration – Example (Success) 2020-07-21 10 PostsToDb Pipeline documentation: DOCUMENTATION.md# data-mining-pipeline
  • 11. Pipeline Orchestration – Example (Failure) 2020-07-21 11 PostsToDb FetchTwitter Pipeline documentation: DOCUMENTATION.md# data-mining-pipeline
  • 13. Database: Migration System 2020-07-21 13 migration_001.sql migration_002.py migration_000.sql migration_003.sql migration_004.sh migration_005.sql $MIGRATION_VERSION=005 $ ./migrate.sh Applying 'migration_005.sql' ... Done. ALTER TABLE tweet ADD COLUMN post_date TIMESTAMP; Migration system documentation: DOCUMENTATION.md#migration-system
  • 14. Continuous Integration 2020-07-21 14 Build Unit tests Minimal mining pipeline Visualization tests Code Analysis CI details: DOCUMENTATION.md# continuous-integration
  • 15. References • Repository: https://github.com/Museum-Barberini- gGmbH/Barberini-Analytics • go~mus API documentation: https://giantmonkey.github.io/gomus- api-doc/public_api.html • Luigi framework documentation: https://luigi.readthedocs.io/ • Pandas framework: https://pandas.pydata.org/ • Power BI website: https://powerbi.microsoft.com/en-us/ • PostgreSQL: https://www.postgresql.org/ 2020-07-21 15

Editor's Notes

  1. These slides were originally presented at HPI on 2019-07-21 to the chair of Information Systems headed by Prof. Dr. Felix Naumann.
  2. Example Power BI dashboard for Instagram posts published by the museum.
  3. Example Power BI dashboard for Instagram posts published by the museum. Inter alia, it allows to compare the engagements and impressions of posts depending on the hashtag they were attributed with.
  4. For more information on the system architecture, see our GitHub repository: https://github.com/Museum-Barberini-gGmbH/Barberini-Analytics
  5. For more information on all supported data sources, see DOCUMENTATION.md#data-sources.
  6. For more information on gomus sources, see DOCUMENTATION.md.
  7. Motivation for containerization: encapsulate + automate all dependencies. For more information on the system architecture, see DOCUMENTATION.md.
  8. For more information on the pipeline, see DOCUMENTATION.md#data-mining-pipeline.
  9. For more information on the pipeline, see DOCUMENTATION.md#data-mining-pipeline.
  10. For more information on the pipeline, see DOCUMENTATION.md#data-mining-pipeline.
  11. For the complete database schema, see all migration scripts in the repository (scripts/migrations).
  12. Fore more information on the migration system, see DOCUMENTATION.md#migration-system.
  13. For more information on the CI, see DOCUMENTATION.md#continuous-integration.
  14. For more information on the system architecture, see our GitHub repository: https://github.com/Museum-Barberini-gGmbH/Barberini-Analytics