This document summarizes a student project that analyzed data to optimize the museum experience. It developed a data pipeline to integrate social media, tickets, reviews and other data sources into visualizations in Power BI. The project implemented a database to store social media posts and other data, along with a migration system. Continuous integration processes were also developed to test the data mining pipeline.

1. Visitors in Focus –
How Data
Analysis Can
Optimize the
Museum
Experience
Bachelor Project 2019/20
Internal Presentation · Revised for publication
Laura Holz, Selina Reinhard, Leon Schmidt,
Georg Tennigkeit, Christoph Thiede, Tom Wollnik
These slides were originally presented at HPI
on 2019-07-21 to the chair of Information
Systems headed by Prof. Dr. Felix Naumann.

2. Data Collection Integration Visualization
Social Media
Tickets,
Bookings
Reviews
Project Structure
These slides were originally presented at HPI
on 2019-07-21 to the chair of Information
Systems headed by Prof. Dr. Felix Naumann.

3. Example Power BI dashboard for Instagram
posts published by the museum.

4. Example Power BI dashboard for Instagram
posts published by the museum.
Inter alia, it allows to compare the
engagements and impressions of posts
depending on the hashtag they were
attributed with.

5. System Architecture
Christoph Thiede
2020-07-21 5
For more information on the system
architecture, see our GitHub repository:
https://github.com/Museum-Barberini-
gGmbH/Barberini-Analytics

6. Data Sources
2020-07-21 6
MUSEUM SYSTEM SOCIAL MEDIA RATING PLATFORMS
All supported data sources:
DOCUMENTATION.md#data-sources.

7. Data Source: go~mus
REST API
Excel Reports
Web Scraper
2020-07-21 7
CUSTOMER
S
BOOKINGS
EXHIBITIONS
ORDERS
All go~mus sources: DOCUMENTATION.md

8. Tech Stack
2020-07-21 8
Ubuntu VM
Power BI Online
Backend
Frontend
Python
PostgreSQL
Docker
GitLab Enterprise
requests
Google
API
Client
beautiful
soup
lxml
⋯
pandas
psycopg2
GitLab CI Runner
GitLab
Runner
for Windows
Power BI
Desktop
Image
Magick
.NET
Standard
Windows 10 VM
Encapsulate + automate all dependencies.
Complete system architecture:
DOCUMENTATION.md

9. Pipeline Orchestration
2020-07-21 9
Task
output(): Target
requires(): Task[]
run()
CsvToDb
copy()
rows()
read_csv(file)
DataPreparationTask
ensure_foreign_keys()
condense_time_series()
DailyEntriesToDb
ExtractDailyEntries
FetchGomusReport
gomus_report.xlsx gomus.daily_entries
daily_entries.csv
requires
requires
Pipeline documentation:
DOCUMENTATION.md#
data-mining-pipeline

10. Pipeline Orchestration – Example (Success)
2020-07-21 10
PostsToDb
Pipeline documentation:
DOCUMENTATION.md#
data-mining-pipeline

11. Pipeline Orchestration – Example (Failure)
2020-07-21 11
PostsToDb
FetchTwitter
Pipeline documentation:
DOCUMENTATION.md#
data-mining-pipeline

12. Database: Post Schema
post
social_media_post
fb_post
fb_comment
ig_post
tweet
museum_review
google_maps_review
app_review
appstore_review
gplay_review
2020-07-21 12
Complete database schema is defined in the
repository (scripts/migrations).

13. Database: Migration System
2020-07-21 13
migration_001.sql
migration_002.py
migration_000.sql
migration_003.sql
migration_004.sh
migration_005.sql
$MIGRATION_VERSION=005
$ ./migrate.sh
Applying 'migration_005.sql' ...
Done.
ALTER TABLE tweet
ADD COLUMN post_date TIMESTAMP;
Migration system documentation:
DOCUMENTATION.md#migration-system

14. Continuous
Integration
2020-07-21 14
Build
Unit tests
Minimal mining pipeline
Visualization tests
Code Analysis
CI details:
DOCUMENTATION.md#
continuous-integration

15. References
• Repository: https://github.com/Museum-Barberini-
gGmbH/Barberini-Analytics
• go~mus API documentation: https://giantmonkey.github.io/gomus-
api-doc/public_api.html
• Luigi framework documentation: https://luigi.readthedocs.io/
• Pandas framework: https://pandas.pydata.org/
• Power BI website: https://powerbi.microsoft.com/en-us/
• PostgreSQL: https://www.postgresql.org/
2020-07-21 15