1.
25/01/2018
1
Agile, cloud-based
DW-development
– experiences from the
field and lessons
learned.
24.1.2018
- Finavia’s journey: the beginning,
setting goals and how to get there?
- Enablers: Snowflake and Solita ADE
- Case examples
- Key takeaways
Tommi Vihervaara
25.1.20182
“Building the Data-Driven culture in aviation business.”

2.
25/01/2018
2
25 January 2018 Finavia yritysesitys 20173
Our task is to develop the competitiveness, mobility and
internationalisation of Finnish society by providing our
customers with safe, high-quality and cost-effective air travel
services.
We enable international accessibility for Finland through the
most widely-linked air traffic network in Northern Europe.
25 January 2018 Finavia yritysesitys 20174
Thanks to the wide network of routes
from Helsinki Airport, we offer superb
connections to all corners of the world.
Transfer connections ensure an
extensive market area for Finland.
Passengers and airlines are our
customers. We produce air traffic
services in cooperation with a large
group of partners.

3.
25/01/2018
3

4.
25/01/2018
4
25 January 2018 Finavia yritysesitys 20178

5.
25/01/2018
5
Our data journey.
In the beginning…
Q1 / 2017 – from where did we start from ?
25 January 2018 Finavia yritysesitys 20179

6.
25/01/2018
6
Target
H2/2017 and beyond…

7.
25/01/2018
7
”Everyone in the organization
should have access to as much data
as legally possible” – DJ Patil and Hilary Mason
(http://bit.ly/data-driven-report)
How the journey began and whats up
next?
There (and back again? )

8.
25/01/2018
8
25 January 2018 Finavia yritysesitys 201715
”Low-hanging fruit”

9.
25/01/2018
9
Cases
25.1.201817
”If the organization wants to be
data driven there has to be an
evangelist that makes noise about
what it needs.”
”Son, your ego is writing checks
your body can’t cash”
http://whitepapers.dataversity.net/content42609/

10.
25/01/2018
10
25.1.201819
We went agile.
Why agile?
Data Warehousing projects have their reputation …

11.
25/01/2018
11
2008 vs 2018 ?
2008

12.
25/01/2018
12
https://i.giphy.com/media/xUPGczofzl3QEh4glW/giphy.mp4
Just do it.
2018

13.
25/01/2018
13
25.1.201826

14.
25/01/2018
14
25 January 2018 Finavia yritysesitys 201728
"keep it short and simple"

15.
25/01/2018
15
Agile Data Warehousing
Old-school DW + ETL + BI doesn’t work anymore!
You need a Data Platform  all data: relational data, big data,
structured/unstructured, batch and realtime.
Manual ETL is a no-no  Automatically generated/maintained ”ETL code” and db schema.
- Time to Market: Let’s get going already!
 Learning by doing!
- Do it once. Do it right. Focus on proper data modeling.
- Dev + Ops  ”Data Devops”
Old-school BI/DW-development vs. software development?
 Getting finally together: ”Data Devops”
 Question for you:
Which vendors can (really) master it?
Enablers:
Tools & methods and architecture
#architecture&standards
#snowflake
#solitaADE
#datavault2.0
#dwautomation
#analyticalMDM

16.
25/01/2018
16
#architecture
Architecture
Flights ERP A
Cloud API A
Legacy sources (relational db)
Customer feedback data,
security throuputs and other
sensor data (non-relational)
csv
Flights ERP B
finavia-stage-data
S3
bucket
AWS Lambda
S3
bucket
COPY
DDL statements
Stage loads
Data vault loads
FINAVIA_DATA_HUB
No separate
”Cube or
inmemory”
hassle”
INTO
(Finavia integration platform)
DW automation/accelerator solution
(”Data Devops”)
Import (cache) or
DirectQuery
#analyticalMDM
(MDS)
csv
Batch
Real-time
json

17.
25/01/2018
17
Legendary 3 tier meets 3½: Bimodal
25.1.201833
”Data Vault. This is where the magik happens”landing / stage consume
experiment
Storage vs publish !!
Since 1974 …
#snowflake

18.
25/01/2018
18
Snowflake
• Data Warehouse as a Service
• Decisions now, must support requirements N years ahead
• Per second pricing (!)
• No-Admin !? ~Zero Admin
• No Backups!?
• Storage vs compute
• Separate compute workloads
• On-line up/downscaling
• No-hadoop etc.
• ”It Just Works”™
• Love it.
25.1.201835
#solitaADE
#dwautomation

19.
25/01/2018
19
Solita Agile Data Engine – Data DevOps
• ”No ETL”: ETL  L(T)
• Separate ETL Tool = Silo ? Does not scale!
• ”Model – Map – and Load”  Automation !
• ~ No transformation  Mapping.
• Put proper master data in work!  Analytical MDM
• Cases
– Dimension  Analytical MDM
– Fact  ”you got problems elsewhere son”
• ”Time to market”
• Proper data modeling / standardization: business entities  hubs / satellites / links
•  No need for lookups etc.
• Automate-everything?
• BI & dashboards automation?
25.1.201837
#others
Task management and monitoring

20.
25/01/2018
20
#trello
Task management
#grafana
Monitoring

21.
25/01/2018
21
#slack
Communicating
Profit? Some case examples…

22.
25/01/2018
22
…just to point out only 3 of them.
Flight & PAX data
• The most crucial data asset
of all – drives the whole
business
• Once we made the data
modeling properly it’s been
easy to connect more data
assets and domains to it
•  Pay serious attention to
data modeling.
Security access control data
• Three sources: Boarding
pass stamps, line queue
length sensors, security
lane throughput sensors
• Analytic MDM: harmonized
master data
• Thanks to business- driven
and holistic approach to
data modeling: Joined the
data to flight data 
detailed insights: dwell
times and throughtput
times per destination etc.
Strategic data acquisitions
• Connect data from various
API sources (internal and
external)
• Once new operational
systems are introduced,
make sure that the data is
available via APIs
• Start collecting and
introduce early prototypes
and test versions in
advance.
25.1.201843
#AI

23.
25/01/2018
23
?
PAX Nationality
User experience
25.1.201846
”Tämä on just niin kuin pitikin.”
Datahubin mahdollistama tulevien lentojen työkalu on
käytössä eikä vain paranna työtäni, mutta myös
vähentää tästä eteenpäin manuaalisen työni määrää
useita tunteja jokainen kuukausi. Power BI -työkalun
vaatimat muutokset Datahubin puolelle sain läpi
monesti jo saman päivän aikana ilmoituksesta ja
useat mieleeni tulleet kehitysehdotukset olivat
itse asiassa työnalla jo ennen kuin ehdin niistä
sanomaan.
Huippuduunia!
Delay seurantaan ja analyysiin kehiteltävänä oleva Delay Dashboard näyttää
jo kehityksen tässä vaiheessa olevan matkalla kohden toimivaa,
monipuolista, joustavaa sekä helppokäyttöistä työkalua jota on jo kauan
kaivattu niiden käyttäjien toimesta jotka tarvitsevat ja analysoivat näitä
tietoja kehittääkseen toimintaa eri vastuualueilla.

24.
25/01/2018
24
Lessons learned and
key takeaways
Lessons learned and key takeaways
25.1.201848
Be brave: ”roppaa likoon”
• Have the courage! Move forward step by step.
• ”Learning by doing”: ask and collect feedback!
Technology
• Technology does matter
• Cloud, [nnn] as a Service, Agile, focus on bringing
value!
Change
• Old habits die hard. Question the old ways to do
things. Doubt the old ways to think.
• But not all of them! Praise the ”good old” stuff!

25.
25/01/2018
25
#bigdata
#saddata*
#IoT
BUT beware of…
”Data you forced to collect even though no-one
wants it as a customer & no-one needs it in your
business & no-one can utilize.”
-Jarno Kartela, Artificial Intelligence Partner @ Fourkind
#machinelearning
#analytics
#API

26.
25/01/2018
26
“Knowledge that does not change
behavior is useless.”
Yuval Noah Harari, Homo Deus: A Brief History of Tomorrow
Thank you.
tommi.vihervaara@finavia.fi
@GreenyDangerous
+358 40 518 1090
https://www.linkedin.com/in/tommi-vihervaara-1593733/