Presentation given at the 2016 Air Transport Research Society (ATRS) conference in Rhodes, Greece. This is the first in a series on methods and means to approach data management, modeling, analytics and predictive analytics. This work uses the T100 data from the USDOT's Office of Airline Information (OAI) to explore options in database keys to establish a scalable foundation for aerospace data analytics.

1. Effec%ve Data Science in
Aerospace Applica%ons
Geoffrey Clark | 2016-06-25 | GR-81-RHO
copyright Lucidata 2016

2. Lucidata Informa%cs?
• Geoffrey Clark, Principal and CEO
– Data Modeler: logical and ra%onal DW
– Solu%on Architect: analy%cs
– Strategy : risks, what-if, sims, wargames
• Associates at Lucidata
– Senior Data Modeler, requirements expert
– Advanced Analy%cs experts
– Senior GIS analysts and FOSS4G development
copyright Lucidata 2015

3. Dual Challenges of Analy%cs
Market
TMS
CRM
Analytics Solution
Standard BI
Formal, Trusted,
Shared, Public
Data Explora0on
Ad-hoc, Agile
Experimental
indivi
duals
Power
Users
Clean, Load
& Join Data
Form New
BI Ques%ons
Analyze
Data
Find New Data
Analytics User Segments
FIN
Your
Data
Knowledge,
Defini%ons,
Hierarchies,
Rela%onships
Ad-Hoc Data
Playground
Standard,
Industry
Data Industry
Reference
Execu%ves,
Managers
Analysts,
Knowledge
Workers
Technical Challenge, Data Integra%on
Cultural Challenge, User Integra%on
How Unique is your data?*
* is that a source of advantage, or confusion?

4. En%ty-Rela%onship (ER) Modeling
This is an example En.ty-Rela.onship Diagram (ERD), which explains how to read the
nota.on. It is also an example of a highly abstract model, which is "data driven",
meaning that new Thing Types, new Things and new Thing Rela.onships may be easily
added to a database based on this design without needing to change the database
structures. This prac.ce was common in early databases, built for on-line transac.on
processing (OLTP). This stands in contrast to the concrete business seman.cs
implemented as part of Dimensional Data Modeling efforts, suppor.ng on-line analy.cal
processing (OLAP).
copyright Lucidata 2016

5. Simulate
Op,mize
Forecast
Derive (data mining)
Summarize & Describe (sta%s%cs)
Visualize
Join & Filter (data warehousing)
Measure & Store (source systems)
En,,es & Rela,onships (data modeling)
BI EA STATS OR
Analy,cs
Quality
AMATEUR PROFESSIONAL
The Progression of Analy%cs
copyright Lucidata 2015
5
... rest upon
this founda%on
These types
of models ...

6. Big Data History, via Google Trends
Source: heps://www.google.com/trends/

7. What research did we do?
"p1sk" - project #1, surrogate keys.
airport_id = 2369
"p2nk" - project #2, natural keys.
airport_iata_cd = 'SEA’
airport_from_dt = '2011-07-01’
"p3uu" - project #3, universally unique iden%fier (UUID),
airport_uuid = 7cbcc311-18c9-4497-99c9-62c42fd1ef2b
"p4hk" - project #4, hash key.
airport_key = 0ed805d25fc96166a5895857a252de4b
What has the Big Data innova.on cycle taught us about data design?

8. Original T100 “Green Book”

9. Data Source: T100 “Green Book”

11. Importance of Reference Data
copyright Lucidata 2015
If you have a flood of .mestamps, beNer know what .me zone they represent.
And, don’t be like the Mars Climate Orbiter, get your units of measure right!

12. Nbr
Full Number
ISO 31
Description
Comments
10^0
1
C62
one (or unit)
"EA" for each from ANSI
10^1
10
ten
10^2
100
CEN
one hundred
10^3
1000
MIL
one thousand
10^4
10,000
ten thousand
10^5
100,000
one hundred thousand
10^6
1,000,000
MIO
one million
Somewhat confusing
10^7
10,000,000
ten million
10^8
100,000,000
one hundred million
10^9
1,000,000,000
MLD
one milliard in EU one billion (US)
Horribly confusing!
10^12
1,000,000,000,000
BIL
one billion in EU (one trillion in US)
Horribly confusing!
10^18
1,000,000,000,000,000,000
TRL
one trillion (EUR)
Horribly confusing!
ISO 31 - Units of Measure
More work remains to beeer coordinate the measurement
ac%vi%es of humankind, the future will appreciate it!

13. project #1, surrogate key
project #2, natural keys
project #3, universally unique id
project #4, hash key

14. Addi%onal Factors to Consider
Source: Dan Linstedt on learndatavault.com.

15. Addi%onal Research Plans
• How does data design change when using
distributed database technology? aliases -
massive parallel processing (MPP), “Sharded”
• How does data design change when using
columnar database technology?
• How does data design change when using graph
database technology? aliases – “RDF”, “triple
store”.
• How does performance change with different
disk op%ons – HDD, SSD, SSD RAIDS, etc.

20. Informa%on Density
Charles Joseph Minard’s Carte Figura.ve from 1869, depic%ng Napolean’s 1812 invasion of
Russia, and aqermath in seven dimensions (la%tude, longitude, %me, temperature, army
group, and military phase). “The best sta.s.cal drawing ever made” -- Edward Tuqe
Source: heps://en.wikipedia.org/wiki/File:Minard.png