Data it's big, so, grab it, store it, analyse it, make it accessible...mine, warehouse and visualise...use the pictures in your mind and others will see it your way!

1. Data Warehousing, Data Mining
&
Data Visualisation
Introduction

2. Data Warehousing

3. What is a Data Warehouse?
• A data warehouse is a database used for
reporting and analysis.
• The data stored in the warehouse is uploaded
from the operational systems.
• The data may pass through an operational data
store for additional operations before it is used
in the data warehouse for reporting.

4. A data-processing database? Wholesaling Data?

5. Benefits of a Data Warehouse
A data warehouse maintains a copy of information from the source
transaction systems. This architectural complexity provides the
opportunity to:
• Maintain data history.
• Integrate data from multiple source systems.
• Improve data quality.
• Present the organisation's information consistently.
• Provide a single common data model for all data of interest regardless of
the data's source.
• Restructure the data so that it makes sense to the business users.
• Restructure the data so that it delivers excellent query performance, even
for complex analytic queries.
• Add value to operational business applications.

6. History of Data Warehousing
• 1990 — Red Brick Systems, founded by Ralph Kimball,
introduces Red Brick Warehouse, a database management
system specifically for data warehousing.
• 1991 — Prism Solutions, founded by Bill Inmon, introduces
Prism Warehouse Manager, software for developing a data
warehouse.
• 1992 — Bill Inmon publishes the book Building the Data
Warehouse.
• 1995 — The Data Warehousing Institute, a not-for-profit
organisation that promotes data warehousing, is founded.
• 1996 — Ralph Kimball publishes the book The Data
Warehouse Toolkit.
• 2000 — Daniel Linstedt releases the Data Vault, enabling real
time auditable data warehouses.

7. Dimensional v Normalised
There are two leading approaches to storing data in a data warehouse
— the dimensional approach and the normalised approach.
• The dimensional approach, whose supporters are referred to as
“Kimballites”, believe in Ralph Kimball’s approach in which it is
stated that the data warehouse should be modelled using a
Dimensional Model (DM). For example, a sales transaction can be
broken up into facts such as the number of products ordered and
the price paid for the products, and into dimensions such as order
date, customer name, product number, order ship-to and bill-to
locations, and salesperson responsible for receiving the order.
• The normalised approach, also called the 3NF model, whose
supporters are referred to as “Inmonites”, believe in Bill Inmon's
approach in which it is stated that the data warehouse should be
modelled using Peter Chen’s Entity-Relationship (ER) model with
which, of course, we are all familiar!

8. Kimball’s Bottom Up Design
• In the bottom-up approach data marts are first
created to provide reporting and analytical
capabilities for specific business processes.
• Data marts contain, primarily, dimensions and facts.
• Facts can contain either atomic data and, if
necessary, summarised data.
• The single data mart often models a specific business
area such as "Sales" or "Production."
• These data marts can eventually be integrated to
create a comprehensive data warehouse.

9. Inmon’s Top Down Design
Inmon states that the data warehouse is:
• Subject-oriented: The data in the data warehouse is
organised so that all the data elements relating to
the same real-world event or object are linked
together.
• Non-volatile: Data in the data warehouse are never
over-written or deleted — once committed, the data
are static, read-only, and retained for future
reporting.
• Integrated: The data warehouse contains data from
most or all of an organisation's operational systems
and these data are made consistent.

10. Hybrid Design
• Data warehouse (DW) solutions often resemble hub
and spoke architecture.
• Legacy systems feeding the DW solution often
include customer relationship management (CRM)
and enterprise resource planning solutions (ERP),
generating large amounts of data.
• To consolidate these various data models, and
facilitate the extract transform load (ETL) process,
DW solutions often make use of an operational data
store (ODS).

11. Data Warehouse Appliances
• IBM Netezza
• Oracle ExaData
• Kognitio 360
• Teradata

12. Demystifying the Data Warehouse
http://www.youtube.c
om/watch?
v=mgEugd5kZgk&featu
re=related

13. Data Mining
(KDD);

14. What is Data Mining?
• Data mining is the analysis step of the
Knowledge Discovery in Databases (KDD)
process.
• It is a relatively young and interdisciplinary
field of computer science.
• It is the process of discovering new patterns
from large data sets involving methods at the
intersection of artificial intelligence, machine
learning, statistics and database systems.

15. The KDD Process
The knowledge discovery in databases (KDD)
process is commonly defined in 5 stages:
(1) Selection
(2) Preprocessing
(3) Transformation
(4) Data Mining
(5) Interpretation/Evaluation

16. The CRISP-DM Process
The CRoss Industry Standard Process for Data Mining
(CRISP-DM) defines six phases:
(1)Business Understanding
(2) Data Understanding
(3) Data Preparation
(4) Modelling
(5) Evaluation
(6) Deployment
The simplified process is (1) Pre-processing, (2) Data
mining and (3) Results validation

17. Spatial Data Mining
• Spatial data mining is the application of data mining methods
to spatial data.
• Spatial data mining follows along the same functions in data
mining, with the end objective to find patterns in geography.
• So far, data mining and Geographic Information Systems (GIS)
have existed as two separate technologies, each with its own
methods, traditions and approaches to visualization and data
analysis.
• The immense explosion in geographically referenced data
occasioned by developments in IT, digital mapping, remote
sensing, and the global diffusion of GIS emphasises the
importance of developing data driven inductive approaches
to geographical analysis and modelling.

18. Build a KPI Dashboard in 5 Minutes
http://www.youtube.c
om/watch?
v=D4S_uIIZyN0&featur
e=related
Build a KPI Dashboard in 5 minutes
with no programming in Excel 2010

19. Data Visualisation
Choose 6 of the Keywords in the above!

20. Data Visualisation Defined
Data visualisation is the
study of the visual
representation of data,
meaning "information
that has been
abstracted in some
schematic form,
including attributes or
variables for the units
of information".
Friendly 2008

21. Tufte and Data Visualisation
‘The success of
visualisation is based
on deep knowledge and
care about the
substance and the
quality, relevance and
integrity of the
content.’
Tufte 1983

22. 5 Principles of Graphic Display
1. Above all else, show the data.
2. Maximise the data-ink ratio.
3. Erase non-data-ink.
4. Erase redundant data-ink.
5. Revise and edit.

23. The Beauty of Data Visualisation
http://www.youtube.com/
watch?v=pLqjQ55tz-U
David McCandless

24. Gapminder
A Data Mining & Data Visualisation Tool

25. Hans Rosling
• The Gapminder application is the brain-child
of Hans Rosling.
• He thought of the title when he heard the
prompt ‘mind the gap’ on the London
Underground.
• He is Professor of International Health at
Karolinska Institute, Stockholm, Sweden.
• He is a Doctor of Medicine and a Doctor of
Philosophy.

26. Hans uses Gapminder
http://www.ted.com/talks/hans_
rosling_shows_the_best_stats
_you_ve_ever_seen.html
http://www.ted.com/talks/hans_
rosling_reveals_new_insights_
on_poverty.html

27. Gapminder Desktop
Gapminder Desktop
allows you to show
animated statistics
from your own laptop.
In short:
• Use Gapminder World
without internet
access.
• Save a list of your own
favourite graphs.
• Updates automatically
when new data is
available

28. Tableau Desktop

29. Gephi

30. VOSViewer

31. Hjalmar Gislason
"Falling in Love with Data"
http://www.youtube.co
m/watch?v=fOg0QHUI-
lM&feature=plcp

32. 20 Top Tools for Data
Visualisation
http://m.netmagazine.
com/features/top-20-
data-visualisation-tools

33. And another angle…
http://deverell.computi
ng.dundee.ac.uk/~cjma
rtin/dataVis.m4v