1.
Welcome

2.
It used to be easy…

3.
they all looked pretty much alike

4.
NoSQL BigData MapReduce Graph Document Shared Column EventualBigTable CAP Nothing Oriented Consistency ACID BASE Mongo Coudera HadoopVoldemort Cassandra Dynamo Marklogic Redis Velocity Hbase Hypertable Riak BDB

5.
Now it’s downrightc0nfuZ1nG!

6.
What Happened?

7.
we changed scale

8.
tack ge dwe ch an

9.
thebig dataconundrum

10.
thebig dataconundrum ?

11.
The Internet

12.
Which isn’t mostly textEverything (500,000) Web Pages (40) ~0.01% is web pages Words (0.6) ~1% of that is text Sizes in Petabytes

13.
And there is lots of other stuff out there mobile weather sensors Social Logs data audio video

14.
Gartner80% of business is conducted on unstructured information

15.
Big Data is now a new class of economic asset* *World economic forum 2012

16.
Yet 80% Enterprise Databases < 1TB Reference from 2009

17.
so what does Big Data mean forthe enterprise?

18.
Insight α data >Data beats Algorithms

19.
Backing up a bit…

20.
We live in a world of, largely, private data Where data is often changed and forwarded on

21.
Sometimes we’re a bit more organized!

22.
But most of our data is not generally accessible Core Operational Exposed Data

23.
Sharing is often an afterthought Core Operational Exposed Data

24.
How do we process, acquire, reason about and act upon information?

25.
The Brain Reptilian Primitive operations: balance, temperature regulation, breathing Mammalian Emotion, short-term memory, flee of fight etc (limbic) Neocortex Plan, innovate, problem solve etc

26.
Our intelligence is segregated in disparate worlds

27.
could our corporations bemore intelligent?

28.
Siloed, closed, bespoke data makes our organisations opaque and unresponsive

29.
What if we exposed it all?

30.
So what might that look like?• Single data store• Federated, homogenous stores• Federated, heterogeneous stores

31.
The Google Approach MapReduce Google Filesystem BigTable Tenzing Megastore F1 Dremel Spanner

32.
The Ebay Approach

33.
so is one approachbetter?

34.
Data Volume?TB 0 1 10 100 1000 10,000 We live well within the overlap region

35.
Academic acumen?

36.
Performance Trade-Off Curve• Volume (pure physical size)• Velocity (rate of change)• Variety (number of different types of data, formats and sources)• Static & Dynamic Complexity (do you need to interpret the affect one message has on another)

37.
ProblemOur ability to model data is much more of a gating factor than raw size, particularly when considering new forms of data Dave Campbell (Microsoft – VLDB Keynote)

38.
Gravitate around a single data model Globally Accessible Application Specific Core Data Models Core Data Model ModelViews /linkages

39.
The data itself follows a similar pattern Globally Accessible Application Specific Core Data Core Data Model dataViews /linkages

40.
Compose Solutions (for now)

41.
Big Data is more than the opportunity for better insight over new data sources

42.
It is the opportunity to make the organisation smarter, simply by making data more accessible

43.
But the harder job, for us, is unifyingthe various domains to make all that data intelligible

44.
Thankshttp://www.benstopford.com @benstopford