Mastering Enterprise Big Data
Concept Overview

March, 2014

Copyright 2014 – Semantech Inc.

1
Introduction

“Big Data” may very well be the most over-hyped and
misunderstood trend in IT today. The goal of this
presentation is to help demystify the topic by gaining
a better understanding of how it ought to exploited
and managed.
This briefing is not meant to address key technical
concepts or a “Big Data 101.” We’re going to focus
instead on the real-world challenges associated with
Big Data implementations in the context of the larger
enterprise environment.
Copyright 2014 – Semantech Inc.

2
Where Does Big Data Fit ?

All Data-related
technologies fit within the
same Data Continuum –
Data must be managed, it
can be discovered and then
hopefully exploited…
Copyright 2014 – Semantech Inc.

3
The Appeal of Big Data

The appeal of Big Data is two-fold;
1) it lies in the implied potential of technology
being able to keep pace with the exponential
growth of data volume and
2) there is an expectation that a leaner approach
to data management will ultimately make the
enterprise easier to run…
These notions are both true and false. Yes, Big Data
technology is designed to both increase volume and
speed – yet it is not a silver bullet and many fail to
recognize that any such capability must reside within
already existing enterprise ecosystems.
Copyright 2014 – Semantech Inc.

4
What is Big Data, Really?
Does it refer to size, to volume to velocity, to flexibility in
data types? Does it refer to the types of systems and
algorithms loosely confederated under the Big Data
umbrella – NoSQL, Hadoop, Key Value Pair Databases,
Document Databases, Graph Databases…

Is Big Data Operational or Analytic or both? The more
people try to define it, the more it’s starting to sound like
the all “legacy” data technologies it is supposed to be
eclipsing. The truth is that there simply isn’t one standard
definition that encompasses the variety of Big Data
technology that now exists. It’s evolved past that point
already. The main thing in common with all Big Data
solutions is that they are looking to shift old paradigms.
Copyright 2014 – Semantech Inc.

5
Where Does Big Data Fit?
Are traditional Data Management and
Architectures now obsolete with the
oncoming waves of Big Data
technology?

Maybe Not – Maybe Big Data isn’t as revolutionary as we think.

Copyright 2014 – Semantech Inc.

6
Things to Keep in Mind
1. Existing database technologies are not going away
anytime soon.
2. All Data is an enterprise asset, thus all Data must be
managed in the context of the larger environment.
3. Technology must serve a purpose – a new technology
can help define new applications – however, to justify
an investment, a valid Use Case (or Use Cases) must
eventually appear.
4. Fast Data and lot’s of Data are of little value without
integrity and context.
5. Databases are systems – systems have architectures
and themselves form parts of larger architectures.
6. All IT solutions require Governance.
Copyright 2014 – Semantech Inc.

7
The Real Challenge with Big Data

There are three main challenges that tend to plague
every Big Data project today:
1. Lack of a Strategy, Use Cases and a Value
Proposition that fits not just in the context of one
project, but in the context of the enterprise.
2. Lack of a Design and Governance Framework that
can manage the solution lifecycle of any Big Data
project.
3. Lack of an Integrated Architecture that properly
leverages Big Data capabilities within the larger
ecosystem of enterprise Data systems.
Copyright 2014 – Semantech Inc.

8
How Do you Design Big Data?

Traditional elements of the
Data Enterprise are well
understood and generally have
clear expectations for both
design and management.
So, can Big Data fit into a
picture like this? If not , is it
acceptable to operate without
such understanding ?
9
“Enterprise Big Data” Defined
When Big Data was limited primarily to Hadoop
databases focused on one or two internet focused Use
Cases driven by Google, the idea of Big Data was much
easier to grasp.
Now that Big Data has “grown up,” there is no simple or
standard way to view Big Data, unless we also expand
our scope. Once Big Data jumped from one technology to
dozens, from one Use Case to dozens, from one industry
to dozens and from one prototype system to a production
element within an enterprise – it became something new.
It became “Enterprise Big Data” – and it’s never going
back.
Copyright 2014 – Semantech Inc.

10
“Enterprise Big Data” Defined 2
Enterprise Big Data:
The collection of technologies designed to handle the
explosion of data associated with 21st century IT
systems and Internet applications. This technology is
not meant to replace all existing database capability
but rather to supplement it in cases where
performance of large or complex data sets requires
more dynamic and flexible management.
Another key aspect is that Enterprise Big Data is just
that – it is for the Enterprise, not just Google – it is a
collection of technologies designed to serve the
enterprise and ultimately reside within it.
Copyright 2014 – Semantech Inc.

11
Understanding the Challenge

Let’s take a look at some of the challenges associated
with deploying Big Data capability in real-world enterprises
12
Challenge 1: Strategy
The Challenge:
Demonstrating a technology is one thing –
a relatively easy thing. Demonstrating the
value of that technology within your
organization is something entirely different.
How do you decide when and how to
employ Big Data capability and more
importantly how do you make it relevant?
Typical Problems that Arise:
1. The typical web-focused Use Cases
don’t seem to apply in your org.
2. There isn’t a clear path as to how
the technology will improve
efficiency or fuel growth.
3. The solution seems to be
competing with similar capability The Strategy is missing…
(both new and legacy) with no
clear plan for reconciliation.
Copyright 2014 – Semantech Inc.

13
Challenge 2: Governance
The Challenge:
Big Data solutions cannot exist separate
from the rest of the mission and
infrastructure of an enterprise. Yet, there is
no standard Big Data management or
Governance framework in IT.
Typical Problems that Arise:
1. Data Integrity is not evaluated at
all (for Big Data).
2. Solution Lifecycle Management is
absent.
3. There are conflicting views as to
whether it can be governed at all.
4. Big Data solutions are seriously out
of touch with business needs or
representatives.
5. There is no metrics framework in
place to understand value.
Copyright 2014 – Semantech Inc.

14
Challenge 3: Enterprise Data Integration
The Challenge:
Both at the Operational and Analytical
level, Big Data represents only part of a
larger picture. And it is no longer easy to
determine just where Big Data fits in that
big picture. In order to actualize any
strategy - data discovery, exploitation and
management must be integrated.

Typical Problems that Arise :
1. There are no standard Big Data
Architecture patterns.
2. There are often no clear design
strategies for integrating Big Data
with ETL, Data Warehouses and
Master Data Management
systems.
3. No one knows how to model Big
Data.
Copyright 2014 – Semantech Inc.

15
Top 10 Lists

The Do’s and Don’t
of Big Data

16
Top 10 Don’ts
1. Don’t initiate Big Data projects without an enterprise strategy.
2. Don’t let your techies run the project without business input.
3. Don’t assume Big Data can’t be designed, modeled or
architected.
4. Don’t assume that Big Data ought to be limited to Internet or
Social Media data.
5. Don’t assume 1 Big Data technology will support all of your
needs. One size doesn’t fit all.
6. Don’t replace exist technologies too soon.
7. Don’t assume that Big Data will be focused on a narrow set of
data formats.
8. Don’t assume that Big Data can’t be governed (as data or as
systems).
9. Don’t separate management of other data systems from Big
Data solutions. (Don’t reinvent the wheel)
10. Don’t start without defining your Use Cases.
Copyright 2014 – Semantech Inc.

17
Top 10 Do’s
1.

Do adopt Big Data Technology, when you’re ready and if it makes sense
– but validate that first.
2. Do integrate Governance and management of Big Data with the rest of
your enterprise data architecture.
3. Do design Big Data solutions – both as systems and as data.
4. Do evaluate All of the available Big Data technologies before deciding
which one/s are the best fit.
5. Do integrate your existing ETL and ESB / Middleware infrastructure
with your Big Data solution from the beginning.
6. Do employ both Semantic modeling and Master Data Management
(MDM) for Big Data – and yes it is possible.
7. Do update and revise enterprise processes to accommodate new
technology and capability when necessary.
8. Do create a security plan as part of the initial Big Data strategy,
especially if that data resides in the Cloud.
9. Do your homework. Do use an Architect to help with your project.
10. Do assess and question your initial assumptions and strategy and
amend after gathering lessons learned.
Copyright 2014 – Semantech Inc.

18
Taking the First Step

Mastering Enterprise
Big Data…

19
A Realization, A Foundation

The first step towards mastering enterprise Big Data
is understanding the realization that regardless of
whether data has a formal structure – like Third
Normal Form (relational), Hierarchy, Schema
Dimensions or little structure (like many Big Data
solutions) – all data can be classified through
Semantics.
Data Classification then facilitates Data
Discovery, Data Management and Data Integration.
Big Data can be classified and modeled within the
context of a larger paradigm.
This is the first step – it is the foundation.
Copyright 2014 – Semantech Inc.

20
Step 1: Provide the Foundation

There needs to be a bridge that
can span every data system,
data source and element. This
is our foundation
21
Big Data in the Enterprise

We will explain this high level or
Conceptual Architecture in greater
depth in our next presentation

22
Conclusion

Semantech Inc. has presented this introductory
topic as the first in a series of briefings on
Enterprise Big Data. The follow-on briefings will
include:
1. How to Architect Enterprise Big Data Solutions
2. How to Model Enterprise Big Data
3. How to Secure Enterprise Big Data Systems
4. How to Govern Enterprise Big Data
5. Enterprise Big Data real-world Scenarios and
Case Studies.
23
Some of Our Clients

24
Thank You
Come visit us at:
http://www.semantech-inc.com
25
25

Semantech Inc. - Mastering Enterprise Big Data - Intro

  • 1.
    Mastering Enterprise BigData Concept Overview March, 2014 Copyright 2014 – Semantech Inc. 1
  • 2.
    Introduction “Big Data” mayvery well be the most over-hyped and misunderstood trend in IT today. The goal of this presentation is to help demystify the topic by gaining a better understanding of how it ought to exploited and managed. This briefing is not meant to address key technical concepts or a “Big Data 101.” We’re going to focus instead on the real-world challenges associated with Big Data implementations in the context of the larger enterprise environment. Copyright 2014 – Semantech Inc. 2
  • 3.
    Where Does BigData Fit ? All Data-related technologies fit within the same Data Continuum – Data must be managed, it can be discovered and then hopefully exploited… Copyright 2014 – Semantech Inc. 3
  • 4.
    The Appeal ofBig Data The appeal of Big Data is two-fold; 1) it lies in the implied potential of technology being able to keep pace with the exponential growth of data volume and 2) there is an expectation that a leaner approach to data management will ultimately make the enterprise easier to run… These notions are both true and false. Yes, Big Data technology is designed to both increase volume and speed – yet it is not a silver bullet and many fail to recognize that any such capability must reside within already existing enterprise ecosystems. Copyright 2014 – Semantech Inc. 4
  • 5.
    What is BigData, Really? Does it refer to size, to volume to velocity, to flexibility in data types? Does it refer to the types of systems and algorithms loosely confederated under the Big Data umbrella – NoSQL, Hadoop, Key Value Pair Databases, Document Databases, Graph Databases… Is Big Data Operational or Analytic or both? The more people try to define it, the more it’s starting to sound like the all “legacy” data technologies it is supposed to be eclipsing. The truth is that there simply isn’t one standard definition that encompasses the variety of Big Data technology that now exists. It’s evolved past that point already. The main thing in common with all Big Data solutions is that they are looking to shift old paradigms. Copyright 2014 – Semantech Inc. 5
  • 6.
    Where Does BigData Fit? Are traditional Data Management and Architectures now obsolete with the oncoming waves of Big Data technology? Maybe Not – Maybe Big Data isn’t as revolutionary as we think. Copyright 2014 – Semantech Inc. 6
  • 7.
    Things to Keepin Mind 1. Existing database technologies are not going away anytime soon. 2. All Data is an enterprise asset, thus all Data must be managed in the context of the larger environment. 3. Technology must serve a purpose – a new technology can help define new applications – however, to justify an investment, a valid Use Case (or Use Cases) must eventually appear. 4. Fast Data and lot’s of Data are of little value without integrity and context. 5. Databases are systems – systems have architectures and themselves form parts of larger architectures. 6. All IT solutions require Governance. Copyright 2014 – Semantech Inc. 7
  • 8.
    The Real Challengewith Big Data There are three main challenges that tend to plague every Big Data project today: 1. Lack of a Strategy, Use Cases and a Value Proposition that fits not just in the context of one project, but in the context of the enterprise. 2. Lack of a Design and Governance Framework that can manage the solution lifecycle of any Big Data project. 3. Lack of an Integrated Architecture that properly leverages Big Data capabilities within the larger ecosystem of enterprise Data systems. Copyright 2014 – Semantech Inc. 8
  • 9.
    How Do youDesign Big Data? Traditional elements of the Data Enterprise are well understood and generally have clear expectations for both design and management. So, can Big Data fit into a picture like this? If not , is it acceptable to operate without such understanding ? 9
  • 10.
    “Enterprise Big Data”Defined When Big Data was limited primarily to Hadoop databases focused on one or two internet focused Use Cases driven by Google, the idea of Big Data was much easier to grasp. Now that Big Data has “grown up,” there is no simple or standard way to view Big Data, unless we also expand our scope. Once Big Data jumped from one technology to dozens, from one Use Case to dozens, from one industry to dozens and from one prototype system to a production element within an enterprise – it became something new. It became “Enterprise Big Data” – and it’s never going back. Copyright 2014 – Semantech Inc. 10
  • 11.
    “Enterprise Big Data”Defined 2 Enterprise Big Data: The collection of technologies designed to handle the explosion of data associated with 21st century IT systems and Internet applications. This technology is not meant to replace all existing database capability but rather to supplement it in cases where performance of large or complex data sets requires more dynamic and flexible management. Another key aspect is that Enterprise Big Data is just that – it is for the Enterprise, not just Google – it is a collection of technologies designed to serve the enterprise and ultimately reside within it. Copyright 2014 – Semantech Inc. 11
  • 12.
    Understanding the Challenge Let’stake a look at some of the challenges associated with deploying Big Data capability in real-world enterprises 12
  • 13.
    Challenge 1: Strategy TheChallenge: Demonstrating a technology is one thing – a relatively easy thing. Demonstrating the value of that technology within your organization is something entirely different. How do you decide when and how to employ Big Data capability and more importantly how do you make it relevant? Typical Problems that Arise: 1. The typical web-focused Use Cases don’t seem to apply in your org. 2. There isn’t a clear path as to how the technology will improve efficiency or fuel growth. 3. The solution seems to be competing with similar capability The Strategy is missing… (both new and legacy) with no clear plan for reconciliation. Copyright 2014 – Semantech Inc. 13
  • 14.
    Challenge 2: Governance TheChallenge: Big Data solutions cannot exist separate from the rest of the mission and infrastructure of an enterprise. Yet, there is no standard Big Data management or Governance framework in IT. Typical Problems that Arise: 1. Data Integrity is not evaluated at all (for Big Data). 2. Solution Lifecycle Management is absent. 3. There are conflicting views as to whether it can be governed at all. 4. Big Data solutions are seriously out of touch with business needs or representatives. 5. There is no metrics framework in place to understand value. Copyright 2014 – Semantech Inc. 14
  • 15.
    Challenge 3: EnterpriseData Integration The Challenge: Both at the Operational and Analytical level, Big Data represents only part of a larger picture. And it is no longer easy to determine just where Big Data fits in that big picture. In order to actualize any strategy - data discovery, exploitation and management must be integrated. Typical Problems that Arise : 1. There are no standard Big Data Architecture patterns. 2. There are often no clear design strategies for integrating Big Data with ETL, Data Warehouses and Master Data Management systems. 3. No one knows how to model Big Data. Copyright 2014 – Semantech Inc. 15
  • 16.
    Top 10 Lists TheDo’s and Don’t of Big Data 16
  • 17.
    Top 10 Don’ts 1.Don’t initiate Big Data projects without an enterprise strategy. 2. Don’t let your techies run the project without business input. 3. Don’t assume Big Data can’t be designed, modeled or architected. 4. Don’t assume that Big Data ought to be limited to Internet or Social Media data. 5. Don’t assume 1 Big Data technology will support all of your needs. One size doesn’t fit all. 6. Don’t replace exist technologies too soon. 7. Don’t assume that Big Data will be focused on a narrow set of data formats. 8. Don’t assume that Big Data can’t be governed (as data or as systems). 9. Don’t separate management of other data systems from Big Data solutions. (Don’t reinvent the wheel) 10. Don’t start without defining your Use Cases. Copyright 2014 – Semantech Inc. 17
  • 18.
    Top 10 Do’s 1. Doadopt Big Data Technology, when you’re ready and if it makes sense – but validate that first. 2. Do integrate Governance and management of Big Data with the rest of your enterprise data architecture. 3. Do design Big Data solutions – both as systems and as data. 4. Do evaluate All of the available Big Data technologies before deciding which one/s are the best fit. 5. Do integrate your existing ETL and ESB / Middleware infrastructure with your Big Data solution from the beginning. 6. Do employ both Semantic modeling and Master Data Management (MDM) for Big Data – and yes it is possible. 7. Do update and revise enterprise processes to accommodate new technology and capability when necessary. 8. Do create a security plan as part of the initial Big Data strategy, especially if that data resides in the Cloud. 9. Do your homework. Do use an Architect to help with your project. 10. Do assess and question your initial assumptions and strategy and amend after gathering lessons learned. Copyright 2014 – Semantech Inc. 18
  • 19.
    Taking the FirstStep Mastering Enterprise Big Data… 19
  • 20.
    A Realization, AFoundation The first step towards mastering enterprise Big Data is understanding the realization that regardless of whether data has a formal structure – like Third Normal Form (relational), Hierarchy, Schema Dimensions or little structure (like many Big Data solutions) – all data can be classified through Semantics. Data Classification then facilitates Data Discovery, Data Management and Data Integration. Big Data can be classified and modeled within the context of a larger paradigm. This is the first step – it is the foundation. Copyright 2014 – Semantech Inc. 20
  • 21.
    Step 1: Providethe Foundation There needs to be a bridge that can span every data system, data source and element. This is our foundation 21
  • 22.
    Big Data inthe Enterprise We will explain this high level or Conceptual Architecture in greater depth in our next presentation 22
  • 23.
    Conclusion Semantech Inc. haspresented this introductory topic as the first in a series of briefings on Enterprise Big Data. The follow-on briefings will include: 1. How to Architect Enterprise Big Data Solutions 2. How to Model Enterprise Big Data 3. How to Secure Enterprise Big Data Systems 4. How to Govern Enterprise Big Data 5. Enterprise Big Data real-world Scenarios and Case Studies. 23
  • 24.
    Some of OurClients 24
  • 25.
    Thank You Come visitus at: http://www.semantech-inc.com 25 25