Developing Enterprise Data Discovery Strategy

Developing Enterprise
Data Discovery Strategy
WHITE PAPER BY INTERSOG
www.intersog.com
2015

Intersog: Premium Digital Solutions for IoT, Cloud, Big Data and wearable tech! 2
Data Discovery is defined as end users’ ability to use Big Data tools and platforms to
access data from many different sources, aggregate and combine data subsets from
these data sources, and analyze them to identify unknown patterns and trends. It’s
also a way for end users to find insights particular to their role, perhaps in a search
for a pattern that a business intelligence (BI) analyst did not have in mind.
Data discovery also eliminates the need for end-users to learn complicated and
cumbersome tools in order to interact with data, and allows for real-time analysis
without the need for cubes or pre-built data models.
For companies looking to analyze and integrate massive volumes of raw data
generated from and residing in multiple disparate sources, data discovery can
enhance the way enterprise data is accessed and used for strategic planning.
Defining Data Discovery

Traditional BI solution providers have tried
hard to meet the ever-growing needs of
Big Data analysts by providing business
user driven data discovery capabilities and
encouraging adoption of data discovery
and visualization tools through bundling
and integration with the rest of their
technology stack. This has led to the
current situation when most of Big Data
tools available in the market can only be
used effectively by a certain group of users
such as data scientists and analysts.
As many enterprises tend to implement a
decentralized and bimodal managed data
discovery approach today, the next-
generation BI tools are emerging to enable
self-service capabilities and interactive
visualization of identified patterns and
trends. To cut it short, any average
business user should have access to Big
Data and ability to translate it into the
effective BI that consists in better data-
driven decisions, more accurate predictive
analytics and optimization of processes
and approaches.
This white paper aims to explain the
process of converting data into BI insights
and help business decision makers
evaluate their Big Data efforts and define
the right strategy of turning raw data into
visually attractive charts of easily
understandable and, what’s more
important, precise information.
“By 2018, United States alone could
face a shortage of 140,000 to
190,000 people with deep analytical
skills as well as 1.5 million managers
and analysts with the know-how to
use the analysis of big data to make
effective decisions” –
McKinsey, 2015
Quote 1
Introduction

Poised to significantly reshape
industries from healthcare to retail to
professional services, Big Data remains
a cumbersome, process-laden and
time-consuming function that is highly
dependent on skilled data analysts
and scientists. Enterprises have yet to
find a new path of converting their
messy data into well-structured
Business Intelligence (BI) that would
enable business continuity,
operational efficiency and user loyalty.
Oracle is wondering whether Big Data
is fact or fiction and whether it can
really be turned into a solid
competitive advantage. Right now, the
Big Data landscape is confusing and
over-hyped, with many enterprises
lacking in-depth understanding of how
to collect, store, query, analyze the
data and benefit from identified trends
and business insights.
New approaches and BI tools are
needed today in order to allow for
free-form exploration and end-to-end
functionality; most of Big Data tools
available now are geared towards the
notion that user knows in advance
what questions they want answered by
their Big Data and predictive analytics
initiatives. Data analysts have to deal
with different tools, which generally
disrupts their work and inhibits agility.
A new generation of data discovery and
predictive analytics tools has just
emerged enabling better data
visualization and interpretation as a
result of it. However, data discovery
tools are oftentimes misused or have
their flaws. With the abundance of data
enterprises have to deal with on a
regular basis, a risk is high that data
discovery tools “will consume unvetted,
inconsistent, or faulty information, or -
worse yet - use this (faulty) information
to generate analytic insights.”
Yet, before rushing to implement any of
available data discovery tools, business
decision makers should evaluate them
in terms of:
 ability to empower all decision
makers, not just a select few;
 data dashboards and their ability
to answer mission critical
questions
 data integrity and elimination of
flawed insights
 ease of adding advanced data
analytics, and more

PROCESS: FROM RAW DATA TO VALUABLE INSIGHTS
Most of Data Discovery tools available
today use a 5-step approach to turn
data into insights:
1. Pinpoint relevant data
Today, any business is able to capture
plenty of raw data such as user
contact information, data on social
engagement and loyalty programs
(e.g., tweets and re-shares), marketing
campaigns, etc. However, it’s not
always easy to determine which data is
timely and reliable enough.
Using traditional Big Data Discovery
tools, Data Analysts can take
advantage of their intuitive interfaces
and dashboards while navigating a rich
catalogue of all raw data available and
fleshing out the relevant one. So, it’s
pretty easy to find and pinpoint the
data in such a way.
Current data discovery tools may
hinder BI adoption as they’re only able
to satisfy a small percentage of the
user base (e.g. data scientists and
analysts) and are hard to comprehend
for the average business user not
skilled in deep analytics.
McKinsey Global Institute (MGI)
predicted back in 2011 that 140,000 to
190,000 data scientist positions will
have remained unfilled by 2018. Four
years later McKinsey projects that “by
2018, the U.S. alone may face a 50
percent to 60 percent gap between
supply and requisite demand of deep
analytics talent.” Starting salaries for
Big Data scientists have already gone
north of $200,000 per annum and this
figure will most likely increase as the
competition for Data Analytics talent
becomes tougher.
If your data error rate is
between 1% and 5% -
you’re already at risk of
having poor data
quality!

With this in mind, enterprises should seek innovative Data Discovery tools and
platforms with seamless UX and capability to allow for precise BI that can be easily
understood by the average user not skilled in in-depth data analytics.
At the moment, the risk of human error and wrong data interpretation remains high
when it comes to Big Data analytics. Enterprise Data Discovery tools should be able to
free professionals other than data analysts from analyzing information in Excel when
they have to make important data-based decisions; instead, surgeons and pilots
should be able to extract, read and understand information from equipment consoles
and dashboards and present data in a visually attractive and easy-to-understand way.
So, what kind of data visualization tools would add most value for different categories of
users?
 Software developers and IT specialists need innovative cross-platform apps and
systems to deliver dashboards and report trends quickly to end users such as
corporate leadership or product owners
 Non-techies need simple applications to explore and analyze data in a down-to-
earth way
 Corporate stakeholders such as clients and business partners need apps and
tools to analyze peer and personal data for new insights into system
performance, troubleshooting, etc.

Lorem Ipsum Dolor
[Street Address]
[City], [State][Postal Code]
[Web Address]
2. Explore data potential
Exploring Big Data is a very time and resource-consuming task. For instance, a facility
engineer seeking to streamline operations and make better use of connected
equipment may spend weeks trying to understand tons of information provided by
equipment. Speed of data interpretation matters a lot today; as such, enterprises
should consider tools able to accelerate time to understand data. Enterprises need
tools that are able to prioritize attributes for different types of users, and provide
analysts with an opportunity to experiment with different data combinations to
understand correlations and make timely conclusions as to whether the given data
set is worth of scrutiny. Such tools should also allow for better quality control and
help prevent businesses from wasted investments in projects with limited or unclear
potential.
While evaluating tools to deploy for Big Data analytics at an enterprise level,
organizations should pay attention to scalability as well. Pure-play data discovery tools
aren’t designed to handle large-scale enterprise deployments and don’t fit well with
wide-scale deployment, especially in cases when you have to go outside the firewall to
meet Big Data needs of external focus groups such as clients, suppliers and partners.
Chosen tools should be able to address many business issues and facilitate data
failover and load balancing, disaster recovery, clustering, monitoring and
administration of information.
Scalability also enables to process adequately the amount of rapidly growing data,
since tool performance degrades significantly as the number of users and mountains
of data grow.

3. Transform data to make it ready for analysis
In order to be suitable for enterprises, data discovery and visualization tools should
provide adequate data management and security. Typically, data in Hadoop should be
manipulated and prepared prior to being used in analytics. Analysts spend on average
up to 80% of their time on data preparation nowadays. New-gen Big Data visualization
tools provide a spreadsheet-like approach for Big Data transformation. They allow for
data enrichment to infer language and location or flesh out topics and sentiment
hidden in the raw data.
For the best results, data should be transformed into the actionable insights on two
interconnected levels: operational and strategic.
Operational BI fosters decision-making at the lower levels of an organization and
facilitates daily operations. It has a direct impact on company’s short and mid-term
goals such as greater profitability and increased sales. Since operational BI bridges
the gap between discovery of issues and opportunities and taking action on them,
enterprises need powerful integration technology to support data transformation in
this level. At the moment, traditional tools need add-ons or partnering strategies to
allow for data integration. It’s especially challenging for real-time data analytics, as
most of tools work with static pre-loaded data.
Strategic BI is associated with analytical data sources, data mart and data warehouse
and aims to improve an existing business process by analyzing a pre-determined set
of metrics and providing historical context of data. Strategic BI is used for forecasting,
goal setting and should be integrated with the overall corporate strategy.
At the enterprise level, tools aim to provide enterprise-class version control and
auditing. When it comes to Big Data analytics, most users modify data and fields with
no audit trail which results in messy data, wrong business decisions and overheads as
administration and error fixing costs rise up. To avoid this, organizations should
establish enterprise-wide data governance that includes, but isn’t limited to rules and
procedures related to dashboard creation, ownership, distribution and usage.

4. Discover insights and put them into Action Plan
Enterprise-class data discovery and analytics tools should provide organizations with
an opportunity to “squeeze” more from their analytics talent by automatically
blending data for deeper perspectives and demonstrating new patterns in rich data
visualizations. Tools should be able to mash up or merge different data sets, filter
data with keywords search and guided navigation and provide a consumer-like
experience.
One of the biggest challenges with visual analytics today is that organizations make
conclusions from their data dashboards without drilling down to the base data for
confirmation of conclusions. Aggregate-level correlations can be completely different
on the base level, therefore metadata matters this much!
Pattern discovery is not something new; statisticians have been working with it for
decades. The process of data pattern discovery consists in creating new subsets
from existing data sets by using filters, aggregators or data grouping. To create
entirely new data sets, these subsets can be merged with other sets and new
variables can be added via calculations and statistics. Most of BI platforms available
today provide sophisticated UIs to manage these operations and guide user through
data discovery process through Venn diagrams and other tools. Poor data quality
results in wrong business decisions and overheads.
To maintain high quality, enterprises should consider tools that are able to address
data profiling and transformation, metadata management and team-based
development of custom plugins and web and mobile systems to ensure these tools
provide profiling and standardization of enterprise information.
Effective data discovery allows for design and deployment of visual dashboards that
are interconnected and communicate in the right way to make sure depicted
information is accurate and truly valid!

5. Data sharing
When choosing data discovery
and visualization tools for your
enterprise, do consider those
that allow for Big Data analytics
democratization and sharing.
Big Data is actually a focal point
of enterprise-wide
collaboration and collective
discovery.
Entire teams (e.g. DevOps,
analytics) should be able to
share projects, bookmarks and
galleries in order to collaborate
and iterate. In Hadoop, for
instance, analysts can publish
their data enrichment and
transformation results to
secure work and maximize Big
Data value.

Next-gen Data Discovery tools that are popping out today have capabilities no
traditional tools could ever boast. Yet, choosing the right ones for enterprise Big Data
strategy can be a daunting challenge not every company can overcome successfully.
When choosing tools for your corporate Big Data solution, check the following
capabilities that are vital for successful data discovery and visualization in 2015 and
beyond:
 Data mashup and modeling: semantic autodiscovery, intelligent joins and
profiling, hierarchy generation, data lineage and data blending on varied data
sources, including multistructured data;
 Integration capabilities: ease of use and user friendly UI, install, query engine,
shared metadata, promotability across all platform components
 Platform administration: users management, security, scalability, ensuring high
availability, performance optimization and disaster recovery
 Metadata Management: the ability to leverage the same systems-of-record
semantic model and metadata; a robust and centralized way for administrators
to search, capture, store, reuse and publish metadata objects, such as
dimensions, hierarchies, measures, performance metrics/KPIs, and report layout
objects;
 Cloud deployment: Platform as a service (PaaS) and analytic application as a
service (AAaaS) for creating, deploying and managing predictive analytics and
analytics apps in the Cloud based on both Cloud-stored and in-house data;
 Development and Integration: a set of programmatic and visual tools and a
development workbench for building reports, dashboards, queries and analysis;
ability to scale and personalize data distribution, schedules and alerts, BI
workflow and analytics content to web and/or mobile; embeddability and
customization of BI components in a business process, app or web / mobile
portal.
EVALUATING DATA DISCOVERY TOOLS FOR THE
ENTERPRISE BIG DATA STRATEGY

 Interactive data exploration: the ability to explore data by manipulating charts
(e.g., use of different colors, sizes and shapes) and visualization of the data
sets being analyzed (e.g. heat and tree maps, geo-maps, scatter plots,
different types of charts, etc);
 Analytic dashboards: the ability to build interactive dashboards and content
with embedded advanced analytics to be consumed by different business
users;
 IT-authored dashboards and reporting: the ability to create well-formatted
and print-ready interactive reports with or without any parameters. Centrally
authored (aka IT-authored) dashboards depict graphically performance
measures and are able to publish linked multi-object reports with intuitive
and interactive displays such as gauges, sliders, maps, checkboxes, etc.
 Ad-hoc query: the ability for users to question data without relying on IT to
generate a report; re-usable semantic layer to allow for navigation of available
data sources, hierarchies, predefined metrics and so on; OLAP to allow for
data analysis with fast query and calculation performance and, as a result,
slicing and dicing of data;

Gartner’s Magic Quadrant of BI and data discovery tools as of February 2015

CONCLUSION
It’s needless to deny the fact that data discovery is crucial to any organization’s
ability to gain true value from enterprise data assets and create sufficient and
effective business intelligence.
Similar to data visualization, data discovery is an interactive function that is
supported by many various software solutions including advanced BI platforms
with innovative data exploration capabilities.
Today’s enterprises need to bridge the gap between business professionals on the
front line that are able to interpret data visualization and draw the right
conclusions and IT professionals supporting data discovery from the back end in
order to make the right business decisions and take advantage of raw data
mountains generated by business on a regular basis.
Traditional Big Data tools that have been out there for years are perfectly suited for
people skilled in both IT and data analytics, but are lagging largely behind when it
comes to usage by business users not skilled in IT and Big Data.
The next-gen data discovery tools are smarter and possess more capabilities than
the first-gen platforms; yet, each enterprise should find tools that would be geared
specifically towards their business use cases and would provide value for all
business users no matter what their key skillsets are.
Using aforementioned criteria, enterprises can find a BI / data discovery vendor
that would add merit to the tool’s capabilities while ensuring an ease of use,
specificity and integration of external sources to the Cloud.

References
Kotorov, R., “5 data discovery pitfalls,” EnterpriseAPPSToday, 2014
Orihuela, R., “Hel wanted: black belts in Data,” Bloomberg, 2015
“Big data: The next frontier for competition,” McKinsey, 2011
“Magic quadrant of BI and data discovery tools,” Gartner, 2015
Images: ShutterStock, Intersog

WHERE INTERSOG FITS
Intersog supports Data Discovery with the 5 A’s – Advisory, Apps, Accessibility,
Automation, and Actual Results
Advisory
Starting at the highest level, we are trying to accomplish a business goal or
address an issue. Our advisory services include defining and prioritizing goals,
establishing measurable success criteria and advising on how mobility and Big
Data can address the need at hand. In addition to assisting with a clear
roadmap for effective enterprise Data Discovery strategy, we advise on how to
lead, not follow innovation in the Big Data world.
Apps
Intersog has been building mobile apps since the advent of the first iPhone.
We have produced embedded firmware solutions for predictive analytics,
connected cars, provided remote control interfaces for electronics and
integrated with many types of machine sensors and communication engines.
Our breadth of experience in all of these areas makes us an ideal Big Data
partner.
Accessibility
Through the availability and accessibility of the mobile platform, Data
Discovery puts your world in the users hands. There could be much more to
your mobile data discovery interface than just a visualization dashboard, it can
also be educating, soliciting and promoting. We understand how an ease of
use adds merit to enterprise data exploration and business decisions. Our
Cloud computing and SaaS/PaaS capabilities enable data discovery and
analysis from anywhere on the planet!

Automation
The customization and automation potential has never been greater with a
combined Big Data, Cloud and mobility solution. Most of Data Discovery
and visualization tools need customization so that organizations only use /
expand features they need. Also, automation accelerates significantly Big
Data deployment within the enterprise. Most Data Discovery tools available
in the market today are complex and envision time and resource-intensive
manual work. Intersog can help automate multiple functions such as
software provisioning, installation, configuration and testing, which helps
improve operational efficiency and eliminate IT burden.
Actual Results
If Data Discovery solution doesn’t meet a market / user demand, then it’s
not worth doing. At Intersog, we emphasize ideation and innovation with
measurable success. Our goal is to be a valued partner that becomes
indispensible in achieving market dominance. You want to make a mark
and change the world for the better and we want to help you get there.
Contact Intersog today to schedule time to
learn more about how we can do that
together!

About Intersog
Intersog Inc. is an international provider of custom software development specializing in full-
service, end-to-end solutions for mobile apps, web, IoT, wearable tech, Big Data and Cloud.
Intersog helps SMEs and established brands deliver disruptive innovation to end users by
providing cost-effective Agile Development Teams and resources onshore (within the United
States) and offshore (in our Ukraine-based R&D Centers).
Founded in 2005 and headquartered in Chicago's iconic Willis Tower, we're a global company
of 150+ people in multiple locations.
Get in touch with Intersog!
233 S Wecker Dr. #9390
Chicago, Illinois 60606
United States
+1 773 305 0885
www.intersog.com
contact@intersog.com

Developing Enterprise Data Discovery Strategy

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (8)

More from Intersog

More from Intersog (20)

Recently uploaded

Recently uploaded (20)

Developing Enterprise Data Discovery Strategy