4 aa4 3925enw

September 2012, IDC Retail Insights #GRI237012
Big Data and Analytics in Retail:
Unlocking Hidden Opportunities
W H I T E P A P E R
Sponsored by: HP
Greg Girard
September 2012
I D C R E T A I L I N S I G H T S O P I N I O N
The new information economy — increasingly digital and intelligent
— defines the terms of reference for retail today, and within this
milieu, mastering Big Data has become a cornerstone for success.
Mastering the four dimensions of Big Data — social,
channel/customer, market, and supplier/product — for actionable and
monetized insight requires newfound technical, organizational,
process, and decision management frameworks to handle three
challenging Vs of Big Data and analytics — volume, velocity, and
variety — in order to deliver the other V — value.
Value from Big Data and analytics can come from three sources —
gaining insight to improve processes and resource allocation,
personalizing and localizing offers, and creating community as the nexus
for branding and customer engagement. Extracting value economically
from Big Data requires a new generation of technologies and
architectures — shared information ecosystems designed to enable the
high-velocity capture, discovery, analysis, and application of insights.
Several analytical underpinnings of value creation are clearly evident,
including context-aware and pattern-based analytics, quantitative natural
language processing and social network analysis, and the extension of
retail data models beyond conventional enterprise entities and ontologies.
Applying these insights to control and optimize operations, tactics, and
strategies may require new business processes, decision management
frameworks, and roles and organizational constructs. Opportunities to
apply Big Data analytical insight reach deep into digital or social
marketing or customer insight — the organizations where these efforts
often originate — but just as important and likely more important,
these opportunities extend into brand management and marketing,
merchandising, product management, localization, pricing, fulfillment,
and all channels of commerce.
What do the dimensions of "big," "fast," and "varied" mean with respect
to the opportunities to create value from Big Data and analytics in your
business? Accurately evaluating competitive and market scenarios that
impact your Big Data and analytics business case calculus requires the
strategic and tactical points of view of line-of-business leadership. Don't
GlobalHeadquarters:5SpeenStreetFramingham,MA01701USAP.508.935.4400F.508.988.7881www.idc-ri.com

#GRI237012 ©2012 IDC Retail Insights
let Big Data and analytics become the bright shiny new object of your
attention. Forgetting that value from Big Data and analytics doesn't
materialize just because you have invested in these capabilities will lead
to too many "dry wells" filled with misspent time, talent, and treasure.
S I T U A T I O N O V E R V I E W
T h e N e w I n f o r m a t i o n E n v i r o n m e n t o f R e t a i l
As the digital world continues to emerge, information has become the
singular environment within which retailers earn customer loyalty,
bring successful new products to market, collaborate through supply
chains with business partners, enable associates, reduce risk, ensure
compliance, and above all, burnish their brand. At least that's the
potential — if only retailers master Big Data.
As more Big Data and potential insight amass within and beyond the
enterprise, traditional approaches to acquiring and analyzing relevant data
and applying insight gained from interrogating the data no longer suffice.
Barriers to capitalizing on this potential include the technical capabilities
of conventional information technologies as well as the processes,
decision management frameworks, and organizational constructs by
which organizations govern data, develop information from the data, and
apply insight to operational, tactical, and strategic decisions.
By some estimates, up to 90% of the world's information lies hidden in
forms and formats beyond the competencies of the information
management technologies commonly used in retail. Retailers own
some of this hidden information within their four walls or a cloud —
some of it unstructured (e.g., in product descriptions, customer emails,
contact center notes, promotional agreements) and some of it
structured (Excel spreadsheets being the nearly ubiquitous example).
More and more information hidden in Big Data, however, lies beyond
a retailer's information management assets — residing in social data,
customer data, market data, and supplier data.
Big Data in the Social Context of Retail
The impact of Big Data — its promise amid peril — is easiest to see in
how it plays out in the inherently social context of retail. A growing
and diverse number of consumers now interconnect and influence
one another with their use of social networking and media —
YouTube, Facebook, Twitter, LinkedIn, Google+, blogs, wikis, and
ratings and reviews. The speed, scale, and viral aspects of social media
and networking influences, whether they are facts, opinions, or
misinformation, dwarf old modalities of "word of mouth" and "power
of the pen" pressures on shaping consumers' attitudes and opinions
about brands as well as consumers' buying patterns and propensities.

©2012 IDC Retail Insights #GRI237012 Page 3
Conventional ways by which retailers seek to understand consumer
sentiments and the sources of influence on those sentiments fall short
of the mark when their customers engage one another through social
media and networks. Traditional information management approaches
will no longer suffice to address these emerging demands and the
resulting complexity.
The disparity between the speed and strength of consumer sentiments
spreading across social networks and the limits of what retailers have
to track these sentiments puts retailers at risk of losing control of their
brand, the loyalty of their best customers, and market share, revenue,
and margins.
The imperative retailers face is deciding how they should ready
themselves to understand, react, and ideally engage their consumers in
this brave new world. The key lies in mastering new competencies in
Big Data and analytics to successfully and automatically integrate their
insight into the various retail business processes.
L a t e n t V a l u e : T h e F i r s t V i n B i g D a t a
Social data is one of four dimensions of Big Data in retail. Customer
data sources, market data sources, and supply data sources round out
the IDC Retail Insights typology of Big Data in retail (see Figure 1).
Insights hidden in the Big Data of each dimension can improve decisions
inherent in that dimension; for example, insight from loyalty and shopper
profile data can improve customer segmentation and promotional
targeting, and in another example, real-time stock (distribution center
[DC], in-transit, store) data can improve next-day or same-day
replenishment. Increasingly, however, new insight can be drawn by
mashing up data across domains. For example, contextual analysis of
contact center call logs, tweets, ratings and reviews, lot tracking of
shipment records, product design information, and loyalty data taken
together could spot product defects from customer feedback, identify
root causes from product design information, identify stores holding any
defective merchandise, and find loyal customers who purchased the item.
In more general terms, Big Data and analytics hold the potential to
dramatically increase three imperatives of omnichannel retail:
● Insight applied in commerce (online and in-store); marketing (brand
management, advertising, and promotions); fulfillment (from private
label management to customer orders); and merchandising (from
curating core assortments to localizing prices and promotions)
● Personalization applied in private promotions and offers, dynamic
configuration of online assortments, and digital content
presentation based on knowing the customer at any touch point and
anticipating his or her needs

● Community applied in physical or digital social venues to harness
the influence of social networks and media to create personal
engagement with the brand
F I G U R E 1
R e t a i l B i g D a t a T y p o l o g y
Source: IDC Retail Insights, 2012
V o l u m e , V e l o c i t y , V a r i e t y : T h e O t h e r V s o f
B i g D a t a O p p o r t u n i t y
Realizing the opportunities from analyzing Big Data in individual
domains and mashing up Big Data across domains, in other words,
harvesting its latent value, is a function of an organization's ability to
Enterprise Systems of Record
Market Data Sources
• Trade
• Syndicated
• Demographic
• Geospatial
• Events
• Venues
• Business
• Economic
• Weather
Social Data Sources
• Twitter
• Facebook
• Games
• Shutterfly
• Pinterest
• Google+
• YouTube
• LinkedIn
• Blogs
• Wikis
• Foursquare
• Shopkick
• Yelp
• Citysearch
Billions of Interactions
Millions of
Transactions CustomerDataSources
SupplyDataSources
• Market baskets
• Shopping carts
• Loyalty and profiles
• Offer/response
• Shopping list
• Click stream
• SMS
• Associates' black books
• Search terms and patterns
• Ratings and reviews
• Downloads
• Geospatial
• QR scans
• App usage
• Web chat
• Contact center
• Sensor
• Survey and focus group
• Email
• Purchase orders
• Shipments
• Returns
• RFID and sensors
• DC stock
• Store stock
• Receipts
• DC issues
• Trade promotions
• Product information
• Design specifications
• Market intelligence
• Compliance

master the inherent volume, variety, and velocity of the types of
Big Data shown in Figure 1:
● Data volumes growing from terabytes to petabytes, exabytes, or
more. IDC Retail Insights estimates a base of 9.6PB held in social
media today with 10.2PB being added in the year ahead. We also
estimate — conservatively — that 19.8PB of created content drives
14,000PB to 18,000PB of content consumed.
● Data varieties extending in formats from structured to semistructured
and unstructured and in sources from core transactional enterprise
systems — one's own or those of trading partners — through which
retailers operate and control their businesses; to collaborative and
communications systems (e.g., email, enterprise wikis, and contact
center logs); and to external informational systems — some structured
(e.g., demographic data) but the vast volume unstructured
(e.g., tweets or posts to a company's Facebook page).
● Data velocities and their business process and decision
management frameworks changing from those amenably serviced
by batch processes to those stubbornly dependent on the streaming
of data acquisition, analysis, and application of insight.
F U T U R E O U T L O O K
Market and financial imperatives present the strategic context of an
urgent and critical need to master the "supply" and "apply" sides of
Big Data and analytics in retail. Merchandising and marketing present
an excellent example of this need in the context of the ongoing shift of
power to the consumer. The forefront of insights in these two domains
has long since marched past the private knowledge domains of the
merchant prince who controlled product risk and is now marching past
the enterprise knowledge domain of spreadsheet merchants as they
control market risk to the Big Data knowledge domain of the social
merchant controlling customer risk. Today's retailer is dependent on
mastering all three knowledge domains.
S u p p l y S i d e a n d A p p l y S i d e o f B i g D a t a a n d
A n a l y t i c s
Mastery of Big Data's four Vs is a function of two capabilities —
supply and apply. Both dimensions require agility in the solutions that
supply Big Data and the processes that apply its insights. Full
realization of the value hidden in Big Data in the strategic context of
real-time omnichannel retail wherein the consumer holds the balance
of power requires new consumption models — from real-time
contextual decision management frameworks to mobile presentation of
visually rendered analytics in an enterprise context and mobile
personalized content in the context of instrumented, informed, and
interconnected customers.

Supply Side of Extracting Value from the Volume, Variety,
and Velocity of Big Data
Extracting value economically from Big Data requires a new
generation of technologies and architectures designed to enable the
high-velocity capture, discovery, analysis, and application of insights.
These IT assets encompass the hardware and the software that
integrate, organize, manage, analyze, and present data that is
characterized by the four Vs of Big Data opportunity. This set of
capabilities is the "supply side" of Big Data competence.
Apply Side of Extracting Value from the Volume, Variety,
and Velocity of Big Data
A second set of capabilities — of equal importance — addresses the
"apply side" of Big Data competence — the business processes,
decision management frameworks, and roles and organizational
constructs by which an organization consumes Big Data insight to
control and optimize its operations, tactics, and strategies to achieve its
business objectives.
Shared Information Ecosystems
The value of Big Data and analytics cannot be fully realized unless its
insights are applied in the retail enterprise ecosystem context of other
business intelligence systems, in particular, in their service of
marketing and positioning, merchandising and product development,
supply and fulfillment, and omnichannel customer engagement
through store and ecommerce channels. The contextualization of Big
Data insight in the ecosystem of enterprise systems of processes and
systems of record includes:
● Aligning analytics with business concerns (e.g., product quality,
brand position, customer relationships, product development,
marketing campaigns)
● Managing the life cycles of Big Data — defining, harvesting,
organizing, analyzing, storing, reporting, and disposing of the data
at the clock speed of the business processes in which it is
consumed
● Reporting and applying insights in process decision management
frameworks (e.g., roles; decision rights; operational, tactical, and
strategic time frames and spans of control; and plan, do, analyze
cycle times)

R e t a i l B i g D a t a a n d A n a l y t i c s M a t u r i t y
Figure 2 presents the five dimensions of Big Data and analytics in the
context of retail:
● Intent — context of strategic, tactical, and operational decision
management
● Process — relevancy of Big Data and analytics to roles,
responsibilities, and outcomes
● People — alignment of organization and cultural norms with the
use of Big Data and analytics
● Data — governance and life-cycle management of voluminous,
varied, and high-velocity data
● Technology — priority focus on hardware, software, and services
for economical supply of relevant Big Data and analytics
F I G U R E 2
R e t a i l B i g D a t a a n d A n a l y t i c s M a t u r i t y M o d e l
Big Data
Maturity
Intent
ProcessPeople
Data Technology
Clear view of business and market trends relating to the power of
Big Data analytics for insight, personalization, and community formation; the competitive
risks and rewards related to the use of Big Data analytics; and commitment to infusing the
omnichannel enterprise with the capabilities of Big Data analytics
IT priority focus on technologies and
architectures designed to extract value
economically from very large volumes of a
wide variety of data by enabling high-velocity
capture, discovery, analysis, and application of
insights to decision making
Governance and MDM of structured and
unstructured Big Data for consistent enterprise
and external data frameworks for analytical
support of business processes
Big Data insight made relevant and applied in
processes that span departments and functions
with clarity of roles, responsibilities, decision
management frameworks, and dependencies for
continuous plan, do, analyze cycles
Organizational alignment to collaborate
with Big Data analytics, define roles and
training for skills to apply the insights of
Big Data analytics, and align
compensation to create a Big Data
analytics retail culture

V a l u e C r e a t i o n O p p o r t u n i t i e s f r o m
B i g D a t a a n d A n a l y t i c s i n R e t a i l
Ultimately, in the business context of retail, value creation from Big
Data and analytics should result in the monetization of hidden insight
revealed and applied. Commercial paths to monetization can lead to
product and service innovation and with it differentiation, premium
pricing, and customer loyalty; process control — compliance, product
quality, security, and fraud prevention; and optimization of processes
and decisions — cost control, margin growth, and efficiencies.
Analytical Underpinnings of Value Creation
The range of new analytical underpinnings for monetizing Big Data
insight includes the following:
● Context-aware analytics: Integrating Big Data insight with
enterprise transactional data and business intelligence (e.g.,
predictive analytics, forecasting, and root cause analysis) such that
the situational awareness of Big Data analytics complements and
supports human judgment
● Broad-scope pattern-based analytics: Proactively seeking
episodic, recurrent, and systemic patterns or systems of factors
within process and organizational domains ("silos") and
supersystems of factors spanning silos from conventional and
unconventional sources that can positively or negatively impact
outcomes
● Quantitative natural language processing analytics: Discovery
and disambiguation of meaning primarily in digital
communications (e.g., emails, tweets, blogs, and Facebook posts)
and content repositories (e.g., product descriptions in product data
management system or customer ratings and reviews)
● Extension of retail data models beyond conventional enterprise
entities and ontologies (e.g., foundational notions of products,
customers, regions, stores, promotions, channels) to include
friends, followers, sentiments, and interests as extensions of
customer attributes; "likes," "wants," and reviews as extensions of
product attributes; and forums and topics at the nexus of people,
places, and products
Opportunities for Applying Big Data and Analytics in the
Line of Business
Opportunities to monetize Big Data and analytics span core retail
processes including brand management and marketing; merchandising;
fulfillment; and commerce. Examples and use cases of Big Data and
analytics for each of these processes are shown in Table 1.

T A B L E 1
O p p o r t u n i t i e s t o M o n e t i z e B i g D a t a a n d A n a l y t i c s i n R e t a i l
Process Process Responsibilities Sample Applications and Use Cases
Brand
management
and marketing
 Brand differentiation
 Support and execution of brand
management differentiation
strategy via development and
execution of media campaigns
 Customer analytics, insights,
and segmentation
 Creation of promotional
campaigns to shape customer
demand in collaboration with
marketing
merchandising
 Identification and characterization of threats to brand values
(e.g., boycotts or campaigns reacting to unmet corporate social
responsibility commitments)
 Intelligence about strengths and weaknesses of competitors by
customer segments, merchandise categories, product quality,
pricing, locations, and other categories of strategic importance
 Intelligence about customer sentiment trends by customer
segments, merchandise categories, product quality, pricing,
locations, and other categories of strategic importance
 Identification of networks, nodes, and personalities of viral
influence
 Assessment of the impact of marketing campaigns and events
on consumer awareness, sentiments, behaviors, and intentions
Merchandising  Selection, curation, localization,
allocation, distribution, and
pricing of merchandise in all
commerce channels
marketing
 Intelligence about new product development, design, and
introduction — aspects that delight customers, meet their
expectations, or are of low value
 Intelligence about local demand for products and services not
carried in local assortment
Fulfillment  Sourcing, developing,
designing, and delivering of
merchandise into commerce
channels in collaboration with
merchandising
 Fulfilling customer orders from
own network, suppliers, and
marketplace
 Early warning of product defects and shortfalls of performance
and attributes against customer expectations
 Intelligence about new product development, design, and
introduction — aspects that delight customers, meet their
expectations, or are of low value
Commerce  Presenting and selling
merchandise and services
across all channels
 Operating all channels of trade,
including stores, catalog, call
centers, and digital — mobile,
ecommerce, social, and third-
party channels
 Store- and market-specific customer concerns and delights
regarding store operations, associates' engagement practices
and product knowledge, customer service, crowds, wait times,
and so forth
 Real-time in-store, near-store customer tweets for customer
service, product information, and product location
 Customer concerns and delights about performance and
characteristics of ecommerce, mobile, social, and other digital
channels

C H A L L E N G E S
What do the dimensions of "big," "fast," and "varied" mean with
respect to the opportunities to create value from Big Data and analytics
in your business? The answers will define the terms of reference for
your Big Data and analytics value creation journey. Gaining clarity
and consensus on these definitions will require you to address
technical, cultural, and organizational challenges. Left unattended,
these challenges will delay and distort identification and definition of
gaps in the five dimensions of your Big Data and analytics maturity
profile — intent, process, people, data, and technology.
Ignoring the dictum that "better can be the enemy of good enough" can
derail your Big Data business case calculus. Accurately evaluating
competitive and market scenarios that impact your Big Data and
analytics business case calculus requires the strategic and tactical
points of view of line-of-business leadership. Against that input, the
ability to define investment tripwires — investment priorities,
dependencies, and time lines for scenarios most likely to come sooner
and create more risk — and using them to sequence and schedule
investments under likely scenarios requires the strategic and tactical
points of view of information technology leadership.
Becoming mesmerized by the allure of Big Data and analytics as a
bright shiny new object of your attention presents another challenge to
be managed. Opportunities to create value from Big Data and analytics
projects don't materialize just because you have invested in these
projects. Forgetting that will lead to too many "dry wells" filled with
lost time, talent, and treasure.
O V E R V I E W O F H P B I G D A T A A N D
A N A L Y T I C S O F F E R I N G S
HP offers a wide range of information management and analytics
(IM&A) capabilities drawn from its enterprise services, software
products, and cloud and security platforms to address Big Data
challenges and opportunities in retail.
HP IM&A services include information strategy and organization,
information management and architecture, business analytics and
information delivery, and social intelligence. HP provides these
consulting services around its own Big Data software assets,
Autonomy and Vertica, and third-party software assets, in particular
SAP HANA and Microsoft SharePoint/BI platform. HP supports this
with a variety of software and solution delivery models including on-
premise installation, hosted, software as a service (SaaS), cloud
computing, and multitenant SaaS (cloud deployment).

HP focuses its capabilities to enable retailers to proactively manage
information-related business risk, enhance customer experiences, and
optimize business performance to create competitive advantage.
HP's Autonomy software asset helps retailers develop connected
intelligence from structured and unstructured data for actionable
decisions that improve business performance.
HP's Vertica analytics database delivers scalable performance on Big
Data queries enabling real-time decision making to be embedded in
retailer processes in order to optimize business performance.
Vertica and Autonomy deliver a powerful combination for real-time
analytics and decision making using structured and unstructured data
across the enterprise.
S t r e n g t h s a n d C h a l l e n g e s
HP offers a product and services portfolio that is consistent with what
IDC Retail Insights expects from a market-leading technology
provider to the retail industry. In addition to the software and services
capabilities noted previously, HP provides the following infrastructure
component to enable Big Data application deployments:
● Cloud computing. HP can offer clients a variety of enterprise-
grade cloud computing solutions, including public, hybrid, and
private. The ability to offer private cloud to retailers is considered
a strength for a number of reasons, not the least of which is a
tighter security model.
● Security. HP offers clients a security strategy through the HP
Security Framework, designed to offer end-to-end information
security plans and execution road maps. Because of the sensitive
nature of customer data and the requirements and penalties imposed
by the regulators, security is top of mind for industry IT executives.
● Mobility. HP's approach to enabling enterprise mobility is suited
for organizations that wish to reach their constituents across
multiple networks and devices by delivering applications, content,
and services in a scalable, secure, and reliable way. This approach
leverages HP's global applications services capabilities to provide
the architecture, systems engineering, development, and support
services. Combined, they help an organization simplify its
applications and extend them where necessary as well as build
mobile business-to-business, business-to-consumer, and business-
to-employee applications. This approach also leverages HP's
service-oriented architecture–based integration architecture and is
enabled by development and security frameworks that help create
componentized building blocks from monolithic legacy
applications to develop and deploy mobile applications.

HP faces several unique market challenges as well as many of the
same market challenges as other enterprise vendors servicing the Big
Data marketplace:
● Demonstrating leadership in the transition from information
management to Big Data. HP's legacy information management
solutions, including TRIM, Data Protect, and others, are not
broadly implemented across the retail marketplace. HP must more
effectively demonstrate leading capabilities at the component level
with high-profile Autonomy and Vertica wins as well as highlight
its ability to deliver end-to-end Big Data software, services, and
hardware with lighthouse and marquee clients.
● Offering competitive, differentiated solutions portfolios. The
retail market, as a traditional early adopter of emerging technologies,
senses the value of the Big Data trend and is budgeting accordingly.
HP's competitors are also focused on expanding their solution
portfolios on the Big Data trend in terms of breadth and depth of
product capabilities, IT infrastructure, and professional services. HP
must leverage its industry position and demonstrate the value of a
Big Data relationship with HP to gain Big Data market share.
● Channel network management. HP's channel strength is also a
weakness. Successful execution of HP's Big Data strategy to
broaden its portfolio will require that HP reinforce its position in
the enterprise space without alienating the channel that has been so
beneficial to the organization.
E S S E N T I A L G U I D A N C E
● Adopt an enterprisewide approach to Big Data analytics. The
design, development, and deployment of Big Data analytical
capabilities should be seen from the outset as an enterprisewide
undertaking even as nascent initiatives of the sort incubate within
marketing, in particular digital marketing, where Big Data
analytics are applied to social Big Data, or in ecommerce and
personalization where Big Data analytics are applied to customer
Big Data. The design objectives of an enterprisewide approach to
Big Data analytics should include:
○ Program governance led by a leadership team drawn from
across four core omnichannel processes: marketing, commerce,
merchandising, and fulfillment
○ Alignment of insights from the four quadrants of Big Data
sources — social, customer, market, and supply — with the
four dimensions of retail intelligence — customer intelligence,
offer intelligence, channel intelligence, and merchandise
intelligence

○ Delivering insights within the decision management context of
the roles they inform — that is, task aligned and just in time —
and the style in which and the speed at which the decisions are
made
● Look for monetization opportunities across the board. Big Data
analytics can inform decisions beyond marketing in
merchandising, commerce, and fulfillment. Actionable insights
include marketing campaigns; product design and quality; new
product development and introduction; assortment localization;
pricing, promotions, and personalization; crisis management,
selling techniques, and store associate management; store
operations and design; and competitive intelligence and
differentiation.
● Maintain a readiness for rapid evolution of Big Data analytics. Big
Data analytics is a new area; new capabilities for and opportunities to
apply Big Data analytics are emerging quickly. Managing rapid
change on the supply and apply sides of Big Data analytics requires
agility in a host of technical competencies (e.g., data management and
governance, commodity hardware configurations); organizational
dimensions of skills, roles, and decision rights; consumption models
and use cases; and more. Retailers will increasingly need to find
sources of innovation and should look to Big Data information
partners. Retailers should screen for partners that provide best-in-class
solutions consisting of open source technologies/capabilities and
proprietary technologies/capabilities, as needed, that are best suited for
their unique requirements.
C o p y r i g h t N o t i c e
Copyright 2012 IDC Retail Insights. Reproduction without written
permission is completely forbidden. External Publication of IDC
Retail Insights Information and Data: Any IDC Retail Insights
information that is to be used in advertising, press releases, or
promotional materials requires prior written approval from the
appropriate IDC Retail Insights Vice President. A draft of the
proposed document should accompany any such request. IDC Retail
Insights reserves the right to deny approval of external usage for any
reason.
This document was reprinted by HP with permission from IDC Retail
Insights.

4 aa4 3925enw

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Similar to 4 aa4 3925enw

Similar to 4 aa4 3925enw (20)

More from TUSHAR GARG

More from TUSHAR GARG (8)

4 aa4 3925enw