Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Mandar R. Mutalikdesai
Head of Data Semantics
mandar@dataweave.in
Data is the new Fabric
COMPETITIVE INTELLIGENCE IN RETAIL
BIG DATA AND DATA PRODUCTS
"Big data": publicly available as well as within intranets
Huge opportunities, because data = i...
BUILDING DATA PRODUCTS
• Aggregate large amounts of data publicly available on the web, and serve
it in readily usable for...
SOME CHALLENGES
Data acquisition: crawling and extraction infrastructure
Data is uncertain and incomplete
Data is human un...
Pricing Date
Open Government Data
Social Media Data
Attributes
Attributes
Big Data
Platform
Unstructured, spread
across so...
AGGREGATION AND EXTRACTION
Extraction Layer: Offline Extraction of Factual Data
Aggregation Layer: Distributed Crawler Infra...
NORMALIZATION
Nomalization Layer
Extraction Layer

Offline extraction of factual data
Remove Noise Fill Gaps in Data
Represe...
DATA STORAGE AND SERVING
Serving Layer
Indexes Views
Filters Pre-Computed Results
Highly
Responsive
Serving Layer
Distribu...
RETAIL ANALYTICS
eCommerce Big Data made easy!
Data science made easy for every retail/brand manager
Make better decisions...
PRICEWEAVE
Real-time Retail Insights Platform across Geographies
Pricing and Assortment Analytics Product and Catalog Inte...
PRICING INTELLIGENCE
Compare and monitor product prices, availability,
offers across competitors in real-time
• Improve ma...
ASSORTMENT INTELLIGENCE
Benchmark and analyze your SKU coverage to
identify gaps and strengths in your catalog
• Increase ...
PRODUCT SIMILARITY MAP
Compare products based on features, tech specs, pricing and
brand attributes. Identify similar and ...
COLORVARIANT ANALYSIS
Decipher trends in colors, patterns, and variants for
apparel, footwear and lifestyle products. Use ...
PROMOTION ANALYSIS
Promote the right products. Enhance stock availability of
popular products. Understand competitor promo...
ALERTS AND NOTIFICATIONS
Set alerts on any attributes that need tracking, in real time. Get
informed on new product introd...
CUSTOM REPORTS
Generate custom reports pivoted around: SKU, sub-
category, category, brand, price, geography and
other att...
HOW IT WORKS
Data Acquisition
Deep Web crawling
• Daily update of 12
million product prices
• 50 million data points
monit...
THANKYOU
Upcoming SlideShare
Loading in …5
×

BIG DATA: LEVERAGING COMPETITIVE INTELLIGENCE IN RETAIL - Mandar Mutalikdesai, Head of Data Semantics, DataWeave

678 views

Published on

“Big Data: Leveraging Competitive Intelligence In Retail" focused on the next wave – enabling real time decisions and real-time responses through big data. Here are the Lounge47 key takeaways: 1. Large enterprises have this far used big data to focus on process improvement and variety of data (Process improvement 47%, Variety of data 26%, Volume of data 16%, Cost Saving & Efficiency 8%, Velocity of Data 3%) 2. Big data is not a new problem; at any point of time, our ability to produce data has always been greater than the sophistication of the tools available to process and make it usable 3. Companies like Uber and Amazon, with products like “Surge Pricing” or “Dynamic Pricing” are ushering in the paradigm of “fast data” to make instant decisions and gain a competitive advantage 4. “Fast Data” unlike “historical data, is live, interactive, automatically generated, and often self-correcting” – the volume and nature will be further accelerated through the Internet of Things (IoT) 5. In the retail vertical – data enablers that push micro decisions in real time and serve to answer – what inventory to hold? or what products to promote? - pose a powerful value proposition 6. A plethora of data products, web-based, Apps, API’s, reports could be built to help enterprises take decisions E.g. a “Color” report that tells a fashion retailer that their inventory should carry more items in blue 7. Data products could serve - ecommerce companies, sellers, brands – each stakeholder, with very specific requirements and specific problems to solve E.g. brands value reports on product discounts offered to flag policy violation 8. Solving the big data challenge would involve the following generic steps – data extraction and aggregation, cleaning, normalizing, standardizing, sorting, storing. Analytics. Visual data presentation, via dashboard interfaces, reports etc. 9. Big data sounds like a simple problem to solve however the challenges are many a) Data acquisition: crawling public websites could be limited if volume and speed of query impact service to users, thus slowing the data collection b) Data cleaning & standardization: raw data could be messy or have gaps c) Storage and retrieval d) Data Accuracy: Careful management of massive machination with minimal human audits to keep the margin of error suppressed 10. Some Big data products: Price comparison by the hour and across competition, color report on product inventory, Market & Business intelligence products, discount tracking of basket of products 11) Finding a “give-back” to encourage E-Commerce companies to part with private data would allow big data companies to build an ecosystem that is mutually beneficial to all stakeholders.

While big data is an often used buzz word, and challenges like “new technology deployment” and the “collection, analysis and measurement of data” are being solved, the full power of this paradigm will be realized when organi

Published in: Business
  • Be the first to comment

  • Be the first to like this

BIG DATA: LEVERAGING COMPETITIVE INTELLIGENCE IN RETAIL - Mandar Mutalikdesai, Head of Data Semantics, DataWeave

  1. 1. Mandar R. Mutalikdesai Head of Data Semantics mandar@dataweave.in Data is the new Fabric COMPETITIVE INTELLIGENCE IN RETAIL
  2. 2. BIG DATA AND DATA PRODUCTS "Big data": publicly available as well as within intranets Huge opportunities, because data = insight Need of the hour: data democratization---intra-, inter-, and trans-firewall Many challenges: aggregating data, cleaning data, normalizing data, learning patterns and insights, presentation This is where data products come in!
  3. 3. BUILDING DATA PRODUCTS • Aggregate large amounts of data publicly available on the web, and serve it in readily usable forms • Serve actionable data through APIs,Visualizations, and Dashboards • Provide reporting and analytics layer on top of datasets and APIs Build data products that provide timely actionable insights for businesses and consumers by aggregating and analyzing public data on the web
  4. 4. SOME CHALLENGES Data acquisition: crawling and extraction infrastructure Data is uncertain and incomplete Data is human understandable, not machine understandable Data storage and retrieval Analytics: more data (often) beats better algorithms Product UX
  5. 5. Pricing Date Open Government Data Social Media Data Attributes Attributes Big Data Platform Unstructured, spread across sources and temporally changing JSON, PDF, XML, DOC, XLS, HTML and much more Data APIs Dashboards Data Services API Feeds Visualizations and Widgets DATAWEAVE PLATFORM
  6. 6. AGGREGATION AND EXTRACTION Extraction Layer: Offline Extraction of Factual Data Aggregation Layer: Distributed Crawler Infrastructure Public Data on the Web
  7. 7. NORMALIZATION Nomalization Layer Extraction Layer Offline extraction of factual data Remove Noise Fill Gaps in Data Represent Data Clustering Machine Learning Techniques Knowledge Base
  8. 8. DATA STORAGE AND SERVING Serving Layer Indexes Views Filters Pre-Computed Results Highly Responsive Serving Layer Distributed Data Storage Crawl Snapshots Processed Data Clustered Data DashboardsVisualizationsData APIs Reports
  9. 9. RETAIL ANALYTICS eCommerce Big Data made easy! Data science made easy for every retail/brand manager Make better decisions on pricing, competitive strategy, product coverage with full visibility.
  10. 10. PRICEWEAVE Real-time Retail Insights Platform across Geographies Pricing and Assortment Analytics Product and Catalog Intelligence Price Monitoring, Availability & Historical Analysis Gap Analysis & Coverage Along Multiple Dimensions Product Similarity Based on Multiple Attributes Catalog Spread & Depth in Key Product Categories
  11. 11. PRICING INTELLIGENCE Compare and monitor product prices, availability, offers across competitors in real-time • Improve margins by responding to pricing opportunities proactively • Increase sales through competitive offers, promos etc. (e.g. free shipping) • Drive price parity across distribution channels. Detect pricing violations. • Develop long term strategies with pricing trends & insights from the past
  12. 12. ASSORTMENT INTELLIGENCE Benchmark and analyze your SKU coverage to identify gaps and strengths in your catalog • Increase coverage by focusing on a wider set of customers • Identify & promote products, brands, categories where one has a unique advantage • Discover product or category gaps in catalog • Reduce customer acquisition costs
  13. 13. PRODUCT SIMILARITY MAP Compare products based on features, tech specs, pricing and brand attributes. Identify similar and substitute products • Fulfil customer demand for a given product with a compelling offer on a substitute to gain share • Cover out-of-stock products with similar ones • Compare competitor coverage, regardless of SKU similarity
  14. 14. COLORVARIANT ANALYSIS Decipher trends in colors, patterns, and variants for apparel, footwear and lifestyle products. Use colors, fabric, and style for assortment analysis. • Spot current and predict future trends in colors and patterns • Ensure most popular colors and sizes are stocked in line with demand • Gain insights from social channels about brands, clothing trends
  15. 15. PROMOTION ANALYSIS Promote the right products. Enhance stock availability of popular products. Understand competitor promotion strategy. • Gain understanding on products that are being featured by competitors • Discounts, coupons and price points at which the promoted products 
 are being offered • Get insights into competitors’ product rankings and promotions on a 
 ongoing basis • Ensure the products customers are interested are always in stock and 
 attractively priced
  16. 16. ALERTS AND NOTIFICATIONS Set alerts on any attributes that need tracking, in real time. Get informed on new product introductions, hot deals, and price slashes, as they happen. • Get insights in to competitor’s strategy • Discover pricing violations & unauthorized discounts on your brand • Keep tabs on the market trends, in real time
  17. 17. CUSTOM REPORTS Generate custom reports pivoted around: SKU, sub- category, category, brand, price, geography and other attributes • Pick your priorities to pivot the reports & gain actionable insights • Understand key trends relative to products, brands, categories and geography • Point-in-time snapshots to understand market offers for product/ category • Compare snapshots for trends / gaps
  18. 18. HOW IT WORKS Data Acquisition Deep Web crawling • Daily update of 12 million product prices • 50 million data points monitored • 3TB of data processed on a daily basis Data Extraction and Normalization • Extract relevant data points • Clean data stored • Normalized records based on knowledge bases • Noise cancellation Machine Learning and Information Retrieval • Classify products into product types • Clustering same products (& similar products) • Multi parametric clustering Data Representation • Dashboards • Visualizations • APIs • Alerts and Notifications • Reports • Integration
  19. 19. THANKYOU

×