Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BIG DATA: LEVERAGING COMPETITIVE INTELLIGENCE IN RETAIL - Mandar Mutalikdesai, Head of Data Semantics, DataWeave


Published on

“Big Data: Leveraging Competitive Intelligence In Retail" focused on the next wave – enabling real time decisions and real-time responses through big data. Here are the Lounge47 key takeaways: 1. Large enterprises have this far used big data to focus on process improvement and variety of data (Process improvement 47%, Variety of data 26%, Volume of data 16%, Cost Saving & Efficiency 8%, Velocity of Data 3%) 2. Big data is not a new problem; at any point of time, our ability to produce data has always been greater than the sophistication of the tools available to process and make it usable 3. Companies like Uber and Amazon, with products like “Surge Pricing” or “Dynamic Pricing” are ushering in the paradigm of “fast data” to make instant decisions and gain a competitive advantage 4. “Fast Data” unlike “historical data, is live, interactive, automatically generated, and often self-correcting” – the volume and nature will be further accelerated through the Internet of Things (IoT) 5. In the retail vertical – data enablers that push micro decisions in real time and serve to answer – what inventory to hold? or what products to promote? - pose a powerful value proposition 6. A plethora of data products, web-based, Apps, API’s, reports could be built to help enterprises take decisions E.g. a “Color” report that tells a fashion retailer that their inventory should carry more items in blue 7. Data products could serve - ecommerce companies, sellers, brands – each stakeholder, with very specific requirements and specific problems to solve E.g. brands value reports on product discounts offered to flag policy violation 8. Solving the big data challenge would involve the following generic steps – data extraction and aggregation, cleaning, normalizing, standardizing, sorting, storing. Analytics. Visual data presentation, via dashboard interfaces, reports etc. 9. Big data sounds like a simple problem to solve however the challenges are many a) Data acquisition: crawling public websites could be limited if volume and speed of query impact service to users, thus slowing the data collection b) Data cleaning & standardization: raw data could be messy or have gaps c) Storage and retrieval d) Data Accuracy: Careful management of massive machination with minimal human audits to keep the margin of error suppressed 10. Some Big data products: Price comparison by the hour and across competition, color report on product inventory, Market & Business intelligence products, discount tracking of basket of products 11) Finding a “give-back” to encourage E-Commerce companies to part with private data would allow big data companies to build an ecosystem that is mutually beneficial to all stakeholders.

While big data is an often used buzz word, and challenges like “new technology deployment” and the “collection, analysis and measurement of data” are being solved, the full power of this paradigm will be realized when organi

Published in: Business
  • Be the first to comment

  • Be the first to like this

BIG DATA: LEVERAGING COMPETITIVE INTELLIGENCE IN RETAIL - Mandar Mutalikdesai, Head of Data Semantics, DataWeave

  1. 1. Mandar R. Mutalikdesai Head of Data Semantics Data is the new Fabric COMPETITIVE INTELLIGENCE IN RETAIL
  2. 2. BIG DATA AND DATA PRODUCTS "Big data": publicly available as well as within intranets Huge opportunities, because data = insight Need of the hour: data democratization---intra-, inter-, and trans-firewall Many challenges: aggregating data, cleaning data, normalizing data, learning patterns and insights, presentation This is where data products come in!
  3. 3. BUILDING DATA PRODUCTS • Aggregate large amounts of data publicly available on the web, and serve it in readily usable forms • Serve actionable data through APIs,Visualizations, and Dashboards • Provide reporting and analytics layer on top of datasets and APIs Build data products that provide timely actionable insights for businesses and consumers by aggregating and analyzing public data on the web
  4. 4. SOME CHALLENGES Data acquisition: crawling and extraction infrastructure Data is uncertain and incomplete Data is human understandable, not machine understandable Data storage and retrieval Analytics: more data (often) beats better algorithms Product UX
  5. 5. Pricing Date Open Government Data Social Media Data Attributes Attributes Big Data Platform Unstructured, spread across sources and temporally changing JSON, PDF, XML, DOC, XLS, HTML and much more Data APIs Dashboards Data Services API Feeds Visualizations and Widgets DATAWEAVE PLATFORM
  6. 6. AGGREGATION AND EXTRACTION Extraction Layer: Offline Extraction of Factual Data Aggregation Layer: Distributed Crawler Infrastructure Public Data on the Web
  7. 7. NORMALIZATION Nomalization Layer Extraction Layer Offline extraction of factual data Remove Noise Fill Gaps in Data Represent Data Clustering Machine Learning Techniques Knowledge Base
  8. 8. DATA STORAGE AND SERVING Serving Layer Indexes Views Filters Pre-Computed Results Highly Responsive Serving Layer Distributed Data Storage Crawl Snapshots Processed Data Clustered Data DashboardsVisualizationsData APIs Reports
  9. 9. RETAIL ANALYTICS eCommerce Big Data made easy! Data science made easy for every retail/brand manager Make better decisions on pricing, competitive strategy, product coverage with full visibility.
  10. 10. PRICEWEAVE Real-time Retail Insights Platform across Geographies Pricing and Assortment Analytics Product and Catalog Intelligence Price Monitoring, Availability & Historical Analysis Gap Analysis & Coverage Along Multiple Dimensions Product Similarity Based on Multiple Attributes Catalog Spread & Depth in Key Product Categories
  11. 11. PRICING INTELLIGENCE Compare and monitor product prices, availability, offers across competitors in real-time • Improve margins by responding to pricing opportunities proactively • Increase sales through competitive offers, promos etc. (e.g. free shipping) • Drive price parity across distribution channels. Detect pricing violations. • Develop long term strategies with pricing trends & insights from the past
  12. 12. ASSORTMENT INTELLIGENCE Benchmark and analyze your SKU coverage to identify gaps and strengths in your catalog • Increase coverage by focusing on a wider set of customers • Identify & promote products, brands, categories where one has a unique advantage • Discover product or category gaps in catalog • Reduce customer acquisition costs
  13. 13. PRODUCT SIMILARITY MAP Compare products based on features, tech specs, pricing and brand attributes. Identify similar and substitute products • Fulfil customer demand for a given product with a compelling offer on a substitute to gain share • Cover out-of-stock products with similar ones • Compare competitor coverage, regardless of SKU similarity
  14. 14. COLORVARIANT ANALYSIS Decipher trends in colors, patterns, and variants for apparel, footwear and lifestyle products. Use colors, fabric, and style for assortment analysis. • Spot current and predict future trends in colors and patterns • Ensure most popular colors and sizes are stocked in line with demand • Gain insights from social channels about brands, clothing trends
  15. 15. PROMOTION ANALYSIS Promote the right products. Enhance stock availability of popular products. Understand competitor promotion strategy. • Gain understanding on products that are being featured by competitors • Discounts, coupons and price points at which the promoted products 
 are being offered • Get insights into competitors’ product rankings and promotions on a 
 ongoing basis • Ensure the products customers are interested are always in stock and 
 attractively priced
  16. 16. ALERTS AND NOTIFICATIONS Set alerts on any attributes that need tracking, in real time. Get informed on new product introductions, hot deals, and price slashes, as they happen. • Get insights in to competitor’s strategy • Discover pricing violations & unauthorized discounts on your brand • Keep tabs on the market trends, in real time
  17. 17. CUSTOM REPORTS Generate custom reports pivoted around: SKU, sub- category, category, brand, price, geography and other attributes • Pick your priorities to pivot the reports & gain actionable insights • Understand key trends relative to products, brands, categories and geography • Point-in-time snapshots to understand market offers for product/ category • Compare snapshots for trends / gaps
  18. 18. HOW IT WORKS Data Acquisition Deep Web crawling • Daily update of 12 million product prices • 50 million data points monitored • 3TB of data processed on a daily basis Data Extraction and Normalization • Extract relevant data points • Clean data stored • Normalized records based on knowledge bases • Noise cancellation Machine Learning and Information Retrieval • Classify products into product types • Clustering same products (& similar products) • Multi parametric clustering Data Representation • Dashboards • Visualizations • APIs • Alerts and Notifications • Reports • Integration
  19. 19. THANKYOU