Download to read offline
Uploaded byConnotate
Using Web Data to Fuel Dynamic Pricing
Connotate provides automated web data extraction to help companies gather pricing intelligence from online sources. Their solution allows companies to monitor competitor websites daily to obtain timely pricing data at national and local levels. This enables businesses to optimize their own pricing strategies and maximize profits. Connotate's workflow transforms raw web data into clean, consumable formats that can feed into business intelligence applications. They also highlight new items and price changes to deliver just the changes from the previous data set. Case studies showed how this approach helped a retailer and manufacturer make better pricing and product decisions.
More Related Content
Similar to Using Web Data to Fuel Dynamic Pricing
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Using Web Data to Fuel Dynamic Pricing
- 1. Using Web Datato Fuel Dynamic Pricing Presenter: Barry Graubart, Vice President Product Strategy, Connotate Moderator: Gina Cerami, Vice President of Marketing, Connotate Date: October 2, 2013
- 2. Presenters 2 Barry Graubart Vice Presidentof Product Strategy Gina Cerami Vice President of Marketing
- 3. About Connotate • Headquarteredin New Brunswick, NJ, with offices in Irvine, CA and London, UK • Developed by world-class scientists at Rutgers University and the University of Southern California • Portfolio of patents for applying machine intelligence to Web data extraction and monitoring • Strong financial backing: .406 Ventures, Castile, Prism
- 4.
- 5. 5 Other Industries: Transform WebData into High-Value Assets • Online ad stats reporting • Competitive intelligence • News aggregation • Background check • Price optimization • Investment research • Market research • Regulatory updates • Sales intelligence • Business risk assessment • Data directories • Aggregate construction bids • Supply chain monitoring • Brand monitoring • Voice of the Customer • Social media monitoring Some of Our Many Use Cases:
- 6. Where Connotate “Fits” •Connotate is an expert in automating Web data monitoring and extraction • Our solution is the “automation engine” for some of the world’s leading innovators in related industries • News and publishing – the Associated Press, Thomson Reuters, Edmunds • Finance – Nasdaq, Lipper, Dow Jones, Alacra • Retail – Shopzilla, O’Reilly • Connotate is enabling technology for gathering pricing intelligence from the Web • We are not a pricing platform or a consultancy 6
- 7. Topics for Today’s Discussion •The challenge: • Extract pricing intelligence accurately and at scale • As often as YOU need to • Our value proposition • How Connotate does this 7
- 8.
- 9. The Pricing Intelligence Challenge •Customers use the Web and smart phones to get up-to-the-minute pricing intelligence • Empowered customers dictate the prices they are willing to pay • They see the prices of your competitors/manufacturers/distributor s • The Web has pushed customers to think about price first • The data volume is enormous and constantly changing • You need to know when competitors raise or lower their prices • You may also need local pricing – this creates a scale problem 9
- 10. What Can YouDo? • Adopt your customers’ same “pricing intelligence” tactics – extract prices from the Web, in near real time • Perform product-by-product comparisons • Compare prices in specific geographies • Make better decisions about which prices need to move up or down • Use automated Web data extraction technology to gain control over the pricing intelligence-collection function • Collect the price data you need, when 10
- 11. Polling Question: HowDo You Obtain Pricing Intelligence Today? How are you currently obtaining pricing intelligence? We contract with outside agencies for pricing strategy/intelligence We use scripts or scrapers to automate collection in-house We are manually collecting pricing data from the Web We are not collecting pricing from the Web but we plan to in the future
- 12.
- 13.
- 14. What Connotate Doesfor Pricing Intelligence Connotate automates pricing intelligence gathering: • Highly scalable data retrieval system • Grab “just the changes” or full extract • Filter by zip code, product category, SKU, etc. • Delivered in XML, CSV or any format More timely, scalable and customizable than agency services Informed decision-making to maximize profits! 14 Optimize prices – grow profits
- 15. Polling Question: For whatpurpose do you use pricing intelligence? Benchmarking Price matching Distribution chain intelligence Other
- 16. Workflow Options 16 Option 1:Traditional pricing workflow Pay 3rd party to collect/analyze data You wait for reports Option 2: Connotate workflow – you specify what you want to collect and how often you want data. Access Web page Transform Data Feed BI Apps
- 17.
- 18.
- 19. Step 1: DataInputs and Configuration 19 Extract: • Product • Item # • Availability • Price • Category Ignore: • Ads, etc.
- 20. Step 2. WebData Transformed 20 Clean, clear, consumable data
- 21. The Power ofAutomated Change Detection 21 Connotate can detect new items and price changes and deliver “just the changes” or highlight the changes within the data set.
- 22. Case Study #1:National Hardlines Retailer • Challenge/Opportunity • Obtain more timely visibility into competitors’ pricing – at the national and local levels – to support dynamic pricing • Reduce dependency on expensive pricing reports (updated weekly) • Solution • Monitor competitors’ websites daily to obtain timely pricing intelligence at both the national and local levels • Business Benefit • Increased market share, moving up in national rankings – optimizing pricing by making decisions based on timely data 22
- 23. Case Study #2:Appliance Manufacturer (Supplier to Big Box Retailer) • Challenge/Opportunity • Obtain a “360 view” of products through the entire distribution chain to optimize product positioning, pricing and branding strategy • Solution • Use automation to extract data from competitors websites daily to gain visibility • Business Benefit • Retaining channels, ensuring repeat orders with well-informed pricing and product enhancement strategies based on continual access to pricing and product reviews at the retail level 23
- 24. Appliance Manufacturer: Extract Dataand Reviews from Web Pages 24 Extract: • Product ID • Specs • Price • Ratings • Comments Ignore: • Ads, etc.
- 25. Appliance Manufacturer: Web DataTransformed 25 • Product • Product ID • Price • Specs • Product • Product ID • Rating • Comments
- 26. Polling Question: The Valueof Automated Web Data Collection Would an automated Web data extraction process help enhance your pricing intelligence function? Yes – we are automating this now but we could improve our processes Yes – we are planning to automate the collection of pricing data soon No – our plans to automate price data collection are not immediate
- 27.
- 28. Scoping Your Project: Stepsto Success 1.Decide which Web sources will provide the best pricing data to support your decision- making 2.Select the product categories and/or specific products you would like to price-match 3.Determine the frequency of collecting price changes and how often you’ll need a full data refresh 4.Carefully consider the types of data formats you’ll need 28
- 29. Q & A Connotatewill email a link to this presentation as well as a copy of the slides to you within 2 business days. If you have an immediate need, please email us at info@connotate.com or call (+1) 732-296-8844. We invite you to attend our upcoming webinars on competitive intelligence: “Aggregating Product Reviews to Optimize Product Positioning” Wednesday October 9, 2013 – Noon ET “Capturing News and Competitive Intelligence from Niche Sites” Wednesday October 23, 2013 – Noon ET 29
Editor's Notes
- #2 <Gina>Welcome to today’s presentation, “How to Use Web Data to Fuel Dynamic Pricing”. My name is Gina Cerami, and I’m the Vice President of Marketing here at Connotate. I’ll be your moderator today.This presentation will explore how companies are tapping into pricing intelligence on the Web so you can:Make better pricing decisionsAdjust product positioning by geography and product category on demandUphold the pricing power of your brands, and moreWe’ll also explain how you can automate the process with precise Web data extraction technology. The formal presentation will last approximately 20 minutes, followed by a live question and answer session. You may submit questions anytime during the session using the Q&A feature. During the presentation, we’ll be asking you several survey questions which can be answered using the Polling feature that will appear when each survey begins.
- #3 <Gina>Our speaker today is Barry Graubart, Vice President of Product Strategy at Connotate. Before I hand the presentation over to Barry, I’d like to take a few minutes to provide some background on Connotate and context for the discussion. Since 2000, Connotate has been helping global companies across a multitude of industries automate the processes that allow them to leverage Web data for strategic advantage.
- #4 <Gina>With technologies founded out of Rutgers University and USC, Connotate has 2 primary locations – one in New Brunswick, NJ and one in Irvine, CA.The company is well-funded by a number of venture capital firms.
- #5 <Gina>Connotate’s client base spans many industries and represents a who’s who of companies who are leveraging automation and technology to help them keep their competitive and strategic advantage in today’s competitive landscape.
- #6 <Gina>iI’d like to take a minute to explain how Connotate serves our clients. We are a trusted provider of enabling technology for many vertical markets. There are actually many use cases for our solution… including data aggregation, market research, social media monitoring and more. Connotate enables companies to automate processes where clients need to interact with Web based data. We speed access and collection times and we transform Web content into structured data so it can be used by business processes, business intelligence systems and big data applicationsToday we’ll focus on how our technology can efficiently extract pricing intelligence from public Web sources in real time to help companies enhance their dynamic pricing strategies.Now, I’d like to hand the presentation over to Barry Graubart.
- #7 <Barry>Thanks, Gina, and thanks to all of you for joining today. For those of you new to Connotate, a little quick positioning:Connotate offers a platform which automates web data monitoring and extraction at scale.Connotate powers many information services which you likely use today.Connotate is used today to automate the collection of pricing data from the web to feed pricing intelligence applications.
- #8 <Barry>Today, we will talk a bit about the challenges that come with monitoring prices at scale.We’ll also talk a bit about how the Connotate approach solves those issues and take a brief look behind the curtain at how the underlying technology works.
- #9 <Barry>
- #10 <Barry>Smart phones have empowered consumers in ways that would have seemed unbelievable just a couple of years ago.Customers today have full transparency into retail pricing across multiple products and retail distribution channels.That puts pressure on retailers and manufacturers to have greater insight into competitive pricing across their entire product catalogs and across multiple channels.Timescales have similarly compressed, rendering the traditional agency model largely useless.
- #11 <Barry>Whether your market is B2B or B2C, and whether you pursue real-time dynamic pricing or not, the Web has become a primary sources of data for a more informed pricing strategy.For Manufacturers, it is critical to know the true street price of your product to assure you’re sustaining value in the marketplace.If your product’s suggested retail price is $100 but Wal-Mart is selling it below cost to move inventory, you need to know.For retailers, you need to know what your competitors are charging at all times, no matter how often you change your prices. Clearly, manufacturers and retailers today must know at least as much as your customer about the competitive pricing landscape to make intelligent, competitive, informed pricing strategy decisions.
- #12 <Barry>If I can ask each of you to take a moment to respond to this brief polling question, it will help guide our discussion going forward.How do you currently obtain pricing intelligence?[read answers]You may check all that apply.
- #13 <Barry>
- #14 <Barry>Connotate does one thing really, really well -- Automate highly repetitive tasks to collect data from the WebData collection costs increase rapidly when you are monitoring high volumes of products – and especially if you are pulling prices frequently. If you are pursuing a dynamic pricing strategy it becomes even more important to be highly accurate.The cost curve is steepest for manual processes – they just don’t scale. Scripts scale better but there are tremendous maintenance headaches. Also, scripts often don’t work with today’s advanced Web technologies like Javascript, AJAX or HTML5.Connotate has completely revised the cost curve of scale, volume and precision – lowering it and flattening it out as well.With Connotate, you get accuracy no matter how many prices you need to collect or how often you need to collect them --and without the headaches of maintaining scripts.
- #15 <Barry>Applying that to pricing intelligence, Connotate’s highly scalable platform can monitor vast numbers of products or categories, across a wide range of distribution channels.Our system can be configured to deliver the full set of result data or just the elements that have changed. This enables support for multiple needs – you can grab the full feeds for comprehensive pricing analyses, while triggering notifications of price changes for tactical response.You can select product categories, search terms or lists of SKUs to monitor, and the system can monitor pricing across specific zip codes.And, of course, we can deliver in whatever format you need, to feed your BI and analytic systems.
- #16 <Barry>OK – another poll question.To help us steer the next part of the discussion, can you tell us in which ways you utilize pricing intelligence today?Again, select all that apply.
- #17 <Barry>So, looking at the workflow, the traditional model is to have a team collect the data, crunch it and send the results to you periodically.Connotate flips that model around, so you can continuously monitor pricing on web pages, driving your business intelligence and analytics applications.
- #18 <Barry>Now I’d like to walk through the Connotate workflow for gathering pricing intelligence
- #19 <Barry>Here is one illustration of where Web data, collected by Connotate, fits into the pricing life cycle.The GRAY boxes here represent the steps in the traditional pricing flow.This approach uses historical pricing data to drive strategy, which informs your pricing models.Connotate changes that approach. The GREEN boxes represent web data and this model shows how you can leverage web data to dynamically adjust pricing to optimize margins.A continuous flow of web data allows you to flip the model so that your decision-making is always informed by current pricing information. The data can be fed into your existing tools – whether a proprietary pricing model or off-the-shelf analytics applications.For retailers, real-time competitive pricing information specific to the consumer products you offer provides insight to drive operational performance, improve sales and minimize online abandonment.For large manufacturers or large brands, competitive pricing information provides added control over how your branded products are priced in online channels, ensuring compliance of your minimum advertised price strategy.
- #20 <Barry>Now we’ll go “under the hood” to see how it works.Connotate offers a hosted option or can be licensed as installed software. Most of our pricing intelligence clients choose the hosted option, in which case, our team configures the requirements, but either way it works the same way.The Connotate platform looks at a web page the same way you would. Then, the user (or, for the hosted solution, our own professional services team) simply clicks on the fields of data which they want to extract, analyze and monitor. As I’d mentioned earlier, websites have gotten a lot more complex. But Connotate acts the same way a user does on a website. Regardless of what technologies are used to build the site, Connotate can easily navigate, search, fill in forms, provide input parameter, all to bring back the specific data on the pages you need.Here’s an example of a subset of the data that we are extracting for this customer as it appears on a competitor’s website.The data includes product name, item number, product category, price and item availability.
- #21 <Barry>The Connotate software then extracts those key elements from the web page, and transforms it into a usable form, in this case, Excel.Here, we’ve highlighted a few of the data elements we capture; in practice, we are collecting many more data elements but to protect the customer’s identity, we are not exposing all of the detail.Once extracted, we can feed this data into your pricing engines, drive alerts to changes and more.
- #22 <Barry>As mentioned, Connotate can deliver the full feed of data, or just detect changes within the data set. Here, we’ve highlighted two changes – a pricing change to an air filter, and a new product – a battery shown at the bottom.In addition to tracking changes and additions, we also track which products have been removed from a site. As a manufacturer, you’ll want to see which of your products may have been removed from a distribution channel’s website.We can also track things like position on a page, so if you want to see if your products are being showcased at the top of the page for a given category or subcategory, that’s easy to do as well.
- #23 <Barry>Let’s take a quick look at a few case studies.In recent years, the Web has completely leveled the playing field for nationalhardlines retailers (electronics, hardware, housewares, automotive, sporting goods, etc.)Brand loyalty is disappearing andCustomers are constantly looking for the best price.To optimize margins, hardlines retailers MUST always be aware of what their competitors are charging.In this case, we helped a hardlines retailer increase market share 10% and overtake their next largest competitor in sales rankings.They were able to optimize their pricing by making decisions based on timely data.The value proposition here is the scale we can support.This retailer had a large set of products they wanted to track, across multiple retailers, and across specific zip codes for regional price differences.You can collect this data manually but not at scale. For this retailer, we are collecting millions of prices with a high degree of accuracy.
- #24 <Barry>The next case is an appliance manufacturer that supplies big box retailers. This business case is different from the previous one; the challenges are different.The manufacturer has a well-established, premium brand. But today, brand loyalty is diminishing across the board and there is no guarantee that this manufacturer will receive repeat orders next season – they can’t depend on retaining this channel just because of brand loyalty. This manufacturer needed a 360 degree view of pricing up and down the distribution chain, as well as visibility into the product specs of its competitorsand consumer product reviews.Here’s why. Let’s say their competitor’s appliance has a new feature that is totally unique. What are consumers saying about that feature?Are they willing to pay more for it? Do the product reviews indicate that this feature is boosting sales or not?Should the manufacturer include this type of feature in their next product model or not?By collecting Web data, we helped the manufacturer answer these questions to help them guide product enhancement, pricing, and ensure repeat orders from the Big Box channel.
- #25 <Barry>Here is an example of the data we extract for them:Feature and function breakdowns of competing products as well as how are they priced, andproduct reviews. We can deliver information to help them understand the pricing as well as all the buzz around a particular storefront. This is important especially among young buyers who base their buying decision on what the consensus says is both the best and the cheapest. It’s absolutely critical to this manufacturer to be informed of this consensus. This helps the manufacturer understand how its products are performing in the retail chain compared to its competitors. Web data also gives this manufacturer visibility into how different retail chains are pricing their products, as well as what kind of discounts are offered by different retailers at checkout. All of this information is helping this manufacturer enhance brand reputation and market share.
- #26 <Barry>This slide illustrates how clean, accurate data is delivered in a structured format – in this case, a spreadsheet which can be used by the manufacturer’s pricing and BI team. The manufacturer has control over what data they collect, and how often they collect it.Previously, this manufacturerspent their market intelligence budget on a 3rd party agency that collected data on a very narrow vertical slice of appliances. This agency controlled the collection and analysis of the data, and presented their findings in a PowerPoint every 3 or 6 months. The data was vertically rich – and the approach worked well in the old days. But today, it is no longer timely enough.Now, this manufacturer is going straight to the source – the Websites of retailers – and compiling the data to present it to their designers, engineers, and financial team as fast as needed instead of waiting 3 or 6 months for an agency report.In other words, it’s possible today to do it on your own – once you have the methodology in place you can get this competitive price intelligence as often as you need it instead of waiting months.
- #27 <Barry>OK – last polling question.Based on what we’ve discussed today, we’d like to better understand how an automated web extraction process might fit into your pricing intelligence strategy.
- #28 <Barry>So if you are thinking about using automation to collect competitive intelligence data on the Web, I’d like to share some best practices we’ve learned over the years to help you get started.
- #29 <Barry> Our team has lots of best practice experience, working with others for pricing intelligence. Here are a few things to start thinking about when developing your web pricing intelligence strategy:What sites are critical to support your decision-making?Which product categories, products or search terms you’d like to price-match. Are there regional issues – do you need to return pricing data for specific zip codes? How much data do you want back? How much can your team or your systems handle?What frequency do you want to see? Daily? Weekly? Hourly? Are there times when you want a full refresh for analysis, then other times that you just want to see changes for tactical response?What will you do with the data? If you’re loading it into analytical systems, what formats do they require?Connotate has lots of experience here and we’re happy to guide you through the process.Now, back to Gina.
- #30 <Gina>Several of you have asked about obtaining a copy of today’s presentation. We will send you a link to the archived presentation within 2 business days.Also – we’d like to invite you attend the 2nd and 3rd Webinars in our three-part series on competitive intelligence. You can find more information about these free webinars on our website.Now, for your questions.