For More Information

  • 702 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
702
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
4
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Contoso, Ltd.: Using Business Analytics to Customize Demographics Reporting and Analysis © 2001 Microsoft Corporation. All rights reserved. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 1
  • 2. The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT. Microsoft, BackOffice, the BackOffice logo, Visual Basic, Win32, Windows, and Windows NT are registered trademarks and ActiveX and Authenticode are trademarks of Microsoft Corporation. Other product or company names mentioned herein may be the trademarks of their respective owners. Microsoft Corporation • One Microsoft Way • Redmond, WA 98052-6399 • USA Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 2
  • 3. Contoso, Ltd.: Using Business Analytics to Customize Demographics Reporting and Analysis.......................................................................................................1 Abstract..........................................................................................................5 Executive Overview............................................................................................5 Highlights .......................................................................................................5 Product Overview ..............................................................................................6 Commerce Server 2000.....................................................................................6 The Business Analytics System...........................................................................6 The Commerce Server Data Warehouse..............................................................7 Step 1: Designing for Scalability....................................................................7 Step 2: Designing for Extensibility.................................................................8 Step 3: Integration/Interoperability with Other Products..................................8 Microsoft Application Center 2000.............................................................8 Initial Considerations for Business Analytics.....................................................8 Step 1: Gathering Business Requirements......................................................8 What Fundamental Metrics Does a Company Use to Measure its Operations and Success? ........................................................................................8 Step 2: Operational Planning........................................................................9 Step 3: Budget Negotiation/Feature Cuts/Prototypes.......................................9 Step 4: Feedback......................................................................................10 Step 5: Training Operations........................................................................10 Step 6: Phased Roll-out..............................................................................10 Step 7: Training Business Users..................................................................10 The Business Analytics Lifecycle......................................................................10 Establish Regular Checkpoints to Evaluate Needs................................................11 Evaluate Needs and Design a Solution for Adding New Data.................................12 Implement Changes to the Test Site.................................................................12 Step 1: Add Data Objects for Demographics.................................................12 To Add Data Objects to the Commerce Server Database..............................................12 Step 2: Add Demographic Properties to the User Profile..................................13 To Add Site Terms for Gender and Location................................................................13 To Add a Group to the User Object (for Demographic Properties)...............................13 To Add New Properties to the Demographics Group....................................................14 Step 3: Add Data Fields to the User Registration Page of the Test Site.............15 Step 4: Allow Site Data to Amass................................................................15 Run DTS Tasks to Update Data in the Data Warehouse........................................15 Step 1: Import Raw Customer Data into the Data Warehouse.........................16 Maintaining the Data Warehouse............................................................16 Data Deletion ................................................................................................................16 Configuration Synchronization .....................................................................................17 Web Server Log Import ................................................................................................17 Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 3
  • 4. Transaction Data Import ...............................................................................................18 Step 2: Import Secondary Data to Provide Context........................................18 Campaign Data Import...................................................................................................18 User Profile Data Import................................................................................................19 Product Catalog Import..................................................................................................19 IP Resolution..................................................................................................................19 Step 3: Export Data Using the Report Preparation Task..................................20 Present Data for the Business Analytics Lifecycle................................................20 Report Implementation: Static or Dynamic...................................................20 Static Reports ..........................................................................................21 Dynamic Reports ......................................................................................21 Create a Static Report Using SQL......................................................................21 Adding a Static Query to the Reports Module.................................................21 Step 1: Modify the Name strings in the Sample Script....................................21 Step 2: Create a SQL Query .......................................................................22 Step 3: Create a SQL Base Report Definition.................................................22 Step 4: Add SQL Report Dimensions............................................................23 Step 5: Add the Appropriate SQL Report Parameters......................................23 Step 6: Display and Export a SQL Report......................................................24 Conclusion................................................................................................25 Predictor Resource: Creating a Customized Analysis Model..................................25 Rolling Out Changes........................................................................................26 Training Business Managers........................................................................26 Summary: Closing the Loop..............................................................................26 For More Information.......................................................................................27 Documentation...............................................................................................27 Related White Papers......................................................................................27 Related Internet Sites.....................................................................................27 Service Packs.................................................................................................27 Commerce Server Newsgroups.........................................................................28 Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 4
  • 5. Abstract The Commerce Server Business Analytics System is a set of tools, database schemas, and methodologies designed to help you analyze the performance of your e-commerce enterprise. Microsoft has combined a variety of technologies—data transformation, data warehousing, data mining, and Analysis Services—that partner with business expertise, third-party products, and Microsoft training to create a rich yet easy-to-use site analytics solution. Customers with deployed Commerce Server sites can use the Business Analytics System to understand and optimize the complex interrelationships between customer preferences, marketing campaigns, product sales, and Web site design. The system is also designed to be expandable, so that as your business grows and your needs change, your business analytics can expand as well. The first part of this paper provides a general introduction to Commerce Server and the Business Analytics cycle. The second part describes best practices and methods for using and expanding Business Analytics. Throughout, the paper follows the course of the fictional company, Contoso, Ltd., as it decides to track new demographic information and make corresponding changes to the Data Warehouse and Analysis components in order to learn vital new information that will enhance their e-commerce venture. Executive Overview This paper follows the progress of Contoso, Ltd., a fictitious company, through the process of designing, configuring, deploying and maintaining an e-commerce Web site with the Business Analytics System tools provided by Microsoft® Commerce Server 2000. As a result of these analyses, Contoso, Ltd. will update their User Object profile definition and create new reports to manage the new information. Highlights Contoso, Ltd. will engage best practice guidelines and examine the following topics during setup and deployment: • Scalability issues • Extensibility issues • Integration issues • Co-location issues • Business requirements gathering • Operations planning • Prototyping the site • Evaluating and adjusting the site • Training personnel • Rolling out the site After the e-commerce site is operational, the company will run its regular Business Analytics evaluation and site maintenance routine. It will be determined that new demographic information must be added to the User Object profile definition. To make Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 5
  • 6. that data useful, corresponding changes must be made to the Business Analytics process. In this paper, Contoso, Ltd. will: • Define new demographic data • Make changes to the user profile • Update the Data Warehouse • Create new reports • Roll out changes • “Close the loop” Product Overview Commerce Server 2000 Contoso, Ltd. chose Microsoft Commerce Server 2000 for its comprehensive, integrated, and ready-to-use features. They wanted tools that could help them quickly develop, deploy, and manage a commerce application for the Web. Commerce Server allowed them to build both scalable business-to-consumer (B2C) and business-to-business (B2B) site applications. As each site was rolled out, Commerce Server enabled Contoso, Ltd. to easily analyze and manage their e-commerce operations. Commerce Server includes sophisticated analytical capabilities, such as an analysis-driven feedback loop that empowers business managers to respond to the ever-changing needs of users and partners. In this paper, Contoso, Ltd. takes advantage of the Commerce Server Business Analytics lifecycle to create a highly dynamic, personalized commerce site with the following results and benefits: • Optimize user experience by obtaining detailed user profiles. • Encourage repeat business by analyzing and implementing site changes based on the new profile. • Allow the company to successfully compete in the e-commerce marketplace by providing more accurate user- and partner-based decision analysis. The Business Analytics System The Commerce Server Business Analytics System helps decision makers evaluate both the internal and external performances of an e-commerce venture. The tools allow you to analyze site usage data in a manner that can justify improvements and identify optimizations. The analysis can affect everything from the layout of the site, to the products in the product catalog, to the marketing strategy used to sell those products. Business Analytics manages three main processes: • Collects raw Web site data across a number of dimensions (for example: user information, advertisements, and products purchased). • Consolidates, transforms, and stores that data in the Data Warehouse. • Enables a variety of decision makers to analyze the results from multiple points of view (for example, by user, product, and marketing campaign). The following figure shows the flow of data for these processes. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 6
  • 7. The Commerce Server Data Warehouse Although Commerce Server uses several databases, most of the Business Analytics tools interact with a Microsoft® SQL Server™ database called the Data Warehouse. The Data Warehouse acts as a central repository for electronic commerce and click-stream data. Source data is taken from Web server logs, Commerce Server transaction databases, and third-party sources such as existing customer databases. Once imported and summarized, the data can be analyzed and presented as part of a static or dynamic Analysis or predictive report. Based on analysis of these reports, enterprises can expand or target vital information. Bottom-line improvements can result from changing the site architecture to: • Enhance user experience • Follow through on successful advertising campaigns • Run more successful promotions The information gained from these reports should also provide more accurate information for partner-based business decisions. Step 1: Designing for Scalability Although the Commerce Server hardware requirements are fairly modest, the needs of an e-commerce site are much more exacting. Contoso, Ltd. implemented their site using redundant, load-balanced servers to provide a single-IP solution. This increased site capacity by distributing HTTP requests proportionally, according to each server’s capacity for handling the required load. Contoso, Ltd. implemented Microsoft® Windows® 2000 Advanced Server to provide Network Load Balancing (NLB) services. In addition, the company found that Microsoft® Application Center 2000 provided NLB enhancements, such as the Request Forwarder, to support multiple users for a single-IP address. At the hardware level, Contoso, Ltd. deployed redundant hardware components, including RAID disc arrays, disc mirroring, and dual disc controllers to minimize disc failures. They used a redundant fiber channel host bus adapter and switches for the Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 7
  • 8. System Area Network (SAN) configuration. In the event of an adapter or switch failure, the backup adapter or switch can provide an alternate path to the SAN. For more information about hardware requirements in other environments, see the Commerce Server 2000 white paper "Planning for Reliability and High Availability" located at http://www.microsoft.com/COMMERCESERVER/techinfo/planning/2000/wp_fivenines.asp . Step 2: Designing for Extensibility Because Commerce Server 2000 is designed for extensibility, it follows that each component in the Business Analytics System can be extended. Based on their business criteria, Contoso, Ltd. implemented a basic retail site modeled on the Commerce Server Retail site. As they used their Business Analytics components to gather more information, it was assumed that the design of the site, along with the underlying databases and report generation mechanisms, would be updated to meet previously discussed as well as newly identified business needs. Contoso, Ltd. also pre-determined a budget for maintaining and updating the site. Step 3: Integration/Interoperability with Other Products Microsoft Application Center 2000 Because Contoso, Ltd. had an existing database for their offline business they made plans to implement a system for synchronizing that data with Commerce Server 2000 at a later date. Contoso, Ltd. intends to use Application Center 2000 to manage this existing group of servers and to integrate their offline business data with their e- commerce efforts. Initial Considerations for Business Analytics Contoso, Ltd. and its consultants reviewed Microsoft case studies, discussed Commerce Server configurations, and developed a detailed plan before implementation and roll out. For a list of similar resources, see the "For More Information" section at the end of this document. The following are the steps that Contoso, Ltd. took and some of the issues they encountered. Step 1: Gathering Business Requirements What Fundamental Metrics Does a Company Use to Measure its Operations and Success? In order to establish metrics for the Web site, the team gathered the existing metrics that the business used to gauge its progress. Consultants found that many people in the company were uncertain about what the current metrics meant. When the Commerce Server 2000 system was introduced, Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 8
  • 9. metrics interpretation caused confusion, and was, in part, a reason for adopting a separate set of goals for the Web-based enterprise. To clarify the requirements issue, the consultants asked the following questions: • Are the metrics appropriate for the current situation? • What benefit does each metric provide in the day-to-day operations of the business? • What do the metrics actually measure? • Business growth? • Attracting new customers? • Retaining customers? • Targeting ads, campaigns, or direct mail? • Based on your experience, could better metrics be used? • How timely must the metric be to be accurate? • There is a reduction in site performance associated with updating statistics. Are reasonable expectations in place for how quickly changes can be evaluated on the site? • What are the costs involved in creating new metrics for our products? Contoso, Ltd. chose a simple set of e-commerce-related metrics as an interim goal for the Web site, with the expectation that they would integrate full metrics at a later date, after the site had amassed a threshold number of user visits. For example, Contoso, Ltd. selected the Browse to Buy ratio as the initial metric of success. Other metrics included Ad Clicks, Page Requests, Products Sold, and Products Browsed. These metrics are already supported by default, built-in Commerce Server reports, and thus carry little overhead or additional cost for an implementation team. Step 2: Operational Planning The consultants moved to the next phase of the project—operational planning. The following are some of the questions they asked: • How much of the existing hardware can be applied to the problem? • How long does it take to obtain new hardware? • How much is budgeted for the initial implementation? • How much is budgeted for recurring maintenance? • Is the staff competent with the applicable technology? • Are the parameters of any particular variable (computer size, storage size, daily volume of data, co-hosted applications, unfamiliar hosting facility, poor site design, and so on) out of line with what is expected? Step 3: Budget Negotiation/Feature Cuts/Prototypes At this point, a realistic budget was presented, using current assets within the company and adding incremental improvements. The budget included a plan to achieve more extensive goals at a later date by building on what will be learned by that point. The team evaluated the budget/resources-required gap and then negotiated the feature set accordingly. After the budget and feature cut process was complete, the consulting Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 9
  • 10. team implemented the initial prototype of the site. The site was then tested using simulated data in order to generate a report based on the new metrics. This in-house testing allowed all parties involved to buy in to the new system. With the time and budget remaining for this phase, the site was opened to a limited number of test engineers who were requested to provide feedback on the functionality of the site. Step 4: Feedback The information gained in "Step 3. Budget Negotiation/Feature Cuts/Prototypes" was used to drive a set of improvements and modifications to the site. Changes were limited by a cut-off date so that the site would be stable in time for a public rollout. Step 5: Training Operations The consulting team prepared a handbook for the Contoso, Ltd. operations group and provided hands-on training to bring the team up to speed. The handbook contained information on day-to-day procedures as well as a section on potential roadblocks and troubleshooting solutions. Step 6: Phased Roll-out Following the training operations, Contoso, Ltd. made plans for rolling upgrades so that the site would not incur downtime for planned changes. Step 7: Training Business Users Finally, the business staff were trained to use Commerce Server 2000 Business Desk, and given the tools to manage the catalogs and Business Analytics System. The Business Analytics Lifecycle The Business Analytics Lifecycle is the day-to day analysis of e-commerce site data that can result in updates to site architecture, product lines, customer service, and partner relationships that improve your bottom line. This lifecycle is often referred to as "closing the loop" within a Commerce Server 2000 implementation. The following figure shows the Business Analytics lifecycle. The Business Analytics lifecycle, as described and presented in this paper, refers to closing the loop between user feedback, site performance, and business decisions. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 10
  • 11. Contoso, Ltd. can now use the business analysis tools in Commerce Server Business Desk to manage day-to-day analysis of site data and update the site content and architecture. Updates are generally in the form of new campaigns. This day-to-day process of closing the loop, involves making regular, modest changes to the system in order to continually tune the site to better serve the user and your business partners, as measured by the “Browse to Buy” metric. The Business Analytics lifecycle is a comprehensive data gathering, analysis, and reporting system that manages changes to this day-to-day process. For example, new information gleaned from the Data Warehouse can suggest the need for additional data. In the case of Contoso, Ltd., the company discovered that they needed to start collecting demographic information to more accurately find appropriate market opportunities for their products. They determined the following user properties would benefit the accuracy of the segmentation and targeting models they will build with the Commerce Server 2000 Business Analytics System: • Gender • Age • Household size • Household income • Geographic location The following sections outline the process that Contoso, Ltd. followed to arrive at this conclusion and to make the necessary changes to their Commerce Server 2000 implementation, from modifying the User Object profile definition to updating analysis reports. This ongoing evaluation of the solution constitutes a complete Business Analytics lifecycle. Establish Regular Checkpoints to Evaluate Needs Contoso, Ltd. retained the consultants who set up the initial solution, and brought them in to evaluate the need for better defining marketing opportunities with their online customers. The consulting team discovered a growing dissatisfaction at the company with the current marketing opportunities, and immediately began asking questions to uncover the reasons. The study revealed a strong desire to perform more specific, targeted marketing than the information available in existing User Object profile definition allowed. In order to collect more demographic data and effectively use reports, Segment models, and Prediction models, Contoso, Ltd. needed to update the User Object profile definition. These updates would allow the Business Analytics staff to better target marketing campaigns and further increase product Browse to Buy ratios. Updates would also allow them to increase their click-through ratio for external advertisements, allowing predictive components to be used to better target users. Although expanding the User Object profile definition meant increased participation from users on the site, Contoso, Ltd. felt that users would be willing to take the time to enter more specific information if they were certain it would result in increased savings and better service. As an added incentive, the company offered free shipping for first orders. As mentioned, Contoso, Ltd. already had a procedure in place to budget for the on-going development and refinement of the site. The consulting team wrote a brief proposal summarizing the staff expectations and the scope of the changes required. After several Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 11
  • 12. content reviews and careful cost analysis, changes to the User Object profile definition were approved. Evaluate Needs and Design a Solution for Adding New Data At this point, the consulting team designed solution specifications and prepared to implement changes to the system. They wrote a task list, evaluated risks, and developed a process for rolling out the changes. This process included testing in a non- production environment, developing contingency plans, and creating backups to the existing site in case of a rollback. Implement Changes to the Test Site Step 1: Add Data Objects for Demographics As a first step, the System Administrator for Contoso, Ltd. added the following data objects to the Commerce Server database, using Commerce Server Manager tools. To Add Data Objects to the Commerce Server Database 1. Click Start, point to Programs, point to Commerce Server 2000, and then click Commerce Server Manager. 2. Expand Global Resources, and then expand the Profiles entry for the site to be managed. 3. Expand Data Sources, expand Data Objects, and then select User Objects. A list appears in the right pane displaying Data Members for the User Object. 4. On the Actions menu, click New Data Member. 5. In the New Data Member dialog box, type d_Gender in the Member Name field, and Gender in the Display Name field. 6. On the Data Type menu, click String, click Add to place the new data member in the list, and then click Finished. Make sure you add a member before you click Finished, or you will lose your changes. Repeat the steps above to add each of the following data objects to the Commerce Server database: Member Name Display Name Data Type d_Gender Gender String d_Age Age Number d_Household size Household size Number d_Household income Household income Number d_Geographic location Geographic location String Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 12
  • 13. Step 2: Add Demographic Properties to the User Profile The User Object profile definition is designed to collect data about users. Since Contoso, Ltd. based their site on the Commerce Server Retail Site, a basic profile definition was already provided. Contoso, Ltd. now needed to expand that definition using Commerce Server Business Desk tools. Some of the data required custom data type definitions. For example: • Gender, with the following types: “Male” and “Female” • Geographic location, with the following types: “North”, “South”, “East”, and “West” This required the addition of custom Site Terms. To Add Site Terms for Gender and Location 1. Open Commerce Server Business Desk. 2. In Users, click Site Terms Editor. 3. In the Site Term Structure list, click User Site Terms, and then click Add. 4. In the Add dialog box, select Add a new Site Term, and then click OK. 5. In the Site Terms Attributes section, do the following: Use this To do this Name Type Gender. Display Name Type Gender. Description Type a description for this Site Term. 6. In the Terms section, click New. 7. In the Term Name box, type Male, in the Term Display Name box, type Male, and then click Accept. 8. In the Terms section, click New. 9. In the Term Name box, type Female, in the Term Display Name box, type Female, click Accept, and then click Apply. To add the location type, repeat the steps above, using North, South, East, and West as the Site Term names. After the data types are available, you can add the appropriate demographic elements to the user profile. Use the following procedure to add properties to the User Object profile definition. To Add a Group to the User Object (for Demographic Properties) 1. Open Commerce Server Business Desk. 2. In Users, click Profile Designer. 3. In the Profiles screen, in the Name box, select User Object, and then click Open on the toolbar. 4. Click Add to open the Add dialog box. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 13
  • 14. 5. In the Add dialog box, select Add new group, and then click OK. 6. In the Profile: User Object screen, in the Attributes section, do the following: Use this To do this Name Type Demographics. Display Name Type Demographics. Description Type a description. 7. Click Apply. To Add New Properties to the Demographics Group 1. Open Commerce Server Business Desk. 2. In Users, click Profile Designer. 3. In the Profiles screen, in the Name box, select User Object, and then click Open on the toolbar. 4. In the Profile: User Object screen, in the Properties list, click Demographics, and then click Add. 5. In the Add dialog box, select Add a new property, and then click OK. 6. In the Profile: User Object screen, in the Attributes section, do the following: Use this To do this Display Name Type Gender. Description Type a description. Type Select Site Term from the drop-down list. Reference Click the ellipsis [...] button to open the Site Term Selection dialog box. In the Site Term Selection box, in the Select a Site Term box, click User Site Terms, select Gender, and then click OK. 7. In the Advanced Attributes section, in the Map to data box, click the ellipsis [...] button to open the Data Source Picker dialog box. 8. In the Data Source Picker dialog box, click ProfileService_SQLSource, and then click User Object. The new members that the System Administrator added to the database appear at the bottom of the data sources list. Select the appropriate member, and then click OK. 9. In the Exported box, click the Yes check box to export the data to the Data Warehouse. 10. Click Apply. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 14
  • 15. Repeat the steps above to add each of the following properties to the User Object profile definition: Display Name/ Data Field Name Type Age Number Household size Number Household income Number Geographic location Site Term, Location Because these new properties are an extension to the profile definition, no columns are designated in the database for these properties. When these properties are added, a new column is automatically created in the Data Warehouse. If you want to create a column with a specific name, or if you want to export a value to a particular column that already exists in the Data Warehouse, use the custom attribute “DWMemberName” and add the name of the column to the value field. More information on this can be found in the topic "Profile Definitions" in Commerce Server 2000 Help. Step 3: Add Data Fields to the User Registration Page of the Test Site The Web designer and Web Administrator for Contoso, Ltd. added the new properties to the site Registration page. Step 4: Allow Site Data to Amass Contoso, Ltd. waited an appropriate length of time to allow new data to be collected from the site and stored in the database. When sufficient data had been collected, the company proceeded to optimize that data by running the following Data Transformation Services (DTS) tasks and then designing and generating analysis reports. Run DTS Tasks to Update Data in the Data Warehouse Data Transformation Services (DTS) tasks are SQL scripts that are provided by Commerce Server to simplify the process of importing and managing data within the Data Warehouse. These scripts may be run manually from Microsoft SQL Server or may be scripted to run automatically by the site developer. In the case of Contoso, Ltd., the tasks were run manually. Contoso, Ltd. needed to evaluate their changes to the User Object profile definition and determine if any changes to the Data Warehouse schema were needed. In this case — the addition of demographic data — the answer was no. For information about changing the Data Warehouse schema, see "Extending the Commerce Server 2000 Data Warehouse" in Commerce Server 2000 Help. Also see the white paper "Extending the Commerce Server Data Warehouse/Analytics" located at http://www.microsoft.com/COMMERCESERVER/techinfo/development/2000/wp_extendin gdatawarehousing.asp. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 15
  • 16. Reports, along with Segment and Prediction models, are the primary Business Analytics tools in Commerce Server. Each Commerce Server DTS task is provided to help collect the data necessary for running those reports and models. DTS tasks import and process data from Web log files and Commerce Server databases, such as electronic commerce and click-stream information, into the Data Warehouse. DTS tasks should be run regularly to update the database with the latest information such as transaction, campaign, user profile, and product information from your commerce site. The following figure shows the steps to process data and generate a report. The first three steps are listed in the following section, and the last is covered in it's own topic, "Present Data for the Business Analytics Lifecycle." Step 1: Import Raw Customer Data into the Data Warehouse Maintaining the Data Warehouse When Contoso, Ltd. imported new data into the Data Warehouse, it needed to delete obsolete data, and then update the Data Warehouse to conform to their site configuration. This was accomplished using the following DTS tasks: • Data deletion • Configuration synchronization • Web server log import • Transaction data import These tasks can be performed manually or programmatically. Data Deletion The data deletion DTS task deletes log file data and summarized data from the Data Warehouse. It is important to frequently delete obsolete log file data. This helps to prevent declining performance and frees up storage space in the Data Warehouse. If you need to maintain historic data from your Commerce Server database, you can use reports to extract and save the data in a convenient format for future reference. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 16
  • 17. Configuration Synchronization The configuration synchronization DTS task synchronizes the Data Warehouse with the Commerce Server configuration database after you make changes to configuration settings. If you change your site configuration, perform this task before you perform any other DTS task. Most of the DTS import tasks provided by Commerce Server, such as user profile and product information imports, require configuration synchronization before they can be successfully run. The following figure shows the synchronization process between the Web Server Log and the Commerce Server databases. Web Server Log Import Use this DTS task to import Web log file data from your site. You can determine the import frequency for the data by analyzing traffic at your site. As a rule, the more traffic your Web site receives, the more frequently you should import Web log file data. You should also run this task to set the properties for the Web log file that determines which data is imported, and how individual users and site visits are distinguished. You should run the Web server log import DTS task just prior to running the report preparation DTS task. For example, if you have hourly Web log imports, import only a few logs at a time. As the size of Web logs increase, your general gigabyte (GB)/hour performance will decrease. There are also performance issues with log files that are minimal in size. For small log files, starting up and flushing each log file may outweigh the benefits derived from the smaller data set. In addition, if the log files contain too little data, the flushing rate may not be optimal. Web log file import causes indexes to become fragmented. This, in turn, can cause performance issues with the Web log import DTS task. As a general rule, you should defragment indexes on all tables that are more than 30 percent fragmented on a weekly basis. For detailed information about running this task, see "Importing Web Log File Data" in Commerce Server 2000 Help. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 17
  • 18. Transaction Data Import Transaction data is created on your Web site whenever a user adds or removes items from a shopping cart or makes a purchase. To analyze transaction data, you must generate reports. This DTS task is used to define and import the data required for those reports. For detailed information about running this task, see "Importing Transaction Data" in Commerce Server 2000 Help. Step 2: Import Secondary Data to Provide Context Campaign information, as well as product catalog information, can be imported to the Data Warehouse. This information is used to resolve product IDs into product names. Actual product names add context and detail to the user transaction reports that you generate through the Reports module in Commerce Server Business Desk. In a similar manner, when you import the user database, you allow the resolution of user IDs to names. Your reports can then present data using actual names instead of unrecognizable, numerically ordered IDs. Internet Protocol (IP) resolution is another feature that can add readability to the reports generated from information in the Data Warehouse. IP resolution converts numerical Domain Name System (DNS) entries to domain names, helping you quickly identify users and companies. The following figure shows three types of information that Commerce Server imports to provide you with meaningful data in your reports. Campaign Data Import A campaign is a marketing program that uses multiple communication vehicles to accomplish a specific result, such as increasing market share, introducing new products, or retaining customers. Campaigns are created using Commerce Server Business Desk. The Campaign data import DTS task imports campaign data, such as target ads or discounts to users of a Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 18
  • 19. specific profile. This data is important for analyzing the success of your marketing campaigns. After the data is imported, it is typically used to generate reports using the Report preparation DTS task. For detailed information about running this task, see "Importing Campaign Data" in Commerce Server 2000 Help. User Profile Data Import Commerce Server collects user profile data when users visit your Web site. The profile data contains a Global Unique Identifier (GUID) for guest users (users who have not registered or completed a customer profile form), as well as behavior data. For registered users, the data contains the personal information entered by the users according to the User Object profile definition you have established for your site. This data can be used in demographic reports and targeted marketing. The User profile data import DTS task imports this user profile data. This task has an option to specify the query interval — the amount of time between queries. For example, a 15-minute query interval (the default for this task) means that the import tool would pull 15 minutes' worth of data at a time, thereby locking fewer records in the source at one time and improving performance. However, this also means running more queries on the source to cover an equal amount of time. Setting this value to 0 (zero) means the query will pull all data at one time, which will lock the entire table. The recommended setting is 30 minutes. The other User Profile import option, “Import Start Date” determines the start time for imports. By default, it is set to the current day’s date. You may want to change this setting the first time you set up your system. For detailed information about the options available for this task, see "User Import Properties" and "Importing User Profile Data" in Commerce Server 2000 Help. Product Catalog Import The product catalog import DTS task imports data from the Product Catalog System into the Data Warehouse. Importing product data supplies your transaction reports with more descriptive product information than the information available simply from transaction data. For example, products can be identified by name, rather than product ID. By running the product catalog import DTS task, you can run reports to help you analyze product performance on your site. Note • Do not confuse this task with the process of importing data into the Product Catalog System from outside sources. For detailed information about the options available for this task, see "Importing Catalog Data" in Commerce Server 2000 Help. IP Resolution The IP resolution DTS task resolves Internet Protocol (IP) addresses from the imported user data. IP resolution converts IP addresses imported from your log files into domain names. For example, 198.105.232.4 becomes www.microsoft.com. This conversion Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 19
  • 20. helps readers of your reports more easily understand or situate the report data in relation to its source. You should run the IP resolution DTS task regularly to resolve new IP addresses. This ensures that the Data Warehouse contains the most recent data possible any time you need to generate a report. When you initiate an IP resolution DTS task, a cache of previously resolved IP addresses is checked in an attempt to resolve new IP addresses. If unresolved IP addresses remain after the cache has been checked, the Domain Name System (DNS) server is then checked. IP resolution typically has a 60 to 75 percent success rate. Note that IP addresses are resolved in a batch file, optimizing the connection with your DNS server. Step 3: Export Data Using the Report Preparation Task Use the report preparation DTS task to populate the online analytical processing (OLAP) cubes with the most recent site and user data stored in the Data Warehouse. Commerce Server 2000 uses OLAP cubes to organize summarized data stored in the Data Warehouse, allowing reports run from the Analysis modules in Commerce Server Business Desk to complete more quickly than they would without this additional step. To keep the data in the OLAP database current, run the report preparation DTS task each time you import or delete data from the Data Warehouse. You will need to run the report preparation DTS task in Full mode when data has been deleted from the Data Warehouse. In all other instances, run Incremental mode to add only the new data from the Data Warehouse to the OLAP cubes. Note • Commerce Server 2000 Service Pack 1 enhances scalability when running this task and should be installed. For more information about OLAP cubes, see the Microsoft® SQL Server™ 2000 documentation and reference the Analysis Services topics. Present Data for the Business Analytics Lifecycle Your optimized data is now stored in the Data Warehouse but you have no way to display that data to your key readers. Reports and analysis models are the tools you must use to manage and present that data in a way that your readers can understand. The Commerce Server 2000 Retail Sample site includes several reports that draw from existing data in the sample site. In order to incorporate custom data, you must create new reports. Report Implementation: Static or Dynamic There are two ways to implement analytical reports — static or dynamic. In each case, the process by which the report is generated and displayed varies significantly. Select an implementation that best suits your technical, reporting, and analytical needs. You can use the Predictor resource to create Segment models, which can then be used for business analysis. Customized analysis models are briefly discussed later in this document. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 20
  • 21. Static Reports Static report definitions consist of a query along with parameter definitions. When you run a static report, the parameters are presented to the operator and then passed to a rendering component. The rendering component then generates an HTML document from the query results. Completed static reports are stored as HTML files on the computer that houses your Commerce Server Business Desk. These reports can be opened using Business Desk and displayed in your browser window (for Commerce Server, your browser must be Microsoft Internet Explorer 5.5 or later). The following section of this document, "Create a Static Report Using SQL," provides six steps for creating a static report and a summary of the results. For more information about implementing static reports, see "Creating Static Reports Using SQL" and "Creating Static Reports Using MDX" in Commerce Server 2000 Help: Dynamic Reports Dynamic report definitions are stored as queries, and the pivot table view is stored in XML format in the Data Warehouse. Query results are displayed in the Microsoft® Office 2000 Web Components (OWC) Pivot Table component. For more information about implementing dynamic reports, see "Creating Dynamic Reports" in Commerce Server 2000 Help. Create a Static Report Using SQL Adding a Static Query to the Reports Module Static reports are defined by SQL code that resides in several SQL scripts. These scripts are used to drive the display of report names in the Reports module in Commerce Server Business Desk and the entry of user parameters when you run the report. Since several scripts need to be changed in order to install a new static SQL report, it is recommended that a user-defined script be used to contain the changes. This script should be run to install or update the report, but is not needed after that point. The following code is based on a sample script located in the Commerce Server 2000 Software Development Kit (SDK). This template is recommended for use in creating custom static SQL reports. Perform the following steps to modify the sample script and create a functional custom report. Step 1: Modify the Name strings in the Sample Script Copy the file Analysis–New static SQL report script.sql from the Commerce Server 2000 SDK (located in the installation folder in your Microsoft Commerce ServerSDKSamplesBusiness AnalyticsScripts directory) to an appropriate place on your computer. Open the script in the text editor of your choice and make the following changes by searching for the strings provided below: 1. Replace "Insert report name here" with a report name. 2. Replace "Report description–short" with a one-line report description. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 21
  • 22. 3. Replace "Insert report category here" with a report category that the user can later use to find reports of this type. 4. Replace "Insert report creator here" with your name. 5. Replace "Report description–long" with a detailed report description (optional). Step 2: Create a SQL Query Create a standard SQL query to return the information you want to include in your reports. The parameters you choose will allow Business Desk users to select alternate values and re-run reports to focus on different reporting needs. The following example contains the query found in the sample script you copied and modified in Step 1: Select @ReportQuery = 'SELECT [$SelectOrder] UserId rcp_guid, DateCreated, DateRegistered, Email rcp_email, FirstName, LastName, TelephoneNumber, UserTitle, UserType Into [$ResultTable] From RegisteredUser, LinkSiteRegisteredUserRel, Site Where RegisteredUser.RegisteredUserID = LinkSiteRegisteredUserRel.RegisteredUserID And LinkSiteRegisteredUserRel.SiteID = Site.SiteID And [$SiteName] And [$DateRange] And [$Expression] [$SimpleValue] Order By rcp_email asc' Step 3: Create a SQL Base Report Definition Create the base report definition using a set of INSERT statements in the Report table. The SQL query that you created in Step 2 is inserted into the [Query] field. The DmExport and UpmExport bits are set so the report can be exported to Direct Mailer (which uses the rcp_email attribute) and Profile Service (which uses the rcp_guid attribute). The ReportType bit is set to Static_SQL (2). The Protected bit is cleared (set to 0), which allows the report to be overwritten or deleted. -- Insert the report definition Insert Into [dbo].[Report] ([DisplayName], [Description], [ReportType], [Category], [Query], [createdby], [Definition]) Values (@ReportName, @ReportShortDescription, @Static_SQL, @ReportCategory, @ReportQuery, @ReportCreator, @ReportLongDescription) Select @ReportID = @@identity For more information about the Report table, see "Analysis/Reporting Schema" in Commerce Server 2000 Help. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 22
  • 23. Step 4: Add SQL Report Dimensions Create the displayable report dimensions using a set of INSERT statements in the ReportDimension table. Specify the fields to display and the field dimension type: column, row, or measure. Note that fields that are specified in the SELECT statement but not inserted in the ReportDimension table may be exported but not displayed. -- Specify report dimensions. Specifies how resultset fields are to be used in the HTML report. Insert Into [dbo].[ReportDimension] ([ReportID], [DimensionType], [DisplayName], [FieldName], [Ordinal]) Values (@ReportID, @Param_ColDim, 'Email', 'rcp_email', 1) Insert Into [dbo].[ReportDimension] ([ReportID], [DimensionType], [DisplayName], [FieldName], [Ordinal]) Values (@ReportID, @Param_ColDim, 'UserId', 'rcp_guid', 2) Insert Into [dbo].[ReportDimension] ([ReportID], [DimensionType], [DisplayName], [FieldName], [Ordinal]) Values (@ReportID, @Param_ColDim, 'FirstName', 'FirstName', 3) Insert Into [dbo].[ReportDimension] ([ReportID], [DimensionType], [DisplayName], [FieldName], [Ordinal]) Values (@ReportID, @Param_ColDim, 'LastName', 'LastName', 4) Insert Into [dbo].[ReportDimension] ([ReportID], [DimensionType], [DisplayName], [FieldName], [Ordinal]) Values (@ReportID, @Param_ColDim, 'DateCreated', 'DateCreated', 5) Insert Into [dbo].[ReportDimension] ([ReportID], [DimensionType], [DisplayName], [FieldName], [Ordinal]) Values (@ReportID, @Param_ColDim, 'DateRegistered', 'DateRegistered', 6) For more information about the ReportDimension table, see "Analysis/Reporting Schema" in Commerce Server 2000 Help. Step 5: Add the Appropriate SQL Report Parameters Add the report parameters that correspond to the query parameter tags in the ReportParam table. Create the report parameters using a set of INSERT statements in the ReportParam table. The parameterizations are in the form: [ Operand1 Value1 Operand2 Value2 FieldName ] Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 23
  • 24. An expression equivalent to the parameterization is given immediately after each VALUES clause, first in the format described above, and second, if needed, in an easier- to-understand format. -- Add report parameters Insert into [dbo].[ReportParam] (ReportID, ParamName, ParamDescription, ParamType, DataType, Opnd1, Val1, Ordinal) Values (@ReportID, '[$SelectOrder]', 'Number of users', @ParamType_SelectOrder, @DataType_integer, @SelectOrderOpnd_Top, 25, 1) Insert into [dbo].[ReportParam] (ReportID, ParamName, ParamDescription, ParamType, DataType, Opnd1, Val1, Val2, FieldName, Ordinal) Values (@ReportID, '[$DateRange]', 'User registration date', @ParamType_DateRange, @DataType_date, @DateRangeOpnd_From, '03/01/2000', '03/31/2000', 'DateRegistered', 2) Insert into [dbo].[ReportParam] (ReportID, ParamName, ParamDescription, ParamType, DataType, Opnd1, Val1, Opnd2, Val2, FieldName, Ordinal) Values (@ReportID, '[$Expression]', 'User type', @ParamType_Expression, @DataType_integer, @ExpOpnd_GreaterThanOrEquals, '0', @ExpOpnd_LessThanOrEquals, '1', 'UserType', 3) Insert into [dbo].[ReportParam] (ReportID, ParamName, ParamDescription, ParamType, DataType, Val1, Ordinal) Values (@ReportID, '[$SimpleValue]', 'Email qualifier', @ParamType_SingleValue, @DataType_text, 'And Email is not Null', 4) Insert into [dbo].[ReportParam] (ReportID, ParamName, ParamDescription, ParamType, DataType, Opnd1, FieldName, Ordinal, Automatic) Values (@ReportID, '[$SiteName]', 'Site Name', @ParamType_SiteName, @DataType_text, @SiteNameOpnd_Equals, 'Site.SiteName', 5, 1) For more information about the ReportParam table and the various parameters, see "Analysis/Reporting Schema" in Commerce Server 2000 Help. Step 6: Display and Export a SQL Report After you have successfully completed steps 1 through 5, run the script on the Commerce Server 2000 Data Warehouse database. The new report will appear in the Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 24
  • 25. Reports module in Commerce Server Business Desk. You can now run or export the report from Business Desk. For more information about running or exporting this report, see "Business Desk Analysis" in Commerce Server 2000 Help. After you run the report, you can view it from Business Desk and begin your analysis. Conclusion As a working example, the script creates a static report based on site visit data that can help you analyze user profile information. The script file is based on the following attributes: • Registration occurred between 3/1/2000 and 3/30/2000. • The UserType equals 1. • An e-mail address was provided during registration. After the steps have been successfully run, the report will be listed in the Reports module in Commerce Server Business Desk. After the report is run, it will display the UserID, DateCreated, DateRegistered, Email, FirstName, LastName, TelephoneNumber, UserTitle, and UserType attributes of the first 25 matched users sorted by Email address. Although this is a basic example, this data can be used as contextual information for deeper analysis. This sample script can also be easily expanded to support the new demographic information collected in the user profile. Predictor Resource: Creating a Customized Analysis Model The Predictor resource is a Microsoft® Windows® 2000 service that is optionally installed with Commerce Server 2000 as part of a complete or custom installation. The Predictor resource enables you to build complex analysis models that add predictive capabilities to your e-commerce site. You use the Predictor resource to build custom analysis models by specifying the table columns or attributes that you want to predict. For example, you can determine the type of content, advertisements, or cross-sells that might interest your site users. After the Predictor resource builds an analysis model, you can add the Predictor Client Object to your Active Server Pages (ASP) pages, thereby adding “intelligent” cross-sell, product recommendations, and property prediction to your site. You can build multiple analysis models for each e-commerce site. However, this process is resource-intensive; it is recommended that you build an analysis model only when the impact on your computer or server will be minimal. The time required to build an analysis model depends on the following elements: • Size of the input data (total number of cases) • Sample size • Number of attributes in the data • Hardware and topology of your servers To deploy your customized analysis models on your Web servers, your site developer must modify the Global.asa file so that it contains a call to the LoadModelFromDB method. This call loads the Prediction model from your Commerce Server Data Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 25
  • 26. Warehouse to your Web servers at application start. As a result, this method requires that your Data Warehouse is always available. For more information about a building custom analysis model, see "Running the Predictor Resource" in Commerce Server 2000 Help. Rolling Out Changes Contoso, Ltd. added new demographic tracking information to their e-commerce site, and then completed the Business Analytics lifecycle described above. As part of its site, the company maintains a secure test environment that is separate from the production environment. Before updating the servers in their production environment, they deployed and tested their updates in this separate staging/test environment, allowing testing to be managed without impact to customers using the existing site. After they were satisfied that the new demographic information and analytics processes were operating successfully, they used SQL Server to replicate the updates to their production servers. Training Business Managers The consulting team updated the handbook for the Contoso, Ltd. operations group, and provided hands-on training to bring the team up to speed with the new data and analytics information. As part of the rollout testing process, managers were encouraged to experiment with the new data in the user profile and generate sample reports. Using formal training and informal testing, Contoso, Ltd. efficiently trained staff and tested the system updates in a timely manner. Summary: Closing the Loop Contoso, Ltd. completed the update to their Web site, and their newly trained staff integrated the updated capabilities into daily routines. By increasing the demographic information available to business managers, the company is now able to perform the following tasks more effectively: • Target special products and offers based on the user's location. • Offer custom services based on user age. • Evaluate the popularity of products based on demographic information. • Suggest and justify additions to partners’ product lines. • Implement a more time- and cost-efficient shipment method, such as a new warehouse location, based on user location. Contoso, Ltd. acquired useful marketing information about their most frequent user. For example, which age and income groups spend the most money on the site? This information allowed the company to revise its catalogs and pricing models, and to confidently improve its e-commerce performance, as proven by their pre-selected metrics. Most importantly, Contoso, Ltd. successfully met their goal of increasing the site’s Browse to Buy ratio. The success of the site validates the well-planned initial focus on scalability, extensibility, integration issues, and requirements gathering. Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 26
  • 27. Establishing procedures for site upgrades and maintaining an ongoing relationship between site architects and business staff priorities made the process of updating the site as simple and economical as possible. Finally, the consulting team followed up with clients to determine the rate of user satisfaction with the recent changes to the site. The resulting data allowed Contoso, Ltd. to schedule time for minor tuning and modifications with minimal impact to their users. The process is already underway to gather suggestions for the next round of improvements. For More Information Documentation • Microsoft Commerce Server 200 Help (installed with product) • Microsoft Commerce Server 2000 TechNet Documentation Related White Papers • Microsoft’s Business Internet Analytics • Extending the Commerce Server 2000 Data Warehouse/Analytics • Planning for Reliability and High Availability Related Internet Sites Microsoft Commerce Server Homepage http://www.microsoft.com/commerceserver/default.asp Microsoft SQL Server Homepage http://www.microsoft.com/sql/default.asp Microsoft Application Center Homepage http://www.microsoft.com/applicationcenter/default.asp Microsoft BizTalk Server Homepage http://www.microsoft.com/biztalk/default.asp Microsoft Windows 2000 Server Homepages http://www.microsoft.com/windows2000/server/ http://www.microsoft.com/windows2000/advancedserver/ http://www.microsoft.com/windows2000/datacenter/ Microsoft Commerce Server Solution Sites http://www.microsoft.com/commerceserver/downloads/solutionsites(fromCS2K).asp Service Packs • Microsoft® Commerce Server Service Pack 1 • Microsoft® Windows® 2000 Service Pack 2 • Microsoft® Internet Explorer 5.5 Service Pack 1 • Microsoft® XML 3.0 Service Pack 1 • For Microsoft® SQL Server™ 7.0: SQL Server 7.0 OLAP Services Service Pack 3 Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 27
  • 28. Commerce Server Newsgroups Announcements microsoft.public.commercserver.announcements Business Desk microsoft.public.commercserver.businessdesk Campaigns CSF microsoft.public.commercserver.campaigns_csf Catalog microsoft.public.commercserver.catalog Data Warehousing microsoft.public.commercserver.datawarehousing General microsoft.public.commercserver.general Setup Deployment Operations microsoft.public.commercserver.setup.deploymentoperations Software Developers Kit microsoft.public.commercserver.sdk Solution Sites microsoft.public.commercserver.solutionsites User Profile Management microsoft.public.commercserver.userprofilemgt Microsoft Commerce Server 2000 Contoso, Ltd.: Using Business Analytics White Paper 28