Data Samples & Data Analyses
NYU | SCPS
Database Management & Modeling
Edward Colet
[email protected]
Asynchronous Session 3, week of June 7 2021
Class material and homework so far
You should be through text chapters 1-3 (introduction), 4-5 (database fundamentals), and the supplemental readings on RDMBS’s and BigData;
HW submissions were a short write-up about yourselves (hw1), a relational database design exercise, and a BigData discussion (hw2)
Questions from the material?
Please feel free to also use the discussion section on our NYU Discussion site to ask, answer, comment on material from this week (for this week, this will be part of hw3)
Content for this week
Chapter 6: The Analysis Sample
Chapter 7: Analyzing and Manipulating Customer Data
Online Khan Academy content to Introduce SQL
Week3 Overview
‹#›
‹#›
Key themes for this week (and the course)
Databases are important for storing data (obviously), but you have to analyze the data as well otherwise you just have a “data tomb”. The analysis of data to gain insights is what gives the data it’s power and makes it really valuable.
This week we’ll learn about some fundamental analytic concepts operations associated with analysis; We’ll review Correlation, a foundational basis for analytics and modeling; We’ll learn some of the fundamental operations to slice and dice data, and we’ll write basic SQL (Structured Query Language) code to create a table, populate it with records, and query the table to extract and summarize information.
Week3 Overview
‹#›
‹#›
The Analysis Sample
Chapter 6
Key Point of the Chapter:
Data analyses are usually (almost always) done on subsets of the data in the database. As such, the following are the key concepts and points to understand about working with subsets of data
Representative samples
Random samples
Frozen files
Test and validation data sets
Chapter 6: The Analysis Sample
‹#›
‹#›
Know some common marketing scenarios that would be suitable to use a sample and test…
To gauge new product offering/response
Price elasticity
Impact of a creative / change
Identify target market for new test
Gain insights on specific groups/segments
. . . Any decision about your product in the market can be tested and analyzed to minimize/gauge risk
Chapter 6: The Analysis Sample
‹#›
‹#›
What is a representative sample?
A sample accurately reflecting the population of interest from which the marketer wants to draw inferences.
Can not extend or apply results from one population to another
Can not purposely exclude names except for “permission opt-outs”, or other recently promoted per rules/regulations
What is a random sample?
When every member equally likely to be chosen
Nth selects is one approach (select every nth name)
Chapter 6: The Analysis Sample
‹#›
‹#›
What is a frozen file?
A file containing a snapshot view of the customer(s) at the time of the promotion, updated with response data to the promotion
Why is a fr ...
Micro-Scholarship, What it is, How can it help me.pdf
Data Samples & Data AnalysesNYU SCPSDataba
1. Data Samples & Data Analyses
NYU | SCPS
Database Management & Modeling
Edward Colet
[email protected]
Asynchronous Session 3, week of June 7 2021
Class material and homework so far
You should be through text chapters 1-3 (introduction), 4-5
(database fundamentals), and the supplemental readings on
RDMBS’s and BigData;
HW submissions were a short write-up about yourselves (hw1),
a relational database design exercise, and a BigData discussion
(hw2)
Questions from the material?
Please feel free to also use the discussion section on our NYU
Discussion site to ask, answer, comment on material from this
week (for this week, this will be part of hw3)
Content for this week
Chapter 6: The Analysis Sample
Chapter 7: Analyzing and Manipulating Customer Data
2. Online Khan Academy content to Introduce SQL
Week3 Overview
‹#›
‹#›
Key themes for this week (and the course)
Databases are important for storing data (obviously), but you
have to analyze the data as well otherwise you just have a “data
tomb”. The analysis of data to gain insights is what gives the
data it’s power and makes it really valuable.
This week we’ll learn about some fundamental analytic concepts
operations associated with analysis; We’ll review Correlation, a
foundational basis for analytics and modeling; We’ll learn some
of the fundamental operations to slice and dice data, and we’ll
write basic SQL (Structured Query Language) code to create a
table, populate it with records, and query the table to extract
and summarize information.
Week3 Overview
‹#›
‹#›
3. The Analysis Sample
Chapter 6
Key Point of the Chapter:
Data analyses are usually (almost always) done on subsets of
the data in the database. As such, the following are the key
concepts and points to understand about working with subsets of
data
Representative samples
Random samples
Frozen files
Test and validation data sets
Chapter 6: The Analysis Sample
‹#›
‹#›
Know some common marketing scenarios that would be suitable
to use a sample and test…
To gauge new product offering/response
Price elasticity
Impact of a creative / change
Identify target market for new test
Gain insights on specific groups/segments
. . . Any decision about your product in the market can be tested
and analyzed to minimize/gauge risk
4. Chapter 6: The Analysis Sample
‹#›
‹#›
What is a representative sample?
A sample accurately reflecting the population of interest from
which the marketer wants to draw inferences.
Can not extend or apply results from one population to another
Can not purposely exclude names except for “permission opt-
outs”, or other recently promoted per rules/regulations
What is a random sample?
When every member equally likely to be chosen
Nth selects is one approach (select every nth name)
Chapter 6: The Analysis Sample
‹#›
‹#›
What is a frozen file?
A file containing a snapshot view of the customer(s) at the time
of the promotion, updated with response data to the promotion
Why is a frozen file important?
Analysis of non-frozen data of customers that have responded
5. may lead to misleading conclusions.
Because the data (and drivers) associated with that customers
response may be different now than what they were during or
prior to the promotion
What if we can’t manage (store) frozen files for every
promotion?
Alternative to using a frozen file for each test is to pull analyses
datasets from database backups (prior to test).
Chapter 6: The Analysis Sample
‹#›
‹#›
What are test (analysis) and validation samples?
Analyses are based on the test file, and calibrated to see if they
hold up on the validation file
2/3rds for Test, 1/3rd for validation is a common rule of thumb
for splitting the file
The reason for doing this is because a sample may have a
certain level of error associated with them.
The validation sample is used to ensure the analyst does not
make erroneous conclusions based on the error variance
associated with the sample.
If the conclusions hold up on the validation sample, then there’s
more confidence that the conclusions from the analyses are
sound, and will generalize in-market.
We can put all of this together in an “analytic workflow” . . .
6. Chapter 6: The Analysis Sample
‹#›
‹#›
Define test segment
Names pulled and saved from the database
Sample split 2/3 for analysis
Application of analysis findings to the database for roll -out
Customer Database
Validate findings and refine results
Responses matched to create the frozen analysis file
Names sent test promotion
Analysis of responders vs.
non-responders
Sample split 1/3 for validation
“The Frozen Analysis File”
Chapter 6: The Analysis Process
7. ‹#›
‹#›
Analyzing and Manipulating Customer Data
Chapter 7
Key Point
It is critically important to be able to explore and analyze the
data – how info is captured, updated, maintained so that what
we get out of the data accurately reflects the state and processes
of the business
Chapter 7:Analyzing & Managing Customer Data
‹#›
‹#›
What is a univariate tabulation?
Generating a table to summarize the categories of values from a
single variable or field.
Age (example below)
Effect is expressed as an Index to some standard of comparison
Or other variables of interest
Past promotion responses,
Total number of orders ever,
Total number of promos. . .
8. Chapter 7: Analysis
‹#›
‹#›
Statistical Correlation
Section
What are some of the reasons a univariate analyses may not be
the full answer?
Judgments and decisions are made about how to cut / slice the
data for a univariate analyses
Deciding on the population of interest,
How to categorize values in buckets,
Determining a threshold/size criteria,
Etc.
…. What if the judgments or decisions made about the above are
not the best?
… And also because it’s univariate, i.e. there may be much
more information available that could be used
Chapter 7: Analysis
9. ‹#›
‹#›
What is a cross-tabulation analyses?
A way to view two or more data elements in combination – e.g.
total promos x total orders
Chapter 7: Analysis
‹#›
‹#›
How would a decision be different with a cross-tabulation vs. a
univariate analyses?
(look at the univariate distribution of each dimension below, >=
3%)
Chapter 7: Analysis
‹#›
‹#›
10. How would a decision be different with a cross-tabulation vs. a
univariate analyses?
(you could promote to 28.27% of customers rather than 14.09%,
i.e. 2x the volume, where >= 3%)
Chapter 7: Analysis
‹#›
‹#›
What are logic-counter variables?
A logical combination of several data elements that represent or
share a common underlying dimension
Each distinct variable can be combined to form a new single
variable
Oftentimes, you can get more lift out of this new variable, than
when put into models, and/or more stable in coefficients in a
regression equation
Also reduces the amount of data to analyze (e.g. variable
reduction)
Notes and considerations:
It is assumed that you know how individual data elements are
related to each other, and/or how these individual elements map
onto the underlying dimension
It may or may not be the case that each component variable is
weighted equally
11. Chapter 7: Analysis
‹#›
‹#›
Note how the following example provides better information on
who to promote, once logic-counter variables were created:
21% of database names can now be promoted (up from 8.7%)
Source: Optimal Database Marketing, Exhibit 7.7 and 7.8
Customer
Name
Customer
Address
Total $
Paid
…
RRP
TSRR
ERL
ELR
Rock Music
16. LEGEND: RRP=Rock and Roll Party, TSRR=The Soul of Rock
and Roll, ERL=Early Rock Legends, ELR=Easy Listening
Rock, PD=Paid, PNO=Promoted but not ordered, NP=Not
promoted.
Chapter 7: Analysis
‹#›
17. ‹#›
How are ratio variables created and used?
Create a new variable, in which one data element is divided by
another
How are ratio variables related to cross-tabs?
The component data elements need to be continuous measures,
while cross-tabs can work for categorical variables.
Note how the use of a ratio variable can provide more
information than a univariate consideration:
Chapter 7: Analysis
‹#›
‹#›
What are longitudinal variables?
Conceptually similar to a time-series, longitudinal variables
allow marketers to see customer responses over time
When are longitudinal variables most useful?
Predicated on the assumption that “Recency” of a customer’s
transaction matters “most”
When coded well, they are useful for seeing which customers
are becoming “better” or “worse” over time
18. Example
Chapter 7: Analysis
‹#›
‹#›
Is there a relationship between two variables, e.g. Age and
Income?
Can we measure or quantify this relationship? How “much” of
a relationship is there?
Correlation (re-visited)
‹#›
‹#›
24
Age
19. Distribution
Income
Distribution
Perfect Correlation (Positive)
When z-scores exactly match
When scores exactly match
25
Age
Distribution
Income
Distribution
Perfect Correlation (Negative)
When z-scores exactly match, but their signs are reversed
How can we quantify (i.e. measure) an association between 2
variables?
The “intuition”
If exact same score on x, and y, then perfect correlation.
But must the scores on x and y be the same value? (e.g. Age
and Income can be perfectly correlated but not match).
Perfect correlation = exact same place on distribution
But can we measure where you are in a distribution, regardless
of scale or units?
Yes, with z-scores
A perfect correlation is when z-scores match exactly
A perfect negative correlation is when the size of z-scores
20. match, but not the sign
No correlation, or a zero correlation, means the association is
random – i.e. where a value is on one distribution is very unlike
it’s corresponding position on the other distribution.
Neat fact:
The sum of the cross products of pairs of numbers are highest
when matched;
An average of cross products, is a definition of correlation
Correlation (re-visited)
‹#›
‹#›
Linking the intuitive understanding of correlation to the math . .
.
Think of the correlation formula as:
Intuitive translation: Correlation is a measure of the association
between X and Y, calculated by multiplying the position of X
and the position of Y from their respective distributions and
then taking the “average” of these cross products.
Reminder: The Z’s are the conversion of raw data into a
standardized score:
Intuitive translation: Take each value, subtract it from the
average of it’s set of values, then divide by the standard
deviation of those values. Do this for all X’s, and do this for all
Y’s.
By converting to z-scores, the underlying scale of raw values
(e.g. age in years, income in $, etc.) no longer matter. Any
variables can now be compared in terms of “standard deviations
21. away from the mean”, i.e. z-scores.
Neat Fact: The average of these “new” z-scores is equal to 0
(always). And the standard deviation of z-scores is equal to 1
(always).
Footnote: the “mu” and the “sigma” in the z-score formula are
the population mean, and the population standard deviation,
rather than the sample mean and sample standard deviation.
(Refer to your past Stats text on this difference if nec.). The
sample standard deviation involves a division by (n-1), and the
population standard deviation divides by (n). When dealing
with large numbers of records, this difference becomes
immaterial.
Correlation (re-visited)
‹#›
‹#›
Database Language: SQL
Analysis: Statistical Correlation
HW 3
Data Analysis (using Microsoft Excel)
Using Microsoft Excel, enter the Age and Income data (textbook
exhibit 7.17) into a spreadsheet.
22. Using the “intuition” (see slides 25-26) of what a statistical
correlation is, calculate the correlation between Age and
Income. Insert columns to show intermediate steps in
calculating z-scores.
Verify that this is equivalent to the correlation output in Excel’s
Data Analysis routines
Excel has the following function to correlate values in cells
A1:A10 with B1:B10
= CORREL(A1:A10,B1:B10)
If correct, then your calculated answer should match Excel’s
function . . . and you should enjoy the “AHA!” magical
moment of insight experience!
Upload your file to NYU Brightspace
Homework assignment #3.1 (Correlation)
‹#›
‹#›
Introduction to SQL (Structured Query Language)
SQL is the programming language commonly used when
working with Relational Databases.
For this part of the HW assignment we'll use an online lesson
and exercise from Khan Academy to introduce SQL coding.
Starting with the following link, work through the series of
short online modules. The modules are based on the "Design a
Store Database" Project in where you learn to how to create,
populate and query a database.
For this part of the HW, your database will be an
implementation of what you designed last week for HW2 – your
database of student information and job opportunities.
23. The HW submission will be an uploaded screen capture/slide
showing your code, the database schema, and the query results.
https://www.khanacademy.org/computing/computer-
programming/sql/sql-basics/v/welcome-to-sql
Homework assignment #3.2 (SQL)
‹#›
‹#›
Head of Household Age
Number
% of
Sample
Number of
Orders
Response
Rate
Index to
Total
30 and under
1,529
15.29%
67
4.38%
175
31-40
1,775
17.75%
63
3.55%
142
25. Sheet: Sheet7
Sheet: Sheet8
Sheet: Sheet9
Sheet: Sheet10
Sheet: Sheet11
Sheet: Sheet12
Sheet: Sheet13
Sheet: Sheet14
Sheet: Sheet15
Sheet: Sheet16
Tabulations for the title Rock Music of the 80's
Head of Household Age
Number
% of Sample
Number of Orders
Response Rate
Index to Total
30 and under
1529.0
0.1529
67.0
0.04381948986265533
175.27795945062132
31-40
1775.0
0.1775
63.0
0.03549295774647887
141.9718309859155
41-50
1879.0
0.1879
46.0
0.024481106971793506
97.92442788717402
51-60
39. Number of Orders
Response Rate
Index to Total
30 and under
1529.0
0.1529
67.0
0.04381948986265533
1.7527795945062132
31-40
1775.0
0.1775
63.0
0.03549295774647887
1.4197183098591548
41-50
1879.0
0.1879
46.0
0.024481106971793506
0.9792442788717401
51-60
2054.0
0.2054
29.0
0.014118792599805257
0.5647517039922103
61 and over
1785.0
0.1785
18.0
0.010084033613445379
0.40336134453781514
No age info available
978.0
0.0978
40. 27.0
0.027607361963190184
1.1042944785276072
Total
10000.0
1.0
250.0
0.025
1.0
Rock N Roll Party (RRP)
Number
% of Sample
Number of Orders
Response Rate
Index to Total
Promoted & Ordered
877.0
0.0877
51.0
0.05815279361459521
2.3261117445838084
Promoted & Not Ordered
3967.0
0.3967
93.0
0.02344340811696496
0.9377363246785984
Not Promoted
3911.0
0.3911
73.0
0.01866530299156226
0.7466121196624904
Not Available
1245.0
0.1245
57. 2.24%
(57/2,544)
3.03%
(108/3,563)
2.21%
(69/3,125)
2.5%
(250/10,000)
Sheet1Tabulations for the title Rock Music of the 80'sHead of
Household AgeNumber% of SampleNumber of OrdersResponse
RateIndex to Total30 and under1,52915.29%674.38%1.7531-
401,77517.75%633.55%1.4241-501,87918.79%462.45%0.9851-
602,05420.54%291.41%0.5661 and
over1,78517.85%181.01%0.40No age info
available9789.78%272.76%1.10Total10,000100.00%2502.50%1.
00Rock N Roll Party (RRP)Number% of SampleNumber of
OrdersResponse RateIndex to TotalPromoted &
Ordered8778.77%515.82%2.33Promoted & Not
Ordered3,96739.67%932.34%0.94Not
Promoted3,91139.11%731.87%0.75Not
Available1,24512.45%332.65%1.06Total10,000100.00%2502.50
%1.00Total Number of Orders Ever (all Prod. lines)Number% of
SampleNumber of OrdersResponse RateIndex to
Total000.00%00.00%0.001-53,31233.12%621.87%0.756-
103,07430.74%682.21%0.8811-152,20522.05%642.90%1.1615
plus1,40914.09%563.97%1.59Total10,000100.00%2502.50%1.0
0Total Number Promotions Ever (all Prod. lines)Number% of
SampleNumber of OrdersResponse RateIndex to Total1-
500.00%00.00%0.006-107687.68%162.08%0.8311-
202,54425.44%572.24%0.9021-303,56335.63%1083.03%1.2131
plus3,12531.25%692.21%0.88Total10,000100.00%2502.50%1.0
0Total Promotions Ever:Total Orders Ever:1-56-1011-2021-
3031 plusTotal00.00% (0/0)0.00%
(0/0)0.00% (0/0)0.00% (0/0)0.00%
(0/0)0.00% (0/0)1-50.00% (0/0)1.63%
(8/491)1.76% (17/967)2.34% (20/856)1.60%
58. (16/998)1.87% (62/3,312)6-100.00% (0/0)2.89%
(8/277)1.85% (14/756)2.51% (29/1,154)1.80%
(16/887)2.21% (68/3,074)11-150.00% (0/0)0.00%
(0/0)3.03% (14/462)3.03% (29/956)2.67%
(21/787)2.90% (64/2,205)15 plus0.00% (0/0)0.00%
(0/0)3.34% (12/359)5.03% (30/597)3.53%
(16/453)3.97% (56/1,409)Total0.00% (0/0)2.08%
(16/768)2.24% (57/2,544)3.03% (108/3,563)2.21%
(69/3,125)2.5% (250/10,000)Rock Logic: (RRP, TSRR, ERL,
ELR)Number% of SampleNumber of OrdersResponse RateIndex
to W/LPurchased 07,85678.56%1181.50%0.60Purchased
19459.45%475.01%2.00Purchased
26336.33%406.34%2.54Purchased
33653.65%277.44%2.98Purchased
42012.01%188.87%3.55Total10,000100.00%2502.50%1.00Cust
omer Actions to Last Three Promotions Sent (Prom-2, Prom-1,
Prom)Number% of SampleNumber of OrdersResponse
RateIndex to W/L(Pay, Pay, Pay)3563.56%195.34%2.13(NR,
Pay, Pay)4224.22%163.79%1.52(NR, NR,
Pay)5285.28%183.41%1.36(Cancel, Pay,
Pay)4274.27%204.68%1.87(Cancel, Cancel,
Pay)2292.29%93.93%1.57………………………………(NR, NR,
BD)1501.50%21.33%0.53(NR, NR,
NR)2982.98%20.67%0.27Total10,000100.00%2502.50%1.00NR
= Non-Response, BD = Bad Debt
&A
Page &P
Sheet21-56-1011-2021-
3031+TOT00.00%000.00%000.00%000.00%000.00%000.00%00
0.00%001-
50.00%002.24%114912.17%219672.10%188561.80%189982.05
%6833122.01%6532336-
100.00%002.53%72772.38%187562.17%2511542.03%188872.2
1%6830742.54%93365511-
150.00%000.00%003.25%154622.82%279562.80%227872.90%6
422052.86%54188715
63. Total
10000.0
1.0
250.0
0.025
100.0
Fig 7.3
Rock and Roll Party (RRP)
Number
% of Sample
Number of Orders
Response Rate
Index to Total
Promoted & Ordered
877.0
0.0877
51.0
0.05815279361459521
232.61117445838084
Promoted & Not Ordered
3967.0
0.3967
93.0
0.02344340811696496
93.77363246785984
Not Promoted
3911.0
0.3911
73.0
0.01866530299156226
74.66121196624904
Not Available
1245.0
0.1245
33.0
0.02650602409638554
64. 106.02409638554215
Total
10000.0
1.0
250.0
0.025
100.0
Fig 7.4
Total Number of Orders Ever (all product lines)
Number
% of Sample
Number of Orders
Response Rate
Index to Total
0
0.0
0.0
0.0
0.0
0.0
1-5
3312.0
0.3312
62.0
0.018719806763285024
74.8792270531401
6-10
3074.0
0.3074
68.0
0.022121014964216004
88.48405985686401
11-15
2205.0
0.2205
64.0
73. 7
7/55 = 12.73%
J. Jackson
111 Rocky Rd
.
12
2
2/12 = 16.67%
Customer
Name
Ratio of total paid
products ever to total
promotions ever
Two
Promotions
Ago
One
Promotion
Ago
Latest
Promotion
A. Flintstone
.2546
Non-Response
Non-Response
Order and Pay
P. Johnson
.3796
74. Order and Pay
Non-Response
Non-Response
X. Wesley
.1408
Non-Response
Order and Pay
Order and Pay
$0$10,000$20,000$30,000$40,000$50,000$60,000$70,000$80,0
00$90,000$100,000020406080IncomeAge
Scatter Plot of Age vs. Income
Sheet: Chart2
Sheet: Sheet1
Sheet: Sheet2
Sheet: Sheet3
Sheet: Sheet4
Sheet: Sheet5
Sheet: Sheet6
Sheet: Sheet7
Sheet: Sheet8
Sheet: Sheet9
Sheet: Sheet10
Sheet: Sheet11
Sheet: Sheet12
Sheet: Sheet13
Sheet: Sheet14
Sheet: Sheet15
Sheet: Sheet16
59.0
32.0
19.0
22.0
45.0
55.0
47.0
36.0
75. 25.0
51.0
92000.0
43000.0
27000.0
24000.0
41000.0
65000.0
60000.0
62000.0
41000.0
85000.0
Age
Income
“Destined to be the definitive guide to database marketing
applications, analytical
strategies and test design.”
—Brian Kurtz, Executive Vice President, Boardroom Inc., 2000
DMA List
Leader of the Year and DMA Circulation Hall of Fame Inductee
“This book is well written with interesting examples and case
studies that both
illustrate complex techniques and tie the chapters together. The
level of detail and
treatment of statistical tools and methods provides both
understanding and enough
76. detail to begin to use them immediately to target marketing
efforts efficiently and
effectively. It is perfect for a course in database marketing or as
a handy reference
for those in the industry.”
—C. Samuel Craig, New York University, Stern School of
Business
“This book should be studied by all who aspire to have a career
in direct
marketing. It provides a thorough overview of all essential
aspects of using
customer databases to improve direct marketing results. The
material is presented
in a style that renders even the technical subjects
understandable to the novice
direct marketer.”
—Kari Regan, Vice President, Database Marketing Services,
The Reader’s Digest Association
“Finally, practical information on database marketing that
tackles this complex
subject but makes it clear enough for the novice to understand.
This book serves as
more than a primer for any senior manager who needs to know
77. the whole story. As
one who has spent over 20 years of his career involved in
publishing and database
marketing, I have a real appreciation for how difficult it is to
explain the finer
points of this discipline, while keeping it understandable. This
book does that
admirably. Well done!”
—Patrick E. Kenny, Executive Vice President, Qiosk.com
“This book is especially effective in describing the breadth and
impact of the data-
base marketing field. I highly recommend this book to anyone
who has anything to
do with database marketing!
—Naomi Bernstein, Vice President, BMG Direct
“Ron Drozdenko and Perry Drake have written a guide to
database marketing that
is thorough and that covers the subject in considerable depth. It
presents both the
concepts underlying database marketing efforts and the all -
important quantitative
reasoning behind it. The material is accessible to students and
practitioners alike
78. and will be an important contribution to improved
understanding of this important
marketing discipline.”
—Mary Lou Roberts, Boston University and
author of Direct Marketing Management
Drozdenko-FM 2/26/02 6:02 PM Page i
“I think it is a terrific database marketing book, it’s got it all in
clear and logical
steps. The benefit to the marketing student and professional is
that complex data-
base concepts are carefully developed and thoroughly explained.
This book is a
must for all marketing managers in understanding database
issues to successfully
manage and structure marketing programs and achieve
maximum results.”
—Dante Cirilli, DMEF Board Member and
Retired President, Grolier Direct Marketing
“An excellent book on the principles of Direct Marketing and
utilization of the cus-
tomer database to maximize profits. It is one of the best direct
marketing books I
79. have seen in years in that it is broad with specific examples. I
am going to require
new hires to read this (book) to get a better understanding of the
techniques used in
Database Marketing.”
—Peter Mueller, Assistant Vice President of Analysis,
Scholastic, Grolier Division
“This is an amazingly useful book for direct marketers on how
to organize and ana-
lyze database information. It’s full of practical examples that
make the technical
material easy to understand and apply by yourself. I strongly
recommend this book
to direct and interactive marketers who want to be able to
perform professional
database analyses themselves, or be better equipped to review
the work of analysts.”
—Pierre A. Passavant, Professor of Direct Marketing, Mercy
College and
Past Director, Center for Direct Marketing, New York
University
“The most useful database marketing reference guide published
today. The authors
do an excellent job of laying out all the steps required to plan
80. and implement an
effective database marketing strategy in a clear and concise
manner. A must have for
academics, marketing managers and business executives.”
—Dave Heneberry, Director, Direct Marketing Certificate
programs,
Western Connecticut State University, and
Past Chair, Direct Marketing Association
“This book is essential for all direct marketers. It serves as a
great introduction to
the technical and statistical side of database marketing. It
provides the reader with
enough information on database marketing and statistics to
effectively apply the
techniques discussed or manage others in the environment.”
—Richard Hochhauser, President, Harte-Hanks Direct
Marketing
Drozdenko-FM 2/26/02 6:02 PM Page ii
Drozdenko-FM 2/26/02 6:02 PM Page iii
Drozdenko-FM 2/26/02 6:02 PM Page iv
82. Drozdenko, Ronald G.
Optimal database maketing: strategy, development, and data
mining/
by Ronald G. Drozdenko and Perry D. Drake.
p. cm.
Includes bibliographical references and index.
ISBN 0-7619-2357-8
1. Database marketing. 2. Electronic commerce.
I. Drake, Perry D. II. Title.
HF5415.126 .D76 2002
658.8’4-dc20 2001005596
This book is printed on acid-free paper.
02 03 04 05 10 9 8 7 6 5 4 3 2 1
Acquisitions Editor: Marquita Flemming
Editorial Assistant: MaryAnn Vail
Production Editor: Claudia A. Hoffman
Indexer: Molly Hall
Cover Designer: Michelle Lee
Drozdenko-FM 2/26/02 6:02 PM Page vi
Contents
Preface xv
Supplemental Material xvii
Acknowledgments xix
Foreword xxi
1. Introduction to Database Marketing Concepts 1
83. What Is a Marketing Database? 3
Trends Leading to the Use of Databases in Marketing 4
Market Segmentation 5
Emphasis on Service and CRM 6
Changes in Media 7
Changes in Distribution Structure and Power 8
Lifestyle and Demographic Trends 9
Accountability for Marketing Actions 10
Integration of Business Functions 11
Technological Advances 11
More Informed Customers 12
Database Marketing Versus Aggregate Marketing 13
Advantages of Database Marketing 15
Disadvantages of Database Marketing 16
Cost Issues 16
Global Markets 17
Competition From Traditional Retailer 17
Negative Perceptions 18
Framework for This Book 18
Chapter Summary 19
Review Questions 19
2. Stategic Database Development in the Marketing
Planning Process 21
Computerized Databases 23
Customer Databases Versus Other Marketing Databases 24
The Need for Strategic Planning 25
Developing a Systematic Plan for Using Marketing
Databases 26
Drozdenko-FM 2/26/02 6:02 PM Page vii
84. The Marketing Planning Process and Database
Implementation 27
Situational Analysis 28
Establishing Marketing Objectives 29
Strategy Development 30
Strategy Development, Targeting, and Product
Positioning 32
Marketing Programs 33
Distribution 34
Promotion 34
Price 35
Product 35
Monitor and Control 36
Databases and the Planning Process 36
Chapter Summary 37
Review Questions 38
3. Defining Customer Data Requirements 39
Data Needs Determination 40
Fulfillment, Marketing, and Prospecting Databases 41
Data Residing on the Marketing Database 41
Internal or House Data 41
Fulfillment Data 43
Marketing Data 43
Customer Contact Data 44
External or Enhancement Data 45
Compiled List Data 45
Census Data 47
85. Modeled Data 49
Lists Versus Data 49
Applying and Using Enhancement Data 51
Chapter Summary 53
Review Questions 53
4. Database Maintenance and Coding 55
Standard Database Maintenance Routines 56
Deduping the Customer File 57
Householding the Customer File 58
Purging Old Customer Records 59
Changing Contact Information 59
Standardizing Addresses 61
Removing Names From Databases at Consumer Request 61
Identifying Customers With Match Coding 62
Merge/Purge Processing 64
Coding Source and Promotional Offers 65
Salting Files and Decoy Records 66
Identifying Credit Risks and Frauds 67
Field Updating Rules 67
Drozdenko-FM 2/26/02 6:02 PM Page viii
Reporting Summary/Aggregate-Level Information 68
Database Storage and Security 70
Database Maintenance Schedules 72
Some Technical Aspects of Database Maintenance 73
Chapter Summary 73
Review Questions 73
5. Basic Database Technology, Organizational
Considerations, and Database Planning 75
86. Computer Hardware and Software 77
Database Hardware 78
Midrange Computers 79
PCs 79
Hardware Decision Factors 80
Database Software 80
Database System Organization 83
Structured Databases 83
Relational Databases 85
Comparison of Structured and Relational Databases 85
Structured Query Language (SQL) and Data Analysis 85
Organizational Considerations in Technical Database Design 86
Outsourcing: The Process to Select a Database Provider 88
Phases of Database Development 91
Comments on Technological Development of the
Database 92
Chapter Summary 93
Review Questions 93
6. The Analysis Sample 95
How We Sample 96
Representative Samples 96
Random Samples 97
Sample Usage 97
Creation of the Analysis Sample 98
Methods of Saving Point-in-Time Sample Data 100
Analysis and Validation Samples 101
Application of Analysis Findings 101
87. Chapter Summary 101
Review Questions 102
7. Analyzing and Manipulating Customer Data 103
Getting to Know Your Data 104
The Analysis 105
Univariate Tabulations 106
Cross-Tabulations 111
Logic Counter Variables 113
Ratio Variables 116
Longitudinal Variables 117
Time Alignment of Key Events 119
Drozdenko-FM 2/26/02 6:02 PM Page ix
Reducing the Amount of Customer Data to
a Manageable Set via Correlation Analysis 120
Statistical Background—Correlation Analysis 123
Chapter Summary 127
Review Questions 128
Notes 128
8. Segmenting the Customer Database 129
Defining Your Segmentation Objective 130
Segmentation Schemes 134
Segmentation for Promotional Product Offerings 134
Corporate-Level Segmentation 134
Product Line-Specific Segmentation 135
Segmentation for Life-Stage Marketing and Research 136
88. Segmentation Techniques 138
Univariate and Cross-Tabulation Analysis 138
Formal RFM Analysis 145
CHAID Analysis 148
Factor and Cluster Analysis 153
Factor Analysis 153
Cluster Analysis 158
Issues to Consider Regarding Segmentation
Implementation 163
Promotional Intensity 163
Too Many Products 164
Cannibalism 164
Overgeneralization 165
Ethical and Public Policy Issues 165
Chapter Summary 166
Review Questions 166
Note 167
9. An Introduction to Simple Linear Regression Modeling 169
The Simple Linear Regression Model 170
The Coefficient of Determination 174
Statistical Background—Simple Linear Regression Analysis 176
Chapter Summary 179
Review Questions 179
10. Multiple Regression Modeling 181
Defining Your Marketing Objective 182
Preparing the Data to Build the Multiple
Regression Model 184
The Multiple Regression Model 187
89. Model Interpretation 187
Assumptions of the Model 192
Multicollinearity 192
Other Properties 193
A Note on Modeling Binary Response Data 193
Drozdenko-FM 2/26/02 6:02 PM Page x
Regression Diagnostics 194
Examining the Model for Indications of
Multicollinearity 195
Examining the Model for Variable Significance 197
Multiple “Logistic” Regression Models 199
Sample Composition 200
Outside List Modeling Options 201
Response Models 202
Clone or Best Customer Models 202
Stepwise Regression Models 205
Neural Networks 206
Data Mining, Tools, and Software 207
Ensuring That Your Model Holds Up in Rollout 213
Chapter Summary 215
Review Questions 215
Notes 216
11. Gains Charts and Expected Profit Calculations 217
The Response Gains Chart 218
Options When Lacking Validation Samples 223
90. Historical Gains Falloff Chart 223
Bootstrapping 225
Expected Profit Calculations 226
Reconciling Gains 231
Chapter Summary 233
Review Questions 233
12. Strategic Reporting and Analysis 235
Key Active Customer Counts 236
List Vitality Customer Statistics 238
Key List Segment Counts and Statistics 238
Calculating LTV 239
LTV Methodologies 240
LTV Profiles 241
Actual and Aggregate LTV Calculations 243
Calculating the Discount Rate and NPV 244
Sample Types Used in LTV Calculations 247
Forecasting LTV 248
Impact Studies 248
Monitoring Promotional Intensity 249
Chapter Summary 250
Review Questions 250
13. Assessing Marketing Test Results 251
Confidence Interval Calculations 252
Confidence Interval Estimation for a Sample Mean 253
Confidence Interval Estimation for a Sample Proportion 256
Confidence Interval Estimation for the Difference
Between Two Sample Means 258
91. Drozdenko-FM 2/26/02 6:02 PM Page xi
Confidence Interval Estimation for the Difference
Between Two Sample Proportions 260
Setting the Confidence Level 263
Single Sample Measures 264
Difference Between Two Sample Measures 264
Making a Business Decision Based on the
Confidence Interval 266
Single Sample Measures 266
Difference Between Two Sample Measures 267
Hypothesis Tests for Significance 268
Establishing the Hypothesis 269
Setting the Error Rate of the Hypothesis Test 269
Establishing the Direction of the Hypothesis Test 270
Hypothesis Test for the Difference Between
Two SampleMeans 270
Hypothesis Test for the Difference Between
Two Sample Proportions 275
Setting the Confidence Level of
Hypothesis Tests for Significance 279
Making a Business Decision Based on
Hypothesis Tests for Significance 279
P Value of the Hypothesis Test for Significance 279
Conducting Hypothesis Tests for Significance
92. Using Confidence Intervals 280
Gross Versus Net 281
Multiple Comparisons 281
Calculating Breakeven 282
Response Rate Required to Break Even 283
Increase in Response Rate Required to Break Even 283
Facts Regarding Confidence Intervals and
Hypothesis Test Results 284
Marketing Test Analysis Software 285
Chapter Summary 285
Review Questions 286
14. Planning and Designing Marketing Tests 287
Marketing Test Design Considerations 288
Rule 1: For Mailers, Include the Control Package
in the Test Plan 288
Rule 2: Reverse Test Package Changes 288
Rule 3: Test One Change at a Time 289
Rule 4: Test for Only Meaningful
Package Element Interactions 290
Rule 5: Define the Universe for Testing Carefully 291
Outside List Test Design Considerations 292
Sample Size Considerations 294
Sample Size Determination for a Sample Mean 295
Sample Size Determination for a Sample Proportion 298
Drozdenko-FM 2/26/02 6:02 PM Page xii
93. Sample Size Determination for the
Difference Between Two Sample Proportions 301
Marketing Test Planning Software 304
Alternative Testing Approaches for
Small Direct Marketers 304
Chapter Summary 308
Review Questions 308
15. Marketing Databases and the Internet 309
Database Integration 310
Growth in Internet Commerce 312
The Internet Versus Other Database Marketing Media 314
Limitations of Internet Marketing 317
Personalization: The Great Promise of the Internet 319
E-Mail Marketing 321
E-Mail Applications 321
E-Mail Formats 322
Chapter Summary 324
Review Questions 325
16. Analyzing and Targeting Online Customers 327
Data Collected via the Internet 327
Registration Data 328
Behavior Data 330
Source Data 332
Understanding Internet Users and Online Buyers 332
Web Site Reporting 334
Driving Customers to Your Web Site 337
Targeting Online Customers 341
94. Conducting Marketing Tests in the
E-Commerce World 345
Banner Ads 345
E-Mail 347
Chapter Summary 348
Review Questions 348
17. Issues in the Marketing Environment and
Future Trends in Marketing Databases 349
The Global Business Environment 351
Social Concerns and Ethics in Database Marketing 357
Industry Organizations 360
Evolution and Trends in Database Marketing 362
Consumer Databases and the Internet 362
B-to-B Databases 364
Not-for-Profit Databases 364
Retailer Databases 365
Service Organization Databases 365
Chapter Summary 366
Review Questions 366
Drozdenko-FM 2/26/02 6:02 PM Page xiii
Glossary 369
Additional Readings in Database and Direct Marketing 381
References 383
Name Index 387
Subject Index 389
95. About the Authors 397
Drozdenko-FM 2/26/02 6:02 PM Page xiv
Preface ___________________________________________
xv
Contemporary direct marketing and e-commerce companies
cannot existin today’s competitive environment without the use
of marketing
databases. Databases allow marketers to reach customers and
cultivate
relationships more effectively and efficiently. Although
databases provide
a means to establish and enhance relationships, they can also be
used
incorrectly, inefficiently, and unethically. Our goal in this book
is to provide
the reader with a complete and solid understanding of how to
properly
establish and use databases to help organizations maximize their
relation-
ships with customers. In fact, we have not found any other book
on the
market today that contains the level of detail regarding database
marketing
applications that this one has.
We have been involved in the direct marketing industry and
academia for
many years. Ron Drozdenko teaches Strategic Marketing
Databases and
Interactive Marketing Management and has been a consultant to
96. many firms
over the course of his career. He is currently the Chair of the
Marketing
Department at Western Connecticut State University. Perry
Drake is an inde-
pendent database marketing consultant and adjunct faculty
member of New
York University, where he teaches Statistics for Direct
Marketers, Database
Modeling, and Advanced Database Modeling in the Direct
Marketing
Master’s Degree program.
In teaching such topics to students, we both have found little
material to
draw upon. As such, we were required to create our own content
from our
industry experience, help from peers, and published case
studies. Several
excellent books have been written on the topic of direct
marketing.
However, many of those books delve only into areas such as
copywriting
and media selection and place less emphasis on database
marketing appli-
cations from a marketer’s perspective. Our intent with this book
is to focus
on the marketing database and take readers systematically
through the
process of database strategy, development, and analysis.
We originally met each other in the summer of 1997 when we
were
approached by the Direct Marketing Educational Foundation
(DMEF) to
develop a database marketing course. The database course is
97. one of a series
Drozdenko-FM 2/26/02 6:02 PM Page xv
of undergraduate courses in direct marketing the DMEF
developed in col-
laboration with the Marketing Department of the Ancell School
of Business
at Western Connecticut State University. (You can contact the
DMEF to
obtain more information about these courses.)
Our target audience for this book is both students and
practitioners:
upper-level undergraduates, graduate students in an MBA
program, and
entry- and middle-level direct marketers. In addition, database
analysts and
statisticians fairly new to the field of direct marketing will find
the book
useful. It will provide a complete overview of the analytical
applications in
the field of direct marketing. Direct marketing executives will
also find the
strategic elements of the book helpful for business planning.
xvi OPTIMAL DATABASE MARKETING
Drozdenko-FM 2/26/02 6:02 PM Page xvi
xvii
98. Supplemental Material
Academic adopters of this book have access to the following
supportmaterial from the authors:
♦ A comprehensive collection of PowerPoint slides for each
chapter
♦ Sample exercises and solutions for each chapter
♦ Sample syllabi and course organization
♦ Sample exams and quizzes
♦ Sample marketing databases for case study work given in
various
formats (delimited text files, SAS, Excel, SPSS, etc.).
Drozdenko-FM 2/26/02 6:02 PM Page xvii
Drozdenko-FM 2/26/02 6:02 PM Page xviii
xix
Acknowledgments
Anumber of people contributed directly or indirectly to the
developmentof this book. Because the book evolved from an
outline developed by
an advisory board established by the DMEF, the members of the
board
deserve acknowledgment. In particular, Richard Montesi and
Laurie Spar
were instrumental in establishing the board and organizing the
meetings.
David Henneberry and Dante Cirilli worked with us to establish
99. the origi-
nal outline that served as the basis for the database marketing
course and
eventually this book. The extensive backgrounds of Dave and
Dan in the
direct marketing industry ensure that the book’s foundation is
solid.
We also wish to acknowledge the indirect contributions of a
number of
small and large professional associations in the direct marketing
community
that allowed us to examine database marketing in application. In
particular,
The Reader’s Digest Association and Grolier Direct Marketing
influenced
our perspectives on database marketing.
One of our primary reasons for writing this book is to provide
market-
ing students with a good foundation in database strategy,
development, and
analysis. Therefore, the feedback we obtained from our students
at the
Ancell School of Business at Western Connecticut State
University and New
York University was especially valuable in translating course
materials into
a coherent book. Students also read drafts of sections of the
book and
offered several cogent suggestions. In particular, Perry gives
special thanks
to some of his past students at NYU—Joe Bello, Janelle
Bowleg, Eric
Chism, Dean Krispin, Steve LaScala, and Bob Wiener—for their
tremen-
100. dous efforts in ensuring that the book content was complete,
consistent,
and understandable.
Industry reviewers of chapters of this book deserve our
gratitude. In
particular, Perry personally thanks, first and foremost, Pierre
Passavant,
the previous director of the NYU Direct Marketing Master’s
Degree pro-
gram, for his support and the many opportunities he provided.
Secondly, we
thank Gary Coles for his significant review of Chapter 10, Rich
Lawsky
for his significant review of Chapter 4, and Elizabeth
Colquhoun for her
review of all the chapters for clarity. In addition, we thank
Craig Ceire,
Drozdenko-FM 2/26/02 6:02 PM Page xix
Mary-Elizabeth Eddlestone, Mary Halloran, Patrick Hanrahan,
Jim Tucker,
Henry Weinberger, and Pat Zamora for their individual
contributions deal-
ing with specific topics. We are also indebted to the following
reviewers of
the completed manuscript. Their comments and suggestions
have helped
improve the final version of the book.
Naomi Bernstein
Dante Cirilli
101. C. Samuel Craig
David Heneberry
Richard Hochhauser
Patrick E. Kenny
Brian Kurtz
Peter C. Mueller
Pierre Passavant
Kari Regan
Mary Lou Roberts
Thanks to the team at Sage Publications, including Marquita
Flemming,
MaryAnn Vail, and our copy editor, Barbara Coster, for their
support and
guidance.
Last, but certainly not least, our appreciation goes to our
families. In
addition to lending moral support, some family members
provided direct
contributions to the development of the book. Rita Drozdenko,
Ron’s wife,
read several chapters and provided feedback from the
perspective of a
novice to the field. Rhonda Knehans Drake, Perry’s wife and an
accom-
plished database marketing consultant, made a significant
102. contribution to
the book. Rhonda wrote Chapter 16, “Analyzing and Targeting
Online
Customers,” and also provided professional critiques of other
chapters.
Tarry Drake-Schaffner, Perry’s sister, an avid book reader and
bookstore
owner, spent a tremendous amount of time editing and rewriting
all the
technically oriented chapters. Words cannot express the thanks
that Perry
has for her invaluable input in ensuring that complex topics
could be under-
stood by a beginner. As a novice to the field herself, this was
not an easy
task for Tarry, especially given the tight deadlines. Thank you,
Tarry, for
your tremendous efforts.
—Ronald G. Drozdenko
—Perry D. Drake
xx OPTIMAL DATABASE MARKETING
Drozdenko-FM 2/26/02 6:02 PM Page xx
xxi
Foreword
With about 70 years of management and teaching experience
betweenus, we know a superior training/reference book when we
read it. This
is a great one. We aren’t surprised, because we both confer with
103. the authors
about the database issues of our own seminars, classes, and
clients.
If you have a database, chances are you have database questions
and
issues: How good are the data? How complete? Are you
capturing the right
data? Are you using the data to the maximum advantage? Will
investments
in new system enhancements pay out? What steps must be
followed when
considering to outsource your database? What issues must be
considered
when examining data mining tools? How do you learn sound
database
management practices? How do you teach them? How do you
provide intel-
ligent leadership to database management departments that
report to you?
The answers require a thoughtful examination of what is
needed, how to
capture it, at what cost—and some knowledge of statistics that
most of us
don’t have. You won’t find a better resource than this book. It
covers all
aspects of database marketing, including database design,
maintenance,
data usage, test design, and data analysis. In all these areas, the
focus is on
how to best utilize the database to optimize marketing efforts.
Important
current issues such as e-commerce, ethics, privacy, and
globalization are
also covered.
104. Coauthor Ronald G. Drozdenko, Ph.D., Professor and Chair of
the
Marketing Department at the Ancell School of Business,
Western
Connecticut State University, teaches Strategic Marketing
Databases and
Interactive/Direct Marketing Management. He was a member of
the advis-
ory board established by the Direct Marketing Educational
Foundation to
develop a model curriculum for direct/interactive marketing.
This model
program is currently being offered at the Ancell School. In his
role as fac-
ulty adviser for student interns, Ron uses company feedback to
enrich the
program with continuing real-life applications. Ron has also
accumulated
more than 20 years of applied marketing experience. Both his
academic
and applied marketing backgrounds are reflected in the
approach taken in
this book.
Drozdenko-FM 2/26/02 6:02 PM Page xxi
Coauthor Perry D. Drake is currently a database marketing
consultant
and faculty member in the Master’s of Science in Direct
Marketing program
at New York University. Prior to this, Perry had spent over 10
years in vari-
ous database marketing roles at The Reader’s Digest
105. Association, most
recently as the director of a special division within the
Marketing Services
group. During Perry’s first year at NYU, word got around that
he had a
remarkable ability to make topics such as statistics and database
modeling
and regression understandable and interesting. In recognition of
his abil-
ities, he won the first Outstanding Master’s Faculty Award.
Perry’s excep-
tional teaching skills are very evident in the chapters of this
book.
The book that Ron and Perry have written tracks a character,
Keri Lee,
as she resolves data and database issues at every step in her
advancement
through the ranks, first in a technology agency servicing clients
and later as
a senior manager in a large publishing company. Her reasoning
and her
solutions to data problems of increasing complexity demonstrate
the
methodology of database management in all its statistics-driven
splendor.
Go as deeply as you need for your purpose. The practical
wisdom and con-
crete examples make it an ideal resource for business managers,
instructors,
trainers, and students.
If you are a business manager, this book will help you oversee
the vari-
ous specialists you must work with to implement a database
marketing
106. strategy. If you are an instructor, trainer, or student, it will give
you a clear
picture of what actually happens in the real world of business
and specific
techniques used by business professionals. Keep the book at
hand to resolve
your next database dilemma.
—David Heneberry
Director, Direct Marketing Certificate Programs,
Ancell School of Business, Western Connecticut State
University
—Pierre Passavant
Professor of Direct Marketing, Mercy College, Westchester,
New York
xxii OPTIMAL DATABASE MARKETING
Drozdenko-FM 2/26/02 6:02 PM Page xxii
Introduction to Database
Marketing Concepts
1
1
It’s 7:15 p.m., Keri Lee, a 29-year-old account executive for a
technology
company, stops at a supermarket in Southbury, Connecticut, on
her way
home from work. After picking up Diet Pepsi, a few tomatoes,
lettuce, and
107. a package of Swiss cheese, she goes to the express checkout
line. Keri hands
her store card to the clerk, who scans it prior to processing her
order. Using
the store card allows her to get a discount on the cheese. With
her sales
receipt, she also gets a $0.40 coupon for Ritz Crackers. The bill
came to
$6.20. Keri paid with her VISA card.
Keri picks up her mail before going into the house. There are
catalogs from
Bloomingdales, L. L. Bean, Macy’s, and Pottery Barn. She puts
the Pottery
Barn catalog to the side. Her sister’s birthday is in two weeks
and the items
in the catalog are consistent with her sister’s decorating style.
In addition to the electric and VISA bills, she has a letter from
the
Volkswagen dealership thanking her for her recent purchase and
a letter
from the American Red Cross. Remembering that the Red Cross
recently
helped her friend who was caught in a flood, she makes a
contribution by
checking a box and entering her VISA number.
Keri also got the new issues of Smart Business, Business Week,
and Self
magazines. An ad in Business Week about a technology
conference attracts
her attention, and she fills out an attached response card
requesting more
information.
108. After dinner, she receives a phone call from an insurance
company. At first,
she is irritated by the call. She then remembers that her car
insurance rates
increased substantially since she leased her new Volkswagen
and asks the
person on the phone for a quote. Later that evening, she goes on
the Internet
to look for other insurance companies and requests three more
quotes online.
Drozdenko01 2/26/02 6:03 PM Page 1
2 OPTIMAL DATABASE MARKETING
Browsing the Web, she remembers that she has almost finished
the book
she has been reading and goes to Amazon.com. The Amazon
page provides
her with suggestions based on her previous purchase, A Certain
Justice, by
P. D. James. A new mystery by Elizabeth George is on the
suggestion list.
It can be shipped within 24 hours. Keri places the book into the
Shopping
Cart and uses 1-Click to check out.
Before leaving the Amazon site, she clicks on the Music tab and
searches for
Sarah McLachlan. She heard a new single by McLachlan on the
radio and
was curious about the other songs on the CD. Keri listens to
five cuts from
McLachlan’s new CD but decides not to order yet.
109. At 10:00 p.m., she scans through the channels on TV and pauses
at QVC
when a bracelet grabs her attention. Calling QVC, she gives her
account
number that she used 2 months ago when she purchased a color
printer. In
less than 1 minute, the bracelet is ordered and she returns to
scanning the
channels.
Keri’s daily routine is similar to the routines of millions of
other peoplein the United States and other countries. These
transactions provide us
with the goods and services that are a part of our lives. In the
scenario
above, databases underlie all the transactions that Keri made.
They underlie
the purchases in the grocery store, catalogs, TV shopping,
Internet, tele-
marketing, and the charitable contributions. Databases are a
collection of
information related to a particular subject or purpose that are
usually
maintained on a computer for easy search, retrieval, and
analysis. Although
databases are not new, they are becoming an essential element
of marketing.
Organizations in consumer products, business-to-business (b-to-
b), char-
ities, health care, politics, media, investments, government,
insurance, and
so on are finding marketing databases essential to their survival
and
success. In addition, because technology has become more
accessible, small
110. businesses are finding the use of databases a cost-effective way
to stay in
touch with their customers.
Several changes in the business, social, and technological
environments
have led to the widespread use of databases in marketing.
However, the
one underlying reason for the adoption of databases is that they
allow
marketers to use information about individual customers to
reach those
customers and cultivate relationships more effectively and
efficiently.
Databases provide a means to establish and enhance
relationships,
but they can also be used incorrectly, inefficiently, and
unethically. Organ-
izations can use databases to help customers make shopping
easier and
make better purchase decisions, or they can use databases to
intrude into
people’s lives. Good marketers know that maintaining customer
satisfac-
tion is the key to long-term success, and using a database to
flood people
Drozdenko01 2/26/02 6:03 PM Page 2
with unwanted promotional materials is not only wasteful but is
unlikely
to build productive long-term relationships with customers. One
of the
111. goals of this book is to look beyond the temptation of the quick
sale and
consider the long-term impact of database marketing techniques
on the
organization, customers, prospective customers, and society in
general.
This first chapter introduces marketing database concepts. We
begin by
defining marketing databases and examining the environmental
trends
that help to explain why the use of marketing databases is
growing so
rapidly. Because one of the principal uses of marketing
databases is in
direct/interactive marketing, we examine this type of marketing,
compare
it to marketing through conventional retail channels, and briefly
explore
its advantages and disadvantages. We conclude the chapter by
providing
a framework for the concepts and techniques covered in this
book.
Introduction to Concepts 3
___________________________ What Is a Marketing
Database?
A marketing database is a file containing information about
individual
customers or potential customers that is relevant to the
marketing process.
This file can be simple or sophisticated. For centuries,
businesspeople
recorded customer information on slips of paper or in
112. notebooks. Some
organizations still use these manual databases. In contrast,
companies
like American Express and Macy’s have computer databases
that contain
millions of names. Some of these names have hundreds of
pieces of
information.
Whether the database uses simple or sophisticated technologies,
the
purpose is the same—to gain a better understanding of
customers in order
to increase the customer’s satisfaction and the organization’s
objectives.
Although some organizations still use paper databases, the focus
of this
book is on computer databases and how data about customers
are stored,
manipulated, and analyzed on a computer.
Where do the data about customers come from? Companies
build mar-
keting databases from a number of sources. Like Keri Lee, if
you receive
product offers in the mail, such as a catalog, you are on a
database. Your
name could have been added to the database from a telephone
book list,
a membership list, or lists of public notices (like a home
purchase). When
you respond to an offer for a product, your name usually gets
added to
another database.
There are companies that specialize in gathering and renting
113. lists of
customers. You might be surprised at the range and diversity of
available
lists. For example, as a marketer, you could rent lists of female
corporate
decision makers, residential pool owners, neuroscientists,
serious collectors
of plates, coins, and stamps, and people who have contributed to
humani-
tarian causes. In deciding on which list to select, you would
match the
Drozdenko01 2/26/02 6:03 PM Page 3
characteristics of these lists to your target market. Often the
lists include
detailed demographic and psychographic data.
4 OPTIMAL DATABASE MARKETING
Trends Leading to the Use of Databases in
Marketing__________
At this point, we define some of the terms we use in the book.
Note that
there is ambiguity in the literature on how these terms are used;
therefore,
our definitions may not correspond with all others in the field.
We have
already defined a marketing database as a file containing
information
about individual customers or potential customers that is
relevant to the
marketing process. Database marketing refers to marketing
114. activities (e.g.,
selecting prospective customers) that utilize a marketing
database. The
term direct marketing is often used interchangeably with
database market-
ing. In this book, we discriminate between database marketing
and direct
marketing on the basis of the marketing activities. We view
direct market-
ing as a broader term that includes other activities such as
development of
offers and advertisements that are indirectly related to the
database.
The Direct Marketing Association’s (DMA) definition takes this
broader
view of direct marketing:
Direct Marketing is an interactive system of marketing that uses
one
or more advertising media to effect a measurable response
and/or
transaction at any location, with this activity stored on
database.
This definition implies that the marketer is obtaining specific
informa-
tion about the customer. Each time a customer orders a product
or requests
literature, this response information is recorded on the database,
allowing
the marketer to determine the effectiveness of specific
marketing programs
such as mailings or Internet promotions.
When we use the term database marketing, our focus is on the
strategy,
115. development, and analysis of the database for marketing
purposes rather
than on the broader range of activities implied in the term direct
market-
ing. In addition, companies that are involved in personal selling
also use
database marketing, and personal selling is often not considered
a part of
direct marketing.
The term interactive marketing is often used interchangeably
with direct
marketing. However, interactive marketing sometimes refers
only to Internet
marketing. In this book, we use the terms interactive marketing
and direct
marketing interchangeably.
Direct marketing has been increasing at a rapid rate. According
to the
DMA (2000), direct marketing sales revenues are expected to
increase by
9.6% from 2000 to 2005. This increase is greater than the
expected increase
of 5.4% in total U.S. sales during that same period. U.S. sales
revenue attri-
Drozdenko01 2/26/02 6:03 PM Page 4
butable to direct marketing is estimated to reach $1.7 trillion in
2000 and
grow to $2.7 trillion in 2005.
A readers’ survey conducted by Direct magazine (Levey, 2001)
116. shows data-
base investment continuing to grow. About 48% of the
respondents indicated
that their company planned to increase database
development/maintenance
budgets in 2002. They indicated that the databases w ere used
for a variety of
purposes, including (in descending order) promotion, cross-
selling products,
customized offers, profiling customers, providing information
to the direct
sales staff, upselling products, supporting the telemarketing
staff, personaliz-
ing offers, modeling customers, obtaining revenues from the
sales of names,
and performing regression analysis.
Why is database marketing increasing so rapidly? Several trends
in the
marketplace may provide insights. These trends include the
following and
are discussed below.
♦ Greater use of market segmentation
♦ Emphasis on service and customer relationship management
(CRM)
♦ Changes in media
♦ Changes in distribution structure and power
♦ Lifestyle and demographic trends
♦ Accountability for marketing actions
♦ Integration of business functions
♦ Technological advances
♦ More informed customers
Market Segmentation
117. Market segmentation means dividing a market into smaller
pieces based
on demographic, psychographic, or behavioral (purchase)
patterns. The
marketer takes a diverse (heterogeneous) market and attempts to
find
similar (homogenous) groups of people or organizations.
Because of intense
competition and diverse customer needs, marketers have to
develop
products and marketing plans that are responsive to more
specific groups
of customers. It is almost impossible to find a market that has
not been
segmented. The automobile market, for example, is extensively
segmented
by a number of variables such as age, gender, income level,
personality,
task situation (e.g., weather conditions, off-road), lifestyle, and
activities,
interests, and opinions (AIO).
For example, Polk, a company offering database management
and analy-
sis services (“Full-Size Sport Utility Market,” 1997),
determined that there
are differences between domestic and import sport utility
vehicle (SUV)
owners. Seventy-nine percent of Tahoe, Suburban, Yukon, and
Ford
Expedition owners are interested in boating and sailing, home
workshop,
Introduction to Concepts 5
Drozdenko01 2/26/02 6:03 PM Page 5
118. 6 OPTIMAL DATABASE MARKETING
camping and hiking, hunting and shooting, crafts, domestic
travel, and fish-
ing, compared to 40% of Land Rover, Land Cruiser, and Lexus
LX450
owners. On the other hand, import owners (36%) are more
interested in
cultural arts and events, tennis, fashion, wines, and foreign
travel, com-
pared to domestic full-size SUV owners (23%). “Our analysis
paints a clear
picture of the differences between domestic and import full -size
SUV
owners,” said Glenn Forbes, Polk’s vice president of
transportation. “A
domestic SUV is more likely to be found with a deer strapped to
its hood,
while an evening at the theater might be a prime time to spot
import full-
size SUVs.” Polk also segmented this market by demographic
characteristics
such as income and geography.
By specifically targeting people who share these defined
characteristics,
a marketer can increase the probability of reaching potential
customers. A
database that can be segmented according to these target
characteristics can
be a valuable marketing tool.
Emphasis on Service and CRM
119. Service offerings such as banking, airlines, and insurance have
grown at
a greater rate than more tangible categories of goods such as
grocery
items and household appliances. Services are estimated to
represent
about three quarters of the U.S. gross domestic product and
nearly 80%
of all jobs. In addition, the service element of products is
becoming a
more important aspect of the overall product. In the b-to-b
market, for
example, buyers are increasingly less concerned with a
product’s tangible
features and technical specifications and more concerned with
whether
the product meets their needs. For this reason, companies like
IBM now
stress “solutions” rather than “boxes.” IBM’s Web site pr ovides
exam-
ples from different industries and business applications to guide
customers and potential customers through sample solutions to
business
problems. IBM is concerned about selling the right combination
of
software, hardware, consulting, and ongoing support that
achieves their
clients’ objectives.
A database allows customers’ needs to be precisely documented
and
tracked. The increased emphasis on CRM has brought database
marketing to
the forefront of many organizations. When a customer calls with
a question,
120. the database allows the customer service representative or
technician to get
a good understanding of the situation rapidly. More responsive
service
increases the probability of developing long-term relationships
with
customers, which leads to repeat purchases. That is a major
advantage to
the marketer, because retaining old customers is usually more
profitable
and less costly than acquiring new customers.
However, CRM has become a controversial topic. Skeptics point
to the
hype that is associated with a concept that is not always clearly
defined.
Drozdenko01 2/26/02 6:03 PM Page 6
Others question the premise that it is even possible for
marketers to
develop true relationships with customers. Few companies
actually can
develop one-to-one marketing, which is the basis of any real
relationship.
For many companies, CRM just means increasing the
probability of
repurchase. (Beardi, 2001, p. 1).
In a response to CRM cynics, Ray Schultz (2001) acknowledges
that
true one-to-one marketing is unlikely, but customer-centric
marketing
(marketing that focuses on customer needs) and a two-way
121. dialogue with
customers is possible. Direct marketers, through their databases,
can
develop programs to establish this dialogue with customers and
measure
the effectiveness of these programs.
Changes in Media
Marketers have traditionally reached customers through media
that are
today becoming more and more fragmented. Just a few decades
ago, when
there were three major television networks, a marketer could
reach a large
audience with a single ad. Now, hundreds of cable and satellite
television
channels target the special interests of segments of viewers
from sports to
classic movies. Some of these categories have been segmented
even further,
such as classic sports, golf, racing, and radical sports. Some
magazine
categories also show increasing fragmentation. In the food
category, you
can subscribe to magazines that focus on home cooking,
gourmet cuisine,
vegetarian fare, cooking with chocolate, low-calorie cooking,
and spicy-hot
food, among others.
The Internet represents an extreme in fragmentation. Althoug e-
commerce
is still in the process of developing, we can be certain that more
people in the
future will use the Internet as a source of information. Internet
122. communica-
tion has the potential to reach even smaller, more specialized
segments of the
market. Databases containing these special-interest cable
viewers, magazine
subscribers, and Internet site registrants provide highly targeted
lists for
goods and services.
Direct marketing, including mail, e-mail, and telemarketing, can
bring
targeted messages to individual consumers and business
customers with
very specific characteristics. Mail and telemarketing have the
potential to
communicate to people based on individual needs and relate
these needs to
the offer. Many organizations have not developed this ability to
adequately
and efficiently target and communicate with individual
customers, and
therefore resources are wasted. The challenge for direct
marketers is both
to increase the relevance of the communication in order to
increase
response to an offer and also to reduce contacts with individuals
who have
a low probability of responding to the offer. Databases provide
the means
to meet the challenge of fragmented media by focusing
marketing commu-
nications on the specific needs of customers.
Introduction to Concepts 7
Drozdenko01 2/26/02 6:03 PM Page 7
123. 8 OPTIMAL DATABASE MARKETING
Changes in Distribution Structure and Power
Power in the distribution channel has shifted. No longer are
manufacturers
in control of distribution channels, as they were in the past.
Now, with the
consolidation of retailing on a regional, national, and even
multinational
level, one retailer has much more impact on the bottom l ine of a
manufac-
turer. If Wal�Mart or Home Depot decides not to carry a
manufacturer’s
product, the manufacturer may lose millions of dollars in sales
annually.
Even large manufacturers like Proctor & Gamble do not take
lightly their
relationships with national retailers. In fact, some
manufacturers have
changed their organizational structure in order to focus on these
important
retail customers. By establishing databases, such as the one at
the supermar-
ket that contains Keri Lee’s information, retailers are gaining
even more
power. They now have extensive databases on the purchasing
patterns of
millions of customers. These customer databases can be used to
locate
segments that may be important to marketers such as customers
loyal to
specific brands, frequent purchasers, high-volume purchasers,
124. brand
switchers, and promotion-sensitive customers.
In an attempt to maintain direct contact with customers,
manufacturers
have developed their own databases. Often these manufacturers’
databases
have been used for promotions rather than direct sales. For
example,
Kellogg’s has used databases for new product introductions,
sending
potential customers free samples and coupons. The objective of
these data-
base promotions is to help generate retail sales. On the other
hand, some
manufacturers have developed alternative distribution channels.
It may not
be possible to sell low margin products, like many of the
products found
in supermarkets, directly to consumers. However, manufacturers
may be
able to develop more exclusive niche products that have the
potential to
become profitable through direct channels. General Foods, the
maker of
Maxwell House Coffee, uses database marketing to sell a
premium coffee
directly to consumers. The brand, Gevalia Kaffee, is positioned
as “fine
coffees of Europe,” and customers receive shipments of coffee
at a regular
interval. To entice customers to become members of this
program, Gevalia
provides a free coffeemaker and the option to drop out of the
program
at any time without obligation. The initial risk for Gevalia
125. management
is great, and profit is not expected until after several purchase
cycles.
However, over the longer term, the product can be profitable.
Some marketing experts, such as Lester Wunderman (1998), see
dramatic changes in the way we think about distribution
channels. Single
distribution channels will become multiple-channel distribution
systems.
Products will be available where people want to buy them. The
Internet
has become a vehicle for moving rapidly to multiple distribution
channels.
A number of products, including cars, computers, greeting
cards, books,
groceries, and even M&M’s (see www.ColorWorks.com), can be
purchased
Drozdenko01 2/26/02 6:03 PM Page 8
Introduction to Concepts 9
directly on the Internet as well as through their conventional
retail
channels.
Proctor & Gamble, the manufacturer of many supermarket
items, includ-
ing Tide, Pringles, Oil of Olay, and Folgers, also markets a
premium coffee
direct to consumers through the mail and the Internet. Although
P&G uses
the Internet and other direct channels to promote its products, it
126. offers very
few directly to consumers. Expansion into direct channels can
be viewed as a
direct challenge to store retailers, and P&G wants to avoid any
impressions
of a threat (“P&G Makes AOL Debut,” 1999). So even though
databases
allow marketers distribution channel options, a number of
factors such as
product category and current channel arrangements have to be
considered.
Lifestyle and Demographic Trends
A number of lifestyle and demographic trends have moved
consumers
away from traditional retailers. Although store-based retailing
is still
strong, people seem to have less and less time for the process of
getting into
the car, driving miles to stores, searching for products, and
waiting in lines
to buy them. In our Keri Lee scenario, you might have noticed
that
she did not leave work until after 7:00 p.m. With the current
pressures on
businesses to perform more work with fewer employees, many
people are
in Keri’s situation. Keri still shops at the supermarket and other
retail
stores, but she shops more often from catalogs, shopping
channels, and on
the Internet. As Internet commerce expands, consumers have
more
opportunities to shop for more products from home. Internet
companies
127. are delivering groceries, drugs, and general merchandise
directly to homes
in certain parts of the United States.
The demographic trends that contribute to the movement of
shoppers
away from store retailers include
♦ Higher percentage of women in the workforce
♦ Higher percentage of family members working
♦ More child-rearing activities that require parents’ time (e.g.,
lessons,
carpools, sports, trips)
♦ Increasing access to the Internet at home, which increases the
chances
of online shopping
♦ Increase in ethnic populations seeking products that may not
be avail-
able from local store retailers
♦ Less brand loyalty, driving people to find convenient
alternative
sources for products
In response to these trends, marketers will make more types of
products
available from nonstore sources (Internet, mail, TV). As more
nonstore
Drozdenko01 2/26/02 6:03 PM Page 9
128. sources become available, competition will increase, driving
more consumers
away from store retailers. As mentioned previously, direct
marketing sales are
expected to increase at a rate that is higher than sales in
general. Because all
forms of direct marketing are dependent on databases, the use of
databases
will also increase.
Accountability for Marketing Actions
Accountability for expenditures is more prevalent in business
today. In pub-
licly held companies, shareholders are becoming more sensitive
to financial
reports. Within the organizations, upper-level managers want to
know
whether expenditures on specific promotions (ad campaigns,
trade promo-
tions, etc.) yield an appropriate return on investment. It is often
difficult,
however, to directly relate mass media advertising to changes in
sales.
Marketing databases allow expenses and revenues to be tracked
and
evaluated.
In particular, the database can be used to track the profitability
of prod-
ucts over time. As mentioned above, companies have to invest
in customers
through the costs of promotions. Sometimes these promotions
include free
items such as the coffeemaker that Gevalia sends new customers
and free
129. CDs offered by record clubs. Even if no incentives are used, an
investment
is needed in list rentals, the cost of the mailing, and overhead
expenses.
Often these investments are not recovered immediately.
However, in the
long term, these promotions may become very profitable as the
customer
makes additional purchases. Similarly, in the b-to-b market,
marketing and
other costs related to customer acquisition can be substantial
and must be
evaluated over a long period of time. Sometimes, return on
investment may
not come for several years, if at all.
With the database, a marketer can track profits from individual
cus-
tomers over time and further break down the effectiveness of
individual
marketing programs such as promotions with incentives. This
long-term
tracking of customers is only possible with a database. Without
the long-
term tracking of individual customers, a potentially profitable
marketing
program may be stopped prematurely.
Not only are databases important to upper management as an
accountabil-
ity tool, but other marketing personnel can directly benefit from
them. As an
account executive, Keri Lee wants to use her time productively.
Part of her
compensation is based on commissions. She uses a database to
manage
130. customer relationships by scheduling contacts at critical times,
such as prior
to contract renewals. She can also respond to customers more
efficiently
and effectively, because critical data on the account are easily
available.
She can determine who her best customers are, and she can
evaluate the
amount of time she spends on less productive accounts. Methods
such as
10 OPTIMAL DATABASE MARKETING
Drozdenko01 2/26/02 6:03 PM Page 10
lifetime value (LTV) analysis have been developed that evaluate
promotional
activities, customer segments, and product lines.
Integration of Business Functions
When all the functional areas of a business work together
(marketing,
accounting, finance, operations, human resources, information
systems,
etc.), the organization usually becomes more effective and
efficient. In
the older business model, the functional areas work almost
independ-
ently. Tasks are moved from one area to another in a sequential
manner.
Problems often arise from this “silo” approach to business
organiza-
tions. For example, Production may not be aware of marketing
131. programs
that might require increases in production levels, and Marketing
may not
be aware of increases in production costs that may require
pricing
modifications.
Today, more and more businesses seek to increase the efficiency
by inte-
grating functional areas. Databases facilitate this integration.
Costs can be
clearly documented, and sales levels can be more accurately
forecasted and
monitored to allow for adjustments in production, inventory,
and staffing
levels. From the financial perspective, sales revenues from
individual items
and product lines can be tracked more closely for better
financial planning
and resource allocation.
A direct marketing organization is centered on its database. The
database
can provide all functional areas immediate access to the
progress of
marketing programs, individual items, product lines, and
divisions.
Although technology as greatly improved the ability of
manufacturers to
monitor products through conventional retail distribution
channels,
problems getting data quickly from the many distributors and
retailers still
exist. Therefore, the organization that uses marketing databases
well can be
more responsive to internal and external changes.
132. Technological Advances
In recent years, the computer technology needed for developing
marketing
databases has decreased in price and increased in power.
Consequently,
more organizations have the financial resources to purchase the
hardware
and software necessary to develop a marketing database. In
addition,
because of the increasing power of low-cost PCs, smaller
organizations
can utilize databases. Within the organization, more people
have access to
technology and, therefore, can take advantage of the database.
In Keri’s
organization, all account executives have a powerful notebook
computer
that allows access to the database even on the road. Through the
network,
Introduction to Concepts 11
Drozdenko01 2/26/02 6:03 PM Page 11
the database is constantly updated, allowing all areas of the
business to be
aware when a transaction is made.
Software has also become more user friendly, so that people
don’t have to
be computer experts to take advantage of the benefits of
databases. Not only
133. are there general database programs that are easy to learn and
use such as
Access and Paradox, but even small businesses can use sales
and marketing
databases such as ACT! and GoldMine. Furthermore, industry-
specific
database programs are available that allow small organizations
to coordinate
marketing and other business functions.
More Informed Customers
Consumers and business customers have access to substantially
more infor-
mation now than in the past. Greater product knowledge brings
more crit-
ical evaluation of products and greater consideration of price.
Information
from marketers extends to areas outside traditional mass
advertising. Sales
promotion methods such as rebates, sweepstakes, contests, and
coupons
continue to be widely used. Event marketing that associates
products with
sporting, music, and other events have been used with a greater
frequency
in the last few years. Marketers of prescription pharmaceuticals
now target
patients directly, providing information about product benefits
(and
side effects) in an attempt to induce patient-physician
discussions about
the possible trial of a drug. The Internet and infomericals have
provided
marketers with an opportunity to present detailed information
about
134. products to consumers.
However, marketers have to compete not only with information
provided by other marketers but also with information from
other sources
such as public interest groups, governmental agencies,
journalists, and
not-for-profit organizations. In our Keri Lee scenario, you may
have
noticed that she was able to get insurance quotes online.
Furthermore, it
is now common for car customers to get product information,
reviews,
ratings, invoice pricing, and price quotes via the Internet (see,
e.g.,
www.edmunds.com). In addition to its magazine, Consumers
Union has
a Web site (www.consumerreports.org), and television news
programs
routinely present CU’s product reviews and consumer
information.
Organizations such as the Center for Science in the Public
Interest
(www.cspinet.org) often present information that is in conflict
with
information given by marketers. For example, the Center
opposes the
marketing of products containing Proctor and Gamble’s low -
calorie fat,
Olestra. Network programs such as Dateline and 20/20
frequently feature
segments on consumer issues. Chat rooms allow consumers to
ask
questions and voice opinions about products. In addition, with
the expan-
sion of specialized cable channels and magazines, consumers
![Data Samples & Data Analyses
NYU | SCPS
Database Management & Modeling
Edward Colet
[email protected]
Asynchronous Session 3, week of June 7 2021
Class material and homework so far
You should be through text chapters 1-3 (introduction), 4-5
(database fundamentals), and the supplemental readings on
RDMBS’s and BigData;
HW submissions were a short write-up about yourselves (hw1),
a relational database design exercise, and a BigData discussion
(hw2)
Questions from the material?
Please feel free to also use the discussion section on our NYU
Discussion site to ask, answer, comment on material from this
week (for this week, this will be part of hw3)
Content for this week
Chapter 6: The Analysis Sample
Chapter 7: Analyzing and Manipulating Customer Data](https://image.slidesharecdn.com/datasamplesdataanalysesnyuscpsdataba-220921023727-8c0f2308/85/Data-Samples-Data-AnalysesNYU-SCPSDataba-1-320.jpg)
