Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beyond RFM: Modeling Applications


Published on

  • Be the first to comment

Beyond RFM: Modeling Applications

  1. 1. Building the CRS Online Community Test #1 Email Campaign February 25, 2005 <ul><li>“ Beyond RFM” </li></ul><ul><li>February 2005 DMFA Roundtable </li></ul><ul><ul><ul><ul><li>Kevin Whorton, Direct Response Fundraising Consultant Catholic Relief Services </li></ul></ul></ul></ul><ul><ul><ul><ul><li>[email_address] </li></ul></ul></ul></ul>
  2. 2. Modeling: Theory and Reality <ul><ul><li>Theory: RFM Has Weaknesses </li></ul></ul><ul><ul><ul><ul><li>Limited use of information: gift history only </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Omits demographics, psychographics </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Mostly provides decision support for marginal audiences </li></ul></ul></ul></ul><ul><ul><ul><ul><li>No prioritization: R<F<M? … M>R=F? … M=R=F? </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Uses language of discrete, not continuous variables </li></ul></ul></ul></ul><ul><ul><li>Reality: RFM Works Well Enough Most Times </li></ul></ul><ul><ul><ul><ul><li>House file mailings—very strong, long histories </li></ul></ul></ul></ul><ul><ul><ul><ul><li>House file telemarketing </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Could be improved but little incentive to do so: </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Can only be so efficient on mailings </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Beyond some point minimizing cost may minimize revenue </li></ul></ul></ul></ul></ul>
  3. 3. CRS: Current Practices Limitations Future Applications
  4. 4. Applying Techniques at CRS <ul><ul><li>House File Model Use </li></ul></ul><ul><ul><ul><li>Target Analysis Group: affinity/other gift behavior </li></ul></ul></ul><ul><ul><ul><ul><li>Powerful to screen the 50% waste, including lapsed in acquisition now outperforms a dedicated lapsed campaign </li></ul></ul></ul></ul><ul><ul><ul><li>Genalytics: full-file scoring by half-decile </li></ul></ul></ul><ul><ul><ul><ul><li>Full house file, by future probability of giving </li></ul></ul></ul></ul><ul><ul><li>Acquisition Model </li></ul></ul><ul><ul><ul><li>Selection criteria used during list selection </li></ul></ul></ul><ul><ul><ul><ul><li>Zip models and “Catholic Finder” </li></ul></ul></ul></ul><ul><ul><ul><li>Full acquisition model </li></ul></ul></ul><ul><ul><ul><ul><li>Created household database from 45 million past contacts </li></ul></ul></ul></ul><ul><ul><ul><ul><li>File scoring after merge purge: typical 20% suppression </li></ul></ul></ul></ul>
  5. 5. Expanding Demographic Data <ul><ul><li>Distinguishing between donors: marketing vs. DM </li></ul></ul><ul><ul><ul><li>Profiling new donors : 62 years avg vs. “ youth movement” </li></ul></ul></ul><ul><ul><ul><li>Drawing linkage between awareness and donation </li></ul></ul></ul><ul><ul><ul><li>Understanding relationship: first gift  ongoing behavior </li></ul></ul></ul><ul><ul><li>We now use data to categorize donors </li></ul></ul><ul><ul><ul><li>By appeal: emergency, region, program area </li></ul></ul></ul><ul><ul><ul><li>By vehicle: catalog, calendar, newsletter, TM, e- </li></ul></ul></ul><ul><ul><ul><li>By timing: seasonality </li></ul></ul></ul><ul><ul><ul><li>By preferences: l imited mailing , no mail, no TM </li></ul></ul></ul><ul><ul><ul><ul><li>Especially critical, post-Tsunami </li></ul></ul></ul></ul><ul><ul><li>Data used to drive frequency </li></ul></ul><ul><ul><ul><li>Segmenting beyond RFM, going deeper into files </li></ul></ul></ul><ul><ul><ul><ul><li>Often based on Interest Codes (next slide) </li></ul></ul></ul></ul>
  6. 6. Example: Interest Codes Used for Inclusions/Exclusions <ul><ul><li>Entire file </li></ul></ul><ul><ul><ul><li>Coded with a mix of Donor Service & DM codes </li></ul></ul></ul><ul><ul><ul><li>Simplify our house file selection </li></ul></ul></ul><ul><ul><ul><li>Behavior captured to: - simplify ad hoc analysis - extend RFM - develop profiles - crosstab “donor types” </li></ul></ul></ul>13843 LD Low Dollar Donors 49677 CD Calendar Donors 55684 95S99 Score 95-99 56137 0S4 Score 0-4 65489 TD Telemarketing Donors 65556 WBD Wooden Bell Donors 67420 HISIND Hispanic Indicator 82993 PRD Premium Donors 87914 CAO Catalog Overlay 103340 RD Renewal Donors 324288 ND Newer Donors 363374 ED Emergency Donors 663157 DSF1 Delivery Point Validation 734794 FY2003 Fiscal Year 2003 counts count ID Interest Code Interest Code Description
  7. 7. Other (Non-Modeling) Data <ul><ul><li>Simulations: gift arrays </li></ul></ul><ul><ul><ul><li>Demographic overlays beyond DM: mid-level PG, MG </li></ul></ul></ul><ul><ul><ul><ul><li>Age & wealth trump typical RFM giving behavior </li></ul></ul></ul></ul><ul><ul><li>Mail sensitivity analysis </li></ul></ul><ul><ul><ul><li>Finding correlation between total mailings, gifts per donor </li></ul></ul></ul><ul><ul><ul><ul><li>Goal: maximize satisfaction without sacrificing revenue </li></ul></ul></ul></ul><ul><ul><li>Maintaining &quot;interest codes&quot; library of preferences </li></ul></ul><ul><ul><li>Merge-purge with greater control </li></ul></ul><ul><ul><ul><li>Moved internally, staff analyst & FirstLogic software </li></ul></ul></ul><ul><ul><li>Conversion analysis </li></ul></ul><ul><ul><ul><li>List life-cycle: tables showing LTV (2-year) by acq. list </li></ul></ul></ul><ul><ul><ul><li>Target Analysis: benchmarking/comparisons </li></ul></ul></ul>
  8. 8. Other Data: Research <ul><ul><li>Donor research </li></ul></ul><ul><ul><ul><li>Analyzing share of market/share of wallet </li></ul></ul></ul><ul><ul><ul><li>Knowing what else donors give to </li></ul></ul></ul><ul><ul><li>Qualitative/focus groups </li></ul></ul><ul><ul><ul><li>Package/teaser/copy testing </li></ul></ul></ul><ul><ul><ul><li>Underlying motivations/drivers/perceptions </li></ul></ul></ul><ul><ul><li>Market research </li></ul></ul><ul><ul><ul><li>Measuring aided/unaided recall, aficionados </li></ul></ul></ul><ul><ul><ul><li>Cluster models (segmentation studies) </li></ul></ul></ul><ul><ul><ul><li>Positioning studies (branding, relative message) </li></ul></ul></ul><ul><ul><ul><li>Competitive intelligence </li></ul></ul></ul>
  9. 9. Limitations: Analyzing Results <ul><ul><li>Most segmentation build to drive reporting </li></ul></ul><ul><ul><ul><li>Pledgemaker report writer </li></ul></ul></ul><ul><ul><ul><li>Occasional use of Business Objects/SAS for ad hoc </li></ul></ul></ul><ul><ul><li>Most segmentation is by discrete RFM buckets </li></ul></ul><ul><ul><ul><li>Segmentation continues in the &quot;normal way&quot; $25-$49, 0-12 months, F1+ $50-$99, 0-12 months, F1+ $100-$249, 0-12 months, F1+ </li></ul></ul></ul><ul><ul><ul><li>Extending universe based on interest codes </li></ul></ul></ul><ul><ul><ul><li>Applying excludes </li></ul></ul></ul><ul><ul><ul><ul><li>Record types (PG, Corp, Spanish-language, Religious Orders) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Individual preferences (1, 2, 6, 12x preferred mail schedules) </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Mutual omits from overlapping camapigns </li></ul></ul></ul></ul>
  10. 10. Best Intentions: Other Applications <ul><ul><li>Original goal in 2003: &quot;family of models&quot; </li></ul></ul><ul><ul><ul><li>Telemarketing </li></ul></ul></ul><ul><ul><ul><li>Early warnings of defection </li></ul></ul></ul><ul><ul><ul><li>Lapsed donors </li></ul></ul></ul><ul><ul><ul><li>Upgrade potential: mid-level program </li></ul></ul></ul><ul><ul><li>Reasons for using: </li></ul></ul><ul><ul><ul><li>High cost per contact/good stewardship </li></ul></ul></ul><ul><ul><ul><li>Sensitivity to complaints </li></ul></ul></ul><ul><ul><ul><ul><li>Predict positive and negative outcomes </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Complaints seen as proxy for reduced lifetime value </li></ul></ul></ul></ul><ul><ul><li>Reasons not pursued </li></ul></ul><ul><ul><ul><li>Not a $$ limitation, but rather management time </li></ul></ul></ul>
  11. 11. Goal/Vision <ul><ul><li>Want to be more &quot;donor focused&quot; </li></ul></ul><ul><ul><ul><li>Finding constructive ways to avoid treating all donors the same </li></ul></ul></ul><ul><ul><ul><li>RFM often treats as identical: </li></ul></ul></ul><ul><ul><ul><ul><li>$500 donor, every year, 1 gift very end of year </li></ul></ul></ul></ul><ul><ul><ul><ul><li>$500 cumulative donor, monthly frequency </li></ul></ul></ul></ul><ul><ul><ul><ul><li>$500 first-time donor </li></ul></ul></ul></ul><ul><ul><ul><li>Goal: sufficiently flexible systems to tailor contact sequence </li></ul></ul></ul><ul><ul><ul><ul><li>Hard to implement CRM systems to reduce costs/maximize efficiency & donor satisfaction </li></ul></ul></ul></ul>
  12. 12. Sample: Donor-focused Grid Use the gift they give to this appeal Consider lifetime seasonal giving activity
  13. 13. Sample Analysis: Years on File <ul><ul><ul><li>Graphing non-linear relationships: finding “sweet spots” </li></ul></ul></ul>
  14. 14. Analysis: Lifetime Avg. Gift <ul><ul><ul><li>And knowing when the relationships really are linear/predictive. </li></ul></ul></ul>
  15. 15. Quick Guide to Models/Techniques
  16. 16. Guide to Models <ul><li>Three major families: </li></ul><ul><ul><li>Parametric Methods </li></ul></ul><ul><ul><ul><li>Linear regression, logistic regressions </li></ul></ul></ul><ul><ul><li>Recursive Partitioning methods (i.e. CHAID) </li></ul></ul><ul><ul><ul><li>Tree diagrams—easier to see interaction between variables. Most time consuming. </li></ul></ul></ul><ul><ul><li>Non-parametric methods </li></ul></ul><ul><ul><ul><li>Neural networks, genetic/natural selection algorithms </li></ul></ul></ul><ul><ul><ul><li>Artificial intelligence—&quot;learning models&quot; used at CRS </li></ul></ul></ul><ul><li>Results are far more important </li></ul><ul><ul><li>Results: more a function of data quality than technique </li></ul></ul>Source: Target Analysis Group: Jason Robbins, statisticians
  17. 17. Sophisticated Techniques, Simple Answers <ul><ul><li>Cross-tabulations </li></ul></ul><ul><ul><li>Shows simple relationships between variables, typically percentages </li></ul></ul><ul><ul><li>&quot;Grids&quot; allow easy audience selection, but complex to review </li></ul></ul><ul><ul><li>Correlation : relationships between two variables </li></ul></ul><ul><ul><li>Regression : </li></ul></ul><ul><ul><li>X=f(x,y,z) or Membership=function of dues level, presence of competition, penetration, service mix </li></ul></ul><ul><ul><li>R 2 “explains” relationship between one variable and everything driving it </li></ul></ul><ul><ul><ul><li>Projections and forecast models </li></ul></ul></ul><ul><ul><ul><li>Logistic regressions: “yes/no” predictions </li></ul></ul></ul><ul><ul><ul><li>Logarithmic: coefficients= percentage contribution </li></ul></ul></ul><ul><ul><ul><li>Dummy variables: use to measure seasonality, time trends, effects of one-time shifts </li></ul></ul></ul>
  18. 18. Introducing Linear Regression <ul><ul><li>Linear regression defined </li></ul></ul><ul><ul><ul><li>PR=aR+bF+cM+dO </li></ul></ul></ul><ul><ul><ul><li>In English, “predicted revenue is a function of donor’s recency of giving, frequency, agg value, other stuff&quot; </li></ul></ul></ul><ul><ul><ul><li>Model for a renewal program: with avg response rate 4.25%, avg gift $36.25, revenue/name mailed of $1.54: </li></ul></ul></ul><ul><ul><li>1.54=-0.068(6.5) + 0.215(2.4) + 0.00465(156) + 0.0087(85) </li></ul></ul><ul><ul><li>Confusing, but potential &quot;Holy Grail&quot; tool for your house file program </li></ul></ul>$0.74 $0.73 $0.52 -$0.44 Contribution to RNM 0.0087 0.00465 0.215 -0.068 Coefficient 85 $156 2.4 6.5 Avg Value Indexed wealth of donor Aggregate total value of gifts Total gifts, relevant time period Months since last gift Equation
  19. 19. More Sense from Regressions <ul><ul><li>Confusing exposition: briefly assume you know what this means! </li></ul></ul><ul><ul><ul><ul><li>Alternative functional forms tell you more </li></ul></ul></ul></ul><ul><ul><ul><ul><li>For example: logarithmic transformations of each independent variable (R, F, M, Wealth) put them on equal &quot;dimensions&quot; </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Average values will no longer make sense, but coefficients will! </li></ul></ul></ul></ul><ul><ul><ul><ul><li>In last equation: 0.182 Months Since 0.215 Total Gifts 0.300 Aggegate Gifts 0.305 Indexed Wealth Means each value represents percentage contribution to results !! </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Note on last slide, many combinations of specific values would add to the average revenue per donor </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>The formula &quot;predicts&quot; it, because it represents the &quot;best fit&quot; expressing relationship between the dependent and independent variables </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>This is an overly simple equation: it assumes only RFM plus wealth </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Often there are other hidden values that also influence </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Equation level metrics (R-squared) and variable-level (t ratios) tell you the degree of prediction and statistical significance </li></ul></ul></ul></ul></ul>
  20. 20. What You Should Know as a User <ul><ul><li>When these techniques are used … </li></ul></ul><ul><ul><ul><li>Generally statistical software runs these: SAS at CRS </li></ul></ul></ul><ul><ul><ul><li>Fast process: takes less time to run than to explain </li></ul></ul></ul><ul><ul><ul><li>Key: some staff need to understand what the results mean </li></ul></ul></ul><ul><ul><ul><ul><li>Younger staff are better, esp. if exposed to it in college—&quot;data kids&quot; </li></ul></ul></ul></ul><ul><ul><li>Once a formula is derived, the real output is a scored file </li></ul></ul><ul><ul><ul><li>&quot;Plotting the residuals&quot; means taking best fit, multiplying through </li></ul></ul></ul><ul><ul><ul><li>Output can be indexed/scored according to predicted Rev/M etc. </li></ul></ul></ul><ul><ul><li>This typically falls on a curve, with an index ranging from 0-99 th percentile of predicted revenue per name mailed </li></ul></ul>First-time donor, modest means Periodic giver, average gift, well-to-do Lapsed occasional donor, big wealthy giver Frequent donor, low gift, well-to-do 98 $750 2 13 $3.89 28 $32 1 2 $0.47 65 $120 4 6 $1.58 65 $240 10 2.5 $3.66 Wealth Tot value Tot gifts Month Predict.
  21. 21. Acquisition Modeling at CRS
  22. 22. Before: List Effectiveness <ul><li>Targeting based on list effectiveness </li></ul><ul><li>Focused on “finding more lists like these” </li></ul>Campaign 1 Campaign 2
  23. 23. New Approach <ul><li>New analytic system to drive programs </li></ul><ul><ul><li>Build prospect universe of likely responders </li></ul></ul><ul><ul><li>Overlay with demographic and census data </li></ul></ul><ul><ul><li>Catalog interaction over time by person </li></ul></ul><ul><ul><li>Develop insights over time with modeling </li></ul></ul><ul><ul><li>Select/suppress based on predicted behavior </li></ul></ul>
  24. 24. After: Prospect Behavior <ul><li>Targeting based on prospect behavior </li></ul><ul><li>Focus on “finding more people like this” </li></ul>+ Marketing History Census & Specialty Demographics + List & Campaign Attributes
  25. 25. Preparation <ul><li>Develop infrastructure </li></ul><ul><li>Collect and organize data </li></ul><ul><li>Response behavior retained </li></ul><ul><li>Other available information added </li></ul>Prospect Universe External Demographics Data Focused Lists Prospect Lists Matchcode and Geography Campaign Data
  26. 26. Applying Analytics to Discover Patterns Prospect Universe Suppression List Model Ready Data Proliferation of Models Actionable Results Structured Data Equation ƒ (x) = * + Equation ƒ (x) = * +
  27. 27. The Final Solution Mailing Universe Suppress To Mail Production Acquisition Promotions Catholic Demographics Donations Census Demographics Data Mart Suppressed Mailing Universe Sample Scoring Equation ƒ (x) = * +
  28. 28. Results/Benefits <ul><li>Focused models on top segments rather than entire universe </li></ul><ul><ul><li>Suppressed mailing to bottom of prospect universe </li></ul></ul><ul><ul><li>Discovered significant numbers of new prospects similar to existing donors </li></ul></ul><ul><li>Savings more than paid for entire analytics program by: </li></ul><ul><ul><li>Removing bottom portion of prospect universe that provides negative ROI </li></ul></ul><ul><ul><li>Providing greater understanding of and insight into characteristics of prospects and donors </li></ul></ul>