Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cesse'11 Using Market & Customer Insight to Make Better Decisions. final


Published on

Presented at CESSE AGM in Vancouver with remote panelists presenting from New York and New Jersey through Blue Sky Broadcast's remote KOL platform.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

Cesse'11 Using Market & Customer Insight to Make Better Decisions. final

  1. 1. Using Market & Customer Insight to Make Better Decisions<br />Jodie Slaughter<br />McKinley Advisors (presenting) American Ceramics Society <br />Jackie Oppenheim & Russ Raman<br />American Society of Mechanical Engineers<br />Marc Beebe, CAE<br />IEEE Corporate Strategy and Communications<br />Peter Turner (moderator)<br />MCI Group<br />Remote Meeting Technology provided by<br />
  2. 2. What does it take to grow a T&D business?<br />
  3. 3.
  4. 4. Research as a Catalyst for Sound Association Decision Making<br />
  5. 5. Today’s Topics<br /><ul><li> Research 101 – benefits, applications, definitions
  6. 6. Research you can conduct yourself with no/little budget
  7. 7. When to call in a professional
  8. 8. Research in Action: Lessons learned by a mid-sized CESSE member</li></li></ul><li>Correctly done…<br />Research:<br /><ul><li>Gives members a voice
  9. 9. Allows you to hear from new audiences
  10. 10. Validates or contradicts whims and curtails lurching
  11. 11. Surfaces new product ideas/unmet needs
  12. 12. Provides insights on competitors
  13. 13. Tests ideas and assumptions 
  14. 14. Determines product/service pricing, positioning and distribution
  15. 15. Monitors performance and member satisfaction</li></li></ul><li>Research 101<br />The First Step in <br />Successful Research<br />
  16. 16.
  17. 17. Research 101—Definitions<br />Primary research: Collecting data that doesn’t already exist<br />Secondary research: Collecting data that already exists<br />Qualitative: Deeply descriptive and anecdotal information<br />Quantitative: Findings that can be expressed numerically and applied to confidence intervals (statistical reliability)<br />
  18. 18. Research 101<br />Qualitative Research<br />+Quantitative Research<br />=Sound Research<br />
  19. 19. Research You Can Do Yourself<br />No Money…No Problem<br />
  20. 20. Research You Can Do Yourself<br /><ul><li>Secondary research
  21. 21. Electronic survey instruments*
  22. 22. Distribute questions to market-facing staff and volunteers</li></li></ul><li>
  23. 23. When do you need a professional?<br />
  24. 24. CESSE Member Mini-Case Study<br />Budget: ~ $6,000,000<br />Members: 5,862 – 6,120 (20% growth in corporate)<br />Why: New exec loves data, needed myth-busting and voices from broader audiences, never did it before<br />What: 69 interviews, two focus groups, electronic survey (19.5% response rate), enviro scan<br />When: Late 2007<br />What’s next: Repeating in 2011<br />
  25. 25. CESSE Member Mini-Case Study<br />Surprises in the Data <br /><ul><li> Members much less conservative than leaders
  26. 26. Very small (older) segment revered sacred cows
  27. 27. Time-consuming and potentially troublesome elections were not necessary
  28. 28. Academics had been over-represented in association decisions
  29. 29. Myths about corporate members were disproved</li></li></ul><li>CESSE Member Mini-Case Study<br />Forced Choice Method<br /><ul><li> A departure from typical scale-based methods
  30. 30. Effective when contemplating key decisions
  31. 31. Major change
  32. 32. Strategic planning
  33. 33. Program development
  34. 34. Must be informed by qualitative phase
  35. 35. Seeks a measure of “practical significance”
  36. 36. Allows for segment-specific solutions</li></li></ul><li>
  37. 37.
  38. 38.
  39. 39. CESSE Member Mini-Case Study<br />
  40. 40. CESSE Member Mini-Case Study<br />In their own words:<br />“As a result of the information we gained from the survey, we were able to better identify our current audience as well as identify our potential audience and in one short year we were able to increase our total membership by 3%. “ <br />Megan Bricker<br />Director, Marketing & Membership Services<br />
  41. 41. Thank youJodie Slaughter,<br />
  42. 42. “If there is one phrase that sets remarkable associations apart from their counterparts, it’s “Data, data, data.. [In remarkable associations, ] whether qualitative or quantitative, research is always put to use, not put on the shelf.”<br /> -7 Measures of Success – What Remarkable Associations Do that Others Don’t<br />
  43. 43. Do you want to be remarkable?<br />
  44. 44. ASME Training & Development <br />26<br />Using Market and Custom Insight to Make Better Decisions2 Primary Research Case StudiesE&T3Wednesday July 201.40pm – 3.00pm<br />
  45. 45. 27<br />Jackie Oppenheim Russ RamanDirector, Training & Development Marketing ManagerASME ASMETel: +1 212 591 7526 Tel: +1 212 591 7745Email: Email:<br />ASME Training & Development <br />
  46. 46. ASME at a glance<br />28<br /><ul><li>Holds numerous technical conferences worldwide
  47. 47. Offers professional development and continuing education courses
  48. 48. Internationally recognized industrial and manufacturing standards enhance public welfare and safety
  49. 49. Educational and technical society of mechanical engineers
  50. 50. Not-for-profit organization founded in 1880
  51. 51. 120,000 members worldwide, including 24,000 student members
  52. 52. Staff of over 350: headquarters in New York City with offices in Washington, Houston, Atlanta, Brussels, India and Beijing</li></ul>ASME Training & Development <br />
  53. 53. 29<br />What are the two case studies?<br />Nuclear Training Opportunities Study<br /> - higher cost, more detailed, longer time frame<br />Workshop Participant Engagement Study<br /> - lower cost, less detailed, quicker<br />ASME Training & Development <br />
  54. 54. For each Case Study we’ll address:<br />Background and Purpose<br />Approach and Methodology<br />Summary of Findings<br />Conclusions and Business Implications<br />What would / should we do differently now?<br />30<br />ASME Training & Development <br />
  55. 55. Background and Purpose<br />Focus on energy; specifically nuclear<br />Understand the current and projected training and skills assessment needs of the nuclear power industry <br />Would help guide us in what additional training courses, Certificate and Certification programs to pursue<br />Nuclear Training Opportunities Study<br />ASME Training & Development <br />31<br />
  56. 56. Nuclear Training Opportunities Study<br />2. Approach and Methodology<br />Conducted in-depth telephone interviews <br />Targeted different industry segments to get balanced viewpoint<br />Average length of interview 40–45 minutes.<br />Interviewers had discussion guide outline<br />Had specific questions to answer (see next slide)<br />Also had ability to listen to unsolicited information<br />All respondents were pre-recruited for the interview<br />Not initially identified as an ASME study <br />ASME Training & Development <br />32<br />
  57. 57. ASME Training & Development <br />33<br />DISCUSSION GUIDE OUTLINE<br /> <br />A. Opening & Comfort Building<br />Purpose of interview<br />Confidentiality – Private study for trade association<br />Audio-taping<br /> <br />B. Company Background<br />Products produced (Suppliers only)<br />Operating units and new plants (Utilities)<br />Number of employees<br />Percent of business in nuclear industry (Suppliers only)<br /> <br />C. Respondent Background<br />Respondent’s position/responsibilities<br />Length of career in nuclear industry<br />Supervision of other employees<br />Involvement with operating plants and/or new construction<br />Positions held throughout career<br />Training received throughout career<br />How maintain/refresh skills <br /> <br />D. Current Training Requirements<br />Involvement with training employees receive in nuclear industry<br />Main categories of workers requiring specific training<br />Specific job training<br /> <br />E. View of Nuclear Industry<br />View of nuclear industry over next 3 years<br />View of nuclear industry over next 10 years<br />Areas of growth/contraction<br />Future needs for training<br /> <br />F. Skill Level and Certifications<br />Industry-wide certifications<br />Peer review systems<br />Certificates/Certifications that are important<br />How training should be provided<br /> <br />G. Training Providers<br />Major training providers in nuclear industry<br />Types of training they provide<br />Strengths/Weaknesses<br />How training is offered/assessed<br />Satisfaction<br /> <br />H. ASME Training<br />Awareness of ASME<br />Would you look to ASME for training?<br />Role of ASME for training<br />Specific type of training ASME should provide<br />
  58. 58. Nuclear Training Opportunities Study<br />3. Summary of Findings<br />Growing need for training & education in the nuclear power industry over next 5-10 years. <br />ASME is perceived as a well-respected, well-known association with national and international awareness<br />ASME should continue dialog with consumers of training as they build their portfolio<br />Primary employees that need to be trained<br />Partnering opportunities <br />Increase in the need for international training within this industry<br />ASME Training & Development <br />34<br />
  59. 59. Nuclear Training Opportunities Study<br />4. Conclusions and Business Implications<br />Continued discussions with industry players to carry on development of a Certificate Program for Nuclear Codes and Standards<br />Additional course development within the nuclear industry in live courses, workshops and eLearning <br />Building a network of global partners to deliver ASME courses around the world <br />ASME Training & Development <br />35<br />
  60. 60. Nuclear Training Opportunities Study<br />5. What would / should we do differently now?<br />Define expectations of results<br />Avoid designing the questionnaire with too many stakeholders <br />Follow up survey would have been good to supplement some of the areas – more statistical focus<br />If we knew then what we know now,<br />would we still carry out the study?<br />ASME Training & Development <br />36<br />
  61. 61. ASME Training & Development <br />37<br />
  62. 62. Workshop Participant Engagement Study<br />Background and Purpose<br />In early 2010 ASME launched a series of educational workshops held in conjunction with quarterly meetings of code committee volunteers<br />Goals:<br />Increase revenues from training and exhibit sponsorships<br />Explore and test new topics<br />Identify and evaluate prospective new instructors<br />Incite interest of attendees to volunteer for participation in future code committee activities<br />Use market research to validate achievement of Goal #iv <br />ASME Training & Development <br />38<br />
  63. 63. Workshop Participant Engagement Study<br />Approach and Methodology<br />Online survey software (email) to query 150 past educational workshop attendees<br />Similar to Zoomerang or Survey Monkey software<br />Pros: easy, quick and affordable<br />Cons: “self-selection bias,” 10-15% response rate<br />Questionnaire to assess participants’ reaction to, interest inand intended involvement withvoluntary code committee activities<br />Requires skills to draft questions / analyze results<br />Good to have trained survey research professional handle<br />Field survey electronically, tab results and analyze<br />ASME Training & Development <br />39<br />
  64. 64. Workshop Participant Engagement Study<br />Summary of Findings<br />Total respondents: 38 (25.3%)<br />Workshops: Very high satisfaction level; would recommend to friends or colleagues<br />Reason for attending: “educational / earn CEU’s”;  only 12.5% cited “to learn more about ASME Code Week Committee Activities”<br />Result: Two-thirds of respondents attended one or more Code Committee meetings –– almost half were first time attendees<br />ASME Training & Development <br />40<br />
  65. 65. Workshop Participant Engagement Study<br />Summary of findings (continued)<br />Code Committee Meeting attendance: found “highly useful;” 80% said future attendance was “highly likely”<br />Reasons for non-attendance: “scheduling conflicts” or “unaware that Code Committee meetings were open to the public”<br />g) Strong interest in new “Introduction to ASME Code Committees” workshop<br />ASME Training & Development <br />41<br />
  66. 66. Workshop Participant Engagement Study<br />Conclusions and Business Implications<br />Overall satisfaction levels with Code Week Workshops – high<br />Objective of attracting new participants to Code Committee activities – achieved<br />Should ASME continue holding educational workshops co-located with Code Committee meetings? – yes<br />ASME Training & Development <br />42<br />
  67. 67. Workshop Participant Engagement Study<br />What would / should we do differently now?<br />Raise awareness among workshop registrants that Committee Meetings are “open to the public”<br />Develop and launch new “Introduction to ASME Code Committees” workshop offering<br />Aggressively seek and schedule “hot topic” subject matter to draw more workshop participants<br />Encourage participants to recommend attendance to peers and colleagues <br />ASME Training & Development <br />43<br />
  68. 68. Conclusions<br />Use research to understand your customers’ wants, needs and preferences – adjust products and services accordingly<br />Clearly identify the objectives and purposes<br />Research is scalable – depending upon importance, budget and time<br />And perhaps most important…<br />Only use research if you are prepared to act upon results – don’t waste time and resources<br />ASME Training & Development - January 2011<br />44<br />
  69. 69. Cheers!<br />ASME Training & Development <br />45<br />Thank you!<br />We welcome any questions or comments…..<br />Goodbye!<br />Russ<br />Jackie <br />
  70. 70. IEEE Education Research<br />Marc Beebe<br />Sr. Manager, Strategic Research<br />
  71. 71. Goal for Today<br />Case study on IEEE Education Research and how you can apply some of our lessons to your own activities<br />7/15/2011<br />47<br />
  72. 72. About IEEE<br />World's leading professional association for the advancement of technology<br />38 technical societies, 325 geographic Sections <br />148 transactions, journals & magazines<br />1,300 sponsored/co-sponsored conferences each year<br />1,200 active standards <br />Approximately $390 million annual budget <br />400,000 members in more than 160 countries<br />52% in US, 48% in all other countries<br />25% students (102,000)<br />51% belong to a technical society<br />Research Area<br />3 Full-time research staff<br />Budget to hire external vendors<br />
  73. 73. IEEE is….<br />
  74. 74. BUT….<br />7/15/2011<br />50<br />
  75. 75. We still need diverse strategies for doing research<br />More projects than internal staff can handle<br />Lack of funding<br />Ability to prioritize research projects<br />7/15/2011<br />51<br />
  76. 76. Case Study: Continuing Education in Singapore<br />
  77. 77. Business questions we wanted to answer<br />Is there a market for Continuing Education in Singapore?<br />What form/topic areas of continuing education would be most useful<br />Would our members consider IEEE as a provider?<br />What is the competitive landscape like in that country?<br />If needed, who could we partner with?<br />7/15/2011<br />53<br />Member Survey<br />Environmental Scan<br />Interviews<br />
  78. 78. Member Survey<br />All non-student, non-retired member in Singapore<br />Conducted in-house, using existing resources<br />364 respondents, margin of error of ±4.5%, response rate of 23.1%<br />7/15/2011<br />54<br />
  79. 79. In-person and customized most of interest<br />7/15/2011<br />55<br />
  80. 80. Practical Technical Topics of most interest<br />7/15/2011<br />56<br />
  81. 81. Singapore members more interested in continuing education from IEEE<br />7/15/2011<br />57<br />
  82. 82. Environmental Scan<br />General background about Singapore<br />Size of the Relevant Industry Market<br />Continuing Education Industry structure and general trends<br />Current state of the market<br />Specific opportunities<br />7/15/2011<br />58<br />
  83. 83. Overview<br />The Singapore CET industry is a complex market with many players <br />Private education was an unregulated industry until late 2009, with new legislation gazetted and the set up of Council for Private Education (CPE) Singapore<br />There are the usual types of ‘bigger players’<br />Universities<br />Vocational institutions (polytechnics, ITEs)<br />Industry Associations/ Unions with a training arm<br />Private for profit providers (e.g. Kaplan)<br />There are many niche small providers with narrow fields of offerings <br />E.g. 60 smaller Accredited Training Organizations (ATOs) in soft skills segment<br />E.g. 90 smaller ATOs in professional skills segment<br />7/15/2011<br />59<br />Sources: NTUC Learning Hub, Company websites<br />
  84. 84. Key Characteristics of some players in the market – tertiary institutes<br />7/15/2011<br />60<br />Sources: Company websites and brochures<br />
  85. 85. Interviews<br />Individual Interviews/Discussions with government officials and industry leaders<br />Served two purposes:<br />Gather information<br />Make connections<br />Used semi-structured interview format<br />7/15/2011<br />61<br />
  86. 86. Gremlins to avoid<br />Not doing research, when it’s necessary<br />Doing research that’s not necessary<br />Using non-representative samples<br />Volunteers<br />Panels that aren’t like your members<br />Doing bad research<br />7/15/2011<br />62<br />
  87. 87. “Bad” Research<br />7/15/2011<br />63<br />Via Brian F. Singh:<br />
  88. 88. Thank you!<br />Marc Beebe<br />Sr. Manager, Strategic Research<br />IEEE<br /><br />+1 732.465.5891<br />7/15/2011<br />64<br />
  89. 89. Thank you<br />