Volume 8 • Issue 3 • 2010
The journal for technology product management and marketing professionals
Auditing Product Management
in Early-Stage Companies
Is Product Management Needed
In a Software Startup?
How Many Data Points
Do You Really Need?
Inside this issue: Volume 8 Issue 3 • 2010
5 ProductCamp Best Practices
By John Milburn and Paul Young
Creator of the world’s most popular ProductCamps are taking the world by storm and
product management and marketing seminars have sprouted up in Silicon Valley, Austin, Toronto,
Amsterdam, Sydney and more. If you haven’t
participated in one, you might wonder what all the fuss
About Pragmatic Marketing® is about. If you have participated, you already know!
Since 1993, Pragmatic Marketing has
conducted product management and
marketing training for 5,000 companies
in 23 countries. Our team of industry
thought-leaders produce blogs, webinars,
podcasts, and publications read by more
than 100,000 every year. 12 Auditing Product Management
in Early-Stage Companies
By L. Kurt Reiss
How do you deal with the “mess” of moving
The Pragmatic Marketer ™ targets and reference points? The tool that
8910 E. Raintree Drive has not been broadly used but can be of
Scottsdale, AZ 85260 great help is the product management audit.
Pragmatic Marketing, Inc. What is a product management audit?
Craig Stull / Founder and CEO
Kristyn Benmoussa / Editor-in-Chief 20 Is Product Management Needed
Graham Joyce / Managing Editor In a Software Startup?
————————————————— By Gopal Shenoy
Interested in contributing an article? What are the benefits that product
PragmaticMarketing.com/submit management can bring to the table during
the early days of a fledging business? Let’s
No part of this publication may be reproduced, stored in look at the typical genesis of a startup
any retrieval system, or transmitted, in any form or by any company and what transpires between
means, electronic, mechanical photocopying, recording
or otherwise, without the prior written permission of the start and first release of the product.
Other product and/or company names mentioned in this
journal may be trademarks or registered trademarks of
their respective companies and are the sole property
24 How Many Data Points
of their respective owners. The Pragmatic Marketer, Do You Really Need?
a Pragmatic Marketing publication, shall not be liable
regardless of the cause, for any errors, inaccuracies, By Ken Allred
omissions, or other defects in, or untimeliness or
unauthenticity of, the information contained within this
Because statistics teach us about minimum
magazine. Pragmatic Marketing makes no representations, confidence levels, confidence intervals, and
warranties, or guarantees as to the results obtained from margins of error, people often struggle with
the use of this information and shall not be liable for any
third-party claims or losses of any kind, including lost the idea they can start leveraging the data
profits, and punitive damages. from their win/loss program from the very
first data point.
The Pragmatic Marketer is a trademark of Pragmatic
Marketing, Inc. Printed in the U.S.A. All rights reserved.
ISSN 1938-9752 (Print)
ISSN 1938-9760 (Online)
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 3
Product Launch Essentials ™
Plan and execute a successful product launch
Are your product launch efforts focused on deliverables rather than results?
Launching a product is more than following a simple checklist. A successful product launch is the
culmination of many, carefully planned steps by a focused, coordinated team. Even good products can
fail because of organizational issues, misunderstanding of roles and responsibilities, and a lack of a
strategic approach to guide efforts.
• Learn a repeatable product launch process to shorten the launch planning cycle, get the resources
needed, and know what to expect at every step.
• Understand the seven product launch strategies your team can use
to maximize sales velocity.
• Measure product launch progress with indicators that identify unforeseen
issues before they become big problems.
Get a free e-book at Daniel
Download a complete agenda and register at PragmaticMarketing.com/seminars
Call (800) 816-7861 to conduct this seminar at your ofﬁce
How did ProductCamps originate?
ProductCamps are based on the concept of
BarCamps, the first of which was held in Palo
Alto, California in 2005. It’s a reference to the
hacker slang term, foobar. BarCamp arose
as a tongue-in-cheek spin-off of Foo Camp,
By John Milburn and Paul Young an annual invitation-only participant-driven
conference hosted by open source publishing
luminary Tim O’Reilly. Where FooCamp was by
invitation only, BarCamp would be democratic
ProductCamps are taking the world and open to all. BarCamps have been held
in over 350 cities around the world, and even
by storm and have sprouted up virtually over the Internet.
in Silicon Valley, Austin, Toronto, The first ProductCamp (aka P-CAMP) was
held on March 15, 2008 in Mountain View,
California. Word-of-mouth spread fast about this
Amsterdam, Sydney and more. If you “unconference” (no registration fee, no agenda,
no selling) and about 170 people attended on a
haven’t participated in one, you might Saturday to discuss topics of interest to product
managers, product marketers and a host of
wonder what all the fuss is about. If you related roles.
have participated, you already know! Following this event, many participants
commented via blogs, wikis, and tweets.
The idea went viral, and other cities started
ProductCamps of their own. Austin was first
out-of-the-gate with others following over
subsequent months in 2008, 2009 and into 2010.
Given the proliferation of ProductCamps, we
thought it would be useful to summarize some
best practices and hopefully these guidelines
will help you organize or participate in an
unconference in your local area.
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 5
ProductCamp Best Practices
Why attend ProductCamp?
Has your travel budget been cut? Do you want to
learn from peers outside your company? Are you
looking for informal ways to “meet-and-greet”
others? Do you enjoy presenting or leading
roundtable discussions on timely industry topics?
Are you unemployed or under-employed, and
want to increase your network? Do you want to
meet others who are passionate about product
management and marketing? If the answer is
“yes,” to any of these, you should participate
Why organize a ProductCamp?
Have you ever attended an industry conference
or company event and felt like you could
have organized it better? Do you feel industry
conferences have become too “sterile” or
dominated by big vendors? Are there a lot
of knowledgeable product management and
marketing professionals in your area, but no
good forum to get together? Would you like
recognition in your community as someone that Many ProductCamps use an
“gets things done”? Dozens of your peers around
the world are planning ProductCamps for these “Open Grid” to set the agenda. Presenters
and roundtable facilitators submit topics
Since ProductCamp is free to attend and typically prior to the event, or sometimes even on
held on a Saturday, it’s a low-risk way to expand
your skills and personal network. Plus, since the morning before the event starts.
it’s “user-sponsored,” it will not be a thinly A final agenda is created real-time,
veiled sales event—participants are strongly
discouraged from selling or promoting products. by and for attendees.
6 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
ProductCamp Best Practices
ProductCamp guidelines A typical day begins with attendee registration and
breakfast. Then participants are led to a “main-tent”
Several common themes have emerged from session that explains the format, rules, and introduction
ProductCamps held to date, though the flow and of ProductCamp planners—but, more importantly, the
tenor may change from city to city. Since it’s an opening session sets the tone that ProductCamp is by
unconference, there are very few rules. The general and for attendees, open and discussion-oriented, and
guidelines (adapted from BarCamp) are: most importantly, fun.
• ProductCamps are free, but there is a “cost” to In today’s economy, several ProductCamps have offered a
attend. There are no passive attendees. Instead, few minutes for anyone who has job openings to present
everyone is referred to as a “participant” and expected a very short overview of the position/company.
to contribute in some way: presenting a session,
coordinating sponsors, managing the venue, ProductCamps use an “Open Grid” to set the agenda.
volunteering for setup and teardown, or sitting Presenters and roundtable facilitators submit topics prior
in a discussion panel—there are many ways to the event, or sometimes even on the morning before
to participate. the event starts. A final agenda is created real-time,
by and for attendees. Session leaders are sometimes
• Information sharing is expected and encouraged. given 30 seconds in the welcome session to introduce
Everyone is urged to share event information and themselves and their topic. Then it’s time to vote! One
experiences, both live and after the fact, via blogging, popular technique is to give each participant three sticky
photo sharing, social bookmarking, tweeting, and notes as they arrive. Volunteers post session topics in a
wiki-ing. This open encouragement is in deliberate central area and ask everyone to place the sticky notes
contrast to the “off-the-record” and “no recording” under their top three choices. Sessions receiving the
most votes are plotted on an “agenda sheet” in such
rules of many conferences.
a way to minimize topic conflicts. Based on interest
• The “2-foot” rule applies: ProductCamp typically level and available time slots, some topics may not
features concurrent sessions. If you feel you’ll get make the agenda. It’s open, it’s participant-driven,
and it really works!
more out of a different session, use your own “two
feet” to move—no one should feel tied down.
Sessions usually run 45-50 minutes with a break for
• Networking is good! ProductCamps are held on lunch and plenty of “slack time” for informal networking.
a weekend and go all day, so participation is a Most ProductCamps reserve a few meeting rooms for
commitment. The sacrifice in time and energy means parallel sessions and list four to six time slots. The
only those who are really interested attend. best ProductCamp sessions are facilitated roundtable
discussions—leveraging participant knowledge.
Venues typically provide basic services. Free internet Notes, videos and photos are posted or linked
access and Wi-Fi are crucial. Sponsors provide funding to the ProductCamp website.
for “nice-to-haves” such as t-shirts, meals/snacks, signage,
lanyards, etc. Attendance is free and usually restricted It’s always fun to bring everyone together at the end
only by venue capacity. Due to the popularity of these of the day for a quick summary and to award prizes for
events (and to assist in planning for meals, t-shirts, etc.) best sessions. By this time, the group is comfortable and
advance registration is often required. can get rather lively celebrating the hard work and start
planning the next ProductCamp.
ProductCamps have been held at university-
donated facilities, professional learning
centers, and corporate offices.
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 7
ProductCamp Best Practices
Tips for running ProductCamp Marketing • Have name tags printed for those
who pre-registered, blanks for
Don’t over-plan! Think of it as a • Create a website and Twitter walk-in registrations (remember to
fun way to get together and meet hashtag. Optional: create a leave space for your participant’s
others to learn and network—not a Facebook page and LinkedIn Twitter ID!). If you can, add
typical conference where events are Group. If you have a local Product
pre-planned. Structure is good, but session break and lunch times
Management Association, ask to the back of the name tag.
too much defeats the purpose.
them to post the event on their
site and in newsletters • Parking directions/assistance
Make registration very easy. People (if needed)
won’t register if they have to jump • Provide regular e-mail updates
through hoops—email address is to registrants • Cleanup—after meals and at the
the only personal information really end of day
needed at this stage. • Get mentions in relevant
newsletters, blogs and websites. • Hall monitors—between sessions
If you have trouble getting sponsors, Contact local media (newspapers, to answer questions and help
sacrifice the t-shirts and breakfast. business and tech journals, shepherd people where they
TV, radio) need to go
Assign a “go-to” person who is very • Session monitors—make sure there
organized and able to delegate— Volunteers is a timekeeper and note-taker in
not to do all the work. The time each session (doesn’t have to be
commitment varies (more time is • Initially just collect names but a
one of the organizers, have the
needed closer to the event). critical role on ProductCamp day
presenter/moderator ask for
• Set up registration table for sign-in, volunteers)
There are five key areas that need
t-shirt, sponsor give-aways, any
leadership, coordination, and • Encourage people to take pictures
accountability to ensure a successful information sheets, etc.
(a camera phone is often good
ProductCamp: enough) and post to Twitter or the
event website or a Flickr page.
Use the Twitter hashtag as a
“keyword.” A dedicated
photographer is a plus.
Don’t over-plan! Think of it as a fun way
to get together and meet others to learn
and network—not a typical conference
where events are pre-planned.
8 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
ProductCamp Best Practices
Sessions Venue If you need help organizing
• Encourage people to sign up to • Coordinate building open/close a ProductCamp, please let
present or host a roundtable times, security and parking Pragmatic Marketing know. We
• Ensure everyone understands • Visit the facility the day/evening have been involved with every
the “Open Grid” scheduling before the event to ensure ProductCamp and can provide
everything is ready sponsorship, promotion and get
• Collect and post presentations
and notes from the sessions to • Set up rooms including projectors, involved with planning teams
the event website or a site like chairs, whiteboards, etc.
as a resource for questions
• Manage details about the food— and guidance.
• Coordinate the “best presentation” set up, trash cans, location,
voting and awards cleanup
• Conduct a post-ProductCamp
Budget John Milburn has “walked the walk” in technology product
management and is an instructor at Pragmatic Marketing.
• Get financial commitment from
Throughout his 20+ year career, he managed or delivered
sponsors over 40 hardware and software products and implemented
• Manage budget and pay bills the Pragmatic Marketing Framework at countless companies.
(ideally, set up a 501(c)(3) and
open a checking account) John is on the planning team for the Austin ProductCamp
and has helped leaders of many ProductCamps. Contact John at
• Order and pay for food, name firstname.lastname@example.org
badges, signs, t-shirts, lanyards
Paul Young is a product management professional with
more than a decade of experience in hardware, software,
and services product management and marketing. He
is an instructor at Pragmatic Marketing. He has also
held a variety of product management and marketing
roles in technology startups and Fortune 50 companies
such as Dell, NetStreams and Cisco Systems.
Paul is founder of ProductCamp Austin and has helped leaders at other
ProductCamps. Read his blog at productbeautiful.com. Contact Paul at
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 9
Living in an Agile World™ Requirements That Work™
Strategies for product management Methods for creating straightforward product
when Development goes agile. plans that product managers can write and
Practical Product Management®
Principles of the Pragmatic Marketing Framework, Effective Product Marketing™
the industry standard for managing and marketing Repeatable, go-to-market process to design,
technology products. execute, and measure high-impact marketing
Techniques to plan, consolidate and communicate New Rules of Marketing™
product strategy to multiple audiences. Reach buyers directly, with information they
want to read and search engines reward
with high rankings.
Product Launch Essentials™
Assess organizational readiness and define team
Executive Briefings responsibilities for a successful product launch.
Designed specifically for senior management,
Executive Briefings discuss how to organize
Product Management and Marketing
departments for optimal effectiveness
In addition to the extensive published schedule, training can be conducted
onsite at your office, saving travel time and costs for attendees, and allowing
a much more focused discussion on internal, critical issues.
Pragmatic Marketing’s seminars have been attended by more than
60,000 product management and marketing professionals.
Auditing Product Management in
Early-Stage Companies By L. Kurt Reiss
Many product managers like to work in startup
companies. I’ve been one of them. Why? Without
doubt, a product manager in a startup has
an opportunity to make broad-ranging and
meaningful contributions to product success,
fast. But, life in that lane is not easy.
Early product definition tends to be quite
elastic, leaving room for creativity. Corporate
structure in a startup tends to be flat, allowing
exposure to serious and far-reaching business
and technology problems that impact decision
making. That’s all good. However, one of the
major difficulties is confusion. It’s all in flux.
Yes, broad-stroke formulations of product vision
and business concept do exist, the ones used to
raise funds for the startup. But almost always,
even these fundamental statements change
with time. Flexibility is good but it’s hard to
succeed without a firm grasp of reality and clear
articulation of objectives, decisions, success
factors and progress metrics.
How do you deal with this “mess” of moving
targets and reference points? The tool that has
not been broadly used but can be of great help
is the product management audit. What is a
product management audit? Let’s take a closer
look, but first, I’ll highlight key issues related to
product management in a startup. Then, we’ll
define the specific problem we propose to
address here and explore an example of an audit
meant as a remedy for this problem.
12 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
The problem we aim to bring to market, and when. And, it’s a matter
of mutual understanding among all the managers in the
The three issues that hold keys to product success and company what these requirements are. Are we building
frequently, create problems in a startup are strategy, what we think we’re building?
market and customer needs, and product requirements.
So are these three issues the problem we address here?
Strategy is extremely important for any endeavor, and Not quite. Entrepreneurs usually know these issues
especially for a startup, both the corporate strategy are important and create problems if not addressed.
and the product strategy. The issue here is two-fold: However, in a startup, people are very busy doing a lot
Is our corporate strategy really sound and clear—as of different things. Moreover, entrepreneurs want to be
articulated—and is its impact on product strategy optimistic, driving to success by focusing on the possible
well understood? Do all key people in the company rather than on the difficult. This combination doesn’t
understand and agree on the current articulation of leave much room for rigorous self-assessment.
The problem often is these three key issues are glossed
Target Market and Customer Needs define, almost over. We tend to think all three are in good shape,
completely, product requirements. Thus any lack and justify these thoughts by superficial considerations
of clarity here will soon lead to difficulties with and wishful thinking. We also want to believe what we
the product. Have we fully decided who our target preach to the point of tweaking the facts to fit our needs.
customers are and truly verified the scope of Although drinking your own Kool-Aid may be enjoyable,
overlap between what they need and what we it usually leads to an upset stomach.
plan to deliver? It’s amazing how often the answer
is: “Sort of, but not really.”
The solution is a product management audit—a practical
way to counter neglect and false optimism and focus on
Product Requirements are key for a startup, thoughtful decisions and a justified positive outlook.
driving both development work and early product
promotion. No it’s not a matter of a polished
text document. It’s a matter of having thought
through and decided on the kind of product
Key Product Management
Issues in a Startup
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 13
Auditing Product Management in Early-Stage Companies
Key Factors in the Audit
fid bi lity
e p o nsi
The solution: a product management audit
What is a product management audit? First, it’s a
review. An analysis of the state of everything related
to product management. The goal is to step out of the
daily grind and take an unbiased look at issues, and
grade the progress achieved towards solving each of
them. It has some similarity to a financial audit. It’s a Can an audit yield easy-to-understand-and-track metrics?
step by step, “by the book,” analysis of everything that Yes it can! And just two metrics will do. An “average”
impacts product management. It’s a short, intense, and score (call it the A-score) and the “fix-it-now” score (the
repeated effort. F-score). The A-score is a geometric average score of all
factors under consideration. And the F-score is the lowest
What’s the typical framework of a product management score of all individual factors.
audit? For each issue related to product management,
one should take a critical look at: Why is geometric average the right one to use? Think of
a car’s tire pressure. If three wheels are at 100% but one
• Facts: What’s known / not-known? is at 0.01%, the arithmetic average (75%) says “it’s still OK
• Confidence: How confident are we to handle problems to drive” but the geometric average (10%) says “fix the
tires now!” Clearly, you need all wheels in good working
related to the issue?
order to succeed.
• Responsibility: Who’s responsible for resolving
each of these problems? The good thing about having numerical scores is they
help track progress. They also help identify the bad
That’s the minimum, but a fundamental one. Of course, outlier factors that need fixing immediately.
there are ways to augment the review by considering
objectives, teams, resources, timelines, plans, etc. I would .
argue against taking it on. Always make sure the basics
are sound before diving into the sea of details. Address
details as a separate exercise.
14 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
Auditing Product Management in Early-Stage Companies
0 20% 40% 60% 80% 100%
Example scoring of Strategy Total score for Strategy in this example: A=47%, F=10%.
Facts: Consider three strategy-related factors that can
be scored with the good-bad percentage metric. For Total Strategy Scores
example, our strategy statement is 80% completed and
articulated (80% good, 20% bad). And, 75% of our F-Score
external strategy reviewers approve of the risks we’ll be
exposed to and bets we have made. And, 20% of our A-Score
managers understand and accept our strategy. That yields
A=49% and F=20%. That’s very bad. 0 20% 40% 60% 80% 100%
Strategy Facts Scores
These are dismal results, butStrategy Scoresthing is: they
Total the good
clearly point to areas where improvement is needed.
Completed Example scoring of Target Market
Reviewed Facts: We know user needs of 80% of the target market
segment. We have articulated competitive win scenarios
Aligned for 20% of anticipated competitive bid types. 100% of
our identified target adopters are committed to do beta
0 20% 40% 60% 80% 100%
testing of our product. A=54%, F=20%.
Confidence: We are 90% confident that our value
proposition is compelling to our target customers. We are
80% confident customer needs will not change before we
Confidence: We’re 80% sure our strategy will not need ship the product. We are absolutely (100%) sure of our
to be modified before we ship the product. And, we’re target market size. A=90%, F=80%. Excellent result—and
50% sure our business can survive bad outcomes of our therefore solid scrutiny is needed to verify this score.
bets. And, we’re 100% sure we will not run into any
technology showstoppers. A=74%, F=50%.
Responsibility: We have individual owners for 50% of
win scenarios that are still missing. A=F=50%
That’s not too bad but not good enough.
Total score for Market: A=67%, F=20%. Better than
Responsibility: For 10% of our still-open strategy issues
A-Score Strategy, but still far, far below target.
we have an individual identified who’s responsible for
closing them. A=F=10%. 40%
0 20% 60%
Really bad! 80% 100%
Total Strategy Scores
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 15
Auditing Product Management in Early-Stage Companies
Example scoring of Product Requirements Audit template
Facts: The Product Requirements Document (PRD) Whether you decide to do it or arrange for an
is 100% completed and reviewed. 90% of individual independent audit, it’s good to have a solid
requirements have defined priorities associated with understanding of how it works. There are three key
them. The review found 80% of these requirements aspects of an audit: First and foremost, it’s the scope:
are clear to reviewers. A=90%, F=80%. Good. the list of questions or factors to examine and review.
Second, it’s the mechanics of organizing results and
Confidence: We are 60% confident that requirements computing metrics. And finally, but perhaps most
define all factors critical for winning against competition importantly, it’s the veracity and validity of the audit.
(Does this agree with the estimate for competitive A template shown on the next page may be of assistance
wins in the Market section? What if it doesn’t?). We for the first two. As for the third, it’s a matter of trust
are 90% confident that priority-1 requirements must be in the person(s) doing the audit.
satisfied before the product ships (Which ones are the
questionable 10%?). A=73%, F=60%. Still needs fixing. Questions can be many and varied, but not all of them
are the “right” questions for a product management audit.
Responsibility: We have one owner for 100% of the What are the right questions? They are largely defined
requirements that need better prioritization and/or by the intersection of the two sets of topics discussed
clarification. For 20% of items that need clarification, earlier: “strategy-market-requirements” and “facts-
we have identified reviewers that will decide if they confidence-responsibility.” The ones in the preceding
have been sufficiently clarified. A=45%, F=20%. Bad. section could be a good starting point. How many
questions to ask? As few as possible that are the most
meaningful in your situation and answerable with a
Total score for Requirements: A=69%, F=20%. Well, this numerical (percentage) metric.
Some questions can be answered with a precise
And finally, aggregating all scores, the total Audit (countable) number. For example: “What fraction of
score for Product Management in this example is: individual requirements in the PRD have priority
A=60%, F=10%. assigned to them?” Others can’t. For example: “What is
our level of confidence in factors that will assure our
Total Audit Scores wins in competitive bids?” Still, a rough percentage
estimate can be defined and be useful. Just make sure
the audit captures and considers not only the estimate
but also its source.
A-Score Going beyond the audit questions used in the example
above, other fact-related questions may include:
0 20% 40% 60% 80% 100%
• What fraction of our value proposition is uniquely
• What fraction of our efforts brings a distinctly new
I wouldn’t invest in this example startup until these value to the market?
numbers are much, much higher. Can they get better?
Total Audit Scores
Of course they can. Once you know—as illustrated by • What fraction of our target market could materially
the audit results—what needs fixing, it’s easier to focus benefit from the full set of features of our product?
your energy and improve the situation.
• What fraction of the cost of the solution implementation
by our customer will be due to the purchase of
16 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
Auditing Product Management in Early-Stage Companies
• What fraction of our product requirements directly map find it difficult to deal emotionally with someone
to our value proposition? else “checking” the validity of their own vision
and perception. Weak entrepreneurs may even fear
• What fraction of our product requirements represents less-than-stellar results of an audit may negatively
new and differentiated value? impact investors’ view of the company. It is possible
that some companies would prefer not to do such
All of these questions are the “right” questions for the audits. But, the value of an audit is not in finding fault,
audit. That’s because for each of these questions, a high it is in confirming that everything is fine and if not, in
percentage-score answer represents a “good” answer. identifying areas that need attention. The audit results
The higher the score, the better. present an independent opinion, a tool facilitating good
decision making for product management purposes.
Mechanics of the audit can be supported by a
spreadsheet, like the one shown below. The auditor You can do an internal audit but it’s best to rely on
can complete the “Questions” fields and associated “%” someone external. Just think about a due diligence
estimates, as the audit proceeds. process a startup goes through prior to a round of
funding. Would an investor ask the company itself to
The example above and the template need a fair amount perform such due diligence? Would you rely on the
of specificity. The main thing about the Audit is doing analysis for fund-raising purposes to actually drive your
it, applying serious qualitative analysis and using the business? The #1 objective of that analysis was to raise
score as a tool, first for critical thinking, then for tracking funds and that calls for a level of optimism that may be
progress—until the next audit. dangerous in day-to-day business operations.
Who needs (and wants) Questions Scores (%)
Certainly top executives in the
startup should want it, as should
the board and investors. If there
is one, so should the product Responsibility
manager. Best startups may not
need product management audits—
all’s working well there. But the vast
majority of startups would certainly
benefit. What if a company does Confidence
not yet have a dedicated product
manager? Often, the founder drives Responsibility
early product definition and its
creation. However, the founder Facts
does not usually have the time
and inclination which often leads REQ
to unjustified optimism about the
product in the market and vague
product requirements. An audit Responsibility
can really help.
POWER(PRODUCT(STR),1/COUNTA(STR)) = A-score for Strategy =
Can it possibly happen that top
MIN(STR) = F-score for Strategy =
executives of a company would
POWER(PRODUCT(MAR),1/COUNTA(MAR)) = A-score for Market =
not want a product management
MIN(MAR) = F-score for Market =
audit? Entrepreneurs are typically
POWER(PRODUCT(REQ),1/COUNTA(REQ)) = A-score for Requirements =
strong willed and may simply
MIN(REQ) = F-score for Requirements =
POWER(PRODUCT(TOTAL),1/COUNTA(TOTAL)) = Total A-score =
MIN(TOTAL) = Total F-score =
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 17
Stay current with industry best practices
Review 10 years of
Annual Product Management
and Marketing Survey results
Watch a webinar by one
of today’s industry experts
Read articles on product management,
marketing and leadership strategies
Subscribe to blogs by
Pragmatic Marketing thought-leaders
Participate in online
networking through LinkedIn,
Facebook and Twitter
Purchase books, clothing
Is Product Management
Needed In a
By Gopal Shenoy
Startup—the early days
A startup is launched when an entrepreneur—a software engineer,
marketer, product manager or salesperson—comes up with an idea
based on past employment or personal experience. The idea could
have been triggered by observations made during sales calls, opinions
expressed by current customers, high volume of prospect inquiries or
customer support calls, personal experience using a technology,
etc. The entrepreneur gets excited about the potential of the idea
and decides to create a product to solve this problem. The typical
questions that consume the founder’s mind include “Who can help
me create the product that will bring this idea to life?”,
“How can I get this done as soon as possible
before someone else does?”, “How can I sell
this and make millions?”
There is a very high sense of urgency at
Is product management the beginning to do something. However,
needed during early stages urgency during the early days is a
double-edged sword. While needing to
of a software startup? be the first to market has its advantages,
rushing to market without doing enough
What benefits can product homework is a sure fire recipe for
management bring to the failure. There is a lot written about why
startups fail. All of them point to the
table during the early days simple fact that most startups build
something nobody wanted. The
of a fledging business? entrepreneur either gets enamored
with a new technology (especially
if the entrepreneur is a software
Before we delve into the engineer or has a technical
need for software product orientation) or a radical
new product (especially
management in a startup, if coming from sales),
let’s look at the typical but then end up never
finding enough customers
genesis of a startup company to buy the product or
and what transpires between
start and first release
of the product.
20 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
Fundamental questions a startup Product pivoting and What skills will this person need?
(or established companies venturing the first release
into new markets) should ask before • Ability to dig into what is stated
building a product or service are: If the early discovery process
by the customer and identify
indicates a potential market, the
startup needs to quickly build a underlying problems.
• What are the problems that exist
minimal product and get it in the • A business orientation to maintain
in the target market?
hands of a few customers who have the big picture of what the
• How many potential customers the problem. Once the customers
company is trying to do without
have this problem? start using the product, someone at
the startup needs to monitor usage, getting caught up in details of the
• How are potential customers be in constant communication product. The business model,
solving the problem today and with these first customers to get target market, and pricing are all
how dissatisfied or frustrated are their feedback and work with the undetermined at this time.
they with the current solution? engineering team to make changes
(pivot) to improve the product. All • Great communication skills to
• Is the frustration large enough that of this should happen at a very fast understand what the customer is
these customers will be willing to pace. Process is not what matters, stating and converting it into a
pay to solve this problem? flexibility is. language engineers understand.
• Is what the customers are willing • Project management skills so to
The sole objective during this prioritize and ensure things don’t
to pay large enough to build a period is to pivot enough times
profitable business? fall through the cracks.
to quickly identify the business
model and final product for the It is extremely important this person
• What should the solution target market. This speed requires be a member of the founding team.
look like? someone focused on this task and You can call this person a product
You will notice that all questions who serves as coordinator between manager but that’s less important
but the last one, are about the customers and engineering. This is than their primary responsibility
market. None of them are about a critical phase and any misstep can to focus on the market and ensure
what technology platform should be catastrophic, resulting in failure the product creates a sustainable
be chosen, what feature set should of the entire business. business. Product-oriented founders
be in Release 1, or how many are typically well suited for this role.
engineers the startup needs. They
are completely independent of
the solution but instead focus on
discovery of market problems and
determination of market size.
While those are great questions to
ask, it is typically not possible to
get complete or accurate answers to
these questions quickly, resulting in
a dilemma—if it is not possible to
find correct and accurate answers to
all of these questions before getting
started, and if the chances of failure
are very high if you do not get the
answers you need, what gives? This
is where product pivoting comes
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 21
Is Product Management Needed In a Software Startup?
If the founding team does not have Even if such a person exists in the • Developers have so much to do
someone with a product orientation, company, here are some reasons that piling on market research is
then it becomes critical that such that make it hard to play dual roles: not a good idea.
a person is brought on board.
This is a tough sell, especially to • Salespeople are inclined to
• Delving deep into customer
those funding the startup with no overcome objections in order
problems is an art not a science.
product. It is much easier to justify to meet their numbers. They
hiring developers or salespeople. Customers often propose solutions
are not trained to understand the
But to create a sustainable business, and it takes skill and effort to
underlying problem, relay it back
it will take product management unearth the real underlying
to the product team and get it
discipline to ensure early market problem.
fixed. Sales is all about the
success continues to create
a growing business. • It is easy to confuse “what the last “here and now.”
two users told me” with “this is
• Early customers need someone
what the entire market wants.”
Some may say the call for a product they can bounce ideas off and
management orientation in a startup • Maintaining an objective opinion someone they can engage with to
is based on false assumptions that when the customer says your build trusted relationships. It is
Development or Sales or Marketing “baby is ugly” (first version of these early customers that turn
are not capable of doing market into references and proponents
your product) is difficult,
research. After all, aren’t we
especially when you have spent when the product starts its
required to wear multiple hats
in a fledgling startup? sleepless nights building it. It is journey into the mainstream.
often difficult to listen to and
understand the customer’s Past the early success
point of view.
Once the company establishes
the viability of the product, creates
the business model and has its
first group of paying customers,
it’s time to assign someone whose
primary responsibility is to take
the early product and transform it
into one designed to appeal to a
wider market. This requires full time
effort and focus. Startups at this
stage of growth must make hiring a
seasoned product manager a priority
and long-term investment.
22 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
Is Product Management Needed In a Software Startup?
“Startup” product manager • Someone who has previously built necessary because often what you
qualifications teams and can grow your product discover in the market requires
management team once the you to change internal opinions
Based on my experience working in company hits the next milestone. and points of view. This requires
startups, I recommend the following Someone you can trust to manage the ability to question currently
criteria when hiring your first what justifies the company’s very held beliefs and opinions
existence—your product and recalibrate based on
or service. market realities.
• Someone who has previously
worked for a successful startup • Someone who can balance a How important is domain
and gone through this critical strategic and tactical role. It is experience? It depends on your
“next” phase. equally important the product market segment. In highly technical
manager lead discussions on fields such as computer networking,
• Someone who has previously product/business strategy and deal virtualization, cloud computing,
worked at least one failed startup CAD, space exploration, network
with day-to-day tactics to get the
and who is able to articulate the security, pharmaceuticals, medical
product out the door. This person services, financial services, etc.,
lessons he/she learned from that should have a keen sense for domain experience will be of
experience. What mistakes were product usability, performance and paramount importance because you
made and why? I personally value functionality because eventually do not have time to teach someone
a person who has failed at least these have major influence on new to the industry the technology,
once over someone who has only product success. jargon, regulations, etc. But in other
had past successes. general areas such as ecommerce,
• Someone you can put in front hire product managers who have
• Someone who can strike the right of customers. Your “product a breadth of experience.
balance between process and manager” should be one who
flexibility. Someone who can enjoys and thrives on customer It is essential that startups have
function without regimented engagement. One who is a good a strong product management
processes, but one who can also listener, can engage in active orientation right from the start.
rein in the chaos so common in dialogue with prospects/customers It is not necessary that the person
startups. As the company grows, and your customers feel have the official title of a product
add process into the mix to comfortable with. manager, but it is of utmost
scale efficiently. importance that someone from the
• Someone your organization founding team has the primary
• Someone who shows immense will respect. Given the product responsibility of doing product
passion and drive. Someone who manager will be ultimately management. Discovering unsolved
can work with very limited responsible for bringing the problems in a target market,
information and takes the voice of the customer into the building a product this target market
initiative. You absolutely need organization, this person needs is willing to buy and figuring
someone who has the energy, will out a business model that helps
to win the respect of internal
and drive to convert early success the company create a sustainable
stakeholders. You also want business is not a part-time effort.
into sustainable growth. someone who can stand up to
the internal stakeholders when
Gopal Shenoy is the Director of Product Management at Gazelle.com where he leads the product
management effort to allow consumers to sell or recycle used electronics. He is a seasoned product
management leader with over 14 years of experience in startups and small businesses. He has led
direct report and cross-functional teams that defined and launched highly successful SaaS-based
and enterprise software products. His passion for product management led him to create a Software
Product Manager blog at productmanagementtips.com. He is also a speaker at ProductCamps and
other software product management conferences. Contact Gopal at email@example.com
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 23
One of the common questions I am act more like a laser pointer that
asked when discussing win/loss illuminates very specific information
analysis is, “How many deals do I about very specific issues and
need to analyze before I can start questions. In other words, to get the
using the data?” In the past, my most out of your win/loss program,
response has always been that you it should not be seen as a survey tool
will get value from the very first win, (which simply gathers information),
loss, or no decision, you analyze. but rather a decision-making tool
However, because statistics teaches and an early warning tool.
us about minimum confidence levels,
confidence intervals, and margins of If you think you can’t get value from
error, people often struggle with the your win/loss program, or believe
idea that they can start leveraging you can’t begin making decisions
the data from their win/loss program
based on win/loss data until you
from the very first data point. achieve a statistically significant
number of data points, our research
shows you risk missing as much as
Get away from
75 percent of the potential ROI you
the “survey” mindset could achieve with your win/loss
analysis program. The key is to
By just listening to the terminology understand how and when to use the
executives use, it is obvious intelligence gathered through your
many initially base their vision win/loss analysis.
of a win/loss program on their
prior knowledge of market
By Ken Allred research or political polls. For
instance, the term “survey” is
The decisions executives
often used to describe what is and managers make
actually an in-depth “interview.” To determine how many interviews
Unfortunately, this market research- are needed for a win/loss program
based mindset often diminishes the to be effective, it is important to
usefulness of a win/loss program understand the kinds of decisions
and gives the false impression the your organization makes and to
program will need hundreds (or understand how win/loss analysis
even thousands) of data points can help you make better decisions.
to be accurate or useful.
Professor Robert A. Harris, in his
Certainly, if you were to use a VirtualSalt blog, outlines three major
win/loss program to determine types of decisions: strategic, tactical,
the general buying habits of and operational.
your buyers (or the populace as
a whole), you would want to
interview a large number of Strategic decisions determine the
random customers in order to long-term direction of an enterprise,
achieve a representative sample product, or department. They
and a high level of statistical also tend to direct and control an
confidence. However, organization’s resources. Strategic
working with hundreds of decisions involve a high degree of
companies over the past ten uncertainty due to the scope and
years has shown that the time frames usually associated with
most successful win/loss these types of decisions.
analysis programs do not
try to cast such a wide
net. Instead, they
24 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
Real-world examples of win/loss Operational decisions support The number of data points needed
analysis that helped clients make tactical and strategic decisions. They to support accurate decision-
better strategic decisions include: help to ensure that daily operations making increases as you move
go smoothly and that tactical from operational (which is often
• Competitive losses indicate buyers decisions are implemented. As the immediate and one data point
name indicates, these are usually can be sufficient) to strategic,
− want a vendor with strong off-
made by managers, although they which may require a large amount
shore capabilities in two specific can also be made by executives. of information to get a clear
countries picture of the entire situation. For
− had issues with key contractual Real-world examples of win/loss instance, deciding which paper
program intelligence that help clients supplier to use for a direct mail
terms the client requires of
make operational decisions include: campaign (an operational decision)
new customers would probably not require client
• Buyers in both wins and losses interviews, but if you wanted to
• A buyer in a lost opportunity change the focus of your company
describe a specific set of needs indicated from services to software (a strategic
that aren’t currently addressed
− the sales representative poorly decision), it is in your best interest
in the market and warrant to interview a representative number
development of a new product presented the solution and did
not understand how their product of current clients to determine how
would solve the buyer’s problems they would react to such a change.
Tactical decisions support or
achieve strategic decisions. In other − they didn’t select the client
words, tactical decisions are made primarily because a product Win/loss as a decision-making
to help bring about the results feature was not available, but the and decision-support tool
envisioned in the organization’s client’s product actually had the
overall strategy. As such, they tend feature From my experience over the
to be more immediate than strategic last ten years, I found that win/
− they are still evaluating solutions loss analysis provides most benefit
decisions, and can be made at
both the managerial and executive and would like to hear from at the tactical and operational
level, whereas strategic decisions the client again (this situation level, although, certainly, it does
are almost exclusively determined happens more often than provide tremendous insight at the
by executives. you think) strategic level as well. It just takes
• A buyer in a won opportunity a little more time before you can
Real-world examples of win/loss indicates they were having trouble start to lean on your win/loss
program data that led clients to with communication and it was program when making long-term,
make tactical decisions include: having a negative impact on their strategic decisions.
• Competitive wins indicate a key On average, we find for each
factor in buyers’ decisions was the win, loss, or no decision interview
presence of superior sales performed, you can expect
engineer support to identify an average of ten
operational or tactical decisions
• Buyers in competitive wins talked that can be acted upon.
about specific sales messaging and
sales collateral that helped them
make their decision, while buyers
in lost opportunities did not
reference these things
On average, for each win, loss, or no decision interview performed,
• Buyers in losses cited over-
confidence of the sales you can expect to identify an average of ten operational or tactical
representatives and disparaging decisions that can be acted upon.
the competition as key reasons
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 25
How Many Data Points Do You Really Need?
When you consider that each • What are the strengths and So, how many data points
analysis is going to help you make weaknesses of your competitors’ do you need?
an average of ten operational or products?
tactical decisions, it becomes clear The case I just described did not
very quickly that you can realize • How are your costs perceived require many interviews or data
great value from the very first in relation to those of the points for the win/loss analysis to
interview of your win/loss program. competition? have a positive impact—in fact, it
Just how much value does each of only took one to begin the decision
these potential decisions represent? Last year, when I was reviewing process. This illustrates you can
Let’s look at an example: one of some interviews we had done with and will receive value from the very
the most common operational our own buyers, I saw an issue first opportunity you analyze. And
decisions identified in win/loss emerge that definitely led to a as you add more data points, you
reviews are sales weaknesses for better tactical decision. One of our will receive further validation and
an individual sales representative respondents commented that their discover important, new insight.
that can be immediately addressed communication with us was almost
through focused training and/or entirely with the salesperson and
they did not have sufficient contact The point is: since win/loss analysis
coaching. If this training can help is a dynamic process, it can and
the sales representative avoid with the Account Consultant who
would be handling their program. should focus on those issues that
making the same mistakes are most important to you and your
in future opportunities, their Since one of our strategic goals is to
be a trusted consultant to our clients buyers. This means, unlike a static
individual win rate will improve. survey, fewer data points are needed
The impact of this one decision rather than just a provider of data,
I realized this was a significant issue. in a win/loss analysis to reach the
to help a sales representative with insights you need to make better
an area of weakness identified by After getting the same feedback in
two other interviews, I determined tactical and operational decisions
a buyer could be in the millions of to help you achieve your current
dollars, depending on the size of this was not an isolated case and
made the tactical decision to involve strategic goals and overall success.
your sales opportunities.
our Account Consultants and other
subject matter experts early in the Don’t wait until you have a
If done correctly, every win/loss “sufficient number” of data points
sales process. The results have been
interview will net key, actionable before you realize value in your
good, with our clients coming out
intelligence such as: win/loss program. Begin analyzing
of the sales process more informed
• What business needs are your and more committed. This tactical and using your win/loss data from
decision was made based on three your very first buyer interview
buyers trying to solve?
data points, but would have been and start seeing returns much
• What could you do to better serve entirely missed if we had not more quickly.
your buyers’ needs? been performing win/loss
reviews on our own
• What is the character and opportunities.
reputation of your company in the
eyes of your buyers?
• How well do your sales teams
communicate your value
Ken Allred, Founder and CEO
• What are the strengths and of Primary Intelligence, is a
weaknesses of your products? thought leader in SaaS-based
sales intelligence, analytics and
sales enablement solutions. He
is committed to the optimization
of sales, marketing and product management
teams through the implementation of
advanced Sales 2.0 intelligence solutions.
Contact Ken at firstname.lastname@example.org
26 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
Sign up for the home
Want Pragmatic Marketing
to come to you?
• Ensure everyone learns at the same level from
start to finish
• Focus on the material most relevant to your
• More of your staff can attend
• Discuss proprietary issues in complete
• Save money on travel expenses by having our
instructor come to you
• Your entire group will become more productive
• Everyone will use consistent methods and speak
the same language
• The skills learned are practiced during the
training session — all seminars are highly
interactive and encourage participation
Call (800) 816-7861 to conduct a seminar at your office
Call (800) 816-7861 or go to PragmaticMarketing.com to register!
Practical Product Management ® Requirements That Work™ Effective Product Marketing™
Introduces a framework that gives product managers the Provides a repeatable method for writing clear requirements Delivers practical tools and processes for product
tools to deliver market-driven products that people want your team will read and use. It discusses techniques for marketing, industry marketing and marketing
to buy. Focuses on the practical aspects of juggling daily prioritizing and organizing market requirements and communication managers who want to improve
tactical demands of supporting the channel with strategic clarifies the roles for team members. This approach their strategic contribution and align with the sales
activities necessary to become expert on the market. enables organizations to deliver solutions that sell. organization. Learn how to build a repeatable process
July 26 - 27 (28)* ............ Newark, NJ July 28 ........................Newark, NJ to develop, execute and measure go-to-market strategies
August 9 - 10 (11)* ........... Boston (Bedford), MA August 11.....................Boston (Bedford), MA that ensure product success.
August 9 - 10 (11)* ........... Toronto, ON, Canada August 11.....................Toronto, ON, Canada August 16 - 17 ...............San Francisco (Burlingame), CA
August 16 - 17 (18)* ......... San Diego, CA August 18 ....................San Diego, CA September 20 - 21 .........Raleigh (Durham), NC
August 23 - 24 (25)* ........ Denver (Littleton), CO August 25 ....................Denver (Littleton), CO October 6 - 7 .................Chicago, IL
September 13 - 14 (15)* ... Atlanta, GA September 15 ..............Atlanta, GA October 18 - 19 ..............San Diego, CA
September 13 - 14 (15)* ... Montreal, QC, Canada September 15 ..............Montreal, QC, Canada November 15 - 16 ..........Boston (Bedford), MA
September 20 - 21 (22)* .. Baltimore, MD September 22 ..............Baltimore, MD December 6 - 7 ..............San Francisco (Burlingame), CA
September 27 - 28 (29)* .. Minneapolis, MN September 29 ..............Minneapolis, MN
September 27 - 28 (29)* .. San Francisco (Burlingame), CA September 29 ..............San Francisco (Burlingame), CA
October 4 - 5 (6)* ............. Chicago, IL October 6 .....................Chicago, IL
October 4 - 5 (6)* ............. New York, NY October 6 .....................New York, NY Product Launch Essentials™
October 18 - 19 (20)* ....... Houston, TX October 20...................Houston, TX
October 25 - 26 (27)* ....... Boston (Bedford), MA October 27 ...................Boston (Bedford), MA AssessEffective Product Marketing
organizational readiness and define team
October 25 - 26 (27)* ....... Raleigh (Durham), NC October 27 ...................Raleigh (Durham), NC responsibilities for a successful product launch.
November 1 - 2 (3)* ......... Herndon, VA November 3 .................Herndon, VA
November 1 - 2 (3)* ......... Toronto, ON, Canada November 3 .................Toronto, ON, Canada August 18 ......................San Francisco (Burlingame), CA
November 8 - 9 (10)* ....... San Jose, CA November 10 ...............San Jose, CA September 22 ................Raleigh (Durham), NC
November 15 - 16 (17)* .... Vancouver, BC, Canada November 17 ...............Vancouver, BC, Canada October 8 .......................Chicago, IL
November 16 - 17 (18)* .... Austin, TX November 18 ...............Austin, TX October 20.....................San Diego, CA
December 13 - 14 (15)* .... San Francisco (Burlingame), CA December 15................San Francisco (Burlingame), CA November 17 .................Boston (Bedford), MA
December 14 - 15 (16)* .... Philadelphia, PA December 16................Philadelphia, PA December 8 ...................San Francisco (Burlingame), CA
*Day 3 is Requirements That Work
New seminar Living in an Agile World™
Shows how product management can ensure an Agile August 12 ........ Boston (Bedford), MA
A discount is available for groups of three or more.
development team remains aligned to company strategy. December 16 .... San Francisco (Burlingame), CA
Don’t see a date or location that works for you?
Let Pragmatic Marketing come to your company! From creating user stories grounded in market problems to
Call (800) 816-7861 for more information. managing a backlog prioritized with market evidence.