Iterating on Developer Marketing..with Metrics from DevRelCon Earth 2020

1. Iterating on Developer Marketing...with Metrics

2. 1
Table of Contents
2
3
Introducing myself and our community
Analyzing three case studies
Reflecting on lessons learned

3. 1 Introducing myself and our community

4. JonHey! I’m Gottfried.
● Co-Founder of Major League Hacking
● Formerly Twilio, StartupBus, Hacker Union, and more
● Yes, I do have a sourdough starter 🍞
@JonMarkGo
fb.com/JonMarkGo

5. Meet Major League Hacking (MLH).
We’re a Public Benefit Corporation & our mission is to
empower hackers.
Hackers join our community by attending events like
hackathons & workshops where they gain hands-on
experience solving problems with their peers. They learn
skills they aren’t getting in a classroom and build a strong
network of other hackers & employers.
100,000
Attendees.
1,200
Worldwide Events.
2.1 million
Hours Spent at MLH Events.
400,000
+Attendees.
~4,000
Worldwide Events.
7.3 million
Hours Spent at MLH Events.
Last Year: All Time:

6. Our customers are the world’s
leading technology companies.
Today, every company is a technology company. As a
result, there is an unprecedented need for developers.
However, demand is outpacing supply and companies
are struggling to stand out.
That’s where we come in. Companies partner with MLH
to scalably and authentically reach our audience for
recruiting & developer marketing. We help our
customers fill hundreds of open roles and onboard tens
of thousands of developers onto their platforms.

7. Willingness to learn, minimal established
preferences, and raw feedback make
students a valuable group to sample.
Students are a Great Sample

8. 2 Analyzing three case studies

9. Developer Onboarding
How we analyzed signup UX with our community to
increase conversions during onboarding.
Documentation
The impact of creating documentation based on
direct feedback from developers.
Messaging
How different ways of describing your product can
impact a developer’s perception and willingness to
try your product.
1
2
3
What we can
learn from
real world
developer
marketing.

10. Developer Onboarding
Goal: Get Developers signed up to
a specific promotional campaign.☁

11. A Proven CTA!
This pre-event email campaign had
a 60% open rate and a 20% click rate.
The audience had increased by 80%
YOY.

12. <50% successfully got started
Let’s measure their experience.

13. Failed verification notices came hours
after they filled out the signup form with
ambiguous explanations.
Problem 1: Bad Verification Flow
Metrics:
Retry requests, support tickets, self-correction rate.

14. Everyone is competing for a developer’s
attention.
If you create friction, they will move on.
Why is this a problem?

15. Give developers real-time feedback as they
fill out the promotional form.
How did we fix it?

16. The new signup form had almost
30 questions on it. People, quite simply,
were getting bored halfway through.
Problem 2: Arduous Signup Process
Metrics:
Time to sign up, churn rate throughout form.

17. Knowing your customer by collecting more
information can be at odds with the ease
of their experience.
Delight and inspire your customers, don’t
drain them.
Why is this a problem?

18. Eliminate excess questions and make a
higher percentage of questions optional.
How did we fix it?

19. People who made it through the form often
churned immediately because they selected
the wrong account type and couldn’t do
what they intended.
Problem 3: Confusing Account Types
Metrics:
Time to first API call, activity rate after signup.

20. Developers want to solve a problem, not
nitpick account features.
Developer experience matters as much in
billing workflows as it does in API calls.
Why is this a problem?

21. Provide clear signup paths for different
account types with explanations of exactly
what each account is for.
How did we fix it?

22. Conversion Rate Grew By 50%

23. Developer Onboarding
Takeaway: Core marketing metrics
should influence how DevX and
DevRel folks think about their
community’s experience.
Signing up should never be the
hard part.
☁

24. Documentation
Goal: Promote a new hardware
platform to developers.��

25. Bleeding Edge Tools
A new device, with all the features.
Powerful cross-platform compatibility.
Thousands of developers trying to use it.

26. 15% conversion rate
This number is important, but
let’s get qualitative.

27. “During the 7 or so hours we spent trying to
figure out how to set up the board…”
Qualitative Feedback

28. New systems shouldn’t just launch to the
world with nonexistent documentation.
Problem 1: New System
Survey Questions:
Tell us about your first experience with the platform.
What resources did you use to get started?

29. “It was very hard to find accessible guides to
the board. We struggled to set it up, and could
not find a comprehensive source for
troubleshooting.
Of course, this is mostly due to our lack of
knowledge in the field, and I'm sure someone
who knew what they were doing would have no
trouble.”
What did they say?

30. If you build it, they will not come.
Developers can easily sign up and move on
if a system lacks clarity or explanation.
Why is this a problem?

31. We wrote 20 single-spaced pages of basic
documentation on how to set up and get
started.
How did we fix it?

32. A lot of feedback had to do with self
doubt. This implies that the marketing
makes it sound simpler than it is, when
more hand holding is actually required.
Problem 2: Seems Simple, But Isn’t
Survey Questions:
Where did you get stuck?
What took the most time?

33. “Not much doc on what certain ports do or are
capable of, with more information I might be
able to do more intricate and complex designs
with it.”
What did they say?

34. Developers are technical, and highly
skilled. But they can’t read your mind.
Things that are obvious to one person are
cryptic to another.
Why is this a problem?

35. We created clear visuals as an onboarding
resource for developers.
How did we fix it?

36. Developers often wrote complex software
that ran perfectly but couldn’t get the
hardware to work with it.
Problem 3: Software Was Not Hard
Survey Questions:
What were you trying to accomplish?
How did it turn out?

37. “the power requirement felt rather high, and left
us unable to envision moving our project away
from a wall-powered source...But in terms of
features offered, if they worked it would really be
an outstanding device to hack on.”
What did they say?

38. Often problems with developer platforms
are hard to investigate because there are
so many points of failure.
Why is this a problem?

39. We created pre-configured OS images that
automatically set up most needed hardware
compatibility.
How did we fix it?

40. Conversion Rate Doubled

41. Documentation
Takeaway: Listen to your
community, look at qualitative
metrics.
Onboarding tutorials and
documentation are a core DevRel
discipline and product requirement.
��

42. Messaging
Goal: Introduce developers to a
new type of tooling.��

43. Build Enterprise
Dev Community
They have a powerful enterprise
platform, but need to get community
developers excited about it.
Build long-term brand awareness and
affinity, and create a marketplace.

44. Multi-Channel Promotion Tracking

45. “Try Our Multi-tenant Auto-scaling
Non-Relational Data Store”
Problem 1: Confusing Jargon
Metrics:
Email open rate, net new signups, survey results.

46. Industry jargon is often about semantics,
not use cases. Focus on inspiring and
exciting developers with the possibilities
of your product.
Why is this a problem?

47. “Build a better database into your
application.”
Keep it simple.
How did we fix it?

48. Problem 2: Wasted Visuals
Metrics:
Sharing rates, clickthrough, viewing time.

49. People are scrolling by. You need to
capture their attention quickly - in a
slideshow, in an ad, in an email campaign.
Make the most of someone’s attention.
Why is this a problem?

50. Fun, shareable and explanatory graphics.
How did we fix it?

51. Developers are making an investment in
your product for the long term. They have
strong, varied opinions that you might be
missing out on.
Problem 3: Missed Opportunities
Metrics:
Survey results, feedback emails, reply rates.

52. As evidenced in previous case studies,
sometimes all you have to do is ask.
A CTA to simply sign up could miss out on
relevant feedback or context.
Why is this a problem?

53. Not only ask people to sign up.
Ask them to email you.
How did we fix it?

54. Brand Preference Increased By 40%

55. Messaging
Takeaway: DevRel folks often
touch many aspects of a product,
and marketing copy and
engagement is key to that.
Avoid jargon, focus on broad
applicability and clarity.
��

56. 3 Reflecting on lessons learned

57. Developer Onboarding
How we analyzed signup UX with our community to
increase conversions during onboarding.
Documentation
The impact of creating documentation based on
direct feedback from developers.
Messaging
How different ways of describing your product can
impact a developer’s perception and willingness to
try your product.
1
2
3
What we can
learn from
real world
developer
marketing.

58. Watch Your Developers
Literally watch someone go through your signup form when
you have the chance, and take notes on the experience.
Track Time Spent
Look at how long it takes someone to sign up, how long it
takes to get to their first use case, and common stumbling
blocks that are time sinks.
Experience developer onboarding.
Engage Developers Personally
Did you meet someone at a conference who signed up for
your platform? Follow up with them personally, ask about
their experience, see if they actually use the tool and how.
1
2
3

59. Survey, Survey, Survey
Utilize surveying for users at all stages of usage. Ask the
hard questions that inform your work.
Keep It Simple
Don’t lose perspective on what it is like to use a tool for the
first time. Create the 101 level resources that new
developers need to be successful.
Actively solicit feedback.
Look For Patterns
A well-crafted qualitative report can be just as valuable as a
bunch of numbers. Patterns often reveal themselves from
large qualitative data sets.
1
2
3

60. Ask Why Metrics Are a Certain Way
By looking at what low engagement, or open/click rates
actually mean, you can identify areas of improvement.
Learn Core Marketing Principles
Understanding marketing tactics, best practices, analytics
and tooling give you a major advantage in improving a
developer’s experience with your product.
Read between the lines of metrics.
Invest in Copywriting
Concise, easy to grok copy can make a huge difference in
the success of a developer and the ROI of your marketing
metrics.
1
2
3

61. Thanks!
Questions?
● Contact me at jon@majorleaguehacking.com
● https://mlh.io/
@JonMarkGo
fb.com/JonMarkGo