Discover Pinterest Engineering
Agenda
!
Mobile & Growth:
Monetization & Data:
Deploying & Shipping Code:
Wendy & Dannie
Pawel
Chris & Jeremy
Wendy Lu
Mobile
Scaling User Education on Mobile
The Experience Framework
Dannie Chu
Growth
User Education
Motivation?
User Education leads to Engagement
First Pin Education
Challenges
1. Preserve a great user experience
Conflicting Education
Never-ending Stream of Nags
Nag that never goes away
2. Targeting
Are you a new user? Invite some friends!
Do you have an empty homefeed? Make it better!
3. Rapid Experimentation
Experiment with messaging w/o requiring a release
4. Keep the client code clean
Solution?
The Experience Framework
Conflicting Experiences
Experiences and Placements
Nag Placement Home view Placement
Pin Tutorial Card
Placement
AB Experimentation, Simple Client Code
Experiences are delivered to the client at runtime
Experience Framework
Decision Engine
Configuration
Handlers
Admin
DashboardAPI
Servers
AB Experiments
Framework Kafka
(log...
Deep Dive: Nags
What are nags?
•Important information which we
occasionally display at the top of
a user’s feed

•Can be a Call to Action ...
Enter: The Experience Framework
Enter: The Experience Framework
•Each time we reload the home
feed, ask the experience
framework: What should I show
in th...
Single Nag Manager that relies on the experience
framework to give it the contents to render
Nag Manager
•Experiment with ...
What does that nag data look like?
Handling actions
"bg_img_url_2x" = "http://mobile-
assets.pinterest.com/iphone/nags/inv...
Handling Actions
•All initialization and presentation of view controllers is handled through a
central “Navigation Manager...
[[NavigationManager sharedManager]
handleURL:[NSURL
URLWithString:@“pinterest://
invite_friends]];
Navigation Manager
self...
Deep Dive: New User Experience
New User Experience
What is NUX?
!
•The set of initial tutorials and education presented to a user after they
register
Personalize a user’s content immediately after signup
Connect with Friends Choose Interests
Next, teach users how to Pin
Experiment between classic user ed and NUX
Experiment between classic user ed and NUX
•When doing experiments where we need to call the network to get the
user’s tre...
1 step vs. 2 steps vs. 3 steps
Experiment with different versions of NUX
Experiment with different versions of NUX
•Backend controls all strings, allowing us to dynamically experiment with
differen...
steps = (
{
"continue_button_text" = Continue;
"detailed_text" = "Tap a network to find people who share your interests.";...
Supports dynamic number and order of steps
NUXViewController : UINavigationController
•Maps an array of display data to an...
Wins from Experience Framework
•Single place in the backend that manages all experiences for all platforms

•Dynamically t...
engineering.pinterest.com
Pawel Garbacki
Software Engineer, Monetization
Pinterest Secor
Zero-data-loss log persistence service
Pinterest is a data driven company
•Data matters

• 100+ experiments active at a given point in time

• 1500+ tracked metr...
We produce a lot of data
•We produce a lot of data

• PBs of data in S3, growing by Tens of TB a day

• Hundreds of produc...
Singer 

(logging
agent)
App
Data pipeline
Local
Disk
S3Kafka 

(log collector)
Secor 

(log saver)
Storm
(realtime
stats)...
Singer 

(logging
agent)
App
Local
Disk
S3
Kafka (log
collector)
Secor (log
saver)
Storm
(realtime
stats)
Hive (hadoop
ana...
Kafka 101
•Distributed pub-sub service

•Designed for high throughput
Producer Producer Producer
Consumer Consumer Consume...
Anatomy of a topic
•Topic is a category to which messages are published

•Partition is a ‘shard’ of a topic controlling th...
Save the day
•Kafka is optimized for local writes

•Local disk capacity is good for a few
days worth of data

•Data needs ...
How soon is “eventually”
•Amazon S3 is a cloud file system

•Eventual consistency model

• No guarantees on when uploaded d...
Secor design guidelines
•Objectives:

• Persist Kafka logs to S3

• Cause no data loss

• Work properly with eventual cons...
No-S3-reads principle
•Secor never reads data from S3

• Lightweight metadata is stored in strongly
consistent state repos...
Date clustering
•Data processing tools rely on date-
partitioned directory structure

• s3n://logs/event/dt=2014-04-04/

•...
https://github.com/pinterest/secor
Chris Danford (@chrisdanford)
Jeremy Stanley (@rouxbot)
Web Team
From Development to Production
The life of web features
Develop Review Deploy Measure
Develop Review Deploy Measure
Developing - Ideal state
•able to iterate quickly

•easy errors caught automatically

•easy-to-understand and powerful abs...
Fast iteration
Developing
•Build tasks and dependencies modeled as a graph

•“cumberbatch” watches for changes of file cont...
Self-healing build graph
sprite/*.png
sprite task
sprite.png sprite.scss
Sass task
component/*.scss
components.css
imagemi...
Self-healing build graph
sprite/*.png
sprite task
sprite.png sprite.scss
Sass task
component/*.scss
components.css
imagemi...
Self-healing build graph
sprite/*.png
sprite task
sprite.png sprite.scss
Sass task
component/*.scss
components.css
imagemi...
Self-healing build graph, example 2
sprite/*.png
sprite task
sprite.png sprite.scss
Sass task
component/*.scss
components....
Self-healing build graph, example 2
sprite/*.png
sprite task
sprite.png sprite.scss
Sass task
component/*.scss
components....
Linting
Developing
•catches easy bugs

•enforces consistent style

•pyflakes

•pep8

•jshint

•CSSLint

•custom RegEx linti...
RegEx-based linter
$(‘.item’).addClass(‘selected’);
BoardPicker.js
this.$(‘.item’).addClass(‘selected’);
Static analysis / type safety
Developing
•Google Closure Compiler

•template static analysis

• validation of template inh...
Closure Compiler - type safety
Generated constants from Thrift
Abstractions
Developing
•component framework

• styles are scoped to the component

• DOM access is scoped to the componen...
component catalog
spriting
.thumbsUpButton {!
@include inline-image(‘sprites/main/thumbUp');!
}
Usage
Develop Review Deploy Measure
Reviewing - Ideal state
•the most-relevant person is reviewing changes

•integration-type issues are caught
Code Reviews
Reviewing
•local test runner script runs Jasmine and Node tests parallelized

•“PR watcher” tool - visibility...
Parallelized Jasmine tests in PhantomJS
Github Watcher tool
Integration problems are caught
Reviewing
•PRs trigger a build and tests - 3 minutes

•latest.pinterest.com is continuousl...
Pull request builds
Selenium
Develop Review Deploy Measure
Deploying - Ideal state
•code deploys are invisible to users

•frequent and non-disruptive to developers

•immediate rollb...
User experience
Deploying
•stickiness to a version

•flip nearly instantaneously between builds

• reduce version thrashing...
Serving multiple application versions
new!
sessions
old!
sticky!
sessions
A B
all!
sessions
B
1 2
Serving multiple application versions, cont’d
3 4
new!
canary!
sessions
all!
non-canary!
sessions
B C
new!
sessions
old!
s...
Asset versioning
logo.png
rename
hash file!
contents
logo.59aa9183.png
bundle.css
background-image: logo.png
background-im...
When things go wrong
Deploying
•experiments dashboard - turn off experiments instantaneously

•version rollback is nearly i...
Develop Review Deploy Measure
Monitoring health
A/B Dashboard
•A/B dashboard key metrics

•Sentry 

•Stats dashboard

•Alarms - Monit, PagerDuty
A/B Dashboard
Sentry
Sentry error emails
Results
Results
•5 engineers on web team

•all teams at Pinterest developing their own web features on our platform

•components r...
F8 tech talk_pinterest_v4
F8 tech talk_pinterest_v4
Upcoming SlideShare
Loading in …5
×

F8 tech talk_pinterest_v4

230 views

Published on

In April 2014, Pinterest engineers presented to members of the engineering community at a series of Tech Talks held at the Pinterest offices in San Francisco. Topics included:

- Mobile & Growth: Scaling user education on mobile, and a deep dive into the new user experience (with engineers Dannie Chu and Wendy Lu)
- Monetization & Data: The open sourcing of Pinterest Secor and a look at zero data loss log persistence services (with engineer Pawel Garbacki)
- Developing & Shipping Code at Pinterest: The tools and technologies Pinterest uses to build quickly and deploy confidently.

You can find more at: engineering.pinterest.com and facebook.com/pinterestengineering

Published in: Engineering, Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
230
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

F8 tech talk_pinterest_v4

  1. 1. Discover Pinterest Engineering
  2. 2. Agenda ! Mobile & Growth: Monetization & Data: Deploying & Shipping Code: Wendy & Dannie Pawel Chris & Jeremy
  3. 3. Wendy Lu Mobile Scaling User Education on Mobile The Experience Framework Dannie Chu Growth
  4. 4. User Education
  5. 5. Motivation?
  6. 6. User Education leads to Engagement
  7. 7. First Pin Education
  8. 8. Challenges
  9. 9. 1. Preserve a great user experience
  10. 10. Conflicting Education
  11. 11. Never-ending Stream of Nags
  12. 12. Nag that never goes away
  13. 13. 2. Targeting
  14. 14. Are you a new user? Invite some friends!
  15. 15. Do you have an empty homefeed? Make it better!
  16. 16. 3. Rapid Experimentation
  17. 17. Experiment with messaging w/o requiring a release
  18. 18. 4. Keep the client code clean
  19. 19. Solution?
  20. 20. The Experience Framework
  21. 21. Conflicting Experiences
  22. 22. Experiences and Placements Nag Placement Home view Placement Pin Tutorial Card Placement
  23. 23. AB Experimentation, Simple Client Code
  24. 24. Experiences are delivered to the client at runtime
  25. 25. Experience Framework Decision Engine Configuration Handlers Admin DashboardAPI Servers AB Experiments Framework Kafka (logging) HBase (User States, 
 Experience States) API Servers API ServersCascading/ MapRed User State Batch Loader Clients SDK SDK What experience should the user see? Experience Framework
  26. 26. Deep Dive: Nags
  27. 27. What are nags? •Important information which we occasionally display at the top of a user’s feed •Can be a Call to Action “Confirm your email” or an announcement “You can now add a map to any board!”
  28. 28. Enter: The Experience Framework
  29. 29. Enter: The Experience Framework •Each time we reload the home feed, ask the experience framework: What should I show in this nag?
  30. 30. Single Nag Manager that relies on the experience framework to give it the contents to render Nag Manager •Experiment with nag messaging, call to actions, images •Add any new nag dynamically, controlled from the backend. •Cool-down management - should not see more than one nag in a set period of time
  31. 31. What does that nag data look like? Handling actions "bg_img_url_2x" = "http://mobile- assets.pinterest.com/iphone/nags/invite- mail@2x.png" "title_text" = "Pinspire your friends!" "detailed_text" = "Know someone who'd like Pinterest? Invite them along." "button1_text" = "No, Thanks”! "button1_uri" = ""! "button2_text" = "Invite Friends"! "button2_uri" = "pinterest://invite_friends"
  32. 32. Handling Actions •All initialization and presentation of view controllers is handled through a central “Navigation Manager.” • Centralizes code to create and present view controllers • Consistency to other platforms for deep links •Allows dynamic insertion of nags from the backend without having to write new client code and submit a new release
  33. 33. [[NavigationManager sharedManager] handleURL:[NSURL URLWithString:@“pinterest:// invite_friends]]; Navigation Manager self.presentationDelegate presentInviteFriends
  34. 34. Deep Dive: New User Experience
  35. 35. New User Experience What is NUX? ! •The set of initial tutorials and education presented to a user after they register
  36. 36. Personalize a user’s content immediately after signup Connect with Friends Choose Interests
  37. 37. Next, teach users how to Pin
  38. 38. Experiment between classic user ed and NUX
  39. 39. Experiment between classic user ed and NUX •When doing experiments where we need to call the network to get the user’s treatment group, need to make sure we’re not adding perceived latency • Structure view controllers in a way that you can asynchronously load in the modules dependent on the treatment group • If need to transition to different view controllers, set a time out in which we transition to the control treatment • “Be Fast or Fail Fast”
  40. 40. 1 step vs. 2 steps vs. 3 steps Experiment with different versions of NUX
  41. 41. Experiment with different versions of NUX •Backend controls all strings, allowing us to dynamically experiment with different text (Messaging, titles, calls-to-action)
  42. 42. steps = ( { "continue_button_text" = Continue; "detailed_text" = "Tap a network to find people who share your interests."; "follow_button_text" = "Follow selected people"; step = 1; "title_text" = "First things first"; "total_steps" = 2; }, { "completion_message" = "Finding Pins for you..."; "continue_button_text" = "Tap at least {0} more to continue"; "detailed_text" = "Tap whatever you're interested in these days."; "finish_text" = Finish; "num_interests" = 5; "skip_text" = "Pinterest is much more interesting when you tell us what you like."; step = 2; "title_text" = "Pick 5 interests"; "total_steps" = 2; } ); After signup, request all data for NUX Enter: Experience Framework
  43. 43. Supports dynamic number and order of steps NUXViewController : UINavigationController •Maps an array of display data to an array of view controllers
 
 
 
 
 •Protocol method advanceToNextStep called by each child view controller • Checks the array it keeps for the next view controller to push JSON dict for Intro JSON dict for Friend Selector JSON dict for Interests Selector NUXIntroViewController NUXConnectViewController NUXInterestsViewController
  44. 44. Wins from Experience Framework •Single place in the backend that manages all experiences for all platforms •Dynamically trigger display of content •Conflict resolution for educations that touch the same views •Experiment with flows, messaging, and images
  45. 45. engineering.pinterest.com
  46. 46. Pawel Garbacki Software Engineer, Monetization Pinterest Secor Zero-data-loss log persistence service
  47. 47. Pinterest is a data driven company •Data matters • 100+ experiments active at a given point in time • 1500+ tracked metrics • 200+ log types
  48. 48. We produce a lot of data •We produce a lot of data • PBs of data in S3, growing by Tens of TB a day • Hundreds of production hadoop jobs, processing about half a PB of data each day
  49. 49. Singer 
 (logging agent) App Data pipeline Local Disk S3Kafka 
 (log collector) Secor 
 (log saver) Storm (realtime stats) Hive (hadoop analytics) Redshift (ad hoc queries)
  50. 50. Singer 
 (logging agent) App Local Disk S3 Kafka (log collector) Secor (log saver) Storm (realtime stats) Hive (hadoop analytics) Redshift (ad hoc queries) Data pipeline 20B messages/day
  51. 51. Kafka 101 •Distributed pub-sub service •Designed for high throughput Producer Producer Producer Consumer Consumer Consumer Kafka cluster
  52. 52. Anatomy of a topic •Topic is a category to which messages are published •Partition is a ‘shard’ of a topic controlling the level of consumption parallelism •Messages are assigned unique identifiers called offsets 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 10 11 12 Writes Partition 0 Partition 1 Partition 2
  53. 53. Save the day •Kafka is optimized for local writes •Local disk capacity is good for a few days worth of data •Data needs to be saved (at least) daily to long term storage - Amazon S3
  54. 54. How soon is “eventually” •Amazon S3 is a cloud file system •Eventual consistency model • No guarantees on when uploaded data will become visible to the readers • No monotonicity - data available in the past may magically disappear
  55. 55. Secor design guidelines •Objectives: • Persist Kafka logs to S3 • Cause no data loss • Work properly with eventual consistency model •Properties: • Horizontal scalability • Fault tolerance • Customizability
  56. 56. No-S3-reads principle •Secor never reads data from S3 • Lightweight metadata is stored in strongly consistent state repository •Strategic choice of file names • s3n://logs/<topic>/ <generation>_<partition>_<start_offset> • <generation> represents software compatibility version • Inconsistencies introduced by consumer failures get fixed automatically by file overwrites
  57. 57. Date clustering •Data processing tools rely on date- partitioned directory structure • s3n://logs/event/dt=2014-04-04/ •Timestamps extracted from messages on the fly •Support for pluggable parsers for thrift, json, etc.
  58. 58. https://github.com/pinterest/secor
  59. 59. Chris Danford (@chrisdanford) Jeremy Stanley (@rouxbot) Web Team From Development to Production The life of web features
  60. 60. Develop Review Deploy Measure
  61. 61. Develop Review Deploy Measure
  62. 62. Developing - Ideal state •able to iterate quickly •easy errors caught automatically •easy-to-understand and powerful abstractions
  63. 63. Fast iteration Developing •Build tasks and dependencies modeled as a graph •“cumberbatch” watches for changes of file contents •“orchestrator” knows tasks and dependencies •build the minimum tasks to heal damage in the graph •maximize parallelization of tasks •built on Grunt - access to large library of build tasks
  64. 64. Self-healing build graph sprite/*.png sprite task sprite.png sprite.scss Sass task component/*.scss components.css imagemin task Autoprefixer task CSSlint task
  65. 65. Self-healing build graph sprite/*.png sprite task sprite.png sprite.scss Sass task component/*.scss components.css imagemin task Autoprefixer task CSSlint task
  66. 66. Self-healing build graph sprite/*.png sprite task sprite.png sprite.scss Sass task component/*.scss components.css imagemin task Autoprefixer task CSSlint task
  67. 67. Self-healing build graph, example 2 sprite/*.png sprite task sprite.png sprite.scss Sass task component/*.scss components.css imagemin task Autoprefixer task CSSlint task
  68. 68. Self-healing build graph, example 2 sprite/*.png sprite task sprite.png sprite.scss Sass task component/*.scss components.css imagemin task Autoprefixer task CSSlint task
  69. 69. Linting Developing •catches easy bugs •enforces consistent style •pyflakes •pep8 •jshint •CSSLint •custom RegEx linting
  70. 70. RegEx-based linter $(‘.item’).addClass(‘selected’); BoardPicker.js this.$(‘.item’).addClass(‘selected’);
  71. 71. Static analysis / type safety Developing •Google Closure Compiler •template static analysis • validation of template inheritance • extract used option variable names •Thrift clients for service calls (Python) and shared constants (JavaScript)
  72. 72. Closure Compiler - type safety
  73. 73. Generated constants from Thrift
  74. 74. Abstractions Developing •component framework • styles are scoped to the component • DOM access is scoped to the component’s DOM • “events up, methods down” • scaffolding script •live component catalog - discovery of existing components •autoprefixer, spriting - remove boilerplate
  75. 75. component catalog
  76. 76. spriting .thumbsUpButton {! @include inline-image(‘sprites/main/thumbUp');! } Usage
  77. 77. Develop Review Deploy Measure
  78. 78. Reviewing - Ideal state •the most-relevant person is reviewing changes •integration-type issues are caught
  79. 79. Code Reviews Reviewing •local test runner script runs Jasmine and Node tests parallelized •“PR watcher” tool - visibility of relevant PRs •code review process - R+, E+
  80. 80. Parallelized Jasmine tests in PhantomJS
  81. 81. Github Watcher tool
  82. 82. Integration problems are caught Reviewing •PRs trigger a build and tests - 3 minutes •latest.pinterest.com is continuously deployed from “head” •Selenium integration tests run against every deploy
  83. 83. Pull request builds
  84. 84. Selenium
  85. 85. Develop Review Deploy Measure
  86. 86. Deploying - Ideal state •code deploys are invisible to users •frequent and non-disruptive to developers •immediate rollback when there’s a problem
  87. 87. User experience Deploying •stickiness to a version •flip nearly instantaneously between builds • reduce version thrashing • worry less about (style mismatches, JS errors due to data format mismatch with server) •asset versioning
  88. 88. Serving multiple application versions new! sessions old! sticky! sessions A B all! sessions B 1 2
  89. 89. Serving multiple application versions, cont’d 3 4 new! canary! sessions all! non-canary! sessions B C new! sessions old! sticky! sessions B C
  90. 90. Asset versioning logo.png rename hash file! contents logo.59aa9183.png bundle.css background-image: logo.png background-image: logo.59aa9183.png update references bundle.css bundle.907389d8.css rename hash file! contents
  91. 91. When things go wrong Deploying •experiments dashboard - turn off experiments instantaneously •version rollback is nearly instantaneous
  92. 92. Develop Review Deploy Measure
  93. 93. Monitoring health A/B Dashboard •A/B dashboard key metrics •Sentry •Stats dashboard •Alarms - Monit, PagerDuty
  94. 94. A/B Dashboard
  95. 95. Sentry
  96. 96. Sentry error emails
  97. 97. Results
  98. 98. Results •5 engineers on web team •all teams at Pinterest developing their own web features on our platform •components re-used across teams •2 scheduled deploys a day •anomalies in key metrics surfaced immediately •100s of simultaneous experiments

×