• Save
Etsy’s Product Development with Continuous Experimentation
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Etsy’s Product Development with Continuous Experimentation

on

  • 1,087 views

Video and slides synchronized, mp3 and slide download available at http://bit.ly/XCxzIh. ...

Video and slides synchronized, mp3 and slide download available at http://bit.ly/XCxzIh.

Frank Harris and Nellwyn Thomas present the deployment process used by Etsy based on experiments. Filmed at qconsf.com.

Frank Harris is a Product Manager on Search, Recommendations and Data at Etsy. Previously, he was a Sr. UX Designer at Google focusing on payments and commerce. @hirefrank Nellwyn Thomas is an analyst on the Data team at Etsy, where she works closely with product, marketing, and engineering to scout, build, instrument and improve Etsy’s product portfolio. @nellwyn

Statistics

Views

Total Views
1,087
Views on SlideShare
1,086
Embed Views
1

Actions

Likes
5
Downloads
0
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Etsy’s Product Development with Continuous Experimentation Presentation Transcript

  • 1. Etsy’s Product Development with Continuous Experimentation November 8th, 2012 QCon San FranciscoThursday, November 8, 12
  • 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /Etsy-Deployment InfoQ.com: News & Community Site• 750,000 unique visitors/month• Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese)• Post content from our QCon conferences• News 15-20 / week• Articles 3-4 / week• Presentations (videos) 12-15 / week• Interviews 2-3 / week• Books 1 / month
  • 3. Presented at QCon San Francisco www.qconsf.comPurpose of QCon- to empower software development by facilitating the spread ofknowledge and innovationStrategy - practitioner-driven conference designed for YOU: influencers ofchange and innovation in your teams- speakers and topics driving the evolution and innovation- connecting and catalyzing the influencers and innovatorsHighlights- attended by more than 12,000 delegates since 2007- held in 9 cities worldwide
  • 4. Frank Harris Group Product Manager, Search & Data frank@etsy.com @hirefrank Nellwyn Thomas Staff Data Analyst nellwyn@etsy.com @nellwynThursday, November 8, 12
  • 5. Deploy Experiment Example TakeawaysThursday, November 8, 12
  • 6. Thursday, November 8, 12
  • 7. The world’s handmade and vintage marketplace.Thursday, November 8, 12
  • 8. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Browse Introducing Etsy Gift Cards, Art the choose-your-own-adventure gift! Home & Living Let the Giving Begin Jewelry Women Handpicked Items See more Picked by violasboutique Men Kids Weddings Vintage Supplies Accessories for iPhone Set of 4 Damaged Goods Or… Embroidered Hoop "Merry Ch… Advent calendar zakka style … Needle Felted Gnome, green… airyobsessions $25.00 USD merriweathe… $50.00 USD arch190 $77.17 USD feltjar $15.00 USD Holidays Daily Finds Email Get gift ideas, editors picks & fresh trends in your inbox. Enter your email modern ceramic Star bowl ca… Mistletoe Art print of original … Disturbance - Modern Pillow … Modern Christmas Tree Lino… Sign Up OneClayBead $9.00 USD TheJoyofColor $21.00 USD ClothandINK $36.00 USD RetroModer… $20.00 USD See our other newsletters. More Ways to Shop Categories Gift Cards Colors Treasury Shop Local Christmas Stocking. Large St… Joy Holiday Decor, Christma… Glass Christmas Tree Ornam… Christmas Ornament Photogr… bluestitchha… $28.00 USD AppleWhite $42.00 USD thewhitepep… $13.50 USD Raceytay $15.00 USDThursday, November 8, 12
  • 9. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In leather ipad sleeve Search Cart All Items leather ipad sleeve 1,714 items Sort by: Relevancy Handmade 1,710 Ships to United States × Vintage 3 Supplies 4 Sponsored Handmade Bags and Purses 1,290 Accessories 329 Geekery 86 Needlecraft 1 Everything Else 1 Vintage Bags and Purses 3 Brown Burgundy Stingray Le… New IPad 3 and 4 Full case … HUGE SALE World Traveler … Leather Canvas 13-inch Lapt… Yehia $299.00 USD PicasoLab $68.00 USD lovetojournal… $53.55 USD xmhazel $39.00 USD Show Only Your Favorites Accepts Etsy Gift Cards Shop Location All locations Enter location ► Ships to United States Leather Ipad Sleeve/ Case/ E… Buckled leather iPad sleeve … Distressed thick brown leathe… 4mm Thick Tan Leather iPad … Change BayTowneL… $74.00 USD LUSCIOUSL… $99.00 USD sord $68.00 USD FleurdeLeather $79.00 USD Price All prices $ to $ ► Leather iPad Sleeve / Case - … Leather Ipad Sleeve/ Case/ F… Hand Stitched Tan Leather Ip… Brown Faux Leather iPad Sle… LWleatherw… $84.00 USD giordanoart $75.00 USD zassicleather $100.00 USD begforabag $35.00 USDThursday, November 8, 12
  • 10. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Home › BayTowneLeatherUSA › IPAD Cases $74.00 USD Only 1 available Like this item? Favorite Add it to your favorites to revisit it later. Add to Cart Leather Ipad Sleeve/ Case/ Envelope/ Folio for IPAD 2 or 3. Full Grain Veg Tan Leather. Handmade by BayTowneLeatherUSA. Favorite Shop BayTowneLeatherUSA BayTowneLeatherUSA 38 items Add shop to favorites See who favorites this shop Shop owner Roger Marion, Illinois, United States Favorites Circle Feedback: 27, 100% pos. This luxurious/rugged and well built IPAD case will protect your IPAD in style. Or you can use it as a folio to carry your necessities . Luxurious enough for use at the office, and rugged Contact enough for use on the trail. Made of beautiful - rugged 5/6 oz. Premium veg tan leather (which will break in beautifully) ; it will develop a wonderful natural patina over time as only full grain veg tan leather can do. Item (Gets better with age) Add item to favorites ***I make these cases ONE BY ONE as orders are received*** See who favorites this itemThursday, November 8, 12
  • 11. Over 15M items from 800k shops in 150 countriesThursday, November 8, 12
  • 12. 1.4B pageviews per monthThursday, November 8, 12
  • 13. 40M unique visitors per monthThursday, November 8, 12
  • 14. $525M sales in 2011Thursday, November 8, 12
  • 15. 30 deploys per day by 200+ people = over 10,000 deploys in 2011. This is continuous delivery at Etsy.Thursday, November 8, 12
  • 16. Deploy Experiment Example TakeawaysThursday, November 8, 12
  • 17. Thursday, November 8, 12
  • 18. Thursday, November 8, 12
  • 19. Thursday, November 8, 12
  • 20. Thursday, November 8, 12
  • 21. Continuous Delivery Why We Like It • Less scary, small and frequent • Move faster • More empowering • Breeds confidenceThursday, November 8, 12
  • 22. Thursday, November 8, 12
  • 23. Thursday, November 8, 12
  • 24. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Similar items 35 Sort by: Relevancy vintage × rubber stamp × office × supply × Add 2 Vintage Rubber Stamps - 1960s - Crown Rubber Stamp Co - Justrite Stamp Co - Back to School Vintage Price Stamp, Garvey… I can provide samples... Very… Very vintage French desk sta… Vintage Rubber Stamp Symb… ThisCharmingManCave SummerHoli… $35.00 USD Mademoisell… $19.95 USD Mademoisell… $14.95 USD ArtfulVintage $13.00 USD Favorite Show Only Your Favorites Accepts Etsy Gift Cards Shop Location Rubber Stamp Holder Vintage Wooden and Rubber… Vintage Crown Bank Teller D… Vintage Crown Ledger Line D… All locations susantique $24.00 USD mrbarnes5 $9.00 USD SydVintage $4.99 USD SydVintage $1.99 USD Enter location ► Ships to United States ► Price All prices $ to $ ► Fab large vintage commercial… Vintage Crown Bank Teller D… Vintage Crown Ledger Line D… Vintage Crown Custom Dual …Thursday, November 8, 12
  • 25. Enter location ► Ships to United States ► Price All prices $ to $ ► Fab large vintage commercial… Vintage Crown Bank Teller D… Vintage Crown Ledger Line D… Vintage Crown Custom Dual … Mademoisell… $29.00 USD SydVintage $5.99 USD SydVintage $2.99 USD SydVintage $4.99 USD Petite Dark Oak Antique Woo… Shaded Metal Fastener Clip … Standard Rubber Stamp Hold… Vintage Crown Club Dater D… TimelessBy… $12.50 USD kawaiigoodies $2.95 USD RosebudsOr… $24.95 USD SydVintage $6.99 USD vintage office funk ... WOOD… VTG rubber stamp Printing Ki… Vintage Wooden Rubber Sta… Rubber Stamp Bag National … Inktiques $12.00 USD dagutzyone $20.00 USD oliday $18.50 USD ThreeOldKeys $11.00 USD Vintage Crown Club Dater D… Vintage Crown Club Dater D… Vintage Crown Club Dater D… Vintage Crown Club Dater D…Thursday, November 8, 12 SydVintage $5.99 USD SydVintage $3.99 USD SydVintage $6.99 USD SydVintage $5.99 USD
  • 26. Vintage Crown Club Dater D… Vintage Crown Club Dater D… Vintage Crown Club Dater D… Vintage Crown Club Dater D… SydVintage $5.99 USD SydVintage $3.99 USD SydVintage $6.99 USD SydVintage $5.99 USD Vintage Crown Club Dater D… Vintage Crown Club Dater D… Vintage Carters Rubber Sta… Vintage Office Rubber Stamp… SydVintage $5.99 USD SydVintage $5.99 USD SydVintage $3.99 USD Vintageworks $18.00 USD Vintage Office Stamp with Si… Dial a phrase stamp with 12 … Vintage Candy Commission … Vintage Rubber Stamp Makin… alifewelllived $8.00 USD FunsizeCrea… $9.95 USD CaprockVint… $6.00 USD heretoforem… $25.00 USD Vintage Jennifer Rubber Stamp Vintage Pullman Shipping Di… Ultra mini French or Europea…Thursday, November 8, 12 iSpyVintage $12.00 USD SydVintage $9.99 USD kawaiigoodies $4.95 USD
  • 27. Vintage Crown Club Dater D… Vintage Crown Club Dater D… Vintage Carters Rubber Sta… Vintage Office Rubber Stamp… SydVintage $5.99 USD SydVintage $5.99 USD SydVintage $3.99 USD Vintageworks $18.00 USD Vintage Office Stamp with Si… Dial a phrase stamp with 12 … Vintage Candy Commission … Vintage Rubber Stamp Makin… alifewelllived $8.00 USD FunsizeCrea… $9.95 USD CaprockVint… $6.00 USD heretoforem… $25.00 USD Vintage Jennifer Rubber Stamp Vintage Pullman Shipping Di… Ultra mini French or Europea… iSpyVintage $12.00 USD SydVintage $9.99 USD kawaiigoodies $4.95 USD © 2012 Etsy, Inc. English (US) / $ United States Dollar / United States Blog About Contact Press Developers Careers Terms Privacy CopyrightThursday, November 8, 12
  • 28. config.php template.tplThursday, November 8, 12
  • 29. Thursday, November 8, 12
  • 30. Welcome to Deployinator _________ ______ _____ _____ ______ /_____ ________ ___ /______ _____ _____(_)_______ ______ ___ /_______ ________ fharris — Switch account _ __ / _ _ ___ __ __ / _ __ __ / / /__ / __ __ _ __ `/_ __/_ __ __ ___/ deploy.etsycorp.com is up / /_/ / / __/__ /_/ /_ / / /_/ /_ /_/ / _ / _ / / // /_/ / / /_ / /_/ /_ / __,_/ ___/ _ .___/ /_/ ____/ ___, / /_/ /_/ /_/ __,_/ __/ ____/ /_/ Change stack: web /_/ /____/ Version: Test: 85c3a6f-20121107-011117-UTC Princess: 85c3a6f-20121107-011117-UTC Production: 85c3a6f-20121107-011117-UTC Deploy log Watch Auto scroll command output? #Push topic: See run logs <prod> cdary (c) Write an arbitrary log message 01. Save the Princess with Tests 85c3a6f ! 125b38c Write Save the Princess with Tests web_config – 2012-11-07 09:08:36 | CONFIG | cdary | CONFIG PRODUCTION Deploy: old 62c689a, new: 125b38c | RUN LOG DIFF DASHBOARD Princess Log Watcher (supergrep) web_config – 2012-11-07 09:06:39 | CONFIG | cdary | CONFIG Princess Deploy: old 62c689a, new: 125b38c | RUN LOG DIFF DASHBOARD 02. Deploy to Production web_config – 2012-11-07 08:28:04 | CONFIG | cdary | CONFIG PRODUCTION Deploy: old f54857b, new: 85c3a6f ! 85c3a6f 62c689a | RUN LOG DIFF DASHBOARD Deploy to Production web_config – 2012-11-07 08:24:20 | CONFIG | cdary | CONFIG Princess Deploy: old f54857b, new: 62c689a | RUN LOG DIFF DASHBOARD What to watch after a push web – 2012-11-07 01:17:11 | PRODUCTION | pseibel | Production Deploy: old a94f728, new: 85c3a6f | RUN Log Watcher (supergrep) LOG DIFF DASHBOARD Forums web – 2012-11-07 01:13:49 | PRINCESS | pseibel | Princess Deploy: old: a94f728, new: 85c3a6f | RUN LOG DIFF DASHBOARD web – 2012-11-07 00:29:57 | PRODUCTION | shardisty | Production Deploy: old ea37991, new: a94f728 |Thursday, November 8, 12
  • 31. .................................................................. Deploy Dashboard Git Commits Princess Supergrep Supergrep Forum Watcher Time: 1 hour Until: Now Hide Deploys Deploys hidden: Config Web / Api / Atlas Search Photos Blog Chef (prod) Schemanator CI Status (1 hour) Push rbateman on qa 6m since 7, 2012 9:54 End-User (1 hour) Three-Armed Sweaters clip Akamai Whoopsies clip Edgecast Whoopsies clip Screwed Users clip error pages · historical origin failures · historical origin failures · historical unique users · historical Error Logs (All) (1 hour) All File or Script Not Found clip Exceptions clip Fatals clip Errors clip Warnings clip current · historical current · historical current · historical current · historical current · historical Segmentation Faults clip PHP Allowed Memory clip PHP Maximum Execution clip Memcache Connection clip Thrift Exceptions clip current · historical Size Exhausted Time Failures current · historical current · historical current · historical current · historicalThursday, November 8, 12
  • 32. Deploy Experiment Example TakeawaysThursday, November 8, 12
  • 33. Easy. Exactly the same way.Thursday, November 8, 12
  • 34. Thursday, November 8, 12
  • 35. Thursday, November 8, 12
  • 36. Experiments Requirements: Bucketing listing cart checkout page pageThursday, November 8, 12
  • 37. Experiments Requirements: Logging hadoop cluster mysql dbThursday, November 8, 12
  • 38. Thursday, November 8, 12
  • 39. Thursday, November 8, 12
  • 40. on offThursday, November 8, 12
  • 41. Reusing config, automated bucketing, automated analysis & graphing with no additional effortThursday, November 8, 12
  • 42. Thursday, November 8, 12
  • 43. Deploy Experiment Example TakeawaysThursday, November 8, 12
  • 44. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Home › BayTowneLeatherUSA › IPAD Cases $74.00 USD Only 1 available Like this item? Favorite Add it to your favorites to revisit it later. Add to Cart Leather Ipad Sleeve/ Case/ Envelope/ Folio for IPAD 2 or 3. Full Grain Veg Tan Leather. Handmade by BayTowneLeatherUSA. Favorite Shop BayTowneLeatherUSA BayTowneLeatherUSA 38 items Add shop to favorites See who favorites this shop Shop owner Roger Marion, Illinois, United States Favorites Circle Feedback: 27, 100% pos. This luxurious/rugged and well built IPAD case will protect your IPAD in style. Or you can use it as a folio to carry your necessities . Luxurious enough for use at the office, and rugged Contact enough for use on the trail. Made of beautiful - rugged 5/6 oz. Premium veg tan leather (which will break in beautifully) ; it will develop a wonderful natural patina over time as only full grain veg tan leather can do. Item (Gets better with age) Add item to favorites ***I make these cases ONE BY ONE as orders are received*** See who favorites this itemThursday, November 8, 12
  • 45. 22% visits enter 53% leave immediatelyThursday, November 8, 12
  • 46. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Home › BayTowneLeatherUSA › IPAD Cases $74.00 USD Only 1 available Like this item? Favorite Add it to your favorites to revisit it later. Add to Cart Leather Ipad Sleeve/ Case/ Envelope/ Folio for IPAD 2 or 3. Full Grain Veg Tan Leather. Handmade by BayTowneLeatherUSA. Favorite Shop BayTowneLeatherUSA BayTowneLeatherUSA 38 items Add shop to favorites See who favorites this shop Shop owner Roger Marion, Illinois, United States Favorites Circle Feedback: 27, 100% pos. This luxurious/rugged and well built IPAD case will protect your IPAD in style. Or you can use it as a folio to carry your necessities . Luxurious enough for use at the office, and rugged Contact enough for use on the trail. Made of beautiful - rugged 5/6 oz. Premium veg tan leather (which will break in beautifully) ; it will develop a wonderful natural patina over time as only full grain veg tan leather can do. Item (Gets better with age) Add item to favorites ***I make these cases ONE BY ONE as orders are received*** See who favorites this itemThursday, November 8, 12
  • 47. How do we know if this theory is right?Thursday, November 8, 12
  • 48. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Similar Items See more Leather Ipad Sleeve/ Case/ E… Leather Ipad Sleeve/ Case/ E… Leather Document Case, Foli… Leather ipad case. Handstitc… Buckled leather iPad sleeve … BayTowneL… $74.00 USD BayTowneL… $74.00 USD BayTowneL… $94.00 USD StudioFi $55.00 USD LUSCIOUSL… $99.00 USD Home › BayTowneLeatherUSA › IPAD Cases $74.00 USD Only 1 available Like this item? Favorite Add it to your favorites to revisit it later. Add to Cart Leather Ipad Sleeve/ Case/ Envelope/ Folio for IPAD 2 or 3. Full Grain Veg Tan Leather. Handmade by BayTowneLeatherUSA. Favorite Shop BayTowneLeatherUSA BayTowneLeatherUSA 38 items Add shop to favorites See who favorites this shopThursday, November 8, 12
  • 49. Buy Sell Registry Community Blogs Mobile Gift Cards Help Register Sign In Search for items and shops Search Cart Home › AHeirloom › About Shop Sections Shop home 73 items US Cutting Boards 49 Custom Request 1 AHeirloom Like 470 Making it digital and heartwarming since 2010 Extra Heart or Star 1 Country Cutting Board 5 About AHeirloom Island.Districts 4 Skewers 2 Engraving 3 Ball Park Series 4 Shop Owner Amy Stringer-Mowat Brooklyn, New York, United States Contact Favorites Circles: 269Thursday, November 8, 12 pos. Feedback: 2208, 100% some boards ready to be packed for the weekly shipments from deep inside brooklyn, NY
  • 50. Amy Stringer-Mowat Brooklyn, New York, United States Contact Favorites Circles: 269 Feedback: 2208, 100% pos. some boards ready to be packed for the weekly shipments from deep inside brooklyn, NY Shop Info AHeirloom Opened on Oct 14, 2009 Accidental Entrepreneurship... About Brooklyn based husband and wife, Amy Stringer-Mowat and Bill Policies Mowat, made their first cutting boards for their own wedding. It is Request custom item rumored that the groom was actually finishing the boards on the day of 16817 sales the wedding... The two wanted a personalized and sentimental way to 5182 admirers display cheese, and came up with state-shaped cutting boards for each of their home states (Michigan and Connecticut), with hearts on their William Henry Mowat III Actions Owner, Maker, Designer hometowns. Delightfully, other people wanted the boards and they are Add to favorites William Henry Mowat III aka Bill now made for all fifty states. Each board is made in the USA of bamboo. is the man behind those nice, See who favorites this borderlines and the smooth AHeirloom listed their original state shaped cutting boards on Etsy in finish. His years of experience Tweet May of 2010. with fabrication helps to speed the process along and keep the Report this shop to Etsy boards moving down the line. A modern day Henry Ford. Our Background Amy and Bill met at Amy’s birthday party when both were getting their Masters in Architecture at Columbia University. Bill walked by, wished Amy a simple and sweet “Happy Birthday,” and the two have been Amy Stringer-… Henry Winslo… collaborating off and on ever since.Thursday, November 8, 12
  • 51. Our Background Amy and Bill met at Amy’s birthday party when both were getting their Masters in Architecture at Columbia University. Bill walked by, wished Amy a simple and sweet “Happy Birthday,” and the two have been Amy Stringer-… Henry Winslo… collaborating off and on ever since. The love of food, design and housewares began with Amys work at Zingermans Delicatessen in the mid 90s and also with Monmouth Coffee Company, in London England. Merging these interests with Bills background in design, his years of experience with project management and skill at fabrication, have made for a good combination on their latest venture. Today, Amy manages the AHeirloom family business and takes care of the firm’s newest member, Henry, born in September of 2011. Bill handles the production of their orders and is a partner in a successful fabrication shop in Williamsburgh, Brooklyn. Our philosophy AHeirloom believe in their small business and manufacturing in the United States. The pieces are designed to be cherished and passed down -- to engage those who use them and resonate with each use. Their quality comes not only from the design and materials, but from Amy and Bill’s belief in new manufacturing techniques. Whether things are made by hand or by machine, they can be timeless -- like these cutting boards, which serve as reminders of where you cameThursday, November 8, 12
  • 52. handles the production of their orders and is a partner in a successful fabrication shop in Williamsburgh, Brooklyn. Our philosophy AHeirloom believe in their small business and manufacturing in the United States. The pieces are designed to be cherished and passed down -- to engage those who use them and resonate with each use. Their quality comes not only from the design and materials, but from Amy and Bill’s belief in new manufacturing techniques. Whether things are made by hand or by machine, they can be timeless -- like these cutting boards, which serve as reminders of where you came from, or where you’d like to be. Featured listings from AHeirloom AHeirlooms USA Cutting Board AHeirlooms Black Board Mini AHeirlooms Texas State Cutt… AHeirlooms Maryland State … AHeirloom $48.00 USD AHeirloom $55.00 USD AHeirloom $48.00 USD AHeirloom $48.00 USD © 2012 Etsy, Inc. English (US) / $ United States Dollar / United States Blog About Contact Press Developers Careers Terms Privacy CopyrightThursday, November 8, 12
  • 53. Thursday, November 8, 12
  • 54. Thursday, November 8, 12
  • 55. Thursday, November 8, 12
  • 56. Thursday, November 8, 12
  • 57. Hypothesis: Showing more items will decrease bounce rate.Thursday, November 8, 12
  • 58. Thursday, November 8, 12
  • 59. And there’s more...Thursday, November 8, 12
  • 60. Listing Page Experiment Results • listing page views (++) • searches (++) • home page views (+) • checkouts (+)! * affect on that listing you landed on (?)Thursday, November 8, 12
  • 61. Listing Page Experiment Next Steps • we do more analysis • we try more variants • we think about how to expand • we invest more resourcesThursday, November 8, 12
  • 62. Deploy Experiment Example TakeawaysThursday, November 8, 12
  • 63. Make it easy to deploy. Invest in tooling.Thursday, November 8, 12
  • 64. Validate ideas with quantitative & qualitative data early and often.Thursday, November 8, 12
  • 65. Allow for time to react and iterate based on what you learn.Thursday, November 8, 12
  • 66. Questions? Learn more etsy.com/careers github.com/etsy Frank Harris frank@etsy.com @hirefrank Nellwyn Thomas nellwyn@etsy.com @nellwynThursday, November 8, 12