SlideShare a Scribd company logo
REST In Action: The Live
Coverage Platform at the
New York Times
WordCamp US
Philadelphia, PA - December 4, 2015
Scott Taylor
• Core Developer, WordPress

Release Lead for 4.4
• Sr. Software Engineer, The
New York Times
• @wonderboymusic on Twitter/
Instagram/Swarm et al
• I like Music, NYC, and
Mexican food
2013 - Present
Me at the New York Times
WordPress at the NYT now
• some “legacy” blogs
• Lens - Photography blog
• First Draft
• some internal corporate sites
• NYT Co.
• Women of the World
• Times Journeys
• The Live Coverage platform
• some forthcoming
International projects
WordPress 4.4 Lead
• REST API (Phase 1)
• Term Meta
• Responsive Images
• WordPress as oEmbed Provider
• Tons of under the hood stuff
Blogs Team
at The New York Times
Blogs at the NYTimes
• NYT used WordPress very early
• NYT was an early investor in Automattic
• Multisite
• ~80 blogs at the height of blog mania - the 00s
were the glory days for Blogs and WordPress
• Many blogs used Live Blogging
When I arrived:
Legacy Blogs Codebase
• Separate from the rest of the NYT’s PHP codebase
• Global NYTimes CSS and JS, CSS for all Blogs,
custom CSS per-blog
• A universe that assumed jQuery AND Prototype
were loaded on every page in global scope
• Challenging amounts of what could generously be
called “technical debt”
<html>
<head>
<script src="prototype.js"></script>
<script src="jquery.js"></script>
</head>
<script>
jQuery('#good-times').click( ... );
$$('.prototype').whatever();
</script>
What could possibly go
wrong here?
• Inline HTML from 2008 that assumes Prototype will
still be a thing in 2015, stored in post_content
• Widgets and inline code that add their own version
of jQuery/Prototype, because YOLO
• Even better: widgets/modules from other teams that
use a different version of jQuery … at times there
could be 4 jQuerys on the page (and 4 different
versions at that)
Things Like ….
<script src="http://graphics8.nytimes.com/js/app/
lib/jquery/jquery-1.6.2.min.js"></script>
<script src="http://js.nyt.com/js/app/lib/jquery/
jquery-1.7.1.min.js"></script>
<!—- also bundles jQuery -->
<script src="http://js.nyt.com/video/vhs/build/
vhs-2.x.js"></script>
<script>
NYTD.jQuery(document).ready(function ($) { ... });
</script>
All the jQuerys
No shared modules
• Code/HTML markup can get out of sync with other
projects regularly: header, footer, navigation
• The CSS and JS files were split across multiple
SVN repos - changes to global assets can affect us
without us knowing. Fixing the code requires
scouring through multiple repos.
At the NYT
• No WordPress Comments: There is an entire team that deals
with these for the site globally, in a different system called CRNR
• No Media: There is another CMS at the Times, Scoop, which
stores the images, videos, slideshows, etc
• WordPress native post-locking: This only landed in WordPress
core in version 3.6 (we have yet to reconcile the differences)
• There is layer for Bylines which is separate from Users: Our
users are employees authenticated via LDAP, most post authors
don’t actually enter the content themselves
NYT5: The New Frontier
My arrival at the New York Times coincided with
the NYT5 project, already in progress
NYT5
• Development requires Vagrant environment
• “apps” are Git repos that require Grunt to transpile the
codebase - you can’t run your repo as a website: it has
to be built
• Impossible to create a “theme” this time with shared JS
and CSS. CSS is SASS build, JS is Require build - both
dynamically built for each app.
• PHP has Composer dependencies and uses
namespaces - the directories are expanded via Grunt in
accordance with PSR-0 Autoloading Standard
require( ['jquery'], function ($) {
$('#cool-link').click(...);
} );
require( ['jquery/1.9'], function ($) {
$('#cool-link').click(...);
} );
require( ['jquery/2.0'], function ($) {
$('#cool-link').click(...);
} );
Require.js fixed the jQuery problem
NYT5 Dealbreakers
• We can’t just point at WordPress on every request
and have our code figure out routing. Routing
happens in Apache in NYT5 - most requests get
piped to app.php
• Because PHP Namespaces are used, WP has to
load early and outside of them (global scope)
• On the frontend, WP cannot exit prematurely before
hitting the framework, which returns the response
to the server via SymfonyHttpFoundation
Apache
NYT5: app.php
Route to NYT5 Blogs app
- Load initial files

- Bootstrap WP
- Capture WP content
- WP complete
- Initialize app
NYT5 Advantages
• “shared” modules - we inherit the “shell” of the page,
which includes: navigation, footer, login, etc.
• our nyt5 theme doesn’t need to produce an entire
HTML document, just the “content” portion
• With WP in global scope, all of its code is available even
when we hit the MVC parts of the NYT5 framework.
• WP output is captured via an output buffer on load - it’s
accessible downstream when the app logic is running.
$wp_query = new WP_Query();
$GLOBALS['wp_query'] = ...


function wp_thing() {
global $wp_query;
. . .
}
GLOBALS!!!
This could be solved
namespace WP;
Namespaces
namespace WP;
class Query {}
function get_post_meta() {}
. . .
new WPQuery();
WPget_post_meta();
. . .
namespace WP;
new Query();
Blogs in Transition
Overall: Bad News for Blogs
• Blogs were duplicating Section Fronts, Columns:



Mark Bittman has column in the paper.

The column also exists on the web as an article. 

He contributes to the Diner’s Journal blog. 

There is a section front for dining. 

He also has his own NYTimes blog. Why?
• Blogs and WordPress were combined in everyone’s
mind. So whenever WordPress was mentioned as a
solution for anything, the response was: aren’t blogs
going away? #dark
But we still had…



First Draft
Lens
Live Coverage
2014 Midterm Elections
required new Live
Blogging tools
2008: Live Blogs at the Times
• A Blog would create a post and check “Start Live
Blogging”
• the updates related to the post were stored in custom
tables in the database
• the APIs for interacting with these tables duplicated tons
of WordPress functionality
• Custom Post Types didn’t exist until WordPress 3.0 (June
2010) - the NYT code was never rewritten to leverage
them (would have required porting the content as well)
Live (actual) Blogs:
Dashboards/Dashblogs
• A Live Blog would be its own blog in the network, its own
set of tables
• A special dashboard theme that had hooks to add
custom JS/CSS for each individual blog, without baking
them into the theme
• Making an entirely new site in the network for a 4-hour
event is overkill
• For every 10 or so new blogs that are added, you are
adding 100 new database tables - gross!
What if…
• Instead of custom tables and
dupe’d API code, new object
types: events and updates!
• To create a new “Live Blog”: create
an event, then go to a Backbone-
powered screen to add updates
• If WP isn’t desired for the front end,
it could be the backend for
anything that wants a JSON feed
for live event data
• Using custom post types, building
a Live Event UI that looks like the
NYT5 theme would be nominal
• Built an admin interface with Backbone to quickly
produce content - which in turn could be read from
JSON feeds
• When saving, the updates post into a service we
have called Invisible City (wraps Redis/Pusher)
• Our first real foray into using the REST API
• Our plan was just to be an admin to produce data
via self-service URLs
What we did
Live Events, the new Live Blogs:
Complete Rewrite of 2008 code
• nytimes.com/live/{event} and nytimes.com/live/{event}/
{update}
• Brand new admin interface: Backbone app that uses the
REST API. Constantly updated filterable stream -
Backbone collections that re-fetch on Heartbeat tick
• Custom REST endpoints that handle processes that need
to happen on save
• Front end served by WordPress for SEO, but data is
received by web socket from Invisible City and rendered
via React
Responsive
on Mobile
An “Interactive
Promo” on an
article page
2015: “Blogs” team
becomes “WordPress”
team, joins the
Interactive News team
Guess What?
• Would rather use Docker instead of Vagrant
• PSR-0 is now PSR-4
• Grunt is now eschewed in favor of Gulp
• RequireJS is ok, but I’d rather use Browserify
• PHP is cool, but why don’t we use Node and
React?
Interactive News Team
• Mostly independent
• Mostly special News projects
• Mostly use whatever language you want
• Mostly does not use NYT5
• Is open to using WordPress, but just as often, does
not
What is a Live
Coverage platform?
nytimes.com/live/{event}
Request is served by WordPress,
PHP generates markup
React wraps the "posts" area
JS listens to Web Socket
Updates are added on the backend (OR via SLACK!)
React updates the content
the elections code
became useful for
Breaking News
WordPress + REST
Register a route:
register_rest_route(
‘nyt/v1’,
‘/live-events/(?P<post_id>d+)’,
[
‘callback’ => [ $this, ‘liveEventRoute’ ],
‘methods’ => WP_Rest_Server::CREATABLE
]
);
Handle the route:
public function liveEventRoute( $request ) {
$post_id = $request['post_id'];
$response = new WP_REST_Response( [
'results' => . . .,
] );
return $response;
}
Some REST API
gotchas…
Most plugins only handle POST
• WP-API and Backbone speak REST
• REST will send you requests via 

GET, PUT, DELETE, POST and friends
$hook = add_menu_page( ... );
add_action( "load-$hook", 'callback' );
function old_custom_load() {
if ( 'POST' !== $_SERVER['REQUEST_METHOD'] ) {
return;
}
...
}
function new_custom_load() {
if ( 'GET' === $_SERVER['REQUEST_METHOD'] ) {
return;
}
...
}
WordPress becomes
a web service
• Monolithic mindset needs to transition into how to
make it into a bare metal service provider
• The serving of requests should be loosely coupled
from objects like WP_Query
• WordPress needs to become supportive of
concurrency
Custom JSON Endpoints for GET
• We do not hit these endpoints on the front-end
• We have a storage mount that is fronted via Varnish
and Akamai
• JSON feeds can show up on the homepage of the
NYT to dynamically render “promos” - these have
to massively scale
HTTP is time-consuming
• It is easy to lose track of how many things are
happening on the 'save_post' hook
• Admin needs to be fast
• The front end is typically cached, but page generation
shouldn’t be bogged down by HTTP requests
• Anything which is time-consuming should be
offloaded to a separate “process” or request who
response you don’t need to handle
wp_remote_post( $url, wp_parse_args( [
'timeout' => 0.01,
'blocking' => false
], $args ) );
Fire and Forget*
* Stolen from Mark Jaquith’s nginx cache invalidation technique:
wp_remote_get( $url, [
'timeout' => 0.01,
'blocking' => false,
'headers' => [ 'X-Nginx-Cache-Purge' => '1' ]
] );
Custom REST Endpoints for POST
• Use fire-and-forget technique on 'save_post',
instead of waiting for responses inline. You can still
log/handle/re-try responses in the separate request.
• Most things that happen on 'save_post' only
need to know $post_id for context, the endpoint
handler can call get_post() from there
Trigger the process:
NYTAdminREST::async_request(
'/varnish-invalidation',
[
'urls' => $urls
]
);
Questions?

More Related Content

What's hot

Modern websites in 2020 and Joomla
Modern websites in 2020 and JoomlaModern websites in 2020 and Joomla
Modern websites in 2020 and Joomla
George Wilson
 
Untangling fall2017 week1
Untangling fall2017 week1Untangling fall2017 week1
Untangling fall2017 week1
Derek Jacoby
 
Untangling spring week12
Untangling spring week12Untangling spring week12
Untangling spring week12
Derek Jacoby
 
Untangling fall2017 week2
Untangling fall2017 week2Untangling fall2017 week2
Untangling fall2017 week2
Derek Jacoby
 
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Frank van der Linden
 
Modern javascript
Modern javascriptModern javascript
Modern javascript
Kevin Ball
 
My Contributor Story
My Contributor StoryMy Contributor Story
My Contributor Story
Marko Heijnen
 
Trying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress TodayTrying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress Today
DrewAPicture
 
Python to go
Python to goPython to go
Python to go
Weng Wei
 
It Takes a Village to Make WordPress
It Takes a Village to Make WordPressIt Takes a Village to Make WordPress
It Takes a Village to Make WordPress
DrewAPicture
 
Untangling the web - fall2017 - class 4
Untangling the web - fall2017 - class 4Untangling the web - fall2017 - class 4
Untangling the web - fall2017 - class 4
Derek Jacoby
 
SharePoint Development 101
SharePoint Development 101SharePoint Development 101
SharePoint Development 101
Greg Hurlman
 
Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...
Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...
Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...
Geoff Varosky
 
Untangling - fall2017 - week 8
Untangling - fall2017 - week 8Untangling - fall2017 - week 8
Untangling - fall2017 - week 8
Derek Jacoby
 
A Personal Journey
A Personal JourneyA Personal Journey
A Personal Journey
Michael Lihs
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
Krish
 
Thoughts on building deployable and updatable share point solutions
Thoughts on building deployable and updatable share point solutionsThoughts on building deployable and updatable share point solutions
Thoughts on building deployable and updatable share point solutions
Serge van den Oever
 
Untangling - fall2017 - week 7
Untangling - fall2017 - week 7Untangling - fall2017 - week 7
Untangling - fall2017 - week 7
Derek Jacoby
 
Migrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoMigrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to Go
Weng Wei
 
ES2015 / ES6: Basics of modern Javascript
ES2015 / ES6: Basics of modern JavascriptES2015 / ES6: Basics of modern Javascript
ES2015 / ES6: Basics of modern Javascript
Wojciech Dzikowski
 

What's hot (20)

Modern websites in 2020 and Joomla
Modern websites in 2020 and JoomlaModern websites in 2020 and Joomla
Modern websites in 2020 and Joomla
 
Untangling fall2017 week1
Untangling fall2017 week1Untangling fall2017 week1
Untangling fall2017 week1
 
Untangling spring week12
Untangling spring week12Untangling spring week12
Untangling spring week12
 
Untangling fall2017 week2
Untangling fall2017 week2Untangling fall2017 week2
Untangling fall2017 week2
 
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
 
Modern javascript
Modern javascriptModern javascript
Modern javascript
 
My Contributor Story
My Contributor StoryMy Contributor Story
My Contributor Story
 
Trying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress TodayTrying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress Today
 
Python to go
Python to goPython to go
Python to go
 
It Takes a Village to Make WordPress
It Takes a Village to Make WordPressIt Takes a Village to Make WordPress
It Takes a Village to Make WordPress
 
Untangling the web - fall2017 - class 4
Untangling the web - fall2017 - class 4Untangling the web - fall2017 - class 4
Untangling the web - fall2017 - class 4
 
SharePoint Development 101
SharePoint Development 101SharePoint Development 101
SharePoint Development 101
 
Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...
Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...
Best Practices in SharePoint Development - Just Freakin Work! Overcoming Hurd...
 
Untangling - fall2017 - week 8
Untangling - fall2017 - week 8Untangling - fall2017 - week 8
Untangling - fall2017 - week 8
 
A Personal Journey
A Personal JourneyA Personal Journey
A Personal Journey
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
 
Thoughts on building deployable and updatable share point solutions
Thoughts on building deployable and updatable share point solutionsThoughts on building deployable and updatable share point solutions
Thoughts on building deployable and updatable share point solutions
 
Untangling - fall2017 - week 7
Untangling - fall2017 - week 7Untangling - fall2017 - week 7
Untangling - fall2017 - week 7
 
Migrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoMigrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to Go
 
ES2015 / ES6: Basics of modern Javascript
ES2015 / ES6: Basics of modern JavascriptES2015 / ES6: Basics of modern Javascript
ES2015 / ES6: Basics of modern Javascript
 

Similar to REST In Action: The Live Coverage Platform at the New York Times

Isomorphic WordPress Applications with NodeifyWP
Isomorphic WordPress Applications with NodeifyWPIsomorphic WordPress Applications with NodeifyWP
Isomorphic WordPress Applications with NodeifyWP
Taylor Lovett
 
You Got React.js in My PHP
You Got React.js in My PHPYou Got React.js in My PHP
You Got React.js in My PHP
Taylor Lovett
 
GeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPressGeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPress
GGDBologna
 
Mobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPressMobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPress
Danilo Ercoli
 
Building a mobile app connected to WordPress with WP-AppKit
Building a mobile app connected to WordPress with WP-AppKitBuilding a mobile app connected to WordPress with WP-AppKit
Building a mobile app connected to WordPress with WP-AppKit
Benjamin LUPU
 
WP REST API - Adding Your Own Endpoint
WP REST API - Adding Your Own EndpointWP REST API - Adding Your Own Endpoint
WP REST API - Adding Your Own Endpoint
Keanan Koppenhaver
 
Why Javascript matters
Why Javascript mattersWhy Javascript matters
Why Javascript matters
Marko Heijnen
 
Mobile and IBM Worklight Best Practices
Mobile and IBM Worklight Best PracticesMobile and IBM Worklight Best Practices
Mobile and IBM Worklight Best Practices
Andrew Ferrier
 
Introduction to react native with redux
Introduction to react native with reduxIntroduction to react native with redux
Introduction to react native with redux
Mike Melusky
 
node_js.pptx
node_js.pptxnode_js.pptx
node_js.pptx
dipen55
 
Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9
Derek Jacoby
 
Wordpress beyond blogging
Wordpress beyond bloggingWordpress beyond blogging
Wordpress beyond blogging
Julien Minguely
 
Hello world - intro to node js
Hello world - intro to node jsHello world - intro to node js
Hello world - intro to node js
Refresh Annapolis Valley
 
Node.js to the rescue
Node.js to the rescueNode.js to the rescue
Node.js to the rescue
Marko Heijnen
 
ASP.NET Core 1.0
ASP.NET Core 1.0ASP.NET Core 1.0
ASP.NET Core 1.0
Ido Flatow
 
How to Contribute to Apache Usergrid
How to Contribute to Apache UsergridHow to Contribute to Apache Usergrid
How to Contribute to Apache Usergrid
David M. Johnson
 
DEV117 - Unleash the Power of the AppDev Pack and Node.js in Domino
DEV117 - Unleash the Power of the AppDev Pack and Node.js in DominoDEV117 - Unleash the Power of the AppDev Pack and Node.js in Domino
DEV117 - Unleash the Power of the AppDev Pack and Node.js in Domino
Heiko Voigt
 
Net training in bhubaneswar
Net training in bhubaneswar Net training in bhubaneswar
Net training in bhubaneswar
litbbsr
 
Extending WordPress as a pro
Extending WordPress as a proExtending WordPress as a pro
Extending WordPress as a pro
Marko Heijnen
 
APIs for modern web apps
APIs for modern web appsAPIs for modern web apps
APIs for modern web apps
Chris Mills
 

Similar to REST In Action: The Live Coverage Platform at the New York Times (20)

Isomorphic WordPress Applications with NodeifyWP
Isomorphic WordPress Applications with NodeifyWPIsomorphic WordPress Applications with NodeifyWP
Isomorphic WordPress Applications with NodeifyWP
 
You Got React.js in My PHP
You Got React.js in My PHPYou Got React.js in My PHP
You Got React.js in My PHP
 
GeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPressGeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPress
 
Mobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPressMobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPress
 
Building a mobile app connected to WordPress with WP-AppKit
Building a mobile app connected to WordPress with WP-AppKitBuilding a mobile app connected to WordPress with WP-AppKit
Building a mobile app connected to WordPress with WP-AppKit
 
WP REST API - Adding Your Own Endpoint
WP REST API - Adding Your Own EndpointWP REST API - Adding Your Own Endpoint
WP REST API - Adding Your Own Endpoint
 
Why Javascript matters
Why Javascript mattersWhy Javascript matters
Why Javascript matters
 
Mobile and IBM Worklight Best Practices
Mobile and IBM Worklight Best PracticesMobile and IBM Worklight Best Practices
Mobile and IBM Worklight Best Practices
 
Introduction to react native with redux
Introduction to react native with reduxIntroduction to react native with redux
Introduction to react native with redux
 
node_js.pptx
node_js.pptxnode_js.pptx
node_js.pptx
 
Untangling - fall2017 - week 9
Untangling - fall2017 - week 9Untangling - fall2017 - week 9
Untangling - fall2017 - week 9
 
Wordpress beyond blogging
Wordpress beyond bloggingWordpress beyond blogging
Wordpress beyond blogging
 
Hello world - intro to node js
Hello world - intro to node jsHello world - intro to node js
Hello world - intro to node js
 
Node.js to the rescue
Node.js to the rescueNode.js to the rescue
Node.js to the rescue
 
ASP.NET Core 1.0
ASP.NET Core 1.0ASP.NET Core 1.0
ASP.NET Core 1.0
 
How to Contribute to Apache Usergrid
How to Contribute to Apache UsergridHow to Contribute to Apache Usergrid
How to Contribute to Apache Usergrid
 
DEV117 - Unleash the Power of the AppDev Pack and Node.js in Domino
DEV117 - Unleash the Power of the AppDev Pack and Node.js in DominoDEV117 - Unleash the Power of the AppDev Pack and Node.js in Domino
DEV117 - Unleash the Power of the AppDev Pack and Node.js in Domino
 
Net training in bhubaneswar
Net training in bhubaneswar Net training in bhubaneswar
Net training in bhubaneswar
 
Extending WordPress as a pro
Extending WordPress as a proExtending WordPress as a pro
Extending WordPress as a pro
 
APIs for modern web apps
APIs for modern web appsAPIs for modern web apps
APIs for modern web apps
 

Recently uploaded

Bengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal BrandingBengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal Branding
Tarandeep Singh
 
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
rtunex8r
 
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
APNIC
 
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
3a0sd7z3
 
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
3a0sd7z3
 
Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
davidjhones387
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
Donato Onofri
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
xjq03c34
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
k4ncd0z
 
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
thezot
 
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
APNIC
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
Paul Walk
 

Recently uploaded (12)

Bengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal BrandingBengaluru Dreamin' 24 - Personal Branding
Bengaluru Dreamin' 24 - Personal Branding
 
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
怎么办理(umiami毕业证书)美国迈阿密大学毕业证文凭证书实拍图原版一模一样
 
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...Securing BGP: Operational Strategies and Best Practices for Network Defenders...
Securing BGP: Operational Strategies and Best Practices for Network Defenders...
 
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
快速办理(Vic毕业证书)惠灵顿维多利亚大学毕业证完成信一模一样
 
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
快速办理(新加坡SMU毕业证书)新加坡管理大学毕业证文凭证书一模一样
 
Discover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to IndiaDiscover the benefits of outsourcing SEO to India
Discover the benefits of outsourcing SEO to India
 
HijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process HollowingHijackLoader Evolution: Interactive Process Hollowing
HijackLoader Evolution: Interactive Process Hollowing
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
 
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
一比一原版新西兰林肯大学毕业证(Lincoln毕业证书)学历如何办理
 
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
Honeypots Unveiled: Proactive Defense Tactics for Cyber Security, Phoenix Sum...
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
 

REST In Action: The Live Coverage Platform at the New York Times

  • 1. REST In Action: The Live Coverage Platform at the New York Times WordCamp US Philadelphia, PA - December 4, 2015
  • 2. Scott Taylor • Core Developer, WordPress
 Release Lead for 4.4 • Sr. Software Engineer, The New York Times • @wonderboymusic on Twitter/ Instagram/Swarm et al • I like Music, NYC, and Mexican food
  • 3. 2013 - Present Me at the New York Times
  • 4. WordPress at the NYT now • some “legacy” blogs • Lens - Photography blog • First Draft • some internal corporate sites • NYT Co. • Women of the World • Times Journeys • The Live Coverage platform • some forthcoming International projects
  • 5. WordPress 4.4 Lead • REST API (Phase 1) • Term Meta • Responsive Images • WordPress as oEmbed Provider • Tons of under the hood stuff
  • 6. Blogs Team at The New York Times
  • 7. Blogs at the NYTimes • NYT used WordPress very early • NYT was an early investor in Automattic • Multisite • ~80 blogs at the height of blog mania - the 00s were the glory days for Blogs and WordPress • Many blogs used Live Blogging
  • 8. When I arrived: Legacy Blogs Codebase • Separate from the rest of the NYT’s PHP codebase • Global NYTimes CSS and JS, CSS for all Blogs, custom CSS per-blog • A universe that assumed jQuery AND Prototype were loaded on every page in global scope • Challenging amounts of what could generously be called “technical debt”
  • 10. What could possibly go wrong here?
  • 11. • Inline HTML from 2008 that assumes Prototype will still be a thing in 2015, stored in post_content • Widgets and inline code that add their own version of jQuery/Prototype, because YOLO • Even better: widgets/modules from other teams that use a different version of jQuery … at times there could be 4 jQuerys on the page (and 4 different versions at that) Things Like ….
  • 12. <script src="http://graphics8.nytimes.com/js/app/ lib/jquery/jquery-1.6.2.min.js"></script> <script src="http://js.nyt.com/js/app/lib/jquery/ jquery-1.7.1.min.js"></script> <!—- also bundles jQuery --> <script src="http://js.nyt.com/video/vhs/build/ vhs-2.x.js"></script> <script> NYTD.jQuery(document).ready(function ($) { ... }); </script> All the jQuerys
  • 13. No shared modules • Code/HTML markup can get out of sync with other projects regularly: header, footer, navigation • The CSS and JS files were split across multiple SVN repos - changes to global assets can affect us without us knowing. Fixing the code requires scouring through multiple repos.
  • 14. At the NYT • No WordPress Comments: There is an entire team that deals with these for the site globally, in a different system called CRNR • No Media: There is another CMS at the Times, Scoop, which stores the images, videos, slideshows, etc • WordPress native post-locking: This only landed in WordPress core in version 3.6 (we have yet to reconcile the differences) • There is layer for Bylines which is separate from Users: Our users are employees authenticated via LDAP, most post authors don’t actually enter the content themselves
  • 15. NYT5: The New Frontier My arrival at the New York Times coincided with the NYT5 project, already in progress
  • 16.
  • 17.
  • 18.
  • 19. NYT5 • Development requires Vagrant environment • “apps” are Git repos that require Grunt to transpile the codebase - you can’t run your repo as a website: it has to be built • Impossible to create a “theme” this time with shared JS and CSS. CSS is SASS build, JS is Require build - both dynamically built for each app. • PHP has Composer dependencies and uses namespaces - the directories are expanded via Grunt in accordance with PSR-0 Autoloading Standard
  • 20. require( ['jquery'], function ($) { $('#cool-link').click(...); } ); require( ['jquery/1.9'], function ($) { $('#cool-link').click(...); } ); require( ['jquery/2.0'], function ($) { $('#cool-link').click(...); } ); Require.js fixed the jQuery problem
  • 21. NYT5 Dealbreakers • We can’t just point at WordPress on every request and have our code figure out routing. Routing happens in Apache in NYT5 - most requests get piped to app.php • Because PHP Namespaces are used, WP has to load early and outside of them (global scope) • On the frontend, WP cannot exit prematurely before hitting the framework, which returns the response to the server via SymfonyHttpFoundation
  • 22. Apache NYT5: app.php Route to NYT5 Blogs app - Load initial files
 - Bootstrap WP - Capture WP content - WP complete - Initialize app
  • 23. NYT5 Advantages • “shared” modules - we inherit the “shell” of the page, which includes: navigation, footer, login, etc. • our nyt5 theme doesn’t need to produce an entire HTML document, just the “content” portion • With WP in global scope, all of its code is available even when we hit the MVC parts of the NYT5 framework. • WP output is captured via an output buffer on load - it’s accessible downstream when the app logic is running.
  • 24. $wp_query = new WP_Query(); $GLOBALS['wp_query'] = ... 
 function wp_thing() { global $wp_query; . . . } GLOBALS!!!
  • 25. This could be solved namespace WP;
  • 26. Namespaces namespace WP; class Query {} function get_post_meta() {} . . . new WPQuery(); WPget_post_meta(); . . . namespace WP; new Query();
  • 28. Overall: Bad News for Blogs • Blogs were duplicating Section Fronts, Columns:
 
 Mark Bittman has column in the paper.
 The column also exists on the web as an article. 
 He contributes to the Diner’s Journal blog. 
 There is a section front for dining. 
 He also has his own NYTimes blog. Why? • Blogs and WordPress were combined in everyone’s mind. So whenever WordPress was mentioned as a solution for anything, the response was: aren’t blogs going away? #dark
  • 29. But we still had…
 
 First Draft Lens Live Coverage
  • 30.
  • 31.
  • 32. 2014 Midterm Elections required new Live Blogging tools
  • 33. 2008: Live Blogs at the Times • A Blog would create a post and check “Start Live Blogging” • the updates related to the post were stored in custom tables in the database • the APIs for interacting with these tables duplicated tons of WordPress functionality • Custom Post Types didn’t exist until WordPress 3.0 (June 2010) - the NYT code was never rewritten to leverage them (would have required porting the content as well)
  • 34. Live (actual) Blogs: Dashboards/Dashblogs • A Live Blog would be its own blog in the network, its own set of tables • A special dashboard theme that had hooks to add custom JS/CSS for each individual blog, without baking them into the theme • Making an entirely new site in the network for a 4-hour event is overkill • For every 10 or so new blogs that are added, you are adding 100 new database tables - gross!
  • 35. What if… • Instead of custom tables and dupe’d API code, new object types: events and updates! • To create a new “Live Blog”: create an event, then go to a Backbone- powered screen to add updates • If WP isn’t desired for the front end, it could be the backend for anything that wants a JSON feed for live event data • Using custom post types, building a Live Event UI that looks like the NYT5 theme would be nominal
  • 36.
  • 37. • Built an admin interface with Backbone to quickly produce content - which in turn could be read from JSON feeds • When saving, the updates post into a service we have called Invisible City (wraps Redis/Pusher) • Our first real foray into using the REST API • Our plan was just to be an admin to produce data via self-service URLs What we did
  • 38. Live Events, the new Live Blogs: Complete Rewrite of 2008 code • nytimes.com/live/{event} and nytimes.com/live/{event}/ {update} • Brand new admin interface: Backbone app that uses the REST API. Constantly updated filterable stream - Backbone collections that re-fetch on Heartbeat tick • Custom REST endpoints that handle processes that need to happen on save • Front end served by WordPress for SEO, but data is received by web socket from Invisible City and rendered via React
  • 39.
  • 40.
  • 42. An “Interactive Promo” on an article page
  • 43. 2015: “Blogs” team becomes “WordPress” team, joins the Interactive News team
  • 44. Guess What? • Would rather use Docker instead of Vagrant • PSR-0 is now PSR-4 • Grunt is now eschewed in favor of Gulp • RequireJS is ok, but I’d rather use Browserify • PHP is cool, but why don’t we use Node and React?
  • 45. Interactive News Team • Mostly independent • Mostly special News projects • Mostly use whatever language you want • Mostly does not use NYT5 • Is open to using WordPress, but just as often, does not
  • 46. What is a Live Coverage platform?
  • 47. nytimes.com/live/{event} Request is served by WordPress, PHP generates markup React wraps the "posts" area JS listens to Web Socket Updates are added on the backend (OR via SLACK!) React updates the content
  • 48. the elections code became useful for Breaking News
  • 50. Register a route: register_rest_route( ‘nyt/v1’, ‘/live-events/(?P<post_id>d+)’, [ ‘callback’ => [ $this, ‘liveEventRoute’ ], ‘methods’ => WP_Rest_Server::CREATABLE ] );
  • 51. Handle the route: public function liveEventRoute( $request ) { $post_id = $request['post_id']; $response = new WP_REST_Response( [ 'results' => . . ., ] ); return $response; }
  • 53. Most plugins only handle POST • WP-API and Backbone speak REST • REST will send you requests via 
 GET, PUT, DELETE, POST and friends
  • 54. $hook = add_menu_page( ... ); add_action( "load-$hook", 'callback' ); function old_custom_load() { if ( 'POST' !== $_SERVER['REQUEST_METHOD'] ) { return; } ... } function new_custom_load() { if ( 'GET' === $_SERVER['REQUEST_METHOD'] ) { return; } ... }
  • 55. WordPress becomes a web service • Monolithic mindset needs to transition into how to make it into a bare metal service provider • The serving of requests should be loosely coupled from objects like WP_Query • WordPress needs to become supportive of concurrency
  • 56. Custom JSON Endpoints for GET • We do not hit these endpoints on the front-end • We have a storage mount that is fronted via Varnish and Akamai • JSON feeds can show up on the homepage of the NYT to dynamically render “promos” - these have to massively scale
  • 57. HTTP is time-consuming • It is easy to lose track of how many things are happening on the 'save_post' hook • Admin needs to be fast • The front end is typically cached, but page generation shouldn’t be bogged down by HTTP requests • Anything which is time-consuming should be offloaded to a separate “process” or request who response you don’t need to handle
  • 58. wp_remote_post( $url, wp_parse_args( [ 'timeout' => 0.01, 'blocking' => false ], $args ) ); Fire and Forget* * Stolen from Mark Jaquith’s nginx cache invalidation technique: wp_remote_get( $url, [ 'timeout' => 0.01, 'blocking' => false, 'headers' => [ 'X-Nginx-Cache-Purge' => '1' ] ] );
  • 59. Custom REST Endpoints for POST • Use fire-and-forget technique on 'save_post', instead of waiting for responses inline. You can still log/handle/re-try responses in the separate request. • Most things that happen on 'save_post' only need to know $post_id for context, the endpoint handler can call get_post() from there