SlideShare a Scribd company logo
1 of 43
Download to read offline
Live Coverage at
The New York Times
WordPress VIP Developer Workshop
Napa, California - May 5, 2015
Scott Taylor
• Core Developer, WordPress
open source project
• Release Lead for 4.4
• Sr. Software Engineer, The
New York Times
• @wonderboymusic on Twitter/
Instagram/Swarm
• I like Music, NYC, and
Mexican food
Blogs at the New York Times
• The 00s were the glory days
for Blogs and WordPress
• Separate from the rest of the
codebase (NYT4)
• Global NYTimes CSS and JS,
CSS for all Blogs, custom CSS
per-blog
• A universe that assumed
jQuery AND Prototype were
loaded on the page in global
scope
What could possibly go
wrong here?
• Inline HTML from 2008 that assumes Prototype will
still be a thing in 2015, stored in post_content
• Widgets and inline code that add their own version
of jQuery/Prototype, because YOLO
• Even better: widgets/modules from other teams that
use a different version of jQuery … at times there
could be 4 jQuerys on the page (4 diff versions)
No shared modules
• Code/HTML markup can get out of sync with other
projects regularly (header/footer/nav)
• The CSS and JS files split across multiple SVN
repos - changes to global assets can affect us
without us knowing. Fixing the code requires
scouring through multiple repos.
• The blogs team divided front-end and back-end
responsibilities. Typically, no one had a full view of
features or the architecture as a whole
At the NYT, we don’t use …
• WordPress Comments: There is an entire team that deals with
these for the site globally, in a different system called CRNR
• Media: There is another CMS at the Times, Scoop, which
stores the images, videos, slideshows, etc
• WordPress native post-locking: This only landed in
WordPress core in version 3.6, we have yet to reconcile the
differences
• There is layer for Bylines which is separate from Users:
Our users are employees authenticated via LDAP, most post
authors don’t actually enter the content themselves
Live Blogs at the Times
Since 2008
• A Blog would create a post and check “Start Live
Blogging”
• the updates related to the post were stored in
custom tables in the database
• the APIs for interacting with these tables duplicated
tons of WordPress functionality
• Custom Post Types didn’t exist until WordPress 3.0
(June 2010) - the NYT code was never rewritten to
leverage them (would have required porting the
content as well)
As Time Went On, A Mess
to Update and Maintain
The Next Iteration:
Dashboards/Dashblogs
• A Live Blog would be its own blog in the network, its own
set of tables
• A special dashboard theme that had hooks to add
custom JS/CSS for each individual blog, without baking
them into the theme
• Making an entirely new site in the network for a 4-hour
event is overkill
• For every 10 or so new blogs that are added, you are
adding 100 new database tables - gross!
2013: The New Frontier
I arrived at the New York Times
NYT5
• NYT5 “apps” are Git repos that are transformed via
Grunt into a new directory structure. You can’t run
your repo as a website: it has to be built
• Impossible to create a “theme” this time with
shared JS and CSS - CSS is SASS, JS is Require.
• PHP for shared components has Composer
dependencies and uses namespaces - the
directories are expanded via Grunt in accordance
with PSR-0 Autoloading Standard
NYT5 Dealbreakers
• We can’t just point at WordPress on every request
and have our code figure out routing. Apache
points to app.php in NYT5
• Because PHP Namespaces are used, WP has to
load early and outside of them (global scope)
• On the frontend, WP cannot exit prematurely before
hitting the framework, which returns the response
to the server via 

SymfonyHttpFoundation
NYT5 Advantages
• “shared” modules - we inherit the “shell” of the page,
which includes: navigation, footer, login, etc.
• our nyt5 theme doesn’t need to produce an entire
HTML document, just the “content” portion
• With WP in global scope, all of its code is available even
when we hit the MVC parts of the NYT5 framework.
• WP output is captured via an output buffer on load - it’s
accessible downstream when the app logic is running.
• We can inherit the global Varnish setup and ditch
Batcache
• Instead of rendering media HTML, our [nytmedia]
shortcodes can just output “markers”
• The NYT5 logic for articles already knows to replace
these markers with markup that is generated from shared
modules - and the data is fetched using parallel HTTP
from internal asset stores (JSON).
• At the end of the day, we were able to eradicate a lot of
old, terrible code and lean on code from foundation
and shared repos
What about the old code that
used inline Prototype JS code?
• I wrote a filter on ‘the_content’ that does a quick string
scan to see if one of the old slideshows exists in post’s
content. If it does, I wrote RegEx that replaces the
slideshow blob on the fly with a new blob that leverages
our Require code.



I am not proud of this, but it works, and it replaces the
need to migrate potentially 100s or 1000s of posts in the
database.
• Whenever I can, I choose to not ignore legacy or archival
content - since it’s all still searchable on the net, why let it
die a death by Prototype?
Bad News for Blogs
• Blogs were duplicating Section Fronts, Columns:



Mark Bittman has column in the paper.

The column also exists on the web as an article. 

He contributes to the Diner’s Journal blog. 

There is a section front for dining. 

He also has his own NYTimes blog. Why?
• Blogs and WordPress were combined in everyone’s
mind. So whenever WordPress was mentioned as a
solution for anything, the response was: aren’t blogs
going away? #dark
First Draft
Lens
Live Coverage
What if…
• Instead of using those custom tables
and all of that weird code, we could
actually create new object types:
events and updates!
• To create a new “Live Blog”: you only
need to create an event, then go to a
Backbone-powered screen to add
updates
• Even if WordPress isn’t desired for the
front end, it could be the backend for
anything that wants a JSON feed for
live event data
• Because we would be using custom
post types, building a Live Event UI
that looks like the NYT5 theme would
be nominal
• Built an admin interface with Backbone to quickly
produce content - which in turn could be read from
JSON feeds.
• When saving, the updates post into a service we
have called Invisible City
• Our first real foray into using the JSON API (WP-
API)
• Our plan was just to be admin to produce data via
self-service URLs
What we did
Live Events
The new Live Blogs
Complete Rewrite of 2008 code
• nytimes.com/live/{event} and nytimes.com/live/{event}/
{update}
• Brand new admin interface: Backbone app that uses the
WP-API. Constantly updated filterable stream - Backbone
collections that re-fetch on Heartbeat tick
• Custom JSON endpoints that handle processes that need
to happen on save
• Front end served by WordPress for SEO, but data is
received by web socket from Invisible City and rendered
via React
Responsive
on Mobile
Some gotchas…
Most plugins only handle POST
• WP-API and Backbone speak REST
• REST will send you requests via 

PUT, DELETE, POST
$hook = add_menu_page( . . . );
add_action( “load-$hook”, ‘custom_load’ );
function old_custom_load() {
if ( ‘POST’ !== $_SERVER[‘REQUEST_METHOD’] ) {
return;
}
. . .
}
function new_custom_load() {
if ( ‘GET’ === $_SERVER[‘REQUEST_METHOD’] ) {
return;
}
. . .
}
HTTP is time-consuming
• Admin needs to be fast
• The front end is typically cached, but page generation
shouldn’t be bogged down by HTTP requests
• It is easy to lose track of how many things are
happening on the ‘save_post’ hook
• Anything which is time-consuming should be
offloaded to a separate “process” or request who
response you don’t need to handle
wp_remote_post( $url, wp_parse_args( array(
‘timeout’ => 0.01,
‘blocking’ => false
), $args ) );
Fire and Forget*
* Stolen from Mark Jaquith’s nginx cache invalidation technique:
wp_remote_get( $url, array(
‘timeout’ => 0.01,
‘blocking’ => false,
‘headers’ => array( ‘X-Nginx-Cache-Purge’ => ‘1’ )
) );
Custom JSON Endpoints for POST
• Use fire-and-forget technique on save, instead of
waiting for responses inline. You can still log/
handle/re-try responses in the separate request.
• Most things that happen on ‘save_post’ only
need to know $post_id for context, the endpoint
handler can call get_post() from there
Register a route:
$routes[ '/live-events/(?P<post_id>d+)/sns' ] = array(
array(
array( $this, 'live_event_sns' ),
WP_JSON_Server::CREATABLE
),
);
$routes[ '/async-cream-invalidation' ] = array(
array(
array( $this, 'async_cream' ),
WP_JSON_Server::CREATABLE
),
);
Handle the route:
public function async_cream() {
$urls = array_map( 'stripslashes', $_POST['urls'] );
if ( $urls ) {
nyt_cream_invalidation( $urls );
}
exit();
}
Trigger the process:
NYT_Admin_JSON::async_request(
'/async-cream-invalidation',
array(
‘body’ => array(
'urls' => $urls
)
)
);
Custom JSON Endpoints for GET
• We do not hit these endpoints on the front-end
• We have a storage mount that is fronted via Varnish
and Akamai
• JSON feeds can show up on the homepage of the
NYT to dynamically render “promos” - these have
to massively scale
An “Interactive
Promo” on an
article page
Questions?

More Related Content

What's hot

Saving Time By Testing With Jest
Saving Time By Testing With JestSaving Time By Testing With Jest
Saving Time By Testing With JestBen McCormick
 
Modern websites in 2020 and Joomla
Modern websites in 2020 and JoomlaModern websites in 2020 and Joomla
Modern websites in 2020 and JoomlaGeorge Wilson
 
Merging two big Symfony based applications - SymfonyCon 2017
Merging two big Symfony based applications - SymfonyCon 2017Merging two big Symfony based applications - SymfonyCon 2017
Merging two big Symfony based applications - SymfonyCon 2017Ivo Lukac
 
Untangling fall2017 week1
Untangling fall2017 week1Untangling fall2017 week1
Untangling fall2017 week1Derek Jacoby
 
Trying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress TodayTrying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress TodayDrewAPicture
 
My Contributor Story
My Contributor StoryMy Contributor Story
My Contributor StoryMarko Heijnen
 
Untangling spring week12
Untangling spring week12Untangling spring week12
Untangling spring week12Derek Jacoby
 
Untangling fall2017 week2
Untangling fall2017 week2Untangling fall2017 week2
Untangling fall2017 week2Derek Jacoby
 
Python to go
Python to goPython to go
Python to goWeng Wei
 
It Takes a Village to Make WordPress
It Takes a Village to Make WordPressIt Takes a Village to Make WordPress
It Takes a Village to Make WordPressDrewAPicture
 
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...Frank van der Linden
 
Java Fundamentals to Advance
Java Fundamentals to AdvanceJava Fundamentals to Advance
Java Fundamentals to AdvanceKrish
 
Introduction to apache maven
Introduction to apache mavenIntroduction to apache maven
Introduction to apache mavenKrish
 
MEAN Stack Workshop at Node Philly, 4/9/14
MEAN Stack Workshop at Node Philly, 4/9/14MEAN Stack Workshop at Node Philly, 4/9/14
MEAN Stack Workshop at Node Philly, 4/9/14Valeri Karpov
 
Riak at Posterous
Riak at PosterousRiak at Posterous
Riak at Posterouscapotej
 
Modern javascript
Modern javascriptModern javascript
Modern javascriptKevin Ball
 
Migrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoMigrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoWeng Wei
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkinsKrish
 
Unlocking the Magical Powers of WP_Query
Unlocking the Magical Powers of WP_QueryUnlocking the Magical Powers of WP_Query
Unlocking the Magical Powers of WP_QueryDustin Filippini
 
SharePoint Development 101
SharePoint Development 101SharePoint Development 101
SharePoint Development 101Greg Hurlman
 

What's hot (20)

Saving Time By Testing With Jest
Saving Time By Testing With JestSaving Time By Testing With Jest
Saving Time By Testing With Jest
 
Modern websites in 2020 and Joomla
Modern websites in 2020 and JoomlaModern websites in 2020 and Joomla
Modern websites in 2020 and Joomla
 
Merging two big Symfony based applications - SymfonyCon 2017
Merging two big Symfony based applications - SymfonyCon 2017Merging two big Symfony based applications - SymfonyCon 2017
Merging two big Symfony based applications - SymfonyCon 2017
 
Untangling fall2017 week1
Untangling fall2017 week1Untangling fall2017 week1
Untangling fall2017 week1
 
Trying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress TodayTrying Out Tomorrow’s WordPress Today
Trying Out Tomorrow’s WordPress Today
 
My Contributor Story
My Contributor StoryMy Contributor Story
My Contributor Story
 
Untangling spring week12
Untangling spring week12Untangling spring week12
Untangling spring week12
 
Untangling fall2017 week2
Untangling fall2017 week2Untangling fall2017 week2
Untangling fall2017 week2
 
Python to go
Python to goPython to go
Python to go
 
It Takes a Village to Make WordPress
It Takes a Village to Make WordPressIt Takes a Village to Make WordPress
It Takes a Village to Make WordPress
 
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...Engage 2019: The good, the bad and the ugly: a not so objective view on front...
Engage 2019: The good, the bad and the ugly: a not so objective view on front...
 
Java Fundamentals to Advance
Java Fundamentals to AdvanceJava Fundamentals to Advance
Java Fundamentals to Advance
 
Introduction to apache maven
Introduction to apache mavenIntroduction to apache maven
Introduction to apache maven
 
MEAN Stack Workshop at Node Philly, 4/9/14
MEAN Stack Workshop at Node Philly, 4/9/14MEAN Stack Workshop at Node Philly, 4/9/14
MEAN Stack Workshop at Node Philly, 4/9/14
 
Riak at Posterous
Riak at PosterousRiak at Posterous
Riak at Posterous
 
Modern javascript
Modern javascriptModern javascript
Modern javascript
 
Migrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to GoMigrate PHP E-Commerce Site to Go
Migrate PHP E-Commerce Site to Go
 
Introduction to jenkins
Introduction to jenkinsIntroduction to jenkins
Introduction to jenkins
 
Unlocking the Magical Powers of WP_Query
Unlocking the Magical Powers of WP_QueryUnlocking the Magical Powers of WP_Query
Unlocking the Magical Powers of WP_Query
 
SharePoint Development 101
SharePoint Development 101SharePoint Development 101
SharePoint Development 101
 

Similar to Live Coverage at The New York Times

Improve WordPress performance with caching and deferred execution of code
Improve WordPress performance with caching and deferred execution of codeImprove WordPress performance with caching and deferred execution of code
Improve WordPress performance with caching and deferred execution of codeDanilo Ercoli
 
Untangling spring week5
Untangling spring week5Untangling spring week5
Untangling spring week5Derek Jacoby
 
The WordPress University
The WordPress UniversityThe WordPress University
The WordPress UniversityStephanie Leary
 
Mobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPressMobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPressDanilo Ercoli
 
GeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPressGeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPressGGDBologna
 
Best Practices for Building WordPress Applications
Best Practices for Building WordPress ApplicationsBest Practices for Building WordPress Applications
Best Practices for Building WordPress ApplicationsTaylor Lovett
 
JavaScript Service Worker Design Patterns for Better User Experience
JavaScript Service Worker Design Patterns for Better User ExperienceJavaScript Service Worker Design Patterns for Better User Experience
JavaScript Service Worker Design Patterns for Better User Experiencereeder29
 
WebNetConf 2012 - Single Page Apps
WebNetConf 2012 - Single Page AppsWebNetConf 2012 - Single Page Apps
WebNetConf 2012 - Single Page AppsPop Apps
 
WordPress Intermediate Workshop
WordPress Intermediate WorkshopWordPress Intermediate Workshop
WordPress Intermediate WorkshopThe Toolbox, Inc.
 
Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo
Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo
Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo Shift Conference
 
Streams API (Web Engines Hackfest 2015)
Streams API (Web Engines Hackfest 2015)Streams API (Web Engines Hackfest 2015)
Streams API (Web Engines Hackfest 2015)Igalia
 
Best Practices for WordPress
Best Practices for WordPressBest Practices for WordPress
Best Practices for WordPressTaylor Lovett
 
Run-time of Node.js : V8 JavaScript Engine
Run-time of Node.js: V8 JavaScript EngineRun-time of Node.js: V8 JavaScript Engine
Run-time of Node.js : V8 JavaScript EngineGary Yeh
 
From WordPress With Love
From WordPress With LoveFrom WordPress With Love
From WordPress With LoveUp2 Technology
 
Building the next generation of themes with WP Rig 2.0
Building the next generation of themes with WP Rig 2.0Building the next generation of themes with WP Rig 2.0
Building the next generation of themes with WP Rig 2.0Morten Rand-Hendriksen
 
Untangling spring week11
Untangling spring week11Untangling spring week11
Untangling spring week11Derek Jacoby
 
Wordpress beyond blogging
Wordpress beyond bloggingWordpress beyond blogging
Wordpress beyond bloggingJulien Minguely
 
Should you use HTML5 to build your product? The pros & cons of using current ...
Should you use HTML5 to build your product? The pros & cons of using current ...Should you use HTML5 to build your product? The pros & cons of using current ...
Should you use HTML5 to build your product? The pros & cons of using current ...boxuno
 

Similar to Live Coverage at The New York Times (20)

Improve WordPress performance with caching and deferred execution of code
Improve WordPress performance with caching and deferred execution of codeImprove WordPress performance with caching and deferred execution of code
Improve WordPress performance with caching and deferred execution of code
 
Untangling spring week5
Untangling spring week5Untangling spring week5
Untangling spring week5
 
The WordPress University
The WordPress UniversityThe WordPress University
The WordPress University
 
Mobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPressMobile Hybrid Development with WordPress
Mobile Hybrid Development with WordPress
 
GeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPressGeneralMobile Hybrid Development with WordPress
GeneralMobile Hybrid Development with WordPress
 
Best Practices for Building WordPress Applications
Best Practices for Building WordPress ApplicationsBest Practices for Building WordPress Applications
Best Practices for Building WordPress Applications
 
JavaScript Service Worker Design Patterns for Better User Experience
JavaScript Service Worker Design Patterns for Better User ExperienceJavaScript Service Worker Design Patterns for Better User Experience
JavaScript Service Worker Design Patterns for Better User Experience
 
WebNetConf 2012 - Single Page Apps
WebNetConf 2012 - Single Page AppsWebNetConf 2012 - Single Page Apps
WebNetConf 2012 - Single Page Apps
 
WordPress Intermediate Workshop
WordPress Intermediate WorkshopWordPress Intermediate Workshop
WordPress Intermediate Workshop
 
Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo
Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo
Shift Remote: JS - Javascript Build Tools: Past & Beyond - Shedrack Akintayo
 
Agile sites2
Agile sites2Agile sites2
Agile sites2
 
Streams API (Web Engines Hackfest 2015)
Streams API (Web Engines Hackfest 2015)Streams API (Web Engines Hackfest 2015)
Streams API (Web Engines Hackfest 2015)
 
Best Practices for WordPress
Best Practices for WordPressBest Practices for WordPress
Best Practices for WordPress
 
Run-time of Node.js : V8 JavaScript Engine
Run-time of Node.js: V8 JavaScript EngineRun-time of Node.js: V8 JavaScript Engine
Run-time of Node.js : V8 JavaScript Engine
 
From WordPress With Love
From WordPress With LoveFrom WordPress With Love
From WordPress With Love
 
Building the next generation of themes with WP Rig 2.0
Building the next generation of themes with WP Rig 2.0Building the next generation of themes with WP Rig 2.0
Building the next generation of themes with WP Rig 2.0
 
Untangling spring week11
Untangling spring week11Untangling spring week11
Untangling spring week11
 
Wordpress beyond blogging
Wordpress beyond bloggingWordpress beyond blogging
Wordpress beyond blogging
 
Should you use HTML5 to build your product? The pros & cons of using current ...
Should you use HTML5 to build your product? The pros & cons of using current ...Should you use HTML5 to build your product? The pros & cons of using current ...
Should you use HTML5 to build your product? The pros & cons of using current ...
 
WordPress Complete Tutorial
WordPress Complete TutorialWordPress Complete Tutorial
WordPress Complete Tutorial
 

Recently uploaded

What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shardsChristopher Curtin
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdfSteve Caron
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...OnePlan Solutions
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptxVinzoCenzo
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?Alexandre Beguel
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITmanoharjgpsolutions
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorTier1 app
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...OnePlan Solutions
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxSasikiranMarri
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfmaor17
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxRTS corp
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...kalichargn70th171
 

Recently uploaded (20)

What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
 
Osi security architecture in network.pptx
Osi security architecture in network.pptxOsi security architecture in network.pptx
Osi security architecture in network.pptx
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh IT
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryError
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
Tech Tuesday Slides - Introduction to Project Management with OnePlan's Work ...
 
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdf
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptx
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
 

Live Coverage at The New York Times

  • 1. Live Coverage at The New York Times WordPress VIP Developer Workshop Napa, California - May 5, 2015
  • 2. Scott Taylor • Core Developer, WordPress open source project • Release Lead for 4.4 • Sr. Software Engineer, The New York Times • @wonderboymusic on Twitter/ Instagram/Swarm • I like Music, NYC, and Mexican food
  • 3. Blogs at the New York Times • The 00s were the glory days for Blogs and WordPress • Separate from the rest of the codebase (NYT4) • Global NYTimes CSS and JS, CSS for all Blogs, custom CSS per-blog • A universe that assumed jQuery AND Prototype were loaded on the page in global scope
  • 4. What could possibly go wrong here?
  • 5. • Inline HTML from 2008 that assumes Prototype will still be a thing in 2015, stored in post_content • Widgets and inline code that add their own version of jQuery/Prototype, because YOLO • Even better: widgets/modules from other teams that use a different version of jQuery … at times there could be 4 jQuerys on the page (4 diff versions)
  • 6. No shared modules • Code/HTML markup can get out of sync with other projects regularly (header/footer/nav) • The CSS and JS files split across multiple SVN repos - changes to global assets can affect us without us knowing. Fixing the code requires scouring through multiple repos. • The blogs team divided front-end and back-end responsibilities. Typically, no one had a full view of features or the architecture as a whole
  • 7. At the NYT, we don’t use … • WordPress Comments: There is an entire team that deals with these for the site globally, in a different system called CRNR • Media: There is another CMS at the Times, Scoop, which stores the images, videos, slideshows, etc • WordPress native post-locking: This only landed in WordPress core in version 3.6, we have yet to reconcile the differences • There is layer for Bylines which is separate from Users: Our users are employees authenticated via LDAP, most post authors don’t actually enter the content themselves
  • 8. Live Blogs at the Times Since 2008
  • 9. • A Blog would create a post and check “Start Live Blogging” • the updates related to the post were stored in custom tables in the database • the APIs for interacting with these tables duplicated tons of WordPress functionality • Custom Post Types didn’t exist until WordPress 3.0 (June 2010) - the NYT code was never rewritten to leverage them (would have required porting the content as well)
  • 10. As Time Went On, A Mess to Update and Maintain
  • 11. The Next Iteration: Dashboards/Dashblogs • A Live Blog would be its own blog in the network, its own set of tables • A special dashboard theme that had hooks to add custom JS/CSS for each individual blog, without baking them into the theme • Making an entirely new site in the network for a 4-hour event is overkill • For every 10 or so new blogs that are added, you are adding 100 new database tables - gross!
  • 12. 2013: The New Frontier I arrived at the New York Times
  • 13.
  • 14.
  • 15.
  • 16. NYT5 • NYT5 “apps” are Git repos that are transformed via Grunt into a new directory structure. You can’t run your repo as a website: it has to be built • Impossible to create a “theme” this time with shared JS and CSS - CSS is SASS, JS is Require. • PHP for shared components has Composer dependencies and uses namespaces - the directories are expanded via Grunt in accordance with PSR-0 Autoloading Standard
  • 17. NYT5 Dealbreakers • We can’t just point at WordPress on every request and have our code figure out routing. Apache points to app.php in NYT5 • Because PHP Namespaces are used, WP has to load early and outside of them (global scope) • On the frontend, WP cannot exit prematurely before hitting the framework, which returns the response to the server via 
 SymfonyHttpFoundation
  • 18. NYT5 Advantages • “shared” modules - we inherit the “shell” of the page, which includes: navigation, footer, login, etc. • our nyt5 theme doesn’t need to produce an entire HTML document, just the “content” portion • With WP in global scope, all of its code is available even when we hit the MVC parts of the NYT5 framework. • WP output is captured via an output buffer on load - it’s accessible downstream when the app logic is running.
  • 19. • We can inherit the global Varnish setup and ditch Batcache • Instead of rendering media HTML, our [nytmedia] shortcodes can just output “markers” • The NYT5 logic for articles already knows to replace these markers with markup that is generated from shared modules - and the data is fetched using parallel HTTP from internal asset stores (JSON). • At the end of the day, we were able to eradicate a lot of old, terrible code and lean on code from foundation and shared repos
  • 20. What about the old code that used inline Prototype JS code? • I wrote a filter on ‘the_content’ that does a quick string scan to see if one of the old slideshows exists in post’s content. If it does, I wrote RegEx that replaces the slideshow blob on the fly with a new blob that leverages our Require code.
 
 I am not proud of this, but it works, and it replaces the need to migrate potentially 100s or 1000s of posts in the database. • Whenever I can, I choose to not ignore legacy or archival content - since it’s all still searchable on the net, why let it die a death by Prototype?
  • 21. Bad News for Blogs • Blogs were duplicating Section Fronts, Columns:
 
 Mark Bittman has column in the paper.
 The column also exists on the web as an article. 
 He contributes to the Diner’s Journal blog. 
 There is a section front for dining. 
 He also has his own NYTimes blog. Why? • Blogs and WordPress were combined in everyone’s mind. So whenever WordPress was mentioned as a solution for anything, the response was: aren’t blogs going away? #dark
  • 23.
  • 24.
  • 25. What if… • Instead of using those custom tables and all of that weird code, we could actually create new object types: events and updates! • To create a new “Live Blog”: you only need to create an event, then go to a Backbone-powered screen to add updates • Even if WordPress isn’t desired for the front end, it could be the backend for anything that wants a JSON feed for live event data • Because we would be using custom post types, building a Live Event UI that looks like the NYT5 theme would be nominal
  • 26. • Built an admin interface with Backbone to quickly produce content - which in turn could be read from JSON feeds. • When saving, the updates post into a service we have called Invisible City • Our first real foray into using the JSON API (WP- API) • Our plan was just to be admin to produce data via self-service URLs What we did
  • 27. Live Events The new Live Blogs
  • 28. Complete Rewrite of 2008 code • nytimes.com/live/{event} and nytimes.com/live/{event}/ {update} • Brand new admin interface: Backbone app that uses the WP-API. Constantly updated filterable stream - Backbone collections that re-fetch on Heartbeat tick • Custom JSON endpoints that handle processes that need to happen on save • Front end served by WordPress for SEO, but data is received by web socket from Invisible City and rendered via React
  • 29.
  • 30.
  • 33. Most plugins only handle POST • WP-API and Backbone speak REST • REST will send you requests via 
 PUT, DELETE, POST
  • 34. $hook = add_menu_page( . . . ); add_action( “load-$hook”, ‘custom_load’ ); function old_custom_load() { if ( ‘POST’ !== $_SERVER[‘REQUEST_METHOD’] ) { return; } . . . } function new_custom_load() { if ( ‘GET’ === $_SERVER[‘REQUEST_METHOD’] ) { return; } . . . }
  • 35. HTTP is time-consuming • Admin needs to be fast • The front end is typically cached, but page generation shouldn’t be bogged down by HTTP requests • It is easy to lose track of how many things are happening on the ‘save_post’ hook • Anything which is time-consuming should be offloaded to a separate “process” or request who response you don’t need to handle
  • 36. wp_remote_post( $url, wp_parse_args( array( ‘timeout’ => 0.01, ‘blocking’ => false ), $args ) ); Fire and Forget* * Stolen from Mark Jaquith’s nginx cache invalidation technique: wp_remote_get( $url, array( ‘timeout’ => 0.01, ‘blocking’ => false, ‘headers’ => array( ‘X-Nginx-Cache-Purge’ => ‘1’ ) ) );
  • 37. Custom JSON Endpoints for POST • Use fire-and-forget technique on save, instead of waiting for responses inline. You can still log/ handle/re-try responses in the separate request. • Most things that happen on ‘save_post’ only need to know $post_id for context, the endpoint handler can call get_post() from there
  • 38. Register a route: $routes[ '/live-events/(?P<post_id>d+)/sns' ] = array( array( array( $this, 'live_event_sns' ), WP_JSON_Server::CREATABLE ), ); $routes[ '/async-cream-invalidation' ] = array( array( array( $this, 'async_cream' ), WP_JSON_Server::CREATABLE ), );
  • 39. Handle the route: public function async_cream() { $urls = array_map( 'stripslashes', $_POST['urls'] ); if ( $urls ) { nyt_cream_invalidation( $urls ); } exit(); }
  • 41. Custom JSON Endpoints for GET • We do not hit these endpoints on the front-end • We have a storage mount that is fronted via Varnish and Akamai • JSON feeds can show up on the homepage of the NYT to dynamically render “promos” - these have to massively scale
  • 42. An “Interactive Promo” on an article page