Here's one we made earlier Building a new infrastructure for bbc.co.uk... ... and rolling out a new Identity platform on it Brendan Quinn Ben Smith 8 May 2008

Building a new infrastructure for bbc.co.uk... ... and rolling out a new Identity platform on it

Brendan Quinn Ben Smith

8 May 2008

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to Identity Development / deployment process

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to Identity

Development / deployment process

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

We’ve come a long way…

Problem #1: Growth without revenue...

Future Media & Technology BBC iPlayer

/programmes Future Media & Technology

Future Media & Technology BBC Homepage

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process And how it applies to identity

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

And how it applies to identity

BBC Identity circa. 2000 Disparate authentication systems Inconsistent user experience Differing levels of security DPA compliance

Disparate authentication systems

Inconsistent user experience

Differing levels of security

DPA compliance

“ Single Sign-On” (SSO) ‏

“ Single Sign-On” (SSO) ‏ Closed-source C interface with Perl bindings mod_perl web-application One-size-fits-all architecture

Closed-source

C interface with Perl bindings

mod_perl web-application

One-size-fits-all architecture

Evolution Gradual evolution of the web Antiquated web platform Inflexible / proprietary architecture SSO was unable to adapt

Gradual evolution of the web

Antiquated web platform

Inflexible / proprietary architecture

SSO was unable to adapt

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process And how it applies to identity

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

And how it applies to identity

Architectural principles Each project must have a clear customer and deliver a real benefit. ideally to an end-user... Don't Repeat Yourself (DRY) (or other people for that matter!) Life is too short to spend your time re-inventing things... Be as simple as possible (YAGNI: “you ain’t gonna need it”) Just do what we need to do now... Be as open as possible Assume that all services can be accessed from outside the BBC, by default...

Each project must have a clear customer and deliver a real benefit. ideally to an end-user...

Don't Repeat Yourself (DRY) (or other people for that matter!) Life is too short to spend your time re-inventing things...

Be as simple as possible (YAGNI: “you ain’t gonna need it”) Just do what we need to do now...

Be as open as possible Assume that all services can be accessed from outside the BBC, by default...

Architectural principles (cont) ‏ Start simple, then iterate Build the smallest thing you could possibly need, deploy it, then build applications on top of it. Think building blocks, not monoliths. Don’t try to do everything straight away, but do something quickly... Don't optimise prematurely… The service might not grow the way we think it will... … but build to scale think stateless, think content delivery networks, think database resilience (Are these two points contradictory? We say no… in our environment at least… because it’s not premature!)‏

Start simple, then iterate Build the smallest thing you could possibly need, deploy it, then build applications on top of it. Think building blocks, not monoliths. Don’t try to do everything straight away, but do something quickly...

Don't optimise prematurely… The service might not grow the way we think it will...

… but build to scale think stateless, think content delivery networks, think database resilience (Are these two points contradictory? We say no… in our environment at least… because it’s not premature!)‏

Architectural principles (cont) ‏ Test often So you know when you need to optimise So you can maintain your code So you can maintain your platform Evolve Teams, systems, support structures The platform These principles..! Let it die (aka “Fall forward fast”) Be prepared to turn your system off... or change it unrecognisably

Test often So you know when you need to optimise So you can maintain your code So you can maintain your platform

Evolve Teams, systems, support structures The platform These principles..!

Let it die (aka “Fall forward fast”) Be prepared to turn your system off... or change it unrecognisably

BBC Tech Refresh Architectural principles Each project must have a clear customer and deliver a real benefit Don't repeat yourself Be as simple as possible Be as open as possible Start simple, then iterate Don't optimise prematurely… … but build to scale Test often Evolve Let it die

Each project must have a clear customer and deliver a real benefit

Don't repeat yourself

Be as simple as possible

Be as open as possible

Start simple, then iterate

Don't optimise prematurely…

… but build to scale

Test often

Evolve

Let it die

What benefits will it bring ? Flexible Capability to build richer functionality Clean separation of application logic from presentation logic improves scale and ease Make it easier to build applications Standardised Systems are easier to understand We can (and do) use third party software off-the-shelf We can draw on industry best practice Working towards a shared technology stack across the BBC Manageable Easier operational support Flexibility to scale & respond to traffic Improved testing and release processes Better visibility for security & performance

Flexible

Capability to build richer functionality

Clean separation of application logic from presentation logic improves scale and ease

Make it easier to build applications

Standardised

Systems are easier to understand

We can (and do) use third party software off-the-shelf

We can draw on industry best practice

Working towards a shared technology stack across the BBC

Manageable

Easier operational support

Flexibility to scale & respond to traffic

Improved testing and release processes

Better visibility for security & performance

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

“Identity” “ the anticipated revolution of identity verification on the internet using emerging user-centric technologies”

“ the anticipated revolution of identity verification on the internet using emerging user-centric technologies”

BBC Pentagram of Data Users?

Identity Links the BBC to its Audience

BBC Pentagram of Data BBC Identity

Linking Content and Users Brendan comments on the PM Blog Ben knows Brendan Ben finds out about BBC Blogs

Brendan comments on the PM Blog

Ben knows Brendan

Ben finds out about BBC Blogs

Linking Content and Users Brendan listens to Portishead Ben knows Brendan Ben finds out about Portishead on /music

Brendan listens to Portishead

Ben knows Brendan

Ben finds out about Portishead on /music

Linking Users via the Internet ...

BBC Pentagram of Data BBC Identity

BBC Application Developers Personal and public profiles Personalisation Customisation Attention tracking Recommendations Notifications Social Networks

Personal and public profiles

Personalisation

Customisation

Attention tracking

Recommendations

Notifications

Social Networks

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

So what is it then ? Page Assembly Layer Caching Layer Data Services Layer Database Layer Traffic Management Layer

Technology stack Services can be exposed via Backstage or to indies Dynamic “page assembly” in PHP Handle the simple stuff separately for performance (CSS, JS, images) ‏ Identity and GeoIP built in, available everywhere

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

Paying off the Technical Overdraft

Ideals Toolbox of Identity services Layered Identity Need-to-know No wheel re-invention

Toolbox of Identity services

Layered Identity

Need-to-know

No wheel re-invention

Architecture Service Oriented Open-source throughout: MySQL Repository OpenSSO Identity Management Core Java REST Services (Spring MVC) ‏ PHP presentation layer (Zend Framework) ‏

Service Oriented

Open-source throughout:

MySQL Repository

OpenSSO Identity Management Core

Java REST Services (Spring MVC) ‏

PHP presentation layer (Zend Framework) ‏

Security Under 16s ~70% of registered user-base Can't ignore them Work hard to maintain trust in the BBC Payment Card Industry standard Treat personal data like credit-card information

Under 16s ~70% of registered user-base

Can't ignore them

Work hard to maintain trust in the BBC

Payment Card Industry standard

Treat personal data like credit-card information

Performance Enormous estimated growth of user-base Identity ubiquitously available across bbc.co.uk Authorisation of every request to bbc.co.uk Horizontal scalability within each layer Intelligent load-balancing Caching and session-management

Enormous estimated growth of user-base

Identity ubiquitously available across bbc.co.uk

Authorisation of every request to bbc.co.uk

Horizontal scalability within each layer

Intelligent load-balancing

Caching and session-management

Overview Why are we doing this? Problems we’re solving With the platform With the identity system Architectural/design principles Vision for Identity Overview of the stack And how it applies to identity Development / deployment process

Why are we doing this?

Problems we’re solving

With the platform

With the identity system

Architectural/design principles

Vision for Identity

Overview of the stack

And how it applies to identity

Development / deployment process

Processes Development environments Local sandbox Internet access Available to indies Release processes Packages (RPMs) ‏ Integration testing Staging and QA Operational support Monitoring (systems and apps) ‏ Capacity planning Live & test environments Consistent with dev Managed upgrades Partitioned

Development environments

Local sandbox

Internet access

Available to indies

Release processes

Packages (RPMs) ‏

Integration testing

Staging and QA

Operational support

Monitoring (systems and apps) ‏

Capacity planning

Live & test environments

Consistent with dev

Managed upgrades

Partitioned

Roadmap – how we get there Build a scale model Wrap processes around it Review traffic Prototype Beta Build a small but complete version of platform Launch some applications Get feedback Live Blueprint Build resilience Add more applications Get more feedback Working platform Live platform

Build a scale model

Wrap processes around it

Review traffic

Build a small but complete version of platform

Launch some applications

Get feedback

Build resilience

Add more applications

Get more feedback

Still to come Content management integration “ inversion of control” ideas Simple data services No coding needed Messaging and notifications MQ, XMPP…? See you next year…?

Content management integration

“ inversion of control” ideas

Simple data services

No coding needed

Messaging and notifications

MQ, XMPP…?

See you next year…?

Questions? [email_address] [email_address]

Photo acknowledgements http://www.flickr.com/photos/stabilo-boss/93136022/ http://www.nostalgiacentral.com/tv/kids/bluepeter.htm

http://www.flickr.com/photos/stabilo-boss/93136022/

http://www.nostalgiacentral.com/tv/kids/bluepeter.htm