XTech 2008 - BBC Tech Refresh and Identity - Brendan Quinn And Ben Smith

Slide 1: Here's one we made earlier Building a new infrastructure for bbc.co.uk... ... and rolling out a new Identity platform on it Brendan Quinn Ben Smith 8 May 2008

Slide 2: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to Identity  Development / deployment process 2

Slide 3: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process 3

Slide 4: We’ve come a long way…

Slide 5: Problem #1: Growth without revenue... 60.0 1700 Thousands Gigabytes Transferred 1600 January 2008 55.0 1500 Internet Traffic Growth [ SIS hosted ] 50.0 1400 1300 1600 TB ~ 51.6 TB/day 45.0 1200 40.0 1100 Transfer Rate (TB/day) 1000 35.0 900 30.0 800 25.0 700 600 20.0 500 15.0 400 300 10.0 200 5.0 100 0 0.0 Jan- Mar- May- Jul- Sep- Nov- Jan- Mar- May- Jul- Sep- Nov- Jan- Mar- May- Jul- Sep- Nov- Jan- Mar- May- Jul- Sep- Nov- Jan- Mar- May- Jul- Sep- Nov- Jan- 03 03 03 03 03 03 04 04 04 04 04 04 05 05 05 05 05 05 06 06 06 06 06 06 07 07 07 07 07 07 08 Month Daily Transfer rate (TB) BOL HTML Data Transferred (GB) RealMedia Data Transferred (GB) Windows Media Data Transferred (GB) NOL NY Data Transferred (GB) NOL UK Data Transferred (GB) Total Data Transferred (GB) Downloads.bbc Data transferred (GB) iPlayer P2P (origin) traffic 5

Slide 6: BBC iPlayer 6 Future Media & Technology

Slide 7: /programmes 7 Future Media & Technology

Slide 8: BBC Homepage 8 Future Media & Technology

Slide 9: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process  And how it applies to identity 9

Slide 10: BBC Identity circa. 2000  Disparate authentication systems  Inconsistent user experience  Differing levels of security  DPA compliance

Slide 11: “Single Sign-On” (SSO)

Slide 12: “Single Sign-On” (SSO)  Closed-source  C interface with Perl bindings  mod_perl web-application  One-size-fits-all architecture

Slide 13: Evolution  Gradual evolution of the web  Antiquated web platform  Inflexible / proprietary architecture  SSO was unable to adapt

Slide 14: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process  And how it applies to identity 14

Slide 15: Architectural principles 1. Each project must have a clear customer and deliver a real benefit. ideally to an end-user... 2. Don't Repeat Yourself (DRY) (or other people for that matter!) Life is too short to spend your time re-inventing things... 3. Be as simple as possible (YAGNI: “you ain’t gonna need it”) Just do what we need to do now... 4. Be as open as possible Assume that all services can be accessed from outside the BBC, by default... 15

Slide 16: Architectural principles (cont) 1. Start simple, then iterate Build the smallest thing you could possibly need, deploy it, then build applications on top of it. Think building blocks, not monoliths. Don’t try to do everything straight away, but do something quickly... 2. Don't optimise prematurely… The service might not grow the way we think it will... 3. … but build to scale think stateless, think content delivery networks, think database resilience (Are these two points contradictory? We say no… in our environment at least… because it’s not premature!) 16

Slide 17: Architectural principles (cont) 1. Test often So you know when you need to optimise So you can maintain your code So you can maintain your platform 2. Evolve Teams, systems, support structures The platform These principles..! 3. Let it die (aka “Fall forward fast”) Be prepared to turn your system off... or change it unrecognisably 17

Slide 18: BBC Tech Refresh Architectural principles 1. Each project must have a clear customer and deliver a real benefit 2. Don't repeat yourself 3. Be as simple as possible 4. Be as open as possible 5. Start simple, then iterate 6. Don't optimise prematurely… 7. … but build to scale 8. Test often 9. Evolve 10. Let it die 18

Slide 19: What benefits will it bring ? Standardised Flexible • Systems are easier to understand • Capability to build richer • We can (and do) use third party functionality software off-the-shelf • Clean separation of • We can draw on industry best application logic from practice presentation logic improves • Working towards a shared scale and ease technology stack across the BBC • Make it easier to build applications Manageable • Easier operational support • Flexibility to scale & respond to traffic • Improved testing and release processes • Better visibility for security & performance 19

Slide 20: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process 20

Slide 21: “Identity” “the anticipated revolution of identity verification on the internet using emerging user-centric technologies”

Slide 22: BBC Pentagram of Data Users?

Slide 25: Identity Links the BBC to its Audience

Slide 26: BBC Pentagram of Data BBC Identity

Slide 27: Linking Content and Users  Brendan comments on the PM Blog  Ben knows Brendan  Ben finds out about BBC Blogs

Slide 28: Linking Content and Users  Brendan listens to Portishead  Ben knows Brendan  Ben finds out about Portishead on /music

Slide 29: Linking Users via the Internet ...

Slide 30: BBC Pentagram of Data BBC Identity

Slide 31: BBC Application Developers  Personal and public profiles  Personalisation  Customisation  Attention tracking  Recommendations  Notifications  Social Networks

Slide 32: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process 32

Slide 33: So what is it then ? Traffic Management Layer Page Assembly Layer Caching Layer Data Services Layer Database Layer 33

Slide 34: Technology stack Handle the Identity and simple stuff Dynamic GeoIP built separately for “page in, available performance assembly” everywhere (CSS, JS, in PHP images) Services can be exposed via Backstage or to indies 34

Slide 35: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process 35

Slide 36: Paying off the Technical Overdraft

Slide 37: Ideals  Toolbox of Identity services  Layered Identity  Need-to-know  No wheel re-invention

Slide 38: Architecture Service Oriented Open-source throughout:  MySQL Repository  OpenSSO Identity Management Core  Java REST Services (Spring MVC)  PHP presentation layer (Zend Framework)

Slide 40: Security Under 16s ~70% of registered user-base   Can't ignore them Work hard to maintain trust in the BBC  Payment Card Industry standard   Treat personal data like credit-card information

Slide 41: Performance  Enormous estimated growth of user-base  Identity ubiquitously available across bbc.co.uk  Authorisation of every request to bbc.co.uk  Horizontal scalability within each layer  Intelligent load-balancing  Caching and session-management

Slide 42: Overview  Why are we doing this?  Problems we’re solving  With the platform  With the identity system  Architectural/design principles  Vision for Identity  Overview of the stack  And how it applies to identity  Development / deployment process 42

Slide 43: Processes Development Live & test environments environments • Local sandbox • Consistent with dev • Internet access • Managed upgrades • Available to • Partitioned indies Release Operational processes support • Packages (RPMs) • Monitoring • Integration testing (systems and apps) • Staging and QA • Capacity planning 43

Slide 44: Roadmap – how we get there Prototype Beta Live Working platform • Build a • Build a small Live platform scale • Build but complete model resilience Blueprint version of platform • Wrap • Add more processes applications • Launch some around it applications • Get more • Review feedback • Get feedback traffic 44

Slide 45: Still to come  Content management integration  “inversion of control” ideas  Simple data services  No coding needed  Messaging and notifications  MQ, XMPP…?  See you next year…?

Slide 46: Brendan.Quinn@bbc.co.uk Questions? Ben.Smith@bbc.co.uk 46

Slide 47: Photo acknowledgements  http://www.flickr.com/photos/stabilo-boss/93136022/  http://www.nostalgiacentral.com/tv/kids/bluepeter.htm 47