BBC's presentation for the GraphDB user group

1. Moving OWLIM
to Amazon Web Services

2. GraphDB
Moving OWLIM
to Amazon Web Services

3. Introduction to LDP
● Stands for Linked Data Platform
● Building APIs and Tools for semantic
publishing at the BBC
● Team of 8-12 people (5-8 developers)
● Based in London (moving to Salford)

4. Introduction to LDP
● Tagging and surfacing content

5. Why move to AWS?
● Started with taking ownership of Owlim
● BBC shift to move to AWS
● Loss of key people
● Freedom with responsibility

6. Current Architecture

7. Cloud Architecture

8. Keeping in sync

11. OpsWorks
● Part of AWS
● Easily deploy things to EC2

12. OpsWorks
● Define infrastructure in a stack
● The stack is comprised of layers
● Each layer has a role (eg. master, slave)
● Layers contain a number of instances

17. OpsWorks – Advantages
● Conceptually, fairly simple.
● Entire stack defined as JSON.
○ Can be (and is) version controlled.
● Very easy to clone an entire stack.
○ We use this to spin up ‘experimental’ environments.
○ Next step: blue/green deployments?

18. OpsWorks – Disadvantages
● It can be a little slow.
● Chef support can be tricky to debug.

20. Chef
● “Infrastructure as code”
● Cookbook, made up of recipes.
○ Each recipe installs/configures a specific thing.
● Each layer has a set of recipes that define it.

21. include_recipe 'tomcat::default'
cookbook_file 'owlim-production.license' do
path File.join(node[:owlim][:basedir], 'owlim.license')
mode 0644
owner 'tomcat'
group 'tomcat'
end
execute 'create_default_cluster_repository' do
command SystemRepository.create_repo 'default-cluster'
not_if SystemRepository.repo_exists? 'default-cluster'
action :nothing
end

23. Lifecycle
events

24. Recipes for each event type

25. Chef – Advantages
● A common and well-documented tool.
○ And it’s Ruby, a skill we possess.
● Can re-use existing cookbooks.
○ eg. Tomcat, Yum.
● Should be platform-agnostic.
○ Can theoretically use the same Chef recipes against
any OS, if written well.

26. Chef – Disadvantages
● OpsWorks support is a little lacking.
○ Using Berkshelf for dependencies is tricky.
○ Many of the error messages are a bit cryptic.

27. What next?

28. What next?
● Autoscaling
○ Not supported by OpsWorks.
○ For now: predict spikes, use time-based instances.

29. What next?
● Autoscaling
○ Not supported by OpsWorks.
○ For now: predict spikes, use time-based instances.
● Backups
○ We still do these from our DCs.

30. What next?
● Autoscaling
○ Not supported by OpsWorks.
○ For now: predict spikes, use time-based instances.
● Backups
○ We still do these from our DCs.
● Chef improvements
○ The recipes do too much at the moment.

31. What next?
● Improve Chef recipes
○ The recipes do too much at the moment.
○ Example: cluster management.

32. Questions?

33. Thank you!
Sam Starling
sam.starling@bbc.co.uk
Johan Hjerling
johan.hjerling@bbc.co.uk