The document discusses GOV.UK's content publishing pipeline and how it is tested. It includes schemas that define content formats and required fields. The publishing app validates content against the schemas before sending to the content store. The frontend then transforms the content for display, expanding links. Both the publisher and frontend are tested by validating examples against the schemas to ensure compatibility as the schemas evolve. This provides confidence in the system and allows it to move faster.

1. GOV.UK Tech Monthly - Publishing 2.0 team
29th May 2015
David Heath
Developer
@dgheath21

2. GOV.UK
Content
Schemas
GOV.UKDavid Heath

3. @bradwright - GOV.UKGOV.UK
What’s it for?
David Heath

4. @bradwright - GOV.UKGOV.UK
What’s it for?
- confidence
David Heath

5. @bradwright - GOV.UKGOV.UK
What’s it for?
- confidence
- go faster
David Heath

6. @bradwright - GOV.UKGOV.UK
Publishing pipeline
overview
David Heath

7. @bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Content
store
Frontend
JSON JSON
{
“base_path”: ...,
…
}
{
“base_path”: ...,
…
}

8. @bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Content
store
Frontend
JSON JSON
{
“base_path”: ...,
…
}
{
“base_path”: ...,
…
}Transformation

9. @bradwright - GOV.UKGOV.UKDavid Heath
Metadata
Details
Links
Content item
{
base_path:
format:
public_updated_at:
details: {
...
},
links: {
…
}
}

10. @bradwright - GOV.UKGOV.UKDavid Heath
Publisher
"required": [
"publishing_app",
"rendering_app",
"update_type",
"format",
"locale",
"public_updated_at"
],
Frontend
"required": [
"base_path",
"format",
"locale",
"public_updated_at"
],
Metadata changes

11. @bradwright - GOV.UKGOV.UKDavid Heath
Publisher Frontend
content_id currently optional (maybe
required in future)
Absent
routes Routes (must be within
base_path)
Absent
redirects Only for format==’redirect’ Absent

12. @bradwright - GOV.UKGOV.UKDavid Heath
Links expansion (publisher)
“links”: {
“lead_organisations”: [
“ab517cc2-80df-48f2-8325-1b8af95b8f71”
]
}

13. Links expansion (frontend)
“links”: {
“lead_organisations”: [
{
"title": "Department for International Development",
"base_path": "/government/organisations/dfid",
"api_url": "https://www.gov.uk/api/organisations/dfid",
"web_url": "https://www.gov.uk/government/organisations/dfid",
"locale": "en"
}
]
}
@bradwright - GOV.UKGOV.UKDavid Heath

14. Details hash
stays the same :-)
@bradwright - GOV.UKGOV.UKDavid Heath

15. Content item transformation
{
base_path:
format:
public_updated_at:
details: {
...
},
links: {
…
}
}
@bradwright - GOV.UKGOV.UKDavid Heath
Metadata (fields change)
Details (stays same)
Links (expanded)

16. @bradwright - GOV.UKGOV.UK
How can we test this?
David Heath

17. @bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Content
store
Frontend

18. @bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Content
store

19. @bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Test that this
produces the
right output
Test that this works
with a variety of valid
inputs

20. @bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Schema

21. Examples
@bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Schema

22. Examples
@bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Schema
Match?

23. Examples
@bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Schema
Match?
GOV.UK Content Schemas

24. @bradwright - GOV.UKGOV.UKDavid Heath
govuk-content-schemas

25. @bradwright - GOV.UKGOV.UKDavid Heath

26. @bradwright - GOV.UKGOV.UKDavid Heath
details.json
metadata.json
links.json
publisher/
schema.json
frontend/
schema.json
curated
examples
curated
examples
combine_publisher_schema
generate_frontend_schema
validate;
ensure_example_base_paths_unique
validated
examples
Build process (Makefile)

27. Examples
@bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Schema
Match?
GOV.UK Content Schemas

28. @bradwright - GOV.UKGOV.UKDavid Heath
Publisher test
jenkins-schema.sh
# Clone govuk-content-schemas dependency for contract tests
rm -rf tmp/govuk-content-schemas
git clone git@github.com:alphagov/govuk-content-schemas.git
tmp/govuk-content-schemas
cd tmp/govuk-content-schemas
git checkout $SCHEMA_GIT_COMMIT
cd ../..
# Run tests
time bundle install --path "${HOME}/bundles/${JOB_NAME}" --deployment
RAILS_ENV=test GOVUK_CONTENT_SCHEMAS_PATH=tmp/govuk-content-schemas time bundle
exec rake test:publishing_schemas --trace

29. @bradwright - GOV.UKGOV.UKDavid Heath
Publisher test
test_publishing_schemas.rake
namespace :test do
Rake::TestTask.new(:publishing_schemas => "test:prepare") do |t|
t.libs << 'test'
t.test_files = FileList['test/unit/presenters/publishing_api_presenters/*_test.rb']
end
Rake::Task['test:publishing_schemas'].comment =
"Test publishing API presenters against external schemas"
end

30. @bradwright - GOV.UKGOV.UKDavid Heath
Publisher test
test/unit/presenters/publishing_api_presenters/case_study_test.rb
test "links hash includes world locations" do
location = create(:world_location)
case_study = create(:published_case_study,
world_locations: [location])
presented_hash = present(case_study)
assert_valid_against_schema(presented_hash, 'case_study')
assert_equal [location.content_id], presented_hash[:links][:world_locations]
end

31. @bradwright - GOV.UKGOV.UKDavid Heath
Publisher test
- use govuk-content-schema-test-helpers
- define jenkins-schema.sh
- contract tests should ideally be stand-
alone (separate test file)

32. Examples
@bradwright - GOV.UKGOV.UKDavid Heath
Publishing
app
Frontend
Schema
Match?
GOV.UK Content Schemas

33. Frontend test
test/contracts/govuk_content_schemas_test.sh
class GovukContentSchemasTest < ActionDispatch::IntegrationTest
include GovukContentSchemaExamples
all_examples_for_supported_formats.each do |content_item|
test "can successfully render #{content_item['base_path']} schema example" do
content_store_has_item(content_item['base_path'], content_item)
get content_item['base_path'].sub(/^//, '')
assert_response :success
end
end
end
@bradwright - GOV.UKGOV.UKDavid Heath

34. Adding a field to a format
1. Branch govuk-content-schemas
2. Add optional field
3. Push branch, all contract tests run
4. Merge!
@bradwright - GOV.UKGOV.UKDavid Heath

35. @bradwright - GOV.UKGOV.UKDavid Heath

36. Adding a new format
1. Branch govuk-content-schemas
2. Add schema and examples
3. Make sure publisher and frontend
tests set up
4. Push branch
@bradwright - GOV.UKGOV.UKDavid Heath

37. Dummy content store
@bradwright - GOV.UKGOV.UKDavid Heath

38. @bradwright - GOV.UKGOV.UK
Summing up
David Heath

39. @bradwright - GOV.UK
Your
safety
net

40. @bradwright - GOV.UK
confidence
publisher
frontend

41. Links
● https://github.com/alphagov/govuk-content-schemas
● https://github.com/alphagov/govuk-content-schema-test-helpers
● https://github.com/alphagov/govuk-dummy_content_store
@bradwright - GOV.UKGOV.UKDavid Heath

42. GOV.UK Tech Monthly - Publishing 2.0 team
29th May 2015
David Heath
Developer
@dgheath21