Identify technical themes and observations from ongoing research Identify stakeholders covering themes Arrange interviews Definition of scorecard criteria to assess potential solutions Iteration and refinement Ideation Identification of candidate solutions Additional ad hoc interviews Feedback workshop Report production Presentation

1. NAO Website
Technical Discovery
Prepared by Stu Mackellar
May 2021

2. Technical Discovery Process
1. Identify technical themes and observations from ongoing research
2. Identify stakeholders covering themes
3. Arrange interviews
4. Definition of scorecard criteria to assess potential solutions
5. Iteration and refinement
a. Ideation
b. Identification of candidate solutions
c. Additional ad hoc interviews
6. Feedback workshop
7. Report production
8. Presentation

3. Emerging Technical Themes (from user research)
● Metadata & Content Discovery
How can content be more easily navigated? Improvements to search and surfacing of related
content. Improvements to metadata and introduction of a structured taxonomy.
● Content Presentation
Transition away from PDF alone toward HTML first or a dual format approach.
● Metrics & Measurement
How effective is the website in exposing and delivering content?
How is the content consumed?
● Accessibility & Responsiveness
How can content - particularly visual assets like infographics - be better presented?

4. Research Topics

5. How do users want to consume content?
Who creates content and what processes
surround this?
What accessibility requirements are there?
Are there any regulatory or legal requirements?
Accessibility
What analytics data is currently available?
What is missing?
Are there any privacy implications?
Analytics
Who is responsible for security within NAO?
Are there any regulatory or legal requirements
applicable to the website’s security?
Security
Content
How could search be improved to better
expose content?
How could task or user oriented journeys be
used to enhance navigation and
discoverability?
User Experience
What skills are available in house?
What is the appetite for building and managing
software vs buying?
What’s the NAO technical strategy?
Technical

6. Constraints

7. Overarching Constraints
● Functional
○ The site must be accessible and available reliably across all device types
○ PDF reports must be available for the foreseeable future
○ PDF as a format does not meet all identified user needs
● Financial
○ The budget for the site is relatively small
● Technical
○ There are no in-house full-stack development skills
■ (although basic WordPress and HTML/CSS skills are available)
○ There is no appetite or budget to increase the website team size
● Operational
○ The website must meet overarching NAO security requirements
○ SaaS hosting is the preferred strategic option

8. Findings

9. Platform Insight
The NAO website was originally introduce in 1997 by the then librarian and was hand-coded in HTML. It has since
been through various revisions.
The site was migrated to a content management system provided by Alterian prior to 2010. It was then migrated
again to WordPress around 2013 by Code For The People. A custom theme and plugin were developed at this
stage.
In 2016 the Sage starter theme was introduced, which is the current basis of the site today. The NAO plugin has
been refined and continually developed and provides a variety of functionality, including custom post types for
reports, widgets for displaying content and workflow elements.
WordPress is a popular and well understood content management system (CMS) and is widely recognised as a
market leader. There has been significant investment in the current implementation and its custom functionality,
such as themes and plugins, and also integration with NAO workflows and processes.

10. Platform Recommendations
Goal: Provide a well understood, secure, affordable and functional platform on which NAO content can be
effectively distributed to its users
Recommendations:
● Continue with the use of WordPress as a platform
● Review theme and plugin usage for fitness for purpose and commission updates or replacements where
necessary
Pros Cons
● Well understood within NAO and externally
● Avoids assessment and migration costs of replatforming
● Highly performant and extensible
● Plentiful off-the-shelf plugin availability
● Relatively inexpensive
● May not cover outlying use cases, although NAO’s usage
patterns are fairly mainstream
● Bespoke functionality requires specialist development
expertise

11. Hosting Insight
The NAO website is currently hosted by CIVIC, who have also provided various software consultancy services
over the past few years.
CIVIC provide a hosted WordPress service, including application of critical security patches and some updates. It
is effectively a PaaS (platform as a service) solution. The NAO IT strategy is to move away from on-premise
hosting within the next 12-18 months, with SaaS the strongly preferred option, or cloud hosting in Azure
otherwise.
Configuration management is seen as a weakness of the current arrangement, particularly from a security
perspective. Access control and limited configuration of identity management functionality is concerning. A move
to a fully-managed SaaS (software as a service) solution employing federated identity management would be
preferred.
The website is currently configured in multi-site mode. This is considered unnecessary and consolidation of the
existing sub-sites with the primary site would be simpler with no loss of function.

12. Hosting Recommendations
Goal: Ensure the NAO website is highly available, cost-effective and manageable by NAO in-house staff
Recommendations:
● Explore moving to a WordPress SaaS (Software as a Service) provider
● Consolidate the current multi-site setup into a single-site configuration
Pros Cons
● SaaS vastly reduces the administrative overheads and
security risks around running a WordPress installation
● SaaS solutions are optimised for efficiency and hence low
cost
● Moving to a single-site configuration reduces complexity
● Changes to hosting can be disruptive
● SaaS is one size fits all so any customisation outside of
basic configuration and plugin usage will generally not be
possible (this is unlikely to be a huge concern given
NAO’s relatively typical usage patterns)

13. Content Insight
Website content is predominantly created via a fairly complex workflow which is centred around Adobe’s InDesign
software. Any changes to how content is produced or to generated formats will consequently require changes to
this process. Numerous opportunities for improvement of the workflow have been identified over recent years and
lack of capacity has generally prevented corresponding action. For example, manually uploading content is
inefficient and automation of this process would save time and be more repeatable.
Reports are the website’s primary content type and comprise of various components such as a synopsis, an
executive summary and the report itself. They are generated in several formats (PDF for web, PDF for print,
epub). None of these formats lend themselves particularly to interactivity, linking into the content or finding
specific segments of interest.
Various data visualisations are also published. These generally are published from one of two data processing
tools: Tableau and Shiny. The visualisations are embedded in the website as independent assets and hosted
elsewhere.

14. Content Recommendations
Goal: Ensure NAO website users are able to find and consume reports and related content easily
Recommendations:
● Prototype an HTML report variant (either full, partial or both) by modifying the InDesign workflow to
generate HTML from source
● Evaluate the Ajar Productions in5 plugin for HTML generation
● Prototype concept designs
● Explore automated ingestion of content into WordPress using plugins such as WP Media Folder
Pros Cons
● Generation from source will provide maximum flexibility
● Different layouts could be applied to each content type
● Automated ingestion will be faster and more repeatable
● Additional effort needed in understanding, setting up and
managing the in5 plugin functionality
● Creating new HTML report layouts will require changes to
workflows and ongoing effort to maintain

15. Search Insight
The website currently has a search function which is prominently displayed at the top of every page. It’s
implemented using the Swiftype WordPress plugin.
There has been strong and consistent feedback from both internal and external users that the search functionality
does not work as well as it could. Multiple users have reported resorting to generic Google web search as an
alternative entry point to NAO content.
Additional content navigation tooling exists internally for NAO users. The Back Catalogue Analyser, built by the
analytics team, is one of the most prominent of these. It has been suggested that, while not a panacea, this tool
could contribute toward provision of an improved externally-facing search function.
The use of enhanced search functionality, such as autocomplete and faceted filtering have been suggested as
another approach to improving search performance and these have been incorporated into the concepts
designed during this discovery work.

16. Search Recommendations
Goal: Provide an enhanced search experience across the NAO website
Recommendations:
● Evaluate the ElasticPress plugin for WordPress to provide autocomplete and faceted filtering
● Explore the plugin’s advanced configuration options to optimise of various search characteristics
Pros Cons
● Using a plugin is a quick and easy solution
● ElasticPress is free and open source
● It is based on ElasticSearch, a hugely popular and well
regarded indexing tool
● No guaranteed technical support as open source

17. Tagging Insight
Content is tagged within WordPress as part of the publication workflow. The taxonomy is hard-code within
WordPress and is thus maintained by the external comms team. Each content item can have multiple tags.
There also exists a separate taxonomy internally, which is used for a variety of purposes. There is a current
ongoing effort to improve this through consistency of language and simplification of the overarching structure.
This taxonomy is considered the master and the system of record is SharePoint.
The lack of consistent tagging of internal and external facing content has caused confusion and there’s a strong
desire to consolidate into a single source of truth. Different terminology is used for the same concepts publicly and
internally in some cases and this has also resulted in confusion on occasion.

18. Tagging Recommendations
Goal: Make it easier for users to find the NAO content they’re looking for and other related content
Recommendations:
● Use a single, unified taxonomy and consistent terminology to annotate content with metadata
● Ensure the taxonomy’s design is flexible and enables support for hierarchical layers of tags
● Explore the viability of synchronising SharePoint master taxonomy data to WordPress
● Explore automated tagging of content through analysis of text and extraction of references to other NAO
reports using tools such as the Back Catalogue Analyser
Pros Cons
● A single taxonomy will ensure consistency and remove
ambiguity
● Synchronisation of internal and external taxonomies will
reduce administrative effort
● Automated keyword analysis will enable quicker and
more complete tagging
● Getting master taxonomy data into WordPress may
present some technical challenges
● Automated tagging may be less effective than hand-
curated tags

19. Data Insight
Many reports are based on or contain statistical analysis of underlying data sets, which are sourced from one or
more government departments. There is often a degree of aggregation and processing needed to ensure related
data sets are consistent and standardised. This function is performed in-house by data and analytics teams.
Data is generally held in a data warehouse and processed using a variety of tools. Tableau and Shiny are the two
commercial products which are integrated with the website. There is an internally developed data service and an
associated REST API which are used as the basis for several bespoke tools.
There is an aspiration toward publication of raw data sets, which is something that has been previously
discouraged within NAO, for various reasons. Some small data sets are currently published alongside
visualisations in CSV format. Other options that have been previously considered include provision of a public
API, either by making the existing internal API publicly accessible or by creating a new one, and making CSV data
sets more widely available.

20. Data Recommendations
Goal: Expose the underlying data sets that are used within NAO reports and visualisations
Recommendations:
● Explore exposing the existing internal REST API publicly with additional access controls if necessary
● Alternatively or additionally, explore the feasibility of providing CSV exports of the raw data used to
generate reports and visualisations
Pros Cons
● Reusing an existing API saves time, effort and duplication
● CSV exports are simple to create and distribute and
widely supported by third party software
● Integration between internal and external systems may
present technical and security challenges
● The existing API may not meet all use cases
● No strategic decision on public data access has been
taken, so action taken now may be premature

21. Analytics Insight
Currently the website is integrated with Google Analytics. Data is made available in Google Data Studio, which is
performing well for existing use cases. A review of the Google Analytics implementation and overall analytics
pipeline is desired and will likely be outsourced to external experts. Metrics include page views and downloads.
The MonsterInsights analytics plugin is already enabled on the site and its efficacy is unclear.
There is a desire to enhance analytics capabilities with additional metrics, such as content engagement (progress,
view time) and capturing progress through specific user journeys. There is a tension between the need for content
engagement metrics and the use of PDFs as the primary consumption format, due to the offline nature of PDF
documents. Adobe have made some progress toward enhancing PDF usage tracking, which may warrant further
investigation.
Dashboards (primarily in Google Data Studio) are the primary consumption method for analytic data. Export to
CSV for ad hoc analysis would be of additional benefit. Data Studio may support this to some extent although no
research has yet been done to assess its capabilities in this area.

22. Analytics Recommendations
Goal: Provide NAO with better visibility of how content is consumed
Recommendations:
● Review existing Google Analytics configuration and optimise to provide more granular tracking Assess
whether the MonsterInsights plugin is adding value
● Assess content groupings as a technique for capturing progress through specific user journeys
● Review Data Studio usage and explore whether dashboards can be enhanced to deliver new metrics
Pros Cons
● Google Analytics is ubiquitous, well understood and free
● Consultancy and advice is plentiful and inexpensive
● Capturing user journeys using built in tools removes the
cost and complexity of introducing additional software
● The granularity of journey that can be expressed with
content groupings will not be as fine as when using
software dedicated to this purpose
● There are potential privacy concerns if user’s personal
data is sent to Google

23. Accessibility Insight
The website was the subject of a recent accessibility audit, for which it received a score of 7 out of 10.
The site scores reasonably well against a selection of high profile accessibility assessment tools. For example, it
has an 85% compliance score at www.webaccessibility.com.
Anecdotally, there are some known issues with the responsiveness of the site on some device types, particularly
smaller screens. Some of the embedded assets which use HTML iframes also have a higher degree of
accessibility issues.

24. Accessibility Recommendations
Goal: Ensure NAO content can be consumed effectively by users with additional needs and on all device types
Recommendations:
● Identify tactical targets for improvement based on the latest audit results and automated scans using
standard accessibility tools
● Optimise WordPress templates to increase responsiveness of iframe content to improve visualisations
Pros Cons
● The site is already reasonably conformant and targeting
tactical improvements removes the need for full rework
● WordPress has excellent accessibility support built in and
through plugins which can be leveraged to mitigate some
of the failings with embedded assets
● Tactical, layered fixes to embedded assets avoids the
need to look at replacing or reworking the entire asset
ecosystem
● Fixing accessibility issues in third party assets by
manipulating HTML is never going to give results as good
as fixing the issues at source

25. SEO Insight
Google tooling is also used to some extent already to measure search engine optimisation performance, although
this is obviously skewed toward a Google perspective. A WordPress plugin called Yoast is currently active on the
site and its efficacy is unclear. No specific SEO concerns have been raised during research, although confidence
in the status quo is uncertain.

26. SEO Recommendations
Goal: Ensure NAO content ranks high in search engine listings for relevant keywords and related searches
Recommendations:
● Optimise existing WordPress templates to ensure that tags and any additional metadata that are introduced
are exposed to search engines using standard protocols
● Review content production workflows to ensure content is optimised for SEO
● Review efficacy of the Yoast plugin
Pros Cons
● WordPress is heavily optimised for good search ranking
out of the box
● There are plenty of plugins available to fine tune results if
deemed necessary
● None

27. Security & Compliance Insight
The website must be compliant with various privacy legislation, including:
● GDPR/DPA
● PECR
Its cookie implementation is not currently compliant with PECR, although there are ongoing efforts to rectify this.
Given that the website does process some personally identifiable information (PII), a PII log should be kept and a
data privacy impact assessment should be undertaken. No evidence of these was found.

28. Security & Compliance Recommendations
Goal: Ensure the NAO website protects sensitive data, is highly resistant to common threat vectors and complies
with all applicable legislation
Recommendations:
● Explore implementation of identity federation to provide seamless, integrated access control
● Conduct a data protection impact assessment (DPIA) and create a personally identifiable information (PII)
log
● Continue ongoing efforts to implement compliant cookie handling
Pros Cons
● Integrated access control helps NAO staff manage
access centrally and removes the need for additional
credentials
● Conducting a DPIA will ensure NAO are able to assess
compliance with relevant legislation and identify any gaps
● None

29. Delivery Models

30. Software as a Service (SaaS)
Description: Procure a website platform as a commoditised service which is hosted and managed by the supplier
on the customer’s behalf
Pros Cons
● Relatively inexpensive and highly predictable cost
● Reduction in administrative overhead
● Hence less in-house technical support required
● Lack of flexibility and control
● Some potential for vendor lock-in
Supplier Responsibilities Customer Responsibilities
● Hosting, availability, security, performance
● Features and functionality
● Content management
● Configuration

31. Outsourced Development
Description: Engage a supplier to create and maintain a website solution independently based on NAO
requirements. Can be structured as an ongoing relationship with one supplier or as discrete units of work,
potentially across many suppliers.
Pros Cons
● Requires very little in-house technical knowledge
● Reduction in administrative overhead
● Relatively expensive
● Lack of technical skills inhibits ability to understand or
influence solution
● High risk of vendor lock-in
Supplier Responsibilities Customer Responsibilities
● Development
● Infrastructure management
● Content management
● Requirement definition
● Acceptance testing

32. Augmented Development
Description: Work collaboratively with a supplier to create a website solution based on NAO requirements.
Generally requires building a good working relationship with a single supplier.
Pros Cons
● Ability to use supplier only as much as needed,
minimising spend
● Ability to leverage a wider set of skills
● Opportunity to transfer skills and knowledge in-house
● Requires a larger (i.e. > 1) internal team to do this well
● Moderately expensive
● Health of supplier relationship is critical to success
Supplier Responsibilities Customer Responsibilities
● Development
● Shared infrastructure management
● Content management
● Requirement definition
● Development
● Shared infrastructure management

33. Appendix
Alternative options

34. Platform
Goal: Provide a well understood, secure, affordable and functional platform on which NAO content can be
effectively distributed to its users
Other options Limitations
Consider another headful CMS platform WordPress is well regarded generally and well understood by
NAO so there is no compelling reason to change
Commission a bespoke solution from external suppliers Expensive, lack of control and potential vendor lock-in
Recruit an internal team to build a bespoke solution Expensive and goes against stated NAO strategy

35. Hosting
Goal: Ensure the NAO website is highly available, cost-effective and manageable by NAO in-house staff
Other options Limitations
Explore hosting a WordPress instance in Azure Additional management overhead and maintenance. Non-
strategic.

36. Content
Goal: Ensure NAO website users are able to find and consume reports and related content easily
Other options Limitations
Continue to explore PDF to HTML conversion techniques Exporting from source data will always give better results than
converting an intermediate format
Create independent production workflows for PDF and HTML Doubles the effort involved in producing content

37. Search
Goal: Provide an enhanced search experience across the NAO website
Other options Limitations
Attempt to improve performance of the Swiftype plugin The Swiftype plugin is not as fully featured
Assess other WP search plugins The ElasticPress plugin is considered best in class
Build a custom plugin Expensive, requires bespoke external development

38. Tagging
Goal: Make it easier for users to find the NAO content they’re looking for and other, related content
Other options Limitations
Keep internal and external taxonomies synchronised manually Additional manual effort and potential for synchronisation to fail,
resulting in inconsistencies
Evaluate dynamically generated relations (e.g. from ElasticPress
plugin)
Dynamic relations will not conform to a predefined taxonomy,
although may still give adequate results

39. Data
Goal: Expose the underlying data sets that are used within NAO reports and visualisations
Other options Limitations
Build a new REST API purely for public consumption Additional cost to build and maintain, although it would potentially
be easier to secure

40. Analytics
Goal: Provide NAO with better visibility of how content is consumed
Other options Limitations
Explore other WordPress plugins for advanced Google Analytics
use cases (e.g. Journey Analytics)
Google Analytics built in functionality is powerful, so be aware of
increasing costs and diminishing returns

41. Accessibility
Goal: Ensure NAO content can be consumed effectively by users with additional needs and on all device types
Other options Limitations
Explore WordPress accessibility plugins to target specific issues Use of multiple plugins increases management overhead and the
likelihood of conflicting behaviour

42. SEO
Goal: Ensure NAO content ranks high in search engine listings for relevant keywords and related searches
Other options Limitations
Explore additional WordPress SEO plugins. Use of multiple plugins increases management overhead and the
likelihood of conflicting behaviour

43. Security & Compliance
Goal: Ensure the NAO website protects sensitive data, is highly resistant to common threat vectors and complies
with all applicable legislation
Other options Limitations
Retain and improve independent access control Harder to apply effective governance and requires users to
maintain additional accounts
Remove PII from the website Limits functionality of the site