ONS Local has been established by the Office for National Statistics (ONS) to promote evidence-based decision-making at the local level. We aim to host insightful workshops which will provide practical, technical support to help users make the most of ONS data. The Cross-Government Data Science Community brings together data scientists and analysts to build data science capability across the UK governments and public sector.
We are delighted to welcome you to our inaugural Workshop in our new series, entitled: 'How to use APIs'. The session will cover what Application Programming Interfaces (APIs) are, the advantages in using them and a practical demonstration of how they can be used. The journey of two Local Authority analysts as they begin using APIs in place of manual processes will be showcased to the audience. The session will conclude by explaining the plan for the forthcoming series of Workshops that will begin in September and introducing the Slack channel that ONS Local and Cross-Government DS community will be using to support users' technical questions going forward.
This event is open to all; however, we anticipate it will be of most interest to anyone working at a local level on creating data dashboards for internal or external use.
If you have any questions, please contact ons.local@ons.gov.uk.
call girls in Tilak Nagar DELHI 🔝 >༒9540349809 🔝 genuine Escort Service 🔝✔️✔️
ONS Local and Data Science Community Workshop 1: How to use APIs
1. Introduction to Application
Programming Interfaces,
and ONS’ APIs
Andrew Fergusson
Lead Data Scientist
ONS ⊃ Digital Publishing ⊃ Data Service
andrew.fergusson@ons.gov.uk
Teams, 6 July 2023
2. Agenda
• What are APIs?
• Advantages of APIs
• Practical demonstration of APIs
3. What is an API?
More importantly why should you care?
5. Data wrangling sucks
• Spreadsheets are presentational, not designed for
analysis
• Cross tabs typically only have two dimensions
• Got to re-interpret the column headers, unpivot the data
• Contains more data than you need
• Not value add!
6. Application Programming Interfaces
• A set of rules and protocols that allows software and data
sources to communicate
• Standards-backed, e.g. HTTP Methods (information
transfer), HTTP Status Codes (like 200 Okay 401
Unauthorized), JSON, XML
• Can be accessed using URLs which contain your
request details (a HTTP GET) or with a JSON file a
(HTTP PUT)
7. Some useful APIs (Not just for reading data!)
• The ONS Census and CMD API:
https://developer.ons.gov.uk/
• The NHS Fingertips API: https://fingertips.phe.org.uk/api
• HMRC’s suite of APIs:
https://developer.service.hmrc.gov.uk/api-
documentation/docs/api
• DWP’s suite of APIs:
https://www.api.gov.uk/dwp/#department-for-work-and-
pensions
8. APIs are a set of
procedures to access
data
There may be lots of new terms here, but consuming APIs is risk-
free. The worst outcome of a bad API call you’ll experience from
an ONS API is that it didn’t understand your query.
Photo by Kevin Fitzgerald on Unsplash
10. Why you should use APIs
• Formatted for analysis
• Filtered for your patch
• No transformation errors
• Automated updates
• Real-time data (depends!)
• Enriched data sources
• Increased efficiency
• Enhanced quality
• Collaboration and sharing
• Agility and adaptability
11. Using or not using an
API is your choice
Configuring a service to use an API may take a bit more
effort but you’ll get all of that time back and then some.
Photo by Bernd 📷 Dittrich on Unsplash
13. Notes about this demonstration
• This demonstration is using Visual Studio Code, a free Integrated
Developer Environment, and Python a programming language
• The examples are configurable, and the presentational document will be
available after this session
• This example is more advanced than the autumn sessions, but it is
necessary to demonstrate how the ONS Census API works
• PowerBI and Tableau can consume APIs directly, and interpret the API’s responses in
JSON for you
• Programming in Python will not be required
• Being familiar with how the API structures data helps you think about your requests
Editor's Notes
Formatted for analysis: no more wrangling and copy/pasting from excel spreadsheets
Filtered for your patch: no more downloading all of England’s data to report on Salford
No transformation errors: the API returns data in the format you request, meaning you won’t make a mistake wrangling and get incorrect analysis
Automated updates: when new data is released, the connections still work so you just need to refresh the API; if you are working with time series this is especially good
Real-time data: It may be true that real-time data is available from an API most analysis ONS Local performs isn’t real time (i.e. live), it’s generally a programmatic view of a static set of observations which get updated monthly, etc. An example of a realtime dataset is Manchester’s Oxford Road Cycle Counter which is updated each time a cyclist passes a set point
Enriched data sources: APIs can return detailed information and notes at almost any level: so instead of a warning on the note section about a set of observations, it’s specific to the exact scope but doesn’t ruin your spreadsheet with symbols or footnotes in an observation cell with the value you want. You can’t sum up observations when the values 3, 4, 5, and x.
Increased efficiency: set it and forget it. APIs are versioned, which means when you program against a version of an API, it will continue to work even if the data updates; no more broken spreadsheets. API developers will release a whole new API version (i.e. v2 after v1) if they’re going to introduce any changes which will break v1 functionality
Enhanced quality: The Census API has perturbation, and there are other advantages
Collaboration and sharing: Imagine being able to provide access to all of the UK’s census information but just with a link, the power you get with an internet connection and not having to download the large amounts of data locally – not that you would be allowed to access the microdata – means it’s easier to collaborate
Agility and adaptability: As these datasets are filterable fully custom, ad hoc analysis is just selecting the new analysis. Is the report originally ethnicity vs age for a local authority’s wards, but someone wants to see highest level of education obtained vs age, it’s just changing one parameter!