Recommended
More Related Content
What's hot
What's hot (20)
Similar to Big Data Means for Aberdeenshire Council
Similar to Big Data Means for Aberdeenshire Council (20)
More from Joe Chapman AMIRMS
More from Joe Chapman AMIRMS (8)
Recently uploaded
Recently uploaded (20)
Big Data Means for Aberdeenshire Council
- 1. Too Much Information? What Big Data means for Aberdeenshire Council Business Transformation Week, Sep 2014
- 2. What makes data ‘Big’? Volume – amount of data generated Velocity – speed of data flow, in and out Variety – range of data types and sources - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Veracity – accuracy and effectiveness
- 3. Up to the end of 2003, the world had accumulated 5 Exabytes (5 billion GB) = 50,000 years worth of DVD-quality video 2011: this amount created every two days 2013: this amount created every 10 minutes Source: University of California Berkeley How big is Big Data?
- 4. If the data world was a shed…
- 5. What can Big Data be used for? Data mining enables organisations to better understand and anticipate customer needs. Examples: •Obama – 2012 election campaign •Amazon – “Recommendations For You” •Walmart – surprising market research…
- 6. How to prepare for a hurricane!
- 7. Customer data at the Council • We use around 180 business systems • Many of these contain customer data • Duplication; inconsistency; inefficiency • Don’t forget shared drives & emails too!
- 10. How can the Council exploit Big Data? • Strategic planning and development • Policy-making and resource allocation • Learning from feedback via social media • Anticipating customer requirements
- 11. What would we be required to do? Firstly, we must establish what data we have and where it is… and ensure it is well structured in order for it to have value… …then we need the tools and know- how to identify how it can help us Then take action using data INPUT PROCESS OUTPUT
- 12. Progress and prospects • Customer Data Integration and the ‘single view of the customer’ • Benefits identified; options appraised; data protection issues assessed • Complex and costly process required
- 13. What are the implications? Privacy concerns Resource requirements Intangible outcomes Political decisions
- 14. Discussion points… • We already hold the data – shouldn’t we use it better? • What would the public response be – welcoming or sceptical? • Are the long-term benefits worth the short-term costs?
Editor's Notes
- Good morning everyone, I’m Joe Chapman – a Project Assistant for the Infosmart information management programme. Today I’m going to explore an issue that’s very much ‘in vogue’ in terms of business – especially digital – transformation, and look at how we as a Council might apply it to improve the service we deliver to our customers. I’ve used the tagline ‘Too Much Information’ and what I mean by that is, asking ourselves whether we actually gather and store customer data in too many different ways and in too many separate areas. So what Big Data might mean for the Council is actually about enabling us to use the information we already have more intelligently and productively.
- First, though, we need to understand the meaning of the term ‘Big Data’. It’s commonly defined in terms of these 3 dimensions – volume, velocity and variety. In other words, an overwhelming array of data, which is generated rapidly and which comes from multiple sources; all of which requires innovative processing and analysis in order to make sense of it and derive business value from it. A fourth ‘V’ is commonly added to this framework, to acknowledge the importance of establishing the reliability of the data – essentially sorting the wheat from the chaff. So in terms of the growth of data over the past decade or two, we’ve gone from talking about Megabytes to Gigabytes to Terabytes and even Petabytes; whereas in the 90s we had batch or periodic processing, now everything is in real-time; and of course we’ve seen the format of data evolve from very primitive databases, to audio/video and now to the mobile, social age.
- To put this in context, I found these figures which show the rate at which data creation is accelerating, largely as a consequence of that increasing complexity and range of formats. In fact, nowadays we’re often generating data without even realising it – what organisations do with things like mobile phone location data and communications records is a controversial issue these days. Then there’s the open-source (like social media) and closed-source (i.e. customer records) data which is more consciously created and better understood, but which is often no less difficult to trace and control – with or without the Data Protection Act, which was passed in a very different era. If you post something on social media, who does that data belong to, and who is responsible for where it ends up?
- Bearing all that in mind, with over half a billion tweets posted on Twitter every day (a handful of them by Aberdeenshire Council!), it’s no wonder that if the data world were indeed a shed, it would be looking a little bloated. But it’s also no wonder that a global army of handymen (otherwise known as data scientists) has appeared who claim to do what many homeowners fail to, and actually make use of half of what is in their shed!
- Now this is by no means a new concept. President Obama’s 2008 and 2012 election victories owed much to a near-military operation that wove pieces of publicly-available information about the electorate into a detailed profile of almost every voter, which enabled careful and productive targeting of grassroots resources by knowing exactly where to deploy the ‘boots on the ground’ and what to say when they got there. But this political giant was merely following in the footsteps of commercial ones. We’re all familiar with those supermarket vouchers that try to entice us with money off products that we are (or might be) interested in, or those Amazon emails that tell you what ‘people like you’ are buying. The more we buy from these retailers, the more information we’re giving them, and so the more tailored (and more accurate) these recommendations become. There are some surprising results too. In 2004, Wal-Mart wanted to find out which products were in demand from customers in Florida as they prepared for the arrival of Hurricane Frances. So they looked at sales data from a similar area in the period before an earlier hurricane – Charley – had struck. And what did they discover they needed to boost their stocks of – sandbags, bottles of water, wind-up torches, portable stoves? No…
- They found that sales of Pop Tarts had risen 7-fold in advance of the hurricane. They didn’t know why… but they didn’t need to know why. They just knew that they needed more Pop Tarts – a lot more. And while they were at it they even made sure those Pop Tarts were moved closer to the checkouts! Now, Wal-Mart does data on an epic scale – via its 3600 stores and 100 million customers, it amasses more data about the products it sells and its shoppers' buying habits than anyone else; so much so that it can claim to know some intimate details about a broad slice of the American population.
- Could we say the same about the people of Aberdeenshire? We know where they live, we receive their council tax, we educate their children, we collect their rubbish, fix their roads, clean their streets, maintain their parks, provide their social care… for many, we even built and own their homes (apologies if I missed anything out!). Yet we can’t really say the Council truly knows its customers in the way that all of that would suggest it should. So why is that? It’s largely because, in terms of our information, we operate not as one single Council but as a collection of semi-independent agencies. For example, separate departments – even separate area teams within departments – have separate network drives, thus holding basically the same information, doing the same work and making the same mistakes. That’s not their fault; it’s not anyone’s fault – it’s just the way that systems and processes have emerged and evolved over time, with little overall control or foresight. I, along with another (former) member of the Infosmart team, recently undertook an extensive service liaison exercise, designed to take a holistic view of how information is managed across the Council. In doing so we found not only many things that could be improved, but also examples of good practice that weren’t being shared. But above all we were able to identify recurring themes which shared the same root causes, as well as where there were opportunities for a commonality of approach. That last point is something we came up against time and again – it’s a bane of records and information management that if an individual or team hoarded too many paper files, before long they’d know about it as shelf, desk and even floor space started to disappear… yet many of us fill up inboxes and shared drives without a second thought. Remember that ‘data world’ shed I showed you – that could just as easily be our network storage, too. Latest estimates put our total network data storage somewhere between 30 and 40 Terabytes – that’s 30 to 40 thousand Gigabytes. And just as sifting through piles of paper would slow you down, the more data we store in the shared drives, the slower they operate… and the longer it takes us to find what we’re looking for.
- So what does this mean for our customers? It means for every service they use, there might be a different account number or password, held by different teams using different systems. If you have a good memory – or a secure stash of post-it notes – this might be OK… until you move home or get married, in which case, for every one of those accounts, you need to phone up or visit an office to change your details (often following different processes and requiring different documentation). Although at the front-end, the Contact Centre and the roll-out of Service Points is improving things, that process won’t be fulfilled and the customer experience will not be completely seamless until we change the way we manage information at the ‘back office’ side. In that situation, the more services you use, the more hassle you have to go through – penalising the most loyal and reliable customers. And many must wonder: ‘all these people on the phone work for the same organisation, so why can’t they just speak to each other?’
- And that’s a key point – in this day and age, our customers expect us to be smarter in the way we operate, including sharing their information internally when doing so allows us to deliver services in a more joined-up way, and to reduce inconvenience for them. That’s what they’re used to – increasingly, they feel that it’s part of what they pay their money for.
- Using big data allows us to extend that so we can even look at delivering personalised services to increase customer satisfaction, but above all it can enable us to anticipate customers’ requirements. For example we could combine the closed-source data we already have about where people live and what schools their children go to, with open-source information about where people work and how they travel and when. This ‘intelligence’ could inform anything from when the Contact Centre is open and how we staff it and when, to where and when we provide transport services such as subsidised buses and car share schemes, as well as allowing us to support community needs such as out-of-school clubs. It could change how we plan new roads and maintain existing ones, reform our waste and recycling services, and even help us with school re-zoning. In summary, across all Services and all areas, it’s about putting the resources in place where and when people need them – possibly even before they know they need them. We could also tap into the social media buzz to identify areas for improvement, and also acknowledge times when we’ve excelled. That’s because there’s a whole new generation of customers who often bypass the official channels and prefer us to come to them in order to communicate – and of course the best publicity of all is word of mouth, and technology provides an opportunity to be a fly on the wall in those informal but increasingly open conversations. If we get to know our citizens better, we can deliver a better customer experience.
- So how do we do that? The critical first step is to discover not only what information we hold, but ways in which data in different systems may be inter-related, and how we might link it all together. Do we want to select a single existing system and merge all information into there, or use a ‘spider’ structure whereby one central dashboard pulls data from all the other relevant sources? Before we can put anything into practice, the data must be accurate in its content and standardised in its format – this includes eradicating duplicates, but also requires us to identify which source holds the ‘master data’; in other words, if different information is stored about the same person, which version is the most up-to-date and reliable? The next step is possibly the most underestimated challenge of this sort of process; you may have lots of data, and the systems used to process it may be big, but it’s a futile exercise without the human capital – the skills required to turn big data into valuable information. This would require input from a range of people with a variety of perspectives – it cannot be driven solely from the centre. Finally, of course, you have to take what you’ve learned and apply it in the day-to-day work. This stage – like all the others – is not a quick or an easy process, and the gains will not be dramatic. Big data fulfils its potential by allowing organisations to spot incremental changes – marginal gains that, when multiplied together over time, deliver meaningful improvement.
- That seems like a big undertaking – quite a daunting one, in fact. But here at Aberdeenshire Council, we have made a start. Around the turn of this year, two of my former Infosmart colleagues carried out some initial scoping work for so-called Customer Data Integration – also known as the ‘single view of the customer’. Liaising with the iCE project team, other ICT staff and the Data Protection Officer, as well as the Improvement Service, NHS and other local authorities in Scotland, they looked into how we would go about uniting all of that disparate customer information that we hold. It’s a complex and costly process, requiring accurate data and essential skills as much as it needs sophisticated technology. It involves using the national Unique Citizen Reference Number, firstly to identify unique individuals and subsequently as the basis for a single set of information about each customer that is held in one place, updated once for each required change, and accessed simultaneous by the relevant business systems for each service we provide.
- That raises a number of issues. Inevitably, many people may be uneasy at the amount of information about them that is being held, and the wide range of purposes for which it is being used. On the flipside (as that screaming baby told us!), more often customers are frustrated when we don’t share information we already hold in ways that save time and hassle both for us and for them. Secondly, it requires significant investment in physical and human capital, especially up-front but also on an ongoing basis, and a commitment to sustaining this in order to realise the benefits. That brings me on to the third point – because those benefits are largely unseen to the average taxpayer (at least at first), justifying that expense in the current climate is likely to be extremely difficult. Consequently, the nature of this organisation is such that the ultimate decision would be a political one, influenced by privacy and cost implications. On the other hand, in the long term, the reputation of a Council that’s at the forefront of this digital information age could also be a significant incentive – rather than one that says we hold Too Much Information.