Data Driven Government 1. May 18th 2018. Data Driven Innovation 2018. Engineering Department, University of Roma Tre

1. Data & Analytics Framework:
how public sector can profit
from its immense asset, data
RAFFAELE LILLO
Chief Data Officer
@ Digital Transformation Team
—

2. Data & Analytics Framework (DAF)
● Vision & Strategy
● Ok, but what is DAF?
● Challenges
● Our Strategy
● Some Architectural Highlights
● What are we doing right now? Use
cases
Q&A
—
DAF

3. Information is a fundamental asset to
interpret social and economic phenomena,
take informed decision, improve services
to citizens, compete in the international
arena.
New Technologies allow to extract
knowledge from the immense amount of
data owned by the State.
Vision

4. Extracting value from data needs a solid
technological platform, a team of experts,
and a proper governance to govern the
generation, integration, standardization
and use of data
Strategy

5. DAF: Data & Analytics
Framework

7. Data & Analytics Framework (DAF) is a combination of:
● A Big Data Platform to centralize and store (data lake), manipulate
and standardize (data engine), re-distribute (API & Data Applications)
data and insights.
● A Data Team (data scientists + data engineers) which uses and
evolve the Big Data Platform to analyze data, create ML models and
build data applications and data viz.
● Laws and regulations to make this activity possible
Give us data and a platform...

8. Interoperability (aka Get out of the Silos!)
Public data is… public and all PP.AA. should have access to it
Democratizing Data (aka Open Data, API & Data Viz)
Data should be open (when legally possible), accessible by anyone (and
anything) and insightful
Data Products (aka deliver value & insights)
Machine Learning in interconnected software applications
Crowdsourcing (aka data is everywhere, let’s help us out)
Citizens (esp. civic hackers) contribute to the surfacing of knowledge
… and we shall move the PA

9. Organizational and Managerial Challenge
Central Data Office and federated analytics teams
Human Resources
Data Scientists & Data Engineers to get knowledge from data
Technology
This is the least complicated one, but still fundamental.
Legislative Challenge
Balancing Privacy and Public Interest
Data Driven Policy needs… Data (and Data
Scientists)

10. Introduction of DAF in Piano Triennale 2017-2019
DAF is one of the building blocks of the official document setting the strategy for
digitalization of the PA, and signed by the Prime Minister
DAF prototype development
TD started the development of the platform from scratch around March ‘17, and
released an Alpha version the first week of October ‘17
Experimental phase
We started working with a selected number of PA to showcase DAF, test it and
listen to PA’s needs so to fine-tune the platform before final release
Institutionalization of DAF
Introduce by law the role of a central data office for the entire PA
Our Strategy

11. Mission: Data driven decision making in efficient ways
Support PA at all levels to implement informed policies, both ex ante (policy
formulation) and ex post (policy monitoring and fine tuning).
Centralize common & non-domain specific tasks
Provide general purpose data platform once and for all, efficiency in standard
data processes, let PA focus on domain specific tasks / analysis
Economy of scope towards a center of excellence
Reach proper dimension to develop and acquire expensive and idiosyncratic
capabilities, and share them with all PA
Design and coordinate implementation of Data Policies
Help interoperability and usage of state-of-the-art standards and processes in
data management and analysis. Stimulates research and collaboration.
End Goal: Chief Data Office for the PA

12. High-Level Architectural Design
Hadoop cluster for
distributed persistence and
processing
Kubernetes cluster
manages dockerized
microservices and external
applications.
Core Managers:
microservices managing
core functionalities of DAF
External applications
natively integrated in DAF
Unique identity
management system,
integrated with HDFS

13. Dataportal - Public Version
https://dataportal.daf.teamdigitale.it

14. Dataportal - Private Version
https://dataportal-private.daf.teamdigitale.it

15. Machine Learning Based Applications (aka Data Products)
Lex Datafication & Citizen Assistant, Fraud Detection, Citizen
Recommendation Engine, Spending Check, Leading Indicators, etc.
Data Visualization
Thematic dashboards and infographics for citizens and firms
API for Interoperability and Open Data
Easy and standard access to data within PP.AA. and citizens
And much more… The limit is imagination
Smart city, analysis for data driven policy making, etc.
What can be done? (examples)

16. Platform, Platform, Platform
Enhance Ux/UI and functionality of the dataportal; API & bulk download;
security and role management; ingestion & standardization procedures; scale the
cluster.
Data quality, standards & Open Data
Implement concept of “standard dataset”; fight the entropy of Open Data; Open
Data in SaaS to all PA; ontologies & controlled vocabularies in Big Data platform.
Data Hackathon
We are organize an hackathon to show case DAF and the value of open data
with civic hacker in solving business and social problems. Stay tuned!
What are we doing right now?

17. Multi-Event Hackathon for Data Science and Social Good
● Online Hackathon: July 8th to September 23rd
● Onsite Hackathon: October, 20th - 21st
Two types of challenge
● Data Science Challenge: machine learning model building to solve a real business
problem → prize: > 5000e
● Civic Challenge: challenge focused on social good and data economy topics
Hack.Data - Save the Date!
hack.data.italia.it

18. Relations with PAs & Use Cases
Onboarding partner PAs; revision of their open data policies and procedures;
data stories; data science prototypes
→ Use Case: Neighborhood Map
Data application that show on map services, facts and a synthetic index of life in
the neighborhood. Working with Turin, Milan, Rome.
→ Use Case: Analysis of public contracts & “enterprise suggestor”
Analysis on public contracts dataset managed by ANAC. This led, among other
things to a data application that suggest companies that are most compatible
with a given contract a PA may want to make. Our Initiative.
What are we doing right now?

19. → Use Case: Document Classificator
Data application based on a trained neural network for automatic
classification of document, normally done manually by “ufficio protocollo”.
Requested by Regione Toscana
→ Use Case: Social Media Monitor and sentiment analysis
Data Application to understand the feeling of people in specific
fields/topics of interest. Requested by Regione Toscana.
What are we doing right now?

20. Raffaele Lillo
Chief Data Officer
raffaele@teamdigitale.governo.it
Twitter, Medium: @lilloraffa
—
Grazie!
Cooperate with us, please :)
Website
http://teamdigitale.governo.it
Forum
https://forum.italia.it/c/daf
Twitter
#DatiPubblici #DAF