The challenges of being an Agile|DevOps
enabled organization implementing GDPR.
Dev & GDPR
Andrea Tino (Software Development Engineer)
Our journey
The agenda for today
A very brief overview of the
most important points of GDPR.
An analysis of the challenges
that an organization faces when
trying to implement GDPR.
Pocus on Process and also
Engineering with highlights on
DevOps practices.
What?
Data is the fuel of modern
business but it belongs to users
and therefore it needs to be
protected.
Protecting user’s data is a long
term commitment and raises
several challenges both in the
process and in the software
architecture.
Why?
As part of the process and daily
worklife inside one organization
and its business.
Where?
GDPR is already in place but
companies are still struggling
to implement it in a
sustainable way which is in
harmony with the process in
the organization.
When?
The main personas involved in
this analysis are developers
inside a DevOps-enabled
organization.
Who?
GDPR can be explained from
many angles. The analysis we
will conduct is from the POV of
developers and managers from
inside an organization.
The aim is to talk about GDPR in
a very practical and
biz--grounded way.
How?
Warning
This is a technical talk
about GDPR from the pov
of Engineers
This is not a lecture
deeply focused on GDPR!
GDPR
The General Data Protection
Regulation became in force
on May 25 2018
Regulation EU 2016/679
supersedes 95/46/EC
Definitions
Wording and terms
Chapter 1, Article 4
provides the definitions we need...
personal data
non personal data
data
Chapter I - General provisions
Article 4: Definitions
1. ‘personal data’ means any
information relating to an identified
or identifiable natural person (‘data
subject’); an identifiable natural
person is one who can be identified,
directly or indirectly, in particular by
reference to an identifier such as a
name, an identification number,
location data, an online identifier or
to one or more factors specific to the
physical, physiological, genetic,
mental, economic, cultural or social
identity of that natural person;
Personal Data
The first name, middle name
and last name of a person are
all considered personal data.
Nicknames can be considered
too as they can be used to
identify a data subject.
Names Identification codes can be
considered personal data when
they can lead to the identity of
a data subject. If an ID isused to
identify a record related to a
data subject but that ID cannot
lead to its identity, then it is not
considered to be personal data.
IDs
store
delete
modify/transform
Chapter I - General provisions
Article 4: Definitions
2. ‘processing’ means any operation or
set of operations which is performed
on personal data or on sets of
personal data, whether or not by
automated means, such as collection,
recording, organisation, structuring,
storage, adaptation or alteration,
retrieval, consultation, use, disclosure
by transmission, dissemination or
otherwise making available,
alignment or combination,
restriction, erasure or destruction;
Processing of data
The simple act of storing
(saving) data, in general, is
considered processing, even
though, computationally
speaking, the data itself is not
touched in any other way.
Storage Moving, transmitting,
disseminating personal data is
also considered a form of
processing of data.
Move
Since the moment data comes into the database of a system to the moment
that data leaves the database, all the operations performed on that chunk of
imformation are considered processing.
Rule of
thumb
Removing data is also
considered processing.
Delete
reverse
pseudonymisation
pseudoed-data
additional data
Chapter I - General provisions
Article 4: Definitions
5. ‘pseudonymisation’ means the
processing of personal data in such a
manner that the personal data can no
longer be attributed to a specific data
subject without the use of additional
information, provided that such
additional information is kept
separately and is subject to technical
and organisational measures to
ensure that the personal data are not
attributed to an identified or
identifiable natural person;
Pseudonymisation
Personal data which has been
processed via
pseudonymisation remains
classified as “personal”
because it can still lead to the
identity of a person.
Still
personal
The system must ensure that
both pseudoed-data and the
additional data alone cannot
lead to an identity and that the
additional data is kept secure in
a different location. This is
achieved by specific algorithms
the software must implement.
Security
controller processor
data processing
purposes
means of data
processing
defines
defines
processes
personal data
according to
processes
personal data
by means of
data subject
consents to
consents to
Chapter I - General provisions
Article 4: Definitions
7. ‘controller’ means the natural or
legal person, public authority, agency
or other body which, alone or jointly
with others, determines the purposes
and means of the processing of
personal data; where the purposes
and means of such processing are
determined by Union or Member
State law, the controller or the
specific criteria for its nomination
may be provided for by Union or
Member State law;
8. ‘processor’ means a natural or legal
person, public authority, agency or
other body which processes personal
data on behalf of the controller;
11. ‘consent’ of the data subject means
any freely given, specific, informed
and unambiguous indication of the
data subject's wishes by which he or
she, by a statement or by a clear
affirmative action, signifies
agreement to the processing of
personal data relating to him or her;
Controller & Processor
The controller defines the main data processing documents required by the
Regulation (according to which the user must give consent to the processing of
his personal data) and the controller executes the processing.
Quick
look
Most relevant articles
The parts of the Regulation Engineers look at
The first articles have an Engineering “translation”
which can be used to identify the challenges arising
to achieve full compliance to the Regulation.
As Engineers, we want to know the
impact of GDPR on our
Organization and system.
Excerpts
from the
Regulation
Description
and
summary
Engineering
challenges
and analysis
“Let’s build a spaceship
GDPR compliant
system!!!”
Personal Data Processing
GDPR-covered
Chapter I - General provisions
Article 1: Subject-ma er and objectives
1. This Regulation lays down rules relating to
the protection of natural persons with
regard to the processing of personal data
and rules relating to the free movement of
personal data.
2. This Regulation protects fundamental
rights and freedoms of natural persons and
in particular their right to the protection of
personal data.
3. The free movement of personal data
within the Union shall be neither restricted
nor prohibited for reasons connected with
the protection of natural persons with
regard to the processing of personal data.
Introducing GDPR
The first chapter, comprehensive of the
most important articles in the
Regulation, explains what the
Regulation is about and, therefore,
what it covers in terms of subjects and
objectives.
Protection
The GDPR is about protecting personal
data of persons. It is not only about
that, it is also about the protection of
the movement of this information.
It is important to note how the
Regulation highlights that the data
being protected is of natural persons.
So this is not about data belonging to
items or animals, the GDPR is
person-centric.
If the data is about companies or other
organizations, and that data relates to
persons in the context of that company,
the Regulation still applies.
Is your organization impacted?
If the software you build is about
handling personal data of persons,
then you should expect to be impacted
by the Regulation.
This is also true if your software deals
with moving of such data.
What is GDPR about?
Chapter I - General provisions
Article 2: Material scope
1. This Regulation applies to the processing
of personal data wholly or partly by
automated means and to the processing
other than by automated means of personal
data which form part of a filing system or
are intended to form part of a filing system.
2. This Regulation does not apply to the
processing of personal data:
(a) in the course of an activity which falls
outside the scope of Union law;
(b) by the Member States when carrying out
activities which fall within the scope of
Chapter 2 of Title V of the TEU;
(c) by a natural person in the course of a
purely personal or household activity;
(d) by competent authorities for the
purposes of the prevention, investigation,
detection or prosecution of criminal
offences or the execution of criminal
penalties, including the safeguarding against
and the prevention of threats to public
security.
Processing of personal data
Article 4 provides all the needed
definitions of personal data and
related. Article 2, however, defines that
the Regulation covers the processing of
personal data.
The protection coverage is full as any
type of processing is included in the
Regulation: both by programs and
humans (manual processing). In other
words: the GDPR is
technology-agnostic.
Exceptions on coverage
The Regulation does not apply in
certain cases. The most prominent are:
Other persons processing
personal data of other persons in
the context of a family unit.
Government authorities are
excepted from the Regulation in
the context of specific tasks.
Is your software impacted?
If your software needs to handle, in any
way, any personal data, then yes! The
Regulation must be considered by your
organization.
What is being protected?
Is your software excepted?
If you can guarantee that your software
is used in a family unit and the users
will only store personal data of other
family members, the exception is valid.
But these conditions are unfeasible in a
business and even if so, in future a
licensing expansion would be difficult
to implement if the Regulation was not
uptaken from the beginning.
If your software is only licensed to
governmental entities, the exception
could be applicable.
Chapter I - General provisions
Article 3: Territorial scope
1. This Regulation applies to the processing
of personal data in the context of the
activities of an establishment of a controller
or a processor in the Union, regardless of
whether the processing takes place in the
Union or not.
2. This Regulation applies to the processing
of personal data of data subjects who are in
the Union by a controller or processor not
established in the Union, where the
processing activities are related to:
(a) the offering of goods or services,
irrespective of whether a payment of the
data subject is required, to such data
subjects in the Union; or
(b) the monitoring of their behaviour as far
as their behaviour takes place within the
Union.
3. This Regulation applies to the processing
of personal data by a controller not
established in the Union, but in a place
where Member State law applies by virtue of
public international law.
Controller and processor’s locations
The coverage focuses on the controller
and processor’s locations. Regardless
of where the data subject is, if the
former are located in the Union, then
the Regulation applies.
Location of the data subject
If the data subject is in the Union but
the controller or processor are not,
then the Regulation applies under
some conditions on the processing
(purposes and means).
Identify the location
If your software handles personal data,
then it must be able to identify/track
the location of the data subjects those
information belong to. This association
is very important.
Where the protection?
Is your software impacted?
If your organization is in the Union,
then yes! Very simple.
If not in the Union, then it applies only
if the data subject is in the Union. For
licensing purposes, it is always better
not to make assumptions on the
location of customers/users to always
be able to scale your application.
GDPR applies
controller processor
data subject
NOT IN UNION
NOT IN UNION
IN UNION
GDPR does NOT apply
IN UNION
GDPR applies
Territoriality
User-centric
The diagram shows how the
GDPR is basically focused on
the user. If the data subject
is in the EU, then its
personal data is protected
by the Regulation!
Non applicable
The only case where GDPR
has no jurisdiction is when
all the parties are outside
the Union.
processor data subject
Union
data subjectcontroller
controller
Microsoft’s solution for small and big
businesses targeting ERP systems.
Companies can host their solution in
the Cloud on Azure and customers can
have access to their business data.
Case of study:
Microsoft Dynamics 365
The controller is the company
developing its solution on the
Microsoft platform, while the
platform itself is the processor.
Customers can be from all over the
world, the data subject can be part
or not of the Union.
Details
The Regulation applies when the data subject is in the Union or the
controller is, because the software is used to sell goods or services.
controller
processor
Union data subject
Shinsei Bank offers an online banking
system for account holders. The system
is offered only to residents in Japan to
track and manage their money, stock
and or savings.
Case of study:
新生銀行 (Shinsei
Bank) Banking System
The controller and the processor
are both located out of the Union.
Customers, data subjects, are
required to be residents in Japan,
therefore outside of the Union’s
jurisdiction.
Details
The Regulation does NOT apply because the controller and the processor
are both in Japan and they offer services only to Japanese residents.
Articles 1, 2, 3 and 4
Understand GDPR coverage
The first step towards GDPR
compliance is to assess whether the
GDPR applies to your organization,
and, if so, to what extent. This analysis
starts with understanding which data
you have and where it resides.
Inventory your data
It’s important to inventory your
organization’s data. This will help you
understand which data is personal,
and to identify the systems where that
data is collected and stored,
understand why it was collected, how
it is processed, and how long it is
retained.
1. Discover
On the journey to compliance
2. Manage
3. Protect
4. Report
Chapter II - Principles
Article 5: Principles relating to processing
of personal data
1. Personal data shall be:
(b) collected for specified, explicit and
legitimate purposes and not further
processed in a manner that is incompatible
with those purposes; further processing for
archiving purposes in the public interest,
scientific or historical research purposes or
statistical purposes shall, in accordance
with Article 89(1), not be considered to be
incompatible with the initial purposes
(‘purpose limitation’);
(c) adequate, relevant and limited to what is
necessary in relation to the purposes for
which they are processed (‘data
minimisation’);
(d) accurate and, where necessary, kept up
to date; every reasonable step must be taken
to ensure that personal data that are
inaccurate, having regard to the purposes
for which they are processed, are erased or
rectified without delay (‘accuracy’);
(f) processed in a manner that ensures
appropriate security of the personal data...
Limitations on processing
The processor cannot freely process
data however it desires. Conditions
apply.
In particular, processing is permitted as
long as the boundaries determined by
the purposes and means stated by the
controller are observed.
Exceptions
The processor can process data out of
the purposes stated by the controller as
long as this further processing is
limited to certain scopes. History
tracking and statistical analysis is
common practice and allowed by the
Regulation.
Security
The Regulation imposes certain
constraints on security. Personal data
must be kept safe and protected from
any event which may cause data-loss.
State purposes and means
Your software must be able to present
the user the list of purposes for which
his personal data is collected. A
description of the means used to
perform the processing must be
provided as well.
How can I process?
Track history and get statistics
If your software is in the Cloud, this is
especially valid. In a DevOps-enabled
organization, keeping track of the
history of operations on personal data
is important as well as extracting
relevant statistics that can help the
business grow.
Since the Regulation provides
clearance on this topic, it is always
possible to perform such activities.
Chapter II - Principles
Article 9: Processing of special categories
of personal data
1. Processing of personal data revealing
racial or ethnic origin, political opinions,
religious or philosophical beliefs, or trade
union membership, and the processing of
genetic data, biometric data for the purpose
of uniquely identifying a natural person,
data concerning health or data concerning a
natural person's sex life or sexual orientation
shall be prohibited.
2. Paragraph 1 shall not apply if one of the
following applies:
(a) the data subject has given explicit
consent to the processing of those personal
data for one or more specified purposes,
except where Union or Member State law
provide that the prohibition referred to in
paragraph 1 may not be lifted by the data
subject;
Not everything can be processed
Recalling that the Regulation by
“processing” also includes the simple
storing of data, processors cannot
process certain categories of
information.
As the recital explains, certain data
concerning data subjects are sensitive
to their rights and freedoms.
Personal data that can lead to a
person’s race, religion, sexual
orientation and other information
directly related to the foundamental
human rights cannot be processed.
Exceptions
There are many exceptions to
Paragraph 1, the most important and
common is the possibility to rely on the
data subject’s explicit consent.
Do not track sensitive information
Your software must not store or
process, in any way, certain categories
of data.
It must be guaranteed that every
form/UI-artifact collecting information
from users will not ask for any of those
sensitive details.
Can I process this?
Create
Read
Update
Delete
data subject
controller
grants rights
Chapter III - Rights of the data subject
Section 2 - Information and access to
personal data
Article 15: Right of access by the data
subject
1. The data subject shall have the right to
obtain from the controller confirmation as
to whether or not personal data concerning
him or her are being processed...
Section 3 - Rectification and erasure
Article 16: Right to rectification
The data subject shall have the right to
obtain from the controller without undue
delay the rectification of inaccurate
personal data concerning him or her...
Article 17: Right to erasure (‘right to be
forgo en’)
1. The data subject shall have the right to
obtain from the controller the erasure of
personal data concerning him or her
without undue delay and the controller shall
have the obligation to erase personal data
without undue delay where one of the
following grounds applies:
Full access to data
The data subject has the right to read,
write and even delete his personal
data.
CRUD capabilities
The system/software must be able to
locate personal data of users and let
them perform tasks related to CRUD.
Data subject & CRUD
Controller’s responsability
It is important to note how it is
responsability of the controller to make
sure the right to access, rectify and
erase is properly exercised by data
subjects.
Articles 5, 9 and 15
Data governance
It is a plan can help you define
policies, roles, and responsibilities for
the access, management, and use of
personal data, and can help you
ensure that your data handling
practices comply with the GDPR.
Data classification
Adopting a classification scheme that
applies throughout your organization
can be particularly helpful for
responding to data subject requests,
because it enables you to identify
more readily and process personal
data requests.
1. Discover
On the journey to compliance
2. Manage
3. Protect
4. Report
Chapter IV - Controller and processor
Section 1 - General obligations
Article 25: Data protection by design and
by default
1. Taking into account the state of the art,
the cost of implementation and the nature,
scope, context and purposes of processing
as well as the risks of varying likelihood and
severity for rights and freedoms of natural
persons posed by the processing, the
controller shall, both at the time of the
determination of the means for processing
and at the time of the processing itself,
implement appropriate technical and
organisational measures, such as
pseudonymisation, which are designed to
implement data-protection principles, such
as data minimisation, in an effective manner
and to integrate the necessary safeguards
into the processing in order to meet the
requirements of this Regulation and protect
the rights of data subjects.
The Regulation supports/advices the
use of Pseudonymisation to achieve
data minimisation.
Data separation
The system must be able to
pseudonymise personal data. There are
many techniques available, however
they all require architectures based on
strong and well scalable databases.
If your organization implements
DevOps, micro-services can help
achieve a good separation between
business logic and data inside systems.
In order to successfully implement
pseudonymisation, the separation of
additional data from replaced data is
possible in maintainable ways.
Pseudonymisation
helps achieving
Minimised data is: adequate,
relevant and limited to what is
necessary in relation to the purposes
for which they are processed.
Article 5, Paragraph 1, Point C
Data minimisation principle
The processing of personal data in
such a manner that the personal
data can no longer be attributed to a
specific data subject without the use
of additional information, provided
that such additional information is
kept separately and is subject to
technical and organisational
measures to ensure that the personal
data are not attributed to an
identified or identifiable natural
person.
Article 4, Paragraph 5
Pseudonymisation
Article 25
Organizations increasingly
understand the importance of
information security.
The GDPR raises the bar. It requires
that organizations take appropriate
technical and organizational
measures to protect personal data
from loss or unauthorized access or
disclosure.
1. Discover
On the journey to compliance
2. Manage
3. Protect
4. Report
Chapter II - Principles
Article 6: Lawfulness of processing
1. Processing shall be lawful only if and to
the extent that at least one of the following
applies:
(a) the data subject has given consent to the
processing of his or her personal data for
one or more specific purposes;
Article 7: Conditions for consent
1. Where processing is based on consent, the
controller shall be able to demonstrate that
the data subject has consented to
processing of his or her personal data.
3. The data subject shall have the right to
withdraw his or her consent at any time. The
withdrawal of consent shall not affect the
lawfulness of processing based on consent
before its withdrawal. Prior to giving
consent, the data subject shall be informed
thereof. It shall be as easy to withdraw as to
give consent.
Consent
For the processing to be lawful, at least
one of many conditions must be met.
The most common one concerns the
data subject to provide its consent to
the processing of its personal data.
Consent can be revoked by the data
subject at any time.
Asking for consent
Your software must be able to present
the user an interface which can be used
to ask for consent to the processing of
personal data. This UI must be clear
and the question must be easy to
understand.
Can I process?
Track consent
The system must be able to track
consent given by users so that it is
possible to exhibit proof later. if ever
required.
Update consent
Your software must be able to give the
user the ability to modify his
authorization to process personal data.
controller
processor
consents to
processing
data subject
1
processes data
provides
information
2
4
gets data
3
Chapter III - Rights of the data subject
Section 2 - Information and access to
personal data
Article 13: Information to be provided
where personal data are collected from the
data subject
1. Where personal data relating to a data
subject are collected from the data subject,
the controller shall, at the time when
personal data are obtained, provide the data
subject with all of the following information:
(a) the identity and the contact details of the
controller and, where applicable, of the
controller's representative;
(c) the purposes of the processing for which
the personal data are intended as well as the
legal basis for the processing;
2. In addition to the information referred to
in paragraph 1, the controller shall, at the
time when personal data are obtained,
provide the data subject with the following
further information necessary to ensure fair
and transparent processing:
(a) the period for which the personal data
will be stored, or if that is not possible, the
criteria used to determine that period;
The role of the controller
Article 13 is all about the
responsabilities of the controller.
Communicating with the data subject
The system must be able to
communicate with the data subject in
order to send him messages. Therefore
it is important to ensure that a reliable
channel can be enstablioshed. Using
e-mail verification techniques is highly
recommended to avoid fake accounts.
Automating the process
The Regulation requires information on
the controller to be sent when the latter
acquires the data. The system will
decide how to perform this operation.
Automation can be a desirab;e solution
to avoid manual work and human error.
Should I communicate?
Timing
It is important to note that the
informartion are required to be sent
when they are acquired by the
controller.
controller
processor
data subject
data
exchange
provides
information
2
4
third party
gets data
3
1
processes data
Chapter III - Rights of the data subject
Section 2 - Information and access to
personal data
Article 14: Information to be provided
where personal data have not been
obtained from the data subject
1. Where personal data have not been
obtained from the data subject, the
controller shall provide the data subject
with the following information:
(a) the identity and the contact details of the
controller and, where applicable, of the
controller's representative;
(c) the purposes of the processing for which
the personal data are intended as well as the
legal basis for the processing;
2. In addition to the information referred to
in paragraph 1, the controller shall provide
the data subject with the following
information necessary to ensure fair and
transparent processing in respect of the data
subject:
(a) the period for which the personal data
will be stored, or if that is not possible, the
criteria used to determine that period;
Same as for Article 13
Nothing changes when data is not
obtained directly from the data subject.
Communicating with the data subject
The system must retrieve the proper set
of information in order to be able to
reach for the data subject when data is
obtained from a different source.
Should I relay?
Same rights
The controller must still reach for the
data subject in order to notify him
about the processing.
Articles 6, 13 and 14
You have to be more transparent
about not only how you handle
personal data, but also how you
maintain documentation defining the
processes and use of personal data.
Auditing
Organizations that process personal
data need to keep records about the
purposes of processing; the categories
of personal data processed; the
identity of third parties with whom
data is shared; whether (and which)
third countries receive personal data,
and the data.
1. Discover
On the journey to compliance
2. Manage
3. Protect
4. Report
DevOps practices
More on the impact on DevOps practices
PLAN
DEVELOP
TEST
PACKAGE
RELEASE
DEPLOY
MONITOR
A huge part of DevOps is based
on collecting data from
customers, as they run the
product, in order to collect
more info about their
experience.
That information is going to be
used later for improving the
Telemetry product even further, detecting
issues before they surface or
detecting errors for quick
escalation/resolution. In order
to be compliant with the GDPR,
it’s important not to log certain
information.
DevOps in a nutshell
DevOps is a very modern trend in the Software
Industry and IT with regards to processes
inside organizations, which has the purpose
to improve the development and maintenance
of cloud services and web applications.
The core concept of DevOps, is to unify
Engineering and Operations departments in
one non-seloed entity handling both
development and running of services. The
overall model is cyclic and relies on iterative
approaches.
Responding to incidents in
order to unblock businesses
and customers relying on your
product is very important.
The GDPR defines strict rules in
terms of what the controller
Support and the processor must do
when incidents impacting
personal data occur and they
all relate to reporting and
communication.
Customer provides its feedback in a passive way! Thanks
to telemetry, it is possible to log user’s activity.
Feedback
The product undergoes development, testing and
deployment. In this phase, the direction is from Teams
to the customer.
Development
Commit
A change (feature or bug
fix) is submitted to the
repository for merge
into the codebase.
Emit telemetry
The product emits telemetry
as users work on it. Services
are responsible for storing
telemetry data in the cloud.
Collect telemetry
Periodically, as part of the development
process, Teams collect telemetry and
process the results by aggregating data
and producing actionable output.
Act on telemetry
Basing on results from telemetry,
the next iteration of the product
will contain bug fixes or features
which improve the product.
Test
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Build
Tests are run against the
generated build. If one
or more tests fail, the
merge job is aborted.
Stage to PPE
Build is deployed into a
pre-production environ-
ment and published for
download.
Stage to PROD
Build is deployed
directly to production
into the customer’s live
environment.
Telemetry
Data relating the application
in the context of a user
session.
It should not contain
personal data, so it is
important to strip sensitive
information from it.
User
The user, using his
browser, interacts with
the web application and
performs his tasks.
Log data
Telemetry data is
processed in order to
remove sensitive
information.
Collect
All feedback from
application is collected
and assembled thanks
to predefined queries.
Analyze
Developers analyze
results in next iteration
and take action accord-
ing to feedback.
Web app
The application resides
on a server and runs in
the Cloud.
Logging telemetry
The user interacts with the
product and, as it does, the
system will raise events that
will be logged in another
system/service in the Cloud.
The data being generated has
to be stripped of any
sensitive information that
might relate to the identity of
the user (personal data) so
that the logged information
are not subject to the
Regulation.
If the data being logged is
required to have personal
data, then it is probably not
telemetry what we are talking
about, but something else. As
a best practice, telemetry
should not contain any
personal data.
The End
Thank you
This work is licensed under a
Creative Commons
Attribution-NonCommercial-NoDerivatives
4.0 International License
Cover: European Infrastructure
This work includes artworks
designed by Freepik.com.
April 2018
v1.0
Keywords
#gdpr #devops #agile
#techtalk #microsoft
#data-protection #privacy
#trust #big-data #telemetry
Presentation info
Duration: 45 mins.
Background: Non-technical,
juridical
Audience: All, students
This work includes artworks
designed by Shutterstock.com.
Andrea Tino
Software Development Engineer II
Twitter:
E-Mail:
@_atino
andrea.tino@microsoft.com

Development & GDPR

  • 1.
    The challenges ofbeing an Agile|DevOps enabled organization implementing GDPR. Dev & GDPR Andrea Tino (Software Development Engineer)
  • 2.
    Our journey The agendafor today A very brief overview of the most important points of GDPR. An analysis of the challenges that an organization faces when trying to implement GDPR. Pocus on Process and also Engineering with highlights on DevOps practices. What? Data is the fuel of modern business but it belongs to users and therefore it needs to be protected. Protecting user’s data is a long term commitment and raises several challenges both in the process and in the software architecture. Why? As part of the process and daily worklife inside one organization and its business. Where? GDPR is already in place but companies are still struggling to implement it in a sustainable way which is in harmony with the process in the organization. When? The main personas involved in this analysis are developers inside a DevOps-enabled organization. Who? GDPR can be explained from many angles. The analysis we will conduct is from the POV of developers and managers from inside an organization. The aim is to talk about GDPR in a very practical and biz--grounded way. How?
  • 3.
    Warning This is atechnical talk about GDPR from the pov of Engineers This is not a lecture deeply focused on GDPR!
  • 4.
    GDPR The General DataProtection Regulation became in force on May 25 2018 Regulation EU 2016/679 supersedes 95/46/EC
  • 5.
    Definitions Wording and terms Chapter1, Article 4 provides the definitions we need...
  • 6.
    personal data non personaldata data Chapter I - General provisions Article 4: Definitions 1. ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person; Personal Data The first name, middle name and last name of a person are all considered personal data. Nicknames can be considered too as they can be used to identify a data subject. Names Identification codes can be considered personal data when they can lead to the identity of a data subject. If an ID isused to identify a record related to a data subject but that ID cannot lead to its identity, then it is not considered to be personal data. IDs
  • 7.
    store delete modify/transform Chapter I -General provisions Article 4: Definitions 2. ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction; Processing of data The simple act of storing (saving) data, in general, is considered processing, even though, computationally speaking, the data itself is not touched in any other way. Storage Moving, transmitting, disseminating personal data is also considered a form of processing of data. Move Since the moment data comes into the database of a system to the moment that data leaves the database, all the operations performed on that chunk of imformation are considered processing. Rule of thumb Removing data is also considered processing. Delete
  • 8.
    reverse pseudonymisation pseudoed-data additional data Chapter I- General provisions Article 4: Definitions 5. ‘pseudonymisation’ means the processing of personal data in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately and is subject to technical and organisational measures to ensure that the personal data are not attributed to an identified or identifiable natural person; Pseudonymisation Personal data which has been processed via pseudonymisation remains classified as “personal” because it can still lead to the identity of a person. Still personal The system must ensure that both pseudoed-data and the additional data alone cannot lead to an identity and that the additional data is kept secure in a different location. This is achieved by specific algorithms the software must implement. Security
  • 9.
    controller processor data processing purposes meansof data processing defines defines processes personal data according to processes personal data by means of data subject consents to consents to Chapter I - General provisions Article 4: Definitions 7. ‘controller’ means the natural or legal person, public authority, agency or other body which, alone or jointly with others, determines the purposes and means of the processing of personal data; where the purposes and means of such processing are determined by Union or Member State law, the controller or the specific criteria for its nomination may be provided for by Union or Member State law; 8. ‘processor’ means a natural or legal person, public authority, agency or other body which processes personal data on behalf of the controller; 11. ‘consent’ of the data subject means any freely given, specific, informed and unambiguous indication of the data subject's wishes by which he or she, by a statement or by a clear affirmative action, signifies agreement to the processing of personal data relating to him or her; Controller & Processor The controller defines the main data processing documents required by the Regulation (according to which the user must give consent to the processing of his personal data) and the controller executes the processing. Quick look
  • 10.
    Most relevant articles Theparts of the Regulation Engineers look at The first articles have an Engineering “translation” which can be used to identify the challenges arising to achieve full compliance to the Regulation. As Engineers, we want to know the impact of GDPR on our Organization and system. Excerpts from the Regulation Description and summary Engineering challenges and analysis
  • 11.
    “Let’s build aspaceship GDPR compliant system!!!”
  • 12.
    Personal Data Processing GDPR-covered ChapterI - General provisions Article 1: Subject-ma er and objectives 1. This Regulation lays down rules relating to the protection of natural persons with regard to the processing of personal data and rules relating to the free movement of personal data. 2. This Regulation protects fundamental rights and freedoms of natural persons and in particular their right to the protection of personal data. 3. The free movement of personal data within the Union shall be neither restricted nor prohibited for reasons connected with the protection of natural persons with regard to the processing of personal data. Introducing GDPR The first chapter, comprehensive of the most important articles in the Regulation, explains what the Regulation is about and, therefore, what it covers in terms of subjects and objectives. Protection The GDPR is about protecting personal data of persons. It is not only about that, it is also about the protection of the movement of this information. It is important to note how the Regulation highlights that the data being protected is of natural persons. So this is not about data belonging to items or animals, the GDPR is person-centric. If the data is about companies or other organizations, and that data relates to persons in the context of that company, the Regulation still applies. Is your organization impacted? If the software you build is about handling personal data of persons, then you should expect to be impacted by the Regulation. This is also true if your software deals with moving of such data. What is GDPR about?
  • 13.
    Chapter I -General provisions Article 2: Material scope 1. This Regulation applies to the processing of personal data wholly or partly by automated means and to the processing other than by automated means of personal data which form part of a filing system or are intended to form part of a filing system. 2. This Regulation does not apply to the processing of personal data: (a) in the course of an activity which falls outside the scope of Union law; (b) by the Member States when carrying out activities which fall within the scope of Chapter 2 of Title V of the TEU; (c) by a natural person in the course of a purely personal or household activity; (d) by competent authorities for the purposes of the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties, including the safeguarding against and the prevention of threats to public security. Processing of personal data Article 4 provides all the needed definitions of personal data and related. Article 2, however, defines that the Regulation covers the processing of personal data. The protection coverage is full as any type of processing is included in the Regulation: both by programs and humans (manual processing). In other words: the GDPR is technology-agnostic. Exceptions on coverage The Regulation does not apply in certain cases. The most prominent are: Other persons processing personal data of other persons in the context of a family unit. Government authorities are excepted from the Regulation in the context of specific tasks. Is your software impacted? If your software needs to handle, in any way, any personal data, then yes! The Regulation must be considered by your organization. What is being protected? Is your software excepted? If you can guarantee that your software is used in a family unit and the users will only store personal data of other family members, the exception is valid. But these conditions are unfeasible in a business and even if so, in future a licensing expansion would be difficult to implement if the Regulation was not uptaken from the beginning. If your software is only licensed to governmental entities, the exception could be applicable.
  • 14.
    Chapter I -General provisions Article 3: Territorial scope 1. This Regulation applies to the processing of personal data in the context of the activities of an establishment of a controller or a processor in the Union, regardless of whether the processing takes place in the Union or not. 2. This Regulation applies to the processing of personal data of data subjects who are in the Union by a controller or processor not established in the Union, where the processing activities are related to: (a) the offering of goods or services, irrespective of whether a payment of the data subject is required, to such data subjects in the Union; or (b) the monitoring of their behaviour as far as their behaviour takes place within the Union. 3. This Regulation applies to the processing of personal data by a controller not established in the Union, but in a place where Member State law applies by virtue of public international law. Controller and processor’s locations The coverage focuses on the controller and processor’s locations. Regardless of where the data subject is, if the former are located in the Union, then the Regulation applies. Location of the data subject If the data subject is in the Union but the controller or processor are not, then the Regulation applies under some conditions on the processing (purposes and means). Identify the location If your software handles personal data, then it must be able to identify/track the location of the data subjects those information belong to. This association is very important. Where the protection? Is your software impacted? If your organization is in the Union, then yes! Very simple. If not in the Union, then it applies only if the data subject is in the Union. For licensing purposes, it is always better not to make assumptions on the location of customers/users to always be able to scale your application.
  • 15.
    GDPR applies controller processor datasubject NOT IN UNION NOT IN UNION IN UNION GDPR does NOT apply IN UNION GDPR applies Territoriality User-centric The diagram shows how the GDPR is basically focused on the user. If the data subject is in the EU, then its personal data is protected by the Regulation! Non applicable The only case where GDPR has no jurisdiction is when all the parties are outside the Union.
  • 16.
    processor data subject Union datasubjectcontroller controller Microsoft’s solution for small and big businesses targeting ERP systems. Companies can host their solution in the Cloud on Azure and customers can have access to their business data. Case of study: Microsoft Dynamics 365 The controller is the company developing its solution on the Microsoft platform, while the platform itself is the processor. Customers can be from all over the world, the data subject can be part or not of the Union. Details The Regulation applies when the data subject is in the Union or the controller is, because the software is used to sell goods or services.
  • 17.
    controller processor Union data subject ShinseiBank offers an online banking system for account holders. The system is offered only to residents in Japan to track and manage their money, stock and or savings. Case of study: 新生銀行 (Shinsei Bank) Banking System The controller and the processor are both located out of the Union. Customers, data subjects, are required to be residents in Japan, therefore outside of the Union’s jurisdiction. Details The Regulation does NOT apply because the controller and the processor are both in Japan and they offer services only to Japanese residents.
  • 18.
    Articles 1, 2,3 and 4 Understand GDPR coverage The first step towards GDPR compliance is to assess whether the GDPR applies to your organization, and, if so, to what extent. This analysis starts with understanding which data you have and where it resides. Inventory your data It’s important to inventory your organization’s data. This will help you understand which data is personal, and to identify the systems where that data is collected and stored, understand why it was collected, how it is processed, and how long it is retained. 1. Discover On the journey to compliance 2. Manage 3. Protect 4. Report
  • 19.
    Chapter II -Principles Article 5: Principles relating to processing of personal data 1. Personal data shall be: (b) collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes; further processing for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes shall, in accordance with Article 89(1), not be considered to be incompatible with the initial purposes (‘purpose limitation’); (c) adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed (‘data minimisation’); (d) accurate and, where necessary, kept up to date; every reasonable step must be taken to ensure that personal data that are inaccurate, having regard to the purposes for which they are processed, are erased or rectified without delay (‘accuracy’); (f) processed in a manner that ensures appropriate security of the personal data... Limitations on processing The processor cannot freely process data however it desires. Conditions apply. In particular, processing is permitted as long as the boundaries determined by the purposes and means stated by the controller are observed. Exceptions The processor can process data out of the purposes stated by the controller as long as this further processing is limited to certain scopes. History tracking and statistical analysis is common practice and allowed by the Regulation. Security The Regulation imposes certain constraints on security. Personal data must be kept safe and protected from any event which may cause data-loss. State purposes and means Your software must be able to present the user the list of purposes for which his personal data is collected. A description of the means used to perform the processing must be provided as well. How can I process? Track history and get statistics If your software is in the Cloud, this is especially valid. In a DevOps-enabled organization, keeping track of the history of operations on personal data is important as well as extracting relevant statistics that can help the business grow. Since the Regulation provides clearance on this topic, it is always possible to perform such activities.
  • 20.
    Chapter II -Principles Article 9: Processing of special categories of personal data 1. Processing of personal data revealing racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, and the processing of genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person's sex life or sexual orientation shall be prohibited. 2. Paragraph 1 shall not apply if one of the following applies: (a) the data subject has given explicit consent to the processing of those personal data for one or more specified purposes, except where Union or Member State law provide that the prohibition referred to in paragraph 1 may not be lifted by the data subject; Not everything can be processed Recalling that the Regulation by “processing” also includes the simple storing of data, processors cannot process certain categories of information. As the recital explains, certain data concerning data subjects are sensitive to their rights and freedoms. Personal data that can lead to a person’s race, religion, sexual orientation and other information directly related to the foundamental human rights cannot be processed. Exceptions There are many exceptions to Paragraph 1, the most important and common is the possibility to rely on the data subject’s explicit consent. Do not track sensitive information Your software must not store or process, in any way, certain categories of data. It must be guaranteed that every form/UI-artifact collecting information from users will not ask for any of those sensitive details. Can I process this?
  • 21.
    Create Read Update Delete data subject controller grants rights ChapterIII - Rights of the data subject Section 2 - Information and access to personal data Article 15: Right of access by the data subject 1. The data subject shall have the right to obtain from the controller confirmation as to whether or not personal data concerning him or her are being processed... Section 3 - Rectification and erasure Article 16: Right to rectification The data subject shall have the right to obtain from the controller without undue delay the rectification of inaccurate personal data concerning him or her... Article 17: Right to erasure (‘right to be forgo en’) 1. The data subject shall have the right to obtain from the controller the erasure of personal data concerning him or her without undue delay and the controller shall have the obligation to erase personal data without undue delay where one of the following grounds applies: Full access to data The data subject has the right to read, write and even delete his personal data. CRUD capabilities The system/software must be able to locate personal data of users and let them perform tasks related to CRUD. Data subject & CRUD Controller’s responsability It is important to note how it is responsability of the controller to make sure the right to access, rectify and erase is properly exercised by data subjects.
  • 22.
    Articles 5, 9and 15 Data governance It is a plan can help you define policies, roles, and responsibilities for the access, management, and use of personal data, and can help you ensure that your data handling practices comply with the GDPR. Data classification Adopting a classification scheme that applies throughout your organization can be particularly helpful for responding to data subject requests, because it enables you to identify more readily and process personal data requests. 1. Discover On the journey to compliance 2. Manage 3. Protect 4. Report
  • 23.
    Chapter IV -Controller and processor Section 1 - General obligations Article 25: Data protection by design and by default 1. Taking into account the state of the art, the cost of implementation and the nature, scope, context and purposes of processing as well as the risks of varying likelihood and severity for rights and freedoms of natural persons posed by the processing, the controller shall, both at the time of the determination of the means for processing and at the time of the processing itself, implement appropriate technical and organisational measures, such as pseudonymisation, which are designed to implement data-protection principles, such as data minimisation, in an effective manner and to integrate the necessary safeguards into the processing in order to meet the requirements of this Regulation and protect the rights of data subjects. The Regulation supports/advices the use of Pseudonymisation to achieve data minimisation. Data separation The system must be able to pseudonymise personal data. There are many techniques available, however they all require architectures based on strong and well scalable databases. If your organization implements DevOps, micro-services can help achieve a good separation between business logic and data inside systems. In order to successfully implement pseudonymisation, the separation of additional data from replaced data is possible in maintainable ways. Pseudonymisation helps achieving Minimised data is: adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed. Article 5, Paragraph 1, Point C Data minimisation principle The processing of personal data in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately and is subject to technical and organisational measures to ensure that the personal data are not attributed to an identified or identifiable natural person. Article 4, Paragraph 5 Pseudonymisation
  • 24.
    Article 25 Organizations increasingly understandthe importance of information security. The GDPR raises the bar. It requires that organizations take appropriate technical and organizational measures to protect personal data from loss or unauthorized access or disclosure. 1. Discover On the journey to compliance 2. Manage 3. Protect 4. Report
  • 25.
    Chapter II -Principles Article 6: Lawfulness of processing 1. Processing shall be lawful only if and to the extent that at least one of the following applies: (a) the data subject has given consent to the processing of his or her personal data for one or more specific purposes; Article 7: Conditions for consent 1. Where processing is based on consent, the controller shall be able to demonstrate that the data subject has consented to processing of his or her personal data. 3. The data subject shall have the right to withdraw his or her consent at any time. The withdrawal of consent shall not affect the lawfulness of processing based on consent before its withdrawal. Prior to giving consent, the data subject shall be informed thereof. It shall be as easy to withdraw as to give consent. Consent For the processing to be lawful, at least one of many conditions must be met. The most common one concerns the data subject to provide its consent to the processing of its personal data. Consent can be revoked by the data subject at any time. Asking for consent Your software must be able to present the user an interface which can be used to ask for consent to the processing of personal data. This UI must be clear and the question must be easy to understand. Can I process? Track consent The system must be able to track consent given by users so that it is possible to exhibit proof later. if ever required. Update consent Your software must be able to give the user the ability to modify his authorization to process personal data.
  • 26.
    controller processor consents to processing data subject 1 processesdata provides information 2 4 gets data 3 Chapter III - Rights of the data subject Section 2 - Information and access to personal data Article 13: Information to be provided where personal data are collected from the data subject 1. Where personal data relating to a data subject are collected from the data subject, the controller shall, at the time when personal data are obtained, provide the data subject with all of the following information: (a) the identity and the contact details of the controller and, where applicable, of the controller's representative; (c) the purposes of the processing for which the personal data are intended as well as the legal basis for the processing; 2. In addition to the information referred to in paragraph 1, the controller shall, at the time when personal data are obtained, provide the data subject with the following further information necessary to ensure fair and transparent processing: (a) the period for which the personal data will be stored, or if that is not possible, the criteria used to determine that period; The role of the controller Article 13 is all about the responsabilities of the controller. Communicating with the data subject The system must be able to communicate with the data subject in order to send him messages. Therefore it is important to ensure that a reliable channel can be enstablioshed. Using e-mail verification techniques is highly recommended to avoid fake accounts. Automating the process The Regulation requires information on the controller to be sent when the latter acquires the data. The system will decide how to perform this operation. Automation can be a desirab;e solution to avoid manual work and human error. Should I communicate? Timing It is important to note that the informartion are required to be sent when they are acquired by the controller.
  • 27.
    controller processor data subject data exchange provides information 2 4 third party getsdata 3 1 processes data Chapter III - Rights of the data subject Section 2 - Information and access to personal data Article 14: Information to be provided where personal data have not been obtained from the data subject 1. Where personal data have not been obtained from the data subject, the controller shall provide the data subject with the following information: (a) the identity and the contact details of the controller and, where applicable, of the controller's representative; (c) the purposes of the processing for which the personal data are intended as well as the legal basis for the processing; 2. In addition to the information referred to in paragraph 1, the controller shall provide the data subject with the following information necessary to ensure fair and transparent processing in respect of the data subject: (a) the period for which the personal data will be stored, or if that is not possible, the criteria used to determine that period; Same as for Article 13 Nothing changes when data is not obtained directly from the data subject. Communicating with the data subject The system must retrieve the proper set of information in order to be able to reach for the data subject when data is obtained from a different source. Should I relay? Same rights The controller must still reach for the data subject in order to notify him about the processing.
  • 28.
    Articles 6, 13and 14 You have to be more transparent about not only how you handle personal data, but also how you maintain documentation defining the processes and use of personal data. Auditing Organizations that process personal data need to keep records about the purposes of processing; the categories of personal data processed; the identity of third parties with whom data is shared; whether (and which) third countries receive personal data, and the data. 1. Discover On the journey to compliance 2. Manage 3. Protect 4. Report
  • 29.
    DevOps practices More onthe impact on DevOps practices PLAN DEVELOP TEST PACKAGE RELEASE DEPLOY MONITOR A huge part of DevOps is based on collecting data from customers, as they run the product, in order to collect more info about their experience. That information is going to be used later for improving the Telemetry product even further, detecting issues before they surface or detecting errors for quick escalation/resolution. In order to be compliant with the GDPR, it’s important not to log certain information. DevOps in a nutshell DevOps is a very modern trend in the Software Industry and IT with regards to processes inside organizations, which has the purpose to improve the development and maintenance of cloud services and web applications. The core concept of DevOps, is to unify Engineering and Operations departments in one non-seloed entity handling both development and running of services. The overall model is cyclic and relies on iterative approaches. Responding to incidents in order to unblock businesses and customers relying on your product is very important. The GDPR defines strict rules in terms of what the controller Support and the processor must do when incidents impacting personal data occur and they all relate to reporting and communication.
  • 30.
    Customer provides itsfeedback in a passive way! Thanks to telemetry, it is possible to log user’s activity. Feedback The product undergoes development, testing and deployment. In this phase, the direction is from Teams to the customer. Development Commit A change (feature or bug fix) is submitted to the repository for merge into the codebase. Emit telemetry The product emits telemetry as users work on it. Services are responsible for storing telemetry data in the cloud. Collect telemetry Periodically, as part of the development process, Teams collect telemetry and process the results by aggregating data and producing actionable output. Act on telemetry Basing on results from telemetry, the next iteration of the product will contain bug fixes or features which improve the product. Test Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Build Tests are run against the generated build. If one or more tests fail, the merge job is aborted. Stage to PPE Build is deployed into a pre-production environ- ment and published for download. Stage to PROD Build is deployed directly to production into the customer’s live environment. Telemetry Data relating the application in the context of a user session. It should not contain personal data, so it is important to strip sensitive information from it.
  • 31.
    User The user, usinghis browser, interacts with the web application and performs his tasks. Log data Telemetry data is processed in order to remove sensitive information. Collect All feedback from application is collected and assembled thanks to predefined queries. Analyze Developers analyze results in next iteration and take action accord- ing to feedback. Web app The application resides on a server and runs in the Cloud. Logging telemetry The user interacts with the product and, as it does, the system will raise events that will be logged in another system/service in the Cloud. The data being generated has to be stripped of any sensitive information that might relate to the identity of the user (personal data) so that the logged information are not subject to the Regulation. If the data being logged is required to have personal data, then it is probably not telemetry what we are talking about, but something else. As a best practice, telemetry should not contain any personal data.
  • 32.
    The End Thank you Thiswork is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License Cover: European Infrastructure This work includes artworks designed by Freepik.com. April 2018 v1.0 Keywords #gdpr #devops #agile #techtalk #microsoft #data-protection #privacy #trust #big-data #telemetry Presentation info Duration: 45 mins. Background: Non-technical, juridical Audience: All, students This work includes artworks designed by Shutterstock.com. Andrea Tino Software Development Engineer II Twitter: E-Mail: @_atino andrea.tino@microsoft.com