Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1 © Nokia 2016
Using Safety-Critical Concepts in
Privacy Engineering
Public
Dr. Ian Oliver
Bell Labs, Finland
2 November 2...
2 © Nokia 2016
Public
Auditing mobile services and associated infrastructure
from an engineering perspective...
3 © Nokia 2016
Public
Auditing mobile services and associated infrastructure
from an engineering perspective...
Privacy
Of...
4 © Nokia 2016
Public
The problem was (and still is, and will be) the lack of
good (any?) techniques for reasoning about p...
5 © Nokia 2016
So what is this privacy thing anyway?
Public
• vs Security...
• [[privacy]]
• Information wants to be free!...
6 © Nokia 2016
Privacy as...
Public
• A legal construct •“The Right to Privacy” (Warren and Brandeis, 1890)
• EU Data Prot...
7 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• morals, ethics etc
• political scien...
8 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• brand/shareh...
9 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding pr...
10 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding p...
11 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding p...
12 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding p...
13 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding p...
14 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding p...
15 © Nokia 2016
Privacy as...
Public
• A legal construct
• A philisophical construct
• An economic construct
• A guiding p...
16 © Nokia 2016
Public
The problem was (and still is, and will be) the lack of
good (any?) techniques for reasoning about ...
17 © Nokia 2016
We developed:
• Epics and Use cases for Privacy
• Checklists
• Software Development Process Integration
• ...
18 © Nokia 2016
Failure
19 © Nokia 2016
Why didn’t it work?
• Despite highly trained personel
- Cessna Single Engine Failure
• FLY THE AIRCRAFT
- ...
20 © Nokia 2016
We developed:
• Simpler ”Checklists”
• Training Courses
• Realised that no-one understood each other
• Tri...
21 © Nokia 2016
Failure
2
22 © Nokia 2016
What’s the problem now?
• Communication
• Emphasis on process over method
• Lack of understanding of role
...
23 © Nokia 2016
What’s the problem now?
Actually it was much worse:
24 © Nokia 2016
What’s the problem now?
Actually it was much worse:
Total emphasis on ”compliance”
25 © Nokia 2016
What’s the problem now?
Actually it was much worse:
Total emphasis on ”compliance”
Whatever ”compliance” m...
26 © Nokia 2016
Compliance
is fragile
Public
char collectDataFlag = 'Y'; // Future proofed boolean
// Y for yes, N for no
...
27 © Nokia 2016
????!!!
28 © Nokia 2016
Help!
Not invented here!
Were there any industries or disciplines from which we could learn?
Or are softwa...
29 © Nokia 2016
29
Serendipity
© 2013 HERE | Title | Author | Company confidential
30 © Nokia 2016
Help!
Consider information to be a dangerous item
This has various meanings in aviation, medicine, civil e...
31 © Nokia 2016
31
A quick introduction to surgical infection control
© 2013 HERE | Title | Author | Company confidential
32 © Nokia 2016
32
A quick introduction to surgical infection control
© 2013 HERE | Title | Author | Company confidential
...
33 © Nokia 2016
33
The Sterile Field
© 2013 HERE | Title | Author | Company confidential
Key:
• Sterile
• Non-sterile
34 © Nokia 2016
34
The Sterile Field
© 2013 HERE | Title | Author | Company confidential
Key:
• Sterile
• Non-sterile
Move...
35 © Nokia 2016
Some things...
• Communication
• Culture
36 © Nokia 2016
Some things...
• Communication, Structure and Semantics
• Culture
Already solved...in other fields
37 © Nokia 2016
Standardised Communication
Public
38 © Nokia 2016
Standardised Communication
Probably not personal data/ Probably personal data
39 © Nokia 2016
Standardised Communication
Forget process, just get
the information about
what’s going on...
40 © Nokia 2016
Nokia Internal
41 © Nokia 2016
Nokia Internal
42 © Nokia 2016
Nokia Internal
43 © Nokia 2016
Public
Checklists
44 © Nokia 2016
Public
Checklists
45 © Nokia 2016
Public
Morbidity and Mortality
Accident Investigation
Reporting
46 © Nokia 2016
Public
Roles and Role Integration
R&D Team
Checklist
(before review)
R&D Team
Checklist
(post-review)
Audi...
47 © Nokia 2016
Public
Experience
Data
Collection
CellID->
Location
Data
Storage
Operator
Privacy
Preprocessing
Extraction...
48 © Nokia 2016
Public
Conclusions...
49 © Nokia 2016
Public
No heroes
50 © Nokia 2016
Public
Treat privacy as a safety-critical aspect
Using Safety-Critical Concepts in Privacy Engineering
Upcoming SlideShare
Loading in …5
×

Using Safety-Critical Concepts in Privacy Engineering

1,090 views

Published on

We describe the use of safety-critical concepts, tools and techniques in privacy engineering.

Published in: Travel
  • Be the first to comment

  • Be the first to like this

Using Safety-Critical Concepts in Privacy Engineering

  1. 1. 1 © Nokia 2016 Using Safety-Critical Concepts in Privacy Engineering Public Dr. Ian Oliver Bell Labs, Finland 2 November 2016 A Lecture Given to CRiM’16, Oulu, Finland
  2. 2. 2 © Nokia 2016 Public Auditing mobile services and associated infrastructure from an engineering perspective...
  3. 3. 3 © Nokia 2016 Public Auditing mobile services and associated infrastructure from an engineering perspective... Privacy Officers & Lawyers Privacy Engineers
  4. 4. 4 © Nokia 2016 Public The problem was (and still is, and will be) the lack of good (any?) techniques for reasoning about privacy in an engineering capacity
  5. 5. 5 © Nokia 2016 So what is this privacy thing anyway? Public • vs Security... • [[privacy]] • Information wants to be free! • Freedom to do/be...? • Freedom from...? • Price, value •Ownership • You are the product • ”Anti-privacy” • Advertising, surveillance, hacking, oversharing • Personal responsibility vs Technilogical complexity • PII, personal data, pseudo-anonymous, anonymisation •variability, entropy, Navier-Stokes • f(p1...pn) -> R •The Privacy Singularity •Unification of disciplines •Mathematical Foundations of Privacy •The Fundamental Theorem of Privacy
  6. 6. 6 © Nokia 2016 Privacy as... Public • A legal construct •“The Right to Privacy” (Warren and Brandeis, 1890) • EU Data Protection Laws • Human Rights •...
  7. 7. 7 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • morals, ethics etc • political science? Kant etc.
  8. 8. 8 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • brand/shareholder value • customer relatonships • business • innovation
  9. 9. 9 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle
  10. 10. 10 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle • A sociological construct
  11. 11. 11 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle • A sociological construct • A game theoretic construct
  12. 12. 12 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle • A sociological construct • A game theoretic construct • A systems engineering construct char collectDataFlag = 'Y'; // Future proofed boolean // Y for yes, N for no void collectDataFunction(){ //collect IMEI, IMSI, MSISDN, TimeStamp and location //and send to the hardcoded IP address ... } void checkDataCollection(){ switch(collectDataFlag){ case 'N' : // don't do anything case 'Y' : // ok to collect everything collectDataFunction(); } } compliance
  13. 13. 13 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle • A sociological construct • A game theoretic construct • A systems engineering construct (pt.2) • Ontological structures • Metrics / Risk Analysis • Modelling • Privacy Engineering • Compliance • Culture & Safety Critical Systems • (Aviation, Medicine)
  14. 14. 14 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle • A sociological construct • A game theoretic construct • A systems engineering construct • An optimisation construct
  15. 15. 15 © Nokia 2016 Privacy as... Public • A legal construct • A philisophical construct • An economic construct • A guiding principle • A sociological construct • A game theoretic construct • A systems engineering construct • An optimisation construct • A mathematical construct 𝑡0 𝑡1 𝐷1 × ⋯ × 𝐷 𝑛 < 𝜀 𝑈 • metrics • topology • ontology • anonymisation & variability • turbulence, chaos theory • link back to economics & game theory • deanonymisation • information entropy
  16. 16. 16 © Nokia 2016 Public The problem was (and still is, and will be) the lack of good (any?) techniques for reasoning about privacy in an engineering capacity... ...so what did we do?
  17. 17. 17 © Nokia 2016 We developed: • Epics and Use cases for Privacy • Checklists • Software Development Process Integration • Audit Procedures (non-functional aspects) - privacy - secuity - performance - continuity (resiliance) and the result was...
  18. 18. 18 © Nokia 2016 Failure
  19. 19. 19 © Nokia 2016 Why didn’t it work? • Despite highly trained personel - Cessna Single Engine Failure • FLY THE AIRCRAFT - Air France AF447 • To much adherence to process - Processes tell everyone the order of what to do - Difficulty in handling exceptions and experts - Aviation Checklists are status checks used to assist in due dilligence in preparation for the next and future phases of flight. - Engineers aren’t stupid • Checklist replaced responsibility and expertise - For both the auditor and develoment teams • Tick-box oriented - Ask questions, Accept answers, TICK! - Limited understanding and context of naswers • Limited time-scale - One-off review
  20. 20. 20 © Nokia 2016 We developed: • Simpler ”Checklists” • Training Courses • Realised that no-one understood each other • Tried to ban (unsuccessfully) the term ”PII” • Tried to formulate requirements • Introduced more risk management ideas, eg: RCA, FMEA and the result was...
  21. 21. 21 © Nokia 2016 Failure 2
  22. 22. 22 © Nokia 2016 What’s the problem now? • Communication • Emphasis on process over method • Lack of understanding of role • Lack of legal and engineering techniques • Lack of integration of legal and engineering • The privacy organisation itself • Humans
  23. 23. 23 © Nokia 2016 What’s the problem now? Actually it was much worse:
  24. 24. 24 © Nokia 2016 What’s the problem now? Actually it was much worse: Total emphasis on ”compliance”
  25. 25. 25 © Nokia 2016 What’s the problem now? Actually it was much worse: Total emphasis on ”compliance” Whatever ”compliance” meant...
  26. 26. 26 © Nokia 2016 Compliance is fragile Public char collectDataFlag = 'Y'; // Future proofed boolean // Y for yes, N for no void collectDataFunction(){ //collect IMEI, IMSI, MSISDN, TimeStamp and location //and send to the hardcoded IP address ... } void checkDataCollection(){ switch(collectDataFlag){ case 'N' : // don't do anything case 'Y' : // ok to collect everything collectDataFunction(); } }
  27. 27. 27 © Nokia 2016 ????!!!
  28. 28. 28 © Nokia 2016 Help! Not invented here! Were there any industries or disciplines from which we could learn? Or are software engineering and legal ’special’?
  29. 29. 29 © Nokia 2016 29 Serendipity © 2013 HERE | Title | Author | Company confidential
  30. 30. 30 © Nokia 2016 Help! Consider information to be a dangerous item This has various meanings in aviation, medicine, civil engineering etc.
  31. 31. 31 © Nokia 2016 31 A quick introduction to surgical infection control © 2013 HERE | Title | Author | Company confidential
  32. 32. 32 © Nokia 2016 32 A quick introduction to surgical infection control © 2013 HERE | Title | Author | Company confidential seriously!
  33. 33. 33 © Nokia 2016 33 The Sterile Field © 2013 HERE | Title | Author | Company confidential Key: • Sterile • Non-sterile
  34. 34. 34 © Nokia 2016 34 The Sterile Field © 2013 HERE | Title | Author | Company confidential Key: • Sterile • Non-sterile Movement of materials from one area to the other must be controlled to prevent contamination of the sterile field with non-sterile items Strict protocols prevent contamination
  35. 35. 35 © Nokia 2016 Some things... • Communication • Culture
  36. 36. 36 © Nokia 2016 Some things... • Communication, Structure and Semantics • Culture Already solved...in other fields
  37. 37. 37 © Nokia 2016 Standardised Communication Public
  38. 38. 38 © Nokia 2016 Standardised Communication Probably not personal data/ Probably personal data
  39. 39. 39 © Nokia 2016 Standardised Communication Forget process, just get the information about what’s going on...
  40. 40. 40 © Nokia 2016 Nokia Internal
  41. 41. 41 © Nokia 2016 Nokia Internal
  42. 42. 42 © Nokia 2016 Nokia Internal
  43. 43. 43 © Nokia 2016 Public Checklists
  44. 44. 44 © Nokia 2016 Public Checklists
  45. 45. 45 © Nokia 2016 Public Morbidity and Mortality Accident Investigation Reporting
  46. 46. 46 © Nokia 2016 Public Roles and Role Integration R&D Team Checklist (before review) R&D Team Checklist (post-review) Audit Team Checklist (sign-in) Audit Team Checklist (time-out) Audit Team Checklist (sign-out) Project development & processes (time) System under audit Privacy Officer Legal Security Architects
  47. 47. 47 © Nokia 2016 Public Experience Data Collection CellID-> Location Data Storage Operator Privacy Preprocessing Extraction Hashing File Storage Raw Data Processing & Enrichment External Data External Cross- referencing Atomic Data Aggregation/ Report Generation Customer Reception Report Storage <<data subject>> Customer
  48. 48. 48 © Nokia 2016 Public Conclusions...
  49. 49. 49 © Nokia 2016 Public No heroes
  50. 50. 50 © Nokia 2016 Public Treat privacy as a safety-critical aspect

×