1. Trust Management as a Service
Enabling Trusted Execution in the Face of Byzantine Stakeholders
Sergei ArnautovSébastien Vaucher, Rafael Pires,
Valerio Schiavoni, Pascal Felber
Franz Gregor, Wojciech Ozga,
Do Le Quoc, André Martin,
Christof Fetzer

2. Real-life use case
Dresden, 09.06.2020
To whom it may concern
This document has been created using online
service for automatic conversion of handwritten
documents into digital data via machine
learning.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been created
using online service for automatic
conversion of handwritten documents
into digital data via machine
learning.
Sincerely
Automatic conversion using
Machine Learning (ML)
Handwritten document Digital document

3. Real-life use case
Stakeholders
Training data owner
Training data Inference
Training Model
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely

4. Stakeholders
Real-life use case
Training code owner
Training data owner
Inference
Model
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely
Training
Training data

5. Training code owner
Stakeholders
Real-life use case
Training data owner
Model owner
Inference
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely
Training
Training data
Model

6. Training data owner
Training code owner
Model owner
Inference code owner
Stakeholders
Real-life use case
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely
Training
Training data
Model
Inference

7. Clients
Training data owner
Training code owner
Model owner
Inference code owner
Stakeholders
Real-life use case
Training
Training data
Model
Inference
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely

8. Clients
Training data owner
Training code owner
Model owner
Inference code owner
Cloud Provider
Stakeholders
Real-life use case
Training
Training data
Model
Inference
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely

9. Stakeholders
Clients
Training data owner
Training code owner
Model owner
Inference code owner
lack of trust
Real-life use case
Cloud Provider
Training
Training data
Model
Inference
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
This document has been
created using online service
for automatic conversion of
handwritten documents into
digital data via machine
learning.
Sincerely

10. What a malicious stakeholder might do?

11. memory
Confidentiality integrity freshness (CIF)
application
data
secrets
libraries

12. memory
Confidentiality integrity freshness (CIF)
application
data
secrets
libraries
I read application’s
secrets
Cloud provider
with root access

13. application
libraries
Confidentiality integrity freshness (CIF)
of code

14. application
I replaced the library
to change application
behavior
Confidentiality integrity freshness (CIF)
of code
IT operations
updating application
libraries

15. Confidentiality integrity freshness (CIF)
of data
memory
data
application

16. Confidentiality integrity freshness (CIF)
I modified the counter of
model executions to work
around licensing issues
of data
memory
data
application
Digitization service
provider

17. application
Confidentiality integrity freshness (CIF)
time
t=1
state:
remaining model
executions: 1

18. Confidentiality integrity freshness (CIF)
time
t=1 t=2
application application
state:
remaining model
executions: 0
state:
remaining model
executions: 1

19. application
I reloaded the previous
application state (t=1)
to get rid of licensing
problems
Confidentiality integrity freshness (CIF)
time
t=1 t=2 t=3
application application
User of
ML model
state:
remaining model
executions: 0
state:
remaining model
executions: 1
state:
remaining model
executions: 1

20. How to guarantee
confidentiality integrity freshness (CIF)
of
data and code execution
in the face of
Byzantine stakeholders?

21. Privileged attack vectors
Application
Data
Secrets
(de/encryption keys)
Libraries
Configuration
passwd=PW1
application
data
secrets
(de/encryption keys)
libraries
configuration
passwd=PW1

22. Privileged attack vectors
Cloud provider
with root access
read secrets
from memory
application
data
secrets
(de/encryption keys)
libraries
configuration
passwd=PW1

23. Privileged attack vectors
Cloud provider
with root access
read secrets
from memory
IT operations
replace
library version
application
data
secrets
(de/encryption keys)
libraries
configuration
passwd=PW1

24. Privileged attack vectors
System
administrator
read secrets
from disk
Cloud provider
with root access
read secrets
from memory
IT operations
replace
library version
application
data
secrets
(de/encryption keys)
libraries
configuration
passwd=PW1

25. Trusted Execution Environment
Existing solutions: SCONE, Graphene
application
data
secrets
(de/encryption keys)
libraries
System
administrator
read secrets
from disk
Cloud provider
with root access
read secrets
from memory
IT operations
replace
library version
configuration
passwd=PW1

26. Trusted Execution Environment
More attacks
Developer
plain text on
developer’s machine
configuration
passwd=PW1
application
data
secrets
(de/encryption keys)
libraries
configuration

27. Trusted Execution Environment
plain text on
developer’s machine
IT operations
updating application
update to
malicious code
More attacks
Developer
configuration
passwd=PW1
application
data
secrets
(de/encryption keys)
libraries
configuration

28. Trusted Execution Environment
plain text on
developer’s machine
IT operations
updating application
update to
malicious code
rollback to
previous state
Client using
3rd party library
configuration
passwd=PW1
More attacks
Developer
application
data
secrets
(de/encryption keys)
libraries
configuration

29. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Our solution: PALÆMON
application
data
secrets
(de/encryption keys)
libraries
Configuration
PALÆMON runtime
configuration

30. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Our solution: PALÆMON
Configuration
inject generated secrets
after attestation
PALÆMON runtime
Developer
application
data
secrets
(de/encryption keys)
libraries
security policy
configuration
passwd=$VAR$
configuration
passwd=secret
plain text on
developer’s machine

31. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Our solution: PALÆMON
Configuration
Stakeholders
control
security policy
inject generated secrets
after attestation
PALÆMON runtime
IT operations
updating application
update to
malicious code
Developer
application
data
secrets
(de/encryption keys)
libraries
security policy
configuration
passwd=$VAR$
configuration
passwd=secret
plain text on
developer’s machine

32. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Our solution: PALÆMON
application
data
secrets
(de/encryption keys)
libraries
Configuration
Stakeholders
control
security policy
security policy
inject generated secrets
after attestation
keep sending integrity hash
of the application’s state
IT operations
updating application
update to
malicious code
rollback to
previous state
Client using
3rd party library
configuration
passwd=$VAR$
configuration
passwd=secret
PALÆMON runtime
Developer
plain text on
developer’s machine

33. PALÆMON: secret management

34. Security policies
ML model
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
document
conversion service

35. ML model
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Document
conversion service
Security policies
Security policy #2 Security policy #3
Security policy #1
security policy #2
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
security policy #3
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
document
conversion service
security policy #1

36. ML model
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Document
conversion service
import
de/encryption key
Security policies
Security policy #2 Security policy #3
Security policy #1
security policy #2
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
security policy #3
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
document
conversion service
security policy #1

37. ML model
Security policy #2
exec
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Security policy #3
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Document
conversion service
Security policy #1
input
ML
encrypted
volume
Handwritten
documents
(encrypted)
output
Digital
documents
(encrypted)
import
de/encryption key
Security policies
security policy #2
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
security policy #3
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
document
conversion service
security policy #1
ML
encrypted
volume
handwritten
documents
(encrypted)
digital
documents
(encrypted)

38. ML model
owner
import
de/encryption key
ML model
security policy #2
exec
govern
govern
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
security policy #3
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
Dresden, 09.06.2020
To whom it may concern
Congratulations, you did the effort of reading this super
secret letter. You are a very good researcher. Thank you for
listening to this talk.
Sincerely,
Wojciech Ozga
document
conversion service
security policy #1
input
ML
encrypted
volume
handwritten
documents
(encrypted)
output
digital
documents
(encrypted)
Client
Security policies

39. Security policies
Tag
policy board
allowed hostsapplication
MRE
application arguments
environment variables
configuration variables
security policy
import / export
secrets

40. Security policies
Tag
policy board
allowed hostsapplication
MRE
application arguments
environment variables
configuration variables
security policy
import / export
secrets

41. Security policies
Tag
policy board
allowed hostsapplication
MRE
application arguments
environment variables
configuration variables
security policy
import / export
secrets

42. Security policies
Tag
policy board
allowed hostsapplication
MRE
application arguments
environment variables
configuration variables
security policy
import / export
secrets

43. Security policies
Tag
policy board
allowed hostsapplication
MRE
application arguments
environment variables
configuration variables
security policy
import / export
secrets

44. Approval process
send policy
(create/update)
PALÆMON
request
approval
Developer Model owner
Inference code
owner
Client

45. Approval process
send policy
(create/update)
PALÆMON
request
approval
Developer Model owner
Inference code
owner
Client

46. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Transparent key distribution
application
libraries
configuration
passwd=$VAR$
attestation
security policy
VAR=fa81c3a4 (generated)
data
secrets
(de/encryption keys)
PALÆMON runtime

47. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Transparent key distribution
inject generated secrets
PALÆMON runtime
application
libraries
configuration
passwd=fa81c3a4
security policy
VAR=fa81c3a4 (generated)
data
secrets
(de/encryption keys)

48. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Transparent key distribution
application
libraries
configuration
passwd=$VAR$
configuration
passwd=fa81c3a4
inject generated secrets
security policy
VAR=fa81c3a4 (generated)
data
secrets
(de/encryption keys)
PALÆMON runtime
source code
repository
Developer

49. PALÆMON: application updates

50. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Application updates
application
libraries
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
data
secrets
(de/encryption keys)
attestation fails!
PALÆMON runtime
IT operations
updating application
update to
malicious code

51. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Application updates
Application
Libraries
Configuration
Security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
Data
Secrets
(de/encryption keys)
PALÆMON runtime
IT operations
updating application
update policy
application
libraries
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
data
secrets
(de/encryption keys)

52. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Application updates
Application
Libraries
Configuration
Security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
approve update
Stakeholders
Data
Secrets
(de/encryption keys)
PALÆMON runtime
IT operations
updating application
application
libraries
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
data
secrets
(de/encryption keys)

53. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Application updates
Application
Libraries
IT operations
updating application
update application
Configuration
Security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 51cc..a9
Data
Secrets
(de/encryption keys)
PALÆMON runtime
application
libraries
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 51cc..ab
data
secrets
(de/encryption keys)

54. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Application updates
Application
Libraries
Configuration
passwd=fa81c3a4
Security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 51cc..a9
Data
Secrets
(de/encryption keys)
inject generated secrets
after attestation
PALÆMON runtime
application
libraries
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 51cc..ab
data
secrets
(de/encryption keys)

55. PALÆMON: rollback protection

56. Trusted Execution Environment
Trusted Execution Environment
PALÆMON
Rollback protection
application
libraries
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
keep sending integrity hash
of the state (tag)
data
secrets
(de/encryption keys)
PALÆMON runtime

57. Trusted Execution Environment
Trusted Execution Environment
rollback to
previous state
Client using
3rd party library
Rollback protection
application
libraries
PALÆMON
configuration
security policy
VAR=fa81c3a4 (generated)
MRE = c9d8..aa
Tag = 8af3..b4
keep sending integrity hash
of the state (tag)
data
secrets
(de/encryption keys)
PALÆMON runtime

58. Implementation
• Built on top of SCONE platform [0]
• Using Intel SGX [1] as a Trusted Execution Environment technology
• Implemented in Rust language [2]
• Uses embedded SQLite database running in the same enclave as
PALÆMON.
[0]: Sergei Arnautov, Bohdan Trach, Franz Gregor, Thomas Knauth, Andre Martin, Christian Priebe, Joshua Lind, Divya Muthukumaran, Dan O’Keeffe, Mark L.
Stillwell, David Goltzsche, Dave Eyers, Rüdiger Kapitza, Peter Pietzuch, and Christof Fetzer. SCONE: Secure Linux Containers with Intel SGX. In 12th
USENIX Symposium on Operating Systems Design and Implementation, OSDI ’16, pages 689–703. USENIX Association, 2016.
[1]: Victor Costan and Srinivas Devadas. Intel SGX explained. IACR Cryptology ePrint Archive, 2016(086):1–118, 2016.
[2]: Nicholas D. Matsakis and Felix S. Klock, II. The Rust language. In Proceedings of the 2014 ACM SIGAda Annual Conference on High Integrity Language
Technology, HILT ’14, ACM, 2014.

59. Evaluation: micro-benchmarks
• Attestation and configuration
• Rollback protection
• Approval service
• Enclave startup times
• Secret injection latency
• Secret access latency
See the paper
for more results!

60. Evaluation: attestation
Attestation and configuration latencies: even when located close to Intel’s IAS server, attestation with IAS
takes about an order of magnitude longer than with PALÆMON.

61. Evaluation: attestation
Attestation and configuration latencies: even when located close to Intel’s IAS server, attestation with IAS
takes about an order of magnitude longer than with PALÆMON.

62. Evaluation: attestation
Attestation and configuration latencies: even when located close to Intel’s IAS server, attestation with IAS
takes about an order of magnitude longer than with PALÆMON.

63. Evaluation: policy update approval
PALÆMON’s approval service: throughput/latency (left) and response latency (right) for different geographical
deployments (from local to intercontinental).

64. Evaluation: rollback protection

65. Evaluation: rollback protection

66. Evaluation: rollback protection

67. Throughput of different MC implementations
Evaluation: rollback protection

68. Evaluation: macro-benchmarks
Evaluated real-world systems executed in the context of PALÆMON:
• MariaDB database server
• Barbican and Vault key management systems
• NGINX web server
• Memcached cache system
• ZooKeeper distributed coordination service
See the paper
for more results!

69. Evaluation: MariaDB

70. Evaluation: MariaDB

71. Evaluation: MariaDB

72. Evaluation: MariaDB
MariaDB with TPC-C benchmark

73. Evaluation: MariaDB

74. Evaluation: MariaDB
MariaDB with TPC-C benchmark

75. Evaluation: MariaDB
MariaDB with TPC-C benchmark

76. Evaluation: production deployment
Native
avg. latency [ms]
PALÆMON
avg. latency [ms]
Document conversion 326 1202
Machine learning use-case executed in production

77. Summary
PALÆMON: service to manage trust in untrusted environments with
Byzantine stakeholders
• Provides confidentiality, integrity, and freshness guarantees
• Stakeholders can cooperate despite a limited trust to each other
• Transparent secret management and rollback protection
• Support for secure software updates where the root of trust is a group
of stakeholders
• Available to the research community (contact us)

78. Summary
Thank you!
wojciech.ozga@tu-dresden.de
PALÆMON: service to manage trust in untrusted environments with
Byzantine stakeholders
• Provides confidentiality, integrity, and freshness guarantees
• Stakeholders can cooperate despite a limited trust to each other
• Transparent secret management and rollback protection
• Support for secure software updates where the root of trust is a group
of stakeholders
• Available to the research community (contact us)