Advancing Apache Nifi Framework Security With David Handermann | Current 2022 As a flexible system for processing data to and from a variety of services, Apache NiFi provides a powerful set of capabilities. Configuration integrity and access security are essential framework features. Recent Apache NiFi releases have included a number of security-oriented improvements, ranging from automated HTTPS configuration to externalization of sensitive application properties. This presentation covers the implementation details involved with automatic certificate generation, password-based key derivation, JSON Web Token signing, repository encryption, and sensitive property management using external services. Through a combination of relevant code samples and capability demonstrations, this presentation describes framework security advances that involve both user interaction and application configuration. Providing a basic summary of selected cryptographic algorithm differences, along with code changes, will enable participants to understand the impact of various improvements. Walking through new and improved configuration capabilities allows administrators to optimize deployment security. Highlighting key implementation details encourages software developers to review and incorporate applicable security strategies.

1. Advancing
Apache NiFi
Framework Security
David Handermann
Principal Engineer, Cloudera
Project Management Committee Member, Apache NiFi 2022-10-05

2. © 2022 EXCEPTIONFACTORY 2
Introduction • David Handermann
Principal Engineer
cloudera.com
PMC Member
nifi.apache.org
Software Development Blog
exceptionfactory.com

3. © 2022 EXCEPTIONFACTORY 3
Summary
Apache 1.14.0 and following have included:
 strengthened access security
 improved configuration protection
 streamlined repository encryption

4. © 2022 EXCEPTIONFACTORY 4
Approach
Review development and deployment
strategies by considering:
 what changed
 why it changed
 how it was implemented

5. © 2022 EXCEPTIONFACTORY 5
Agenda
■ Background
■ Access Security
■ Configuration Protection
■ Repository Encryption
■ Conclusion
■

6. © 2022 EXCEPTIONFACTORY 6
Apache NiFi • Core Capabilities
 Directed Graph Data Routing and Transformation
 Extensible Component Design
 Web Interface for Design and Monitoring

7. Process Group
Processors
Status

8. © 2022 EXCEPTIONFACTORY 8
Cryptographic Terminology • Hashes
 Cryptographic Hash Function

Deterministic mathematical algorithm for deriving
a concise representation of arbitrary information

Examples: SHA-2 and SHA-3
 Key Derivation Function

Deterministic mathematical algorithm for deriving
a secret key from simple sources and iterative operations

Examples: PBKDF2 and Argon2

9. © 2022 EXCEPTIONFACTORY 9
Cryptographic Hashing • SHA-256
 Input String
 Lorem ipsum dolor sit amet
 Hexadecimal-encoded hash
 16aba5393ad72c0041f5600ad3c2c52e
c437a2f0c7fc08fadfc3c0fe9641d7a3

10. © 2022 EXCEPTIONFACTORY 10
Key Derivation • PBKDF2
 Password
 Lorem-ipsum-dolor
 Salt
 monosodium
 Pseudorandom Function
 SHA-512
 Iterations
 325000
 Hexadecimal-encoded key
 38b7834dd4d678d5c8cfc31df251528c
f880a783e7ac707340603f5b05247d00

11. © 2022 EXCEPTIONFACTORY 11
Cryptographic Terminology • Ciphers
 Symmetric Cipher

Algorithm for enciphering or deciphering
using the same key

Examples: AES or ChaCha20
 Asymmetric Cipher

Algorithm for encrypting or verifying with a public key
and decrypting or signing with a paired private key

Examples: RSA or Ed25519

12. © 2022 EXCEPTIONFACTORY 12
Agenda
■ Background
■ Access Security
■ Configuration Protection
■ Repository Encryption
■ Conclusion
■

13. © 2022 EXCEPTIONFACTORY 13
Access Security Improvements
 Single User Login and HTTPS in default settings
 JSON Web Token asymmetric signing
 HttpOnly Session Cookie storage for Tokens

14. © 2022 EXCEPTIONFACTORY 14
Single User Login and HTTPS • Changes
 Single user authentication required

HTTPS on port 8443 in NiFi 1.14.0

Single User Login Identity Provider and Authorizer

Automatic username and password generation

Automatic key pair and certificate generation

15. © 2022 EXCEPTIONFACTORY 15
Single User Login and HTTPS • Why?
 Add Minimum Authentication Requirements
 Provide Secure Standard Properties
 Support Simplified Access

16. © 2022 EXCEPTIONFACTORY 16
Single User Login and HTTPS • How?
 Random Username Generation
 Java UUID.randomUUID()
 Random Password Generation
 Java SecureRandom with Base64 encoding
 Password Stored with bcrypt Hashing
 at.favre.lib:bcrypt hashing library

17. © 2022 EXCEPTIONFACTORY 17
Single User Login and HTTPS • Password
Generate Random Password with 32 Characters
static final int RANDOM_BYTE_LENGTH = 24;
static final Base64.Encoder ENCODER = Base64.getEncoder()
.withoutPadding();
protected String generatePassword() {
final SecureRandom secureRandom = new SecureRandom();
final byte[] bytes = new byte[RANDOM_BYTE_LENGTH];
secureRandom.nextBytes(bytes);
return ENCODER.encodeToString(bytes);
}

18. © 2022 EXCEPTIONFACTORY 18
JWT Asymmetric Signing • Background
 JSON Web Tokens for HTTPS Access

Alternative to X.509 Client Certificates

Credentials exchange returns JWT
 JSON Web Tokens support multiples strategies

Username and Password with Kerberos or LDAP

Single Single-On with OpenID Connect or SAML

19. © 2022 EXCEPTIONFACTORY 19
JWT Asymmetric Signing • Changes
 Replaced Hashed Message Authentication Code and SHA-256

HS256 algorithm in NiFi 1.14.0

Random UUID as symmetric key for each user

Symmetric key stored without encryption on file system
 RSA Probabilistic Signing with SHA-512

PS512 algorithm in NiFi 1.15.0

RSA 4096 bit shared key pair with automatic rotation

Private key retained in memory and public key stored on file system

20. © 2022 EXCEPTIONFACTORY 20
JWT Asymmetric Signing • Why?
 Meet Key Length Requirements
 Avoid Storing Unencrypted Secret Keys
 Enforce Scheduled Key Rotation

21. © 2022 EXCEPTIONFACTORY 21
JWT Asymmetric Signing • How?
 Refactored using Spring Security OAuth 2.0 JOSE
 Built on Nimbus JOSE JWT library
 RSA Key Pair Generation
 Public key storage in application State Provider
 Configurable Key Pair Rotation
 Default rotation every hour
 Unique Token Identification and Revocation
 User log out caches revoked token identifier and expiration

22. © 2022 EXCEPTIONFACTORY 22
JWT Asymmetric Signing • Rotation
Generate Key Pair and Notify Services
public void run() {
final KeyPair keyPair = keyPairGenerator.generateKeyPair();
final String keyIdentifier = UUID.randomUUID().toString();
verificationKeyListener.onVerificationKeyGenerated(
keyIdentifier, keyPair.getPublic()
);
final JWSSigner jwsSigner = new RSASSASigner(keyPair.getPrivate());
signerListener.onSignerUpdated(
new JwsSignerContainer(keyIdentifier, JWSAlgorithm.PS512, jwsSigner)
);
}

23. © 2022 EXCEPTIONFACTORY 23
HttpOnly Cookie for Token • Changes
 Refactored JSON Web Tokens stored in Browser Local Storage

Local Storage persistence in NiFi 1.14.0

Bearer Token presented in HTTP Request Authorization Header

JavaScript JWT storage management across browser restarts
 JSON Web Tokens stored in Browser HttpOnly Session Cookie

HttpOnly Session Cookie persistence in NiFi 1.15.0

Bearer Token presented in HTTP Request Cookie Header

Session Cookies cleared when closing browser

24. © 2022 EXCEPTIONFACTORY 24
HttpOnly Cookie for Token • Why?
 Avoid Browser Local Storage for Tokens
 Avoid JavaScript Access to Tokens
 Resolve Advanced Interface Access Issues

25. © 2022 EXCEPTIONFACTORY 25
HttpOnly Cookie for Token • How?
 Spring Security Cookie Request and Response Filtering
 Custom Bearer Token Resolver for Cookie or Header
 ResponseCookie.Builder for Strict Same Site configuration
 Double Submit Cookie Pattern
 Cross Site Request Forgery mitigation

Random Cookie with matching Request Header

Custom Spring CSRF Token Repository

26. © 2022 EXCEPTIONFACTORY 26
HttpOnly Cookie for Token • Headers
HTTP Response with Request and Bearer Cookies
HTTP/1.1 201 Created
Set-Cookie: __Secure-Authorization-Bearer=eyJ; Secure; HttpOnly; SameSite=Strict
Set-Cookie: __Secure-Request-Token=8d7af4d0f938; Secure
HTTP Request using Double Submit Cookie Pattern
GET /nifi-api/flow/status HTTP/1.1
Host: localhost:8443
Cookie: __Secure-Authorization-Bearer=eyJ;
__Secure-Request-Token=8d7af4d0f938
Request-Token: 8d7af4d0f938

27. © 2022 EXCEPTIONFACTORY 27
Agenda
■ Background
■ Access Security
■ Configuration Protection
■ Repository Encryption
■ Conclusion
■

28. © 2022 EXCEPTIONFACTORY 28
Configuration Protection Advances
 Sensitive Properties Key Generation
 New Sensitive Properties Algorithms
 Externalized Application Properties

29. © 2022 EXCEPTIONFACTORY 29
Properties Key Generation • Changes
 Removed Default Sensitive Properties Key

Hard-coded default Properties Key in NiFi 1.13.2

Warning message logged when not configured
 Sensitive Properties Key Required

Random Properties Key generated in NiFi 1.14.0

Minimum key length of 12 characters

30. © 2022 EXCEPTIONFACTORY 30
Properties Key Generation • Why?
 Eliminate Hard-Coded Default Key
 Improve Security of Standalone Deployments
 Remove Ignorable Warning Messages

31. © 2022 EXCEPTIONFACTORY 31
Properties Key Generation • How?
 Empty Properties Key Throws Exception
 Startup fails without Sensitive Properties Key
 Secure Random Generation
 SecureRandom 24 bytes Base64 encoded to 32 characters
 Shell Command for Simplified Upgrades
 One command to upgrade from hard-coded key

32. © 2022 EXCEPTIONFACTORY 32
Properties Key Generation • Command
Shell Command to Set Properties Key
nifi.sh set-sensitive-properties-key lorem-ipsum-dolor
Flow Configuration Processed [conf/flow.xml.gz]
Flow Configuration Processed [conf/flow.json.gz]
NiFi Properties Processed [conf/nifi.properties]

33. © 2022 EXCEPTIONFACTORY 33
New Properties Algorithms • Changes
 Deprecated Weak Cryptographic Hashing and Encryption

MD5 with 1,000 iterations and AES-CBC in NiFi 1.13.2
 Key Derivation Function with Authenticated Encryption

Optional Argon2 and AES-GCM added in NiFi 1.12.0

Default PBKDF2 with AES-GCM in NiFi 1.14.0

Optional bcrypt and scrypt added in NiFi 1.14.0

34. © 2022 EXCEPTIONFACTORY 34
New Properties Algorithms • Why?
 Deprecate Weak Cryptographic Functions
 Promote Authenticated Encrypted
 Support Basic FIPS-140-2 Standards

35. © 2022 EXCEPTIONFACTORY 35
New Properties Algorithms • How?
 Refactored Complex Class to Interface and Implementations
 Replaced monolithic StringEncryptor with new components
 Leveraged Strong Key Derivation Functions
 Configurable algorithms with Argon2, bcrypt, PBKDF2, or scrypt
 Toolkit and Shell Commands for Upgrades
 Changing algorithms requires decrypting and encrypting values

36. © 2022 EXCEPTIONFACTORY 36
New Properties Algorithms • Command
Shell Command to Set Properties Algorithm
nifi.sh set-sensitive-properties-algorithm NIFI_PBKDF2_AES_GCM_256
Flow Configuration Processed [conf/flow.xml.gz]
Flow Configuration Processed [conf/flow.json.gz]
NiFi Properties Processed [conf/nifi.properties]

37. © 2022 EXCEPTIONFACTORY 37
Externalized Properties • Changes
 Externalized Storage of Application Properties

New Sensitive Property Providers in NiFi 1.15.0

Popular secret storage managers supported

Sensitive values stored outside of application

Delegated encryption supported

Secret keys stored outside of application

38. © 2022 EXCEPTIONFACTORY 38
Externalized Properties • Why?
 Address the Secret Zero Problem
 Standardize Access to Configuration Secrets

39. © 2022 EXCEPTIONFACTORY 39
Externalized Properties • How?
 Redesigned Sensitive Properties Loading
 Java ServiceLoader for Provider implementations
 Optional Bootstrap Configuration files
 Implemented Standard Providers
 Amazon Web Services Secrets Manager and KMS
 Google Cloud Platform KMS
 Microsoft Azure Key Vault Cryptography and Secrets
 HashiCorp Vault Key Values and Transit Encryption

40. © 2022 EXCEPTIONFACTORY 40
Agenda
■ Background
■ Access Security
■ Configuration Protection
■ Repository Encryption
■ Conclusion
■

41. © 2022 EXCEPTIONFACTORY 41
Repository Encryption Refinements
 Standard Secret Key Provider
 Simplified Configuration Properties

42. © 2022 EXCEPTIONFACTORY 42
Secret Key Provider • Changes
 Added Standard Java Keystore for Secret Keys

Replaced hexadecimal encoded key properties

Manage Secret Keys using Java keytool in NiFi 1.15.0

43. © 2022 EXCEPTIONFACTORY 43
Secret Key Provider • Why?
 Streamline Secret Key Handling
 Implement Standard Protection for Secret Keys

44. © 2022 EXCEPTIONFACTORY 44
Secret Key Provider • How?
 Implemented New Key Provider
 Java KeyStore for reading Secret Keys
 Documented Examples using keytool
 Command references for generating AES Secret Keys
 Retained Support for Previous Properties
 Upgrading supported with historical properties

45. © 2022 EXCEPTIONFACTORY 45
Simplified Configuration • Changes
 Streamlined Repository and Secret Key Properties

Five properties required in NiFi 1.15.0

Single version number maps to Repository classes
 Refactored Repetitive Properties

Eliminated secret key duplication

Removed class references

46. © 2022 EXCEPTIONFACTORY 46
Simplified Configuration • Why?
 Avoid Partial Configuration
 Avoid Class References
 Reduce Code Duplication

47. © 2022 EXCEPTIONFACTORY 47
Simplified Configuration • How?
 Introduced Encryption Protocol Version
 Encapsulated current capabilities as Version 1
 Deprecated Alternative Properties
 Maintained compatibility when upgrading

48. © 2022 EXCEPTIONFACTORY 48
Simplified Configuration • Properties
Command to generate AES-256 Secret Key
keytool -genseckey
-alias primary-key
-keyalg AES
-keysize 256
-keystore conf/repository.p12
-storetype PKCS12
Application Properties required for repository encryption
nifi.repository.encryption.protocol.version=1
nifi.repository.encryption.key.id=primary-key
nifi.repository.encryption.key.provider=KEYSTORE
nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
nifi.repository.encryption.key.provider.keystore.password=2fRKmwDyMYmT7P5L

49. © 2022 EXCEPTIONFACTORY 49
Agenda
■ Background
■ Access Security
■ Configuration Protection
■ Repository Encryption
■ Conclusion
■

50. © 2022 EXCEPTIONFACTORY 50
Review
Structural security changes since Apache NiFi 1.14.0
 Better default settings
 Targeted automatic property generation
 Additional configuration strategies

51. © 2022 EXCEPTIONFACTORY 51
Apache NiFi • Version Security Highlights
 1.14.0 - Default Single User Login with HTTPS
 1.14.0 - New Sensitive Properties Algorithms
 1.14.0 - Required Sensitive Properties Key
 1.15.0 - JWT Asymmetric Signing with HttpOnly Cookies
 1.15.0 - New External Property Providers
 1.15.0 - Simplified Repository Encryption
 1.16.0 - Standardized HTTP Access Logging
 1.17.0 - Supported Sensitive Dynamic Properties

52. © 2022 EXCEPTIONFACTORY 52
References
 Single User Access and HTTPS in Apache NiFi
 Deciphering Apache NiFi Component Property Encryption
 Improving JWT Authentication in Apache NiFi
 Configuration Apache NiFi Repository Encryption
 Introducing Apache NiFi HTTP Request Logging
 Apache NiFi Support for Sensitive Dynamic Properties
EXCEPTIONFACTORY.COM

53. THANK YOU
David Handermann
Principal Engineer, Cloudera
Project Management Committee Member, Apache NiFi 2022-10-05