Why Security is important in Cassandra

1. Securing Cassandra
for Compliance (or Paranoia)

2. Hi, I'm Nate.
@zznate
https://www.linkedin.com/in/zznate
http://www.slideshare.net/zznate/
Co-Founder, CTO
The Last Pickle
Cassandra user since 2009 (v0.4)
Austin, Texas

3. Security presentations can be scary.
Here's a cat.

4. First, how did we get here and why is
securing Cassandra important?

5. "Target CEO Gregg Steinhafel Resigns In
Data Breach Fallout"
http://www.forbes.com/sites/clareoconnor/2014/05/05/target-ceo-gregg-steinhafel-resigns-in-wake-of-data-breach-fallout/
First, how did we get here and why is
securing Cassandra important?

6. I have
your
personal
information
Customers place a lot of trust
in technology companies

7. LOL! Me too!
Sometimes too much.

8. Ease of scalability comes with a price

9. HA! A bin-packed
message format with no source
verification!*
Ease of scalability comes with a price
* <currently reading o.a.c.net.MessageIn#read>

10. nmap -Pn -p7000
-oG logs/cass.gnmap 54.88.0.0/14

11. I'm publicly
discussing your
technical
shortcomings
Then you end up in this situation.

12. Meanwhile, at the FCC...
We have to require two
factor, secure socket transport
encryption, something something...
ZZZzzzzzzzZZZzz

13. We did a regulation!
My staffers still print
out my email :)

14. Why
are we doing
this again?
Sssshhhh.
I'm AES'ing...
...even though the traffic
never leaves a backplane.
Some industries will require node to node SSL

15. 1. Encrypting data at rest
2. Encrypting data on the wire
3. Authentication and authorization
4. Management and tooling
Focusing our Discussion: Architecture

16. 1. Encryption at rest

17. No matter what:
understand the failure modes

18. bit rot, entropy, etc.
Horrible things can happen with on disk encryption.

19. Don't mind me, I'm just
your key server.

20. Haha! Later!
x
What's on this
disk again?
Shrug.

21. ...but you may not have a choice.
Because we said "at rest"

22. dmcrypt, eCryptFS
Open source options:

23. Vormetric, Gazzang
Commercial options:

24. DSE Encryption
CREATETABLE users
...
WITH compression_parameters:sstable_compression = 'Encryptor'
and compression_parameters:cipher_algorithm = 'AES/ECB/
PKCS5Padding'
and compression_parameters:secret_key_strength = 128;

25. DSE Encryption
CREATETABLE users
...
WITH compression_parameters:sstable_compression = 'Encryptor'
and compression_parameters:cipher_algorithm = 'AES/ECB/
PKCS5Padding'
and compression_parameters:secret_key_strength = 128;
WARNING:
commitlog not included*
*eCryptFS would work fine for this

26. EBS Encryption
(a.k.a "not my problem")

27. (Looks like this)
EBS Encryption
(a.k.a "not my problem")
http://www.slideshare.net/AmazonWebServices/bdt323-amazon-ebs-cassandra-1-million-writes-per-second
See Crowdstrike's presentation on
Cassandra GP2 performance (with encryption):

28. Maybe Client Side?
The Java Driver now has custom codecs
which would make this easy to implement
https://github.com/datastax/java-driver/tree/3.0/manual/custom_codecs

29. Maybe Client Side?
The Java Driver now has custom codecs
which would make this easy to implement
https://github.com/datastax/java-driver/tree/3.0/manual/custom_codecs
Column-level encryption!

30. New in Cassandra 3.4
(DSE 5.1?):
Commitlog Encryption: CASSANDRA-6018
Hint File Encryption: CASSANDRA-11040
https://issues.apache.org/jira/browse/CASSANDRA-6018
https://issues.apache.org/jira/browse/CASSANDRA-11040

31. 2. Encryption on the wire

32. Because:
It is really easy to attack
an un-protected cluster

33. It takes a single Message
to insert an admin account
into the system table

34. -Dcassandra.write_survey=true
How to steal writes in real time:

35. The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic

36. Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic

37. Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic
Bonus: can be done
with NO downtime!!!

38. Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate
authentication to cluster traffic
Bonus: can be done
with NO downtime!!!
How-to guide:
http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-
server.html

39. When you are done it should look like:

40. Things to note:
Use "dc" or "rack" to limit encryption to
connections between racks and data centers

41. Thanks for that!!
Huzzah!
(But AES on modern hardware
will not be a bottleneck)

42. Things to note:
Keystore and key password must match
(artifact of JDK X.509 Impl complexity)

43. Things to note:
256 bit means export restrictions
(requires JCE provider JAR)
http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#importlimits

44. Don't forget this part or else...
Things to note:

45. Hahaha!
Now I'm hacking you over SSL.
*Still* vulnerable AND you can't see what the
attacker is doing.

46. Client to Server SSL

47. Client to Server SSL
(see slides 30 to 35)

48. Client to Server SSL
(see slides 30 to 35)
Now with NO downtime!!!
https://issues.apache.org/jira/browse/CASSANDRA-10559
Available in: 2.1.12, 2.2.4, 3.0.0

49. Need to Debug SSL?
-Djavax.net.debug=ssl
http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/ReadDebug.html

50. Certs are hard :(
Netflix Lemur:
x.509 Certificate Orchestration Framework
http://techblog.netflix.com/2015/09/introducing-lemur.html
https://github.com/Netflix/lemur

51. Certs are hard :(
Hashicorp Vault
"secures, stores, and tightly controls access to
tokens, passwords, certificates, API keys, and
other secrets in modern computing. "
https://www.vaultproject.io/

52. 2. Encryption on the wire
But wait! There's more!

53. The internode authentication API:
BYO identity verification

54. Looks like this:

55. 3. Authentication and Authorization

56. Best practices should not be new to you.
user segmentation
schema access limitation
etc.

57. (Everything we did with an RDBMS)
Best practices should not be new to you.
user segmentation
schema access limitation
etc.

58. Best practices should not be new to you.
user segmentation
schema access limitation
etc.
(Everything we did with an RDBMS)
New in 2.2:
Role-based access control!

59. An Example

60. An Example

61. An Example

62. An Example

63. An Example
buzzword compliant!

64. An Example

65. An Example

66. Turning it all on
authenticator: PasswordAuthenticator
Tip: keep your read-only cqlsh credentials in
$HOME/.cassandra/cqlshrc
of the system's admin account

67. Turning it all on
authorizer: CassandraAuthorizer

68. Turning it all on
role_manager: CassandraRoleManager

69. Turning it all on
authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
WARNING:
potential downtime!

70. authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
Turning it all on
WARNING:
potential downtime!
WARNING:
stupid defaults

71. authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
Turning it all on
WARNING:
potential downtime!
WARNING:
stupid defaults
TIP: turn these WAY UP:
permissions_validity_in_ms
roles_validity_in_ms
Also: use permissions_update_interval_in_ms
for async refresh if needed

72. authorizer: CassandraAuthorizer
authenticator: PasswordAuthenticator
role_manager: CassandraRoleManager
Turning it all on
WARNING:
potential downtime!
WARNING:
stupid defaults
NEW in 3.4:credentials_validity_in_ms*
* https://issues.apache.org/jira/browse/CASSANDRA-7715

73. Turning it all on
authorizer: TransitionalAuthorizer
authenticator: TransitionalAuthenticator
DSE plugins to avoid downtime

74. Turning it all on
system.schema_keyspace
system.schema_columns
system.schema_columnfamilies
system.local
system.peers
These tables have default read permissions for every
authenticated user:

75. Turning it all on
IMPORTANT cassandra.yaml line note:
"Please increase system_auth keyspace
replication factor if you use this..."
Tip: replication factor for the system_auth
keyspace should be the same as the number
of nodes in the data center

76. Turning it all on
IMPORTANT cassandra.yaml line note:
"Please increase system_auth keyspace
replication factor if you use this..."
Tip: replication factor for the system_auth
keyspace should be the same as the number
of nodes in the data center
WARNING:
stupid defaults*
*https://issues.apache.org/jira/browse/CASSANDRA-11340

77. 4. Management and tooling

78. 4. Management and tooling

79. Securing JMX

80. nmap -Pn -p7199
-oG logs/cass.gnmap 54.88.0.0/14
Always a few suckers that
TL,DR'ed

81. Why do I need to secure JMX?

83. Works as Advertised!

84. also
good for
some
LOLs

85. Securing JMX
SSL setup is like node to node and client to server
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html

86. Securing JMX
JMX Authentication is straightforward
and well documented
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html

87. Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
Now you can:
nodetool -u admin -pw secret compactionstats
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
JMX Authentication is straightforward
and well documented

88. Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
Now you can:
nodetool -u admin -pw secret compactionstats
Tip: -pwf option will read the password from a file
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
JMX Authentication is straightforward
and well documented

89. Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access
$JAVA_HOME/jre/lib/management/
jmxremote.password.template
Now you can:
nodetool -u admin -pw secret compactionstats
JMX Authentication is straightforward
and well documented
THIS JUST IN!!!
RBAC for JMX Authentication and Authorization
https://issues.apache.org/jira/browse/CASSANDRA-10091

90. Thanks!@zznate