Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cassandra-Powered Distributed DNS


Published on


Published in: Technology

Cassandra-Powered Distributed DNS

  1. 1. Highly Available DNS and Request Routing Using Apache Cassandra A Real-World Introduction to Cassandra's Data Structures + Python's pyCassa module David Strauss / Founder + CTO / Pantheon Systems
  2. 2. Why another DNS server? <ul><li>DNS servers either have no replication or require writing to a defined master server . </li><ul><li>Exceptions (Active Directory and ApacheDS) require backing DNS with a heavyweight and annoying directory service like LDAP. </li></ul><li>Critical DNS services need replication </li><ul><li> withstand DDoS attacks
  3. 3. maintain uptime when major regional links fail </li></ul><li>The zone file formats in use are awful.
  4. 4. Maintaining data persistence and replication should not be the DNS server's problem. </li></ul>
  5. 5. Why Cassandra? <ul><li>Easy cluster setup and management
  6. 6. Built-in replication and high availability
  7. 7. Multi-master: writers don't need to understand the replication topology.
  8. 8. Data model similarity to DNS
  9. 9. Eventual consistency isn't a problem
  10. 10. Not a perfect match, though: </li><ul><li>Write scalability is overkill
  11. 11. High memory requirements </li></ul></ul>
  12. 12. Demo break <ul><li>Let's set up a basic, three-node Cassandra cluster... </li></ul>
  13. 13. Creating the data model <ul><li>Think in terms of a nested dictionary.
  14. 14. Design for eventual consistency. </li><ul><li>Columns are the units (atoms) of replication.
  15. 15. Some columns may be replicated before others.
  16. 16. Column names are unique in each row or SuperC.
  17. 17. When possible, dissect objects into columns, keeping in mind that Cassandra may replicate those columns in any order. </li></ul><li>Design for common read/write patterns. </li><ul><li>The ability to arbitrarily query is limited. </li></ul></ul>
  18. 18. My initial data model <ul><li>I started with normal Column Family:
  19. 19. names (Column Family) </li><ul><li>Key: fully qualified domain name (FQDN)
  20. 20. Columns </li><ul><li>Name: Record type (A, AAAA, MX, …)
  21. 21. Value: All data (addresses, TTL, etc.) as JSON </li></ul></ul><li>Efficient for lookups for a type or ANY
  22. 22. But: All records of one type must be replaced at once. Cassandra keeps latest column written. </li><ul><li>Can't rely on reading, modifying, then writing </li></ul></ul>
  23. 23. Data Model <ul><li>Then, I dissected records into sub-columns:
  24. 24. names (Super Column Family) </li><ul><li>Key: fully qualified domain name (FQDN)
  25. 25. Super Columns </li><ul><li>Name: Record Type (A, AAAA, MX, …)
  26. 26. Sub-Columns </li><ul><li>Name: Data (e.g. IP address)
  27. 27. Value: Metadata as JSON (TTL, preference) </li></ul></ul></ul><li>Still efficient for lookups for a type or ANY
  28. 28. Using data as sub-column name results in keeping the latest metadata for any record. </li></ul>
  29. 29. Visualizing as a dictionary { ””: { ”A”: { ””: {”ttl”: 86400} ””: {”ttl”: 86400} } ”MX”: { ””: {”preference”: 10, ”ttl”: 86400} } } } Key Super Column Name Super Column Name Sub- Column Names Stored in Cassandra as a JSON-encoded sub-column value. Sub- Column Name Sub- Column Values
  30. 30. Structuring the application <ul><li> + CassandraNames </li><ul><li>DNS-centric Python API wrapping Cassandra </li></ul><li> </li><ul><li>Shell-based import tool for BIND files </li></ul><li> </li><ul><li>Python unit test to exercise the persistence </li></ul><li> + CassandraNamesResolver </li><ul><li>Twisted-based DNS server using CassandraNames </li></ul></ul>
  31. 31. Want to follow along with code? <ul><li>Setup directions: Cassandra+DNS+server+setup
  32. 32. Code on GitHub: </li></ul>
  33. 33. Demo break <ul><li>Let's clone the code down to two boxes on our demo cluster and run the test suite... </li></ul>
  34. 34. Schema setup def install_schema(drop_first=False, rf=3): keyspace_name = &quot;dns&quot; sm = pycassa.system_manager .SystemManager(&quot;;) [snip the drop_first implementation] sm.create_keyspace(keyspace_name, replication_factor=rf) sm.create_column_family(keyspace_name, &quot;names&quot;, super=True, key_validation_class= pycassa.system_manager.UTF8_TYPE, comparator_type= pycassa.system_manager.UTF8_TYPE, default_validation_class= pycassa.system_manager.UTF8_TYPE)
  35. 35. The CassandraNames class class CassandraNames: def __init__(self): self.pool = pycassa.connect(&quot;dns&quot;) [rest on upcoming slides]
  36. 36. Adding new records def insert(self, fqdn, type, data, ttl=900, preference=None): # Connect to the ColumnFamily cf = pycassa.ColumnFamily(self.pool, &quot;names&quot;) # Start the metadata with just a TTL metadata = {&quot;ttl&quot;: int(ttl)} # Add in a ”preference” if requested. if preference is not None: metadata[&quot;preference&quot;] = int(preference) # Actually perform the insertion. cf.insert(fqdn, {str(type): {data: json.dumps(metadata)}})
  37. 37. Reading records def lookup(self, fqdn, type=ANY): cf = pycassa.ColumnFamily(self.pool, &quot;names&quot;) try: columns = {} if type == ANY: # Pull all types of records. columns = dict(cf.get(fqdn)) else: # Pull only one type of record. columns = {str(type): dict(cf.get(fqdn, super_column=str(type)))} # Convert the JSON metadata into valid Python data. [snip] return decoded_columns except pycassa.cassandra.ttypes.NotFoundException: # If no records exist for the FQDN or type, # fail gracefully. pass return {}
  38. 38. Deleting records def remove(self, fqdn, type=ANY, data=None): cf = pycassa.ColumnFamily(self.pool, &quot;names&quot;) if type == ANY: # Delete all records for the FQDN. cf.remove(fqdn) elif data is None: # Delete all records of a certain type from the FQDN. cf.remove(fqdn, super_column=str(type)) else: # Delete all records for a certain type and data. cf.remove(fqdn, super_column=str(type), columns=[data])
  39. 39. Making it actually serve DNS class CassandraNamesResolver(common.ResolverBase): implements(interfaces.IResolver) def __init__(self): self.names = cassandranames.CassandraNames() common.ResolverBase.__init__(self) def _lookup(self, name, cls, type, timeout): log.msg(”Type %s records for name: %s&quot; % (type, name)) all_types = self.names.lookup(name, type) results = [] authority = [] additional = [] [continued on next slide] Python's Twisted includes a complete DNS server implementation with a pluggable resolver base (IResolver and common.ResolverBase).
  40. 40. Making it actually serve DNS def _lookup(self, name, cls, type, timeout): [function started on previous slide] for type, records in all_types.items(): for data, metadata in records.items(): if type == A: payload = dns.Record_A(data) elif type == MX: payload = dns.Record_MX( metadata[&quot;preference&quot;], data) elif type == NS: payload = dns.Record_NS(data) header = dns.RRHeader(name, type=type, payload=payload, ttl=metadata[&quot;ttl&quot;], auth=True) results.append(header) return defer.succeed((results, authority, additional))
  41. 41. Demo break <ul><li>Let's actually play with the cluster: </li><ul><li>Query the records left around by the test suite
  42. 42. Use the Python shell to manage records
  43. 43. Import a BIND zone file on one server
  44. 44. Query the imported records on a different server </li></ul></ul>
  45. 45. Next steps <ul><li>Properly firewall the cluster </li><ul><li>Cassandra needs port 7000 for replication with other cluster servers.
  46. 46. Port 53 needs to be open for DNS requests. </li></ul><li>Accelerate DNS by fronting each server with a djbdns cache
  47. 47. Finish the CNAME implementation (and other record types)
  48. 48. Consider a non-blocking library, like txCQL
  49. 49. GeoDNS using a Python GeoIP library </li></ul>
  50. 50. Conclusion <ul><li>Questions?
  51. 51. Questions for later ? </li><ul><li>I'm David Strauss (@davidstrauss) </li></ul><li>Setup directions: Cassandra+DNS+server+setup
  52. 52. Code on GitHub:
  53. 53. Pantheon Systems is hiring engineers and developers in the San Francisco Bay Area </li></ul>