Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Connect Everything with NATS - Cloud Expo Europe

2,059 views

Published on

NATS is a high performance publish/subscribe messaging system that has as one of its main goals connecting services in the simplest, most secure and reliable way possible.

The project has a long history of being part of production deployments as the core component for the internal communication of a distributed system, and in its latest release NATS v2.0, its feature set was enhanced so that it can also be used to create a global and decentralized communication network.

Published in: Technology
  • My brother found Custom Writing Service ⇒ www.HelpWriting.net ⇐ and ordered a couple of works. Their customer service is outstanding, never left a query unanswered.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Paid To Facebook? Earn up to $200/day on social media sites. ▲▲▲ https://tinyurl.com/rbrfd6j
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Connect Everything with NATS - Cloud Expo Europe

  1. 1. Connect Everything with NATS Cloud Expo Europe / March 2019 Waldemar Quevedo / @wallyqs
  2. 2. About me ● Waldemar Quevedo / @wallyqs ● Software Engineer at ● NATS core maintainer ● Using NATS based systems since 2012 ● Author of Practical NATS (Apress, 2018)
  3. 3. The original book about NATS Available at: https://www.apress.com/us/book/9781484235690
  4. 4. Agenda ● Intro to the NATS project ● Show some of the latest NATS features
  5. 5. Intro to NATS
  6. 6. NATS is a simple, production proven, cloud-native messaging system.
  7. 7. About NATS ● Created by Derek Collison in 2011 for CloudFoundry ● Project known for its resiliency and great performance ● TCP/IP based with a simple plain text protocol and at- most-once delivery guarantees. | PUB | SUB | UNSUB | CONNECT | INFO | MSG | -ERR | +OK | PING | PONG |
  8. 8. CNCF Landscape Joined CNCF as an incubation project in 2018 https://landscape.cncf.io
  9. 9. CNCF Landscape Joined CNCF as an incubation project in 2018 https://landscape.cncf.io
  10. 10. CNCF Landscape https://landscape.cncf.io
  11. 11. Core of NATS: 3 Simple Patterns
  12. 12. Request/Response (1:1)
  13. 13. Publish/Subscribe (1:N) SUB foo PUB foo
  14. 14. Load Balanced Queues SUB foo workers
  15. 15. Load Balanced Queues Randomly distributed
  16. 16. Load Balanced Queues Randomly distributed
  17. 17. Load Balanced Queues Randomly distributed
  18. 18. Load Balanced Queues Randomly distributed
  19. 19. Wildcards SUB > SUB foo.bar workers PUB foo.bar
  20. 20. Wildcards SUB > PUB foo.bar SUB foo.bar workers
  21. 21. Wildcards SUB > SUB foo.bar workers PUB foo.bar
  22. 22. Wildcards SUB > PUB foo.bar SUB foo.bar workers
  23. 23. Wildcards SUB > PUB foo.bar SUB foo.bar workers
  24. 24. Wildcards SUB > SUB foo.bar workers PUB foo.bar SUB foo.*
  25. 25. Wildcards SUB > SUB foo.bar workers PUB foo.bar SUB foo.*
  26. 26. Wildcards SUB > SUB foo.bar workers PUB foo.bar SUB foo.*
  27. 27. Wildcards SUB > SUB foo.bar workers PUB foo.bar SUB foo.*
  28. 28. Do not assume the audience! SUB > SUB foo.bar workers PUB foo.bar SUB foo.*
  29. 29. Simplicity: The Road Not Taken Image: https://www.theatlantic.com/video/index/555959/robert-frost-road-not-taken/
  30. 30. Simplicity: The Road Not Taken http://www.curry-on.org/2018/sessions/simplicity-the-road-not-taken.html
  31. 31. Conventional wisdom might be off sometimes... ‘How HTTP 2 nearly derailed our latency initiatives’ - Altitude San Francisco 2018: HTTP/2 Tales: Discovery and Woe https://www.slideshare.net/Fastly/altitude-san- francisco-2018-http2-tales-discovery-and-woe
  32. 32. Things evolving fast but getting too complex? ‘Yes! We're replaying hits from the 1990s and 2000s (and adding some new things)’ QUIC Tutorial A New Internet Transport https://datatracker.ietf.org/meeting/98/materials/slides- 98-edu-sessf-quic-tutorial-00.pdf
  33. 33. Things evolving fast but getting too complex? QUIC ... deeply integrates aspects of TCP, TLS and HTTP/2 into one cross-layer implementation over UDP. However, such ambition comes at the cost of high complexity, which in turn leads to misinterpretations, bugs and unwanted behaviour in implementations. This was also witnessed in the recently standardized HTTP/2 protocol. https://quic.edm.uhasselt.be/files/QUIC_debuggability_C ameraReadyFinal_18oct2018.pdf
  34. 34. Goal of NATS: Be the simplest, fastest, most secure and reliable way to communicate among services.
  35. 35. NATS as an always available dial tone to connect everything
  36. 36. The NATS Server
  37. 37. The NATS Server (aka. core NATS) ● Written in Go a. https://github.com/nats-io/gnatsd ● Extremely high performance ● Full-mesh one hop clustering for HA ● Auto discovery via gossip ● Auto prune of interest ● TLS support ● Authorization and Authentication
  38. 38. The NATS Server NATS Client NATS NATSNATS NATS Client
  39. 39. The NATS Server NATS Client NATS NATSNATS NATS Client
  40. 40. The NATS Server NATS Client NATS NATSNATS NATS Client
  41. 41. The NATS Server NATS Client NATS NATSNATS NATS Client
  42. 42. NATS Streaming
  43. 43. NATS Streaming (aka. STAN) ● Supports at-least-once delivery guarantees a. https://github.com/nats-io/nats-streaming-server ● Persistence of messages / ‘Message replay’ ● Raft based replicated log for clustering ● Protocol based on NATS Request/Reply
  44. 44. NATS Streaming (aka. STAN) NATS Streaming Client NATS NATSNATS NATS Streaming Server ‘cluster-A’ NATS Streaming Server ‘cluster-A’ NATS Streaming Server ‘cluster-A’
  45. 45. Edge NATS v2
  46. 46. What’s in edge NATS v2 ● NATS v2 is a very special release of the server. ● Expands the security and project capabilities of the server to become a core component used to build a global communication network.
  47. 47. What’s in edge NATS v2 ● Gateways & Super clusters ● New clustering protocol ● Accounts isolation ○ Containers for messaging ● NKEYS (ed25519 based keys) ● Decentralized authorization with JWTs ● System Accounts ● Graceful shutdown
  48. 48. Super Clusters
  49. 49. Services & Streams Service definitions are a secure RPC endpoint ✓ Export a service to allow other accounts to import ✓ Import a service to allow requests to be sent and securely, seamlessly, and anonymously to another account ✓ Usage include monitoring probes, certificate generation services, secure vault, geolocation Stream definitions allow data flow between accounts ✓ Export a stream to allow egress ✓ Import a stream to allow ingress ✓ Use cases include stock quotes, weather, Twitter feeds, Slack, global alerts Zero client configuration or client API changes!
  50. 50. Accounts Example accounts { synadia { users = [ {user: nats, password: $2a$10$BYItxVAGPCbHakeKXegN7uGNJQB45p5sQT4D5Jrlb/gOI13Orx.RK} {nkey: UC53TQCCXLUYSYTJ7PHSHDAORV6OSON7SNZQAWVMJUGM5JC3GR2AAD2M} ] # For sharing streams and services with others. exports = [ # Network status updates available for anyone. {stream: "cloud.network.status"} # Service to request developer statistics {service: "private.devstats", accounts: [CNCF]} ] } }
  51. 51. Accounts Example accounts { synadia { users = [ {user: nats, password: $2a$10$BYItxVAGPCbHakeKXegN7uGNJQB45p5sQT4D5Jrlb/gOI13Orx.RK} {nkey: UC53TQCCXLUYSYTJ7PHSHDAORV6OSON7SNZQAWVMJUGM5JC3GR2AAD2M} ] # For sharing streams and services with others. exports = [ # Network status updates available for anyone. {stream: "cloud.network.status"} # Service to request developer statistics {service: "private.devstats", accounts: [CNCF]} ] } }
  52. 52. JWTs JWTs are used to represent identities in NATS ● User, Account, Cluster, or Server User JWTs Contain ● Account NKey (Issuer) ● Public NKey (Subject) ● Friendly Name ● Permissions ● Limits ● Not Before and Expiration
  53. 53. JWTs { "jti": "3Y2OIRCSQLHOZI2KWXPS7JCRIR5BT5ZGZ5G74VHFCMUJAZUPCYCA", "iat": 1544140248, "iss": "ADQO262SKHLYIQTIBU3VG2K4GWRVO4TXYYJDHKI7QBMWYW6HACLQZIVB", "name": "Waldemar", "sub": "UCZRG6WDXWMIKDPLUMMRS2UAO2NSA5GOU2WCTXQLK7TRUWLLQ2CAXY7M", "type": "user", "nats": { "pub": { "allow": [ "public.>" ] }, "sub": { "deny": [ "private.>" ] } }
  54. 54. NGS: A NATS based global communication network https://synadia.com/ng
  55. 55. NGS: A NATS based global communication network telnet connect.ngs.global 4222 Trying 34.241.195.68... Connected to euwest1.aws.ngs.global. Escape character is '^]'. INFO {"server_id":"NBZCGYFTHLCMZC5FWCJXQHVK7VUXSDRYEALBY6GFYCGEBZHGJLWHZR7J", "version":"2.0.0-RC5","nonce":"kBpF7HCm6rb_TW0", "cluster":"aws- euwest1","connect_urls":["34.241.195.68:4222","34.243.238.95:4222","34.249.187.230:4222"]}
  56. 56. NGS: A NATS based global communication network telnet connect.ngs.global 4222 Trying 34.241.195.68... Connected to euwest1.aws.ngs.global. Escape character is '^]'. INFO {"server_id":"NBZCGYFTHLCMZC5FWCJXQHVK7VUXSDRYEALBY6GFYCGEBZHGJLWHZR7J", "version":"2.0.0-RC5","nonce":"kBpF7HCm6rb_TW0", "cluster":"aws- euwest1","connect_urls":["34.241.195.68:4222","34.243.238.95:4222","34.249.187.230:4222"]}
  57. 57. Gathering all server metrics opts := []nats.Option{nats.Name("NGS Tools - PING")} opts = append(opts, nats.UserCredentials("./ngs-system.jwt", "./ngs-system.nkey")) ncs, err := nats.Connect(*urls, opts...) if err != nil { log.Fatal(err) } // Replies for ping responses. reply := ncs.NewRespInbox() if _, err := ncs.Subscribe(reply, processServerUpdate); err != nil { log.Fatal(err) } // Send our ping for statusz updates if err := ncs.PublishRequest("$SYS.REQ.SERVER.PING", reply, nil); err != nil { log.Fatal(err) }
  58. 58. Roadmap ● Other Messaging Project Integrations ● Augmenting other messaging systems with NATS ● Jetstream (NATS Streaming V2) ● Native MQTT support ● Websocket Support ● Microcontroller Clients for IoT
  59. 59. Summary ● Consider NATS if: ○ Require low operational overhead ○ Looking for low latency RPC solution ○ Not constrained to use HTTP for communicating ○ Fan of simple solutions that just work
  60. 60. Questions?
  61. 61. Thanks! github.com/nats-io / @nats_io https://nats.io

×