An Introduction to Twisted

883 views

Published on

An introduction to Python's Twisted event-driven, networking library (PyCon 2014).

Published in: Software

An Introduction to Twisted

  1. 1. An Introduction to Twisted ! Stacey Sern shira@twistedmatrix.com @staceysern
  2. 2. Web Server twistd web --port tcp:8080 --path . (HTTP)
  3. 3. Chat Server twistd words --irc-port tcp:6667 --auth file:passwords.txt --group channel (IRC)
  4. 4. Secure Shell Server sudo twistd conch -p tcp:2222 (SSH)
  5. 5. Name Server twistd dns -p 5553 --hosts-file=hosts (DNS)
  6. 6. File Server twistd ftp --port 2121 --root . (FTP)
  7. 7. BitTorrent Client https://github.com/staceysern/bittorrent
  8. 8. Twisted An event-driven, networking engine
  9. 9. Twisted An event-driven, networking engine • Event-driven, networking engine • Event-driven programming abstractions • Networking abstractions • Low-level APIs • High-level APIs • Applications
  10. 10. Twisted An event-driven, networking engine
  11. 11. Twisted An event-driven, networking engine
  12. 12. time task 1 task 3 task 2 multi-threaded synchronous single-threaded asynchronous single-threaded synchronous event-driven
  13. 13. • Event-driven, networking engine • Event-driven programming abstractions • Networking abstractions • Low-level APIs • High-level APIs • Applications Twisted An event-driven, networking engine
  14. 14. Reactor Twisted’s event loop
  15. 15. Reactor Twisted’s event loop Interface server & client ! run() stop() callLater() server ! listenTCP() listenUDP() listenSSL() client ! connectTCP() connectSSL()
  16. 16. Reactor Code callback Application Code
  17. 17. Deferred Callback Chain Errback Chain An abstraction for managing callbacks ... Callback 2 Callback 1 Callback 3 Errback 2 ... Errback 1 Errback 3
  18. 18. Synchronous try: p = getpage_sync() except Exception: unavailable() else: t = translate(p) display(t) finally: cleanup() d = getpage_async() d.addCallbacks(translate, unavailable) d.addCallback(display) d.addBoth(cleanup) Asynchronous display cleanup unavailable cleanup translate failure from getpage_async() successful result from getpage_async()
  19. 19. cleanup() getpage_sync() translate() display() Synchronous
  20. 20. Asynchronous translate() display() cleanup()getpage_async() addCallbacks() addCallback() addBoth()
  21. 21. callback2 both4 errback1 errback3 both4 callback1 successful result of asynchronous operation failure from asynchronous operation addCallbacks(callback1, errback1) addCallback(callback2) addErrback(errback3) addBoth(both4) Deferred
  22. 22. Twisted An event-driven, networking engine • Event-driven, networking engine • Event-driven programming abstractions • Networking abstractions • Low-level APIs • High-level APIs • Applications
  23. 23. Application Transport Link Network HTTP, FTP, SMTP, POP, IMAP, DNS, IRC TCP, UDP, SSL/TLS IP Ethernet Internet Protocol Suite
  24. 24. Application Transport Link Network Client Application Transport Link Network Server 0100100011001010010111 GET /index.html HTTP/1.1
  25. 25. Application Transport Link Network Internet Protocol Transport OS Twisted
  26. 26. Protocol Represents one side of an application layer protocol which determines the format and meaning of data sent and received over a Transport Transport Represents one end of a connection between two endpoints over which data can be sent and received ProtocolFactory Used to create the appropriate Protocol when a new connection is established
  27. 27. Transport OS poll/event notification API sockets API write() writeSequence() loseConnection() connectionMade() dataReceived() connectionLost() Protocol Reactor
  28. 28. Twisted An event-driven, networking engine • Event-driven, networking engine • Event-driven programming abstractions • Networking abstractions • Low-level APIs • High-level APIs • Applications
  29. 29. from twisted.internet import protocol, reactor ! class EchoServer(protocol.Protocol): def dataReceived(self, data): self.transport.write(data) ! class EchoServerFactory(protocol.Factory): def buildProtocol(self, addr): return EchoServer() ! reactor.listenTCP(8000, EchoServerFactory()) reactor.run() Echo Server
  30. 30. from twisted.internet import protocol, reactor ! class EchoClient(protocol.Protocol): def connectionMade(self): self.transport.write(b’Hello, world!’) ! def dataReceived(self, data): print(data) self.transport.loseConnection() ! class EchoClientFactory(protocol.ClientFactory): def buildProtocol(self, addr): return EchoClient() ! reactor.connectTCP(’10.0.1.56’, 8000, EchoClientFactory()) reactor.run() Echo Client
  31. 31. Transport OS poll/event notification API sockets API write() writeSequence() loseConnection() connectionMade() dataReceived() connectionLost() Protocol Reactor
  32. 32. Twisted An event-driven, networking engine • Event-driven, networking engine • Event-driven programming abstractions • Networking abstractions • Low-level APIs • High-level APIs • Applications
  33. 33. SMTP Trying 173.194.68.26... Connected to aspmx.l.google.com. Escape character is '^]'. 220 mx.google.com ESMTP ew5si11028094qab.7 - gsmtp $ HELO 250 mx.google.com at your service MAIL FROM:<sender@example.com> 250 2.1.0 OK ew5si11028094qab.7 - gsmtp RCPT TO:<recipient@example.com> 250 2.1.5 OK ew5si11028094qab.7 - gsmtp DATA 354 Go ahead ew5si11028094qab.7 - gsmtp From: Sender <sender@example.com> To: Recipient <recipient@example.com> Subject: This is a test ! This is only a test. . 250 2.0.0 OK 1392752225 ew5si11028094qab.7 - gsmtp telnet aspmx.l.google.com 25
  34. 34. SMTPClient A Protocol which implements the client side of the SMTP protocol API getMailFrom() getMailTo() getData() sentMail()
  35. 35. class SingleMessageSender(smtp.SMTPClient): def __init__(self, from_addr, to_addr, data): smtp.SMTPClient.__init__(self, None) self.from_addr = from_addr self.to_addr = to_addr self.data = data self.done = False ! def getMailFrom(self): if not self.done: self.done = True return self.from_addr ! def getMailTo(self): return [self.to_addr] ! def getMailData(self): return self.data ! def sentMail(self, code, resp, numOk, addresses, log): pass
  36. 36. • SMTPClient • SMTPSender • SMTPSenderFactory • sendmail() • Extended SMTP (ESMTP) equivalents Twisted Mail Client Building Blocks
  37. 37. • SMTP • ESMTP • twistd mail Twisted Mail Server Building Blocks
  38. 38. Twisted An event-driven, networking engine • Event-driven, networking engine • Event-driven programming abstractions • Networking abstractions • Low-level APIs • High-level APIs • Applications
  39. 39. twistd • Starting and stopping • Logging • Daemonizing • Custom reactor • Profiling A cross-platform utility for deploying Twisted applications
  40. 40. Twisted • Event-driven, networking engine (building blocks) • Event-driven programming abstractions (Reactor, Deferred) • Networking abstractions (Transport, Protocol, ProtocolFactory) • Low-level APIs (TCP, UDP, SSL/TLS) • High-level APIs (HTTP, SMTP, FTP, SSH, IRC, DNS) • Applications (twistd) An event-driven, networking engine
  41. 41. Resources • An Introduction to Asynchronous Programming and Twisted (http://krondo.com/?p=1327) • Twisted Network Programming Essentials - Jessica McKellar & Abe Fettig • Twisted Website (https://twistedmatrix.com)

×