Marrow: A Meta-Framework for Python 2.6+ and 3.1+
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

Marrow: A Meta-Framework for Python 2.6+ and 3.1+

  • 2,898 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,898
On Slideshare
2,898
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
15
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MarrowMeta–Framework for Python 2.6+ and 3.1+ Alice Zoë Bevan–McGregor
  • 2. Overview
  • 3. ConfigurationYAML-Based Application Configuration
  • 4. Introspective Scripting Non-Imperative Command-Line Parsing
  • 5. Blueprint Template–Derived Directory TreesInteractive & Command–Line Interrogation
  • 6. Streaming Templates A Python Micro–Language
  • 7. Server InterfaceModified Tornado IOLoop and IOStream Server and Protocol Wrappers
  • 8. HTTP/1.1 WSGI 2 Server Highly Performant Pure Python HTTP/1.1 Implementation
  • 9. Object Wrappers PEP 444 Request / Response ObjectsHTTP Status Code Exception Applications
  • 10. Middleware / Filters Compression, Sessions, etc.
  • 11. Performance & Optimizations Time for Timeit
  • 12. Compatibility Python 2.6+ and 3.1+
  • 13. Configuration
  • 14. marrow.config
  • 15. Unfortunately…
  • 16. Least Developed (So far.)
  • 17. Paste Deploy
  • 18. 1 [server] 2 use = marrow.server.http:HTTPServer 3 host = 127.0.0.1, ::1 4 port = 8080, 8088 5 6 [mapping] 7 / = root 8 9 [app:root]10 use = marrow.server.http.testing:hello11 name = ConFoo
  • 19. INI = Evil
  • 20. Typecasting
  • 21. 1 [server] 2 use = marrow.server.http:HTTPServer 3 host = 127.0.0.1, ::1 4 port = 8080, 8088 5 6 [mapping] 7 / = root 8 9 [app:root]10 use = marrow.server.http.testing:hello11 name = ConFoo
  • 22. String to List
  • 23. String to Integer
  • 24. YAML to the Rescue
  • 25. 1 version: 1 2 3 server: 4 use: marrow.server.http:HTTPServer 5 host: ["127.0.0.1", "::1"] 6 port: [8080, 8088] 7 8 mapping: 9 /: *root1011 root: &root12 use: marrow.server.http.testing:hello13 name: ConFoo
  • 26. References
  • 27. 1 version: 1 2 3 server: 4 use: marrow.server.http:HTTPServer 5 host: ["127.0.0.1", "::1"] 6 port: [8080, 8088] 7 8 mapping: 9 /: *root1011 root: &root12 use: marrow.server.http.testing:hello13 name: ConFoo
  • 28. Direct Object Access (Entry points are for chumps.)
  • 29. 1 version: 1 2 3 server: 4 use: marrow.server.http:HTTPServer 5 host: ["127.0.0.1", "::1"] 6 port: [8080, 8088] 7 8 mapping: 9 /: *root1011 root: &root12 use: marrow.server.http.testing:hello13 name: ConFoo
  • 30. Logging
  • 31. 16 logging:17 formatters:18 brief:19 format: %(levelname)-8s: %(name)-15s: %(message)s20 handlers:21 console:22 class: logging.StreamHandler23 formatter: brief24 level: INFO25 stream: ext://sys.stdout26 loggers:27 foo:28 level: ERROR29 handlers: [console]30 root:31 level: DEBUG32 handlers: [console]
  • 32. Scripting
  • 33. marrow.script
  • 34. Existing Solutions
  • 35. sys.argv (painful)
  • 36. sys.argv (inconsistent)
  • 37. getopt(old–school)
  • 38. optparse (old–school)
  • 39. optparse (deprecated)
  • 40. argparse(new old–school)
  • 41. Paste Script (fancy)
  • 42. Paste Script (entry point magic)
  • 43. Paste Script (paster <name> […])
  • 44. Paste Script (context–aware)
  • 45. Commonality?
  • 46. Commonality? (un–Pythonic…)
  • 47. Commonality?(…hideous, hideous, imperative parser objects…)
  • 48. 1 import optparse 2 3 if __name__=="__main__": 4 parser = optparse.OptionParser("usage: %prog [options] arg1 arg2") 5 parser.add_option("-H", "--host", dest="hostname", 6 default="127.0.0.1", type="string", 7 help="specify hostname to run on") 8 parser.add_option("-p", "--port", dest="portnum", 9 default=80, type="int",10 help="port number to run on")11 (options, args) = parser.parse_args()12 if len(args) != 2:13 parser.error("incorrect number of arguments")14 hostname = options.hostname15 portnum = options.portnum
  • 49. 1 import argparse 2 3 parser = argparse.ArgumentParser(description=Process some integers.) 4 parser.add_argument(integers, metavar=N, type=int, nargs=+, 5 help=an integer for the accumulator) 6 parser.add_argument(--sum, dest=accumulate, action=store_const, 7 const=sum, default=max, 8 help=sum the integers (default: find the max)) 910 args = parser.parse_args()11 print(args.accumulate(args.integers))
  • 50. Most needed?
  • 51. Simplicity
  • 52. Arguments ➢ Variables
  • 53. 1 # encoding: utf-823 def ultima(required, value=None, name="world", switch=False, age=18, *args, **kw):4 print "Hello %s!" % (name, )567 if __name__ == __main__:8 __import__(marrow.script).script.execute(ultima)
  • 54. Usage: ultima.py [OPTIONS] [--name=value...] <required> [value...]OPTIONS may be one or more of:-a, --age=VAL Override this value. Default: 18-h, --help Display this help and exit.-n, --name=VAL Override this value. Default: world-s, --switch Toggle this value. Default: False-v, --value=VAL Set this value.
  • 55. Additional Detail
  • 56. __doc__ = Help Text
  • 57. Decorators
  • 58. @annotateArgument Typecasting & Validation Callbacks
  • 59. @describe Help Text
  • 60. @shortArgument Abbreviations
  • 61. @callbacks Simple Callbacks
  • 62. optparse Example
  • 63. 1 # encoding: utf-8 2 3 import marrow.script 4 5 6 @marrow.script.describe( 7 host = "specify a hostname to run on", 8 port = "port number to run on" 9 )10 def serve(arg1, arg2, host="127.0.0.1", port=80):11 pass121314 if __name__ == __main__:15 marrow.script.execute(serve)
  • 64. Sub–Commands
  • 65. Method = Command
  • 66. __init__ + method
  • 67. Blueprint
  • 68. marrow.blueprint
  • 69. Paste Script
  • 70. Á La Carte Templates
  • 71. Best way to describe it…
  • 72. 1 class PackageBlueprint(Blueprint): 2 """Create an installable Python package.""" 3 4 base = marrow.blueprint.package/files 5 engine = mako 6 7 settings = [ 8 Setting( 9 name,10 "Project Name",11 "The name to appear on the Python Package Index, e.g. CluComp.",12 required=True13 ),14 Setting(15 package,16 "Package Name",17 "The name of the Python package, periods indicating namespaces, e.g. clueless.compiler.",18 required=True19 ),20 # ...21 ]2223 manifest = [24 # ...25 File(setup.py),26 File(setup.cfg),27 Folder(tests, children=[28 File(.keep, keep)29 ]),30 package31 ]
  • 73. 1 def package(settings): 2 def recurse(name): 3 head, _, tail = name.partition(.) 4 5 return [ 6 Folder(head, children=[ 7 File(__init__.py, namespace.py if tail else init.py) 8 ] + (recurse(tail) if tail else [])) 9 ]1011 return recurse(settings.package)
  • 74. class Setting
  • 75. target
  • 76. title
  • 77. help
  • 78. required
  • 79. validator
  • 80. condition
  • 81. values
  • 82. default
  • 83. cast
  • 84. hidden
  • 85. class File
  • 86. target
  • 87. source
  • 88. condition
  • 89. data
  • 90. class Folder
  • 91. ≈ File - data
  • 92. Class Inheritance
  • 93. pre/post Callbacks
  • 94. Interactive Questioning
  • 95. Command–Line Answers (marrow.script + **kw ;)
  • 96. Templating
  • 97. marrow.tags
  • 98. Streaming
  • 99. yield
  • 100. Enter / Exit
  • 101. Text / Flush
  • 102. HTML5
  • 103. High–Level
  • 104. Widgets
  • 105. Python ±
  • 106. 1 #!/usr/bin/env python 2 # encoding: utf-8 3 4 from __future__ import unicode_literals 5 6 from marrow.tags.html5 import * 7 8 from master import SITE_NAME, site_header, site_footer 91011 def welcome():12 return html [13 head [ title [ Welcome!, — , SITE_NAME ] ],14 flush, # allow the browser to start downloading static resources early15 body ( class_ = "nav-welcome" ) [16 site_header,17 p [18 Lorem ipsum dolor sit amet, consectetur adipisicing elit…19 ],20 site_footer21 ]22 ]232425 if __name__ == __main__:26 with open(welcome.html, w) as fh:27 for i in welcome().render(utf8):28 fh.write(i)
  • 107. 1 login = Form(sign-in, class_="tabbed", action=/users/action:authenticate, children=[ 2 HiddenField(referrer), 3 FieldSet(local, "Local Users", TableLayout, [ 4 TextField(identity, "User Name", autofocus=True), 5 PasswordField(password, "Password") 6 ]), 7 FieldSet(yubikey, "Yubikey Users", TableLayout, [ 8 TextField(identity, "User Name"), 9 PasswordField(password, "Password"),10 PasswordField(yubikey, "Yubikey")11 ]),12 FieldSet(openid, "OpenID Users", TableLayout, [13 URLField(url, "OpenID URL")14 ])15 ], footer=SubmitFooter(form, "Sign In"))
  • 108. Guts
  • 109. 1 class Tag(Fragment): 2 def __call__(self, data_=None, strip=NoDefault, *args, **kw): 3 self = deepcopy(self) 4 5 self.data = data_ 6 if strip is not NoDefault: self.strip = strip 7 self.args.extend(list(args)) 8 self.attrs.update(kw) 910 return self
  • 110. 12 def __getitem__(self, k):13 if not k: return self1415 self = deepcopy(self)1617 if not isinstance(k, (tuple, list)):18 k = [k]1920 for fragment in k:21 if isinstance(fragment, basestring):22 self.children.append(escape(fragment))23 continue2425 self.children.append(fragment)2627 return self
  • 111. 29 def __unicode__(self):30 """Return a serialized version of this tree/branch."""31 return .join(self.render(utf8)).decode(utf8)3233 def enter(self):34 if self.strip:35 raise StopIteration()3637 if self.prefix:38 yield self.prefix3940 yield u< + self.name + u.join([attr for attr in quoteattrs(self, self.attrs)]) + u>4142 def exit(self):43 if self.simple or self.strip:44 raise StopIteration()4546 yield u</ + self.name + u>
  • 112. 48 def render(self, encoding=ascii):49 # ...5051 for k, t in self:52 if k == enter:53 # ...54 continue5556 if k == exit:57 # ...58 continue5960 if k == text:61 # ...62 continue6364 if k == flush:65 yield buf.getvalue()66 del buf67 buf = IO()6869 yield buf.getvalue()
  • 113. 71 def __iter__(self):72 yield enter, self7374 for child in self.children:75 if isinstance(child, Fragment):76 for element in child:77 yield element78 continue7980 if hasattr(child, __call__):81 value = child(self)8283 if isinstance(value, basestring):84 yield text, unicode(value)85 continue8687 for element in child(self):88 yield element8990 continue9192 yield text, unicode(child)9394 yield exit, self
  • 114. 29 def __unicode__(self): 30 """Return a serialized version of this tree/branch.""" 31 return .join(self.render(utf8)).decode(utf8) 96 def clear(self): 97 self.children = list() 98 self.args = list() 99 self.attrs = dict()100100 def empty(self):101 self.children = list()
  • 115. Server Interface
  • 116. Asynchronous IO
  • 117. Callbacks
  • 118. Low–Level
  • 119. marrow.io
  • 120. Py3K Tornado + Patches IOLoop + IOStream
  • 121. Apache License
  • 122. 1 # encoding: utf-8 2 3 """An example raw IOLoop/IOStream example. 4 5 Taken from http://nichol.as/asynchronous-servers-in-python by Nicholas Piël. 6 """ 7 8 import errno, functools, socket 910 from marrow.util.compat import exception11 from marrow.io import ioloop, iostream1213 def connection_ready(sock, fd, events):14 while True:15 connection, address = sock.accept()1617 connection.setblocking(0)18 stream = iostream.IOStream(connection)19 stream.write(b"HTTP/1.0 200 OKrnContent-Length: 5rnrnPong!rn", stream.close)2021 if __name__ == __main__:22 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)23 sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)24 sock.setblocking(0)25 sock.bind(("", 8010))26 sock.listen(5000)2728 io_loop = ioloop.IOLoop.instance()29 io_loop.set_blocking_log_threshold(2)30 callback = functools.partial(connection_ready, sock)31 io_loop.add_handler(sock.fileno(), callback, io_loop.READ)3233 try:34 io_loop.start()35 except KeyboardInterrupt:36 io_loop.stop()
  • 123. 1 # encoding: utf-8 2 3 """An example raw IOLoop/IOStream example. 4 5 Taken from http://nichol.as/asynchronous-servers-in-python by Nicholas Piël. 6 """ 7 8 import errno, functools, socket 910 from marrow.util.compat import exception11 from marrow.io import ioloop, iostream1213 def connection_ready(sock, fd, events):14 while True:15 connection, address = sock.accept()1617 connection.setblocking(0)18 stream = iostream.IOStream(connection)19 stream.write(b"HTTP/1.0 200 OKrnContent-Length: 5rnrnPong!rn", stream.close)2021 if __name__ == __main__:22 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)23 sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)24 sock.setblocking(0)25 sock.bind(("", 8010))26 sock.listen(5000)2728 io_loop = ioloop.IOLoop.instance()29 io_loop.set_blocking_log_threshold(2)30 callback = functools.partial(connection_ready, sock)31 io_loop.add_handler(sock.fileno(), callback, io_loop.READ)3233 try:34 io_loop.start()35 except KeyboardInterrupt:36 io_loop.stop()
  • 124. 1 # encoding: utf-8 2 3 """An example raw IOLoop/IOStream example. 4 5 Taken from http://nichol.as/asynchronous-servers-in-python by Nicholas Piël. 6 """ 7 8 import errno, functools, socket 910 from marrow.util.compat import exception11 from marrow.io import ioloop, iostream1213 def connection_ready(sock, fd, events):14 while True:15 connection, address = sock.accept()1617 connection.setblocking(0)18 stream = iostream.IOStream(connection)19 stream.write(b"HTTP/1.0 200 OKrnContent-Length: 5rnrnPong!rn", stream.close)2021 if __name__ == __main__:22 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)23 sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)24 sock.setblocking(0)25 sock.bind(("", 8010))26 sock.listen(5000)2728 io_loop = ioloop.IOLoop.instance()29 io_loop.set_blocking_log_threshold(2)30 callback = functools.partial(connection_ready, sock)31 io_loop.add_handler(sock.fileno(), callback, io_loop.READ)3233 try:34 io_loop.start()35 except KeyboardInterrupt:36 io_loop.stop()
  • 125. 1 # encoding: utf-8 2 3 """An example raw IOLoop/IOStream example. 4 5 Taken from http://nichol.as/asynchronous-servers-in-python by Nicholas Piël. 6 """ 7 8 import errno, functools, socket 910 from marrow.util.compat import exception11 from marrow.io import ioloop, iostream1213 def connection_ready(sock, fd, events):14 while True:15 connection, address = sock.accept()1617 connection.setblocking(0)18 stream = iostream.IOStream(connection)19 stream.write(b"HTTP/1.0 200 OKrnContent-Length: 5rnrnPong!rn", stream.close)2021 if __name__ == __main__:22 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)23 sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)24 sock.setblocking(0)25 sock.bind(("", 8010))26 sock.listen(5000)2728 io_loop = ioloop.IOLoop.instance()29 io_loop.set_blocking_log_threshold(2)30 callback = functools.partial(connection_ready, sock)31 io_loop.add_handler(sock.fileno(), callback, io_loop.READ)3233 try:34 io_loop.start()35 except KeyboardInterrupt:36 io_loop.stop()
  • 126. 1 # encoding: utf-8 2 3 """An example raw IOLoop/IOStream example. 4 5 Taken from http://nichol.as/asynchronous-servers-in-python by Nicholas Piël. 6 """ 7 8 import errno, functools, socket 910 from marrow.util.compat import exception11 from marrow.io import ioloop, iostream1213 def connection_ready(sock, fd, events):14 while True:15 connection, address = sock.accept()1617 connection.setblocking(0)18 stream = iostream.IOStream(connection)19 stream.write(b"HTTP/1.0 200 OKrnContent-Length: 5rnrnPong!rn", stream.close)2021 if __name__ == __main__:22 sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)23 sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)24 sock.setblocking(0)25 sock.bind(("", 8010))26 sock.listen(5000)2728 io_loop = ioloop.IOLoop.instance()29 io_loop.set_blocking_log_threshold(2)30 callback = functools.partial(connection_ready, sock)31 io_loop.add_handler(sock.fileno(), callback, io_loop.READ)3233 try:34 io_loop.start()35 except KeyboardInterrupt:36 io_loop.stop()
  • 127. Not fun!
  • 128. High–Level
  • 129. marrow.server
  • 130. 1 # encoding: utf-8 2 3 """A simplified version of the raw example.""" 4 5 6 from marrow.server.base import Server 7 from marrow.server.protocol import Protocol 8 91011 class HTTPResponse(Protocol):12 def accept(self, client):13 client.write( b"HTTP/1.0 200 OKrnContent-Length: 7rnrnPong!rn", client.close)141516 if __name__ == __main__:17 Server(None, 8010, HTTPResponse).start()
  • 131. functools.partial (Pass the client object to your callbacks.)
  • 132. Your BFF
  • 133. Single–Thread Async
  • 134. Futures–Based Threading
  • 135. Multi–Process (incl. processor detection)
  • 136. r".+"
  • 137. PEP 444
  • 138. Warning!
  • 139. PEP 444 Hurts babies!
  • 140. PEP 444Is highly addictive?
  • 141. PEP 444Is a draft of one possible WSGI 2 solution.
  • 142. WSGI
  • 143. Complete Rewrite
  • 144. Simplified
  • 145. Consistent
  • 146. 1 def hello(environ):2 return b200 OK, [3 (bContent-Type, btext/plain)],4 (bContent-Length, b12)5 ], b"Hello world!"
  • 147. Distinctions
  • 148. Byte String
  • 149. Unicode String
  • 150. Native String
  • 151. RFC–Style
  • 152. Applications…
  • 153. An application is any function, method, or instance with a __call__ method. Applications must: 1. Be able to be invoked more than once. If this can not be guaranteed by the application implementation, it must be wrapped in a function that creates a new instance on each call. 2. Accept a single positional argument which must be an instance of a base Python dictionary containing what is referred to as the WSGI environment. The contents of this dictionary are fully described in the WSGI Environment section. 3. Return a 3-tuple of (status, headers, body) where: 1.status must contain the HTTP status code and reason phrase of the response. The status code and reason must be present, in that order, separated by a single space. (See RFC 2616, Section 6.1.1 for more information.) 2.headers must be a standard Python list containing 2-tuples of (name, value) pairs representing the HTTP headers of the response. Each header name must represent a valid HTTP header field name (as defined by RFC 2616, Section 4.2) without trailing colon or other punctuation. 3.body must be an iterable representing the HTTP response body. 4.status and the name of each header present in headers must not have leading or trailing whitespace. 5.status, and the contents of headers (both name and value) must not contain control characters including carriage returns or linefeeds. 6.status, headers, and the chunks yielded by the body iterator should be returned as byte strings, though for implementations where native strings are unicode, native strings may be returned. The server must encode unicode values using ISO-8859-1. 7. The amount of data yielded by the body iterable must not exceed the length specified by the Content-Length response header, if defined. 8. The body iterable may be a native string, instead of a native string wrapped in a list for the simple case, but this is not recommended.Additionally, applications and middleware must not alter HTTP 1.1 "hop-by-hop" features or headers, any equivalent features in HTTP 1.0, orany headers that would affect the persistence of the clients connection to the web server. Applications and middleware may, however,interrogate the environment for their presence and value. These features are the exclusive province of the server, and a server should considerit a fatal error for an application to attempt sending them, and raise an error if they are supplied as return values from an application in theheaders structure.
  • 154. Servers…
  • 155. A WSGI 2 server must: 1. Invoke the application callable once for each request it receives from an HTTP client that is directed at the application. 2. Pass a single positional value to the application callable representing the request environment, described in detail in the WSGI Environment section. 3. Ensure that correct response headers are sent to the client. If the application omits a header required by the HTTP standard (or other relevant specifications that are in effect), the server must add it. E.g. the Date and Server headers. 1. The server must not override values with the same name if they are emitted by the application. 4. Raise an exception if the application attempts to set HTTP 1.1 "hop-by-hop" or persistence headers, or equivalent headers in HTTP 1.0, as described above. 5. Encode unicode data (where returned by the application) using ISO-8859-1. 6. Ensure that line endings within the body are not altered. 7. Transmit body chunks to the client in an unbuffered fashion, completing the transmission of each set of bytes before requesting another one. (Applications should perform their own buffering.) 8. Call the close() method of the body returned by the application, if present, upon completion of the current request. This should be called regardless of the termination status of the request. This is to support resource release by the application and is intended to complement PEP 325s generator support, and other common iterables with close() methods. 9. Support the HTTP 1.1 specification where such support is made possible by the underlying transport channel, including full URL REQUEST_URI, pipelining of requests, chunked transfer, and any other HTTP 1.1 features mandated in the relevant RFC.Additionally, 1. HTTP header names are case-insensitive, so be sure to take that into consideration when examining application-supplied headers. 2. The server may apply HTTP transfer encodings or perform other transformations for the purpose of implementing HTTP features such as chunked transfer. 3. The server must not attempt to handle byte range requests; the application can optimize this use case far more easily than a server. (For example an application can generate the correct body length vs. generating the whole body and having the server buffer and slice it.) 4. Servers must not directly use any other attributes of the body iterable returned by the application.
  • 156. More Demanding
  • 157. (Optional = Never)
  • 158. HTTP/1.1
  • 159. Chunked Encoding
  • 160. (Request)
  • 161. (Response)
  • 162. Expect/Continue
  • 163. Pipelining / Keep–Alive
  • 164. HTTP/1.1 Server
  • 165. 4.5KR/sec(Single process, single thread.)
  • 166. C10K(4 processes, single thread.)
  • 167. 10KR/sec(4 process, single thread, lower concurrency.)
  • 168. Pure Python!
  • 169. (171 Opcodes)
  • 170. PEP 444
  • 171. Async
  • 172. Threading
  • 173. Futures!
  • 174. Multi–Process
  • 175. Explicit
  • 176. Processor Detection
  • 177. Request Cycle
  • 178. Socket Accept
  • 179. Protocol .accept()
  • 180. Read Headers
  • 181. Pre–Buffer Body
  • 182. Dispatch to Worker
  • 183. Emit Status & Headers
  • 184. Stream Body
  • 185. Keep–Alive
  • 186. wsgi.errors ➢ logging
  • 187. Object Wrappers
  • 188. marrow.wsgi.objects
  • 189. Request / Response
  • 190. Exceptions
  • 191. WebOb
  • 192. 1 #!/usr/bin/env python 2 # encoding: utf-8 3 4 from __future__ import unicode_literals 5 6 from pprint import pformat 7 8 from marrow.server.http import HTTPServer 9 from marrow.wsgi.objects.decorator import wsgify101112 @wsgify13 def hello(request):14 resp = request.response15 resp.mime = "text/plain"16 resp.body = "%rnn%snn%s" % (request, request, pformat(request.__dict__))171819 if __name__ == __main__:20 import logging21 logging.basicConfig(level=logging.DEBUG)2223 HTTPServer(None, 8080, application=hello).start()
  • 193. Request
  • 194. Dict Proxy
  • 195. WSGI Environment
  • 196. Singleton
  • 197. Accessor Objects
  • 198. 24 class Request(object):28 body = RequestBody(wsgi.input)29 length = Int(CONTENT_LENGTH, None, rfc=14.13)30 mime = ContentType(CONTENT_TYPE, None)31 charset = Charset(CONTENT_TYPE)
  • 199. 102 def __getitem__(self, name):103 return self.environ[name]104105 def __setitem__(self, name, value):106 self.environ[name] = value107108 def __delitem__(self, name):109 del self.environ[name]
  • 200. 1 class ReaderWriter(object): 2 default = NoDefault 3 rw = True 4 5 def __init__(self, header, default=NoDefault, rw=NoDefault, rfc=None): 6 pass # save arguments 7 8 def __get__(self, obj, cls): 9 try:10 return obj[self.header]1112 except KeyError:13 pass1415 if self.default is not NoDefault:16 if hasattr(self.default, __call__):17 return self.default(obj)1819 return self.default2021 raise AttributeError(WSGI environment does not contain %s key. % (self.header, ))2223 def __set__(self, obj, value):24 if not self.rw:25 raise AttributeError(%s is a read-only value. % (self.header, ))2627 if value is None:28 del obj[self.header]29 return3031 obj[self.header] = value3233 def __delete__(self, obj):34 if not self.rw:35 raise AttributeError(%s is a read-only value. % (self.header, ))3637 del obj[self.header]
  • 201. Filtering
  • 202. Ingress
  • 203. Egress
  • 204. “Light-Weight Middleware”
  • 205. 38 class CompressionFilter(object):39 def __init__(self, level=6):40 self.level = level4142 super(CompressionFilter, self).__init__()4344 def __call__(self, request, status, headers, body):45 """Compress, if able, the response.4647 This has the side effect that if your application does not declare a content-length, this filter will.48 """4950 # TODO: Remove some of this debug logging; itll slow things down and isnt really needed.5152 if request.get(wsgi.compression, True) is False:53 log.debug("Bypassing compression at applications request.")54 return status, headers, body5556 if request.get(wsgi.async) and hasattr(body, __call__):57 log.debug("Can not compress async responses, returning original response.")58 return status, headers, body5960 if bgzip not in request.get(HTTP_ACCEPT_ENCODING, b):61 log.debug("Browser support for GZip encoding not found, returning original response.")62 return status, headers, body
  • 206. 38 class CompressionFilter(object):39 def __init__(self, level=6):40 self.level = level4142 super(CompressionFilter, self).__init__()4344 def __call__(self, request, status, headers, body):45 """Compress, if able, the response.4647 This has the side effect that if your application does not declare a content-length, this filter will.48 """4950 # TODO: Remove some of this debug logging; itll slow things down and isnt really needed.5152 if request.get(wsgi.compression, True) is False:53 log.debug("Bypassing compression at applications request.")54 return status, headers, body5556 if request.get(wsgi.async) and hasattr(body, __call__):57 log.debug("Can not compress async responses, returning original response.")58 return status, headers, body5960 if bgzip not in request.get(HTTP_ACCEPT_ENCODING, b):61 log.debug("Browser support for GZip encoding not found, returning original response.")62 return status, headers, body
  • 207. 38 class CompressionFilter(object):39 def __init__(self, level=6):40 self.level = level4142 super(CompressionFilter, self).__init__()4344 def __call__(self, request, status, headers, body):45 """Compress, if able, the response.4647 This has the side effect that if your application does not declare a content-length, this filter will.48 """4950 # TODO: Remove some of this debug logging; itll slow things down and isnt really needed.5152 if request.get(wsgi.compression, True) is False:53 log.debug("Bypassing compression at applications request.")54 return status, headers, body5556 if request.get(wsgi.async) and hasattr(body, __call__):57 log.debug("Can not compress async responses, returning original response.")58 return status, headers, body5960 if bgzip not in request.get(HTTP_ACCEPT_ENCODING, b):61 log.debug("Browser support for GZip encoding not found, returning original response.")62 return status, headers, body
  • 208. Exit Early
  • 209. Stream Process
  • 210. Flat Stack
  • 211. Performance & Optimization
  • 212. timeit FTW
  • 213. s="Content-Type: text/htmlrn" ^
  • 214. Split or Partition?
  • 215. a,b = s.split(":", 1)
  • 216. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909
  • 217. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909 a,b = s.split(":")
  • 218. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909 a,b = s.split(":")• Python 2.7: 0.631 • Python 3.1: 0.837
  • 219. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909 a,b = s.split(":")• Python 2.7: 0.631 • Python 3.1: 0.837 a,c = s.partition(":")[::2]
  • 220. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909 a,b = s.split(":")• Python 2.7: 0.631 • Python 3.1: 0.837 a,c = s.partition(":")[::2]• Python 2.7: 0.642 • Python 3.1: 0.690
  • 221. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909 a,b = s.split(":")• Python 2.7: 0.631 • Python 3.1: 0.837 a,c = s.partition(":")[::2]• Python 2.7: 0.642 • Python 3.1: 0.690 a,b,c = s.partition(":")
  • 222. a,b = s.split(":", 1)• Python 2.7: 0.665 • Python 3.1: 0.909 a,b = s.split(":")• Python 2.7: 0.631 • Python 3.1: 0.837 a,c = s.partition(":")[::2]• Python 2.7: 0.642 • Python 3.1: 0.690 a,b,c = s.partition(":")• Python 2.7: 0.407 • Python 3.1: 0.429
  • 223. s="Content-Type: text/htmlrn"
  • 224. .upper() or .lower()?
  • 225. "Content-Type: text/htmlrn".upper()
  • 226. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469
  • 227. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469"CONTENT-TYPE: text/htmlrn".upper()
  • 228. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469"CONTENT-TYPE: text/htmlrn".upper()• Python 2.7: 0.417 • Python 3.1: 0.616
  • 229. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469"CONTENT-TYPE: text/htmlrn".upper()• Python 2.7: 0.417 • Python 3.1: 0.616"CONTENT-TYPE: TEXT/HTMLrn".upper()
  • 230. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469"CONTENT-TYPE: text/htmlrn".upper()• Python 2.7: 0.417 • Python 3.1: 0.616"CONTENT-TYPE: TEXT/HTMLrn".upper()• Python 2.7: 0.291 • Python 3.1: 0.407
  • 231. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469"CONTENT-TYPE: text/htmlrn".upper()• Python 2.7: 0.417 • Python 3.1: 0.616"CONTENT-TYPE: TEXT/HTMLrn".upper()• Python 2.7: 0.291 • Python 3.1: 0.407"Content-Type: text/htmlrn".lower()
  • 232. "Content-Type: text/htmlrn".upper()• Python 2.7: 0.479 • Python 3.1: 0.469"CONTENT-TYPE: text/htmlrn".upper()• Python 2.7: 0.417 • Python 3.1: 0.616"CONTENT-TYPE: TEXT/HTMLrn".upper()• Python 2.7: 0.291 • Python 3.1: 0.407"Content-Type: text/htmlrn".lower()• Python 2.7: 0.319 • Python 3.1: 0.497
  • 233. a="foo"; b="bar"
  • 234. Efficient Concatenation?
  • 235. ", ".join((a, b))
  • 236. ", ".join((a, b))• Python 2.7: 0.405 • Python 3.1: 0.319
  • 237. ", ".join((a, b))• Python 2.7: 0.405 • Python 3.1: 0.319 a + ", " + b
  • 238. ", ".join((a, b))• Python 2.7: 0.405 • Python 3.1: 0.319 a + ", " + b• Python 2.7: 0.257 • Python 3.1: 0.283
  • 239. a="://"; b="http://www.example.com/"
  • 240. Determine Presence
  • 241. b.find(a)
  • 242. b.find(a)• Python 2.7: 0.255 • Python 3.1: 0.448
  • 243. b.find(a)• Python 2.7: 0.255 • Python 3.1: 0.448 a in b
  • 244. b.find(a)• Python 2.7: 0.255 • Python 3.1: 0.448 a in b• Python 2.7: 0.104 • Python 3.1: 0.119
  • 245. a="foo.bz"
  • 246. Test Filename Extension
  • 247. a.endswith(".bz")
  • 248. a.endswith(".bz")• Python 2.7: 0.338 • Python 3.1: 0.515
  • 249. a.endswith(".bz")• Python 2.7: 0.338 • Python 3.1: 0.515 a[-3:] == ".bz"
  • 250. a.endswith(".bz")• Python 2.7: 0.338 • Python 3.1: 0.515 a[-3:] == ".bz"• Python 2.7: 0.229 • Python 3.1: 0.312
  • 251. uri="/foo/bar/baz"
  • 252. Absolute Path?
  • 253. uri.startswith("/")
  • 254. uri.startswith("/")• Python 2.7: 0.324 • Python 3.1: 0.513
  • 255. uri.startswith("/")• Python 2.7: 0.324 • Python 3.1: 0.513 uri[0] == "/"
  • 256. uri.startswith("/")• Python 2.7: 0.324 • Python 3.1: 0.513 uri[0] == "/"• Python 2.7: 0.133 • Python 3.1: 0.146
  • 257. (Negative case identical.)
  • 258. Compatibility
  • 259. marrow.util.compat
  • 260. formatdate rfc822 vs. email.utils
  • 261. range vs. xrange
  • 262. str vs. bytes
  • 263. unicode vs. str
  • 264. “foo” vs. b“foo” vs. u“foo”
  • 265. from __future__ import unicode_literals
  • 266. No implicit conversion!
  • 267. (Stop that!)
  • 268. StringIO vs. BytesIO
  • 269. Exception Handling
  • 270. 216 def _handle_read(self):217 try:218 chunk = self.socket.recv(self.read_chunk_size)219220 except socket.error:221 e = exception().exception222 if e.args[0] in (errno.EWOULDBLOCK, errno.EAGAIN):223 return
  • 271. Questions?
  • 272. Chasing Corporate care of Air Review Relevant Specificationsmyspace.com/airreview PEP 333 WSGI 1.0 PEP 391 Dict Logging ConfigurationCore Developers PEP 444 WSGI 2.0Alice Bevan-McGregor PEP 3148 FuturesAlex Grönholm PEP 3333 WSGI 1.1Resources RFC 1945 — HTTP 1.0HTTP: The Definitive Guide, David RFC 2616 — HTTP 1.1Gourley & Brian Totty, O’Reilly PressPorting to Python 3, Lennart Regebro, Get GA to GA for PyCon!CreateSpace http://pledgie.com/campaigns/14434