HTTP::Parser::XS - writing a fast & secure XS module

HTTP::Parser::XS writing a fast & secure XS module Cybozu Labs, Inc. Kazuho Oku

Pros & Cons of XS Pros Faster (well, not always, but…) Cons Difficult and time-consuming Higher security risk Mistakes in C programming leads to remote code injection May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 2

The Best Practice, in General Don’t use XS Until you hit performance bottleneck Rewrite the bottleneck (only) using XS May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 3

HTTP::Parser::XS May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 4

Summary An HTTP request parser Designed and implemented to be simple, fast & secure Stateless PSGI-compatible Required or recommended by many Plack servers Starman, Starlet, Twiggy, etc. May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 5

Two-layered Approach Picohttpparser HTTP request / response parser written in C has its own test suite does not parse request / response content request line (response line) and headers only copy-less faster, lesser probability of security holes and memory leaks HTTP::Parser::XS glue code to access picohttpparser from perl May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 6

Speed and Complexity HTTP::Parser::XS is simple, and fast most of the time is not spent in picohttpparser, but in the glue code constructing hashref picohttpparser/trunk can handle >1Mreqs/sec. reqs/sec. lines of code HTTP::HeaderParser::XS 116,000 1,166 HTTP::Parser::XS 140,000 487 Plack::HTTPParser::PP 10,100 104 May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 7

Why is it a Stateless? Faster and simpler than a stateful parser lower security risks most HTTP requests / responses arrive in a single packet, anyway if written optimally in C, the cost of re- parsing is smaller than storing headers into a perlhashref May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 8

Why is it Stateless? (cont’d) Easy to determine the end of an multi- packet HTTP request by looking for “rnrn” within the last packet (and preceding three bytes) mainly as a countermeasure for Slowloris May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 9

Consistent Design Functions take same arguments  buf – points to current char  buf_end – points to end of buffer  *ret – error value  returns pointer to the next char  or null on error (the reason will be stored in *ret) const char* parse_http_version(const char* buf, const char* buf_end, int* minor_version, int* ret) { EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T'); EXPECT_CHAR('P');EXPECT_CHAR('/'); EXPECT_CHAR('1'); EXPECT_CHAR('.'); return parse_int(buf, buf_end, minor_version, ret); } May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 10

Macros Consistent design is #define CHECK_EOF() if (buf == buf_end) { *ret = -2; essential to heavy use } return NULL; of macros #define EXPECT_CHAR(ch) CHECK_EOF(); Good abstraction ⇒ safe if (*buf++ != ch) { *ret = -1; code } return NULL; const char* parse_http_version(const char* buf, const char* buf_end, int* minor_version, int* ret) { EXPECT_CHAR('H'); EXPECT_CHAR('T'); EXPECT_CHAR('T'); EXPECT_CHAR('P');EXPECT_CHAR('/'); EXPECT_CHAR('1'); EXPECT_CHAR('.'); return parse_int(buf, buf_end, minor_version, ret); } May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 11

Micro-optimization (only in picohttpparser) Reduce # of conditional branches, optimize for pipeline (27% faster) unlikely(x) __builtin_expect(!!(x), 0) #define for (; ; ++buf) { CHECK_EOF(); if (unlikely((unsigned char)*buf<= 'r') && (*buf == 'r' || *buf == 'n')) gotoEOL_FOUND; } Unroll loops (36% faster) May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 12

Release History of HTTP::Parser::XS Current version: 0.07 No security hole found since initial release Two memory leaks were found and fixed Please let me know if you find any security holes (especially the ones that lead to arbitrary code execution) May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 13

Conclusion – KISS Keep it simple, stupid for fast development simple design leads to more secure code use perl whenever possible simple operations (like tokenization) is worth converting to XS complex operations (from handling of strings to database queries) are not so slow in perl May 28 2010 HTTP::Parser::XS - writing a fast & secure XS module 14

http://developer.cybozu.co.jp	2115
http://developer-old.cybozu.co.jp	11
http://webcache.googleusercontent.com	9
http://translate.googleusercontent.com	5
http://static.slidesharecdn.com	3
http://freerss.net	2
http://reblog.rot13.org	1
http://rssreader.isolinear.jp	1
http://anonymouse.org	1

HTTP::Parser::XS - writing a fast & secure XS module

by Kazuho Oku

Accessibility

Categories

Upload Details

Usage Rights

9 Embeds 2,148

Statistics

HTTP::Parser::XS - writing a fast & secure XS module Presentation Transcript