• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
The Perdition and NGINX IMAP Proxies
 

The Perdition and NGINX IMAP Proxies

on

  • 21,212 views

 

Statistics

Views

Total Views
21,212
Views on SlideShare
20,956
Embed Views
256

Actions

Likes
9
Downloads
158
Comments
0

5 Embeds 256

http://jpmens.net 94
http://blog.fupps.com 93
http://www.slideshare.net 67
http://webcache.googleusercontent.com 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

The Perdition and NGINX IMAP Proxies The Perdition and NGINX IMAP Proxies Presentation Transcript

  • The Perdition and NGINX IMAP Proxies UKUUG Spring 2010 Conference Manchester, UK March 2010 Jan-Piet Mens mens.de
  • Overview IMAP servers: one, two, .... lots How an IMAP proxy works Perdition NGINX Summary.
  • Single server User configures IMAP server name in MUA MUA connects to IMAP server Expansion means Larger server Multiple servers Distribute accounts IMAP client server
  • Multiple servers More than one server? Users "know" server names Non-transparent to users User "freddy" server1? User "paula" server3? Use DNS CNAMES to associate users to servers IMAP IMAP IMAP server server server a-f.example.net g-m.example.net n-z.example.net client
  • How a proxy works Client connects to proxy (in) Proxy retrieves username 01 login sue@example.org secret Proxy looks up username in "map" Optional: authentication Optional: translate username Map returns address of back-end IMAP server Proxy hands off connection to IMAP server (out) and authenticates with target server IMAP IMAP IMAP server server server IMAP client proxy map lookup
  • Pros and Cons Advantages Transparent to users: imap.example.net Replace/maintain back-end servers at will Migrate users from server1 to serverN Synthetic usernames on IMAP servers Different ways of lookup up users (DB, LDAP, etc.) Consolidate heterogenous brands of IMAP servers Integrate monitoring to provide access to available IMAP servers only Central logging Disadvantages Additional code Possible single point of failure Load-balance / heartbeat
  • IMAP Capabilities Different servers have different capabilities http://www.iana.org/assignments/imap4-capabilities Clients typically query them before login List capabilities of your IMAP server $ openssl s_client -connect imap.gmail.com:993 * OK ready for requests 01 capability * CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA XLIST CHILDREN XYZZY 01 OK Thats all she wrote! Adapt proxy’s CAPABILITY to common denominator Lotus Dovecot Cyrus Domino IMAP client CAPABILITY proxy
  • Testing IMAP connections Telnet / SSL $ telnet imap.example.net imap $ openssl s_client -connect imap.example.net:993 Watch your IMAP connection with IMAP Proxy Server http://www.aboutmyip.com/AboutMyXApp/ImapProxy.jsp $ java -jar IMAPProxy.jar
  • IMAP Proxies Perdition NGINX (Engine X) Others Courier IMAP/POP3 proxy Cyrus Murder Dovecot proxy
  • Perdition Created by Simon Horman POP3 and IMAP4 One process per connection Can bridge between SSL/TLS and plain back-ends Perdition can authenticate user locally if compiled with PAM support Supported lookups: CDB, GDBM, BDB, MySQL, PostgresSQL, LDAP, NIS, regex Custom C function Username replacement No support for CRAM-MD5
  • Perdition configuration /etc/perdition/perdition.conf bind_address 192.168.1.120 connection_logging log_facility mail log_passwd fail imap_capability IMAP4REV1 UIDPLUS IDLE map_library /lib/libperditionXXX.so map_library_opt "this or that" username_from_database ssl_mode tls_listen,ssl_listen ssl_ca_chain_file /etc/perdition/chain.pem ssl_cert_file /etc/perdition/imap.crt ssl_key_file /etc/perdition/imap.key ssl_cert_verify_depth 0 ssl_listen_ciphers "ALL:!ADH:!NULL:+HIGH: ...
  • Perdition map: GDBM Options map_library libperditiondb_gdbm.so.0 map_library_opt /etc/my_popmap.gdbm.db Data $ cat popmap sue:localhost jane:s009@example.org:143 joe@foo.net:j999@imap-1.internal $ makegdbm popmap.gdbm.db < popmap
  • Perdition map: POSIX regexp Options map_library libperditiondb_posix_regex.so.0 map_library_opt /etc/popmap.re Data First match wins $ cat popmap.re ^john: user2@imap.example.net:143 ^[a-m]: server2.example.net (.*)@(.*): $1_$2@localhost
  • Perdition map: MySQL, PostgresSQL Options map_library libperditiondb_mysql.so.0 map_library_opt dbhost:dbport:dbname:dbtable:dbuser: dbpwd:dbservercol:dbusercol:dbportcol Data SELECT * FROM users; +------+------------------+------+ | user | servername | port | +------+------------------+------+ | sue | localhost | NULL | | jane | s009@example.org | 143 | +------+------------------+------+
  • Perdition map: LDAP Options map_library libperditiondb_ldap.so.0 map_library_opt 3:ldap://localhost/o=example.net? username,mailhost,port?one?(uid=%s) Data Attributes returned from search new username (optional), server, port (optional) perdition.schema has a structural objectclass dn: uid=jane, o=example.net objectClass: uidObject objectClass: perditionPopmap uid: jane username: s009 mailhost: example.org port: 143 Can use mailHost from inetLocalMailRecipient
  • Perdition map: custom C Options map_library libperditiondb_XXYY.so.0 map_library_opt "this or that" Data Look up user in map int dbserver_get(const char *key, const char *options_str, char **str_return, int *len_return) Initialize / Terminate int dbserver_init(char *options_str) int dbserver_fini(void) Make gcc -shared $(L).c -L/usr/lib -o $(L).so
  • Perdition map: custom C (2) Example int dbserver_get(const char *key, const char *opts, char **sret, int *lret) { int status = -2; // not found if (key && *key && !strcmp(key, "jp")) { char *server = "192.168.1.20"; // [user@]host[:port] *lret = strlen(server) + 1; if ((*sret = calloc(1, *lret)) == NULL) status = -3; // other error else { status = 0; // OK strcpy(*sret, server); } } return (status); }
  • Case study: Perdition with Lotus Domino Domino cluster contains n hosts User has mail on >= 2 hosts Custom Perdition database map Retrieves username Finds user’s back-end IMAP servers (cldbdir.nsf) Attempts TCP connect Fails over to backup server Returns first live server to Perdition Perdition connects to that IMAP server Domino Domino Domino client Perdition LDAP socket()
  • NGINX (Engine X) Fast HTTP server and reverse proxy, and IMAP/POP3 and SMTP proxy created by Igor Sysoev for rambler.ru Fixed process pool and non-blocking code Powers WordPress, Github, Ohloh, SourceForge, ... Doesn’t require special database or LDAP schema POST to a URL to do authentication/authorization via HTTP or UNIX socket Authorization / authentication Apache, NGINX (HTTP), Lighttpd, ..., thttpd NGINX Web server is built-in FastCGI Embedded Perl
  • NGINX IMAP/POP3 proxy Client connects to NGINX NGINX passes authorization request to URL via POST or UNIX socket Auth back-end looks up user (optionally password, etc.) Auth back-end determines target server/port Auth back-end passes data back to NGINX NGINX proxies connection to user’s IMAP/POP3 server:port (optionally using new credentials) NGINX IMAP IMAP client IMAP proxy IMAP headers HTTP IMAP process
  • NGINX authorization NGINX passes authorization info in HTTP headers HTTP_AUTH_PROTOCOL: imap HTTP_HOST: nano.mens.de HTTP_CLIENT_IP: 192.168.1.154 HTTP_AUTH_METHOD: plain HTTP_AUTH_USER: sue@example.net HTTP_AUTH_PASS: seacret HTTP_AUTH_LOGIN_ATTEMPT: 2 HTTP_YY_AUTH: jp-mysecret Authmethod CRAM-MD5 sets salt and HMAC-MD5 hashed password (need cleartext on server to verify and hand back to NGINX) HTTP_AUTH_METHOD = cram-md5 HTTP_AUTH_SALT = <1885293406.1268902260@nano.mens.de> HTTP_AUTH_PASS = 1b0864a115d8ae5f3562e3bc7d05cb59
  • NGINX authorization Authorization module (e.g. CGI) can Authenticate Redirect to back-end server Report failure Should complete fast
  • NGINX authorization response Good response HTTP/1.0 200 OK Auth-Status: OK Auth-Server: 192.168.1.20 Auth-Port: 143 Auth-User: newusername Auth-Pass: newseacret Bad response HTTP/1.0 200 OK Auth-Status: Invalid login or password Auth-Wait: 5 Don’t forget CGI Content-type
  • NGINX configuration nginx.conf error_log logs/error.log info; events { worker_connections 1024; multi_accept on; } mail { auth_http web.example.net:80/nginx/auth.cgi; auth_http_header YY-Auth "jp-mysecret"; imap_auth login plain cram-md5; imap_capabilities "IMAP4rev1" "UIDPLUS" "IDLE"; server { listen *:143; protocol imap; proxy on; proxy_pass_error_message on; } }
  • Sample NGINX authenticator Perl CGI my $q = new CGI; my $username = $q->http(’HTTP_AUTH_USER’); my $password = $q->http(’HTTP_AUTH_PASS’); if ($username eq ’jpm’ && $password eq ’ooh’) { print "Auth-Status: OKn"; print "Auth-Server: 192.168.1.20n"; print "Auth-Port: 143n"; } elsif ($username eq ’sue’) { ... print "Auth-Server: 192.168.3.47n"; } else { print "Auth-Status: Invalid login or passwordn"; print "Auth-Wait: 3n"; } print "Content-type: text/plainn"; print "n";
  • Some numbers Perdition 40,000 users with 2 x Perdition, 9 Courier IMAP ISP: 10 Perdition, 10 M POP3/IMAP connections/day NGINX fastmail.fm: 10,000 connections on 10% CPU (3.2 GHZ Xeon) (FastMail.fm moved from Perdition to NGINX)
  • Summary Perdition has built-in database lookups Easier to deploy: no coding Perdition’s custom functions allow complex setups NGINX as IMAP/POP3 proxy requires custom code Consider using memcached for caching "expensive" user-lookups Integrate your monitoring system to influence choice of target IMAP server
  • Further reading Perdition http://www.vergenet.net/linux/perdition/ NGINX http://wiki.nginx.org/Main Alternative DNS Servers, UIT, 2009, Jan-Piet Mens http://mens.de/:/altdns
  • Thank you Questions?