IndiMail - The Flexible Messaging Platform


Published on

IndiMail is a messaging platform utilizing the following packages qmail, serialmail, qmailanalog, dotforward, fastforward, mess822, daemontools, ucspi-tcp, Courier IMAP/POP3, Bogofilter - A Bayesian Spam Filter, Fetchmail, other useful utilities (pack, unpack, altermime, ripmime, flash). IndiMail provides Management of Virtual domains with large number of users. Some of the components like qmail have been significantly modified to provide a seamless user experience, manageability, superior performance and high scalability. IndiMail works on both 32 / 64 bit Unix platforms.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

IndiMail - The Flexible Messaging Platform

  1. 1. IndiMailThe Flexible Messaging Platform Manvendra Bhangui
  2. 2. IndiMailThe Flexible Messaging Platform Strengths & Advantages Components Feature List ArchitectureInstallation & Configuration Administration Maintenance Long-term Goals Support Hardware Requirements Questions
  3. 3. IndiMail The Flexible Messaging PlatformStrengths & Advantages –All parameters customizable through environment variables (around 200 parameters). –Customizable globally, locally, dynamically or specifically for a user or group of users. –Mail can be intercepted before injection, after queuing or before final delivery –Single instance multiple queues. Concurrency customizable –Horizontally scalable architecture. –Crash proof –Faster than qmail, postfix (the only two noteworthy competitors) –Full featured (SMTP, POP3, IMAP, SPAM Filter, Virus Scanning, SSL, Domainkeys, DKIM, BATV, etc) –Highlysecure. Can protect vulnerable messaging servers like MS Exchange by acting as frontend SMTP, IMAP and POP3 servers. –Open Standards – Compliant with most of the messaging RFCs –Open Source – Source code available (GNU GPL V3). –accesslist– control email exchanges between senders & recipients. Control exchanges between internet and users. (unique to indimail & satisfies corporate requirements) –High speed virus/spam filtering by using inline scanning.
  4. 4. IndiMail The Flexible Messaging PlatformComponents MTA -Based on qmail. Extensive changes to make it a modern MTA and achieve highest possible delivery speed. IMAP/POP3 - Based on courier-imap. Changes made to support multiple open standards authentication modules (v4.10.x) Virus Filter - clamAV with automatic signature updates using freshclam (v0.96.x) SPAM Filter- Bogofilter – robinson fisher algorithm (v1.2.2)
  5. 5. IndiMail The Flexible Messaging PlatformFeature List – Speed ● Probably the fastest MTA. Multi-queue architecture allows 1.5 millions+ deliveries on a intel commodity hardware – Setup ● automatic adaptation to your UNIX variant ● Linux, SunOS, Solaris, and more ● automatic per-host configuration - gnu autoconf ● High degree of automation of configuration through svctool ● RPM packages for multiple Linux Distros.
  6. 6. IndiMail The Flexible Messaging PlatformFeature List (cont...) – Security ● clear separation between addresses, files, and programs ● minimization of setuid code (qmail-queue, qhspi, qscanq, systpass) ● minimization of root code (qmail-start, qmail-lspawn) ● five-way trust partitioning---security in depth ● optional logging of one-way hashes, entire contents, etc. (EXTRAQUEUE, mailarchive control file) ● virus scanning through qscanq, clamav. ● Extensible plugin feature for virus scanners ● Inbuilt virus scanner ● sender/recipient accesslist, hostaccess using tcprules
  7. 7. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Message construction ● RFC 822, RFC 1123 ● full support for address groups ● automatic conversion of old-style address lists to RFC 822 format ● sendmail hook for compatibility with current user agents ● header line length limited only by memory ● host masquerading (control/defaulthost) ● user masquerading ($MAILUSER, $MAILHOST) ● automatic Mail-Followup-To creation ($QMAILMFTFILE) ● ability to add signature/content to messages using altermime ● Abuse report format (ARF) generator using qarf
  8. 8. IndiMail The Flexible Messaging PlatformFeature List (cont…) – SMTP ● RFC 2821, RFC 1123, RFC 1651, RFC 1652, RFC 1854, RFC 1870, RFC 1893 ● 8-bit clean ● 931/1413/ident/TAP callback ● relay control---stop unauthorized relaying by outsiders (control/rcpthosts) ● no interference between relay control and aliases ● automatic recognition of local IP addresses ● per-buffer timeouts ● hop counting * parallelism limit (tcpserver) ● per host limit (tcpserver - MAXPERIP)
  9. 9. IndiMail The Flexible Messaging PlatformFeature List (cont…) – SMTP (cont…) ● refusal of connections from known abusers (tcpserver, badmailfrom, badmailpatterns, badhelo, blackholedsender, blackholedpatterns, badhost, badip) ● goodrcptto, goodrcptpatterns which override the above ● blackholercpt, blackholercptpatterns for blackholing mails to specific senders. ● Control files spamignore, blackholedsender, badmailfrom, relaymailfrom, badrcptto, chkrcptdomains, goodrcptto, blackholercpt, badip can be specified in plain text, cdb format as well as stored in MySQL tables. ● relaying and message rewriting for authorized clients ● authenticated SMTP PLAIN, LOGIN, CRAM-MD5, CRAM-SHA1, CRAM- RIPEMD, DIGEST-MD5 HMAC (RFC 1321, RFC 2104, RFC 2554, RFC 2617)
  10. 10. IndiMail The Flexible Messaging PlatformFeature List (cont…) – SMTP (cont…) ● STARTLS extension, TLS ● Support for SMTPS ● POP/IMAP before SMTP ● ETRN (RFC 1985) ● ODMR (RFC 2645) ● RBL/ORBS support (rblsmtpd) ● DNSBL support using plugin ● SURBL (SURBL Blacklist) using surblfilter ● SPAM Control (Reject/Tag/Accept) using Bayesian techniques ● High Performance MS Virus Control via control file viruscheck and control file signatures ● Content Filtering and blocking of prohibited attachments via control file bodycheck
  11. 11. IndiMail The Flexible Messaging PlatformFeature List (cont…) – SMTP (cont…) ● Ability to reject/bounce mails for unknown/inactive users (CHECKRECIPIENT) ● ability to have the RECIPIENT check for selective domains using control file chkrcptdomains ● Antispoofing mode (turned on by environment variable ANTISPOOFING) ● Masquerading ability. ● Multiline greetings via control file smtpgreeting ● Message Submission Agent – MSA (RFC 2476) ● Domain IP address pair access control via control file hostaccess ● Per User accesslist via control file accesslist ● SPF – Sender Permitted From
  12. 12. IndiMail The Flexible Messaging PlatformFeature List (cont…) – SMTP (cont…) ● Per User control of environment variable by envrules (rules file set by environment variable FROMRULES) ● Greylisting[3] capability using qmail-greyd or greydaemon ● Bounce Address Tag Validation (BATV)[4] ● Notify recipient when message size exceeds databyte limits (by setting environment variable DATABYTES_NOTIFY) ● SMTP Plugins using external plugins in /var/indimail/plugins – Queue management ● instant handling of messages added to queue ● parallelism limit (control/concurrencyremote, control/concurrencylocal) ● split queue directory---no slowdown when queue gets big
  13. 13. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Queue Management (cont…) ● quadratic retry schedule---old messages tried less often ● independent message retry schedules ● automatic safe queueing---no loss of mail if system crashes ● automatic per-recipient checkpointing ● automatic queue cleanups (qmail-clean) ● queue viewing (qmail-qread) ● detailed delivery statistics (qmailanalog) ● Ability to hold local, remote or both deliveries (holdlocal, holdremote control file) ● Qmail Queue Extra Header – Ability to pass extra headers to local and remote deliveries via qmail-queue (Environment variable QQEH).
  14. 14. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Queue Management (cont…) ● Configurable number of queues and time slicing algorithm for load balancing via qmail-multi. A queue in indimail is configurable by three environment variables QUEUE_BASE, QUEUE_COUNT, and QUEUE_START. A queue in IndiMail is a collection of queues. Each queue in the collection can have one or more SMTP listener but a single or no delivery (qmail-send) process. It is possible to have the entire queue collection without a delivery process (e.g. SMTP on port 366 – ODMR). The QUEUE_COUNT can be defined based on how powerful your host is (IO bandwidth, etc). NOTE: This configurable number of queues is possibe with a single installation and does not require you to install multiple instances of qmail.
  15. 15. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Queue Management (cont…) ● External Virus scanning via QHPSI – Qmail High Performance Scanner Interface ● Ability to extend QHPSI interface through plugins. The keyword plugin:shared_lib defined in the environment variable QHPSI denotes shared_lib to be loaded. ● Virus scanner qscanq. Ability to detect virus via a third party scanner defined by SCANCMD environment variable (clamscan, clamdscan, etc) ● Blocking of prohibited filename extensions via qscanq program ● Domainkeys (qmail-dk) RFC 4870 * DKIM[5] with ADSP/SSP (qmail- dkim) RFC 4871
  16. 16. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Queue Management (cont…) ● Set all header values listed in envheader control file as environment variables. ● Log all headers listed in control file logheaders to stderr. ● Remove all headers listed in control file removeheaders from email. ● Ability to do line processing instead of block processing. ● qmail-nullqueue – blackhole the mail silently. ● rule based mail archival using control file mailarchive (SOX, HIPAA compliance) ● Added additional recipients for a message using extraqueue or mailarchive control file. ● X-Originating-IP header to record the original IP from which the mail originates
  17. 17. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Bounces ● QSBMF bounce messages---both machine-readable and human-readable ● HCMSSC support---language-independent RFC 1893 error codes ● double bounces sent to postmaster ● Ability to discard double bounces ● Ability to preserve MIME format when bouncing. ● Control of bounce process via envrules (rules file controlled by environment ● variable BOUNCERULES or control files bounce.envrules) ● limit size of bounce using control file bouncemaxbytes ● Ability to process bounces using external bounce processor (environment variable BOUNCEPROCESSOR)
  18. 18. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Routing by domain ● any number of names for local host (control/locals) ● any number of virtual domains (control/virtualdomains) ● domain wildcards (control/virtualdomains) ● configurable percent hack support (control/percenthack) ● Clustered Domain. Same virtual domain can exist on multiple hosts, each having its own set of users. Provides Load Balancing and infinite scalability.
  19. 19. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Remote SMTP delivery ● RFC 2821, RFC 974, RFC 1123, RFC 1870 ● 8-bit clean ● automatic downed host backoffs ● Configurable tcp timeouts for downed host backoffs. ● automatic switchover to next best MX ● artificial routing---smarthost, localnet, mailertable (control/smtproutes) ● Support for jumbo ISP (control/smtproutes.cdb) ● per-buffer timeouts ● passive SMTP queue---perfect for SLIP/PPP (serialmail) ● AutoTURN support (serialmail) ● Spam control (SPAMFILTER environment variable)
  20. 20. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Remote SMTP delivery (cont…) ● Authenticated SMTP (userid/passwd in control/smtproutes) - PLAIN, LOGIN, CRAM-MD5, CRAM-SHA1, CRAM-RIPEMD, DIGEST-MD5 ● STARTTLS, TLS ● Static and Dynamic Routing. (SMTPROUTES environment variable) ● Environment variable control via envrules (rules file controlled by environment variable RCPTRULES) * QMAILREMOTE environment variable to run any executable/script instead of qmail-remote ● QMTP support, artificial routing using (control/qmtproutes) ● ONSUCCESS_REMOTE, ONFAILURE_REMOTE scripts run on successful or failed remote deliveries environment variables SMTPTEXT, SMTPCODE, ERRTEXT available for these scripts ● IP address binding on domain, sender address, recipient address and random selection from a pool of IP addresses
  21. 21. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Local delivery ● user-controlled address hierarchy : fred controls fred-anything ● mbox delivery ● reliable NFS delivery (maildir) ● user-controlled program delivery: procmail etc. (qmail-command) ● optional new-mail notification (qbiff) ● detailed Delivered-To Headers ● optional NRUDT return receipts (qreceipt) ● autoresponder RFC 3834 compliance (provide Auto-Submitted, In-Reply-To, References fields (RFC 3834)) ● conditional filtering (condredirect, bouncesaying, vfilter)
  22. 22. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Local delivery (cont…) ● Environment variable control via envrules (rules file controlled by environment variable RCPTRULES) ● Eliminate duplicate messages ● QMAILLOCAL environment variable to run any executable/script instead of qmail-local ● X-Forwarded-To, X-Forwarded-For headers – Other ● Unix Client Server Program Interface (UCSPI) through programs tcpserver and tcpclient ● Change concurrency of tcpserver without restart
  23. 23. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Other ● TLS/SSL Support in tcpserver ● STARTTLS extension in IMAP, STLS extension in POP3 ● Ability to restrict connection per IP (MAXPERIP) ● run shutdown script if present on svc –d ● ability to log svscan output using multilog ● nssd Name Service Switch which allows extending of the system passwd database to IndiMails database. ● pam-multi - Generic PAM module allows any external programs to authenticate against IndiMails database. ● multiple checkpassword modules sys-checkpwd, ldap-checkpwd, pam-checkpwd, vchkpass, systpass ● Proxy for IMAP/POP3 Protocol
  24. 24. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Other ● inlookup – High Performance User Lookup Daemon. ● indisrvr – Indimail Administration Daemon. ● spawn-filter - Ability to add disclaimer, run multiple filters before local/remote delivery. ● Post Execution Handle - Allows functionality of indimail to be extended by writing simple scripts ● On the fly migration of users by defining MIGRATEUSER environment variable. ● ready to use QMQP service ● ability to distribute QMQP traffic across multiple servers ● sslerator - TLS/SSL protocol wrapper for non-tls aware applications
  25. 25. IndiMail The Flexible Messaging PlatformFeature List (cont…) – Other ● svctool – Configuration tool for IndiMail. ● adminclient protocol – Interface for external programs to administer IndiMail ● mrtg graphs for detailed statistics ● ability to specify commands in control files
  26. 26. Internet IndiMail Architecture Internet (25) (110, 143) Load Balancer 1 (25) SMTP (110/143) Port 25 (80) (80) Incoming Incoming Incoming Webmail Webmail Webmail Proxy Proxy Proxy Relay Relay Relay imap/pop3 imap/pop3 imap/pop3 Inlookup Inlookup Inlookup inlookup inlookup inlookup Clamd Clamd Clamd bogofilter bogofilter (25/587) MySQ SMTP Conn L ecti on (for authenticated My user Co SQ to submit their IMAP/POP3 nn L outgoing email) ec ti o n Online Replication MySQL MySQL SMTP Load Balancer 2 Host Control DB Host Control DB (Master) (Slave) MySOutgoing Outgoing Con QL MySQL nec Connection Relay Relay tion Inlookup Inlookup clamd clamd SMTP Mailstore Mailstore Mailstore (delivery from imap/pop3/smtp imap/pop3/smtp imap/pop3/smtp mailstore) Inlookup Inlookup Inlookup vdelivermail vdelivermail vdelivermailInternet SMTP SMTP (for localqmail-remote domains) (external domain) MySQL MySQL MySQL
  27. 27. IndiMailThe Flexible Messaging Platform
  28. 28. IndiMailThe Flexible Messaging Platform
  29. 29. IndiMailThe Flexible Messaging Platform
  30. 30. IndiMailThe Flexible Messaging Platform
  31. 31. IndiMailThe Flexible Messaging Platform
  32. 32. IndiMailThe Flexible Messaging Platform
  33. 33. Installation IndiMail – The Flexible Messaging Platform RPM for both 32 / 64 Bit – rpm -ivh rpm_filename – openSUSE • openSUSE 11.3 • openSUSE 11.2 • openSUSE 11.1 • SUSE Linux Enterprise 11 SP1 • SUSE Linux Enterprise 11 • SUSE Linux Enterprise 10 • OpenSUSE Factory – Red Hat • Fedora14 • Fedora13 • Fedora12 • RHEL6 • RHEL5 • Centos 5 – Mandriva Linux • Mandriva 2010.1 • Mandriva 2010 – Yum Repository – – Copy repo file to /etc/yum.repos.d for hands-free installation/upgrade ● % sudo yum install indimail.x86_64 ● % sudo yum install indimail.i386
  34. 34. IndiMailConfiguration Flexible Messaging Platform The – svctool (service Tool) • SMTP, IMAP, POP3, ssl, fetchmail, certificates, verification of installation • Backups of data, configuration • Repair of Database, queue • Configuration for MySQL, Spamfilter, virus filter, qmail • Reports – GUI Iwebadmin, indium, shit, cindimai, osh • – Supervise for 100% uptime • Envdir • Logging
  35. 35. IndiMail The Flexible Messaging Platform● Administration ● Web Administration – iwebadmin ● ncurses GUI – Secure Host Interface – Configurable menu & commands ● Indium – TCL/TK GUI – Powerful & provides in-depth administration ● Operator Shell – osh – Bash like shell with restricted access to commands – Access to commands based on user roles ● cindimail – Small shell providing access to all indimail commands with tool tips
  36. 36. IndiMail The Flexible Messaging PlatformMaintenance – Backup of Configuration ● /var/indimail/control ● /var/indimail/etc ● /var/indimail/users ● /var/indimail/share – Mail Backup – Update of Clamav Software – Update of IndiMail (rpm, yum)
  37. 37. IndiMail The Flexible Messaging PlatformLong Term Goals ● To provide a modern technologically superior technology and solutions specifically related to Messaging Protocols (SMTP, IMAP, POP3) ● Provide a highly scalable, bug free, MSP class mailing solution using the Bazaar Model under GNU GPL V3 ● Provide configuration for any property that can potentially affect the behaviour of messaging (Install the software once). ● FHS 2.3 compliance ● Build a community and get shipped with Linux distros by default
  38. 38. IndiMail The Flexible Messaging PlatformSupport ● indimail-support - You can subscribe for Support at ● You can email for posting messages to this list. ● indimail-devel - You can subscribe at ● indimail-announce - This is only meant for announcement of New Releases or patches. You can subscribe at You can email for posting messages to this list. ● Archive at Google - This group acts as a remote archive. Any discussions posted here goes to indimail-support. ● There is also a Project Tracker for IndiMail (Bugs, Feature Requests, Patches, Support Requests) at
  39. 39. IndiMail The Flexible Messaging PlatformDocumentation– INSTALL, INSTALL-RPM– IndiMail Documentation – indimail.pdf– WIKI -– Frequently Answered Questions – FAQ.pdf– Man Pages - /var/indimail/man– All docs in /var/indimail/docs
  40. 40. IndiMail The Flexible Messaging PlatformHardware Requirements (100,000 users) – Messaging Server • DL 360 (Relay Server) – 2 Nos • DL 380 (Message Store) – 1 Nos • Storage (MSA 1000/MSA 2000) - Groupware & Chat • DL380 (Egroupware & Jabber) – 1 Nos - Load Balancer – 1 Nos
  41. 41. IndiMail The Flexible Messaging PlatformTimelines– Messaging Server – 1 day installation + 4 weeks training– Groupware & chat - 6 weeks installation & configuration + 2 weeks training