Marc Chantreux, Floss Evangelist at Renater, presented at OW2con'19 (June 12-13 2019 in Paris) the Sympa project, the well know and largely used open source mailing list manager.

1. List service
What is Sympa ?
Like mailman, phplist, listserv, etc.
●
Sympa = SYstème de MultiPostage Automatique
●
It's was a mailing list manager
●
GPLv2, writen in perl
●
First version : 1st April 1997
●

2. large organizations have special needs
Tracking
Listes
auto
Custom
conditions
UTF 8
Named
filters
2007
5.3
Full
V. hosts
VERP
2006
5.2
RSS
2005
5.0
SSO
SOAP
2003
4.0
Antivirus
2001
3.2
S/MIME
HTTPS
Shared
2000
3.0
Archives
Modération
Scénarios
1997
0.001 6.
2
Web
1999
2.3
DKIM
XSS
Merge
2010
6.1
Bulk.pl
Mot de passe
MD5
Autotools
CSS
2009
6.0
Sessions
Custom
attributes
2008
5.4
BD
Inclusions
Bounces
2.2
1998
V. hosts
Task
manager
Auth
LDAP
3.3
2002
TT2
Familles
2004
4.2

3. sympa userbase
●
Serveurs : 3 500
●
Listes : ………………….. 1 300 000
●
Utilisateurs : …………. 35 000 000
●
Abonnements : ……. 130 000 000
●
Messages entrants : … 1 200 000
●
Messages sortants : .. 17 000 000

4. big ones
Biggest list : 1,6 millions subscribers
Most lists on a single server : 32 000
Most virtual hosts on a single server : 30 000
Most subscribers on a server : 3 millions
Users :
●
Research and education : 90 % of organizations in France
●
French ministries (defense, foreign affairs, finances, etc.)
●
Private companies (Illiad, Orange, Atos)
●
Hosting services (Infomaniak)
●
(Riseup.net, queernet)
●
NASA, UNESCO, CGT, etc.
●
Majority of users out of France (fully translated to a dozen languages, 30
other on their way...)
☺

5. sympa management
1997
CRU
2011
RENATER
2016
RENATER
2017
Communauté
RENATER

6. State nowaday
●
IS integration (produce/consume)
●
List management industrialization
●
Expressive authorization mechanism
●
Group management
●
Heavily customizable
●
And so many more, but I only have 50
minutes...

7. Sending mails
●
Bulk mailing :
– Group mails by domain
– Performance management (don't become a spammer but get mail out
anyway)
●
S/MIME (signing and crypting)
●
state of art (VERP,DMARC, DKIM, ARC)
●
Bounce automatic management
●
(privacy respecting) mail tracking
●
Web archives – controlled access
That's the basic purpose after all

8. spool
msg
bounce
task
outgoing
arc
config
bounced.pl
sympa.pl
task_manager.pl
archived.pl bulk.pl
Incoming
MTA
Incoming
MTA
Outgoing
MTA
Outgoing
MTA
bulk.pl bulk.pl
Sending workflow

9. MTA DB
Web serverAuthentication
Referential
IS integration

10. MTA
●
Sendmail
●
Postfix
●
Exim
IS integration

11. DB
●
MySQL
●
Postgres
●
Oracle
●
SQLite
●
Sybase
IS integration

12. Web server
●
Apache
●
Lighthttpd
●
Nginx
IS integration

13. Referential
●
Database
●
LDAP
●
Files
●
Web service
IS integration

14. Authentication
●
Native
●
CAS
●
LDAP
●
Shibboleth
IS integration

15. MTA DB
Web serverAuthentication
Referential
Sympa alone
Is useless...
IS integration

16. Industrialisation
Key1 val1
Key2 val2
Key3 val3
Key1 val1
Key2 val2
Key3 val3
Key4 val4
Key5 val5
Key1 [% key1 %]
Key2 [% key2 %]
Key3 [% key3 %]
Key4 val9
Web form
General
configuration
(defines defaults)
List template
Key1 val1
Key2 val2
Key3 val3
Key4 val9
Key5 val5
List configuration file
Manual creation

17. Industrialisation
Key1 val1
Key2 val2
Key3 val3
Key4 val4
Key5 val5
Key1 [% key1 %]
Key2 [% key2 %]
Key3 [% key3 %]
Key4 val9
XML document
General
configuration
(defines defaults)
List template
Key1 val1
Key2 val2
Key3 val3
Key4 val9
Key5 val5
Lists configuration files
List families <list>
<Key1>val1</Key1>
</list>
<list>
<Key1>val3</Key1>
</list>
Key1 val3
Key2 val2
Key3 val3
Key4 val9
Key5 val5

18. Industrialisation
●
Automatic lists = List families ++
●
Large number of potential lists – very few actually used
●
Example :
– Three categories : rank, location, unit.
– 20 values per category
– => 8000 potential lists
●
Solution : preparing a family but create lists only the first
time they're used.
●
The name of the list contains the values normally provided
by the XML file
●
Example : auto+engineer-sofia-research@domain.tld

19. Industrialisation
Key1 val1
Key2 val2
Key3 val3
Key4 val4
Key5 val5
Key1 [% key1 %]
Key2 [% key2 %]
Key3 [% key3 %]
Key4 val9
Mail to
auto+val3@domain.tld
General
configuration
(defines defaults)
List template
Key1 val3
Key2 val2
Key3 val3
Key4 val9
Key5 val5
List configuration files
Automatic lists
Listname
analysis Key1 = val3

20. Industrialisation
●
Automatic lists on the web
●
Automatic lists : difficult because you need
to remember :
– The structure of the list names : prefix+field1-field2...
– The values these fields can take
●
Sympa evolved to integrate them to its web
interface

21. Industrialisation

22. Authorization
●
Authorizations are handled by scenarios
●
File named <action>.<scenario>
– <action> :
●
what the user tries to do : send, web_archives, review, etc.
●
The name of the parameter to use in configuration
– <scenario> :
●
whatever you want to identify the scenario
●
The value to set for the parameter <action>
●
Example :
– Config : « send subscribers »
– Scenario used : « send.subscribers »

23. is_subscriber([listname],[sender]) smime -> do_it
is_subscriber([listname],[sender]) smtp -> request_auth
is_editor([listname],[sender]) smime -> do_it
is_owner([listname],[sender]) smime -> do_it
Authorization
A scenario looks like this :
General structure :
<test> <authentication method> -> <decision>
Senario evaluated from top to bottom. It stops at the first test that returns true

24. Authorization
Decision :
●
do_it : do the requested action (send the message, grant access to archives, etc.)
●
reject : don't execute the action
●
owner : request moderation from the list owner
●
etc.
Authentication method :
●
smime : request from a message containing a vlid S/MIME signature
●
smtp : no authentication except the From field
●
md5 : message confirmed by mail or request from an authenticated user from the web
interface
●
dkim : DKIM signed message
Tests :
●
is_subscriber : check whether the user is subscrbed to a given list
●
is_owner : check whether the user is owner of a given list
●
match : checks a string against a regexp
●
etc.
●
If none fits your need, just use custom_condition that uses your own module...

25. List service
SOAP service
Third party
application
request
Sympa groups
Group manager
List = group
Lists can be merged
to form larger lists
Manage groups :
●
Manually (manual subscription)
●
Automatically (from external data sources)
●
Both...

26. Customization
●
You can modify most Sympa behaviours :
– At the server level,
– At the virtual host level,
– At the list level
●
What can you customize ?
– Web interface,
– Service messages,
– List creation templates,
– Scenario (and create new conditions or scenarios),
– Create your own list parameters,
– Add new attributes to users (in addition to email),
– Tasks
– Default list and virtual hosts configuration,
– You name it...

27. but
●
lack of documentation
●
installing is hard
●
need of more commercial support
●
hard to get new regular contributors
●
i hope smtp to be back