Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Rspamd testing

657 views

Published on

Rspamd proxy description

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Rspamd testing

  1. 1. RSPAMD TESTING
  2. 2. PROBLEM STATEMENT WHY TESTING IS HARD ▸ Need to test on live traffic ▸ Testing environment might be less powerful (e.g. a VM) ▸ Experimental machines can fail or die ▸ Need to compare all results
  3. 3. PROBLEM STATEMENT GOALS: TEST NEW VERSIONS STABLE VERSION TESTING VERSION COMPARE RESULTS
  4. 4. PROBLEM STATEMENT GOALS: TEST NEW RULES OLD RULES NEW RULES COLLECT STATISTICS NEW PLUGINS
  5. 5. PROBLEM STATEMENT GOALS: COMPARE SPAM ENGINES RSPAMD OTHER SCANNER COMPARE QUALITY
  6. 6. PROBLEM STATEMENT GOALS: COLLECT GLOBAL STATISTICS RSPAMD GATHER STATS RSPAMD RSPAMDRSPAMD 1% 2% 10% 0.5% STATISTICS
  7. 7. ARCHITECTURE SCAN SCHEME RSPAMD PROXY STABLE CLUSTER TESTING CLUSTER TESTING CLUSTER Proxy stable result COMPARE RESULTS
  8. 8. ARCHITECTURE MAIN FEATURES ▸ Reply immediately when get results from the main cluster ▸ Fast and low latency architecture ▸ Can use multiple compare result scripts ▸ Compare scripts could use all API functions from rspamd
  9. 9. ARCHITECTURE ENCRYPTION PROXY RSPAMD PROXY STABLE CLUSTER Encrypt using HTTPCrypt Scan local file
  10. 10. ARCHITECTURE ENCRYPTION PROXY ▸ Encrypt with HTTPCrypt: ▸ low latency (0 RTT before data sending) ▸ zero copy ▸ provable secure ▸ simple keys management ▸ Can open local files and send encrypted data stream ▸ Each cluster can have its own unique encryption key ▸ Local keys are rotated frequently
  11. 11. ARCHITECTURE LOAD BALANCING RSPAMD PROXY STABLE CLUSTER TESTING CLUSTER TESTING CLUSTER COMPARE RESULTS 50% 10% Balance within clusters
  12. 12. ARCHITECTURE LOAD BALANCING ▸ Send certain amount of traffic to each testing cluster ▸ Balance within each cluster: ▸ balancing schemes: round-robin, master-slave, random ▸ each server can have its own priority ▸ can detect if an upstream is down ▸ lazily resolve upstream names (DNS balancing)
  13. 13. ARCHITECTURE FOREIGN EXTERNAL SCANNERS RSPAMD PROXY STABLE CLUSTER Encrypt using HTTPCrypt Scan local file FOREIGN CLUSTER TESTING CLUSTER Use LUA script to parse results
  14. 14. ARCHITECTURE FOREIGN EXTERNAL SCANNERS ▸ Can scan external scanners, e.g. SA or Cloudmark ▸ Can evaluate their efficiency comparing to rspamd ▸ Use Lua filter to parse external scanners results
  15. 15. COMPARE EXAMPLES AN EXAMPLE OF COMPARISON SCRIPT return function(results) local log = require "rspamd_logger" for k,v in pairs(results) do if type(v) == 'table' then log.infox("%s: %s", k, v['default']['score']) else log.infox("err: %s: %s", k, v) end end end
  16. 16. FUTURE PLANS POTENTIAL FEATURES ▸ Balance not merely HTTP but also SMTP ▸ Perform retries when master connection fails somehow ▸ Use mirrors results if the whole stable cluster is dead ▸ Location based balancing (select the nearest or the fastest server among possible choices)

×