Rspamd

Fast, free and open-source spam filtering system.

Rspamd 1.7.9 has been released

2018-08-01 00:00:00 +0200

We have released Rspamd 1.7.9 today. There are no incompatible changes introduced with this version to our best knowledge.

The most important features and fixes

  • Ratelimits are reworked and now work as intended (and documented)
  • Clickhouse module supports data retention policies
  • Reworked C modules to avoid global contexts (simplifies leaks detection on reload)
  • Reputation plugin now supports SPF records reputation
  • WebUI code is now even more conformant to the modern JS standards
  • Maps are now distributed remotely with local file safety fallback to allow faster maps update without waiting for a new release
  • Antivirus module checks attachments only (as decoded content) in attachments_only mode to improve AV performance by hiding the mime content from them

Full list of the meaningful changes

  • [CritFix] Fix caseless comparison of equal length strings
  • [Feature] Add HTTP basic auth support to elastic and clickhouse plugins
  • [Feature] Add SPF selector to reputation
  • [Feature] Add support of the fallback backends for HTTP maps
  • [Feature] Allow to print full mime structure when extracting mime data
  • [Feature] Allow to split symbols in reputation plugin
  • [Feature] Check attachments only on AV scanners in attachments_only mode
  • [Feature] Disable all SSL checks if ssl_no_verify flag is set
  • [Feature] Implement parsing of scoped IPv6 addresses
  • [Feature] Improve rspamc counters output
  • [Fix] Add sanity checks when expanding SPF macros
  • [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
  • [Fix] Avoid one extra byte writing
  • [Fix] Deal with direct hash table
  • [Fix] Detect empty text part as text, not HTML
  • [Fix] Do not reduce map watch timeout for mixed http/file maps
  • [Fix] Fix HTML part detection heuristic
  • [Fix] Fix double free in redirectors cleanup
  • [Fix] Fix legacy history handling in the controller
  • [Fix] Fix messages insertion
  • [Fix] Fix sending string method
  • [Fix] Fix statconver command line arguments
  • [Fix] Fixed argument checking for being null
  • [Fix] Fixed issues reported by luacheck
  • [Fix] Freeze updates queue when do actual storage update
  • [Fix] HTTP map hash is per-backend and not per-map
  • [Fix] Plug memory leak in fuzzy updates
  • [Fix] Prefer ‘MTA-Name’ when producing authentication results
  • [Fix] Replace bad unicode sequences instead of stopping on them
  • [Fix] Set classifier version on learning
  • [Project] Reworked ratelimits
  • [Project] Apply topological sorting for symbols in Rspamd
  • [Project] Remove global contexts from C modules
  • [Project] Move performance critical hash tables to khash
  • [WebUI] Avoid unused indexes
  • [WebUI] Do not execute on_success callback
  • [WebUI] Fix history reset for “All SERVERS” (#2346)
  • [WebUI] Fix query URL for selected server
  • [WebUI] Fix symbols display in legacy history,
  • [WebUI] Hide symbols order selector for legacy history
  • [WebUI] Refactor query functions into one
  • [WebUI] Remove previously-attached event handlers
  • [WebUI] Save symbols to the selected server
  • [WebUI] Unify arguments of query functions
  • [WebUI] Use common query functions to get graph data
  • [WebUI] Use common query functions to save symbols

Rspamd 1.7.8 has been released

2018-07-12 00:00:00 +0200

We have released Rspamd 1.7.8 today. There are no incompatible changes introduced with this version to our best knowledge.

The most important features and fixes

  • Rspamd mime tool can now show you fuzzy hashes extracted from text
  • Fuzzy hashes are now updated when being hitted to prevent expiration of the important hashes
  • Fuzzy updates queue is now deduplicated that allows to reduce amount of Redis update requests by 10 times in some cases
  • HTTP maps are now cached on disk to provide preload on startup
  • WebUI code is now more conformant to the modern JS standards (special thanks to Alexander Moisseev)

Full list of the meaningful changes

  • [Feature] Add more extended statistics about fuzzy updates
  • [Feature] Add more non-conformant Received headers support
  • [Feature] Add preliminary function to get fuzzy hashes from text in Lua
  • [Feature] Allow to configure AV module rejection message
  • [Feature] Implement fuzzy hashes extraction in mime tool
  • [Feature] Improve WHITE_ON_WHITE rule
  • [Feature] Improve integer -> string conversion
  • [Feature] Reuse maps in multimap module more aggressively
  • [Fix] Avoid race condition in skip map as pool lifetime is not enough
  • [Fix] Eliminate all specific C plugins pools
  • [Fix] Fix DKIM check rule if DNS is unavailable
  • [Fix] Fix build where ucontext is defined in ucontext.h
  • [Fix] Fix crash in base url handling
  • [Fix] Fix descriptors leak in sqlite3 locking code
  • [Fix] Fix messages quarantine
  • [Fix] Fix padded numbers printing
  • [Fix] Fix race condition on maps reinit
  • [Fix] Fix regexp functions when no data is passed
  • [Fix] Fix specific urls extraction
  • [Fix] Fix styles propagation
  • [Fix] Improve resetting of the limit buckets
  • [Fix] Initialize sqlite3 properly
  • [Fix] Work with broken resolvers in resolv.conf
  • [Project] Implement HTTP maps caching
  • [Project] Refresh fuzzy hashes when matched
  • [Project] Add logic to deduplicate fuzzy updates queue
  • [WebUI] Add missed declarations
  • [WebUI] Avoid using “undefined” property
  • [WebUI] Do not accept passwords containing control characters
  • [WebUI] Do not redeclare variables
  • [WebUI] Enable strict mode,
  • [WebUI] Fix variable assignment
  • [WebUI] Initialize variables at declaration
  • [WebUI] Remove duplicated path from RequireJS config
  • [WebUI] Remove unused block
  • [WebUI] Remove unused variable
  • [WebUI] Remove unused variables
  • [WebUI] Use self-explanatory notation
  • [WebUI] Use type-safe equality operators

Rspamd 1.7.7 has been released

2018-07-02 00:00:00 +0200

We have released Rspamd 1.7.7 today. There are no incompatible changes introduced with this version to our best knowledge.

The most important features and fixes

  • Add rspamadm mime tool to do various email operations:
    • extract text/HTML content
    • extract statistical tokens
    • exctact URLs
  • Fixed encryption mode in Rspamd proxy
  • Fixed various crashes in maps during reload
  • Preload maps data before starting of the worker processes when possible
  • Better HTML styles processing: add ZeroFont exploit filtering rules
  • Fix ED25519 DKIM signatures as described by the latest RFC draft
  • Added crash reporting system via libunwind

Full list of the meaningful changes

  • [CritFix] Check NM part of pubkey to match it with rotating keypairs
  • [CritFix] Do not overwrite PID of the main process
  • [CritFix] Fix maps after reload
  • [CritFix] Fix maps race conditions on reload
  • [CritFix] Fix shmem leak in encrypting proxy mode
  • [Feature] Add a concept of ignored symbols to avoid race conditions
  • [Feature] Add ability to print bayes tokens in rspamadm mime
  • [Feature] Add method to get statistical tokens in Lua API
  • [Feature] Add preliminary mime stat command
  • [Feature] Add rspamadm mime tool
  • [Feature] Add urls extraction tool
  • [Feature] Address ZeroFont exploit
  • [Feature] Allow rspamadm mime to process multiple files
  • [Feature] Allow to extract words in rspamadm mime
  • [Feature] Allow to print mime part data
  • [Feature] Allow to show HTML structure on extraction
  • [Feature] Distinguish IP failures from connection failures
  • [Feature] Improve output for mime command
  • [Feature] Improve styles propagation
  • [Feature] Main process crash will now cleanup all children
  • [Feature] Preload file and static maps in main process
  • [Feature] Print stack trace on crash
  • [Feature] Process font size in HTML parser
  • [Feature] Propagate content length of invisible tags
  • [Feature] Read ordinary file maps in chunks to be more safe on rewrites
  • [Feature] Support base tag in HTML
  • [Feature] Support more size suffixes when parsing HTML styles
  • [Feature] Support opacity style
  • [Fix] Another fix for nested composites
  • [Fix] Fill nm id in keypairs cache code
  • [Fix] Fix colors alpha channel handling
  • [Fix] Fix destruction logic
  • [Fix] Fix double free
  • [Fix] Fix maps preload logic
  • [Fix] Fix nested composites process
  • [Fix] Fix proxying of Exim connections
  • [Fix] Fix reload crash
  • [Fix] Fix rspamadm -l command
  • [Fix] Update ed25519 signing schema
  • [WebUI] Stop using “const” declaration
  • [WebUI] Update RequireJS to 2.3.5