Rspamd

Fast, free and open-source spam filtering system.

Rspamd 1.7.8 has been released

2018-07-12 00:00:00 +0200

We have released Rspamd 1.7.8 today. There are no incompatible changes introduced with this version to our best knowledge.

The most important features and fixes

  • Rspamd mime tool can now show you fuzzy hashes extracted from text
  • Fuzzy hashes are now updated when being hitted to prevent expiration of the important hashes
  • Fuzzy updates queue is now deduplicated that allows to reduce amount of Redis update requests by 10 times in some cases
  • HTTP maps are now cached on disk to provide preload on startup
  • WebUI code is now more conformant to the modern JS standards (special thanks to Alexander Moisseev)

Full list of the meaningful changes

  • [Feature] Add more extended statistics about fuzzy updates
  • [Feature] Add more non-conformant Received headers support
  • [Feature] Add preliminary function to get fuzzy hashes from text in Lua
  • [Feature] Allow to configure AV module rejection message
  • [Feature] Implement fuzzy hashes extraction in mime tool
  • [Feature] Improve WHITE_ON_WHITE rule
  • [Feature] Improve integer -> string conversion
  • [Feature] Reuse maps in multimap module more aggressively
  • [Fix] Avoid race condition in skip map as pool lifetime is not enough
  • [Fix] Eliminate all specific C plugins pools
  • [Fix] Fix DKIM check rule if DNS is unavailable
  • [Fix] Fix build where ucontext is defined in ucontext.h
  • [Fix] Fix crash in base url handling
  • [Fix] Fix descriptors leak in sqlite3 locking code
  • [Fix] Fix messages quarantine
  • [Fix] Fix padded numbers printing
  • [Fix] Fix race condition on maps reinit
  • [Fix] Fix regexp functions when no data is passed
  • [Fix] Fix specific urls extraction
  • [Fix] Fix styles propagation
  • [Fix] Improve resetting of the limit buckets
  • [Fix] Initialize sqlite3 properly
  • [Fix] Work with broken resolvers in resolv.conf
  • [Project] Implement HTTP maps caching
  • [Project] Refresh fuzzy hashes when matched
  • [Project] Add logic to deduplicate fuzzy updates queue
  • [WebUI] Add missed declarations
  • [WebUI] Avoid using “undefined” property
  • [WebUI] Do not accept passwords containing control characters
  • [WebUI] Do not redeclare variables
  • [WebUI] Enable strict mode,
  • [WebUI] Fix variable assignment
  • [WebUI] Initialize variables at declaration
  • [WebUI] Remove duplicated path from RequireJS config
  • [WebUI] Remove unused block
  • [WebUI] Remove unused variable
  • [WebUI] Remove unused variables
  • [WebUI] Use self-explanatory notation
  • [WebUI] Use type-safe equality operators

Rspamd 1.7.7 has been released

2018-07-02 00:00:00 +0200

We have released Rspamd 1.7.7 today. There are no incompatible changes introduced with this version to our best knowledge.

The most important features and fixes

  • Add rspamadm mime tool to do various email operations:
    • extract text/HTML content
    • extract statistical tokens
    • exctact URLs
  • Fixed encryption mode in Rspamd proxy
  • Fixed various crashes in maps during reload
  • Preload maps data before starting of the worker processes when possible
  • Better HTML styles processing: add ZeroFont exploit filtering rules
  • Fix ED25519 DKIM signatures as described by the latest RFC draft
  • Added crash reporting system via libunwind

Full list of the meaningful changes

  • [CritFix] Check NM part of pubkey to match it with rotating keypairs
  • [CritFix] Do not overwrite PID of the main process
  • [CritFix] Fix maps after reload
  • [CritFix] Fix maps race conditions on reload
  • [CritFix] Fix shmem leak in encrypting proxy mode
  • [Feature] Add a concept of ignored symbols to avoid race conditions
  • [Feature] Add ability to print bayes tokens in rspamadm mime
  • [Feature] Add method to get statistical tokens in Lua API
  • [Feature] Add preliminary mime stat command
  • [Feature] Add rspamadm mime tool
  • [Feature] Add urls extraction tool
  • [Feature] Address ZeroFont exploit
  • [Feature] Allow rspamadm mime to process multiple files
  • [Feature] Allow to extract words in rspamadm mime
  • [Feature] Allow to print mime part data
  • [Feature] Allow to show HTML structure on extraction
  • [Feature] Distinguish IP failures from connection failures
  • [Feature] Improve output for mime command
  • [Feature] Improve styles propagation
  • [Feature] Main process crash will now cleanup all children
  • [Feature] Preload file and static maps in main process
  • [Feature] Print stack trace on crash
  • [Feature] Process font size in HTML parser
  • [Feature] Propagate content length of invisible tags
  • [Feature] Read ordinary file maps in chunks to be more safe on rewrites
  • [Feature] Support base tag in HTML
  • [Feature] Support more size suffixes when parsing HTML styles
  • [Feature] Support opacity style
  • [Fix] Another fix for nested composites
  • [Fix] Fill nm id in keypairs cache code
  • [Fix] Fix colors alpha channel handling
  • [Fix] Fix destruction logic
  • [Fix] Fix double free
  • [Fix] Fix maps preload logic
  • [Fix] Fix nested composites process
  • [Fix] Fix proxying of Exim connections
  • [Fix] Fix reload crash
  • [Fix] Fix rspamadm -l command
  • [Fix] Update ed25519 signing schema
  • [WebUI] Stop using “const” declaration
  • [WebUI] Update RequireJS to 2.3.5

Rspamd 1.7.6 has been released

2018-06-15 00:00:00 +0200

We have released Rspamd 1.7.6 today. There are no incompatible changes introduced with this version to our best knowledge.

The most important features and fixes

  • Fix multiple neural networks support: it is now possible to learn multiple neural networks with different settings as documented
  • Rework rspamadm to use mostly Lua for subcommands for better documentation and extensions support
  • Add pubkey checks for dkim_signing module (#2277)
  • DMARC reports are now compressed using gzip as suggested by RFC
  • Settings module can now skip message processing to improve performance
  • Bayes classifier now consider more metatokens from the headers
  • ED25519 DKIM signatures are now supported
  • Fixed serious issues with composites, maps and other components
  • Major memory leak hunting and eliminating (especially those that occurs during reload)
  • Add more tests and allow to create fake DNS records to make certain tests self-contained (e.g. DKIM or DMARC)

Full list of the meaningful changes

  • [CritFix] Fix multiple neural networks support
  • [Feature] Add decryption function to keypair command
  • [Feature] Add gzip compression for HTTP requests in elastic module
  • [Feature] Add gzip methods to lua util
  • [Feature] Add maps based on Top Level Domains
  • [Feature] Add pubkey checks for dkim_signing
  • [Feature] Add support of fake DNS records
  • [Feature] Add tool to encrypt files
  • [Feature] Allow to add symbols using settings directly
  • [Feature] Allow to match private and public keys for DKIM signatures
  • [Feature] Allow to set task flags via settings
  • [Feature] Allow to specify fake DNS address from the config
  • [Feature] Implement signatures verification using rspamadm keypair
  • [Feature] Implement signing using rspamadm keypair
  • [Feature] Improve error reporting for DKIM key access issues
  • [Feature] Provide $HOSTNAME variable in UCL
  • [Feature] Rework levenshtein distance computation
  • [Feature] Split message parsing and processing
  • [Feature] Support ED25519 DKIM signatures
  • [Feature] Support encrypted configs in UCL
  • [Feature] Suppress duplicate warning on very large radix tries
  • [Feature] Use OSB to combine header names
  • [Fix] Cleanup maps data on shutdown
  • [Fix] Fix ‘~’ behaviour in composites
  • [Fix] Fix HTTP maps updates
  • [Fix] Fix NIST signatures
  • [Fix] Fix RFC822 comments when processing a mime address
  • [Fix] Fix double free
  • [Fix] Fix dynamic settings application
  • [Fix] Fix for CommuniGate Pro maillist
  • [Fix] Fix keypair creation method to actually create keypair…
  • [Fix] Fix matching patterns with no paths
  • [Fix] Fix memory leak in parsing comments
  • [Fix] Fix parsing of urls with numeric password
  • [Fix] Fix plugins intialisation in configwizard
  • [Fix] Fix potential crash on reload
  • [Fix] Fix potential race condition for a finished HTTP connections
  • [Fix] Fix race-condition leak on processes reload
  • [Fix] Fix signing in openssl mode
  • [Fix] Free language detector structures
  • [Fix] Relax alignment requirements
  • [Fix] Send DMARC reports compressed
  • [Fix] Try to fix leak in dmarc module
  • [Fix] Try to plug memory leak in metric exporter
  • [Project] Convert rspamadm subcommands to Lua
  • [WebUI] Display smtp sender/recipient in history
  • [WebUI] Fix elements disabling in “Symbols” tab
  • [WebUI] Limit recipients list in history column to 3
  • [WebUI] Match envelope and mime addresses following in arbitrary order
  • [WebUI] Update column header
  • [WebUI] Wrap addresses in history