I've discussed spam filtering, and in particular the use of blacklist services, in the recent past. I've been testing spam filtering mechanisms and want to share some insights.
Although blacklist services do help reduce the amount of unwanted email your users might receive in their inboxes, they also introduce some amount of processing overhead. Blacklist filters rely on DNS lookups, so message processing time is increased by the number of DNS lookups per message along with any network lag time involved in those lookups. Heavily used blacklist services sometimes take up to 1 second or longer to respond to queries.
The order in which your mail filters are used can make a performance difference. You might be able to reduce processing lag time by performing blacklist queries after other, simpler processing has taken place. For example, you might have filters that use whitelists or look for foreign languages, various countries of origin, various character sets, banned word lists, nonstandard message formatting, malformed HTML, banned scripts, file attachments, etc. These types of filters can typically process mail much faster than filters that rely on network communication to outsourced services such as blacklist providers. These types of filters can also process mail much faster than typical Bayesian filtering systems, especially Bayesian filters that have accumulated a big database of tokens (a database that probably grows larger by the hour). Whether you use Bayesian filters before or after blacklist service filters probably depends on how much mail your server processes and what kind of processing power your server has as compared to the sum of network lag time between your network and the blacklist service providers.
Another thing I've found, which is probably to be expected, is that blacklist services tend to respond to DNS queries much faster at night (in the US) than they do during the day. This phenomenon is undoubtedly due to far more people picking up mail during the day then at night. Most server-based filtering solutions are probably on dedicated connections and therefore process mail any time of day or night. But when you factor in the millions of individual computer users who run desktop-based mail filtering solutions, it stands to reason that there is a much greater load on blacklist services during daytime hours.
If your mail filtering solution lets you adjust the priority or processing order of the various filtering mechanisms that it uses, consider testing to see which priority or order works best for your needs. You might find that the out-of-the-box configuration works better after some tweaking.
Register
