This guide will help you get started using the Incoming Filtering service.
This guide assumes you have purchased and provisioned a SpamExperts Incoming Filtering service.
Configuring MX Records
To use Incoming Filtering, you need to point the MX Records for the domain to the SpamExperts Hosted Cloud routes.
First remove any existing MX records from the DNS of your server.
Then set the MX records with the following priorities as SpamExperts describe:
- mx.spamexperts.com (Priority 10)
- fallbackmx.spamexperts.eu (Priority 20)
- lastmx.spamexperts.net (Priority 30)
Now you will need to allow DNS to propagate for SpamExperts to recognise the MX records have changed and to filter the emails.
No further configuration is necessary. An email will be routed through the SpamExperts Cloud System initially, Spam filtered out, and the resulting clean emails forwarded on to your local mail server.
Managing Quarantined Emails
The quarantine system can be accessed directly via IMAP or the SpamExperts Control Panel.
Messages that are rejected with a 5xx SMTP rejection code at SMTP level are quarantined. So legitimate sending servers will have informed the sender about the rejection. Enabling the quarantine is optional. If you do not want to use the SpamExperts quarantine , it's possible to configure the systems to deliver all messages to the destinaion mail-server and use "Subject Notation" instead. This will tag messages on the subject , so that filters can be used directly on the destination mail-server.
By default, SpamExperts stores the quarantined spam for 14 days, however on a Local Cloud setup customers may overrule this value. Spam messages that were temporarily rejected at SMTP level are not listed in the quarantine, and will be automatically retried by legitimate sending servers
Web Interface access
From the Control Panel you can either Release, Release & Train, Blacklist & Remove, Release and Whitelist, or Remove messages blocked as spam. In case an email has been incorrectly blocked, clicking 'Release & Train" will result in delivery of the false positive to the original recipient. It will also trigger a report about the wrong classification to our systems to further improve the filtering.
It's also possible to do all the above actions directly from the log search page. Using the drop down menu next to the blocked message will give you the available options.
IMAP access
The quarantine system is powered by an IMAP backend. The backend is accessible as super-administrator (using the special "global" account), domain (using the domainname), and recipient (using the email address). Customers using the SpamExperts hosted cloud can connect to quarantine.antispamcloud.com.
Users on a SpamExperts Local Cloud can use the hostname of the primary server in that cluster (or a CNAME associated with it). We have instructions available for Thunderbird and Outlook. Using IMAP you can also apply mass actions to specific emails by dragging to specific folders.
For Local Cloud users there is also a special "global" account which can be used to retrieve IMAP access to all quarantined messages. This is for super administrators only.
IMAP folders
There are several special folders in the IMAP account:
- Caught: All incoming messages which have been quarantined can be found in this folder. It's not necessary to report these emails as Spam again, as they have already been classified as such. These emails will automatically expire.
- Caught (Outgoing): All outgoing messages which have been quarantined can be found in this folder (optional). It's not necessary to report these emails as Spam again, as they have already been classified as such. These emails will automatically expire.
- Training Requested: This is only relevant if a special option as been activated via the Software API to store a copy of unsure classifications in this folder. These messages have been delivered, but can be manually dragged/dropped to "Not Spam" or "Spam" to further fine-tune the systems. Please contact support@spamexperts.com if you would like more information on this one.
- Release & Train: This is a write-only folder (which cannot be accessed by the email client). Emails drag/dropped from "Caught" to this folder will be delivered to the recipient and reported as a classification mistake to our central systems
- Release: This is a write-only folder (which cannot be accessed by the email client). Emails drag/dropped from "Caught" to this folder will be delivered to the recipient only.
- Not Spam: This is a write-only folder (which cannot be accessed by the email client). Emails drag/dropped from "Caught" or "Training Requested" to this folder will be reported as a classification mistake to our central systems (it will not be delivered to the recipient)
- Spam: This is a write-only folder (which cannot be accessed by the email client). Emails drag/dropped to this folder will be reported as a Spam to our central systems (it will not be delivered to the recipient). This is useful to report spam which was not blocked correctly directly from the email client.
Managing Message Queues
Generally emails are directly delivered to the destination server. However, if the delivery attempt to the destination server is returning a temporary failure, all email sent to known valid recipients are queued locally on the filtering servers for delivery retry. Emails which have been permanently rejected by the destination server with a 5xx error code, will NOT be queued and are rejected by the systems. If the destination mailserver acts as catch-all for the domain, no recipients will be cached and therefore email will only be queued if the valid recipients have been explicitly set as Local Recipient in the SpamExperts control panel. In case a recipient is not cached as valid, the message will be temporarily rejected and queued at the sender instead of on the filtering server (unless Local Recipients is enabled).
Queue Access
You can access the email queue from the webinterface, from which you can also manually retry delivery of a queued message.
Automatic Retry Schedule
Messages queued for known valid recipients because of temporary problems with the destination route (for example network problems) are automatically retried for delivery at the following approximate intervals:
- During the first 2 hours, delivery is retried at a fixed interval of 15 minutes.
- During the next 14 hours, delivery is retried at a variable interval, starting at 15 minutes and multiplying by 1.5 with each attempt (e.g. after 15 minutes, then 22.5 minutes, then 34 minutes, and so on).
- From 16 hours since the initial failure, until 4 days have passed, delivery is retried at a fixed interval of every 6 hours.
- After 4 days we generate a bounce to the sender. If the bounce cannot be delivered immediately, it will be frozen. After this time, delivery of the message will have permanently failed. Optionally via the Software API the 4 days can be overruled to a longer (or shorter) period.
When a message is frozen, (when a message can neither be delivered to its recipients nor returned to its sender) no more automatic delivery attempts are made. A super administrator can “thaw” ( force retry) such messages when the problem has been corrected.
SpamExperts caches valid recipients up to 4 days. When such entry has expired, SpamExperts will not queue email for such recipients and instead temporarily reject the message so it's queued on the sending server. The sending server in such case will automaticallty retry delivery. When using the "Local Recipients" feature, no caching is involved and SpamExperts continues to accept and queue the emails for all specified recipients.
Messages Queued
The SMTP RFC 5321 specifies a sending server must queue messages which cannot be directly delivered because of a temporary failure at the receiving end. Therefore in case of temporary issues with the email infrastructure, emails will not be bounced immediately but are instead queued on the sending server(s) and automatically retried for delivery. In case of downtime of the destination mailserver, messages are only accepted for delivery by the filtercluster if the recipient is known to be valid. Valid destination recipients are cached up to 96 hours.
When a destination server cannot be reached for 4 days, all messages will be bounced after 4 days and no new email will be accepted/queued until the destination server is back online. This 4 day period is conform the SMTP RFC. The reason it's not longer than 4 days, is because it's important for the sender to be aware that delivery of their message has been failing for 4 days so they can try and contact the recipient in another way.
Your own fallback server(s)
Please note that when you specify multiple destination routes, the SpamExperts system will assume you run your own fallback system. If the specified fallback server is not responding to recipient callouts, there will be no database built up of valid recipients internally. We recommend not to specify any fallback server(s) unless you've specifically designed your infrastructure to handle outages of the main destination server.
Troubleshooting messages in the queue
If there are messages stored in the queue there will always be a (temporary) error when delivering to the destination server. To investigate the issue:
- Verify you have set a correct destination route (also ensure there are not multiple destination routes specified, normally there should just be 1 route)
- Check the logs on your destination server to investigate why it's not accepting the delivery attempts
- Run a telnet test to check the response of your destination mailserver
- If after following these steps you still have an issue, please contact SpamExperts support providing a sample sender/recipient/date to investigate
Replying to messages in the queue
Whilst a message is queued you can reply to it directly from the webinterface by using the "Reply" option from the message options:
The resulting page allows you to immediately respond to the email, ensuring you can continue to manage your email whilst your destination mailserver environment is unavailable.
