E-mail Forwarders, Spam, and Blacklisting
A note about forwarders
This article goes into detail about e-mail forwarders. Please keep in mind that this only applies to e-mail forwarders that forward mail off of the server. This does not include forwarding mail within the same domain name. If you have e-mail forwarders set up that forward within your domain name, then this would not apply to you. For example, you may have one POP/IMAP account set up on yourdomain.com, but have five or six e-mail forwarders forwarding to that one POP/IMAP account. This is fine and completely safe. Forwarding within the same domain does not subject your messages to the same environments that forwarding your mail off of the server will.
To a lesser extent this also does not apply to any other domain or account that may be hosted by us. However, arrangements may need to be made to accommodate this type of setup.
Lately we have been experiencing some issues regarding the blacklisting of our servers by other remote mail servers due to spamming. When we investigate almost all of these cases, the block is instigated because of e-mail forwarders set up on the server that are forwarding mail off of the server to another e-mail service. This means that spam messages are also forwarded off of the server and those remote mail servers see our server as being the server responsible for sending out that spam message. This leads to server blacklisting. Some additional information is available in our guide detailing the ongoing forwarder problem.
We have developed a method of scanning messages for spam, and rejecting those messages if that address forwards off of the server and the server deems the message as spam. There appears to be some confusion over what exactly this does. This article will attempt to explain the function of this spam fighting method in more detail.
In the examples and details that are listed below, the mail service hotmail.com is used a lot. It should be pointed out that these issues do not apply to just hotmail.com, but to any remote mail server or e-mail service. A remote mail server or e-mail service is any mail server that is not associated with your domain name. The same principles listed here apply to any other remote mail server such as Yahoo, Gmail, Verizon, Bellsouth, Comcast, etc. This article is not meant to imply that only one or a handful of these remote mail servers are affected by this. Instead this article applies to all remote mail servers and all services where you might be forwarding mail off of the server. The hotmail.com service is just meant to be used as an example.
Any time an e-mail message is sent, there are basically two parties involved, the sender and the recipient. This also means that there are at least two servers involved, the sending server or the outgoing mail server that the sender used to send out the message and the receiving server the mail server that received the message sent from the sender. In this manner a typical e-mail delivery process will look something like:
In this image the message is sent by the sender who sends the message to their sending server. Typically this sending server may be the sender's ISP or other mail server. Once the message reaches the sending server it goes out through the Internet through a series of networks until it reaches the receiving server. Once the message reaches the receiving server, the receiving server is responsible for delivering that message to the individual recipient whom the message was intended for.
By using an e-mail forwarder this process becomes a little bit more complicated. An added entity, a forwarding server is added to the mix. In terms of web hosting a forwarding server would be the server hosting your domain name. For example if you have joe@yourdomain.com forwarding mail to joe@hotmail.com then joe@hotmail.com becomes the recipient and yourdomain.com becomes the forwarding server because mail sent to joe@yourdomain.com forwards that message on to joe@hotmail.com. This is detailed below:
Here the sender is going to send a message to an address that forwards. The message goes from the sender onto the sending server where it reaches the Internet and reaches the forwarding server. Had this message not been sent to an address that forwards, this is where the message would end, and the diagram would look exactly like the first image above. However, because the address is a forwarder, the message has not yet reached its final destination. The forwarding server determines that this message is intended for another recipient on another server, so it sends the message out through the Internet to the receiving server where the message is accepted and the receiving server delivers the message to the final recipient. As you can see this adds a point of complexity to the message delivery process. The forwarding server, your domain, our server, becomes a like a middle man throughout the entire e-mail delivery process.
It is important to note that each identifiable server can only determine that the message came from the last server in the e-mail delivery process. For example, when the message reaches the sending server that server knows that the message was sent from the sender. When the message reaches the forwarding server that server only knows that the message came from the sending server. When the message reaches the receiving server that server only knows that the message came from the forwarding server. This is where issues involving spam begin to arise.
Consider that if you only have a single e-mail address, bob@yourdomain.com. You have this e-mail address set up as a POP account and when someone writes you at bob@yourdomain.com that message is stored on our server, on your hosting account, until you check that mailbox by logging into the POP3 server, the IMAP server, or through webmail. When an e-mail is sent to your bob@yourdomain.com e-mail address that message will follow the process:
That is, a sender will send a message to bob@yourdomain.com, which will go through the sending server out through the Internet, onto the receiving server (our server) and finally delivered to the recipient or your bob@yourdomain.com mailbox on the server.
If this sender was a spammer and was sending you a spam message, the process would look similar except the message would stop just before reaching the recipient:
Here the message is stopped just before reaching the recipient on our server. What is done at this step is determined by how you have your individual SpamAssassin setup configured. You may have SpamAssassin configured to flag the message by rewriting the subject. You may have SpamAssassin set up to deliver this message into your spam box. Or you may have SpamAssassin set up to automatically delete this message. The point is, regardless of how you have SpamAssassin set up, no harm is done to the server. Our server never becomes blacklisted and there is no risk that our server will ever be blacklisted. What happens to this spam message is completely in control of the account owner, the individual that owns yourdomain.com.
Now consider that if you have bob@yourdomain.com set up to forward mail to bob@hotmail.com. If you have this set up, then the process a message will take will be similar to:
That is the sender will send a message to their sending server. The message will reach the Internet and come to our server, the forwarding server. Our server will then forward that message on through the Internet to the receiving server or in this example the hotmail.com mail servers. Once there, the hotmail.com mail servers will deliver the message to the recipient or bob@hotmail.com.
Again, lets consider the scenario where this message is a spam message. The message will continue its traversal through this same path, until it gets to the receiving server or hotmail.com's mail servers.
Here, the spam message is blocked from being delivered to the recipient. The hotmail.com mail servers view this message as spam, and rightfully so, and do not deliver the message to the recipient. This may seem like a good thing, but remember, the receiving server or hotmail.com's mail server can only see and know that the message was sent from the last server that sent the message, or the forwarding server, which in this case is our server, the server hosting yourdomain.com.
So according to the receiving server, in this case hotmail.com, this spam message was sent from the forwarding server or yourdomain.com, our server. Because hotmail.com has such a large user base and because this user base is always wanting to receive less spam, the hotmail.com servers have to try and limit the amount of spam that comes into their servers. They do this by blocking spam sources or mail servers that they believe are sending spam. In this particular case, even though our server did not originally send this spam message, our server will be considered a spam source. This will lead hotmail.com to blacklist our server. This is generally why forwarding your mail off of the server is such a bad idea.
This issue with e-mail forwarders, spam, and blacklisting is really starting to become a problem. If an e-mail address on our server is forwarding to a hotmail.com address and that causes our server to become blacklisted at hotmail.com's mail server, then this means that nobody, including the e-mail forwarders, would be able to write a hotmail.com address from the server. If a forwarder causes the server to be blacklisted, then this is not fair to another user on the server who may have a friend with a hotmail.com address and they would be unable to communicate.
Originally it was thought that we had two choices in regards to this issue. We could either continue to allow these e-mail forwarders to be in place and continue to have issues with our server becoming blacklisted or we could disabling e-mail forwarders and not allow e-mails to be forwarded off of our server. The former did not seem to be fair to our users who are not forwarding e-mails, but the latter seemed to be too strict. To combat this, our developers created a compromise. This compromise only applies to e-mail addresses that forward off of the server. When a message comes into our server destined for an address that forwards off of the server, that message is scanned for spam. If our server determines that the message is spam, then the message is rejected and not forwarded on. If the message is not determined to be spam, then it continues on to its destination. In this manner, we are able to reject most spam messages, lessening the chance that our server would be viewed as a spam source by remote mail servers, while allowing users who are forwarding their mail off of the server to continue to use that forwarding option.
This new spam scanning algorithm only applies to address on our server that are forwarding off of the server. To illustrate this I will take the diagram of an e-mail messages process that involves an e-mail forwarder and cut it off once it reaches our server or the forwarding server.
So again, if the sender is sending a message to an address, such as bob@yourdomain.com that forwards off of the server, this is what the e-mail process will look like. The message will go through the sending server onto the Internet and finally reaching our server, the forwarding server. Here the message will stop momentarily. At this stop, our server will perform a series of test:
- Is the message addressed to an address that forwards off of the server? If no, then the message is addressed to a POP/IMAP account on the server and deliver it accordingly. If yes, then proceed to the next step.
- Scan the message for spam. Does the spam scanning determine that this message is spam? If yes, then reject the message and prevent the message from being accepted onto our mail queue and forwarded on to its destination. If no, proceed to the next step.
- The message was scanned for spam and determined to not be a spam message. Continue on with forwarding this message.
The test is really very simple. If the message is determined to not be spam, then the forward continues on as usual:
There is still the off chance that the receiving server would determine that the message is spam, but this method certainly limits that chance and helps weed out spam from ever reaching the receiving server through an e-mail forwarder.
Please understand that you really should not notice any major changes to your e-mail because of this new function. The only difference you will notice is that you may see less spam in your mailbox. There is always the chance that a legitimate message might get caught as spam and rejected, but it should be stressed that if you were using a POP/IMAP account on the server, then this message would not be scanned or rejected because of spam (depending on how you have your SpamAssassin setup configured, something YOU control). If a legitimate message is determined to be spam by our system, then it's a good bet that once the message reaches the receiving server then it would also be deemed as spam, which increases the chance that our server would be blacklisted. This is what you need to consider if you receive a report from someone concerning their message being rejected because of spam.
One major note to consider concerning this spam scanning function. If you have your default box set to forward mail off of the server, then this subjects all of the e-mails addressed to yourdomain.com to this spam scanning, regardless of whether or not the e-mail address the message is addressed to is a POP/IMAP account on the server or a forwarder. Really you should not be forwarding your default box off of the server. Your default box is just going to receive a lot of spam, and forwarding your default box off of the server just increases the chances that spam sent to this default box will lead to the server becoming blacklisted because of the default box. More information concerning the default box is available in our Default Address Guide. If you don't want this spam scanning affecting all of your domain's e-mail addresses, then you should not forward your default box off of the server.
It should also be noted that this spam scanning method is not in place on all of our servers. Currently we are just testing this method on a few of our servers and gathering results. So far the results have been very positive and very helpful to the overall health of the servers.