Greylisting

from Wikipedia, the free encyclopedia

The term gray list or greylisting (British) or graylisting (USA) describes a form of spam control for e-mails in which the first e-mail from unknown senders is initially rejected and only accepted after a further delivery attempt.

Greylisting is both a method of detecting spam and a method of notifying the sender of rejected emails.

functionality

If an SMTP server is contacted so that it can receive an e-mail, this mail server knows the following three data before the mail server has to accept the e-mail (the "SMTP envelope"):

  1. IP address of the sending mail server
  2. The sender 's email address according to this (see Envelope Sender )
  3. E-mail address of the addressees

If an e-mail with this combination of addresses has never been received, the delivery attempt is blocked by the SMTP server during greylisting with a message that a temporary error has occurred and that the SMTP client should try the delivery again later. The next time an attempt is made to deliver an email with the same combination of data (which a regular and RFC- compliant SMTP server should definitely do), this email is accepted (after a configurable time interval). Whether and when a new delivery attempt is made depends solely on the sender. There are also greylisting implementations that relax the rules a bit, e.g. For example, the domains involved can be entered and checked instead of the e-mail addresses.

advantages

Typical software for mass sending e-mails (especially worms or Trojans) often does not attempt to deliver a (spam) e-mail to the same SMTP server a second time. Such e-mails are successfully filtered by "greylisting". It is currently very effective in combating spam, reducing spam to up to a tenth.

Due to the delayed delivery, spam detection methods based on network checks are also more effective (such as RBLs , Vipul's Razor and DCC ), since the spam wave may already have been recognized between the first and second delivery attempts and entered on the corresponding blacklists.

An e-mail can be rejected if only the e-mail envelope with sender and recipient data has been received and not only after the complete e-mail (with body and, if applicable, attachments) has been received. In this way, additional spam filters such as B. Spamassassin is not burdened with a rejected e-mail, which saves considerable resources.

In contrast to heuristic spam control procedures, “greylisting” does not normally result in the loss of emails. Most greylisting implementations maintain a dynamic whitelist . After a successful e-mail delivery, the combination of sender, recipient and e-mail server is entered in the whitelist. Combinations that are noted in the whitelist bypass greylisting, which means that the e-mail is delivered on the first attempt. If e-mails are repeatedly sent between two people, this is not hindered by the greylisting.

disadvantage

(Incorrectly configured) mail server programs cannot make any further delivery attempts in the event of temporary errors. The responsible administrator of the sending mail server should be encouraged to correct this gross misconfiguration of his system. Furthermore, many greylisting implementations offer a whitelist, which, however, should be used for legitimate rather than incorrect senders, for example for whitelisting large providers. A useful whitelist is, for example, the DNSWL. The proportion of spam, which has increased again due to the high proportion of forged sender addresses, can be reduced by using SPF .

Another disadvantage is the time lag. A desired email can arrive a few minutes or hours later thanks to the greylisting. Anti-spam systems can, however, keep desired e-mail domains or senders available so that the time delay only affects the first e-mail.

Some mail server programs generate a preliminary delivery report for the sender the first time an e-mail is rejected by greylisting. Often times, this report is not read carefully or is not understood and is therefore often treated as a report of a definitive delivery failure.

As with all methods of combating spam, greylisting becomes less efficient as the spam software continues to develop. The more greylisting spreads, the more the spammers will adapt. It therefore makes sense to use other methods such as SPF or DKIM at the same time .

It should also be noted that if possible, all mail servers responsible for a domain have activated greylisting, as spam senders already today often use the - often poorly protected - MX with the lowest priority for delivery.

In the case of implementations on cluster servers, it must be ensured that the greylist database is replicated on all server nodes, otherwise the receipt of mail can be severely delayed.

In addition to the principle-related delay in delivery by a few minutes to a few hours, there were significant delays in e-mails from T-Online customers in mid-June 2009. For example, if T-Online rejected a delivery attempt due to greylisting, the second delivery was only made with a delay of 12 hours to more than 5 days. T-Online initially referred to the non-RFC-compliant behavior of the rejecting mail server, but then eliminated the delays.

field of use

To distinguish between the terms bulk e-mail (UBE) and commercial e-mail (UCE), see Types of Spam .

Greylisting can be used against UBE because it is usually based on infected PCs that are connected to botnets. Against this target group, it also helps to generally reject e-mails from dial-in access. Lists of dial-in access must be constantly updated.

As a rule, greylisting is not applicable against UCE. Since it can be a personal decision which of it is spam or not, blacklisting is the most accurate solution; Automatic spam filters help relatively well.

Changing server address

Some larger mail server operators distribute the sending of their customers' emails to several servers with different IP addresses in order to cope with the load of the emails to be sent. Depending on the system configuration of this server farm, regular delivery attempts from different sender addresses can then occur if they are repeated, but they all come from the IP address range of the one mail server operator. Some greylist programs offer the option of weak greylisting in order to partially compensate for the associated problems. For example, in the case of IPv4 addresses, the last byte of the address cannot be taken into account in the evaluation. This is based on the assumption that the different servers for sending e-mails are in a common address range with the network mask / 24. When using IPv6 , other methods must be used.

Adaptation of the spammers

In the period from 03/2008 to 08/2009, insignificant waves of spam were sent via botnets, which repeated the insertion of spam, possibly for test purposes. Features of the multiple objections were:

  1. 3 times in parallel or one after the other from the same PC
  2. 3 times in parallel from several worldwide distributed PCs
  3. Throw-in with permuted senders and content
  4. 2 times in succession at a greater distance from the same PC

Accidental or intentional - none of the waves was designed to circumvent a greylisting as described here. The combination "own PC with the same sender and time interval" was simply not included. Greylisting was the most accurate spam filter for UBE until October 2010.

Since the end of October 2010, a small proportion of UBE spam from botnets has also been sent several times, and since October 11, 2011, a notable one. Greylisting is still a criterion for the spam filter, but is no longer the only solution.

RFC compliance

The RFCs relevant for greylisting are RFC 5321 , which defines the SMTP protocol , and RFC 6647 , which is dedicated to greylisting. In RFC 5321 , mail is temporarily rejected at the discretion of the server operator ( mailbox ... temporarily blocked for policy reasons ).

Notification of the sender

If an SMTP server first accepts e-mails and only then filters spam, it cannot notify the sender: the sender information in almost all spam e-mails is falsified and the notifications would go to innocent victims .

The SMTP server can, however, initially accept the mail with the envelope, header and content and deliver a temporary error instead of the confirmation. If it has completed the spam check by the next delivery attempt, it returns a 250 (OK) or 5xy (error) response.

It is even conceivable that the recipient personally sorts the emails, so that the check only takes place after hours.

By rejecting the e-mail when it is thrown in, the real sender of the e-mail is usually notified. As a result, the recipient is no longer forced to look through a spam folder. In the event of an erroneous refusal, the sender will follow any instructions in the error message or pick up the phone.

See also

Web links

Individual evidence

  1. ↑ dnswl.org: DNS Whitelist - Protect against false positives
  2. ↑ t-online.de: T-Home team's opinion on delivery problems with active greylisting from June 14th and 24th, 2009