Want to stop spam? Multiple techniques in unison is the answer - Internet

Computer Technology Review, Jan, 2004 by John Korsak

Spammers have proven to be both determined and technologically savvy. Despite miniscule response rates and the ire of email users everywhere, the number of people sending spam continues to grow. And, despite concerted efforts of businesses to block spam, it continues to sap productivity and drain resources. Spammers are leveraging technology not only to increase the number of messages they send, but also to thwart some of the rudimentary anti-spam approaches that are now in place at some businesses. Simply filtering on phrases such as "Get Rich Quick!" is no longer reliable, as many spammers now use HTML formatting tags to break up the message, disguising it from filters while leaving it readable to end users.

To fight spam effectively today, organizations must employ a multi-layered approach, which combines a broad set of techniques to turn spam's own objectives, characteristics, and defenses against itself. No one method can do it all. By combining a variety of techniques, businesses can create an exceptionally effective anti-spam barrier that is custom tailored to the particular needs of the organization. Organizations looking to reduce spam must consider the following techniques.

Connection Filtering

A key objective of spammers is to avoid being traced. The more anonymously they can send email, the more likely it is that they will be able to continue using the same systems and services they are using without threat of interruption. Connection filtering detects many of the methods that spammers use to avoid being traced and also includes mechanisms for blocking spam that comes from known spam senders. Connection filtering techniques identify spam by checking characteristics of the sending server and information presented by the sending server before it begins to transfer mail.

A common connection filtering technique is the use of Black Lists. Black lists are maintained by various organizations and are generally used to track IP addresses used to send spam. Black lists are often used to identify open relays, computers that allow anyone to send outbound email. Relay prevention requires the email server to know who is sending the message, or at least trust the IP address of the computer used to send the email. A large portion of the spam sent daily is sent using an open relay to help spammers hide their identities.

Other connection filtering techniques include reverse lookups, verifying computer names and verifying the from email address.

SMTP Filtering

After two mail servers establish a connection with each other, they initiate a dialog in which the sending mail server tells the receiving server who the next email message is from, and to whom it is being sent. During this SMTP (Simple Mail Transfer Protocol) exchange, the receiving server employs filtering rules to stop spam before it is received into the organization's mail system. SMTP filtering is similar to connection filtering but it relies more heavily on the information provided by the sending server, rather than the TCP/IP connection information.

At this stage the receiving email server can help prevent dictionary attacks, where spammers attempt to validate random email addresses through the use of the verify command (VRFY) or by faking an email to series of email addresses.

Content Filtering

Because the goal of all spam is essentially the same--selling or promoting a product or service--a great deal of spam content shares common characteristics. Certain words and phrases such as "Silk ties" or "Eliminate debt" appear with such frequency in spam that they can be used as excellent indicators of unwanted email. Other characteristics are also reliable spam identifiers, such as the call to action--"Find out how, click here"--or even the ubiquitous removal notification--"If you want to be removed from our mailing lists ...". Content filtering turns the spammers' need to promote and sell against them by analyzing the words, phrases, structure and URLs contained within an email message to separate spam from legitimate email.

Bayesian Filtering

With Bayesian statistical filtering, the words in an incoming email message are evaluated based on the frequency that they appear in spam and non-spam email. A probability is then calculated on the likelihood of the email being spam. The statistical filters can be updated with an organization's own sample of good and bad messages to improve the accuracy of the filter. One particularly effective way of helping the filter "learn" is to update it with any spam that it failed to identify on its first pass. Very quickly the filter will be able to improve its ability to accurately identify what constitutes spam for a particular business.

URL domain black list

The email server searches through the body of the message for specific URLs that have been cultivated from a large sample of spam. This is a very effective way to identify spam since all spam has some call to action that typically urges the user to visit a web site or another online resource.