The Grinch that stole Quarantine Reports

No boys and girls, The Grinch didn’t steal the Christmas Quarantine Reports from Santa ExchangeDefender. They were just delayed while we ran some final tests on the new reporting engine. All OK.

The quarantine reports were sent out at 5:00 PM GMT on Monday, December 25th, 2006. Tonight they will resume their regularly scheduled reporting time of 8:00 AM GMT. The new reporting engine has already improved the performance of ExchangeDefender and opens up possibilities to integrate many features you have been asking for in 2006. We will soon publish our product roadmap for Q1/2007 and welcome input and suggestions from all of you.

Now back to killing SPAM.

ExchangeDefender Report Management Updates

For Christmas this year Santa (Pablo Averbuj) Claus will be bringing our customers a new report system management update. We expect to start on Christmas eve, December 24th at 6 PM GMT and be done by midnight.

There will be no downtime associated with this upgrade and no mail will be delayed. The upgrade will however make some sections of the system inaccessible while your particular domain is being upgraded. Because the upgrade requires us to migrate large collections of data from one database to another we do not want to give you incomplete information during the switch – so if you get your timing just right you might see “We’re moving, come back in 5 minutes” banner when you access your queue reports.

And yes, we’re really doing this on Christmas eve.

ExchangeDefender Begins Managed Tar-Pitting & Back-Scatter

Brilliant title, I know. But give me a chance to explain. Tar pitting and back scatter protection are two technologies that we have been working on for quite some time and in all current tests believe them to be ready for production. First the definitions:

Tar-pitting — The process of selectively delaying a HELO/EHLO response to an open SMTP connection.

Back-scatter — The bounce notifications of messages you never sent to begin with.

Tar-Pitting Protection

Tar pitting is simply a flow control mechanism. By default, SMTP servers answer and handle all connections at full speed. When a remote mail server connects on port 25 the destination mail server responds with its welcome banner and waits for the SMTP dialog to start, eventually leading to message delivery. Because SMTP is unaware of any reputation scoring it accepts communication from anyone in its access list and works as fast as the network and system resources allow. Because all SMTP servers by default work as fast as they possibly can it stands to reason they would become perfect spam targets for spammers that rely on sending as many messages in as little time as possible.

Enter tar-pitting. This mechanism is not new but until now presented significant shortcomings in terms of reputation. For example, your standard tar-pitting mechanisms either had a pre-configured timeout or timed out depending on how many connections the remote server opened. System administrators could configure their own tar-pitting settings which usually ended up in two unforeseen circumstances:

• The server would wildly fork and keep open processes listening on port 25 until all system resources were exhausted.
• The server would not recognize whitelists or reputation – applying the same throttling process to all mail server. So a large AOL mail exchanger that likely delivers thousands of messages a day to your organization would be greeted with the same delay as the SBS server that only delivered a few every other week.

Both approaches were faulty and resulted in a significant mail delay as well as performance degradation.

ExchangeDefender Tar-pitting

ExchangeDefender approach adapts our scoring and reputation technology. Servers become trusted over time and in real-time. Our research indicated that there are two polar types of mail servers out there: massive mail gateways and low volume mail servers. Massive mail gateways can be evaluated and over time trusted or whitelisted to get immediate service and reduce the amount of time they would have to wait for delivery. It stands to reason that a high activity mail server legitimately sends large volumes of mail over time.

However, if a low volume mail server started attempting thousands of connections per minute to our network we would get suspicious. Enter tar-pitting. By throttling low volume mail servers we can reduce overall network load and allow our spam filters to catch up in case these low volume mail servers were compromised to send malware, viruses, trojans or spam.

Back-scatter Protection

Back-scatter is simply the returned message (“bounce”) from a mail server that replied to a message you never sent. This threat has been around since the early days of commercial Internet in form of Joe Job attacks. In a Joe Job attack a spammer would assume identity and forge (“spoof”) a message to a remote recipient. The remote recipient would get upset at the “forged” identity and spammer would be in the clear.

Many faulty spam control mechanisms and Microsoft Outlook / Outlook Express exploits over the years created the environment in which trust was used to implicitly bypass spam filtering. Today when a computer is compromised by a virus or a trojan it can access files on the computer, mailboxes, email messages and more. By scanning the system for email addresses the trojan can build an impressive mailing list and then start spreading itself to those remote systems. The trojan assumes an identity of one of the email addresses on the system and uses it to reach another.

Think of all the email addresses stored on your computer in your address books, text files, accounting packages, etc. Most of them are outdated or gone, some may have their mailboxes overflowing, others have been disabled or have autoresponders on them. Viruses, trojans and other automatically generated content for the sake of spreading around the Internet does not care to validate the existence of the email – it only cares about getting as wide of a distribution as possible.

Thats where back-scatter comes into picture. All those bounced messages can come back into your mailbox to clog it with NDRs or even worse, infected messages that can compromise your security.

ExchangeDefender Back-scatter

Because our mail servers track where you send your messages we can also validate all the back scatter as legitimate or not. Typical non-delivery receipt (“NDR”) lists the subject, time, date, recipient email address and other information that is only known by the sending system and the recipient system. Considering that we control that sending mechanism, we can validate whether the NDR was caused by a message sent from our network or if it is simply a spoof.

Either way, we keep your mailbox clean. 

Implementation

We will begin the wide-spread implementation of tar-pitting and back-scatter protection this week. We have already tested the system adequately and believe that the reduced mail load over the holiday break will allow us to adapt the new processes while the network has more than its usual amount of spare resources.

No change will be required on your mail server and your users and customers will not see a change in the way that mail is being delivered our routed.

ExchangeDefender eliminates Image SPAM

Spam wars tend to evolve over time. Initially, SPAM looked just like the offers you still find in your fax machine – direct, informative, actionable. You’re almost pushed to buy something with countless incentives, promotions and reinforcement of just what a great deal you’re getting. We eliminated a bulk of this years ago through use of Bayesian analysis, or text patterns, found in the SPAM messages. Notice how when you get a spam you can tell within seconds that it is garbage just by glancing at its formatting?

The second evolution of SPAM was when it became convenient to make a purchase. No longer were you sold and promoted to but just asked to click on a link and proceed to buy that latest watch or drug. We eliminated those easily through the use of URIBL, specific blacklists of URL (web site addresses) and additional HTML analysis.

The latest evolution of SPAM has been the most difficult to isolate by far. You’ve seen dozens of these in your inbox nearly every day: Image SPAM. The email is very easy to characterize, it has a big gif or jpeg image followed by paragraphs of garbage text. At first, there was just an image – which contained text that used to be a part of the SPAM you’ve been receiving for years. Except because it was stored in an image it bypassed all SPAM filters. Fine, we easily discarded messages that contained no text. Then spammers started adding text. No problem, we eliminated them by calculating the ratio of the screen being taken up by text vs. image. Think about how often you get an email message that starts with an image that takes up most of your screen? Easy solution. Following the natural evolution of the spam war, image spam became harder and harder to detect.

We have finally come up with a set of solutions that effectively eliminates nearly all known strains of Image SPAM:

• All inline JPG and GIF images are OCR’ed. By using optical character recognition we can convert the image into plain text and determine whether it is SPAM or not.
• Parsing JPG and GIF image info. Each picture has series of image attributes, such as the Camera maker, model, F-Stop, Max aperture and so on. Dynamically generated image spam does not.
• Finally, we have spent the past month developing an image footprint database.

Image footprint database is something exclusive to ExchangeDefender. We strip known SPAM messages from our honeypot (public email addresses that only exist to collect SPAM) and store the known images in spam into a database. We then run analysis on them and compare all new incoming messages against the known samples of SPAM.

OCRing images is very expensive in terms of processor cycles and as expensive as it is for us to analyze each incoming message it is even more expensive for the spammers to create these images for each SPAM they send out. They create a single SPAM message that is then broadcast millions of times – and we’re ready for it!

So thank you for your continuing support of ExchangeDefender and as always, we’ll keep your mailbox clean for you.

Exporting Exchange SMTP Addresses

One of the most common Exchange questions we get is “How do I export all the SMTP addresses on my Exchange server?”

First of all, SMTP addresses are part of the User object in the Microsoft Active Directory. Active Directory is an LDAP-based database and by using LDIF (LDAP Data Interexchange Format) you can run queries against this database. You can use ldifde to accomplish this.

The following ldifde syntax will export the proxyaddress list for each user object in the directory on the server SERVERNAME.DOMAIN.TLD:

ldifde -f C:\DATA.ldf -s SERVERNAME -d “dc=DOMAIN,dc=TLD” -p subtree -r “(objectClass=user)” -l “cn,proxyaddresses”

The data is then dumped in C:\DATA.ldf. But suppose your server name was SBSBOX.MYDOMAIN.LOCAL. The syntax changes to:

ldifde -f C:\DATA.ldf -s sbsbox -d “dc=mydomain,dc=local” -p subtree -r “(objectClass=user)” -l “cn,proxyaddresses”

If you open it up with notepad you’ll see a number of entries in this file:

dn: CN=Administrator,CN=Users,DC=MYDOMAIN,DC=LOCAL
changetype: add
cn: Administrator
proxyAddresses: smtp:Administrator@MYDOMAIN.LOCAL
proxyAddresses: SMTP:Administrator@mydomain.com
proxyAddresses: smtp:postmaster@mydomain.com
proxyAddresses: X400:c=US;a= ;p=MYDOMAIN;o=Exchange;s=Administrator;
proxyAddresses: smtp:postmaster@MYDOMAIN.LOCAL

Nothing very private in here, just the display name and a few containers along with the proxyAddresses. Now another interesting bit – notice how there are both SMTP and smtp addresses here? What is the difference you may wonder? SMTP is the default SMTP address and smtp are the additional addresses and aliases.

The process is quick and convenient but unfortunately it does not export proxyAddress attributes from other objects such as distribution lists and public folders. For that you can use a third party script, such as this one that our friend emailed over. Simple vbScript that will create a list of users and addresses in C:\EmailAddresses.txt.

Pick your way but as you can tell, exporting SMTP addresses in Exchange is a piece of cake.

ExchangeDefender Starts Deployment of 3.0

We will begin deploying ExchangeDefender 3.0 over the next two months. Because ExchangeDefender is a large cloud-based network all updates are all-or-none so we will be deploying features one by one. We will be announcing them here on our blog so please stay tuned.

The first significant change to come is the end to open relaying:

Starting November 1st, 2006 we will no longer blindly relay domains through ExchangeDefender for our customers. The following email address requirements will be established for the new accounts:

• All users and valid email addresses must be provided at the time of account activation. By default ExchangeDefender will block traffic to addresses it is not aware of.
• All users will be programmed into the Valid Recipients database, either by the customer, reseller or ExchangeDefender staff (through the LDF file)
• All users that activate their accounts will automatically be allowed through the ExchangeDefender cloud, there is no need for double entry on the administrative side.

All current customers will have until December 1st, 2006 to provide a list of valid recipients or establish a protocol for address verification. We will of course be available to assist you with any administrative tasks that may be required to get the data exported correctly.

Reasons for Address Restrictions

ExchangeDefender has always applied address restrictions but until now it was an optional feature. Unfortunately over the past year address book attacks and mail probes have made open addressing impossible to facilitate. Many major ISP’s (Own Web Now Corp included) have stopped allowing “catch-all” addresses. Furthermore, many sites have even stopped issuing NDR’s (Non-Delivery Receipt) alltogether.

In the modern world of email threats, NDR’s can and are often used to flood the third party mailbox with SPAM. All servers are configured by default to send an NDR if there is a problem with delivery. Spammers can identify mail servers that openly issue NDR’s and can forge the From: address that the message is being sent from. When the target SMTP server rejects the message by issuing an NDR the message is sent back to the forged From: email address.

The result? Flooded mailbox. Quota Warnings. Diminished effectiveness of spam filtering. Increased overall load on the mail server. To protect our customers from all of these we’ll begin enforcing address validations through ExchangeDefender.

Any questions? Let us know – support@ownwebnow.com