New ExchangeDefender Reports! (Part 1)

New ExchangeDefender Reports are out! We have worked hard to bring the new reporting functionality to you and we hope you like the new information we are providing as well as what we are allowing you to remove from the report:

If you pardon the eraser tool you will notice that the layout of the reports has changed slightly and you will also notice some more information showing up in an unobtrusive way. So let’s look at the improvements:

• Summary Guide – Top right blue box “You are reading an email summary…” has been the most demanded feature by our customers and partners alike because  it gives you single-click access to the ExchangeDefender portal. Click on the link to access your settings, searchable quarantines and more.
• Email Activity Stats – Directly above the SPAM quarantines. This new summary field is showing you the total activity during this reporting period and helps address those “we are not receiving any email” complaints that users tend to feel once ExchangeDefender comes to life. 
• Warnings & Caveats –  Bottom of the message. There are two warnings so let’s look at them carefully. “You chose to be notified of all SPAM quarantines.” is printed when you chose to receive full/complete SPAM reports. If you have a ton of email addresses and aliases it can get annoying to scroll down rows and rows of email addresses that never receive email just to see “No messages in this quarantine” so we gave you an option to suppress empty quarantine reports. Second warning is to let the user know that these reports do not include viruses, address book attacks, NDR storms, mailbox floods, mailbombs and other causual Internet annoyances. On a daily basis we only accept and process up to 80% of the inbound mail, most of the 20% being part of multiple RBLs or confirmed SPAM content. Majority of that 20% is dismissed anyhow! 
• Branding – Background, messages and logo are now brandable. If you are a service provider you can make this your own!

That’s all for the user facing problems! Stay tuned for the Administrator reports tomorrow!

Note: We addressed a bug in the reports that did not print a header message for the SureSPAM category. If you received a report prior to 8 AM EST on Monday, September 17th, you would have noticed your SPAM and SureSPAM bundled together.. After 8 AM EST you will see both quarantine contents broken down individually.

Alert: ExchangeDefender bounce notifications

We have received some reports of certain users emails bouncing on receipt. We are currently looking into the problem and will update the advisory shortly.

Update: 1:19 PM EST: Problem solved, 100% of the accounts are now online.

Update: 1:53 PM EST: ExchangeDefender has been reloaded and refreshed to assure absolutely everything is working perfectly. We have taken off administrative console access offline for the moment to determine the cause of network configuration failure. As mentioned in the 1:19 PM update, everything should be working perfectly fine and there should be no bounces.

Update: 2:16 PM EST: Notified ExchangeDefender administrators, updated trouble tickets and the recovery effort to deliver 800+/20+ inbound/outbound messages continues. This was a minor (albeit catastrophic) error in the ExchangeDefender network configuration that affected a small portion of our customer base but due to the distributed nature of the system it may have affected just about everyone. As a precaution, we have temporarily taken administrative interfaces offline to determine how this happened in the first place. Again, network, performance and system are at 100% at the moment with no known issues.

Update: 2:28 PM EST: Cause of network failure identified, fixed. Moving to the testing phase, control panels are still offline. 1/4 of the bounced messages have been recovered and delivered to the end users.

Update: 3:19 PM EST: Everything is still working perfectly fine. Our team is decrypting messages from the standby spool and dropping the messages by hand into the delivery queue. All “bounced” messages will still be delivered. Thanks to the technology behind LiveArchive, we are able to cache delivery so in case of bounces, like today, we can still manually drop the message into your mail server.

Update: 4:46 PM EST: Everything is back to normal, all external bounced mail has been delivered, 100% service restored and administrative control panels are restored as well.

ExchangeDefender: Back to Normal

The following message was sent as a part of the daily and intraday reports ExchangeDefender provides to users that choose the quarantine their mail:

You may have noticed that more SPAM messages than usual have gotten to your inbox between Thursday and Saturday of last week. This was an isolated case related to our software updates which have been designed to prevent future issues.

We are seeing a change in the way threats are delivered. Spammers used to rely on small text messages and links to their sites in the past, today they are using attachments, images, PDF files and other dangerous content to get your attention. We have kept up with them and minimized your exposure but as the SPAM problem evolves and becomes more threatening we found it neccessary to both increase the size of our network and the way we process messages.

Everything should be back to normal and you should be seeing less SPAM than you ever have before. Our network and software improvements have been training for a few days and were put in full effect at roughly midnight GMT.

Thank you for your patience and we’re sorry for any inconvenience the increased amount of SPAM may have caused you. During the software upgrade we still filtered out over 99.7% of all inbound mail but with the increasing number of SPAM operations even that small 0.3% of non-filtered mail can result in a dozen or more messages that got through.

We’re seeing a significant change in the way SPAM is being designed, delivered and spread. Where in the past we could simply rely on virus scanners and RBLs the future of threats and SPAM has gone to the new level.

Over the past month we have quadrupled the size of ExchangeDefender both in physical assets and bandwidth and have rewritten major parts of the system in expectation of worse SPAM problems in the future. The transition to the new engine, new systems, new networks and new software has been tough on us and on our customers but it has prepared us all for whats coming.

On behalf of the whole team thank you for your patience with the transition and problems that came up during August. If it helps in any way, we will be refunding all the ExchangeDefender fees for August. Even though our performance was well within the SLA (service level agreement) we believe in providing excellent service and you should expect no less from us.

Sincerely,
Vladimir Mazek
CEO, Own Web Now Corp

ExchangeDefender Detours

Expect some delays in mail delivery starting at about 11 AM EST today. We are adding new servers, new switches, new bandwidth and more routes to our data centers and as we scale each network we will need to refresh configuration and shut that particular load balancer down.

The delays will not be significant and should not be uniform. No mail will be dropped or deleted nor will mail “sit” on the network while we are moving it up. We are taking this opportunity of a really light few days before the holiday to further improve our network and expand our offering (something which you will hear about very, very shortly!)

New SPAM Reports hit Inboxes today!

New SPAM reports for daily and intraday activity are already hitting our customers inboxes today. We have taken so much feedback from our customer base on these reports and taken every bit we could to help improve them. Among notable options, SPAM reports are now:

• Brandable – Your background color, your colors, your corporate identity and product name in both From: and every line that otherwise mentions ExchangeDefender
• Flexible schedule – Reports can be delivered at any 30 minute interval and are adjusted for your home time zone as well as your date format options (m/d/y, d/m/y)
• Daily and Intraday – Daily reports outline past 24 hours of quarantined mail, intraday show only mail since the last report.
• Custom Message – Reports can be branded with a custom message: Alert your customers and users about network events, new services or just general announcements.

Every level of ExchangeDefender user (administrator, service provider, end user) can manage their SPAM report settings and administrators and service providers can now override all settings for all users under their control. Some best practices are to remember not to set intraday reports to run earlier in the day than the daily reports. Also keep in mind that it can take a few minutes for the reports to be generated because they are prepared in realtime.

Finally, remember that daily reports are a great user self-management tools but should not be used as the primary SPAM management option. Create a shortcut to ExchangeDefender instead. To do so, right click on the destktop, select new Shortcut, type in

https://admin.exchangedefender.com/login.php?

username=theirusername&password=theirpassword

Change theirusername and theirpassword values and they’ll have realtime, searchable access to their SPAM quarantines. Enjoy!

Alert: In order to adjust for the reports around the world everyone will receive multiple reports today (one at 9AM EST, and one at their scheduled time). Starting tomorrow you will only see the reports scheduled at your preset time.

ExchangeDefender Mail Delays & Non-receipts

I am writing this blog post to address the issue of ExchangeDefender mail receipt delays or mail simply not arriving at all. Nearly three weeks after we have implemented the new networks, and nearly a month after we have notified all our ExchangeDefender customer administrators we are still fighting with the ad-hoc issues related to delayed mail, mail that was not received, mail that was received hours later.

In 100% of the cases the issue was the recipient policy on the target mail server. Please, please, please make sure you have added the following IP address blocks in order to allow our new servers to relay mail to you:

64.182.140.0/24

64.182.139.0/24

If you do not allow those IP address ranges access to your network the system will not bounce the messages. Instead, our intelligent routing system will route messages internally to the server that is able to establish a connection with you. This system, however, was not designed to handle sysadmin apathy but instead to respond to major interruptions in the Internet backbone. If a system is unable to deliver the message directly to the server it reattempts every 15 minutes. After the first hour it sends a broadcast message asking other networks to see if they can establish a route and receive the SMTP banner. If the connection can be established the message is routed to that server and then delivered. By not having the proper IP address restrictions in place you are forcing your inbound mail to be put through our DR scenario which is automatic but time consuming.

Please, either do not use IP restrictions at all or update them properly. For a little more positive note, tune in later tonight when we’ll announce our new email SPAM reports.

ExchangeDefender Network Status Update

I wanted to offer you an interim update on the status of ExchangeDefender network and codebase. As I mentioned on Friday, we have brought the platform back to normal and since then we have not had any even unusual events (sans a few DDoS attacks which are common against large networks). All our data centers are performing well, now at 11:56 AM which is our peak time we are running at 38% network utilization and 61% system utilization meaning we can sustain twice the load without seeing any effects on the network itself.

All the issues that have been reported over the past week or so have been resolved and we have not had any reports of additional problems since. Now, back to features….

ExchangeDefender LiveArchive

ExchangeDefender LiveArchive is easilly the most popular feature we have introduced in years. Simple to understand why, its a part disaster recovery part business continuity solution that doesn’t cost you anything and works without any maintenance whatsoever. The idea is simple: mail servers and Internet connections tend to be unreliable. Hard drives fill up, Internet connections go down, Outlook profiles or Exchange mailboxes/stores become corrupted our dismount, you name it — we’ve seen it. But the solutions for Exchange business continuity and disaster recovery are very expensive, require appliances and still do nothing when those Internet connections go down.

We have an answer in ExchangeDefender LiveArchive: Your mail, in addition to being delivered if the server is up or queued/mailbagged in case it’s down, is also simultaneously delivered to a separate mail system on our network. We keep a realtime storage of past seven days of your inbound mail along with your profile settings “Vlad Mazek <vlad@ownwebnow.com>” along with all your other mail identities. When you can’t connect to your mail server, you’ll be able to connect to ExchangeDefender LiveArchive server just by going to this web site:

https://archive.exchangedefender.com

Username: your email address

Password: your exchangedefender password

Both should be easy to remember and are even easier to enable. To get started with ExchangeDefender LiveArchive you have to enable it. As a Domain Administrator go to Configuration, scroll down to LiveArchive and select Enabled. If you want this new setting to apply to all the users in the domain just select the checkbox that says “Make this the default setting for all existing users.” and you are set. By default, without this box checked, only new accounts created within this domain will have ExchangeDefender LiveArchive enabled so making sure this box is checked is important. As the ExchangeDefender Service Provider you can accomplish the same by going to Management, selecting the domain to manage and clicking on Change Configuration. Same screen, same configuration.

That is all you need to do and all you need to know. It is important to recognize that this is a true live mail system that you can read all incoming mail, respond to it, delete it, forward it or whatever you wish and can do with your Outlook, Notes or Groupwise clients. The site is always on and always has the latest 7 days of email available. Email is delivered in realtime, meaning that you can communicate with people even while your mail server or Internet connections are down, it does not stop delivering to LiveArchive just because it cannot reach your mail server.

Finally, reading and managing mail here does not mean we start bouncing or redirecting mail – the mail will still be delivered to your server  once it or the Internet connection are back online.

We did everything we could to make Exchange LiveArchive an easy deployment and as easy to use as your own mail client. However, trick to properly executing business continuity is in making sure your employees know this exists and training them on how to access it before they need to do it in an emergency so here are a few checks you need to do:

[ x ] Make sure your employees know and remember the address of https://archive.exchangedefender.com

[ x ] Make sure your users know and remember their email address and password to be used with https://archive.exchangedefender.com – for security purposes this should be different than their server password.

[ x ] Make sure your users are aware of the dangers of using insecure Internet connections, computer kiosks and third party equipment that could be keylogging their usernames and passwords.

[ x ] Consult a lawyer for mandatory legalese in case you have to disclose that your infrastructure is temporarily down. Work on the customer service skills so employees properly notify their contacts that response times may be lagging due to an emergency.

There is a lot more to business continuity but I hope that the ExchangeDefender team at least makes it less of a burgen to communicate in an emergency and gives you the time and resources to deal with high priority problems and not fixing the email.