ExchangeDefender – ExchangeDefender Network Operations

We’re currently addressing a large scale DDoS attack on our outbound network; We’re slowly clearing out the queues from the attack and the latency has been reduced down to normal levels.

Now that the major launch of ExchangeDefender 5 has been completed and we have our processes together I wanted to resume the helpful NOC posts we did in 2009 that gave you a bit of a heads up about maintenance and network changes that were being made. Here is our agenda for March:

ExchangeDefender upgrade to Exchange 2010

ExchangeDefender LiveArchive has been tremendously popular and we’ve planned some enhancements to the product that will just blow you away. Coming in March, ExchangeDefender LiveArchive will be upgraded to Exchange 2010. Unfortunately due to the volume of mail, a migration will just not be possible (think restoring or mail moving a year worth of your email multiplied by every ExchangeDefender customer – not easy). To alleviate the pain of getting a fresh mailbox, we will continue to run the two ExchangeDefender systems in parallel – old LiveArchive with Exchange 2007 won’t go offline for 30 days after the new one is up and running. Now let me address a concern:

But Vlad, you guaranteed 1 year of archiving free, that’s what we’ve sold to our client and they absolutely need 1 year to be there for their compliance.

This seems to be a common misconception because people focus on numbers and not the language and general spirit of the solution. To be clear, LiveArchive is a business continuity solution that keeps up to a year worth of both inbound and outbound mail in an always on, always available Exchange OWA environment so you can access it at any time and continue working where you left off when your connection was interrupted.

LiveArchive does not guarantee 1 year of archived mail nor is it advertised as an archiving solution – if your clients require that, we do offer a mail archiving solution called ComplianceArchive which is designed for compliance, HIPAA/SOX regulatory archiving and guarantees access to mail for as many years as you wish to archive it for. If you’ve sold your client LiveArchive contact them and tell them that due to migration constraints the new LiveArchive (with new features, see below) will start as a clean mailbox and that they should save any old items they absolutely must have. If they absolutely must have them all, offer to upgrade them to a ComplianceArchive solution.

LiveArchive was designed as a business continuity solution, so that clients that got inconvenienced by server performance, outage, downtime or maintenance could continue working without having to manage a disaster scenario or start from a blank mailbox – typically a day or two worth of email is sufficient to refer back and we offer up to a year because we understand that sometimes people need more and it’s not incredibly expensive for us to provide it.

Now, the beauty of ExchangeDefender 2010 is all in it’s Outlook Web App – no more Web Access –  The new solution is compatible with all browsers which means full mobility and a ton more functionality. We’re also building in rule sets so that outbound mail goes to send items and adding a layer of quality assurance that will address the issues we faced with 2007.

Rollout of Exchange 2010 Hosting & New Control Panels

We will begin offering Exchange 2010 hosting this month as well, with a new quota and a new set of control panels. With the new controls you will be able to manage mail enabled contacts, public folders and distribution groups, so pretty much the same type of control as if you had your own Exchange cluster!

As with all new software, the price is going up but we will keep it at the same level for our ExchangeDefender Service providers. We’ll also be the only company to offer it under your own brand while not competing with you for the business – not directly with our brand and not indirectly through the major distributors. With ExchangeDefender we believe we can deliver the reporting and business intelligence, something that is unparalleled in our industry not to mention integrated with Autotask and ConnectWise.

Other Fun

We’re rolling out new power switches and reboot switches which will make us a lot more green and improve power. There will be several announced reboots / server moves as we move systems around to improve power utilization and cooling. We’ve grown tremendously since ExchangeDefender 5 launched in December 2009 and our products are a hit in the marketplace. ExchangeDefender will be getting a new major data center (delivery point) and we’ll advise of the maintenance as we get closer to the announcement.

We also have some really great EU news but look for that in the newsletter coming around the 2nd or 3rd week of March 😉


Vlad Mazek, MCSE

CEO, Own Web Now Corp

We have received reports from a few partners that ExchangeDefender users are not receiving mail released from Quarantine. We are investigating into this issue and assure that no mail has been lost. If you experience issues in releasing mail from quarantine please open a support ticket at including the senders email address , recipients email address, and time stamp.

Update – Noon EST – We have resolved the issue that caused the release mechanism to fail, the problem has been fixed and as of about 11:30 the full functionality has been restored. Now a word from Vlad Mazek, CEO:

As we have widely blogged, we have been migrating from the old systems and old software with the release of ExchangeDefender 5. The final step, which was taken this weekend, involved removal of the old infrastructure at the core of the command center and as is usually the case with complex software, some middleware got in the way.

Just release the messages again and they will flow in. Furthermore, if you’re not familiar with this would have been a perfect time to check it out, in ExchangeDefender 5 we do offer full access to SPAM so you can view it live without releasing – you can even reply directly from the site!

Again, my apologies about the inconvenience this has caused.



Cell: (407) 536-VLAD

Throughout the day the ExchangeDefender outbound grid has been fighting extremely large mail queues and hour long delivery delays. The source has been identified as a DDoS attack and we’ve taken all mesures to remove the mail.

Legitimate mail that hasn’t been delivered will be delivered throughout the next couple of hours.

We highly apologize and we are making modifications to the outbound grid throughtout the next limit to prevent flooding.

We will be holding an extended maintenance window this weekend affecting systems:

Sunday, January 31st, 2010

4 AM – 7 AM EST

During this time window access to will be intermittent as we undergo a major networking and hardware update to handle the expansion and additional services.

We will start posting updates to this blog during the 4 AM – 7 AM.

The livearchive database for some ExchangeDefender users is starting to show mail routing issues. We’ve disabled this database and temporarily put up a blank database. Over the weekend we will attempt to diagnose the issue with the database and remount the affected database.

Update 12-06-09: After many attempts to restore the database, the decision was made to leave the database dismounted in preparation of LiveArchive 3.0 (Due for release in Feb 2010). The current running database is >6 TB in size and direct repairs would take at least a month, leaving customers without the ability to utilize LiveArchive. All users currently have new mailboxes and we plan to migrate the >6TB database into the new LA 3.0 database.

We’ve received reports from a couple partners that they’ve received an email titled “your mailbox has been deactivated” that has an executable attachment that gets stripped by ExchangeDefender. This seems to be a blind attack from the outside and we’ve already implimented the checks to block these messages from coming through ExchangeDefender.

Just as a reminder, we will never email end users about issues with ExchangeDefender, we only contact our registered partner.

We are in the process of rebooting the livearchive server. Our alerting software showed periods of inaccessibility which we believe will be resolved with a reboot.

Update 8:30 AM Eastern: The livearchive server has been rebooted and is back online. Services are running 100%.

In preparation for the release of ExchangeDefender 5.0 we’ve installed 4 new servers to process outbound mail for ExchangeDefender clients.

This transition was seamless and shouldn’t require any work from our partners, however any clients who are using SPF records will need to add the following IP addresses


Over the weekend our replication servers experienced a larger than normal lag in replicating statistical data from our Dallas Colo. This caused quarantine reports to show up as empty as there was no new data replicated when the reports were generated. Reporting is expected to be fully functional by the next scheduled cycle.

We have been running an extended maintenance interval process on ExchangeDefender to run some optimization processes with regard to PSA integration. Some users have reported that they were unable to access, something we have no explanation for at the moment as we’ve not had any service interruption on it (just high loads and periodic latency).

Our Exchange 2007 network is also under extended maintenance due to the issues we’ve experienced between Exchange 2007 Update Rollup 9 and Blackberry Enterprise Services. We are still working on it and will provide updates as we have them.

We anticipate receiving reports about delays in ExchangeDefender for mail accepted in the earlier half of today. In short, the livearchive server experienced a larger than normal queue load, which delayed messages from the inbound grid being copied to livearchive.

This issue has been resolved and any queued mail should be delivered momentarily.

There have been a few reports about NDR error “ExchangeDefender does not protect this address” on new aliases/accounts added to ExchangeDefender today. The NOC has identified the issue with replication of new accounts and applied a fix. Accounts should start working properly by 6:30 PM Eastern.

Over the weekend our reports service was under maintenance and reported all zeros. This issue has been addressed, things should be back to normal within an hour or so. I wanted to take a moment of your time and address the email reports again.

As usual, the 8 partners that have clients using this service complained, loudly. I have talked about this numerous times and heard the feedback but the fact remains that the life cycle for this feature has come and passed. Email reports are not timely, are not realtime, get caught by other filtering software due to their content and admin’s improper deployment, crash Outlook when they report thousands upon thousands of SPAM messages (volume of which is still growing), and are naturally ignored by over 99.9% of our client base.

If you have sold your client base or your employees on the email reports please inform them of the other, better, methods to access their junk mail. With ExchangeDefender 5 we will deliver a cross-platform desktop agent and a more responsive web UI, completely eliminating the need for digest mail reports. I know many of you will miss them, and they were a great solution for SPAM reporting back in early 2000’s, but with users receiving tens of thousands of messages a day this process is no longer supportable.

Please consider the following:

ExchangeDefender Client Software

ExchangeDefender Desktop Shortcut

Lastly, I would like to address a concern that a few resellers have brought to me regarding the “email reports as a competitive advantage” which may work against appliances but is still several levels below the functionality and value that ExchangeDefender delivers with the client software suite. We designed this software to give our partners and clients a huge gateway to the users desktop while allowing them to remain in the workplace environment they are used to. If you are not leveraging, branding, deploying and marketing these tools you are missing an incredible opportunity to demonstrate the value of the service you are providing – and price it accordingly.

Vlad Mazek, MCSE
CEO, Own Web Now Corp

We are experiencing an issue with the ExchangeDefender email SPAM reports: they are reporting all 0’s. Reporting will be restored within the hour.

As usual, we do not recommend or offer support for email reports as it is a feature that has long ago been replaced with more reliable and efficient ways of obtaining SPAM data and statistics through either our administrative portal, ExchangeDefender Outlook 2007 plugin or ExchangeDefender Desktop Agent. If you have not considered these alternatives please take a look at:

Earlier last week we identified an issue with ExchangeDefender whitelists. At first we believed the issue was simply that of certain notes not receiving full replication but after that turned out not to be the case we focused on the internal processes that generate note whitelists. The problem turns out to be a bug in the replication of whitelist content between users and master whitelist.

We have been working on it through our maintenance interval but have added a few more hours for additional testing today to make sure it is working properly. As a result of the bug we have rewritten how the whitelist data is replicated and as a result monitoring scripts that watch for replication failures had to be adjusted as well.

As of 1 AM EST / 7 AM GMT the monthly global maintenance cycle has been completed. Please allow up to 2 hours for routes to converge and new networks to be announced.

Among updates, features and bug fixes are:

– Provisioning for the new Los Angeles, CA and Chantilly, VA networks.

– Expansion of LiveArchive network.

– Fix for the ExchangeDefender routing network priority hack (for low bandwidth mail servers)

– Fix for Australia/Pacific routing group (* extending smtp_greet from 30 to 120 seconds to compensate for trans-Pacific)

– Extended nameserver network to improve load balancing.

– Provisioning for the new RR routing strategy for international customers with complex compliance issues (ex: “route all mail through UK, route through EU in case of network outages, never route outside of EU”)

It will take at most two hours for the routes to converge, during this time you may see delayed messages or deliveries out of sequence as the new paths are being distributed. You also may see delays in extended-outage-queues we use for disaster recovery (ETRN/queueing/spooling) mechanisms which are affected by all of the above but primarily the name server expansion.

These efforts are critical to the continued growth and reliability of the ExchangeDefender network. No downtime or outage has occurred.

We are currently running an experimental delivery process on ExchangeDefender mail that has been queued on ExchangeDefender delivery queue for over an hour. If our management system indicates that the target server is up and the message is continuously deferred/rejected, the message will be handed off to another delivery agent that will attempt to stream it through a different process.

To review, if the following conditions are met:

  1. Recipients mail server is up and running
  2. Recipients SMTP port is available and accepting connections (no greylisting).
  3. Message is older than an hour and smaller than 100 Mb.

.. our system will attempt to flush the message through a different system.

These messages may be malformed in some way and the recipients server never would have received them directly from a third party server. However, with ExchangeDefender in the middle, we believe we can attempt to compensate for the problems in the message.

What to watch out for:

Our redelivery queue currently stands at 14 days. It is possible that you will see messages that are older than just a few days.

Why are we doing this:

We have decided to get to the bottom of the sporadic issues that have been reported through the years and build a concise, managed system that monitors for clients server problems so support requests can have a more timely and adequate response. We will extend this Managed ExchangeDefender service as we go along.

Yesterday we made performance enhancements to our ExchangeDefender Email Reports to help cope with the amount of junk arriving in reports as of late. As of this morning, it still does not show signs of improvement we have been expecting. While we are still trying to address the issue, we do not recommend using ExchangeDefender SPAM reports and suggest moving to Outlook or Desktop agents for ExchangeDefender or preferably the web interface.

Update: We believe the issue has been resolved and is now behaving correctly. We are continuing to watch it.

Symptoms: Empty intraday, fractional daily reports.


Our readiness kit contains valuable resources designed specifically to help businesses with GDPR requirements.


IoT Security Solution

Introducing our newest security solution for IoT devices. Protect and secure your IoT environment with robust built in Security.


Are you an MSP?

See why you should consider our partner program. Become a partner at no cost, with no annual commitment, cancel anytime.