June 2008

I wanted to share with you some details of the upcoming upgrade to our Offsite Backup service over the next 60 days. If you work closely with us you are perhaps already well aware of the information below but we thought it would be nice to have it in a blog post.

First, starting the first week of July, our European Offsite Backup offering goes online based in United Kingdom with a replication server infrastructure in Netherlands. In United States we already do triple replication between Dallas, Los Angeles and Chicago. The two offsite backup systems will be physically and logically separate, no data stored in European Union will be replicated to United States or vice versa. However, nothing stands to prohibit you from purchasing space on either network and to the surprise of many of our EU clients, the price will indeed be the same as in United States.

Oh, and we’ll also give you quota and account management functionality so you don’t have to work through the tickets and can have centralized service management functionality straight out of our portal 🙂

So without further ado, feature set from AhSay 5.5:

  • AhsayOBS
    • Backup
      1. N/A – No updates have been made in this category
    • Restore
      1. N/A – No updates have been made in this category
    • Customization
      1. [Pre May-07] Include link to download OBM / ACB based on user type in the welcome email. (ref: SQN-381142)
      2. [Pre May-07] Client requests to have the encrypted password included in the welcome email or make this it available as an optional feature. (ref: TRK-830097)
    • GUI / Reporting / Logging
      1. [Pre Dec-06] Ability to set catalina log files to another directory (it is the catalina.out file in particular. Let the user know how to set the location of this file)
      2. [Pre May-07] Show the expiration date of all trial user accounts on the [List Users] and [User Profile] pages (ref: JMN-416336) 
      3. [Pre Dec-06] Show total number of files and uncompressed file size for each backup set (ref: FMZ-208525)
      4. [Pre May-07] File search and filter function (ref: BFC-291647, QBH-761540, VRT-928212, DOU-264678, 383)
    • API
      1. N/A – No updates have been made in this category
    • Administration / Maintenance / Deployment (e.g user account maintenance, billing, monitoring, system jobs, installer)
      1. (Key Feature) Load balancing support for AhsayOBS
      2. (Key Feature) Add LDAP (or Windows Active Directory) support to AhsayOBS
        1. [Pre Dec-06] Allow backup quota to be shared by group of users (ref: SYA-626989)
        2. [Pre May-07] Allow the default backup set setting for different users or for user groups customizable by administrators (ref: 291)
        3. [Pre Dec-06] Global filter settings – allow client setting to be overridden by server setting at system level, at user level and at backup set level
        4. [Pre May-07] Allow administrator to change the in-file default settings from incremental to differential delta (ref: TST-934738)
    • Replication
      1. [Pre Dec-06] Multi-user homes for RPS, e.g. allow replicating from user home to user home
    • General
      1. N/A – No updates have been made in this category
  • AhsayOBM / AhsayACB
    • Integration with other applications (e.g. AS/400, Postgres)
      1. (Key Feature) Continuous data protection – start backup whenever a file is being modified (ref: KLY-561803)
        1. [Pre May-07] Start backup whenever the computer is not in use (no network traffic) (ref: KLY-561803)
        2. [Jun-07] Monitor changes on computer but backup only when CPU utilization is below a configurable percent (e.g. 20%) (ref: WPC-416541)
      2. [Pre Dec-06] Integrate with Microsoft Exchange recovery storage group feature
      3. [Pre May-07] Enable/Disable truncate logs after backing up Microsoft Exchange Information Store (ref: 216)
    • Backup (core backup logics and scheduler) 
      1. [Pre May-07] Add options for user to choose to backup using 256-bit Encryption (ref: XWM-772999)
      2. [Pre May-07] Alert users when listing backup sets with “Run scheduled backup on this computer” not checked (ref: FBT-783473)
      3. [Pre May-07] If a user set “\\” or “\\SERVER” as backup source or filter’s top directory, backup job failed with an unclear error message. Better error messages should be presented to users to explain the problems. (ref: YSH-282296)
    • Restore
      1. [Pre May-07] Allow OBM to run decrypt.bat without contacting OBS as a standalone EXE executable (ref: DPV-515226)
    • Auto upgrade agent
      1. N/A – No updates have been made in this category
    • Customization
      1. N/A – No updates have been made in this category
    • GUI / Reporting / Logging
      1. (Key Feature) Re-design of AhsayOBM / AhsayACB GUI
      2. [Pre May-07] Add the functionality of “Backup and Sleep (Hibernate)” and “Backup and standby” (ref: MMP-653617)
      3. [Pre May-07] Remove “Run all backup sets” from system tray in ACB (ref: CCV-309638)
      4. [Pre May-07] In the restore window, it should give a label to the other radio button (not the original location but the one which allows to choose restoring to another location) (ref: 935)
      5. [Pre May-07] Variable retry interval – it will only wait for 5 seconds for the first retry and all subsequence retries interval will a double of previous retry interval until an interval of 300 seconds is reached (ref: ERY-169217)
      6. [Pre Dec-06] Add the option to skip all files with invalid encrypting key during the decryption process.
      7. [Pre May-07] Ensure that the “UnsupportedLookAndFeelException” will not display when running “RunBackupSet.sh” with a backup set that could not be found on server (ref: LRU-783282)
    • Administration / Maintenance / Deployment (e.g. monitoring, installer)
      1. registry before installation of AhsayOBM to fix the bug of system tray icon not showing during startup when there are duplicates entries to the same executable
      2. [Jan-08] Bug fix – The point-in-time view from AhsayOBM drop down list under the [Restore] tab is not correct and is not consistent with the view of AhsayOBS on the web interface when using advanced retention policy
    • General
      1. [Pre May-07] With the current restore.bat or restore.sh, Java will pop-up a prompt to reconfirm whether to overwrite previous files when restoring to an existing folder. This doesn’t make sense as people may not have a GUI when using command-line, besides, it doesn’t work when they use a OS level scheduler to trigger the script. Provide a switch to the script file so that no reconfirmation will be prompted. We may want to consider putting in a synchronization logic to the restore function as well. (ref: TYW-997121) 
      2. [Pre May-07] (bug) OBM doesn’t timeout in 6 hours (ref: VOZ-927176)
      3. [Pre May-07] Users first logon to OBM using the encrypted password, it should prompt the users to change their password. (ref: TRK-830097)
  • AhsayOBX (AhsayOBS, AhsayOBM & AhsayACB)
    • Administration / Maintenance / Deployment (e.g. monitoring, installer) 
      1. (Key Feature) Ability to initiate backup from OBS (ref: QDP-817039)
    • Backup
      1. [Pre Dec-06] Allow backup schedule to run every X minutes (ref: 234)
    • Restore
      1. [Pre May-07] File search and filter function in OBS and OBM (ref: BFC-291647, QBH-761540, VRT-928212, DOU-264678, 383)
    • GUI / Reporting / Logging
      1. [Pre May-07] Change “Show jobs after….” to “Show job as of ….” (ref: HSO-458501) 
      2. [Pre May-07] In the restore window, it should use “AS OF” instead of “After” a certain jobs (ref: 935)
  • Network Attached Storage (NAS) integration
    • [Nov-07] Integrate OBS into Thecus NAS products:
      1. Building a client utility which install AhsayOBS into NAS products
      2. Enhance user experiences by streamlining the installation process with easier installation instructions
AhsayOBS v6.0 (beta: 1-Oct-2008, stable: 1-Dec-2008)

This release will contain bug fixes and enhancements listed below. 

  • AhsayOBS
    • Backup
      1. (Key Feature) New backup file system architecture that can span multiple partitions and run faster. For example, multiple user homes for the same user, allowing a user to span multiple user homes (ref: HUM-981718, 548)
        1. [Pre May-07] Permission updates should not increase the storage massively (ref: AIC-665482)
    • Restore
      1. N/A – No updates have been made in this category
    • Customization
      1. [Pre May-07] Add the option to change where OBM stores the log files (ref:  HOU-743486, 382)
    • GUI / Reporting / Logging
      1. (Key Feature) Re-design GUI using AJAX
      2. [Pre Dec-06] Auditing of all administrative activities 
      3. [Pre Dec-06] Add remaining quota to usage report 
      4. [Pre Dec-06] Once detects the client has exceeded storage quota subscribed, can show an option for the user to request how much more storage is needed, and allow user to buy additional storage and proceed with backup
      5. [Pre Dec-06] Raise quota automatically and send confirmation emails to user and admin
      6. [Pre May-07] Generation the following reports on OBS side (ref: 790):
        1. Report on when users were first registered so I can keep track of renewals (I know the usage report gives this info but it is static and without manual intervention, I can’t manipulate it)
        2. Any point in time report between dates on when and which users restored files
        3. Missed user backups
        4. Backups with errors (essential from a customer services point of view)
      7. [Pre May-07] File search and filter function as in AhsayOBS (ref: BFC-291647, QBH-761540, VRT-928212, DOU-264678, 383)
    • API 
      1. [Pre May-07] Enhance ListBackupJobs API with more details (e.g. username, job name, start time, end time, upload size and job status) (ref: EEU-184625, 547) (This has been delayed because a simple testing showed that this could lead to significant slow down in response time of this API because it would take too long to get the backup job status of all backup jobs)
    • Administration / Maintenance / Deployment (e.g user account maintenance, billing, monitoring, system jobs, installer)
      1. [Pre Dec-06] A routine job that merges delta files with full backup files on AhsayOBS
      2. [Pre Dec-06] Allow retention policy to keep a certain number of versions of the same file 
      3. [Pre Dec-06] Allow user type to be defined by partners, e.g. paid, trial, others
      4. (Key Feature) Integration with Windows Active Directory and OpenLDAP
        1. [Pre Dec-06] Backup report setting per user group or per user instead of per system right now (NTT request)
        2. [Pre Dec-06] Enable different administrator accounts for different group. These group admin accounts will have privileges to manage accounts under their own groups (ref: XSB-581351, RER-199696, RWI-435051, TKD-379677)
        3. [Pre May-07] Delegate user account management to other support staff without giving them full administrative privileges (ref: 347)
        4. [Pre May-07] Add “Company” Field in User Profile (ref: XSB-581351, RER199696, RWI-435051)
        5. [Pre May-07] Add “phone number” field for user account (ref: RUX-816258)
        6. [Pre May-07] Provide a “Company Name” field for each user account. This field should only be visible and modifiable by the administrator. Administrators should also be able to sort and filter accounts by company name. (ref: 650)
        7. [09-05-2008] Add functionality to backup/restore full OBS configurations, not including userdata (ref: UQT-815782)
    • Replication
      1. (Key Feature) Many-to-one and one-to-many server data replication
      2. [Pre Dec-06] Replicating selective users (or backup sets) from OBS to RPS (ref: FBT-783473)
    • General
      1. [Pre Dec-06] Add support of time zone GMT+9.5 (Adelaide) 
  • AhsayOBM / AhsayACB
    • Integration with other applications (e.g. AS/400, Postgres)
      1. (Key Feature) Bare metal backup / restore
      2. (Key Feature) Backup Framework (allow OBM to be integrated with other applications)
        1. [Pre Dec-06] Allow AhsayOBM to be easily extended to support Sharepoint, Sybase’s iAnywhere, Progress database, Quicken, Quickbooks (*.qbw files), Peachtree, MAS90 module for Peachtree
        2. [Pre Dec-06] Install OBM in silent mode – Embedded “User Account” / API to create user account
        3. [Pre Dec-06] Automatically create a “standard” backup set for target application
        4. [Pre Dec-06] Expose OBM features as external API
        5. [Pre Dec-06] Allow OBM to be hidden from end users, file restore can be invoked through API
        6. [Pre May-07] IMAP mailbox backup – Scalix support. (NB: Scalix is a Linux based Exchange server replacement ) (ref: RFZ-476395)
        7. [Pre Dec-06] Add filter to exclude files from a certain date
        8. [Pre Dec-06] Extract individual email from Outlook (PST) file
        9. [Pre Dec-06] Upload checksum incorrect backup file again during backup
        10. [Pre Dec-06] Add multi-thread delta file calculation & upload for performance
    • Backup (core backup logics and scheduler)
      1. N/A – No updates have been made in this category
    • Restore
      1. N/A – No updates have been made in this category
    • Auto upgrade agent
      1. N/A – No updates have been made in this category
    • Customization
      1. N/A – No updates have been made in this category
    • GUI / Reporting / Logging
      1. [Pre May-07] (bug) OBM backup source of a mapped directory disappears after the mapped drive is unmapped. The backup set will still backup that directory but cannot find the path in the backup source. Web interface will still show the mapped path. (ref: WVK-850816, ORO-725960)
      2. [Pre May-07] Add the option to “Skip logging failed backup of all open files as error” when running backup in OBM / ACB (ref: GXH-275737)
      3. [Pre Dec-06] Add wizard to detect if SQL or Exchange is installed and configure it automatically
    • Administration / Maintenance / Deployment (e.g. monitoring, installer)
      1. N/A – No updates have been made in this category
    • General
      1. N/A – No updates have been made in this category

We have completed the rollout of PHP 5.1.6 scripting language on our clustered web hosting platform. PHP 5.1.6 is our first upgrade from PHP 4.x and opens up a new world of possibilities for our partners and customers to utilize many open source and commercial web applications free of charge on top of their web hosting accounts.

Due to the enormous amount of feedback by our customer base we are stepping up the defense from NDRs received for the emails that were not originated by your users to begin with. This is often called NDR blowback, backscatter, fake virus or worm storm, etc. It happens when someone uses your email address to relay an enormous amount of SPAM to the remote servers and encounters a lot of dead mailboxes that may have already been removed or had their quotas filled with SPAM. Naturally, an error bounces back to you because the remote server thinks you sent it.

We have had NDR backscatter protection for quite some time but the cries from our customer base have forced us to take away our liberal stance on this issue. We are now strictly enforcing NDR legitimacy, meaning that we will only deliver NDR mail if the message was sent through one of our outbound servers. Anything else, because we cannot validate it, will be automatically thrown into the SPAM queue if you choose to quarantine SPAM messages.

Are NDRs SPAM?

No, the non-delivery receipts and delivery status notifications are not SPAM. They do not contain any unsolicited commercial communication, they are not selling anything, they are not dangerous in any way. They are annoying, very annoying when you receive a few hundred in a span of a minute. How did this happen? Well, someone you previously emailed likely got infected by a worm or a virus that searched their hard drive (mailboxes) for email addresses. It then took a random address and joined a botnet and sent thousands of messages and made them appear they came from you. Because the remote (recipient) server did not have proper SPAM protection it blindly accepted the message and issued a rejection.

How does ExchangeDefender know what passed through it and what did not?

ExchangeDefender outbound network stamps each outgoing message with a hash key. When the message is returned in a form of a DSN or NDR we check the SMTP header for the presence of our hash key, we decode it and compare with the local copy stored in our server along with the matching From: message. If the hash key matches the sender of the message the email is passed on to other filters. If it doesn’t it means that  the message is a bounce to the message you never sent in the first place because it did not go through our network and it did not get stamped.

What to do if you still keep on getting NDRs?

There are a few things:

  1. Check that you are sending mail using outbound.exchangedefender.com as your organizations smarthost.
  2. Check that you only have inbound30.exchangedefender.com as your only MX record. If you have more than one your configuration is broken, follow the deployment guide.
  3. Check that you are enforcing IP restrictions, port 25 only and from our exchangedefender.com network only.
  4. If everything looks correct and the NDR was received after Tuesday, May 10th, open a support request with the text of the NDR as well as full SMTP headers of the message for review.

Thank you for trusting us with your mail.