We’re currently investigation an issue where a limited number of mailboxes are not accessible on LOUIE . Please be sure to have your users use LiveArchive while we investigate and resolve this issue. We will be updating this post shortly with further details.
Update 9:48 AM Eastern: The issues with the mailboxes that are unavailable seems to be related to user mailbox moves that began last night. To clear up the issues, we will have to restart the individual mailbox servers in LOUIE in a staggered manner. Clients on each server will experience an outage of 15 minutes.
We’ve exhausted all possible ways to resolve the communication issue, leaving the restart as the last viable option.
The LOUIE mailbox servers will begin the reboot process at 10:00 AM Eastern and will last 15 minutes per mailbox server.
Update 10:14 AM Eastern: In the interest of restoring service as soon as possible, we will be doing a dial tone database restore. In short, users will receive a temporary blank mailbox as we work to restore full service to users on LOUIEMBOX1.
Update 10:18 AM Eastern: The outage appears to only affect about 15% of the users on the LOUIE network. We are running a test restore of a user mailbox and if all goes well, we will be able to bring the mailboxes back to the state they were in right before the move requests completed.
Update 11:00 AM Eastern: The test restore went through successfully. We are beginning to seed in the mail for the affected users. We anticipate that this process may take the rest of the day to complete, however, users will being to see mail populate in their mailbox.
Update 12:51 PM Eastern: The restore process is about 10% done and appears to be processing the larger mailboxes first. We will continue to provide updated statistics throughout the restore process.
3/5/11 Update 12:08 PM Eastern: The process is well over 50% completed and we anticipate the restores to be completed by midnight.
3/6/11 Update 9:44 AM Eastern: The final batch of 50 users is taking an unusual amount of time to complete. While most mailboxes have successfully restored, we have a few restores that failed which we are working towards restoring full content.
3/6/11 Update 6:30 PM Eastern: We are retrying the mailbox restores on users that failed to restore content during the weekend restore. In the background we are also replaying the log files from the database on Friday morning as a possible restore point for any mailboxes that continue to fail on restore.