[Mar 31, 2006, 6:49 AM] Network Problems Resolved
At 5:48am Eastern time, one of our central aggregation switches suddenly stopped passing most traffic, without logging any problems or explanation for the failure. This was not a normal failure mode.
Over the past several months, we have been upgrading our switching infrastructure to utilize newer equipment with higher capacity, and our investigation of this problem was initially focused on the new equipment. As a result, the legacy switch was not immediately identified as faulty; it was rebooted and returned to normal service by 6:40am.
For the duration of the outage, traffic to the majority of our servers and services was disrupted. We apologize for this outage; our recent upgrades have been designed precisely to avoid this type of scenario. We currently expect to have the offending legacy switches out of service within the next seven to ten days, although this will be accelerated further if there are any additional problems.
Upon deployment of the new switches, we have further enhancements planned which will make it nearly impossible for any one device to cause significant problems in network connectivity.