[LCH Announce] cloud0.lchost.co.uk EMERGENCY MAINTENANCE

[Phillip Baker]

Hi,

Those of you hosted on cloud0.lchost.co.uk will no doubt have noticed that it crashed earlier today, causing an outage for all services on this server. Since the server came back up I have been attempting to troubleshoot 

a) the cause of the crash 
b) considerably slower than expected mail processing since rebooting

Despite attempts at nailing down b), we are still seeing considerably slower than expected mail processing, leading to many MUAs (mail clients, such as outlook, outlook express, thunderbird, et al) to complain that the server is not responding. 

I believe that both the crash and the poor performance are due to issues with the current driver and kernel versions in use on the server which were built manually with currently available code before Debian (the distribution of linux we use) natively supported the RAID controller in use on this server.

The latest version of Debian includes built in support for this controller as part of a considerably newer kernel. It is with this in mind that I am scheduling an emergency rebuild of this server this evening in an attempt to nail down these issues entirely. This will unfotunately involve the server being taken completely offline so that it can be backed up safely prior to being rebuilt.

The rebuild will start at approx 23:40, and is expected to take no more than 5 hours. 

We apologise for any inconvenience this may cause; but we cannot permit the current situation with potential instability and poor perfomance to continue. In the event that initial testing reveals the newer software release will not function correctly with the system, we will instead look to seperate the mail function out to another backend server over the next 24 hours.

-- 
Regards,
Phillip Baker
LCHost Chief Technical Officer
0870 068 HOST ext 50