Notice of upcoming downtime

[orudge]

Hi all,

Just a little notice to let you all know that a hard disk on the server has failed, and needs to be replaced. While the forum is safe (we run a RAID array, plus have regular off-site backups made), the disk needs to be replaced as soon as possible.

We have scheduled this maintenance for the 27th of June (a Saturday), at 23:00 GMT. You can check what time this is in your local timezone here. Note that the server may be down for up to 90 minutes, although it is hoped that the replacement can be made within 30 minutes. Upon restoration of service, the server may be a little slower than usual due to the disk array resynchronising - rest assured that this will only be temporary.

If you have any questions, please let me know.

[CommanderZ]

I thought hot-swap disk arrays are being used in the servers...am I mistaken?

[orudge]

It is possible that it won't be necessary to reboot the server, but in case it is, I put out the announcement.

[Dave]

NOT GOOD ENOUGH!

Luckily I will be out this Saturday getting slaughtered.

So we're alright, right?

[jonty-comp]

This time it wasn't me that broke it. Or was it?

We'll never get over 100 days uptime on this thing at this rate!

[Rubidium]

We'll never get over 100 days uptime on this thing at this rate!

I rather have scheduled down time than unscheduled down time. Also high uptime usually means not recently patched against kernel exploits, which might increase the chance of unscheduled down time.

[Hyronymus]

Does it affect Yoda, Owen?

[orudge]

Does it affect Yoda, Owen?

No, otherwise you would have received an e-mail about it.

[Hyronymus]

Does it affect Yoda, Owen?

No, otherwise you would have received an e-mail about it.

OK, just checking.

[Benny]

There was some downtime just now. Was this it, or was this something else?

[Ameecher]

Do the dates and time match? Oh yes, they do...

[Dave]

Do the dates and time match? Oh yes, they do...

No they don't.

Owen said this would occur on Sat 27th June, at 2300GMT

From what I saw, it occurred at Fri 26th June at 2300GMT (Sat 0000 BST)

I thought it was tomorrow.

[jonty-comp]

Code: Select all

»» 02:10:56 <@orudge> seems the scheuled maintenace was carried out 24 hours earlier
»» 02:10:58 <@orudge> than I expected
»» 02:11:03 <@orudge> conveniently when I was away
»» 02:11:06 <@orudge> the HD was replaced though
»» 02:11:07 <@orudge> so it is fine
»» 02:11:09 <@orudge> and sites are now back up
»» 02:13:57 <+Andel> they took a long time over it
»» 02:13:58 <+Andel> 3 hours
»» 02:14:01 <+Andel> :/
»» 02:15:58 <@orudge> well, no, the replacement was fine
»» 02:16:06 <@orudge> however, VPSes aren't set to start up on boot
»» 02:16:11 <+Andel> ahhh
»» 02:16:16 <@orudge> as a faulty VPS could render the entire machine inoperable
»» 02:16:19 <+Andel> yes
»» 02:16:39 <@orudge> so it was a case of waiting for a technician to restart the VPSes
»» 02:16:39 <@orudge> again
»» 02:16:44 <@orudge> I apologise for the inconvenience
»» 02:16:49 <@orudge> it was not at all intended this way

I should know, I woke up this morning wondering where my VPS had gone. T'is all good now, though.

[orudge]

Basically, it seems the technician in the datacentre got confused and carried out the process a day early. Which is a little bit annoying, but it's done now.

It's possible there may need to be further downtime tonight (as scheduled originally) to carry out updates, which were meant to have happened at the same time as the disk replacement. This will basically just involve a simple reboot though and shouldn't take anything near as long!

[orudge]

Maintenance all done, everything should be good again now. Sorry for the downtime.