[conspire] Another motherboard was _not_ burned out today

[Rick Moen]

This morning, my server was cut off from power, and that was A Good
Thing[tm].

Let's roll the timeline backwards a bit, then forward again.  


Saturday, January 24, 2015:

We had a CABAL meeting and I finally caught up on two more
deferred tasks:

1. Swapping out a failed drive forming half of a mirrored pair, and
remirroring the RAID1 filesystems (/home, /var/lib, /usr/local,
/var/spool, /var/www) onto a replacement hard drive.

2.  Finally putting the machine behind a (very good) AC power
conditioner unit, about which more below.


~Sunday, January 4, 2015:

Michael Paoli and I work out that, against all probability, I indeed had
suffered another Intel L440GX+ motherboard failure, and so now move my
hard drives and RAM to yet another spare VA Linux Systems 2230 system
box and PSU.  Fixing the software problems that were simultaneous with
the hardware failure on August 27th takes a while after this.  (I am now
finally out of spares, by the way, but it's past time to cease using
2001-era PIII servers anyway.)


Wednesday, September 3, 2014:

I swap out another failed motherboard, after finding that it had burned
out while I was on holiday in the UK and Ireland.


Wednesday, August 27, 2014:  In the middle of a software upgrade, I get
a kernel panic and then the machine fails to even produce video at all.  
Eventually, I isolate the cause to burned-out motherboard and swap it
for a spare Intel L440GX+.


2010:  

Deirdre's ShuttlePC-style lunchbox-sized Celeron server that she's had
on house voltage dies and appears to be completely burned out.  She's
had it running since 2004, but it's now totally destroyed.  She moves
her domains to a VPS provider.


Wednesday, April 15, 2009:  Power fluctuations destroy my VA Research
model 500 server including all hard drives, the motherboard, all but one
stick of RAM, and the power supply unit.  I rebuild onto a spare VA
Linux Systems 2230 box and restore from backup.  Time to rebuild:  3
hours for basic server function, 2 days for restoration of all
services.[1]



Rolling forward again to this morning:  My server is buffered from the AC 
supply by a Furman PS-8R Series II power conditioner / sequencer that I 
got for very little money at a De Anza College Electronics Fleamarket.  
It came with no manual, but there is one here:
http://www.furmansound.com/pdf/manuals/PS-8R_II_manual.pdf

This morning, my server wasn't pingable.  A visit to console showed it
to be powered off.  The Furman unit underneath it showed a red LED
marked 'Extreme Voltage'.  Quoting the manual:

   If the unit has been operating with an acceptable input voltage 
   and subsequently that voltage exceeds 135V, it will shut off power 
   to the outlet and the Extreme Voltage LED will light.

OK, so we have power spikes for some unclear reason, and server hardware
is at risk if not behind quality power conditioning.  


Yay, Furman PS-8R Series II!

I'll not even be unhappy about the need for a manual power reset when
this happens, long as it happens not too often.  ;->


I also own a (spare) APC-branded, smaller, less heavy duty power
conditioner unit, currently still in retail box. I think I'll deploy it
with my planned back-end server that will serve as the house LAN's
regular backup target, NIDS box, and configuration management master.
Possibly a Raspberry Pi 2 with attached hard drive.


[1] Large amounts of data had to be rsync'd from offsite storage on a
Joyent.com OpenSolaris box.  Because the data had been rsync'ed to there 
using Deirdre's non-root customer UID, all file-ownership metadata had
been flattened out (lost), and so I had to very carefully chown data
subtrees as needed, and rebuild many services manually to get ownerships
right.