[sf-lug] SF-LUG (& BALUG) VM infrastructure, etc. [Re: "all better" (up again): Re: (temporarily) down: SF-LUG (but lists up) ... likewise some BALUG bits down ...]

[Rick Moen]

Quoting Michael Paoli (Michael.Paoli at cal.berkeley.edu):

> For various reasons, also, laptop isn't set up to boot, etc. when power
> is restored to it from a powered down state (and for various reasons I'm
> not currently planning to change that, even if it might be feasible).
> So ... once battery charge was too low, things went down (they were
> already offline), and stayed down until wee bit 'o manual intervention
> was applied.

This is one of the few disadvantages of the otherwise genius idea of
using a laptop as the poor man's host + UPS -- an idea pretty much every
Linux server administrator has entertained from time to time.

Laptops all, to my knowledge, remain in their current state when AC power
is restored.  Thus, a laptop that went to 'off' state upon exhausting
its battery remains that way when AC comes back.  In this, they have the
same problem that almost all ATX power supplies used in workstations do: 
Such PSUs typically go to a 'standby power' mode after restoration of AC
power, and there's no way to change that.

And that is one of the several reasons why workstation boxes tend to
make really bad Linux servers:  They don't come back online after power
outages.

My wife didn't realise this error when she migrated her deirdre.org
domain to a ShuttlePC box (at our house in Menlo Park), circa 2004,
realising it only after the first PG&E power glitch.  Her semi-solution?
buying a UPS, and placing the ShuttlePC behind it.  This had the same
effect, and the same limits, as using a laptop with ~1 hour battery
runtime:  It meant that her site now returned online after short power
outages, but still went down and remained down after long ones.

The only real (complete) fix, to the best of my knowledge, is to use
server-type equipment, which has the desired PSU behaviour.  But, man, a
laptop is certainly an excellent 90% solution, isn't it?


['vicki':]

> And ... powered up?  Depends a bit how busy it is, but powered up and
> idle, it suck about 113W or so.  And powered up and actually doing some
> work (e.g. disk I/O and/or more CPU activity) ... nominally it goes up
> to around 115W.  Not a huge range, ... but too, not a whole helluva lot
> for it to do (only has two hard drives, 2 GiB of RAM if I recall
> correctly, some pretty good CPUs, but most of the time more CPU
> horsepower than its common workloads really need or can put to use).

And doubtless a bit of noise, too.

Pretty soon, I'll check using my Kill A Watt power meter to see how much
AC power my long-delayed next-generation home server draws.  It's a
CompuLab Intense PC with a pair of external SSDs.  No moving parts
anywhere (e.g., no fans), so it's utterly silent.  

Kill A Watt meters are probably pretty inaccurate at low power
draws, though.  Could be a problem.
http://forums.anandtech.com/showthread.php?t=137169

> Other random thing on high(er) availability possibilities.  At some
> point, may want to add more virtual/cloud into the mix.  E.g. can do
> some AWS stuff for free to cheap.

Also worthy of pondering:  any of several different types of 'hot spare
host'.

One type that doesn't aspire to instant _or_ automatic failover, but IMO
is easily good enough for any LUG server I know of, involves a second
(failover) machine kept either constantly or periodically synced with
the main (primary) host.  The failover has a completely different IP.
Ideally, it would be in a different building on different routes /
provider, and certainly not on the same electrical circuit.

The failover host would rsync over data files from the primary, so at
any given moment it would have very close to perfect replication.  (I
submit that, for a LUG server, nobody's going to cry if you lose < 1
hour of state, so an hourly cron job would be adequate.)

If the sysdamin notices that the primary is down and it looks serious,
then repoint the DNS.  TTL, yeah, caching, yeah.  But still, pretty fast
changeover.   An more than robust _enough_.  

For extra credit, scripting and heartbeat monitoring could make the DNS
changeover automatic or semi-automatic, though I have my doubts about
this being desirable.

The hot spare would also give you backup for free.

And none of this requires paying Amazon or other commercial cloud
service.  So, again, LUG-appropriate, IMO.


> Another thought ... Raspberry Pi.  Could at some point take what's
> presently on VM, and run it on Raspberry Pi.

I think a RPi2B would make an interesting LUG host, and also would make a
good dedicated backup / configuration management host.  RAM capacity on 
Raspberry Pi 2 Model B (current best model) is 1GB, which is enough for
meaningful VM but only barely IMO, mostly because of antispam in your
MTA typically gobbling lots of RAM.  I'd be fascinated to see it tried,
though.

RPi and other ARM options make me a bit concerned on account of reliance
on out-of-tree kernel patches.  Imagine a big kernel security flap.
Oops, you cannot just compile mainline code.  You have to wait for the
port.  There are times when waiting for ported code is a problem.

That is the main reason I went with x86_64 for the new server prototype,
and not ARM.  Of course, mainline support for real ARM machines is
supposed to happen Real Soon Now, but not today.