2016 note: These old postings are preserved here even though the specific backup technologies discussed are now ancient, because the issues discussed are still with us: Failure to distinguish backup from redundancy from archiving, failure to do rotational backup, failing to choose among backup options based on the full cost including sufficient media for a rotational backup scheme, using cheap and inadequate and/or proprietary hardware schemes out of ignorance, failure to understand incremental vs. differential backup (and the importance of single points of failure), failure to plan for orderly retirement of backup media, Linux-using gadget freaks gravitating towards inappropriate technology solely out of fascination with it and ignoring practical considerations, and failure to consider capacity, cost-effectiveness, backup speed, verification, cataloguing, and appropriate backup sets. The point is that, if you're truly serious about doing backup, you need to think about such things.

I should also add that backup schemes are at risk of turning out to be worthless when you really need them, unless you do a period restore operation under conditions of simulated host failure.





From rick Thu Mar 14 08:26:27 2002
Date: Thu, 14 Mar 2002 08:26:27 -0800
From: Rick Moen <rick>
To: kmself@ix.netcom.com
Subject: (forw) Re: [ILUG] Cheap tape drives

Relevant to your Linux Backups mini-FAQ:

1. Below you'll find the outline of a battle-tested rotational backup scheme. Useful for realistic costing of backup options.

2. Cleaning tapes! Your FAQ doesn't mention them. They're absolutely crucial for DAT, because it's a helical-scan tape technology. (There are stories I can tell you.)

3. Likewise, if you don't have a scheme for ensuring that old tapes get retired, you're doomed. (More stories.)

4. Differential vs. incremental. Just a reminder.

5. Later in that discussion, another participant made a good case for using DDS3 tapes with a DDS4 drive. Will send, separately.

6. Distribution terms for the FAQ. I might want to mirror, at some point. And, if you haven't addressed them, I might preface it with text of my own regarding points 1-5, above.


----- Forwarded message from rick -----

Date: Wed, 13 Mar 2002 07:10:40 -0800
To: ilug@linux.ie
Subject: Re: [ILUG] Cheap tape drives

Quoting Eoin Phillips (eoinp-ilug@celt.dias.ie):

> I'm in charge of a small server (20GB) used for Samba, and it's high
> time we put a proper backup scheme in place! There's not a lot of
> money for this and I don't want to go to SCSI unless necessary (we
> currently have a UDMA 100 card), so I was thinking of going for a
> Seagate internal IDE drive, using Travan tapes (4/8GB). Has anyone
> any experience of these?

They seem inexpensive, but they aren't, because the tapes cost too much.

> The backup will be done weekly (incremental) and monthly (full) for a
> 10GB partition.

First of all, do differential instead of incremental. (A differential backup is one of all changes to date, relative to the last full backup.) This avoids the first few incremental tapes being single points of failure for restores.

Second, you need to plan on retiring the differential (or incremental) tapes and replacing them on an occasional basis, because they will wear out. And I personally think you should keep the monthly backup for at least several months before overwriting them, and I would urge a full year.

Third, I'd do differential tapes every weeknight (M-Th), and full backups every Friday night. So, you'll have five Friday tapes in service, at any given time (plus four daily ones).

I say daily because you'll be kicking yourself if you ever lose most of a week's worth of work, despite functional backup.

So, for the daily tapes, you might (as a wild guess) need four sets of four tapes for a year. As you say, they wear. And then thirteen Friday tapes for the first year, because you remove one from rotation every month as the monthly archival tape. But then no new Friday tapes for the second and later years, because you're rotating the prior year's back in as they become over a year old (but a new set of sixteen daily tapes per year). And those weekly tapes will probably last until the drive is obsolete.

Given the cost of media, then, I'd say DDS3, not Travan. http://kmself.home.netcom.com/Linux/FAQs/backups.html

2016 note: The Travan / QIC (Quarter inch cartridge) standard for physically large 8mm tape cartridges and drives, also adopted in variant forms by Colorado (later, HP Colorado), Iomega, and Aiwa, was a bad late 1990s idea. Until very late in the evolution of this technology, the drives interfaced from a PC's floppy drive header, hence they were extremely slow, but were perceived as "inexpensive" because of ability to avoid purchasing a SCSI host adapter card. QIC cartridges were very expensive both per piece and per GB compared to competing DAT technology, yet still were perceived as "inexpensive" if you greatly underestimated your backup media requrements, as was sadly common in these discussions. Travan/QIC economics are explored further down this page. It should also be noted that Travan/QIC, unlike DAT, had no write verification, such that users frequently found the hard way that they had only corrupted backup sets caused by dirty tapes, worn recording heads, etc. Again, the point in 2016 is not to documented the deficiencies of a specific bad but obsolete technology, but to learn from the planning errors that lead otherwise intelligent people to use it. That problem is still with us.

Don't forget to buy and use cleaning tapes.

-- 
Cheers,        "Linux means never having to delete your love mail."
Rick Moen                                              -- Don Marti
rick@linuxmafia.com
McQ!  (4x80)

----- End forwarded message -----





From rick Thu Mar 14 08:27:02 2002
Date: Thu, 14 Mar 2002 08:27:02 -0800
To: kmself@ix.netcom.com
Subject: (forw) Re: [ILUG] Cheap tape drives

----- Forwarded message from Wesley Darlington wesley@yelsew.com -----

Date: Thu, 14 Mar 2002 05:06:39 -0500
From: Wesley Darlington wesley@yelsew.com
To: ilug@linux.ie
Subject: Re: [ILUG] Cheap tape drives

Hi All,

Rick's advice is, of course, excellent. I'd just like to add a couple of points...

On Wed, Mar 13, 2002 at 07:10:40AM -0800, Rick Moen wrote:
> Given the cost of media, then, I'd say DDS3, not Travan.
> http://kmself.home.netcom.com/Linux/FAQs/backups.html

I've found DDS4 drives with DDS3 tapes to be a good combination. A DDS4 drive is only about a hundred quid (I think in sterling... :-) dearer than a DDS3 drive, but can write to DDS3 media about twice as fast as a DDS3 drive can.

This way you get the speed of DDS4 and the cheapness (*) of DDS3 tapes. And you can even use DDS4 tapes for your full backups and DDS3 tapes for your differentials, say. Horses for courses.

> Don't forget to buy and use cleaning tapes.

And get a tape drive from a supplier with a no-quibble warranty. One whose first reaction, when confronted with a "my tape drive doesn't work any more" complaint, is to send out a new one immediately, rather than to put you through a laborious cycle of proving it (complete with downtime). *cough* Dell *cough*

Finally, a big advantage of DDS is that it's much easy (**) to borrow a tape drive that can read your media if/when your computer/s and its/their tape drive/s get stolen.

<Insert rant here about the general unsatisfactoryness of tape "technology">

Wesley.

(*) All things are relative.

(**) Compared to, say, some weirdo, proprietary, tape format.

----- End forwarded message -----





Date: Mon, 12 Aug 2002 00:29:34 -0700
From: Rick Moen rick@linuxmafia.com
To: "Karsten M. Self" kmself@ix.netcom.com
Subject: Backups, again

The topic of data backup herewith returns, like a troublesome data set — occasioned by my addressing the matter on a mailing list, and again referring people to your Linux Backups mini-FAQ. Comments will concern that FAQ and surrounding cosmic truths.

Cosmic truth #1: Part of the reason it's a FAQ topic is that people are confused about what a backup is, and what it is not.

These are very distinct concepts, yet many people have them hopelessly confused, and call all of them "backup".

A lot of the people with dumb opinions on the subject have no friggin' clue what it takes to foil Thor and Moriarty: They think quantity one duplicate copy, stored within Mjolnir distance of the server, and overwritten every Saturday night with a fresh data set, is "backup".

Cosmic truth #2: Nobody wants to admit he/she's bought a fake diamond. The more people have bought into cheap-bastard pseudo-solutions, the more they need to convince others, you, and themselves that those are ideal solutions. The more they screwed up, the louder they'll be.

Cosmic truth #3: People are innumerate. Nobody would assert that CDRs are cost-effective backup (even ignoring the 31-CD issue you raise) if they bothered to do the arithmetic. But they never do the arithmetic. If you want to clarify that point, do a table comparing media cost per full backup media set for a 20 GB backup on CDR, DDS3, DLT, SDLT, LTO, AIT, DLT7000, etc.

Include 1.44MB floppies for comic relief. (Actually, it may be time to drop analysis of floppy backup from the FAQ, since I really doubt that anyone will seriously propose that, any more.

You probably do want to include the cost of hard drives in that comparison. Why? Because the 3Ware cult (who do have some good points) tend these days to advocate peeling off half of a RAID1 pair and putting it in a vault as a backup set. Not half bad, actually — but I really don't think it's cost-effective.

FAQ observation #1: The "Hardware" section discusses SCSI in contrast to QIC/Travan. (First of all, you might want to clarify that QIC/Travan uses the floppy port.) However, the buyer's dilemma, these days, isn't usually floppy-port/QIC versus SCSI; it's SCSI vs. ATAPI. And you don't mention ATAPI at all. This is a big hole in your FAQ.

FAQ observation #2: You still don't explain incremental vs. differential in it.

FAQ observation #3: I'm glad you mention storing copies of "fdisk -l". I personally like just doing dd to store sector zero to a file, then mcopy to put that on a (date-marked) floppy. (2016 equivalent: Put thst 512-byte file on a USB pen drive or stored with your backup sets.) That way, you get the boot code as well as the partition table, in one neat package.

FAQ observation #4: Remember I mentioned that a lot of people think a single data set, overwritten every Saturday night with a fresh data set, is "backup"? Your FAQ wording as written isn't going to get through to those people. You say "For a typical single-user system, periodic full archives on a set of rotated tapes should be reasonably sufficient."

Understand that the aforementioned people will read that through the lens of their existing strategic error. They'll think, OK, two tapes to make certain will be good enough. Used in perpetuity. Back before hard drives passed the 2.1 GB mark and kept on going, this reasoning always lead people to buy Colorado Jumbo (QIC) drives, on economic grounds.

And, honestly, two tapes (with replacement as they wear out, and therefore with tracking of their service history) might be enough for a single-user system. But, this depends on consideration of threat models. Which you don't yet discuss.

That is, backups exist to protect you in case of mishaps. Which mishaps are you protecting against? Do you need to be able to recover last week's data state? Do you also need to be able to restore data from six months ago? (If the latter is the case, then two tapes used weekly or monthly won't cut it.) Do you need to be able to recover from fire? If so, then offsite storage is required.

FAQ observation #5: Backups are pointless unless tested. Do you have adequate ability to recover from a fault in the system that causes only useless backups to be made, followed by a hard drive failure? Most people don't. The only practical way to ensure that a backup set is adequate is to do a test restore onto (one assumes) a different machine. Your FAQ doesn't yet cover that, either.

FAQ observation #6: Cleaning tapes, and use thereof. Not mentioned yet at all, in your FAQ.

FAQ observation #7: Some salient aspects of alternative tape technologies are covered here: http://linuxmafia.com/faq/Hardware/tape-backup-types.html It also points to (without further discussion) other criteria that must be weighed in the choice among them.

FAQ observation #8: if you don't have a scheme for ensuring that old tapes get retired, you're doomed.

FAQ observation #9: Remember the Irish guy's observation I quoted to you some months ago, that DDS4 drives can reliably read/write DDS3 tapes, which he considered the most cost-effective combination. You get the speed of DDS4 and the cheapness of DDS3 tapes. And you can even use DDS4 tapes for your full backups and DDS3 tapes for your differentials, say.

-- 
Cheers,   "This is mad, egotistical, sick, twisted, and stretches the bounds of
Rick Moen   good taste right off the tongue, past the uvula, and down around 
rick@linuxmafia.com      the duodenum.  It has other merits, but that should 
           indicate positive interest."  -- The Cube, http://www.forum3000.org/




Date: Fri, 16 Aug 2002 15:39:53 -0700
From: Rick Moen rick@linuxmafia.com
To: kmself@ix.netcom.com
Subject: (forw) Re: [ILUG] Backup solutions

Since I do try to make sure I know what I'm talking about before sounding off in public, I attempted to research ADR, and found it to be a defensible choice. My reply follows:


----- Forwarded message from Rick Moen rick@linuxmafia.com -----

Date: Fri, 16 Aug 2002 15:33:40 -0700
To: ilug@linux.ie
Subject: Re: [ILUG] Backup solutions
From: Rick Moen rick@linuxmafia.com

Quoting Ciaran Johnston (cj@nologic.org):

> Thanks for all the replies.
>
> I see from
> http://www.sosonline.co.uk/Shop/Product_Details.asp?DP=1215&RP=0 that
> I can get a Seagate internal DDS-4 capable drive for Euro1,018.43.
> This is about the limit of my spending. However, this OnStream IDE
> tape drive - http://web6.scan.co.uk/Products/Info.asp?WPID=14022
> (thanks for the link, Martin) - by my calculations provides more
> storage at only slightly slower speeds at half the price. Is there any
> overriding reason why I should go with the more expensive and
> lower-capacity DDS-4 drive other than speed?

The ADR (Advanced Digital Recording) tape format is an interesting option, one I hadn't seen much of, before. Look as if OnStream (spun off from Philips Electronics in 1998) invented and rather tightly control the format, though other companies such as Verbatim do make compatible media under licence.

If you're reasonably serious about backup, you need to plan for a rotational backup scheme and retirement of tapes as they become worn (and well before they start failing). That requires at least a couple of dozen tapes for the first couple of years. Therefore, cost and availability of media are (or should be) a major factor. Also, tape heads wear out and need to be replaced (more rapidly for helical-scan systems — which ADR turns out not to be), and somehow that always ends up being cheaper for tape technologies in which there is heavy competition.

The less-tangible consideration that comes most immediately to mind is that, if one's OnStream ADR drive fails, the only possible replacement that would suffice to restore your accumulated backup sets is another OnStream ADR drive. In that sense, you're somewhat locked in: I'm not seeing ADR drives from anyone else (though I might have missed them).

If OnStream are smart, they've priced the drives to attract people into the system, and are making up the shortfall on media cost after they get people to buy in. I see the following prices for mail order (http://www.finnsoft.com/priclist/verbat.htm) of Verbatim-brand tapes:

42 euros for 30 GB ADR tape
33 euros for 30 GB ADR tape, four-pack.
66 euros for 50 GB ADR tape.
44 euros for 50 GB ADR tape, four-pack.
47 euros for ADR cleaning tape.

(You don't get much lower pricing than Verbatim, without going no-name.)

Capacities are usually quoted on the basis of a nominal 2:1 compression ratio. The PDF datasheet for Onstream's "ADR2" series says this is a linear-serpentine recording technology (a good thing) like DLT, SDLT, and LTO, rather than a helical-scan method typified by DDS/DAT, AIT, and 8mm (which puts heavy wear on tapes on heads).

I'm not sure what software other than NovaStor's TapeCopy and Yosemite Technology's Tapeware will support OnStream's drives. My recollection is that BRU will do it.

On balance, I'd personally still go with a DDS4 drive using either DDS3 or DDS4 media, mostly because once you've survived life with one oddball, less-standard tape format, you're not in a hurry to repeat the experience. But ADR does seem at a quick glance to be well designed, and might be a good bet after all. As they say in the gaming industry, "Ya pays your money, and ya takes your chances."

-- 
"Is it not the beauty of an asynchronous form of discussion that one can go and 
make cups of tea, floss the cat, fluff the geraniums, open the kitchen window 
and scream out with operatic force, volume, and decorum, and then return to 
the vexed glowing letters calmer of mind and soul?" -- The Cube, forum3000.org

----- End forwarded message -----





From rick Fri Aug 16 19:04:19 2002
Date: Fri, 16 Aug 2002 19:04:19 -0700
To: kmself@ix.netcom.com
Subject: (forw) Re: [ILUG] Backup solutions

This is one variation on the sort of solution people are promoting as so-called "backup", on LUG mailing lists. At least the guy has bothered to do the math, and realises that the scheme isn't economically viable past about 3-4 generations of data.

The other variation is periodically peeling off half of a RAID1 pair, as one's backup medium.

----- Forwarded message from Vincent Cunniffe vincent@cunniffe.net -----

Date: Fri, 16 Aug 2002 15:48:49 +0100
From: Vincent Cunniffe vincent@cunniffe.net
To: ilug ilug@linux.ie
Subject: Re: [ILUG] Backup solutions

Ciaran Johnston wrote:
>Hi folks,
>I maintain a colocated server on behalf of a small group of individuals,
>and am looking at backup solutions. Is it possible to get some sort of low-
>end internal tape / other solution that could be used to back up approx. 40
>Gigs of data or am I just dreaming? My ISP does offer backups at extra cost
>but the only problem with that is, well, the extra cost.
>
>What I was hoping to do was to install some kind of internal tape device,
>then swap tapes round every month, so I had an onsite backup of say the
>last 24 hours and an offsite backup of the last month. Is this feasible?
>I'm beginning to think it isn't. External devices are not an option as part
>of the charge for colocation is rackspace.

I run several co-located servers, and the solution I have adopted is IDE drives. They're cheap, fast, and you don't have to keep buying media for them. Combine it with a removable HD caddy from Peats or Maplins, and you have a complete onsite/offsite backup solution with 4 entire generations of data for about €250 (2 * 80GB/5400 drives and a caddy for the server).

You do need 60 seconds of downtime to replace the drive, but that's pretty trivial if done once per month.

Regards,

Vin

----- End forwarded message -----





From rick Wed Jan 8 16:00:47 2003
Date: Wed, 8 Jan 2003 16:00:47 -0800
To: conspire@linuxmafia.com
Subject: Re: [conspire] Burn DVDs...
User-Agent: Mutt/1.4i

Quoting Jose Sanchez (jose_sanchez79@yahoo.com):

> It is very true what you have posted. I get "maybe it will work" from
> almost everyone I talked too. I need this solution to backup important
> data not movies and stability is what I am after. I might just have to
> put a tape drive instead :)

Yeah. Doing backup to DVD sounds attractive and nicely high-tech, but I'm not sure you'd like the results:

1. Capacity is OK (4.7 GB). About right for home machines, I guess.

2. Cost-effective media? I haven't priced the blanks, but suspect they're pretty damned expensive per MB, compared to, say, DDS3 tape.

3. Backup speed? Is each backup going to take a dog's age? I'm not sure.

4. Verification. A backup is worse than useless if you can't trust its integrity. (If you make unreliable backups, you're suffering a false sense of security. If you know that you have no backup, you're at least scared and cautious.) Standard backup mechanisms have verify cycles and/or extra tape heads that do hardware-level read-and-compare, to ensure that you actually have useful data.

5. Cataloguing and media rotation/reuse. Regular backup mechanisms keep records of what data are on which backup sets, so you can determine which set has the last version of your master's thesis before you accidentally deleted it. And they also tell you how much wear each backup set has gone through, and therefore which ones to recycle next and when to retire sets from service. (Of course, if you're using write-once media, this isn't an issue.)

6. Software support for incremental and differential backups, not just "grab everything" backup jobs. Easy to do with tape; can you do it with DVD?

All of the above issues come up all the time on Linux mailing lists, whenever someone proposes goofy ideas like "Hey, lets do backup onto CDRWs!" just on account of being gadget freaks and not thinking through the details. And you'll hear a lot of really bad advice in most such discussions.


There's also a difference between backup and archival storage. Are you sure you're trying to do the former and not the latter? I'll not get into the difference right at the moment, but it has to do with what you're trying to protect against.

Maybe you'll feel like expanding a bit on what you're trying to accomplish, if you'd like to get better-aimed advice.

I have a partial summary of common tape-backup types, here (omitting some crucial details such as pricing): http://linuxmafia.com/faq/Hardware/tape-backup-types.html

Some rather acidly worded mail from me to Karsten Self regarding the gadget-freak problem in backup discussions is preserved here: http://linuxmafia.com/faq/Admin/backup-strategy.html (Also includes explanations of differential vs. incremental and backup vs. archival storage.)

'Hope that helps!

-- 
Cheers,                                      "My file system's got no nodes!"
Rick Moen                                    "How does it shell?"
rick@linuxmafia.com  
McQ!  (4x80)




From rick Sun Apr 13 03:07:16 2003
Date: Sun, 13 Apr 2003 03:07:16 -0700
To: luv@luv.asn.au
Subject: Re: Tape backup

Quoting Andre Marenke (andre@arintech.com.au):

> For that much we would usually install a Travan drive. Would sure be
> nice to replace it with a DDS4!

Installing a Travan drive for meaningfully systematic backup doesn't make economic sense, and hasn't for a very long time. Cartridges are far too expensive (US $26 in quantity for TR-4 4/8GB format, versus US $8 for DDS3 [1] 12/24GB format), and capacity is low. They're also slow. Ignoring the capacity and speed issues, consider just the tape count:

For a branch office or small business, you might start the year with four tapes for Monday, Tuesday, Wednesday, and Thursday differential backups — and ten tapes covering full backups on 1st, 2nd, 3rd, 4th, and 5th Fridays, so that you have two months' full backups and weekly intervals at any given time. The first backup of every month would be removed and sent offsite for archival storage (12 tapes). Half-way through the year, you'd probably want to retire and replace the differential tapes (on account of wear). So, that's 4 + 10 + 12 + 4 = 30 tapes for the year — a US $540 (AUS $900) difference between Travan and DDS3.

So, I imagine that thirty Travan tapes plus a Travan drive will always cost significantly more than 30 DDS3 tapes plus a DDS4 drive. The economics have worked out that way, in my experience, for well over a decade.

About the only way it doesn't work out that way is if you kid yourself into believing you need only a few tapes. Which in my view means you're just not serious about backup.

More at: http://linuxmafia.com/faq/Hardware/tape-backup-types.html
http://linuxmafia.com/faq/Admin/tape-backup.html
http://linuxmafia.com/faq/Admin/backup-strategy.html


[1] DDS3 tapes in a DDS4 drive is an excellent combination, getting you the economy of DDS3 media with DDS4 speed. If you have very large backup sets, you could use DDS4 for the full backups and DDS3 for the differentials.

-- 
Cheers,     Founding member of the Hyphenation Society, a grassroots-based, 
Rick Moen   not-for-profit, locally-owned-and-operated, cooperatively-managed,
rick@linuxmafia.com     modern-American-English-usage-improvement association.
McQ!  (4x80)




Date: Mon, 22 Dec 2003 00:02:31 -0800
From: Rick Moen <rick@linuxmafia.com>
To: luv@luv.asn.au
Subject: Re: USB drive enclosure review

Quoting Rob Hodge (rshodge@optushome.com.au):

> One final suggestion for a backup solution for you.....have you
> thought of using a DVD-RW for backups?? DVD writers are very cheap at
> the moment. You can get a Pioneer 4x + and - DVD-RW burner for under
> $200. DVD-RW media is about $6 a disk...much cheaper than 2 x 200GB
> HDD's....not to mention the price of the New Motion enclosures....but
> the New Motion stuff look really nice!!!

Capacity per DVD-RW disc is 4.7 GB. (I'm ignoring higher advertised capacity figures that rely on data compression.)

Envision full backup of a nearly full 80 GB hard drive: That's 18 DVD-RW discs. Seventeen changes. Wow, that's a lot of babysitting. And what software manages all this? Also, where's the error-detection and correction (so that you know your backup is readable)? Where's the cataloguing of what files (as of what time stamp) was written to which disc? And how long does the backup take, given really slow DVD-burnning speeds and ignoring time the backup isn't progressing because you weren't instantly there for one of the 17 changes?

And how many generations of backup do you plan to keep in rotation? Would that be a rotation cycle of one set? Two sets? Three?

Personal opinion: With fewer than twelve sets, one per month, you're not being serious about backup. You might justify recycling the January 2003 set in January 2004 — but otherwise, how can you preserve the ability to restore month n's data after discovering that your hard drive's data started getting corrupted during month n+1?

Cost: AUS $37.50 DDS4 20GB DAT media. I saw a Dell DDS4 drive for AUS $600 on eBay. Comparing cost for a full 80 GB backup media set:

Thus the tape backup would probably be a little more expensive initially, but a great deal faster, more convenient, and able to do systematically and reliably.

Economics, however, is also affected by expected number of re-use cycles. How many times can a DVD-RW disc reliably be erased and rewritten? I figure they're good for a few cycles. DAT tape media, by contrast, are good for many, many erase/write cycles.

And any DAT or better tape backup will be an order of magnitude or better faster than DVD-RW.

As always, your mileage may differ.

-- 
Cheers,        "A raccoon tangled with a 23,000 volt line, today.  The results
Rick Moen       blacked out 1400 homes and, of course, one raccoon."
rick@linuxmafia.com                                  -- Steel City News