[nylug-talk] CD-R as archival media

[David Panofsky]

Ok, so I have some data which I don't need frequent or quick access to, but
which I'll almost certainly need to use in the future.  I want to archive it
so I can free up some space on my system.  I've been planning on just burning
the files to a couple of CD-Rs, but I'm concerned about the possibility of
data loss due to media degradation.  I'm looking for an efficient method to
minimize my risks, and I don't really care if the data is stored in a standard
or user friendly way.  Note, I'm looking for technological advice, not
information about off-site or fire-proof storage, etc.

First off, I'm wondering if any of you have an opinion of how likely I am to
have any problems down the line.  I know that the red-book format has some ECC
built in which helps with small scratches, but I'm sure we've all experienced
cases where that is not enough protection. I have the sense that CD-Rs are
pretty likely to develop significant errors, but if I'm being overly paranoid,
just let me know....

The simplest thing is probably to make multiple copies of each disk on the
assumption that no two will fail in exactly the same way. This can get
cumbersome with a lot of data, but I'll do it if nothing better comes up.  I
was also thinking I could use an old, small (by today's standards) HD to hold
one data copy and CD-Rs for other copies on the assumption that only one of
the two media types is likely to fail in a given storage environment.

Here are some ideas I've been having which use some sort of Forward Error
Correction scheme to add redundancy to the data set, but none is ideal:

1) For each 700MB CD-R, make an (X) MB ISO file and use something like RAS
(http://www.cleaton.net/ras) to create a (700-X) MB file which could be used
to recreate the original ISO assuming errors are somewhat limited.  Both files
would be burnt onto the CD as separate sessions.

2) Same as above, but run algorithm on tgz file and make ISO out of tgz, the 
generated redundancy information, and the software/instructions for recovery.

3) Use same method with the complete set of standard ISO images to create a
single "redundancy CD" which could be used to recreate any damaged CD image
from the rest of the set.  The RAS software was actually written for this
purpose, but using floppies. This method would be a pain because of the large
temporary storage spaces involved to create/use the redundancy information,
but unlike the other options would survive a TOC failure of one of the disks.

4) Make a tar file of the data and then use rar (http://www.rarlab.com) to
compress, split, and add redundancy to the file.  Each segment then goes on a
CD.  My main concern here is that while rar is a cool archive format for
including redundancy options, it is not free software.

Anyway, I've already wasted enough of your time, but would love any advice
regarding this matter which you might have. Possibly there is already software
to do this sort of thing, or maybe a filesystem / archive format which has
error correction which I could burn onto the CDs.

  Thanks in advance,
  dave p.

__________________________________
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.
http://search.yahoo.com