[sf-lug] Linux backup software .. that meets unique requirements

[David Hinkle]

That's a neat trick, but isn't the contents of your encrypted filesystem available for browsing on the remote end while your backup is taking place?   From looking at the FAQ my understanding is that EncFS mounts an encrypted filesystem as unencrypted and available to the OS, and then SSHFS simply provides a transport mechanism to make any generic directly appear at the remote end over an ssh link.

If my understanding is correct, and that's acceptable, I'd recommend using a more standard encrypted filesystem and using rsync's built in SSH transport.   The SSHFS website indicates there are posix problems related to SFTP (Which it uses for a transport, unlike rsync) not being able to atomically move files over other files, which causes problems with subversion, among other things. 

Also, I expect you'll run into pretty stiff performance problems if you try to use rsync to rsync against an SSHFS mounted directory.  Rsync is going to try to read every file in it's entirety if you have checksumming on.  This will require a complete download of the repository one file at a time.   You can leave checksumming off, and then you'll be fine for files that have the same date and size, but any other file (i.e. files that have changed) will be downloaded in it's entirety and then re-uploaded, which is still suboptimal.

I don't know if you've ever used Linux's built in crypto filesystem stuff, but if you haven't here's a quick intro.  You create a file large enough to hold all your data, then turn that file into a device node using the loopback device.  You can then mount any crypto filesystem on top of that that's supported by your kernel  (Or any other filesystem for that matter, you can treat it just like a disk).  Though, all of this probably requires root and I don't know if you have root.  If you do, however, you won't beat the performance of this solution.

Another option is to use fuse's encfs (http://www.arg0.net/encfs) alone and back up to that directly with rsync, skipping potential compatibility problems with the SSHFS transport by using the tried and true rsync over ssh transport.  I've never used fuse before, so I can't attest to it's effectiveness.

David

-----Original Message-----
From: sf-lug-bounces at linuxmafia.com [mailto:sf-lug-bounces at linuxmafia.com] On Behalf Of David Rosenstrauch
Sent: Monday, March 15, 2010 10:11 AM
To: sf-lug
Subject: Re: [sf-lug] Linux backup software .. that meets unique requirements

On 03/11/2010 03:03 PM, David Hinkle wrote:
> Have you considered writing a script that produces a copy of the tree
> you're working on, but encrypts each file with a simple symmetric
> cipher using the filename as the initialization vector for the
> encryption?   Then you can run rsync to back that tree up.   Rsync
> will have to copy any file that changes in it's entirety, but the
> data should be safe.  You could take it one step further and cache
> the md5sums of the files before you encrypt them (like duplicity) so
> you can avoid re-encrypting files that haven't changed.   An approach
> like that might work well enough for a source code repository, but
> will be bad for anything like a database.
>
> David

Thanks much for the suggestion, David.

This is indeed the way I was leaning.  But I wasn't too thrilled about 
having to go through 2 full rsync steps - particularly because it's a 
pretty sizable file system tree.

Fortunately I think I've found a neat workaround:  Continue backing up 
as I've been (i.e., using rsync/rsnapshot) but have the backup process 
backup to a mount point that is an encfs encrypted directory on top of 
an sshfs remote mounted directory, as described here: 
http://wiki.rdiff-backup.org/wiki/index.php/BackupToEncfsAcrossSshfs

Result meets all my requirements:  files are backed up using rsync, to 
my existing storage space, and are encrypted both in transit and on disk.

Going to give this a whirl tonight.

Thanks much for helping me bat this problem around with you, David.

DR

_______________________________________________
sf-lug mailing list
sf-lug at linuxmafia.com
http://linuxmafia.com/mailman/listinfo/sf-lug
Information about SF-LUG is at http://www.sf-lug.org/