[sf-lug] Linux backup software .. that meets unique requirements

[David Hinkle]

> Not sure I understand.  Does an sshfs file system work differently than 
> a regular remote file system in this regard?  A typical rsync over ssh 
> won't wind up having to read every file?

No, a typical rsync over ssh passes file checksums back and fourth.   Once it detects a file that doesn't match it breaks it into blocks and trades checksums again.  The end result is that most of the time only small portions of only changed files need to be transferred.

> I'm not sure I understand how this would work.  I want to rsync from 
> <local src> to <remote dest>, with the <local src> being the unencrypted 
> data on my server, and <remote dest> being the unencrypted remote file 
> system.  How would it be possible to introduce encfs alone into the mix 
> to make this be encrypted?

My assumption, based on the faq and cursory examination of the website, is that this is like other encrypted filesystems with which I'm familiar.   A file containing encrypted data is mounted as if it were any other filesystem on the remote end, and you can use standard rsync symantics to sync to it.  Then you unmount the filesystem and all that's left is the data stored encrypted in a file.   The difference between this and something like cryptfs would then be that this filesystem is userspace only, so you shouldn't need root to make it happen, but you might need root to get it installed anyway.

You've got me intrigued, if this is a niche that can't be filled maybe I should write some software for it, I'm going to post a proposal to the lug and see what people think.

David



-----Original Message-----
From: sf-lug-bounces at linuxmafia.com [mailto:sf-lug-bounces at linuxmafia.com] On Behalf Of David Rosenstrauch
Sent: Monday, March 15, 2010 1:18 PM
To: sf-lug
Subject: Re: [sf-lug] Linux backup software .. that meets unique requirements

On 03/15/2010 12:33 PM, David Hinkle wrote:
> That's a neat trick, but isn't the contents of your encrypted
> filesystem available for browsing on the remote end while your backup
> is taking place?   From looking at the FAQ my understanding is that
> EncFS mounts an encrypted filesystem as unencrypted and available to
> the OS, and then SSHFS simply provides a transport mechanism to make
> any generic directly appear at the remote end over an ssh link.

I don't think that's an issue.  The way encfs works is that it maps an 
encrypted directory to an unencrypted one.  So if I choose the encrypted 
directory to be an sshfs remote dir, and the unencrypted one to be a 
local mount point then this isn't an issue.  The clear text data will be 
local, and the remote data will be encrypted.


> If my understanding is correct, and that's acceptable, I'd recommend
> using a more standard encrypted filesystem and using rsync's built in
> SSH transport.

I'd be open to it, except for a couple of things:  1) I don't know of 
any other FS that works similar to Encfs, and 2) remember that I have no 
control over the remote file storage.  All I essentially have is sftp 
access to it, so I can't install things on it.


> The SSHFS website indicates there are posix problems
> related to SFTP (Which it uses for a transport, unlike rsync) not
> being able to atomically move files over other files, which causes
> problems with subversion, among other things.

That is a bit worrisome.  I'll have to look into that.


> Also, I expect you'll run into pretty stiff performance problems if
> you try to use rsync to rsync against an SSHFS mounted directory.
> Rsync is going to try to read every file in it's entirety if you have
> checksumming on.  This will require a complete download of the
> repository one file at a time.   You can leave checksumming off, and
> then you'll be fine for files that have the same date and size, but
> any other file (i.e. files that have changed) will be downloaded in
> it's entirety and then re-uploaded, which is still suboptimal.

Not sure I understand.  Does an sshfs file system work differently than 
a regular remote file system in this regard?  A typical rsync over ssh 
won't wind up having to read every file?


> I don't know if you've ever used Linux's built in crypto filesystem
> stuff, but if you haven't here's a quick intro.  You create a file
> large enough to hold all your data, then turn that file into a device
> node using the loopback device.  You can then mount any crypto
> filesystem on top of that that's supported by your kernel  (Or any
> other filesystem for that matter, you can treat it just like a disk).
> Though, all of this probably requires root and I don't know if you
> have root.  If you do, however, you won't beat the performance of
> this solution.

But again, I have no control over the remote server, so I can't set up a 
looback crypto filesystem there.



> Another option is to use fuse's encfs (http://www.arg0.net/encfs)
> alone and back up to that directly with rsync, skipping potential
> compatibility problems with the SSHFS transport by using the tried
> and true rsync over ssh transport.  I've never used fuse before, so I
> can't attest to it's effectiveness.
>
> David

I'm not sure I understand how this would work.  I want to rsync from 
<local src> to <remote dest>, with the <local src> being the unencrypted 
data on my server, and <remote dest> being the unencrypted remote file 
system.  How would it be possible to introduce encfs alone into the mix 
to make this be encrypted?

Thanks,

DR

_______________________________________________
sf-lug mailing list
sf-lug at linuxmafia.com
http://linuxmafia.com/mailman/listinfo/sf-lug
Information about SF-LUG is at http://www.sf-lug.org/