[conspire] FileZilla isn't a file manager (but you can use it as one) (was: FTP)

[Michael Paoli]

> From: "Rick Moen" <rick at linuxmafia.com>
> Subject: Re: [conspire] FileZilla isn't a file manager (but you can  
> use it as one) (was: FTP)
> Date: Tue, 17 Nov 2020 12:03:51 -0800

> The ftp protocol is problematic for firewalls on account of how it
> connects, which is one of the reasons it's been little used for about
> the last 20 years.

Not to mention it still being unencrypted!  So, e.g., if one is using
password to authenticate, that goes across the network in the clear,
at least if one is using FTP and not FTPS.  And of course with sftp
(which is a ssh subsystem) that's a non-issue, as that's encrypted*.
*well, possibly excepting if one goes through extreme gyrations of
non-default configuration, and probably also compiling, to force it
to do otherwise (e.g. use the "null" encryption algorithm for testing).

>> Moving files within a particular system, I would either use mv or  
>> cp or whatever GUI file program was installed.  I've usually be OK  
>> with the default that comes with the installation.

Yes, locally, or remote, I'll almost always be doing such through
various CLI means.  E.g. among them may use/involve any of:
cp, mv, tar, pax, cpio, ssh, rsh, find, sh, vi, grep, sort, file,
stat, etc.  I may often use shell's fc capability, then within
vi, do whatever editing and running of commands, etc., to compose
the command proper to deal with exactly what I do/don't want it to
deal with in terms of file manipulations - then execute that command.
  
> A good basic way of copying files is
>
>     rsync -avz olddirectory/ [host:]newdirectory/

I do quite like rsync ... but I also quite dislike at least some
bits of its default behavior.  What, ... edge cases?  Well,
yes, very much so.  But edge cases that may also very much violate the
Principle of Least Surprise.
https://en.wikipedia.org/wiki/Principle_of_least_astonishment
E.g. - these could as well be to/from a remote host, but for
simplicity/example, I'll do it on same host - but over ssh (to
also help illustrate how it would be to/from remote):
$ cd "$(mktemp -d)"
$ mkdir d1 d2
$ cat < /usr/share/dict/words > d1/f
$ tac < /usr/share/dict/words > d2/f
$ touch -m -r d1/f d2/f
$
// So, now we have two files, different content, but same length and
// mtime:
$ cmp d[12]/f
d1/f d2/f differ: char 1, line 1
$TZ=GMT0 stat -c '%y %s %n' d[12]/f
2020-11-18 05:59:19.148152628 +0000 972398 d1/f
2020-11-18 05:59:19.148152628 +0000 972398 d2/f
$
// Now let's try to make d2/f matched to d1/f ... we'll also use most
// all reasonably suitable options to try and make them identical,
// including also attributes and extended attributes, etc.
$ rsync --archive --acls --xattrs --hard-links --sparse --rsh=ssh  
"$(pwd -P)"/d1/ localhost:"$(pwd -P)"/d2/
$ echo "$?"

$ cmp d[12]/f
d1/f d2/f differ: char 1, line 1
$
// But note that the files still don't match!
// Let's change our rsh command very slightly and try again:
$ rsync --archive --acls --xattrs --hard-links --sparse --rsh=ssh  
--checksum "$(pwd -P)"/d1/ localhost:"$(pwd -P)"/d2/
$ echo "$?"

$ cmp d[12]/f
$

So, in the last example bit above, notice that time they actually ended up
being matched, but not in the rsync command immediately before that.
What was the difference?  In the latter, I added the --checksum option,
which isn't rsync's default.  That forces rsync to actually read the files
and calculate hashes of chunks of the files and compare those to see if
they match or not.  Without that non-default option, rsync defaults to
taking some shortcuts and making some presumptions.
Most notably, if the filename matches, the length matches, and the
mtime matches, rsync will presume the contents are the same, and not
read the contents and not calculate hashes and compare those hashes.
I don't think I'd mind at all if the default were the other way around,
and one instead had an option something like
--use-possibly-dangerous-shortcuts-that-may-miss-stuff
... but that's not how rsync is ... by default.
However with some suitably well and carefully chosen options,
rsync can work quite well, and without such possibly unpleasant
surprises.  (E.g., just think, if by coincidence, or intent, someone
manipulated file mtime and length to avoid having a file copied.  What if
you were relying upon that for your backups or audit trail?  Yes, even
rsync's --archive option doesn't disable that potentially hazardous
shortcutting).  So, that's why I generally advise on rsync - use due
caution ... great tool, but it has some potentially hazardous default
behavior.)

> Page goes on to partially satisfy our resident edge-case fanatics with
> small improvements on that recipe.

Hey, I resemble that remark!  ;-)

> by running an ftpd on the remote host.  Which means among other things
> firewall headaches.  (Diehards say you deal with that by initiating the
> ftp connection in 'passive ftp mode'.  I'm not even going to go further
> into that.  Look it up, if curious.)

Yep, passive mode gets around FTP's default port funkiness, thus better
dealing with firewalls.  But that also has some drawbacks in
functionality, and does nothing for encryption, and also often many
clients (or maybe even servers? - not sure) don't support passive
mode, or it may be difficult/inconvenient to configure them for
passive mode (may be buried deep in some configuration file somewhere,
maybe not even showing the options there by default, or buried way down
in some layers of menu settings, or dear knows).

> For the record, I really cannot understand what insisting on a graphical
> tool gets you over just typing 'rsync -avz filetree linuxmafia.com:/tmp'
> It's dead-simple, logical, and Just Works.[tm]  But OTOH, it's always
> cool to like what you like because you like it.

I find it exceedingly rare - at least for myself, that some GUI tool for
managing files would be to advantage.  And consequently I use such
almost never.  But I could see some scenarios where it might be
advantageous.  E.g.:
o if one is in environment where CLI sucks or is
   (effectively) inaccessible or non-existent.
o where one quite wants to pick and choose which files/directories to be
   dealt with, e.g. manually selecting/deselecting individual files and
   groups of files, directories, etc., (re)sorting them by any number of
   possible various criteria, and further refining by (de)selecting more
   files, directories, etc., until one has fully (de)selected exactly
   what's (un)desired, and then can execute the operation (move, or
   copy, or what have you).
But, at least for myself, generally using Linux (or *nix) environments,
and being well familiar with CLI tools (and fine text editor, etc.),
and shell's edit capabilities, etc.,  I can probably (darn near?) always
do these types of (de)selections to build up the desired command operation,
faster and better in CLI.  However, for others that may not be nearly so
familiar with CLI ... and/or want a (much) more visual representation,
I could certainly see where GUI (or even menu/text) interface could be
to advantage.