[sf-lug] Example with more structure/comments Re: program comments/examples, domain(s), DNS, ... Re: sh, awk, Perl, ...
Michael Paoli
Michael.Paoli at cal.berkeley.edu
Fri Jun 7 08:34:36 PDT 2019
And if y'all want an example with more structure/comments ...
The below example I did highly recently (started yesterday
evening, finished(? - at least thus far) this A.M. - as based
on some earlier bits that were much more specialized on specific
to/from pair hardcoded in each, this is significantly more
generalized version (at least for my VM infrastructure)):
# < Live_Migrate_from_to expand -t 4
#!/bin/sh
# Live migration of virtual machine
# arg0 vm localhost host2 [--no-autostart]
# arg0 vm host2 localhost [--no-autostart]
# $1 is VM name
# $2 is from
# $3 is to
# $4 if present must be --no-autostart
# after migration:
# we disable autostart on from,
# we enable autostart on to unless 4th arg is --no-autostart
# vi(1) :se tabstop=4
virsh_migrate_base_cmd_and_args='virsh migrate --live --p2p
--tunnelled --persistent --copy-storage-all --verbose --abort-on-error'
# if conditional must be true for good syntax
if \
case "$#" in
4)
case "$4" in
--no-autostart)
autostart=--no-autostart
autostart_disable=--disable
;;
*)
# 4th arg other than --no-autostart - disallowed
false
;;
esac
;;
3)
autostart=--autostart
autostart_disable=
;;
*)
# bad # of arguments
false
;;
esac &&
case "$3" in
localhost)
case "$2" in
localhost)
# to and from can't both be localhost
false
;;
*)
localhost=to
non_localhost="$2"
;;
esac
;;
*)
case "$2" in
localhost)
localhost=from
non_localhost="$3"
;;
*)
# to and from can't both be non-localhost
false
;;
esac
;;
esac &&
case "$non_localhost" in
192.168.55.2|192.168.55.25[23])
:
;;
vicki)
non_localhost=192.168.55.2
;;
host2)
non_localhost=192.168.55.252
;;
host1)
non_localhost=192.168.55.253
;;
*)
# non_localhost isn't a recognized one
false
;;
esac
then
Name_of_Virtual_Machine="$1"
case "$localhost" in
from)
time \
$virsh_migrate_base_cmd_and_args \
"$Name_of_Virtual_Machine" \
qemu+ssh://"$non_localhost"/system &&
virsh autostart --disable "$Name_of_Virtual_Machine" &&
ssh -ax -l root "$non_localhost" '
virsh autostart '"$autostart_disable "'\
'"$Name_of_Virtual_Machine"'
'
;;
to)
# try to determine our IP for the subnet we're using
myip=$(
ip -4 a s |
sed -ne '
s/^ inet \(192\.168\.55\.[0-9]\{1,3\}\)\/.*$/\1/p
t q
d
: q
q
'
) &&
case "$myip" in
?*)
:
;;
*)
echo "$0: failed to determine our IP address,
aborting" 1>&2
exit 1
;;
esac
ssh -ax -l root "$non_localhost" '
time \
'"$virsh_migrate_base_cmd_and_args"' \
'\'"$Name_of_Virtual_Machine"\'' \
qemu+ssh://'"$myip"'/system &&
virsh autostart --disable \
'\'"$Name_of_Virtual_Machine"\''
' &&
virsh autostart $autostart_disable "$Name_of_Virtual_Machine"
;;
*)
echo "$0: internal error, bad value for
localhost(=$localhost), aborting" 1>&2
exit 1
;;
esac
else
cat <<- __EOT__ 1>&2
$0: usage:
$0 Name_of_Virtual_Machine from_host to_host [--no-autostart]
Exactly one of from_host and to_host must be localhost,
and that which is not localhost must be a recognized configured host.
__EOT__
exit 1
fi
#
It's mostly got (almost?) just enough comments to keep me from
getting lost in my own code. The comments are mostly there to
explain what might not be highly obvious from the code itself,
e.g. intent/reason, general purpose/function, without (redundantly
and wastefully) stating what the code itself is doing and is or
should be pretty obvious from the code itself. It could probably do
with a modest bit more commenting, but what's there pretty well covers
the more/most important part. I should probably add comments about
it always (not an option) disabling autostart on the VMs from
location after it's successfully migrated from that VM's host.
And, yeah, that one ... "too" complex* and too infrequently used
(thus it would tend to not still be in my shell's history to
be easily reinvoked or edited and reinvoked) and also too useful
as to want to use it later again and again, as to be some quick
throw-away CLI thingy ... so ... it's a program saved in a
program file.
*Well, not really *that* complex, but significantly more complex
than something I'd [re-]whip up from scratch in 10 minutes or
less at the CLI.
references/excerpts:
http://linuxmafia.com/pipermail/conspire/2019-June/009846.html
https://lists.balug.org/pipermail/balug-admin/2019-June/000994.html
http://linuxmafia.com/pipermail/sf-lug/2019q2/014184.html
(etc.)
> From: "Michael Paoli" <Michael.Paoli at cal.berkeley.edu>
> Subject: program comments/examples, domain(s), DNS, ... Re: [sf-lug]
> sh, awk, Perl, ...
> Date: Tue, 04 Jun 2019 22:41:52 -0700
> *Well* ...
>
> readability 'n all that. Many of the shorter examples I often give are
> essentially CLI throw-away bits of code - too short and sufficiently easy
> enough (heck, at least for me, anyway), that I don't bother to save 'em
> in some script file ... heck, "next time around" I need the same or
> quite similar, they're "easy enough" I rewrite those small(ish) bits
> from scratch, and can typically do that, often much faster, than if I saved
> such as a script/program - notably as I'm typically doing half-dozen
> to hundreds(ish) of such relatively ad hoc use of shell in CLI per day,
> and where it gets sufficiently non-trivial that it's more than just
> some really basic stuff like redirection/piping and command
> substitution, but still far too simple(ish) for me to bother putting it
> in script/program file - and - egad, trying to find the dang thing later.
> So ... often it remains "throw-away" CLI code/example.
>
> Often when they *do* make it into a program file, they were sufficiently
> useful, and "oops" - no longer in my history to [re[edit] and] reuse,
> and, my gosh, maybe it had evolved up to something that, oh, might take
> me more than 10 minutes to recreate from scratch ... or I keep doing quite
> similar often enough, that, well, I should stick it in a program file -
> notably because it's "the same" thing very commonly done, and doesn't
> change, or the bits that do change are easy to deal with in program,
> e.g. via options, arguments, and/or inputs/outputs.
>
> So, let's see, on, e.g.:
> http://linuxmafia.com/pipermail/sf-lug/2019q2/014173.html
> we have ...
>
> $ lsb_release -d; man dash | col -b | expand | grep '[^ ]' | wc -l
> Description: Debian GNU/Linux 9.9 (stretch)
> 1200
> $ echo 1200/66 | bc -l
> 18.18181818181818181818
> $
> That's pretty straight forward.
> could comment the heck out'a it, but why, it's a throw-away.
>
> Flushed out with comments it'd look something like this:
> #!/bin/sh
> # use standard shell for our interpreter
>
> # What's the description of the operating system we're running?
> lsb_release -d
>
> # for the man page for dash, a minimally compliant POSIX shell,
> man dash |
> # if we filter out backspace sequences, e.g. change _^Hx to just x, etc.
> col -b |
> # and then expand the tabs to spaces
> expand |
> # and then filter it to just lines that contain some non-whitespace
> grep '[^ ]' |
> # how many lines does that gives us?
> wc -l
>
> #!/bin/sh
> # standard shell
> # what's 1200 (obtained from earlier) divided by 66 (number of printable
> # lines on yee olde std 11" long line printer paper with 6 LPI, so
> # 66 lines that can be printed, standard, per page
> # so we divide by that to get at least approximate number of pages
> echo 1200/66 | bc -l
>
> But this *is* a Linux User Group.
> It's got ...
> http://linuxmafia.com/pipermail/sf-lug/2019q2/014153.html
> ... about 279 folks on the list.
> I give what I think are "interesting" examples - e.g. of powerful
> stuff one can do from CLI. I don't feel like necessarily explaining
> every bloody teensy 'lil itty bitty step. There's lots 'o folks on
> the list, someone else can always break it down and and explain it, or
> geez, even take a crack at it. Likely if they don't get it quite right
> other(s) will provide additional hints/corrections, etc. Why folks might
> even learn that way. :-)
> Y'all got man pages, you can look 'em up.
> If you don't, STFW (Search The Fine Web).
>
> And, when it gets to a program file, I may reasonably well comment
> it ... does also depend too upon audience ... much of the time it's
> mostly for me much of the time (but too, different in different contexts)
> ... and ... so N months/years later I might have some clue as to why I did
> it as I did ... or the intended purpose or
> whatever ... though one can always read the code itself - but can
> be useful to have comments that summarize to higher level, or explain
> what's not obvious in the code itself.
>
> Let's see, a somewhat longer one in program file - did this one pretty
> recently, comes in dang handy ... it probably ought have more
> comments, but ... whatever. I know what it does for me. :-)
>
> It's a little more non-trivial.
> It's got some high-level comment(s) so I can recall what I was
> doing(/attempting) with it ... but alas, that comment is already
> somewhat out-of-date.
> I also trimmed the list of domains a bit in the, uh, "example" below.
>
> Anyway, what it *really* does, is some fair bit more than that ...
> it gets and displays the delegating authority NS records from nameserver
> upstream of the domain,
> then for each delegated NS, it queries all the IP addresses of each for
> the domains' SOA record. Oh, and it also tags those outputs
> with the NS and IP address, so I know from whence I got the SOA results.
> I mostly use it to check that the delegated NS are as I expect, that they're
> serving up DNS data for the domain, and that they're (at least reasonably)
> current on SOA serial number (if they have most current on all for domain,
> they're fully caught up and have the current data).
> Now, that's not a *full* check of all the DNS checks I might want to do,
> but it well covers those I tend to want to examine pretty regularly,
> and, well, I got tired of typing out the longer commands to do it
> manually at CLI, and particularly fairly regularly, so ... time to
> put it in a program ... and so I did.
>
> Anyway, y'all can always comment the heck out of it, if you wish.
> I know pretty dang well what it does from a fairly quick glance/read
> over the code. If I can't figure that out so easily in 6 months or
> more from now, I probably should'a written better comments. ;-)
> But then again, I might alter it to add/change the functionality
> anyway, so, spending a whole lot 'o time on comments might be
> rather premature.
>
> Yeah, not exactly like it's a highly mature unchanging program ...
> $ (cd ~/bin && ls -ond DNS_SOA_CK)
> -rwx------ 1 1003 1229 May 13 23:11 DNS_SOA_CK
> $
> Hasn't yet even made it a month yet without changes (or having
> been created). And in the meantime, no shortage of other stuff
> to do.
>
> $ expand -t 4 < ~/bin/DNS_SOA_CK
> #!/bin/sh
>
> LC_ALL=C export LC # avoid non-ASCII surprises
>
> # vi(1) :se tabstop=4
>
> # check SOA records for domain(s) - given as arguments or default set
> default_domains='balug.org sf-lug.com sf-lug.org sflug.org sflug.com
> sflug.net berkeleylug.com berkeleylug.org'
>
> [ "$#" -ge 1 ] || set -- $default_domains
>
> tmpf=$(mktemp) || exit
> trap "trap - 0; rm $tmpf" 0
> for sig in 1 2 3 15
> do
> trap '
> trap - 0
> rm '"$tmpf"' || exit
> trap - '"$sig"'
> kill -'"$sig"' "$$"
> ' "$sig"
> done
>
> for FQDN
> do
> FQDN=$(echo "$FQDN" | tr A-Z a-z)
> case "$FQDN" in
> *.)
> :
> ;;
> *)
> FQDN="$FQDN".
> ;;
> esac
> set -- $(
> echo "$FQDN" |
> sed -e 's/\(.*\)\.\([^\.][^\.]*\.$\)/\1 \2/'
> )
> [ "$#" -eq 2 ] || continue
> sd="$1"
> TLD="$2"
> echo FQDN="$FQDN" authority:
> aNS=$(dig +short "$TLD" NS | head -n 1)
> dig @"$aNS" +noall +authority +norecurse "$FQDN" NS |
> sort |
> tee "$tmpf"
> for NS in $(
> < "$tmpf" tr A-Z a-z |
> awk '{if(($1=="'"$FQDN"'")&&($2 ~
> /^[0-9][0-9]*$/)&&($3=="in")&&($4=="ns"))print $5;}' |
> sort -u
> )
> do
> for IP in $(
> dig +short "$NS" A "$NS" AAAA
> )
> do
> #echo "$FQDN $NS $IP"
> echo $(dig @"$IP" +noall +answer +norecurse +nottl
> "$FQDN" SOA) '@'"$IP"' ('"$NS"')'
> done
> done
> done
> $
>
> Yep, code not mature ... I just spotted and fixed a typo in comment.
>
> Oh, and even if the comments are mostly missing/lacking, the structure in
> the file is much more readable. Often long(ish) CLI stuff, not so
> readable, but dang handy to have it all on single line, as then one can
> most easily repeat it, or edit it and reuse altered version, and repeat
> as necessary/desired - often building up the desired results
> piece-wise with chunks of code typed at CLI ... and often "throw
> away" - need that
> info just that once, and probably never need to do quite the exact same
> thing ever again (or at least rather improbable). ... or sufficiently rare
> and not too horribly complex, that explicitly saving the code - not
> worth the bother, as "next time", it would be easier/faster to recreate
> than to try and find the earlier. (heck, if I saved all those ad hoc
> more than slightly trivial ones, I'd probably have dozens to hundreds
> of ad hoc "programs" written per week ... most of which I'd never want
> to use again exactly as they were - let alone try 'n find the specific
> one I might want again when I actually had some need to repeat something;
> again, faster to do it again from scratch in most cases for such
> shorter bits.
>
> Ah, yep, everybody likes to complain & critique (okay, sure, I'm
> guilty too) ;-)
>
> But yeah, as Rick (and others) occasionally point out ... yeah, often
> most/all others don't pick up and run with what they ought, or even
> bother to say "thanks" once in a while on such tasks or whatever.
> Often, "crickets", as Rick says.
>
> Maybe a bit early to complain ;-) but, e.g., hey y'all, sf-lug.ORG isn't
> approaching imminent expiration now. Yup, somebody took care of that ...
> me. Oooh, and folks ought figure out the plan for renewing (or not?)
> sf-lug.COM ... haven't heard a peep on that, but maybe y'all will get
> that figured out by end of the one remaining SF-LUG meeting before
> sf-lug.COM expires. Oh, and Joker.com ... I know Jim likes it, but
> rather sucky registrar - notably as Rick also pointed out, ... they
> won't even allow one to have the whois data properly public. So,
> NOT within 30 days of expiration!!! ... or even soon after a renewal,
> but y'all might want to think about transferring to a registrar that
> doesn't suck - or at least sucks less. I know I quite like gandi.net,
> I think Rick has his favorite(s) - there are at least a moderate number
> of rather to quite good ones. But, anyway, y'all didn't plan that
> out well in advance of being within 30 days of expiration, so, that's
> a lost opportunity on transfer timing ... could do it *later*, but
> Now is *NOT* the time to be transferring - at least for those two
> domains (impending expiration, and just recently renewed).
>
>> From: aaronco36 <aaronco36 at SDF.ORG>
>> Subject: Re: [sf-lug] sh, awk, Perl, ... Re: calculating meeting dates ...
>> Date: Tue, 4 Jun 2019 23:43:40 +0000 (UTC)
>
>> As somewhat related to "sh, awk, Perl" and to "calculating meeting
>> dates", Akkana Peck <akkana at shallowsky.com> wrote in [1]:
>>> How's this? (Add to your .bashrc or wherever.)
>>>
>>> dayofweek() {
>>> echo -n "Date as YYYY-MM-DD: "
>>> read d
>>> date --date=$d
>>> }
>>>
>>> If you really only want the day, change the last line to
>>>
>>> date --date=$d +%a
>>
>> IMHO, one major benefit this bash-script function has over other
>> coding constructs is its *incredibly* straightforward Readability.
>> Readability as in the ease of figuring out what the code is really
>> supposed to do, as well as the more extensive and respective
>> Software Engineering Stack Exchange and Psychology of Code
>> Readability treatments of the same in refs [2] and [3].
>>
>> Well, sure, *of course*(!!) the following and longer command
>> pipeline is easier to enter and may be more efficient to run than
>> most bash scripts...
>> '$ (cd /usr/bin && ls -iL $(apropos awk | awk '{if(($1 ~
>> /awk/)&&($1 !="igawk"))print $1;}') | sort -bn); read -rsp $'Press
>> any key to continue...\n' -n1 key; (for a in mawk awk; do echo "$a"
>> $(echo $(man "$a" | col -b | expand | grep '[^ ]' | wc -l)/66 | bc
>> -l); done)'
>>
>> But such a command pipeline is more efficient at the cost of any
>> semblance of good Readability (again, IMHO).
>>
>> It also greatly helps Readability to have appropriate comments
>> placed throughout a bash script, such as those which the author of
>> this bash script [4] does ;->
>>
>> -A
>>
>>
>> Any fool can write code that a computer can understand.
>> Good programmers write code that humans can understand.
>> Martin Fowler
>>
>> Programs should be written for people to read, and only
>> incidentally for machines to execute.
>> Abelson and Sussman
>>
>>
>> ============================================================
>> References
>> ============================================================
>> [1]http://linuxmafia.com/pipermail/sf-lug/2019q2/014174.html
>> [2]https://softwareengineering.stackexchange.com/questions/162923/what-defines-code-readability [3]https://medium.com/@egonelbre/psychology-of-code-readability-d23b1ff1258a
>> [4]http://www.rawbw.com/~mp/unix/sh/examples/shrink2fs
>> ============================================================
More information about the sf-lug
mailing list