[conspire] BekeleyLUG today and ... Re: letsencrypt.org - not that hard, even from "scratch", and for wildcards, ...
Michael Paoli
Michael.Paoli at cal.berkeley.edu
Sun Jul 12 08:19:23 PDT 2020
So,
BerkeleyLUG today:
https://groups.google.com/d/msg/berkeleylug/DFCG_Kh_em8/iuO0bLvgAwAJ
https://www.wiki.balug.org/wiki/doku.php?id=balug:covid-19
https://berkeleylug.com/
Have a bunch 'o certs that expire 2020-07-21, was at first thinkin' I'd
get go ahead and get updated certs 10 days in advance, but then I
remembered there's BerkeleyLUG (online virtual) today so ... why don't
I just do that live during BerkeleyLUG (can share/so screen) ... so,
I'm planning and intending to do that live during today's BerkeleyLUG
meeting today - get nice fresh cert(s) (at least starting with those
for BerkeleyLUG - and probably at least installing that too), and
likely get updated replacements for all the certs that'll be expiring
around that same date/time, and likely get most or all of 'em
installed.
And, yeah, one can browse lots of the relevant programs and helper
programs/scripts and such:
https://www.balug.org/~mycert/
> From: "Michael Paoli" <Michael.Paoli at cal.berkeley.edu>
> Subject: letsencrypt.org - not that hard, even from "scratch", and
> for wildcards, ...
> Date: Fri, 08 May 2020 06:30:03 -0700
>> From: "Ruben Safir" <ruben at mrbrklyn.com>
>> Subject: Re: [conspire] Lets Encrypt
>> Date: Thu, 7 May 2020 10:08:34 -0400
>
>> On Wed, May 06, 2020 at 10:01:09PM -0700, Michael Paoli wrote:
>>>> From: "Rick Moen" <rick at linuxmafia.com>
>>>> Subject: Re: [conspire] Lets Encrypt
>>>> Date: Wed, 6 May 2020 20:46:07 -0700
>>>
>>>> Quoting Michael Paoli (Michael.Paoli at cal.berkeley.edu):
>>>>
>>>>>> From: "Ruben Safir" <ruben at mrbrklyn.com>
>>>>>> Subject: [conspire] Lets Encrypt
>>>>>> Date: Wed, 6 May 2020 16:11:55 -0400
>>>
>>>>>> Has anyone used it successfully?
>>>>>
>>>>> Heck yeah! Been usin' it for years now!
>>>>> It's not *that* hard.
>>>>
>>>> I think Ruben might appreciate a look at your implementation scripts,
>>>> Michael.
>>>
>>> Yup, ... watch this space for updates. ;-)
>>
>> I am sick of banging my head on this. I wish I could do this
>> with a wildcard domain, which evidently needs to be done through
>> DNS. There instructions (IMO) are completely incoherent on this.
>
> Stop banging your head. Read(/skim) some documentation. It's not all
> that hard! Uhm, no, their (not there) instructions aren't that difficult
> to comprehend. Let's see ...:
>
> Ain't that hard, let's see - from (relative) "scratch" ...
> o a relatively minimal Debian stable installation
> o not generally covered: generating key, generating CSR, changing DNS,
> rationale on not covering those bits:
> o key and CSR you'd need to do anyway for any CA, not specific to
> letsencrypt.org
> o DNS maintenance needs be done anyway, so not a DNS tutorial or the
> like, will presume reader knows how to make and verify requisite
> DNS changes
>
> Okay, start with aforementioned Debian installation.
> Debian stable now likely more than suffices on sufficiently new
> client - we'll use certbot - widely supported, from the project itself,
> simple enough, and will well do the needed (there are lots of other
> clients also available - or heck, don't like 'em? Write your own - the
> API is quite well documented, so any capable programmer can write their
> own client if they so choose).
>
> Let's make sure our information on package versions available is current,
> and that we're current ... I'll show also config lines I have in
> /etc/apt/sources.list:
>
> # grep '^[ \t]*[^ \t#]' /etc/apt/sources.list
> deb http://deb.debian.org/debian/ buster main non-free contrib
> deb-src http://deb.debian.org/debian/ buster main non-free contrib
> deb http://deb.debian.org/debian-security buster/updates main
> contrib non-free
> deb-src http://deb.debian.org/debian-security buster/updates main
> contrib non-free
> deb http://deb.debian.org/debian/ buster-updates main contrib non-free
> deb-src http://deb.debian.org/debian/ buster-updates main contrib non-free
> # apt-get -y update
> # apt-get -y upgrade
> # apt-get --no-install-recommends -y install certbot python-certbot-doc man
>
> In the above, I give the --no-install-recommends option to avoid pulling
> in additional stuff I don't need, and prefer not to drag in here.
> That option can also be configured in the apt configuration if one wants
> that for default behavior.
> I do also explicitly pull in the documentation, to make things easier -
> including checking against the exact version installed.
> Oh, I did say it was relatively minimal ... also installed the man package,
> so we can now usefully do:
> $ man certbot
>
> And how did I determine the package for that documentation?:
> $ apt-cache search certbot-doc
>
> By default, certbot likes to do stuff as root and automatically.
> For illustration purposes here, we're not gonna do that. No
> privileged operations needed - a regular unprivileged user will
> quite suffice as we're going to do it. That also means we'll have to
> pass along a few more options/arguments and/or configure things a bit
> differently to do it that way - that's fine - mostly want to show that
> any user can get the certs - no particular privilege needed.
> Heck, could even install under user's HOME directory and not need root
> to even install the client software - but I'm not going to show
> that here. I also figure if you can manage DNS and install certs,
> you can probably install certbot.
>
> # let's disable certbot's cron-driven stuff:
> # find /etc/cron* -name '*certbot*' -print
> /etc/cron.d/certbot
> # grep -n '^[ \t]*[^ \t#]' /etc/cron.d/certbot
> 14:SHELL=/bin/sh
> 15:PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
> 17:0 */12 * * * root test -x /usr/bin/certbot -a \! -d
> /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot
> -q renew
> #
>
> Hmmm, ed would be nice ...
> # apt-get --no-install-recommends -y install ed
>
> and comment out that certbot cron line:
> # ed /etc/cron.d/certbot
> 775
> 17
> 0 */12 * * * root test -x /usr/bin/certbot -a \! -d
> /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot
> -q renew
> s/^/#/p
> # 0 */12 * * * root test -x /usr/bin/certbot -a \! -d
> /run/systemd/system && perl -e 'sleep int(rand(43200))' && certbot
> -q renew
> w
> 776
> q
> #
>
> For example/illustration here, I'll use domain example.com and email
> postmaster at example.com - substituting those in for the actual domain
> I ran these tests on.
>
> So, let's say we've generated our key and CSR.
> We'll put the CSR, in PEM format, in file:
> CSR.pem
> On the CSR, we request these for CN and SAN
> CN = *.example.com
> X509v3 Subject Alternative Name (SAN):
> DNS:*.example.com, DNS:example.com
> For regular user ID here, I've created an account named "test" - no special
> privileges.
> So, hereafter, the stuff shown with prompt of "$ " is done with the test ID.
>
> $ man certbot
>
> So, we read/skim it. Doing it as non-root, there are a fair number
> of non-default options we're going to want ... and consistently so.
> We'll need to register first (that sets up our API key, so we can do
> subsequent requests using that, without needing to register again).
> And when we request certs, likewise we'll want a quite same set of
> options each time - and with some of those options in common to both.
> So, rather than type that all out repeatedly, let's create a few
> fairly simple scripts to handle that - and also a file with the
> common options ... actually two such files.
> Let's do these files/scripts:
> .my_certbot_common - for common settings (sourced in from others), we
> also set some options so that it won't explicitly ask us some
> questions (you may want to review and possibly set these differently)
> my_certbot.register - to register (generally only need do that once)
> my_certbot.getcert - to get actual cert
> my_certbot.getcert.test - like above, but use staging/test cert - so it
> essentially does everything the same, except instead of the production
> server, it instead uses test/staging server, and the certificates
> issued are not valid (not CA trusted root signed - and clearly so) -
> test/staging is also not rate limited, and is suitable for testing
> when wants to test everything out but not get an actual cert. We'll
> also configure this one slightly different on the email (which I
> think it respects? Documentation says it does for registration at
> least) - hoping that will suppress reminders about expiring cert (in
> this case we don't care for or want the reminder ... the reminders
> are also clear enough that they're invalid test/staging certs, so one
> can also fully test the reminder email stuff - if so desired)
> .my_certbot.getcert_common - the two above scripts will have a whole lot
> in common - so lets shove most of that commonality here, for easier and
> more consistent maintainability
>
> $ vi .my_certbot_common my_certbot.register
> .my_certbot.getcert_common my_certbot.getcert my_certbot.getcert.test
> $ chmod u+x my_certbot.*
>
> And, let's have a look at what we have for and in our relevant files now:
> $ ls -ld .my_certbot_common my_certbot.register
> .my_certbot.getcert_common my_certbot.getcert my_certbot.getcert.test
> -rw------- 1 test test 374 May 8 11:37 .my_certbot.getcert_common
> -rw------- 1 test test 428 May 8 11:21 .my_certbot_common
> -rwx------ 1 test test 877 May 8 12:09 my_certbot.getcert
> -rwx------ 1 test test 895 May 8 12:24 my_certbot.getcert.test
> -rwx------ 1 test test 750 May 8 12:09 my_certbot.register
> $ more .my_certbot_common my_certbot.register
> .my_certbot.getcert_common my_certbot.getcert
> my_certbot.getcert.test | expand -t 4
> ::::::::::::::
> .my_certbot_common
> ::::::::::::::
> # file to be used to source in our common settings
>
> MY_CERTBOT_DEFAULT_EMAIL='postmaster at example.com'
> MY_CERTBOT_CONFIG_DIR="$HOME"/etc/letsencrypt
> MY_CERTBOT_LOGS_DIR="$HOME"/var/log/letsencrypt
> MY_CERTBOT_WORK_DIR="$HOME"/var/lib/letsencrypt
> MY_CERTBOT_COMMON_OPTIONS="--agree-tos --no-eff-email --config-dir
> $MY_CERTBOT_CONFIG_DIR --logs-dir $MY_CERTBOT_LOGS_DIR --work-dir
> $MY_CERTBOT_WORK_DIR --manual-public-ip-logging-ok"
> ::::::::::::::
> my_certbot.register
> ::::::::::::::
> # !/bin/sh
>
> # Register with letsencrypt.org - generally only need to do this once.
> # see also:
> # "$HOME"/.my_certbot_common - for where relevant local data is stored
> # certbot(1)
>
> # vi(1) :se tabstop=4
>
> set -e # exit on failures
>
> umask 077 # security / least privilege principle
>
> . "$HOME"/.my_certbot_common # get our common settings
>
> # ensure that we have our needed directories:
> for dir in \
> "$MY_CERTBOT_CONFIG_DIR" \
> "$MY_CERTBOT_LOGS_DIR" \
> "$MY_CERTBOT_WORK_DIR"
> do
> [ -d "$dir" ] &&
> continue # already have this directory, check next
> mkdir -p "$dir"
> [ -d "$dir" ] || {
> echo "$0: failed to create directory $dir, aborting" 1>&2
> exit 1
> }
> done
>
> certbot \
> --email "$MY_CERTBOT_DEFAULT_EMAIL" \
> $MY_CERTBOT_COMMON_OPTIONS \
> register
> ::::::::::::::
> .my_certbot.getcert_common
> ::::::::::::::
> # file to be used to source in our common settings when getting cert
>
> # get our common settings
> . "$HOME"/.my_certbot_common || exit
>
> MY_CERTBOT_GETCERT_COMMON_OPTIONS="$MY_CERTBOT_COMMON_OPTIONS
> --duplicate --force-renewal --manual --preferred-challenges dns
> --csr $HOME/CSR.pem --cert-path $HOME/cert.pem --fullchain-path
> $HOME/fullchain.pem --chain-path $HOME/chain.pem"
> ::::::::::::::
> my_certbot.getcert
> ::::::::::::::
> # !/bin/sh
>
> # Get letsencrypt.org CA signed cert
> # See file: "$HOME"/.my_certbot.getcert_common
> # for locations of: expected CSR file, and where output files (including
> # cert) will be written
> # see also: certbot(1)
>
> # vi(1) :se tabstop=4
>
> set -e # exit on failures
>
> umask 077 # security / least privilege principle
>
> . "$HOME"/.my_certbot_common # get our common settings
>
> # get our common settings when getting cert:
> . "$HOME"/.my_certbot.getcert_common
>
> # ensure that we have our needed directories:
> for dir in \
> "$MY_CERTBOT_CONFIG_DIR" \
> "$MY_CERTBOT_LOGS_DIR" \
> "$MY_CERTBOT_WORK_DIR"
> do
> [ -d "$dir" ] &&
> continue # already have this directory, check next
> mkdir -p "$dir"
> [ -d "$dir" ] || {
> echo "$0: failed to create directory $dir, aborting" 1>&2
> exit 1
> }
> done
>
> certbot \
> --email "$MY_CERTBOT_DEFAULT_EMAIL" \
> $MY_CERTBOT_GETCERT_COMMON_OPTIONS \
> certonly
> ::::::::::::::
> my_certbot.getcert.test
> ::::::::::::::
> # !/bin/sh
>
> # Get test (invalid) letsencrypt.org cert
> # See file: "$HOME"/.my_certbot.getcert_common
> # for locations of: expected CSR file, and where output files (including
> # cert) will be written
> # see also: certbot(1)
>
> # vi(1) :se tabstop=4
>
> set -e # exit on failures
>
> umask 077 # security / least privilege principle
>
> . "$HOME"/.my_certbot_common # get our common settings
>
> # get our common settings when getting cert:
> . "$HOME"/.my_certbot.getcert_common
>
> # ensure that we have our needed directories:
> for dir in \
> "$MY_CERTBOT_CONFIG_DIR" \
> "$MY_CERTBOT_LOGS_DIR" \
> "$MY_CERTBOT_WORK_DIR"
> do
> [ -d "$dir" ] &&
> continue # already have this directory, check next
> mkdir -p "$dir"
> [ -d "$dir" ] || {
> echo "$0: failed to create directory $dir, aborting" 1>&2
> exit 1
> }
> done
>
> certbot \
> --test-cert \
> --register-unsafely-without-email \
> $MY_CERTBOT_GETCERT_COMMON_OPTIONS \
> certonly
> $
>
> And, expectedly, we have relatively little difference between the
> my_certbot.getcert and my_certbot.getcert.test programs:
> $ diff <(<my_certbot.getcert expand -t 4)
> <(<my_certbot.getcert.test expand -t 4)
> 3c3
> < # Get letsencrypt.org CA signed cert
> ---
>> # Get test (invalid) letsencrypt.org cert
> 36c36,37
> < --email "$MY_CERTBOT_DEFAULT_EMAIL" \
> ---
>> --test-cert \
>> --register-unsafely-without-email \
> $
>
> So, let's see if we can smoothly do the registration, and get our cert!
> Registration:
> $ ./my_certbot.register
> Saving debug log to /home/test/var/log/letsencrypt/letsencrypt.log
>
> IMPORTANT NOTES:
> - Your account credentials have been saved in your Certbot
> configuration directory at /home/test/etc/letsencrypt. You should
> make a secure backup of this folder now. This configuration
> directory will also contain certificates and private keys obtained
> by Certbot so making regular backups of this folder is ideal.
> $
>
> Well, that was certainly easy.
> We have our CSR file in the appropriate place:
> $ ls -l CSR.pem
> -rw------- 1 test test 1001 May 8 10:04 CSR.pem
> $
>
> So, let's get a cert - we'll get a test cert - if that works fine,
> actual production cert would be trivially different to obtain (for
> production, just use my_certbot.getcert.test instead of
> my_certbot.getcert.test).
>
> $ ./my_certbot.getcert.test
> ...
> Please deploy a DNS TXT record under the name
> _acme-challenge.example.com with the following value:
>
> urC99wtxH-Xla7_pQxAefCbb2JG7NEIIeeXRez4qO44
>
> Before continuing, verify the record is deployed.
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> - - - - - -
> Press Enter to Continue
> ...
> Please deploy a DNS TXT record under the name
> _acme-challenge.example.com with the following value:
>
> dmS_i4cDPXo_JvwKfMQRn196i4sTDdu8aewPxe2RGxo
>
> Before continuing, verify the record is deployed.
> ...
> Press Enter to Continue
> ...
> Server issued certificate; certificate written to /home/test/cert.pem
> - Congratulations! Your certificate and chain have been saved at:
> /home/test/fullchain.pem
> ...
> $
>
> And done!
>
> And peeking a bit at the (test) cert:
> $ openssl x509 -in cert.pem -text -noout | sed -ne '/Issuer:/p;/Not
> After/p;/Subject: CN = /p;/Subject Alternative Name/{N;p;q}'
> Issuer: CN = Fake LE Intermediate X1
> Not After : Aug 6 11:30:02 2020 GMT
> Subject: CN = *.example.com
> X509v3 Subject Alternative Name:
> DNS:*.example.com, DNS:example.com
> $
>
> So, in conclusion ...
> wildcard and/or (multi-name/domain) CA signed certs from letsencrypt.org,
> ooh, and no superuser privileges required on client at all to obtain
> such certs. Not that hard. Sure, need to read a bit of documentation,
> maybe poke at it slightly to possibly work out a (minor) kink or two,
> but pretty straight-forward.
>
> And ... subsequent certs? Easy peasy. In our example shown, just
> drop the CSR in place in the file (and probably not a bad idea
> to remove the cert.pem chain.pem fullchain.pem files - presumably having
> safely saved them elsewhere first). Then just run the program again,
> make the requisite DNS validation entries ... boom, done.
> Definitely not rocket science.
> And if/when that's "too much work" - or one gets tired of the
> manual step parts of that ... well, can automate (much!) more of it.
> Generally, with privilege, certbot can install certs for you,
> automagicly checking and replacing those (if you trust it that much
> and want it to do that for you). Even without that, can use
> other options/programs, e.g. "hook" programs, to automate otherwise
> manual steps (such as DNS updates). The certbot program has quite
> a set of options for configuring various ways for it to automate
> DNS updates (and clean out those "temporary" entries after, when it's
> done validating and they're no longer needed). Validation can also
> be done other ways, e.g. (for non-wildcard) using http (or https),
> just place the requisite files on the webserver. It can even
> fire up a minimal temporary web server and serve up that data (which
> might be particularly useful and convenient if you're actually wanting
> to use the certs for something other than http/https - e.g.
> SMTP STARTTLS).
>
> http://linuxmafia.com/pipermail/conspire/2020-May/010723.html
More information about the conspire
mailing list