[conspire] "Time zones exist to make programmers' lives miserable" ...
Michael Paoli
Michael.Paoli at cal.berkeley.edu
Sun Feb 7 00:00:06 PST 2021
Ought such quote be more like:
"Time zones exist to make programmers' lives miserable, but ..."
... slightly less miserable than without them? ... or something like
that.
Time and timezones has always been a relatively messy problem,
certainly has throughout history, etc.
Unix seeks/sought to simplify that, and POSIX to further standardize that,
and Linux, with LSB, etc., mostly follows along same, but also extends it
a wee bit - or at least also makes that possible.
An oft useful exercise, when moaning over the misery of timezones,
is to compare to the misery without timezones and timezone files ...
and of course short of getting all the humans on the planet to do
something much more simple and predictable, like UTC for the entire
planet. No, such is not the way - not only do relatively stable
countries muck about with their timezones, but some jurisdictions
even change such on very short notice - e.g. a new set of timezone
definition files is released because ... yes, next week, someone
changes timezones next week because those laws passed this week for
that to happen.
So, UNIX ... in the beginning, there was the epoch:
1970-01-01T00:00:00+00:00
Or more specifically the UNIX epoch. System time was in seconds from
the epoch. That stuff for humans, that would be converted back and forth
as needed. Early on, it was fixed libraries covering at least common
US timezones. As more timezones needed to be covered, and laws changed
timezone details, things got more messy and unwieldy. Somewhere around
roughly System V, or perhaps as early as III, things had evolved to
dynamic linked libraries - so at least those libraries could be updated
without recompiling all the programs using them. And
then there was TZ - initially simple, to specify which routine in
the libraries. Then, as governing bodies tended to change the rules, TZ
was made more flexible - so that TZ could in fact within it, describe
the rules themselves, e.g. that the change from standard to daylight time
happened on the nth day of week of the nth month, at nAM local time, and
the offset was n hour(s) and n minute(s), and what the zone abbreviation
changed to for daylight time. POSIX
actually picked that up. However, even that wasn't sufficient.
As having the rules in TZ, per POSIX - at least last I read of it - the
rules were fixed, not variable. E.g. it had no means to specify that
the rule was different for different years.
E.g. just look at PST8PDT US/Pacific from 1970 through 2021.
The start of DST happens on which Sunday of which month? Well, the question
the immediately is, for which year?
$ echo $(zdump -v US/Pacific | sed -ne '/ 1970 /,/ 2021
/{/PDT/!d;/Oct/d;/Nov/d;s/^[^=]*= *//;s/ *isdst.*$//;s/^Sun
//;s/^Mar/03/;s/^Jan/01/;s/^Feb/02/;s/^Apr/04/;s/ PDT$//;p}' | awk
'{printf("%s-%s-%02d\n",$4,$1,$2);}') | fold -s -w 72
1970-04-26 1971-04-25 1972-04-30 1973-04-29 1974-01-06 1975-02-23
1976-04-25 1977-04-24 1978-04-30 1979-04-29 1980-04-27 1981-04-26
1982-04-25 1983-04-24 1984-04-29 1985-04-28 1986-04-27 1987-04-05
1988-04-03 1989-04-02 1990-04-01 1991-04-07 1992-04-05 1993-04-04
1994-04-03 1995-04-02 1996-04-07 1997-04-06 1998-04-05 1999-04-04
2000-04-02 2001-04-01 2002-04-07 2003-04-06 2004-04-04 2005-04-03
2006-04-02 2007-03-11 2008-03-09 2009-03-08 2010-03-14 2011-03-13
2012-03-11 2013-03-10 2014-03-09 2015-03-08 2016-03-13 2017-03-12
2018-03-11 2019-03-10 2020-03-08
$
January or February anyone, just to mix it up? Sure, we got that too.
So, to help address that mess, we got ... timezone files. Timezone files
would have a more complex rule specification - it wasn't fixed for forever
some static rule, but described various rules at various times, so, they
were - or ought be - correct, for any given time (well, at least since
epoch, and into reasonable future) and for any given timezone - TZ would
more commonly now refer not to something in some library or give a
specific rule, but would be a reference to a file that would specify
the rules which should apply for that timezone for most any time of
interest. And so it goes ... mostly. Mostly UNIX/Linux/BSD uses
TZ and timezone files to figure out how to convert between system time
and human time. But wait there's more.
Leap seconds. How do you do rules to cover leap seconds, when those can't
be known well in advance - typically only months or so, at best. And how
do you accurately convert, to the second, dates/times that go further into
the future. Well, POSIX has a "simple" solution for that. POSIX mostly
pretends leap seconds don't exist. So, that makes those time conversions
much more predictable - so long as one doesn't care about leap seconds
... which sort'a kind'a mostly works, ... except when leap seconds
actually come along. How *nix and NTP, etc. deal with that is a whole
'nother topic - and there's more than one way. Regardless, that makes
things a bit messy around leap seconds - there is no perfect solution.
Unix, etc. filesystems have timestamps to the second on files. Some such
filesystem types have sub-second accuracy. So, how does one record the
time, on the timestamsps, for the files, that occur over a leap second
that's added? Per POSIX, leap seconds don't exist (mostly), so all such
in that interval would be the same number of seconds since the epoch.
What if your filesystem has sub-second accuracy? What was the order in
which things actually happened with those files? That might actually
matter. Anyway, whole 'nother topic.
But, back to timezones and timezone files, and leap seconds. There's the
POSIX way - essentially pretend they don't exist. Then there's the, uh
"right" way - the rightness/correctness of which could be greatly debated.
And, there's a "right" set of alternative timezone files, e.g. for
Linux. The "right" timezone files include leap seconds, whereas the
standard (POSIX) set does not. So, now for a given timezstamp on a
file, there are two possible ways to interpret it - the POSIX way will
give one time, and the "right" way will give a different time - differing
by leap seconds to that point, that have happened since the epoch. So
much for disambiguating when something happened or what a particular
timestamp means. E.g. most any backup format for *nix files, will
generally record times in seconds since the epoch - some might even
store sub-second accuracy. But if there's POSIX vs. "right" ambiguity,
how does one interpret all that data. I don't know of any backup format
that says, "This uses POSIX interpretation vs. 'right' interpretation.",
or vice versa or has such flag/indicator.
So, yes, timezones and timezone files, ... messy/"miserable". However,
consider without.
You write your program for your clock - it picks up GPS, and set the clock
to display the correct local time - it uses GPS to map location to a
timezone. You have to put all this logic into the firmware on your clocks.
You sell millions of these. Now some governing body changes the timezone,
or its standard/daylight/summer time rules. Now whatch'a gonna do, have
everybody update all their firmware on all those clocks? ... Yet again?
Are you going to update all that code and introduce no bugs or oversights?
So, yeah, sure, timezones and timezone files ... miserable.
And doing it all without such mechanism ... generally even more miserable.
But wait, there's more, you also get ...
E.g. Java. Java doesn't use the *nix timezone files. Nope, Java has it's
whole own separate independent set of timezone definition files. Whee!!!
So, e.g. Java's idea of time in a given timezone, may differ from the host
operating system's idea of same. Either may be out-of-date or inconsistent
relative to the other.
Anyway ... converting between timezones?
Well, a very first caution - timezone abbreviations aren't necessarily
unique. So, same abbreviation may mean quite different things, depending
upon which timezone one uses to interpret it. Fun, huh?
In any case, lots of languages and their libraries, etc., have various
capabilities of doing conversions - most notably from system time to
human time, and more-or-less mostly vice versa. The latter generally also
depending upon proper setting of TZ and available timezone file - if those
don't match, e.g. to timezone abbreviation, they might not get it right or
may not be able to do it at all.
GNU date also has all kinds of non-POSIX extensions. Many of which can be
handy for date calculations. It has %s to get seconds since epoch,
and it also has -d @s to convert from seconds since epoch.
Add or subtract integral multiple of 24 hours to a given time, and that'll
always give you date that many days into future or past, right? Nope.
Not all days have 24 hours. They're mostly 24 hours ... +- a second,
+- an integral multiple of of 15 minutes of up to about 2 hours ...
mostly, ... possibly also excepting if the timezone rules change even further
along the way. If you do it from noon, and round to the day, will probably
mostly work - but there might even be exceptions there. Did the
International Date Line move on you? "Oops". Stuff happens.
In general, there are pretty good libraries typically available for
most languages, to do such calculations (why reinvent the wheel ... poorly).
And ... how exactly did you want to count days? 'Round the world sea
cruise? Which direction? And when did you cross the
International Date Line? Or did your island get moved from one side of the
International Date Line to the other in the intervening days?
Wee bit before the epoch, but did you change calendars?
$ cal 9 1752
September 1752
Su Mo Tu We Th Fr Sa
1 2 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
$
So, tell me again, how many days hath September?
Or the day after September 2nd is? ... or how many days from
September 2nd to September 14th? How many birthday parties the
night of the 2nd?
So, a year, ... 365 days, or 366 with leap day, or ... how many days
in a year?
Did everyone who lived through that calendar change, statistically on
average live a bit longer than those right before and after? Or does
one just call that a calendar fluke?
Death rate was very low also that month from 3rd through 13th - as was
the birth rate ... and fewer birthdays on those dates in many following
years.
$ upcoming_meetings
usage: upcoming_meetings: [{+|-}n]
{1[st]|first,2[nd]|second,3[rd]|third,[45]th|fourth|fifth,last}
{Sundays,Mondays,Tuesdays,Wednesdays,Thursdays,Fridays,Saturdays - or
can be truncated as remains unique}
$ echo $(upcoming_meetings 2 Sa | sed -ne '/^2022/q;p') | fold -s -w 72
2021-02-13 2021-03-13 2021-04-10 2021-05-08 2021-06-12 2021-07-10
2021-08-14 2021-09-11 2021-10-09 2021-11-13 2021-12-11
$
Oh, and how many timezone names and variations currently?
$ find /usr/share/zoneinfo/*/ -follow -type f -print | wc -l
1779
$
And ... how many actually with unique timezone data?
$ find /usr/share/zoneinfo/*/ -follow -type f -exec sha512sum \{\} \;
| awk '{print $1;}' | sort -u | wc -l
774
$
What's the most common one - in terms of same data and distinct
names/pathnames?
$ find /usr/share/zoneinfo/*/ -follow -type f -exec sha512sum \{\} \;
| awk '{print $1;}' | sort | uniq -c | sort -bnr | head -n 2
30
01e5216822bc00070c7728249ed4443b070f901f6337de4ee72b7f4b6623b2638be69f72e5eb0838ad3c78e70618f1c839e681928316305f9b0ab9922c039f51
22
57a60cbbf067bc6d41c359a0ea23aaad3325652a7fefb33dbf015de41d851afc182c1472f651b4f562fe8b42c74e6aabb45f2f8d3fc8d496a9c6b2050cbb7ca5
$ echo $(find /usr/share/zoneinfo/*/ -follow -type f -exec sha512sum
\{\} \; | fgrep
01e5216822bc00070c7728249ed4443b070f901f6337de4ee72b7f4b6623b2638be69f72e5eb0838ad3c78e70618f1c839e681928316305f9b0ab9922c039f51 | sed -e 's/^[^ ]* *\/usr\/share\/zoneinfo\///' | sort) | fold -s -w
72
America/Anguilla America/Antigua America/Dominica America/Grenada
America/Guadeloupe America/Marigot America/Montserrat
America/Port_of_Spain America/St_Barthelemy America/St_Kitts
America/St_Lucia America/St_Thomas America/St_Vincent America/Tortola
America/Virgin posix/America/Anguilla posix/America/Antigua
posix/America/Dominica posix/America/Grenada posix/America/Guadeloupe
posix/America/Marigot posix/America/Montserrat
posix/America/Port_of_Spain posix/America/St_Barthelemy
posix/America/St_Kitts posix/America/St_Lucia posix/America/St_Thomas
posix/America/St_Vincent posix/America/Tortola posix/America/Virgin
$ $ ls -ld $(cd /usr/share/zoneinfo && find */ -follow -type f -exec
sha512sum \{\} \; | fgrep
01e5216822bc00070c7728249ed4443b070f901f6337de4ee72b7f4b6623b2638be69f72e5eb0838ad3c78e70618f1c839e681928316305f9b0ab9922c039f51 | sed -e 's/^[^ ]* *//' | sort) | cut -c-1 | sort | uniq -c | sort
-bnr
29 l
1 -
$
One file, 29 symbolic links. So, may be as easy as - location changes
timezone, change its symbolic link - so long as it matches to some other
existing timezone definition file.
> From: "Rick Moen" <rick at linuxmafia.com>
> Subject: [conspire] "Time zones exist to make programmers' lives miserable"
> Date: Sat, 6 Feb 2021 21:16:51 -0800
> Deirdre cited for me the Subject header as a saying by Jamis Buck,
> and it seemed amusing to quote, here.
>
> Anyway, what follows is what _some_ of us do for fun on a Saturday
> evening.
>
> Extra credit question: Why am I grepping out ".tab" and "posixrules"
> from the timezone names list? Class?
>
>
> ----- Forwarded message from Wayne <Wayne at TradeTimer.com> -----
>
> Date: Sat, 6 Feb 2021 17:33:03 -0800
> From: Wayne <Wayne at TradeTimer.com>
> To: Felton Linux Users Group LUG <felton-lug at googlegroups.com>
> Subject: [Felton LUG] More online meeting fun...
>
> In case you're interested, here is a partial list of time-practical
> online LUG meetings you might like to check out. I hope I got the
> time translations correct...
>
> https://meet.jit.si/SurreyLUGBaB - 2nd and 4th Saturday, 11:00 to
> 17:00 UTC, 3AM-9AM PT
> https://jitsimeet.jitsimeeting.co.uk/ManLUG - 3rd Saturday, 2-4PM UTC, 6AM PT
> https://meet.jit.si/ALE-NW - every Sunday at 3PM ET, Noon PST
> https://meet.jit.si/sf-lug.org - 1st Sunday, 11:00 AM PT
> https://meet.jit.si/BerkeleyLUG - 2nd and 4th Sundays, 11AM PT
> https://meet.jit.si/Pi.BerkeleyLUG - 3rd Sunday, 11AM PT
> https://meet.jit.si/TheCall - every-other Monday, 8:30AM PT
> https://meet.jit.si/livlug - 1st Wednesday 19:00 UTC, 11AM PST
>
> More at the BALUG wiki...
> https://www.wiki.balug.org/wiki/doku.php?id=balug:covid-19
>
> ----- End forwarded message -----
> ----- Forwarded message from Rick Moen <rick at linuxmafia.com> -----
>
> Date: Sat, 6 Feb 2021 18:56:19 -0800
> From: Rick Moen <rick at linuxmafia.com>
> To: Felton Linux Users Group LUG <felton-lug at googlegroups.com>
> Subject: Re: [Felton LUG] More online meeting fun...
> Organization: If you lived here, you'd be $HOME already.
>
> Quoting Wayne (Wayne at TradeTimer.com):
>
>> In case you're interested, here is a partial list of time-practical
>> online LUG meetings you might like to check out. I hope I got the time
>> translations correct...
>
> Time calculations involving multiple time zones are vexingly difficult.
> In fact, even _without_ multiple time zones, they're a pain. E.g.,
> this tiny shell script of mine is not entirely 100% correct because of
> an awkward definitional problem about when a day begins and ends for
> purposes of day count:
>
> $ cat $(which daysuntil)
> #!/bin/sh
> echo "Type the future date in ISO 8601 (YYYY-MM-DD) format."
> read futuredate
> echo $futuredate | tr -s '-' ' ' | awk '{dt=mktime($0
> "00-00-00")-systime(); print int(dt/86400+1) " days";}'
> $
>
> I haven't been moved to improve it lately, but maybe will come back to
> it. Anyway, point is, doing a similar trick involving inputting a time
> in zone X and programmatically converting it to zone Y seems like an
> even more daunting task. Anyone feeling cocky about that? Maybe a
> perlista? ;->
>
> What I _have_ done is the lazy 80% solution of saying "What is the time
> _right now_ in zone Y?" That's a lot simpler:
>
> :r! grep worldclock .bashrc
> alias worldclock='zdump
> America/{Los_Angeles,Phoenix,Mexico_City,New_York,Toronto,Sao_Paulo}
> Europe/{London,Berlin,Oslo,Moscow} Asia/{Calcutta,Shanghai,Tokyo}
> Australia/{Perth,Melbourne} Pacific/Auckland'
>
> I personally like that a lot. It's elegant.
>
> Sooner or later, I'll nail down the larger problem, maybe using this set
> of Python tips:
> https://stackoverflow.com/questions/10997577/python-timezone-conversion
> With all these Zoom/Jitsi meet things happening all over the world, I
> don't know about you, but I really need something like that -- and I
> feel a little sheepish relying on some stranger's hidden-away script via
> CGI, e.g, using things like
> https://www.timeanddate.com/worldclock/converter.html . That's not the
> Unix way, Grasshopper. ;->
>
> ----- End forwarded message -----
> ----- Forwarded message from Rick Moen <rick at linuxmafia.com> -----
>
> Date: Sat, 6 Feb 2021 19:47:02 -0800
> From: Rick Moen <rick at linuxmafia.com>
> To: Felton Linux Users Group LUG <felton-lug at googlegroups.com>
> Subject: Re: [Felton LUG] More online meeting fun...
> Organization: If you lived here, you'd be $HOME already.
>
> I wrote:
>
>> Sooner or later, I'll nail down the larger problem...
>
> Actually, I think this is a lot easier than I imagined. Just manipulate
> shell variable 'TZ' in scripting, and then ask date(1) to state what a
> time in a certain specified time converts to, in the TZ-value zone:
>
> $ TZ=MST date -d "19:43 EST"
> Sat Feb 6 17:43:00 MST 2021
> $
>
> ----- End forwarded message -----
> ----- Forwarded message from Rick Moen <rick at linuxmafia.com> -----
>
> Date: Sat, 6 Feb 2021 21:06:13 -0800
> From: Rick Moen <rick at linuxmafia.com>
> To: Felton Linux Users Group LUG <felton-lug at googlegroups.com>
> Subject: Re: [Felton LUG] More online meeting fun...
> Organization: If you lived here, you'd be $HOME already.
>
> I wrote:
>
>> Actually, I think this is a lot easier than I imagined. Just
>> manipulate shell variable 'TZ' in scripting, and then ask date(1) to
>> state what a time in a certain specified time converts to, in the
>> TZ-value zone:
>>
>> $ TZ=MST date -d "19:43 EST"
>> Sat Feb 6 17:43:00 MST 2021
>> $
>
> Also, I made a little thing to let me type 'tzall' or 'tzall | less' or
> 'tzall | grep [something]' as the lazy way to find a timezone descriptor
> for somewhere I care about.
>
> Voila, a new shell function defined in my .bashrc:
>
>
> tzall () {
> find /usr/share/zoneinfo/* -type f -print | sed -r
> 's_/usr/share/zoneinfo/__' | grep -v -e .tab -e posixrules
> }
>
>
> I first tried defining it as a shell _alias_ (out of habit), but the
> gyrations you'd have to go through to escape the quotation characters
> (for sed) seemed painful, so I did an end-run around that problem by
> writing it as a shell function, instead.
>
> There are a _whole_ lot of valid timezone names, many of them existing
> for weird purposes. There are actually 'only' 38 timezones in the
> world[1] ; it's just that Linux systems let you refer to them by a lot
> of distinct names. How many names?
>
> $ tzall | wc -l
> 1755
> $
>
> That many names.
>
>
> [1] https://en.wikipedia.org/wiki/List_of_UTC_time_offsets
>
> ----- End forwarded message -----
More information about the conspire
mailing list