[conspire] July schedule posted

[Tony Godshall]

On Tue, Jul 7, 2009 at 4:15 PM, Rick Moen<rick at linuxmafia.com> wrote:
> Quoting Tony Godshall (tony at of.net):
>
>> Oh, OK.  So then parsing HTML is not needed.  Very good.
>
> Again, I'm not 100% certain I understand what you mean when you make
> that remark, so I'll review.
>
> 1.  The CABAL page is (still) using a flat HTML table for its calendar
> events.  Its HTML source is the canonical upstream data.  Absent some
> rewrite of the page, the only way you're going to generate iCalendar
> from it is to parse HTML.
>
> 2.  The BALE page _used_ to have the same sort of HTML table.  In fact,
> I seem to recall creating the CABAL page's calendar data originally by
> copying and pasting HTML snippets out of the original incarnation of
> BALE.
>
> Some years ago, to make BALE easier to maintain and (arguably) more
> consistently accurate, Deirdre coded an ingenious PHP/MySQL replacement
> for the primordial BALE setup, thus using index.php rather than
> index.html, and running a monthly Python cronjob to create "events"
> table entries for an additional upcoming month as specified in separate
> table "eventtemplate".
>
> It is not necessary to "parse HTML" to convert the HTML output data
> created by BALE's index.php into iCalendar format.  The less-roundabout
> alternative is to query it out of MySQL, much the way index.php does.
>
> I could swear I stressed that BALE is PHP/MySQL-based right when you
> started talking about this matter, specifically to avert any such
> confusion -- though I could be misremembering.

Or perhaps I was overwhelmed by detail and confused between the two
calendars.  Just because you were clear on something doesn't mean it
made it properly into my mind.  I suppose I could go back and re-read
the messages in light of my new understandings of your methodology.


>> > BALE _used_ to be flat HTML.  Then, we templatised and automated it.
>> > These days, I have to use my choice of MySQL tool, if I have to edit
>> > either an event template, or edit an event already generated from a
>> > template or entered by me manually.
>> >
>> > Having to chase down events already generated by template is
>> > particularly tedious.  For example, Smaug in Santa Cruz, which meets
>> > every Monday night, has just decided to change locations, which means I
>> > need to chase down and edit three months of events rows for them, in
>> > addition to editing the template to fix future months not yet generated.
>> >
>
>> Going vcalendar-based would help you there since repetitions can
>> have expiration dates.  You just replicate the event, change one's
>> expiration, the other's start, and adjust the new one's details.
>
> Again, I'm well aware of that feature of the vCalendar / iCalendar
> specs.

OK.  I'm sorry I overcommunicated there.  Sometimes I do that.
It helps me think.  I don't mean to demean.  You probably understand
them much better than I do.  Since this entire conversation is CC'd
from the beginning to conspire@ it makes sense to err on the side
of clarity even if it may seem pedantic to you- perhaps others can
assist or learn.  I'd probably assume more of a word to the wise
kind of attitude if it were private and we hadn't alread a bit of
miscomm.

> One of the advantages of Deirdre's design is that its treatment of
> recurring events is, in general, smarter than those of the RFC 2445 and
> related specs.  There's a clever bit she has to indicate an event being
> on "5th or last" days, which was one of the bizarre variations that one
> of the groups threw at us (Bay Area NeXT Group, now deceased).

Oh, good to know.  I don't know the details and assumed from your
frustration at tracking down generated repetitions that it wasnt.

Perhaps you and she should propose such a feature to the next
rev of the spec.  Or write an RFC.

> She didn't implement beginning and end dates for templated events -- for
> the simple reason that it wasn't part of the static design that she was
> replacing -- but the great thing about the database design is that such
> things can be added easily, just by adding two more columns to the
> eventtemplate table, and then putting appropriate "if" statements in the
> Python script.

Oh very cool

> And I actually could, on reflection, avoid having to hunt down and
> _edit_ already-generated events, by instead editing "eventtemplate",
> then hunting down and _deleting_ the erroneous event rows, then
> re-running newmonth.py:  My recollection is that newmonth.py is designed
> to fill in missing events for the months currently shown, but not
> overwrite them if they exist in the "events" table.
>
> I hadn't really worked out that logic, until this moment.  Also, I've
> been favouring editing generated events rather than deleting and
> regenerating them in order to not have the events missing while I'm in
> the middle of that work.

I often find when I explain something to someone else that it debugs
my thought process and the issue I was explaining becomes solved.

> [Holidays:]
>
>> I think I saw a tool in Linux that has those.  Shouldn't have to do them
>> manually.
>
> I sort of like "doing" them, to a degree.  It allows me to express what
> I think is worth expressing, in the way I consider worthwhile.
>
> The selection of which holidays are worth including, and how to describe
> them, is a matter of my personal expression.
>
> I'm a writer.  Writing and publishing is what writers do.  Because I'm a
> writer, and as a reflection of who I am, BALE includes a "Summer Solstice
> / Midsummer Day / Midsommardagen" entry in June, because it pleases me
> to include the Norwegian / Old Norse spelling.

Well, now I'm going to actually have to pay attention to the events
I'd skimmed past in the past.

> It includes "Luciadagen (St. Lucia's Day), Sweden" because I have a
> Swedish mother-in-law.  It includes "Mohandas K. Gandhi's birthday
> (1869)" and "Juneteenth" because... well, just because I want them there.
>
> It includes "Spring Equinox / Ostara / Eostar / Eostre" because I wanted
> to remind people of the Saxon, Old High German, and Old English
> antecedents of Easter (without hitting them over the head with it).

I understand now.

> And BALE omits a whole lot of other allegedly noteworthy occasions
> because it reflects my taste rather than that of, say, Hallmark (or "a
> tool in Linux that has those").

Yes.

> In short, it is not a bug that I am an originator of information that is
> written in my personal style and reflecting my personal tastes and
> preferences.  To the contrary:  It's a design goal -- a consideration
> that applies to the selection, wording, and presentation of all the
> entries:  recurring events, holidays, one-time technical events,
> everything.  The whole megillah.  It's what I care to list, omitting
> what I wish to omit, written and presented the way I like.
>
> People sometimes act astonished and aggrieved when they hear that BALE
> omits something they care about, or includes something they don't.
> They tell me it's a "community resource", which sort of misses the
> point:  It's _my_ resource, my work (with Deirdre's code), written by
> me, on my server, running in my house on bandwidth and electricy I write
> chequees for.  I'm happy if they use and like it, but ultimately it's an
> expression of me, by me, for my purposes.  Including the holidays.

Absolutely.

Maybe if it had a clever name...

>> Nah, the way I'd do it is switch the "master" to something easier than
>> HTML and switch the HTML to generated.  But it turns out you already
>> have a better master than the HTML, so it makes sense to generate
>> both the HTML and the iCalendar from the SQL, I guess.
>
> Yes, that's what I was thinking.  However, the way your Python script
> parses HTML would be neat for, say, the CABAL calendar -- if we can fix
> the year-end and all-day-event problems.

So you don't want to switch the CABAL calendar to something else?
Are there events in CABAL that are not in BALE?  I guess I was assuming
it was a subset and you would get the CABAL events by updating the
BALE list and filtering them out.  I even wrote you that assumption and
you didnt contradict it.

>> Yes, I understand now.  You are testing using iCalendar to give people a
>> dynamically- generated html of the icalendar stuff.
>
> Well, sort of.  (Where you say "using iCalendar", above, I gather that
> you meant to say "using PHP iCalendar".)

Yes, of course. Sorry about that.  To the peanut gallers: There are
actually three different iCalendars under discussion: iCalendar, the
standard.  PHP iCalendar, the specific PHP program.  And the
Python library (in a fork of this thread)

> PHP iCalendar (0.9.4) was already there -- has been there for many
> years.  All I did was plunk the bale.ics file generated by your script
> into its data directory in order to see how PHP iCalendar presented it.
> At first, the answer was "not at all", because your script omitted the
> mandatory vCalendar header and footer -- and then "rather badly" because
> your script wrote DESCRIPTION where SUMMARY is apparently what was
> needed.  And then, it turned out that the data needed some rearrangement
> -- and all of those things were appropriate only after I loaded the file
> in PHP iCalendar.  (Please note that I'm not complaining in any way!
> I'm just recapping.)
>
> Basically, it was bog-standard QA:  Given that the script allegedly
> generated an iCal file, the necessary next step was to verify that at
> least one iCal-parsing tool could read it and do useful things with it.
> As I always say in a sysadmin context, "The task's not done until the
> results are tested and verified."   And PHP iCalendar was the tool I had
> handy for that purpose.
>
>
>> I'd do it statically with make file.
>
> Well, of course.  I plunked a copy into PHP iCalendar because it was
> there.  Didn't I _say_ that PHP iCalendar 0.9.4's been sitting around
> since forever?  [/me checks.]  Yes, I did.

Oh, and your complaining about how long it took to come up and that
you'd stick with the old version was supposed to tell me you *weren't*
considering it for production use?

> You know, we'll shorten the cycle time on these exchanges if you don't
> make odd assumptions.

Heh.  Likewise.

>> Like you found, iCalendar processing is rather CPU-intensive.
>
> PHP iCalendar 0.9.4 is perfectly tolerable, FWIW.  2.3.1 appears to be
> ghastly.

Well, yes, perhaps it has to parse the more the complicated
options of a higher rev of the standard.  Or maybe it's prettier.
But what I don't see is why you care so much if it takes a
second or five if it's only done statically after an edit.
Premature optimization IMHO.

Tony