Date: Sun, 23 Jul 2000 20:25:16 -0700 (PDT)
From: Deirdre Saoirse <deirdre@deirdre.net>
To: rick@linuxmafia.com
Subject: Gnome Dependencies

Great post from havoc.

-- 
_Deirdre   *   http://www.sfknit.org   *   http://www.deirdre.net
"Trust me, I'm a science fiction writer" -- Larry Niven @ Conolulu

---------- Forwarded message ----------
Date: Sun, 23 Jul 2000 22:16:54 -0500
From: kelly@poverty.bloomington.in.us
To: aaron@munge.net, deirdre@deirdre.net


------- Forwarded Message

To: gnome-hackers@gnome.org
Subject: dependencies
From: Havoc Pennington <hp@redhat.com>
Date: 23 Jul 2000 15:18:36 -0400
Message-ID: <y5wzon8y7sj.fsf@icon.labs.redhat.com>
User-Agent: Gnus/5.0803 (Gnus v5.8.3) Emacs/20.5


Hi,

Since there aren't enough flames in my life today, I thought I'd post
this mail. ;-) It's going to gnome-hackers to keep things relatively
quiet and let everyone speak openly.

For a while I've been saying things like "our packages shouldn't have
gratuitous dependencies" - Elliot (who strongly disagrees) says that I
have been saying this without giving reasons for it. Also, some have
taken to calling this stance "anti-gnome", which I think is kind of
inaccurate, because I'm not anti-gnome. So here is some clarification
of what I mean and why I've been working on integrating certain
features into GTK+.

Hopefully this will mean future discussions about module dependencies
result in rational debates instead of reactionary flameage.
Please followup in the spirit of this post, which is supposed to be a
careful analysis, not just statement of opinion.

I do realize that some of my less-careful expositions of this
viewpoint in the past could have inspired the flameage I've received,
this post is an attempt to start over.

What is the argument exactly
===

The point, in essence, is that when a significant number of users want
to use or predictably will want to use thing A without thing B, and
thing A is not fundamentally bound to thing B and can be easily
separated from thing B, it is good to separate thing A in order to
meet the needs of users (modulo exceptions created by unusual
circumstances or temporary expediency).

Example
===

The most classic example in GNOME-land is GnomeCanvas. We put it in
libgnomeui to get people to use libgnomeui; I initially defended
that. Go back a long time in the gtk-list and gtk-devel-list archives
and you will find me arguing for having it in libgnomeui with Shawn
and others disagreeing (notably, you'll see that most people were
_not_ on my side).

Fast forward to the present time; I read gtk-list and
gtk-app-devel-list in their entirety, and it is a regular event that
someone asks for the canvas separate from GNOME. I know of several
people that have cut-and-pasted the whole thing into their app.  And
today Andy Tai actually packaged up the separated canvas for
distribution. Andy has nicely done this as a sed job rather than a
fork; contrast with the unfortunate CSCHTML which is some kind of
separate project, Andy's tarball is just a mechanical transformation
of the GNOME code that can track the latest GNOME version.

Now, we can't actually change the canvas situation in the short term,
because of backward compatibility constraints. But it's perfectly
sensible IMO to support Andy's hacked canvas, and to long-term look at
providing a GTK-only canvas solution.

Non-arguments
===

Things I am not saying:

 - "delete libgnomeui" - a strawman. I have suggested that 
   we break libgnomeui into "things that depend on the 
   GNOME runtime (runtime = e.g. config files)" and "extra 
   widgets" and "internal use only unreviewed APIs" but this 
   is a totally separate argument, and I have always said 
   it was just speculation anyway.

 - "why don't we just make one shared library per function?" - 
   not deserving of a response, clearly there is a balance 
   point between the extremes of one-big-ass-library and 
   one-library-per-function.

 - "dependencies are never OK" - that is not the point at all

 - "code reuse is bad" - actually I'm suggesting more code reuse,
   see below

So, some of the reasons we should look hard at making our code have
fewer dependencies:

Reason #1: We no longer have to "get people to use GNOME"
===

There is just no reason for this anymore. We are on track to win the
"desktop wars".

I think winning developers for GNOME away from plain GTK+ is missing
the forest for the trees. What we should be doing is winning
developers for free systems (such as Linux) away from proprietary
systems (e.g. Windows).

Allowing developers to take baby steps (learn GTK first, then learn
about Bonobo) helps to do that. 

Let's forget about KDE; they are not our primary competitor anymore.
We shouldn't grow GNOME by cannibalizing the plain-GTK+ userbase,
though we can expect a continuous flow of people from there; we should
grow GNOME by attracting the 3 million Windows developers (there are
only a few hundred thousand UNIX developers, so you have an order of
magnitude difference there).

Reason #2: User demand
===

A number of people have told me things like "no one cares about the
dependencies".

I'll just present some concrete counterexamples:

 - If you dig through the gtk-list and gtk-app-devel-list archives, 
   my guess is that about weekly someone asks for either an HTML 
   or Canvas widget, with the stipulation that it only depends 
   on GTK+. Latest example is this one:
    http://mail.gnome.org/pipermail/gtk-app-devel-list/2000-July/008107.html

   If I had more time on my hands, I could easily give you a couple 
   dozen more examples.

 - Loki games only uses GTK and esound. Probably they wouldn't use 
   esound if it required gnome-libs.

 - Applix only uses plain GTK

 - Andy's Canvas separation work

 - The GIMP
 
 - The CSCHTML fork of GtkHTML (though in this case, Larry did say 
   he would take a patch to allow GTK-only use of GtkHTML, and 
   I therefore think the fork is ill-advised, I also think that
   some people encouraged the fork by flaming the guy for not 
   wanting to use GNOME)

In any case; you can dismiss the evidence by saying all these people
are on crack, but IMHO that is not a good way to attract users of
other systems and toolkits.

Users that avoid dependencies are typically looking at all the reasons
I'm enumerating here. As an app author, I've been burned by
dependencies before; I'm sure many others have the same experience.
So their gut reaction is "carefully justify each dependency."
Dismissing that out of hand is a mistake.

Reason #3: Modularity
===

In general, fewer dependencies between A, B, and C means that changing
A results in fewer recompilations/reinstalls of B and C, and that
changing A is easier to do (since B and C won't need fixing).
It also means that development of A, B, and C is more parallelizable.
This is sound engineering practice; quoting Herb Sutter, not that 
it's a big revelation:

 "[object-oriented programming and generic programming] are both 
  fundamentally tools to help manage dependencies and, therefore,
  manage complexity. It's telling that all of the common OO/generic 
  buzzwords - including encapsulation, polymorphism, and type
  independence - and all the design patterns I know of describe ways
  to manage complexity within a software system by managing the 
  code's interdependencies."

Note that one of the primary goals of COM is to achieve full
encapsulation, instead of the half-ass encapsulation provided by
C++. It reduces the size of the ABI/API you care about dramatically
and allows you to ignore the dependencies of the COM object, because
they don't affect your app. In "Essential COM" by Box he describes
this as pretty much the main reason for using components.

Dependencies create an "accordion effect." In bike or car racing, this
is what happens when you go around a curve - gaps open between the
bikes or cars, because everyone had to brake for the curve and then
reaccelerate. 

In software, if you break module A then B and C break and you have to
go unbreak B and C, and if D depends on C then D can't be fixed until
you fix C, the net effect being that the length of time it takes D to
work again is a function of the number of dependencies in the chain
between D and A.

On the micro scale, an example of this is the pre-GNOME-1.0 situation
where we were constantly breaking gnome-libs and GTK 1.1.x, and having
to rebuild all the stuff depending on them. So on bad days, we'd spend
all day recompiling and no time hacking. People complained about it a
lot anyway.

If you avoid that situation, you get a macro scale accordion effect:
we release GTK+ 2.0, wait a month or two to port gnome-libs to it,
wait a month or two to port the next layer of libs to it, wait a month
or two to port apps to it, etc. Point being the GTK+ 2.0 deployment
will take on the order of 6 months at least. Go down a layer, take
Keith Packard's X render extension to accelerate antialiased vector
graphics; that will take at least a year, probably more, until the
first GNOME deployment using it.

We also get severe accordion effect with language bindings, which tend
to "lag" as a result. Guillaume pointed this out long ago, but didn't
generalize the point to say that it's true of all libraries with a
dependency, not just language bindings. The accordion effect is
totally inevitable, but the higher your stack of dependencies, the
worse it becomes. (So for example a language binding which includes
GNOME is going to take much longer to release post-GTK+-2.0 than one
which doesn't.)

Reason #4: Code reuse
===

Anytime you add a new dependency to a module, you create a reason why
someone may not be able to use your module.

Extreme example: Someone's writing an office print daemon. It listens
on a port, accepts print data, and dumps it to the printer. They want
to use gnome-print; they find out it links to libgnomeui. Quite
reasonably, they would expect sysadmins to flame them if their
nongraphical daemon links to libgnomeui. So they don't use
gnome-print.

Another example: A number of people are trying to use GTK+ as a
portable toolkit that runs on Windows or BeOS; they can use GTK+ and
libxml here, but if you have an unportable dependency they can't use
your module.

But these are just examples; the point is, any module can be a barrier
to entry for whatever reason, and we should have some reluctance to
add barriers as developers start migrating from Windows to GNOME.

If several users say "your dependency X is a barrier for me," we
should listen to them seriously. Screwing users is not productive.


Reason #5: Unified, sensible API
===

This is actually a specialization of the code reuse argument.

If a module you depend on should use a facility in your module, you're
out of luck. For example, all the image APIs in GTK+ (CList, etc.)
should take a pixbuf. Federico was very upset at these CList APIs. But
they couldn't possibly take a pixbuf, because gdk-pixbuf depended on
GDK. Solution: move gdk-pixbuf below GTK+ in the dependency
chain. Prerequisite: bust out the GnomeCanvas dependency.

Another example: tons of stuff in GTK+ should have used the stock
image API, and many things were in libgnomeui only because they used
stock images. The most important thing that should have used stock
images was themes; stock icons should be themeable. Not possible
because they were in libgnomeui. Solution: move stock images down the
dependency chain so GTK+ can use them.

In general, if you don't like a lower-in-the-dependency-chain API,
duplicating it is a pretty stupid long-term solution, though it may be
a necessary short-term solution. However bad the first API is, if it's
usable, two competing APIs is typically worse than the first API,
because you end up with developer confusion and duplication of effort.


Reason #6: Modularity is good for free software development
===

More dependencies mean more people have to talk to each other and
agree on any given change. This just isn't good for the way free
software development (or any software development) works; we can
develop faster by decreasing the need for communication bandwidth.

More importantly, the more communities we can have contributing to
GNOME the better off we are. Examples: GIMP, AbiWord, and libxml each
have large non-GNOME communities with contributors not taken from the
pool of "core GNOME developers." If we can get these extra
communities, then our development powers are much larger. I've seen
many bug reports and fixes on the libxml list that weren't from GNOME
people.

Reason #7: Ease of deployment
===

Something many users mention about why they don't want to use
gnome-libs is that their sysadmins complain about deploying it to
non-Linux systems.

For anyone, compiling Nautilus is a good bit harder than compiling 
something that depends only on gnome-libs, or only on GTK+.

Proprietary developers typically ship a cut-and-paste of GTK+ with
their app; the runtime dependencies of gnome-libs cause them problems,
along with the size of gnome-libs.

Counterargument #1: Strict maintenance
===

In the specific case of moving things to GTK+, a number of people have
complained about the difficulty of getting patches in to GTK+. This is
a legitimate issue that no one discusses because they don't want to 
hurt Owen and Tim's feelings.

IMO this is the same situation with the Linux kernel core, GNU C
library, etc.; many Linux kernel patches have been around for years
without going in. I think it's a good thing if GTK+ is a relatively
strictly-maintained library, and I think it's fine to have less-strict
libraries like GtkExtra and libgnomeui. (Although sometimes the
problem with getting patches included is that the maintainers have no
time, rather than that the maintainers are strict, and you can hardly
fault them for that.)

However, note that this counterargument doesn't actually apply in the
case where someone else is doing the work to merge things in to GTK+,
and it only applies to merging things in to GTK+. Two cases where it
doesn't apply, for example, are me putting stock icons in GTK+, and 
removing the GNOME dependency of GtkHTML. So let's be realistic 
about the scope of this counterargument; it is only relevant in very
specific cases.

I'll also point out that if we want to make gnome-libs a lasting and
widely-used library, at some point it will need stricter maintenance,
if only to be sure bin/source compat are retained.


Counterargument #2: The dependency in question is actually needed
===

I have no problem with this argument. An example is that some widget
(e.g. GnomeFileEntry) is crucially bound to a dependency
(e.g. gnome_config_*).  In those cases, you have a real dependency and
not a gratuitous one, and that's fine. Users don't even complain about
these cases.

Counterargument #3: Slower development
===

The argument here is that it's much easier and faster to add a feature
to the topmost layer of the dependency lasagna than to the bottom. The
easiest place to add a feature is your app; next easiest during the
push for GNOME 1.0 was libgnomeui; etc. Higher in the dependency
layers means less recompilation, and coincidentally in our case
less-strict maintenance and easier patch integration.

I think this is true, but it's only short-term true. What it means is,
when you're first developing a module or widget, you stick it in your
app or in a higher-level more specific library or in something like
GtkExtra; as it matures, you can migrate it downward to gain users and
decrease module interconnections. For example, GtkDialog could learn
from the mistakes I made with GnomeDialog, and now we can know that
it's close to the right thing for GTK+.

It was in some sense unfortunate to put GtkCList in GTK immediately;
it may have been better to distribute it as a separate module for a
while first, let everyone learn from the mistakes, and then migrate it
downward.

Anyway, the overall point being that once we've already developed
something like GnomeCanvas, this argument ceases to apply. This
argument also doesn't apply in the case where removing a dependency is
just a matter of library split (such as GObject and GTK+, libgconf and
libgconf-gtk, or the dialogs in gnome-print which could be broken
out).

Counterargument #4: Too much work
===

This is just "it's hard to separate GConf into two libraries" or "it's
too much work to make GObject" etc. 

This can be legitimate, but:
 - it's definitely not a reason to reject patches, only 
   an excuse for not doing the work yourself
 - it really isn't that much work in most cases; GnomeCanvas is 
   trivial to break out of gnome-libs, and gconf-gtk is trivial 
   to maintain.

Counterargument #5: Lots of little libs are slow
===

The dynamic linker starts up slower if you have a million little
libraries.

This argument actually cuts both ways. Sometimes removing a dependency
actually reduces the number of libraries:

 - a given app becomes able to avoid libraries it isn't using
 - moving things down in the dependency layers reduces the number
   of libs you have to link to. For example, GObject allows 
   gconf-gtk to go away and GConf will now contain fewer libs
   and link to fewer libs.
 - using components instead of libraries nukes library
   dependencies. For example, with components you can use 
   GtkHTML without linking to it.

In general, though, code reuse encourages tons of shared libraries. 
Our long-term answer to this should be components, and maybe some sort
of performance hacks (either a way to merge libraries, or speed up the
linker, or wait for computers to get faster). 

My personal sense is that increased modularity outweighs this concern
in many cases, though this concern does mean we should be looking for
concrete motivations (such as user demand) before breaking a package
into two packages. Again, a balance between the extremes of
one-library-per-function and one-giant-big-ass-library is required.




Anyway, the summary is that sometimes one of the counterarguments is
compelling, and in those cases of course we should not sweat the
dependency. But also in those cases we can give users a good rationale
for the dependency.

The situation I would like to see disappear is the one where we have a
dependency and if users complain all we can do is flame them and say
"just link to all our dependencies, you weenie" - because this
alienates users, and users of libraries become developers of
libraries, and long-term it undermines GNOME to lose developers.
If we find ourselves flaming library users with no good rationale,
we're forgetting why we are working on libraries.

Havoc

------- End of Forwarded Message