Date: Sun, 23 Jul 2000 20:25:16 -0700 (PDT) From: Deirdre Saoirse To: rick@linuxmafia.com Subject: Gnome Dependencies Great post from havoc. -- _Deirdre * http://www.sfknit.org * http://www.deirdre.net "Trust me, I'm a science fiction writer" -- Larry Niven @ Conolulu ---------- Forwarded message ---------- Date: Sun, 23 Jul 2000 22:16:54 -0500 From: kelly@poverty.bloomington.in.us To: aaron@munge.net, deirdre@deirdre.net ------- Forwarded Message To: gnome-hackers@gnome.org Subject: dependencies From: Havoc Pennington Date: 23 Jul 2000 15:18:36 -0400 Message-ID: User-Agent: Gnus/5.0803 (Gnus v5.8.3) Emacs/20.5 Hi, Since there aren't enough flames in my life today, I thought I'd post this mail. ;-) It's going to gnome-hackers to keep things relatively quiet and let everyone speak openly. For a while I've been saying things like "our packages shouldn't have gratuitous dependencies" - Elliot (who strongly disagrees) says that I have been saying this without giving reasons for it. Also, some have taken to calling this stance "anti-gnome", which I think is kind of inaccurate, because I'm not anti-gnome. So here is some clarification of what I mean and why I've been working on integrating certain features into GTK+. Hopefully this will mean future discussions about module dependencies result in rational debates instead of reactionary flameage. Please followup in the spirit of this post, which is supposed to be a careful analysis, not just statement of opinion. I do realize that some of my less-careful expositions of this viewpoint in the past could have inspired the flameage I've received, this post is an attempt to start over. What is the argument exactly === The point, in essence, is that when a significant number of users want to use or predictably will want to use thing A without thing B, and thing A is not fundamentally bound to thing B and can be easily separated from thing B, it is good to separate thing A in order to meet the needs of users (modulo exceptions created by unusual circumstances or temporary expediency). Example === The most classic example in GNOME-land is GnomeCanvas. We put it in libgnomeui to get people to use libgnomeui; I initially defended that. Go back a long time in the gtk-list and gtk-devel-list archives and you will find me arguing for having it in libgnomeui with Shawn and others disagreeing (notably, you'll see that most people were _not_ on my side). Fast forward to the present time; I read gtk-list and gtk-app-devel-list in their entirety, and it is a regular event that someone asks for the canvas separate from GNOME. I know of several people that have cut-and-pasted the whole thing into their app. And today Andy Tai actually packaged up the separated canvas for distribution. Andy has nicely done this as a sed job rather than a fork; contrast with the unfortunate CSCHTML which is some kind of separate project, Andy's tarball is just a mechanical transformation of the GNOME code that can track the latest GNOME version. Now, we can't actually change the canvas situation in the short term, because of backward compatibility constraints. But it's perfectly sensible IMO to support Andy's hacked canvas, and to long-term look at providing a GTK-only canvas solution. Non-arguments === Things I am not saying: - "delete libgnomeui" - a strawman. I have suggested that we break libgnomeui into "things that depend on the GNOME runtime (runtime = e.g. config files)" and "extra widgets" and "internal use only unreviewed APIs" but this is a totally separate argument, and I have always said it was just speculation anyway. - "why don't we just make one shared library per function?" - not deserving of a response, clearly there is a balance point between the extremes of one-big-ass-library and one-library-per-function. - "dependencies are never OK" - that is not the point at all - "code reuse is bad" - actually I'm suggesting more code reuse, see below So, some of the reasons we should look hard at making our code have fewer dependencies: Reason #1: We no longer have to "get people to use GNOME" === There is just no reason for this anymore. We are on track to win the "desktop wars". I think winning developers for GNOME away from plain GTK+ is missing the forest for the trees. What we should be doing is winning developers for free systems (such as Linux) away from proprietary systems (e.g. Windows). Allowing developers to take baby steps (learn GTK first, then learn about Bonobo) helps to do that. Let's forget about KDE; they are not our primary competitor anymore. We shouldn't grow GNOME by cannibalizing the plain-GTK+ userbase, though we can expect a continuous flow of people from there; we should grow GNOME by attracting the 3 million Windows developers (there are only a few hundred thousand UNIX developers, so you have an order of magnitude difference there). Reason #2: User demand === A number of people have told me things like "no one cares about the dependencies". I'll just present some concrete counterexamples: - If you dig through the gtk-list and gtk-app-devel-list archives, my guess is that about weekly someone asks for either an HTML or Canvas widget, with the stipulation that it only depends on GTK+. Latest example is this one: http://mail.gnome.org/pipermail/gtk-app-devel-list/2000-July/008107.html If I had more time on my hands, I could easily give you a couple dozen more examples. - Loki games only uses GTK and esound. Probably they wouldn't use esound if it required gnome-libs. - Applix only uses plain GTK - Andy's Canvas separation work - The GIMP - The CSCHTML fork of GtkHTML (though in this case, Larry did say he would take a patch to allow GTK-only use of GtkHTML, and I therefore think the fork is ill-advised, I also think that some people encouraged the fork by flaming the guy for not wanting to use GNOME) In any case; you can dismiss the evidence by saying all these people are on crack, but IMHO that is not a good way to attract users of other systems and toolkits. Users that avoid dependencies are typically looking at all the reasons I'm enumerating here. As an app author, I've been burned by dependencies before; I'm sure many others have the same experience. So their gut reaction is "carefully justify each dependency." Dismissing that out of hand is a mistake. Reason #3: Modularity === In general, fewer dependencies between A, B, and C means that changing A results in fewer recompilations/reinstalls of B and C, and that changing A is easier to do (since B and C won't need fixing). It also means that development of A, B, and C is more parallelizable. This is sound engineering practice; quoting Herb Sutter, not that it's a big revelation: "[object-oriented programming and generic programming] are both fundamentally tools to help manage dependencies and, therefore, manage complexity. It's telling that all of the common OO/generic buzzwords - including encapsulation, polymorphism, and type independence - and all the design patterns I know of describe ways to manage complexity within a software system by managing the code's interdependencies." Note that one of the primary goals of COM is to achieve full encapsulation, instead of the half-ass encapsulation provided by C++. It reduces the size of the ABI/API you care about dramatically and allows you to ignore the dependencies of the COM object, because they don't affect your app. In "Essential COM" by Box he describes this as pretty much the main reason for using components. Dependencies create an "accordion effect." In bike or car racing, this is what happens when you go around a curve - gaps open between the bikes or cars, because everyone had to brake for the curve and then reaccelerate. In software, if you break module A then B and C break and you have to go unbreak B and C, and if D depends on C then D can't be fixed until you fix C, the net effect being that the length of time it takes D to work again is a function of the number of dependencies in the chain between D and A. On the micro scale, an example of this is the pre-GNOME-1.0 situation where we were constantly breaking gnome-libs and GTK 1.1.x, and having to rebuild all the stuff depending on them. So on bad days, we'd spend all day recompiling and no time hacking. People complained about it a lot anyway. If you avoid that situation, you get a macro scale accordion effect: we release GTK+ 2.0, wait a month or two to port gnome-libs to it, wait a month or two to port the next layer of libs to it, wait a month or two to port apps to it, etc. Point being the GTK+ 2.0 deployment will take on the order of 6 months at least. Go down a layer, take Keith Packard's X render extension to accelerate antialiased vector graphics; that will take at least a year, probably more, until the first GNOME deployment using it. We also get severe accordion effect with language bindings, which tend to "lag" as a result. Guillaume pointed this out long ago, but didn't generalize the point to say that it's true of all libraries with a dependency, not just language bindings. The accordion effect is totally inevitable, but the higher your stack of dependencies, the worse it becomes. (So for example a language binding which includes GNOME is going to take much longer to release post-GTK+-2.0 than one which doesn't.) Reason #4: Code reuse === Anytime you add a new dependency to a module, you create a reason why someone may not be able to use your module. Extreme example: Someone's writing an office print daemon. It listens on a port, accepts print data, and dumps it to the printer. They want to use gnome-print; they find out it links to libgnomeui. Quite reasonably, they would expect sysadmins to flame them if their nongraphical daemon links to libgnomeui. So they don't use gnome-print. Another example: A number of people are trying to use GTK+ as a portable toolkit that runs on Windows or BeOS; they can use GTK+ and libxml here, but if you have an unportable dependency they can't use your module. But these are just examples; the point is, any module can be a barrier to entry for whatever reason, and we should have some reluctance to add barriers as developers start migrating from Windows to GNOME. If several users say "your dependency X is a barrier for me," we should listen to them seriously. Screwing users is not productive. Reason #5: Unified, sensible API === This is actually a specialization of the code reuse argument. If a module you depend on should use a facility in your module, you're out of luck. For example, all the image APIs in GTK+ (CList, etc.) should take a pixbuf. Federico was very upset at these CList APIs. But they couldn't possibly take a pixbuf, because gdk-pixbuf depended on GDK. Solution: move gdk-pixbuf below GTK+ in the dependency chain. Prerequisite: bust out the GnomeCanvas dependency. Another example: tons of stuff in GTK+ should have used the stock image API, and many things were in libgnomeui only because they used stock images. The most important thing that should have used stock images was themes; stock icons should be themeable. Not possible because they were in libgnomeui. Solution: move stock images down the dependency chain so GTK+ can use them. In general, if you don't like a lower-in-the-dependency-chain API, duplicating it is a pretty stupid long-term solution, though it may be a necessary short-term solution. However bad the first API is, if it's usable, two competing APIs is typically worse than the first API, because you end up with developer confusion and duplication of effort. Reason #6: Modularity is good for free software development === More dependencies mean more people have to talk to each other and agree on any given change. This just isn't good for the way free software development (or any software development) works; we can develop faster by decreasing the need for communication bandwidth. More importantly, the more communities we can have contributing to GNOME the better off we are. Examples: GIMP, AbiWord, and libxml each have large non-GNOME communities with contributors not taken from the pool of "core GNOME developers." If we can get these extra communities, then our development powers are much larger. I've seen many bug reports and fixes on the libxml list that weren't from GNOME people. Reason #7: Ease of deployment === Something many users mention about why they don't want to use gnome-libs is that their sysadmins complain about deploying it to non-Linux systems. For anyone, compiling Nautilus is a good bit harder than compiling something that depends only on gnome-libs, or only on GTK+. Proprietary developers typically ship a cut-and-paste of GTK+ with their app; the runtime dependencies of gnome-libs cause them problems, along with the size of gnome-libs. Counterargument #1: Strict maintenance === In the specific case of moving things to GTK+, a number of people have complained about the difficulty of getting patches in to GTK+. This is a legitimate issue that no one discusses because they don't want to hurt Owen and Tim's feelings. IMO this is the same situation with the Linux kernel core, GNU C library, etc.; many Linux kernel patches have been around for years without going in. I think it's a good thing if GTK+ is a relatively strictly-maintained library, and I think it's fine to have less-strict libraries like GtkExtra and libgnomeui. (Although sometimes the problem with getting patches included is that the maintainers have no time, rather than that the maintainers are strict, and you can hardly fault them for that.) However, note that this counterargument doesn't actually apply in the case where someone else is doing the work to merge things in to GTK+, and it only applies to merging things in to GTK+. Two cases where it doesn't apply, for example, are me putting stock icons in GTK+, and removing the GNOME dependency of GtkHTML. So let's be realistic about the scope of this counterargument; it is only relevant in very specific cases. I'll also point out that if we want to make gnome-libs a lasting and widely-used library, at some point it will need stricter maintenance, if only to be sure bin/source compat are retained. Counterargument #2: The dependency in question is actually needed === I have no problem with this argument. An example is that some widget (e.g. GnomeFileEntry) is crucially bound to a dependency (e.g. gnome_config_*). In those cases, you have a real dependency and not a gratuitous one, and that's fine. Users don't even complain about these cases. Counterargument #3: Slower development === The argument here is that it's much easier and faster to add a feature to the topmost layer of the dependency lasagna than to the bottom. The easiest place to add a feature is your app; next easiest during the push for GNOME 1.0 was libgnomeui; etc. Higher in the dependency layers means less recompilation, and coincidentally in our case less-strict maintenance and easier patch integration. I think this is true, but it's only short-term true. What it means is, when you're first developing a module or widget, you stick it in your app or in a higher-level more specific library or in something like GtkExtra; as it matures, you can migrate it downward to gain users and decrease module interconnections. For example, GtkDialog could learn from the mistakes I made with GnomeDialog, and now we can know that it's close to the right thing for GTK+. It was in some sense unfortunate to put GtkCList in GTK immediately; it may have been better to distribute it as a separate module for a while first, let everyone learn from the mistakes, and then migrate it downward. Anyway, the overall point being that once we've already developed something like GnomeCanvas, this argument ceases to apply. This argument also doesn't apply in the case where removing a dependency is just a matter of library split (such as GObject and GTK+, libgconf and libgconf-gtk, or the dialogs in gnome-print which could be broken out). Counterargument #4: Too much work === This is just "it's hard to separate GConf into two libraries" or "it's too much work to make GObject" etc. This can be legitimate, but: - it's definitely not a reason to reject patches, only an excuse for not doing the work yourself - it really isn't that much work in most cases; GnomeCanvas is trivial to break out of gnome-libs, and gconf-gtk is trivial to maintain. Counterargument #5: Lots of little libs are slow === The dynamic linker starts up slower if you have a million little libraries. This argument actually cuts both ways. Sometimes removing a dependency actually reduces the number of libraries: - a given app becomes able to avoid libraries it isn't using - moving things down in the dependency layers reduces the number of libs you have to link to. For example, GObject allows gconf-gtk to go away and GConf will now contain fewer libs and link to fewer libs. - using components instead of libraries nukes library dependencies. For example, with components you can use GtkHTML without linking to it. In general, though, code reuse encourages tons of shared libraries. Our long-term answer to this should be components, and maybe some sort of performance hacks (either a way to merge libraries, or speed up the linker, or wait for computers to get faster). My personal sense is that increased modularity outweighs this concern in many cases, though this concern does mean we should be looking for concrete motivations (such as user demand) before breaking a package into two packages. Again, a balance between the extremes of one-library-per-function and one-giant-big-ass-library is required. Anyway, the summary is that sometimes one of the counterarguments is compelling, and in those cases of course we should not sweat the dependency. But also in those cases we can give users a good rationale for the dependency. The situation I would like to see disappear is the one where we have a dependency and if users complain all we can do is flame them and say "just link to all our dependencies, you weenie" - because this alienates users, and users of libraries become developers of libraries, and long-term it undermines GNOME to lose developers. If we find ourselves flaming library users with no good rationale, we're forgetting why we are working on libraries. Havoc ------- End of Forwarded Message