sprezzatech blog #0018
SprezzOS Weeklyish Oblations, Objections, and News (SWOON) Vol. 1 Issue 6.
Thu Apr 25 12:40:54 EDT 2013
SprezzOS Weeklyish Oblations, Objections, and News (SWOON)
Volume I, Issue 6. 2013-04-25. Editor: Nick Black.
I. INTRODUCTORY RITES
“It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them. Operations of thought are like cavalry charges in a battle — they are strictly limited in number, they require fresh horses, and must only be made at decisive moments.”
—Alfred North Whitehead, An Introduction to Mathematics (1911)
“Hey there, fella, whose soul are you tormenting now?”
—Clutch, “Binge and Purge” (1993)
Well, SWOON seems to have become more Monthlyish than Weeklyish of late, but SMOON just doesn't have the same ring to it (“S-M-O-O-N, that spells SMOON!”), so we'll let things stand. I've spent a lot of time recently preparing for the class I'll be teaching this summer at Georgia Tech, playing around with the newly complete C++11 support in LLVM, preparing Sprezzatech's initial hardware line, hacking on OpenGL and XCB, and doing some work to keep everything funded. Busy, busy, busy, as they said in the Books of Bokonon.
Work on RAPTORIAL continues, but my focus has shifted to improving performance of the core, rather than outlying pieces like apt-file. With the aptitude maintainers having publicly stated that they're “uninterested in performance” and generally unwilling to accept patches devoted towards that end, I've gained significant liberty and freedom of action—essentially, I've accepted that Debian isn't going to welcome this work anytime soon, and thus I needn't make such an effort of keeping my changes minimal. So, I've been reading the dpkg, aptitude and APT sources, familiarizing myself with the dirty depths of the APT stack. I think that, with a bit of effort, APT could be made fast enough that never again will I sit there, wondering “what the hell is taking you so long?” Not only APT itself, but the package-building infrastructure (e.g. dh-shlibdeps, dh-install, etc) can all be vastly accelerated. Hurrah, differentiation!
fish is one of the coolest programs I've seen in awhile.
II. LITURGY OF THE WORD
Updates ahoy! As always, this list is neither autoritative nor exhaustive:
- Wayland ⇒ 1.1.0
- Harfbuzz ⇒ 0.9.16
- GHC ⇒ 7.6.3
- GNOME ⇒ 3.8.1
- libtins ⇒ 1.0
- cgminer ⇒ 3.0.1
- gparted ⇒ 0.16.0
- libpipeline ⇒ 1.2.3
- librelp ⇒ 1.0.5
- mpich ⇒ 3.0.4
- gprolog ⇒ 1.4.4
- openimageio ⇒ 1.1.10
- smokekde ⇒ 4.10.2
- oxygen-gtk ⇒ 1.1.3
- libav ⇒ 9.5
- graphicsmagick ⇒ 1.3.18
- gjs ⇒ 1.36.1
- libvision ⇒ 0.0.26
- libcdr ⇒ 0.0.13
- pinfo ⇒ 0.6.10
- muffin ⇒ 1.7.3
- byobu ⇒ 5.37
- gmic ⇒ 126.96.36.199
- hlint ⇒ 1.8.44
- compiz8 ⇒ 0.8.8
- stk ⇒ 4.4.4
- fish-shell ⇒ 2.0.0-pre
- snappy-player ⇒ 0.3
- mysql ⇒ 5.5.31
- pulseaudio ⇒ 3.99.1
καὶ εἰς ἕνα Κύριον Ἰησοῦν Χριστὸν τὸν υἱὸν τοῦ Θεοῦ τὸν Μονογενῆ.
III. LITURGY OF THE EUCHARIST
With 2318 source packages in SprezzOS, all of them maintained by moi, I think I've got room to speak about Debian's vaunted packaging practices. That said, I do not speak the following as a Debian Developer, nor do I consider myself tremendously well-informed regarding Debian as a whole. I'm sure there will be some incorrect statements and omissions in the next few paragraphs, and I welcome your corrections. My main thrusts, however, I expect to stand.
- Has at least three major implementations
- Is nowhere completely documented or even described
- Cannot scale without adding more developers
- Is unsuitable for analysis at the source level
- Is painfully slow to learn, debug, and perform
- Is rife with violations of what standards do exist
But that's about it. You can build this ar archive with debuild, or dpkg-buildpackage or debian/rules binary or ar or cat. It appears that the autobuilders use dpkg-buildpackage, looking at the porting guide, so let's use that as our minimum bar for packaging materials -- you ought build in a fresh chroot + all listed deps.
Except what do you build? The process of autobuilding relies on a source package. But when you're doing the packaging itself, you haven't yet built a source package. When you're taking over a package, there's no assurance that you can find the original tarballs on the original source, even when that source still exists. This is one purpose of watch files—to point at the original source, thus facilitating automatic checks for new versions. Obviously, this is not a complete solution (packages move sometimes), but it allows the vast majority of upstream updates to be automatically detected. Unfortunately, Debian doesn't require that these files be correct or included (it's recommended now, but I'd estimate a good 20% of source packages to have incorrect watchfiles). The first thing I do when I take over a package for SprezzOS is validate and correct the watchfile. Some upstream packages, however, are released in a format not amenable to uscan + dpkg-buildpackage, and require some intermediate processing. This processing is nowhere codified and seems to be done totally ad hoc; either way, it breaks our attempts to automatically update.
Thankfully, these changes tend to be oriented towards e.g. DFDL satisfaction, and are thus inappropriate for SprezzOS. Once the upstream is being properly unpackaged, we blow away any debian/ directory it might have shipped, and copy in our own. At this point, we run into any number of problems in the packaging itself, i.e. that would not cause us problems when building from a local tarball without intent to package:
- specified build tools no longer build the source
- install manifests no longer cover all installed files
- install manifests specify files that are no longer installed
- configure options are used that no longer exist
- new library versions no longer build the source
- sonames change, requiring package control/symbol/shlibs changes
- symbols change, requiring symbol changes
- libraries installed change, requiring shlibs/symbols changes and de facto requiring control changes
- package building process is invalidated any number of a thousand different ways
The fine-grained division of packages that suffuses Debian like a thousand paper cuts are their own problem.
But it's the free-form rules file that provides me the most constant serious problems. Who thought that a Turing-complete logic programming system was the best way to build more than, like, three packages? You can use your own free-form Make; you can use debhelper, or you can use cdbs. There's almost as many packaging implementations for this one format as there are build systems in use, and just like the build systems, they're all fucking terrible. They are embarrassments. I've been shocked and profoundly disturbed over this past half-year of SprezzOS development, having run Debian for thirteen years or so now, at how the sausage is made. If we consider obviously future-unproofed chicanery having clearly superior and immediately identifiable improvements as “bugs” —which they certainly are to the poor SprezzOS bastard who has to come along and update your package for the first time in years—then rules files probably have, on average, bugs proportional to and greater than their length. I.e. more than one bug per line. When I open a rules file of more than 10 lines, I deflate. When it's more than 30 lines, I look around really hard for something else to do, like check the price of blue-ring octopodes or practice knotting my dick into a pretzel. When it's, say, gcc-4.8 I waste twenty to forty curse-dense hours trying to get it to work, and eventually throw up my hands.
CDBS is actually pretty good. If every package had a watchfile and a CDBS-based rulesfile, I'd be a pretty happy guy. Happier than I am today, for sure.
More on this topic at a later date. Know that a solution inchoate is forming itself in my fetid mind.
IV. COMMUNION RITE
Not so many patches accepted upstream recently...Allegro5, Muffin, argyle, maybe something else. Actually, I guess that's one a week, which has been pretty steady since we started—it just seems less impressive now? Who knows.
Agnus Dei, qui tollis peccata mundi, miserere nobis.
V. CONCLUDING RITE
- The most recent release of SprezzOS is 1.1.0 ("von Neumann").
- There are now 2318 source packages in SprezzOS.
- There are now 9569 binary packages in SprezzOS.
- There are now 2 registered Sprezzadevs.
Ite, missa est.