Packaging for 2021q2

2021-02-26

I spent a good deal of the past couple weeks working on actually getting init up and running, preparing kernel configurations for multiple architectures, and learning about Das U-Boot in order to put up bootable images. Taking a little break from that the last few days, I am now once again working at building a set of packages using pkgsrc.

Inevitably considering how many utilities have been replace in Hitch Hiker for either BSD versions or scratch implementations, I have been running into breakage in odd places. Most are relatively easy fixes. A few kept repeating in an infuriating manner...

GNU autotools is an attempt to make a framework that allows building software on multiple architectures and operating systems. I won't attempt to elaborate much on how it works, but in order to support as many different systems as possible the lowest common denominator is assumed when it comes to the infamous 'configure' shell script, meaning that it is supposed to be POSIX compliant and not use any extensions to the POSIX spec in relation to shell grammar. I'm mentioning this because a whole bunch of Gnome related packages use the same test for intltool in their configure sript. The gist of the test is this:

intltool-update --version | head -1 | cut -f" " -d 3

I previously ran into this when compiling and testing Servo, the browser engine formerly sponsored by Mozilla written in Rust. I was pissed at the time, because it's such a simple thing. Basically, the POSIX spec only includes the -n option when describing the behavior of the head utility. And my version of head did not support the older syntax head -[num] as it is not possible to code that sort of thing by just using getopt. The BSD head actually parses and rewrites argv in order to support this. Not sure what GNU does but I'm sure it's a few hundred lines that shouldn't be necessary.

The fact that this is the same test, with exactly the same syntax and whitespace, is a dead giveaway that the package authors basically have just been copying and pasting the same code without ever reading or understanding it. It's also indicative of another thing I hate; passing --version to a utility should return a string of numbers, not multiple lines including a copyright statement and author credit. There should be no need for calling head to begin with.

Furthermore, checking for package versions is bad practice to begin with. Say I created a fully functional drop-in replacement for intltool. It does exactly what intltool does, it accepts the exact same input and gives identical output. If I then tried to build any of these packages that check for a specific version of intltool they would fail. Best practice is to check for capabilities, not version numbers.

To fix the issue I did what I had said previously that I didn't want to do; made the head utility in Hitch Hiker accept the non-conforming syntax. I also hacked our BSD originating uname utility while I was at it to accept the GNU -o option and just alias it to -s, as a few things had choked on that option.

Creating a Hitch Hiker overlay for pkgsrc

One of the changes that I have been tinkering with in the base system without success is the ability to build the other components of gcc beyond the C and C++ compilers. I have had no luck building the Go, Fortran, Objective C or Objective C++ frontends with the sysroot build. So I tried to build the gcc10 version from pkgsrc, only to have the Go compiler fail at the last moment. Interestingly, I was able to build all of the components manually with zero issues. In looking at the Makefile in pkgsrc/lang/gcc10 I realized that there's quite a lot of conditionals to aid in building on different platforms and there are also quite a lot of options passed to configure beyond what I normally use. In short, for our limited purpose I thought that I could do better.

What I have created is a Hitch Hiker specific overlay for pkgsrc. Basically, the overlay gets extracted in the same location as pkgsrc and adds some Hitch Hiker specific packages, prefixed with hhl_ in their name. The hhl_gcc10 package is beautifully simple, builds in approximately 1/3 the time as the pkgsrc version (we skip the three stage bootstrap of gcc as we're compiling it with the same version from the base system) and builds the C, C++, Fortran, Go, Objective C and Objective C++ frontends. Additionally, the pkgsrc gcc10 package puts everything under a different prefix, /usr/pkg/gcc10, rather than /usr/pkg. I don't like adding extra prefixes in general, so I didn't follow suite with that practice.

I think this is actually a better solution in the long run than continually patching pkgsrc for things that fail to build on Hitch Hiker. If a package requires a specific change, I can create a new package that does build and it can be installed with the name of the original to satisfy dependencies while we try to get changes pushed upstream. In this way we can still have a fairly maintainable set of Hitch Hiker specific changes, as they will live in their own repo.

Tags for this post: