Patching pkgsrc, building rust and Arm headaches

2020-07-20

Having been working with pkgsrc for a couple weeks now there's a bit of a flow developing, and quite a bit of progress, with a big caveat. The issues I'm running into are that there are quite a few more build failures for Arm than there are for x86_64.

Interestingly, both libdrm and MesaLib fail with similar messages about assembly code that isn't supported on the current architecture (armv7l). In situations like this it's hard to blame pkgsrc, as the fault is upstream from there. I'm getting quite used to adding packages to my local tree and tweaking BSD style Makefiles. I'm making every attempt to send fixes upstream; however, waiting for a response each and every time is just not really going to be an option here, so in the meantime I've decided to import the entirety of pkgsrc into git and use that tree as the "official" branch for HitchHiker. This will enable our project to build up a substantial package collection even if upsteam pkgsrc begins to lag behind.

One of the problematic packages that is now often being pulled in as a dependency is rustc, the rust compiler. It's problematic on two fronts. First, unless your system has substantial amounts of memory a parallel build will fail as the memory fills up (mine has 8 gig and still fails). Second, and quite frustrating, is that rustc is only compatible with certain versions of either openssl or libressl, and our version of libressl was too new. After seeing if we could use either libressl or openssl from pkgsrc and encountering repeated failures, I just rolled back libressl to the newest version that rust supports.

Let me pause here for a brief commentary on rust as a programming language. There is a lot to like about rust, and it seems that the language may be with us for quite some time, as it seems to see more usage every day and has achieved a critical mass of developers. I've played around with it myself a little bit and the documentation is very good, it's probably one of the easier to learn languages, and the ecosystem as a whole is very nice when you look at all of the features that cargo brings to the table. However, I am not a fan of the fact that the rust standard library, libstd, is compiled in statically by default. This results in a simple "Hello, world!" executable compiling into a 7.1m binary on my machine before stripping, which comes down to 220k after stripping. When compiling the same program dynamically it comes all the way down to 16k after stripping, so we have approximately a 204k overhead per binary for the static libstd.

I am also not happy about rolling back libressl, as I would much rather have the newest possible version of what is a security critical piece of software. However, it is I believe still a better and more secure option than openssl, which most distros are still using. It's a shame that a major piece of software like rust, that is a dependency of quite a number of other packages now, has not updated to use the newer abi presented by newer versions of libressl.

After a couple weeks of building packages on both x86_64 and armv7l, using an identical set of packages in the base system, it is very apparent just how much more development x86_64 gets than arm. Two packages in particular illustrate this quite well - libdrm and MesaLib - both of which are major dependencies of modular Xorg. In both cases assembly code is generated by gcc that contains instructions not supported by the processor, and the only fix is to disable features in the Arm build. In the case of MesaLib this involves patching the source code. Thanks to the Arch Linux Arm team for showing the way, as my patches are adapted from their PKGBUILD.

There have been just a few more tweaks to the base system as I attempt to fully finalize the build in preparation for release. The rpcgen utility that I had incorporated proved to not fully support all features needed for building certain packages in pkgsrc. After some search I found a newer version that had been autotooled in the rpcsvc-proto package, which was originally a compatibility package during the time that SunRPC was being removed from Glibc. I've imported the source for just the rpcgen utility from that package into hitchhiker's build tree, removed all of the autotools material, moved all C-preprocessor flags into a config.h file, disabled NLS and made a very lean, mean Makefile for it. While I was at it I used similar techniques to update the builds for mksh and pax. Both of those packages now include their CPPFLAGS as defines in config.h, and I have even begun eliminating ifdefs out of the C files to make for a cleaner and faster build.

Tags for this post: