jumbo-builds

Sometimes we have a massive build. Massive builds make Arch kids laugh at us, so let's run them sparingly.

1. Minimizing recompiles

Suppose you compile a software, change a USE flag, then recompile again. Rebuilding the same .o files wastes time.

To cache .o files, you might think of using ccache due to its Portage integration.

This is old.

Use sccache instead, discussed in a footnote on the wiki's ccache page ⇗. I find this more relevant to rebuilds of local .ebuild stuff so I discuss it in overlay-funroll ⇒ instead.

2. getbinpkg

Binary caches are a curious thing.

  1. On the user side, Arch Linux is binary first, PKGBUILD later.
  2. On the user side, NixOS is "technically" source first, binary later. However Cachix is so massive that most of your packages are going to be binary downloads, given that the same package-build-hash you're looking for is available.
  3. On the user side, Gentoo is source first, binary much later. You typically only manually override source builds; many even disrecommend it.

I use binary packages if I'm in a time crunch or if building from source doesn't work for some reason. However Gentoo binary packages are often unavailable for your USE flag list; it's easonable since the amount of builds to cache would be upper-bounded by two-factorial and lower-bounded by "typically-changed USE flag"-factorial.

However, the clever of us run into a sad bug: Bug 337456 ⇗.

Observe:

/etc/portage/env/getbinpkg
FEATURES="${FEATURES} getbinpkg"

/etc/portage/package.use/jumbo-builds
net-libs/webkit-gtk keyring lcms pdf spell
media-libs/gst-plugins-base opengl #dep

/etc/portage/package.env/jumbo-builds
net-libs/webkit-gtk getbinpkg

In the above conf files, one would expect for FEATURES=getbinpkg to be only conditionally enabled for webkit-gtk. But on attempted install, webkit-gtk will still only be built from source.

That's a Gentoo bug since 2010.

Preemption. "Fix it yourself." Thanks smartass, but there'd be a lot of liaising to do first, since eclass architecting necessitates caution. (Just an excuse, since I haven't gotten to it yet, nor has anyone else who ran into bug 337456.)

3. surrender

Therefore the best thing to do is to

emerge --oneshot --getbinpkg -av $some_jumbo_pkg

every time a @world update asks to upgrade such a package.