As we already know from this previous article, it is possible to configure Gentoo to distribute the compilation of ebuilds among different compilation nodes, but this method had a limitation: all machines had to share the same architecture. In this guide, we will solve this problem and make distcc compile for a different architecture than the one it is running on.

In my case, I will configure a compilation server with an amd64 architecture and a RaspBerryPi with an ARM architecture.

The first step will be to install distcc on both the compilation servers and the Rasp:

To make the magic possible, we will have to install crossdev on the server, as it is the necessary tool to compile the toolchains for another architecture:

Now we will need to maintain two profiles, one for the server architecture and another for the Rasp architecture, for which it is necessary to have the ebuilds from /etc/portage in directory format:

!/bin/bash
PROFILE_DIR="/etc/portage"
if [ ! -e ${PROFILE_DIR} ]; then
  mkdir ${PROFILE_DIR};
fi;

for PACK_DIR in package.accept_keywords package.keywords package.use package.unmask package.mask; do
  CUR_DIR="${PROFILE_DIR}/${PACK_DIR}"
  if [ ! -e ${CUR_DIR} ]; then
    mkdir ${CUR_DIR}
  fi
  if [ -e ${CUR_DIR} -a ! -d ${CUR_DIR} ]; then
    mv ${CUR_DIR} ${CUR_DIR}.moving
    mkdir ${CUR_DIR}
    mv ${CUR_DIR}.moving ${CUR_DIR}/monolithic
  fi
done
echo "Completed!"

We compile the toolchains:

If the compilation fails, we can try the following:

We add the IP of the RaspberryPi to the distcc configuration:

DISTCCD_OPTS="${DISTCCD_OPTS} --allow IP_RASP/32"

We finished setting up distcc:

On the Rasp:

MAKEOPTS="-jN -lM"
FEATURES="distcc buildpkg"

NOTE: N = Number of computers that will compile code (local and remote) + 1, M = Number of local cores, M is used when the compilation is local.

• distcc –> Uses distcc to distribute the compilation load ;)
• buildpkg –> Generates binary packages for all packages that are ordered to compile, so we can use this rasp as a package server.

DISTCCD_OPTS="--port 3632 --log-level notice --log-file /var/log/distccd.log -N 15 --allow 1.1.1.1"

NOTE: Actually, the Rasp will not compile code for anyone, but in recent versions of distcc, we are forced to configure at least one IP, so we put 1.1.1.1

We will have to indicate the architecture to use to distcc, what we will do is an intermediate script:

lrwxrwxrwx 1 root root 15 abr 7 16:30 armv6j-hardfloat-linux-gnueabi-c++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 armv6j-hardfloat-linux-gnueabi-g++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 armv6j-hardfloat-linux-gnueabi-gcc -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 c++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 cc -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 g++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 gcc -> /usr/bin/distcc

#!/bin/bash
exec /usr/lib/distcc/bin/armv6j-hardfloat-linux-gnueabi-g${0:$[-2]} "$@"

lrwxrwxrwx 1 root root 15 abr 7 16:30 armv6j-hardfloat-linux-gnueabi-c++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 armv6j-hardfloat-linux-gnueabi-g++ -> /usr/bin/distcc
lrwxrwxrwx 1 root root 15 abr 7 16:30 armv6j-hardfloat-linux-gnueabi-gcc -> /usr/bin/distcc
-rwx--x--x 1 root root 85 abr 7 16:34 armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 abr 7 16:35 c++ -> armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 abr 7 16:35 cc -> armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 abr 7 16:35 g++ -> armv6j-hardfloat-linux-gnueabi-wrapper
lrwxrwxrwx 1 root root 38 abr 7 16:35 gcc -> armv6j-hardfloat-linux-gnueabi-wrapper

We can see which compilation server the tasks are being distributed to from the rasp with:

And we can see from the build server which part of the software is being compiled with:

NOTE: It should be noted that this does NOT work perfectly, as occasionally we may see errors of the following type on the build server:

distccd[24197] (dcc_readx) ERROR: unexpected eof on fd5
distccd[24197] (dcc_r_token_int) ERROR: read failed while waiting for token "DOTI"
distccd[24197] (dcc_job_summary) client: 192.168.69.13:32797 OTHER exit:0 sig:0 core:0 ret:108 time:0ms

However, the load distribution continues during the compilation, so…