We already covered how Linux executes files and how to run ARM binaries “natively” in Linux in the last two posts. We ended up running a whole ARM root filesystem transparently in a chroot jail using QEMU user mode and binfmt_misc support.
Now that we have that covered, nothing prevents us from applying that to Docker containers. At the end of the day, containers are chroots in steroids, and they also share kernel with the host so all the basic mechanics remain the same.
Running ARM containers
If you haven’t yet, install QEMU user mode, by default it will install also binfmt_misc support.
| # apt-get install qemu-user |
Now, we only need to place the qemu-arm-static interpreter inside the container. This is as easy as mount-binding it in the container invocation and voila! We are running an armhf container in our x86 machine.
| $ docker run -v /usr/bin/qemu-arm-static:/usr/bin/qemu-arm-static –rm -ti arm32v7/debian:stretch-slim root@49176286e89e:/# |
Building ARM containers
This is a bit of an uglier trick, because we need to copy the QEMU interpreter to the container. The problem is that it adds ~5MiB to the container, but the benefit of being able to build foreign containers natively is immense!
Consider the following simplified Dockerfile to create an armhf nginx container
| FROM arm32v7/debian:stretch-slim
COPY qemu-arm-static /usr/bin RUN apt-get update; apt-get install -y nginx; echo “ndaemon off;” >> /etc/nginx/nginx.conf CMD [“nginx”] |
We end up with a container that can be run both natively in an ARM device, and in an x86 system with a properly configured binfmt_misc support. Not bad!
We can even test it locally
| docker build . -t nginx-armhf:testing
docker run –rm -ti -d -p 80:80 nginx-armhf:testing firefox localhost |
We are now running the ARM nginx web server locally.
This means that we can create a build pipeline with automated testing without requiring any ARM boards. After all automated tests are passing, we can create the production image.
| FROM nginx-armhf:testing
RUN rm /usr/bin/qemu-arm-static |
| docker build . -t nginx-armhf:production |
Reclaiming the space in Docker API 1.25+
If we really want to reclaim those extra 5MiB from qemu-arm-static, we can use the new experimental
–squash flag.
| docker build –squash . -t nginx-armhf:production |
Honestly, it was about time that they did something to help us keep the image size down. The piling up of layers and their refusal to add a simple
–chown argument to the COPY command made the creation of Dockerfiles a black art where you had to chain all the installation steps including the cleanup in a single RUN statement, therefore denying the benefits of caching for big blocks of code.
Thankfully, they are beginning to react and we now have not only
–chown , but also the
–squash flag for the build command. In order to use it, we need to enable experimental features. Add the following to the daemon.json file, which might not exist
Restart the Docker daemon, after this.
Reclaiming the space in Docker API <1.25
We are left with the old hacky way of flattening the images, by exporting the container
| docker container create –name nginx-armhf-container nginx-armhf:production
docker export nginx-armhf-container | docker import – nginx-armhf:raw |
, and finally rebuild the metadata
| FROM nginx-armhf:raw
CMD [“nginx”] |
| docker build . -t nginx-armhf:latest |