Creating Tiny Docker Images

Posted on June 4, 2020

I never understood why people wanted their Docker images to be based on a full Linux distribution1, or why each command created a new layer.

Recently I learned of Google’s Jib tool for Java applications. The JVM, your project’s dependencies, and your project itself all get their own layers to maximise sharing which I thought was really cool. Inspired by this, I tried to do something similar a native executable (specifically, the Monero daemon).

Building the Executable

I built Monero like usual, install dependencies, git clone, configure, and make…

apt install -y --no-install-recommends \
  build-essential \
  cmake \
  git \
  libboost-all-dev \
  libpgm-dev \
  libsodium-dev \
  libssl-dev \
  libunbound-dev \
  libzmq3-dev \
  pkg-config
git clone https://github.com/monero-project/monero.git
cd monero
git checkout v0.16.0.0
git submodule init
git submodule update
cmake \
  -Bbuild \
  -DCMAKE_BUILD_TYPE=Release \
  -DBUILD_SHARED_LIBS=On
make -j$(nproc) -Cbuild daemon wallet_rpc_server

-DBUILD_SHARED_LIBS=On tells CMake to build the Monero specific libraries (like librandomx and libringct) as shared libraries and not statically link them into the executable. It will help sharing between our two executables.

Creating the Layers

For my first2 layer, I’m going to put all of the system libraries that my executables require. You can get all of an executable’s dependencies with the ldd command. I will also copy the executable loader.

mkdir -pv ~/layer-syslibs/{lib/x86_64-linux-gnu,lib64}
ldd build/bin/* | 
  awk '/=>/ && !/monero\/build/ { print $3 }' |
  sort -u |
  xargs -I '{}' cp -v '{}' ~/layer-syslibs/lib/x86_64-linux-gnu/
cp -v /lib64/ld-linux-x86-64.so.2 ~/layer-syslibs/lib64/

The second layer will be almost the same, but I’ll copy Monero’s libraries instead.

mkdir -pv ~/layer-applibs/lib/x86_64-linux-gnu
ldd build/bin/* | 
  awk '/monero\/build/ { print $3 }' |
  sort -u |
  xargs -I '{}' cp -v '{}' ~/layer-applibs/lib/x86_64-linux-gnu/

The third and final layer will contain the executable. I’m packaging two executables (the daemon and wallet RPC server) so I’ll create two of these.

mkdir -pv ~/layer-monerod/bin
cp -v build/bin/monerod ~/layer-monerod/bin/
mkdir -pv ~/layer-wallet/bin
cp -v build/bin/monero-wallet-rpc ~/layer-wallet/bin/

We can then put all of these layers into archives:

for i in layer-*; do
	tar -C $i -cvf $i.tar .
done

Creating Container Images

Now that we have our layers, we can put them into an image. The easiest way is to write a Dockerfile. I think there might be some voodoo magic you can do using docker import/save/load and modifying the manifests by hand, but I haven’t been able to figure it out yet.

My two Dockerfiles looked like this:

# Dockerfile.monerod
FROM scratch

ADD layer-syslibs.tar /
ADD layer-applibs.tar /
ADD layer-monerod.tar /

EXPOSE 18080/tcp 18081/tcp

ENTRYPOINT ["/bin/monerod"]

# Dockerfile.monero-wallet-rpc
FROM scratch

ADD layer-syslibs.tar /
ADD layer-applibs.tar /
ADD layer-wallet.tar /

ENTRYPOINT ["/bin/monero-wallet-rpc"]

When ADDing an archive, Docker will automatically extract it, which is just what we want.

I can now build my new containers like this:

docker build \
  --rm \
  --tag moneromint/monerod:v0.16.0.0 \
  -f Dockerfile.monerod .
docker build \
  --rm \
  --tag moneromint/monero-wallet-rpc:v0.16.0.0 \
  -f Dockerfile.monero-wallet-rpc .

You can find my containers on Docker Hub:

Troubleshooting

You may need to include the root CA certificates (/usr/share/ca-certificates) into your image if your application needs to make any TLS connections.

Going Further

I’d like to automate this whole procedure, and maybe split things up further by creating more layers (libc gets a layer, libboost on another, etc.) but this might not be a good idea due to the limit on the number of layers3.


  1. I understand that it doesn’t result in any extra disk space being used (if multiple images are based on the same distro), but it still seems a bit weird.↩︎

  2. The order of the layers doesn’t really matter. You could put the executable first if you really wanted.↩︎

  3. https://github.com/docker/docker.github.io/issues/8230↩︎