← all posts

Troubleshooting Docker containers starting before an NFS mount

Gordon Beeming
Gordon Beeming
On this page4 sections ▾

After moving my home automation data from the server's SSD to a UniFi NAS, Home Assistant showed its first-run setup screen following a reboot. The existing files were still on the NAS, but the container was using a local directory instead.

The problem was the order in which the NFS mount and the container became available. A directory can exist at the expected path without being backed by the expected filesystem.

#How the local directory gets used

With Compose's short bind-mount syntax, Docker creates the host source directory if it doesn't exist. If the NAS hasn't mounted yet, that directory is created on the local filesystem. The application can then initialise new data there. Docker documents this behaviour.

On my server, these directories existed under /mnt:

Terminal
oak@pokedex:~$ ls -la /mnt/
drwxr-xr-x 3 root root 4096 Jan 10 23:16 homeassistant
drwxr-xr-x 3 root root 4096 Jan 10 23:16 homebridge

That listing showed the paths existed, but it didn't show which filesystem backed them. When I browsed the network share directly, my original files were there.

Mounting the NAS over the local directory can hide the local files from the host's view. An existing container bind mount can still refer to the original local directory. Docker's bind-mount documentation explains that mounts obscure existing contents and use private propagation by default.

#Check the backing filesystem

On the host, check the path used by the container:

Terminal
df -h /mnt/homeassistant/homeassistant_data

This output means the host path is on the local SSD:

Output (local SSD)
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p2  1.8T   42G  1.7T   3% /

With the NAS mounted, the host output shows its export instead:

Output (NAS mounted)
Filesystem                            Size  Used Avail Use% Mounted on
10.42.40.101:/var/nfs/shared/pokedex  9.1T  284G  8.9T   4% /mnt

A correct host result doesn't prove that an already-running container uses the same mount. If the mount arrived after the container started, recreate the container after checking the host path and the original files.

#Recover without discarding either copy

Stop the affected Compose stack before working on the mount:

Terminal
docker compose down

Check the NAS directly and confirm the original data is present. If the expected share is not mounted on the host, mount it using your existing mount configuration, then repeat the filesystem check. For a share configured at /mnt in fstab:

Terminal
sudo mount /mnt
df -h /mnt/homeassistant/homeassistant_data

Only start the stack once the path resolves to the expected NAS export and contains the application data:

Terminal
docker compose up -d

If you need to inspect the files beneath the mount, stop every service using it and unmount it normally with sudo umount /mnt. If it reports that the mount is busy, find what is using it before proceeding. Preserve any local files separately; they may include changes made while the application was running against the SSD. Don't delete everything under /mnt as a recovery step.

#Prevent automatic directory creation

Use the long bind-mount syntax to stop Compose from creating a missing source directory:

docker-compose.yml
services:
  homeassistant:
    # ...
    volumes:
      - type: bind
        source: /mnt/homeassistant/homeassistant_data
        target: /config
        bind:
          create_host_path: false

create_host_path: false makes a missing source path an error. It doesn't verify that NFS is mounted: an old local directory at the same path can still satisfy the existence check.

Startup also needs to wait for the expected filesystem. Configure that in the service or mount setup that starts your containers, and check it with a reboot. The useful verification is that the NAS mounts before the application starts and that Home Assistant opens the existing installation.

Gordon Beeming
Gordon Beeming

Father • Husband • Triathlete • SSW Solution Architect

Related posts