I have a working system based on Sumo. The system boots with a read-only rootfs, then applies are read-write overlay for /etc.

When I migrate to Warrior, systemd-resolved fails to start. If I mount the same rootfs via NFS, it starts and works fine.  systemd-timesyncd is also failing, but I haven't looked into that yet. It also works fine on the NFS mounted system.

The resolve problem seems to be caused by two things:

    - /var/volatile is read-only

    - /run/systemd/resolve has the wrong ownership

      drwxr-xr-x  2 systemd-network systemd-journal  80 Jul 12 16:23 resolve/

      I think this permissions problem may be a result of the /var/volatile mounting

      problems; it looks fine on the NFS mounted system.

If I manually mount /var/volatile (it's in fstab) and change the ownership on /run/systemd/resolve, the service starts just fine.

I also notice that /tmp is not mounted at all, which may be related.

Here are the various tmp mount points on my read-only rootfs:

$ mount | grep tmp

devtmpfs on /dev type devtmpfs (rw,nosuid,size=112036k,nr_inodes=28009,mode=755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
overlay on /etc type overlay (rw,relatime,lowerdir=/tmp/lower/etc,upperdir=/tmp/upper/etc,workdir=/tmp/upper/work/etc)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=23840k,mode=700)

On the NFS mounted system, I see these:

$ mount | grep tmp
devtmpfs on /dev type devtmpfs (rw,relatime,size=118180k,nr_inodes=29545,mode=755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev)
tmpfs on /var/volatile type tmpfs (rw,relatime)
tmpfs on /run/user/0 type tmpfs (rw,nosuid,nodev,relatime,size=23840k,mode=700)

As you can see, NFS has these extra mounts:

tmpfs on /var/volatile type tmpfs (rw,relatime)

I've tried reverting a few commits that may be related, but I haven't had any luck working out things have changed, eg:

c4acf1b531  2018-10-19  volatile-binds: use overlayfs if available                      [Matt Hoosier] 

Advice would be appreciated. Are there any particular areas I should be looking to work out what's going wrong?

Kind regards,

Ryan.