= LSE atomics problem =
Large System Extensions, part of ARMv8.1, provides new atomics instructions. These instructions are essential for high performance locking when you have lots of cores.
Recommendation: distributions should provide an alternative optimized glibc with LSE atomics based locks for end users.
= Scalable Vector Extensions =
May have an ABI issue. Situation under investigation.
= Page size =
Don't assume your binaries will always execute in the page size they were built on. Even if your distribution is built on 4K page size, be aware that some of your users might opt to run with 64K page size kernel - for example in containers.
Users be aware, that switching between 4K and 64K kernels may break some data structures that depend on page size. Swap needs be reformatted, and btrfs will explode on page size change-
= OCI spec =
With LSE, SVE, armv8.x releases coming, any containers using newer instructions should be identified in variant field.
= Booting =
One day, booting on arm will be so boring, that nobody will bring it up on our cross-distribution BoF. That day was not today.
RHEL only supports ACPI on arm64. Even if you are not RHEL, you should prefer ACPI over device tree, if the former is available.
U-boot can now start EFI binaries, making u-boot an excellent platform to implement an UEFI bios. This allows single-path installers for distributions, where install of distribution starts always by loading grub from EFI.
On arm64, Kernel doesn't self-decompress. The bootloaders are stringly recommended to support decompressing kernel images. It's optional in grub, make sure your grub does. At least iPXE and u-boot don't support booting Image.gz on arm64, and should be fixed.
Riku