Hi all,
(Sent to cross-distro with debian-arm on cc.)
We have an 'interesting' situation ahead of us, or indeed some of us
have already fallen into it:
ARM64 platforms with > 512GB between the lowest and highest RAM
addresses end up getting their amount of usable memory truncated if
the kernel is built for 39-bit VA (which is what currently happens for
Debian kernels). For 4.7, the arm64 defconfig was changed to enable
48-bit VA by default.
While itself not a critical error (but really annoying), in
combination with GRUB putting the initrd near the top of available
RAM, we end up with systems not booting. We think we've also seen
issues with ACPI tables above this waterline.
Simple - all we need to do then is enable 48-bit VA in the arm64
kernel config? Well, yes. I know Fedora are already doing this, and I
have raised a bug[1] for Debian to do the same.
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=834505
The problem is - some pieces of software have had time to be written
in a ... let's charitably call it a "focused on amd64" fasion ... with
the embedded assumption that anything above virtual address bit 44 is
a pointer-tag free-for-all.
On the Debian-ish side, we're coming up on both Ubuntu 16.10 and the
freeze for Stretch, leaving a pretty short window to resolve this
unholy kernel->initrd->userland triangle.
The applications we know are affected are luajit and mozjs (libv8 is
not a problem). But this has a follow-on cost: both of these are used
by other packages. Other jit/runtime packages could have their own
issues.
The mozjs bug is fixed on trunk, and will hopefully make it into
release 49[2], but it remains to be seen if that's too late for some
distributions.
[2] https://bugzilla.mozilla.org/show_bug.cgi?id=1143022
For luajit, I'm told this has been fixed on 2.1 branch, but not merged
to master?
Now, Jeremy (cc:d) tells me the list of currently-known Fedora
packages affected by this are:
couchdb
elinks
erlang-js
freewrl
libEAI
libproxy-mozjs
mediatomb
pacrunner
plowshare
polkit
cinnamon
cjs
cjs
cjs-tests
gjs
gjs
gjs-tests
gnome-shell
0ad
mongodb
mongodb-server
Some of these may only need an updated luajit/mozjs package, but some
may need more invasive changes.
Anyway, this is just a heads up - anyone sitting on more information
than I've put into this email is very welcome to share it.
/
Leif
Recently my angry post on Google+ [1] got so many comments that it was
clear that it would be better to move to some mailing list with discussion.
As it is about boot loaders and Linaro has engineers from most of SoC
vendor companies I thought that this will be best one.
1. https://plus.google.com/u/0/+MarcinJuszkiewicz/posts/J79qhndV6FY
All started when I got Pine64 board (based on Allwinner A64 SoC) and had
same issue as on several boards in past - boot loader written in some
random place on SD card.
Days where people used Texas Instruments SoC chips were great - in-cpu
boot loader knew how to read MBR partition table and was able to load
1st stage boot loader (called MLO) from it as long it was FAT filesystem.
GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and
reads firmware files from there as long it is FAT.
Chromebooks have some SPI flash to keep boot loaders and use GPT
partitioning to find where from load kernel (or another boot loader).
And then we have all those boards where vendors decided that SPI flash
for boot loader is too expensive so it will be read from SD card
instead. From any random place of course...
Then we have distributions. And instead of generating bunch of images
per board they want to make one clean image which will be able to handle
as much as possible.
If there are UEFI machines on a list of supported ones then GPT
partitioning will be used, boot loader will be stored in "EFI system
area" and it boots. This is how AArch64 SBSA/SBBR machines work.
But there are also all those U-Boot (or fastboot/redboot/whateverboot)
ones. They are usually handled by taking image from previous stage and
adding boot loader(s) by separate script. And this is where "fun" starts...
GPT takes first 17KB of storage media as it allow to store information
about 128 partitions. Sure, no one is using so many on such devices but
still space is reserved.
But most of chips expects boot loader(s) to be stored:
- right after MBR
- from 1KB
- from 8KB
- any other random place
So scripts start to be sets of magic written to handle all those SoCs...
Solution for existing SoCs is usually adding 1MB of SPI flash during
design phase of device and store boot loader(s) there. But it is so
expensive someone would say when it is in 10-30 cents range...
Even 96boards CE specification totally ignored that fact while it could
be a way of showing how to make popular board. Instead it became
yet-another-board-to-laugh (EE spec did not improve much).
Is there a way to get it improved? At least for new designs?
> Message: 1
> Date: Thu, 5 May 2016 13:45:57 +0200
> From: Marcin Juszkiewicz <marcin.juszkiewicz(a)linaro.org>
> To: linaro-dev(a)lists.linaro.org
> Cc: cross-distro(a)lists.linaro.org
> Subject: Let's talk about boot loaders
> Message-ID: <89cb7539-fbea-e9d6-10a7-2ea4333dc756(a)linaro.org>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Recently my angry post on Google+ [1] got so many comments that it was
> clear that it would be better to move to some mailing list with discussion.
>
> As it is about boot loaders and Linaro has engineers from most of SoC
> vendor companies I thought that this will be best one.
>
> 1. https://plus.google.com/u/0/+MarcinJuszkiewicz/posts/J79qhndV6FY
>
>
> All started when I got Pine64 board (based on Allwinner A64 SoC) and had
> same issue as on several boards in past - boot loader written in some
> random place on SD card.
>
> Days where people used Texas Instruments SoC chips were great - in-cpu
> boot loader knew how to read MBR partition table and was able to load
> 1st stage boot loader (called MLO) from it as long it was FAT filesystem.
>
> GPU used by Raspberry/Pi is able to read MBR, finds 1st partition and
> reads firmware files from there as long it is FAT.
>
> Chromebooks have some SPI flash to keep boot loaders and use GPT
> partitioning to find where from load kernel (or another boot loader).
>
> And then we have all those boards where vendors decided that SPI flash
> for boot loader is too expensive so it will be read from SD card
> instead. From any random place of course...
>
>
> Then we have distributions. And instead of generating bunch of images
> per board they want to make one clean image which will be able to handle
> as much as possible.
>
> If there are UEFI machines on a list of supported ones then GPT
> partitioning will be used, boot loader will be stored in "EFI system
> area" and it boots. This is how AArch64 SBSA/SBBR machines work.
>
> But there are also all those U-Boot (or fastboot/redboot/whateverboot)
> ones. They are usually handled by taking image from previous stage and
> adding boot loader(s) by separate script. And this is where "fun" starts...
>
> GPT takes first 17KB of storage media as it allow to store information
> about 128 partitions. Sure, no one is using so many on such devices but
> still space is reserved.
>
> But most of chips expects boot loader(s) to be stored:
>
> - right after MBR
> - from 1KB
> - from 8KB
> - any other random place
>
> So scripts start to be sets of magic written to handle all those SoCs...
>
> Solution for existing SoCs is usually adding 1MB of SPI flash during
> design phase of device and store boot loader(s) there. But it is so
> expensive someone would say when it is in 10-30 cents range...
>
> Even 96boards CE specification totally ignored that fact while it could
> be a way of showing how to make popular board. Instead it became
> yet-another-board-to-laugh (EE spec did not improve much).
>
> Is there a way to get it improved? At least for new designs?
Unfortunately, SoC vendors don't give a crap; as long as *their* SDK works,
anyone else can take a hike up a tree. If it wasn't for JCM pushing against
this, aarch64 servers would be in the same state, and they'd never ever
gain traction against Intel.
John.
Hi,
We have a few sessions at this Linaro connect that could be in
interest of Linux distributions working on Arm/Linux:
Tuesday 14.00 Bangkok time: (07.00 UTC):
https://bkk16.pathable.com/meetings/372827 BKK16-212: What's broken on ARM64?
Thursday 10.10 Bangkok time: (03.00 UTC):
https://bkk16.pathable.com/meetings/372890 BKK16-402: Cross distro BoF
Friday 10.10 Bangkok time: (03.00 UTC):
https://bkk16.pathable.com/meetings/372923 BKK16-501: Kernel
Consolidation and Other Short Stories
Talks are being live-streamed (thou there were problems today..) and
recorded in case time happens to be inconvenient in your timezones.
If you have anything specific in your mind, feel free to reply to this mail to
cross-distribution list, and I'll get it added to the agenda slides.
Cheers,
Riku
ILP32 loader path and triplet
ILP32 is an ABI to run on ARM v8 64-bit-only cores. Thus it has the
same basic properties as aarch32 or armv7 code (ints, long and
pointers are 32bit), but only uses the armv8 A64 instruction set. It
is exactly the same concept as x32 on the x86 architecture (but is
interesting for somewhat different reasons).
There has been some discussion over the last year about the degree to
which this was actually neeed/useful in the real world, and it seems
that enough people care that the development work will be finished and
it will be upstreamed. Binutils and gcc already have basic support
upstream, a kernel ABI has finally been agreed, and thus the glibc
implementation can now be finished.
So, whilst distros (certainly Debian, but I presume others) remain
entirely uninterested in this as a supported architecture, some people
do want to be able to build it, and that means we should agree the
triplet and loader path to be used so it's not gratuitously different
between distros.
This page has been up for a year or so giving current status,
rationale and sugested names:
https://wiki.linaro.org/Platform/arm64-ilp32
It was brought up in the cross-distro session at Linaro Connect in
November 2015 and no-one seemed to object violently to what is there.
So, before people start actually building this for real (beyond OE)
I'd like to confirm that everyone is happy to use these, which are
essentially just following on from the existing aarch64 and
aaarch64_be names, and ensuring that the loader path is unique.
GNU name ('triplet'):
aarch64_ilp32-linux-gnu
Loader path:
/lib/ld-linux-aarch64_ilp32.so.1
And Big endian versions: (quite which crazies are going to build
ilp32-big-endian I don't know, but lets define them whilst we are here :-)
GNU name ('triplet'):
aarch64_be_ilp32-linux-gnu
Loader path:
/lib/ld-linux-aarch64_be_ilp32.so.1
Do we have a shed which is already adequately painted, or would people
like to argue for something different?
I would like to send in a dpkg patch soon so that it's possible to
build this stuff with debian tools, and there will no doubt be further
discussion about actually building ILP32 toolchains and packages at
Linaro connect in March. I'm hoping this naming isn't controversial.
Aside:
Oh and if you didn't know (I didn't until I looked it up for
this). ILP32 just comes from "Integer, Long, and Pointer 32"
Wookey
--
Principal hats: Linaro, Debian, Wookware, ARM
http://wookware.org/
Hi everyone,
Since cross-distribution meetings at Linaro Connect have been popular,
we are continuing the tradition and having a session[1] in Linaro Connect next
week.
https://sfo15.pathable.com/meetings/302652
The event is on Monday September 21, 16.10 PST (aka 23:10:00 UTC).
Details of google hangout should be coming closer to event.
If you have anything specific in your mind, feel free to reply to this mail to
cross-distribution list, and I'll get it added to the agenda.
Cheers,
Riku
Hi,
Here's quickly cleaned notes from etherpad:
People affiliated with at least Debian, Fedora, OpenEmbedded,
OpenSUSE, Redhat, Ubuntu and Yocto were present and voiced opinions.
Kernel:
- Good: most Interesting platforms mainline now
- Qualcomm IFC (in progress), raspi2, hikey probably mostly wanted?
- Bad: sometimes mainline kernels features limited "it boots to serial console"
- OEMs still put their own kernels on everything they ship
Kernel testing:
- http://kernelci.org is gaining traction by kernel devs, but so far
tests mostly defconfigs (which are rather minimal)
- all options needed by the new pid 1 and other hard requirements to
boot a modern distro should be in the arm64 defconfig and multi_v*
configs
- if not, kernel developers present think adding them is definetly OK
- make allmodconfig testing at kernelci.orghttp://kernelci.org/build/mainline/kernel/v3.19-rc7-200-gda2d96d3aa18/defco…
- TODO: make allmodconfig boot!
- ACTION Koen volunteered to put some distro-style config fragments
into the kernel
Kernel command line options
- Now firmware can tell where console= is instead of defining in command
- https://www.kernel.org/doc/Documentation/devicetree/bindings/chosen.txt
- Similar setting exists for ACPI (SPCR)
- Userspace support needs to be done
- ACTION device tree files need to get chosen/stdout-path added
- Systemd should magically use that as serial console if
CONFIG_FHANDLE is enabled
Page size:
- 64/4K? (AArch64 Fedora/Red Hat kernels will differ: 4K for Fedora
for now, 64K for RH, there are a few issues with 64K left for triage)
- 32bit binaries compiled with latest binutils work with 64k pages
(trunk, so not in 2.25)
- 20-30 packages already fixed.
- probably more things making assumptions (BTRFS was glaring example)
Kernel trapping for old instructions not supported on v8. (mostly
android). SWP, SETEND, cp15 barriers.
- Some current packages have SWP due to embedded libatomicops. being fixed.
- need to make the personality do the right thing for chroots. linux32
says armv8l Would be nice to have a armv8l for building (or just fix
things)
- coreutils repeats uname name (aarch64 x3 - could be improved) Riku
will look into
32bit kernels on 64bit machines?
- Works with QEMU, but you don't get KVM acceleration.
- has worked on XEN - should still be working. IJC will fix if not.
Graphics
- framebuffer/drm/hdmi/panel drivers often half-done
- should DRM be built as modules or builtin?
- graphics developers prefer built-in, server people want them as modules...
- libdrm: enable tegra, exynos, freedreno, omap
- conclusion: nobody in the session was confident enough to take lead
on graphics issues
Applications still needing porting?
- golang in progress. Should arrive in 1.5 (release in August)
- gccgo also works for docker (esp gcc5)
- ocaml already fixed upstream
- mogodb, libv8, nodejs is tangled. (only new libv8 has arm64
support, mongodb forked the old one - big job to update). nodejs .12
needs to be released. ubuntu still maintains external libv8. suse gave
up. some pressure to stabilise ABI would help (and stop everyone
embedding copies)
- mono is done/built, still needs upstreaming ("some legal
issues"). $someone should build Microsofts code?
- GHCi - give some people a board, and wait longer
- freepascal, etc have bugs - time to tell upstreams that hardware exist
maybe hand out some boards to people to get stuff done.
Is booting solved problem?
- Document requirements/recommendations for OEMS
- Common requirements:
- boot into EL2 / Hyp mode!
- SPCR is serial port console redirection. No problem to
implement.(MS promise)
- there are two sets of EFIvars. You need to mount the newer
psedofilesystem (efivarfs). This is kernel build option.
ILP32 (some network people want it). Distros don't want to support
it!. Demo OE in a container and tell them to use that. (ssh broken on
ILP32) has different syscalls. Are they turned on?
Post-meeting discussion:
- Monthly Cross-Distro Google hangout planned (ACTION Riku)
Hi all,
This is just a quick heads-up to advise that I've found quite a nasty
bug in the Mozilla Javascript engine:
https://bugzilla.mozilla.org/show_bug.cgi?id=1143022
On systems running with a 48-bit VA, bit [47] is being incorrectly
masked out by the Javascript engine due to it being used internally
for tagging. This will cause random crashes (depending upon where mmap
returns memory).
In my case it even prevented the system from booting properly due to
polkitd crashing.
I would recommend anyone packaging the Mozilla Javascript engine to
keep an eye on this bug, I've flagged it as a "blocker".
Cheers,
--
Steve
Hi,
At connect, it was discussed if it is possible to run 32bit arm guests
on arm64 machines. This wanted especially by distributions that do
native builds, as server class 32bit hardware is rare. I've now added
this as testcase to linaro KVM CI, and wrote a small howto:
https://wiki.linaro.org/Core/Virtualization/HowTo/Arm32GuestOnAarch64
At least in my tests the VM returns "armv7l" for "uname -m", so this
is less confusing for build systems than chroots under Aarch64 where
linux32 personlity returns "armv8l".
Riku
