Hi all,
I wanted to start a discussion on defining a unified feature set that
makes it simpler for the different distros to support ARM systems using
u-boot. I have based a lot of my thoughts on how calxeda ship their
systems configured as it works fairly well, recently i sent in a patch
implementing most of what I would like to see for the wandboard[1]
right or wrong we want things to be simple for the user and to largely
look like a linux system on x86 would. The user and distro should never
need to worry about memory locations
so this would mean similar partitioning. i.e. /boot on ext4 root and
swap on lvm or as raw partitions. people should be able to have
just / on ext4 also. Down the road xfs /boot support would be a nice to
have.
pxe boot suport, two reasons, one is to be able to easily network boot
the distro installer, the other is to use sysboot support post install
with a extlinux.conf file to specify the kernel, initrd and bootargs
bootz and raw initrd support. not having to wrap kernels and initrds
really is a must. raw initrd support means that its much simpler for a
user to rebuild an initramfs if needed and bootz means we do not need
to worry about making sure that we specify the correct addresses to
load the kernel to when calling mkimage.
when it comes to memory addressing a distro and user shouldn't need to
know anything. Ideally u-boot will auto allocate addresses based on the
size of loaded objects. starting with a base address internal to u-boot
you load a kernel, when loading an initramfs u-boot automatically
calculates an address that ensures it does not overlap with the kernel.
same for a fdt if loaded. I say auto calculated because what we think
today will be enough room may not be tomorrow, dynamically calculating
gives the flexibility for whatever may come.
fdt will be automatically loaded and provided fedora ships dtbs
in /boot/dtb-<kernelver>/ I am not sure where other distros provide
them, however u-boot should automatically load dtb and provide access
to a fdt in a ${fdt_addr} variable that can be used by pxe_cmd but still
allows the user to specify their own in extlinux.conf if desired.
ideally the devicetree files need to be decoupled from the kernel,
along the lines of what is being discussed in the kernel now[2].
Distros should agree on a single location for the dtbs to be
placed. /boot/dtb/ or /boot/dtbs/ are probably good locations. u-boot
can then look in the path with and without /boot
output and input should happen on all possible devices and not just the
serial console. If a user has a trimslice with only a HDMI monitor and
usb keyboard they should be able to see the menu listing their possible
kernels and be able to choose which one they want to boot.
longer term being able to edit the boot arguments should also be an
option with the menu functionality.
"""
# extlinux.conf generated by anaconda
ui menu.c32
menu autoboot Welcome to Fedora. Automatic boot in # second{,s}. Press
a key for options. menu title Fedora Boot Options.
menu hidden
timeout 60
#totaltimeout 9000
label Fedora (3.9.9-302.fc19.armv7hl) 19 (Schrödinger’s Cat)
kernel /vmlinuz-3.9.9-302.fc19.armv7hl
append console=ttyAMA0,115200
root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet
LANG=en_US.UTF-8 initrd /initramfs-3.9.9-302.fc19.armv7hl.img
label Fedora (3.9.4-301.fc19.armv7hl) 19 (Schrödinger’s Cat)
kernel /vmlinuz-3.9.4-301.fc19.armv7hl
append console=ttyAMA0,115200
root=UUID=04f8c4be-2890-4d26-bb79-e5555060a142 ro rhgb quiet
LANG=en_US.UTF-8 initrd /initramfs-3.9.4-301.fc19.armv7hl.img """
the above is an example of a extlinux.conf file on a fedora 19 system
that works with the u-boot as shipped by calxeda on thier highbank
energy cards. some of the valid extlinux options written out by
anaconda cause noise on ARM as they are not implemented longer term it
would be good to deal with them correctly.
"""
Ignoring unknown command: ui
Ignoring malformed menu command: autoboot
Ignoring malformed menu command: hidden
Fedora Boot Options.
1: Fedora (3.9.9-302.fc19.armv7hl) 19 (Schrödinger’s Cat)
2: Fedora (3.9.4-301.fc19.armv7hl) 19 (Schrödinger’s Cat)
Enter choice:
"""
is the output on boot, the user can then enter 1 or 2 to select a
kernel to boot. if nothing is selected the first option is booted after
the timeout expires.
What this gets us in the end is that with a unified kernel, device tree
and u-boot implementing the system specific knowledge, supporting new
ARM systems is along the same lines as supporting new x86 systems. Get
the drivers and platform support in the mainline kernel and as long as
the Distro enables it it will just work in the next OS release. The one
part then that we as Distro's need to do is to ensure we install
u-boot into the disk correctly for the board in question if it is a
system that does not ship with some kind of storage for u-boot. If it
is a system with NOR, Nand, SPI storage for u-boot it is then entirely
in the vendors hand, and assuming that they implement everything, all
distros will be able to simply support them. Giving users and vendors
more choice in what to run on their hardware.
I may not be entirely clear here and there may be things I am missing,
I feel it is a good place to start the discussion.
Regards
Dennis
[1] http://lists.denx.de/pipermail/u-boot/2013-July/159522.html
[2] https://lwn.net/Articles/560523/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi all,
As part of the unified features in u-boot thread I said that I would
work to get distros to agree to a common location that u-boot will be
able to assume exists for it to transparently load dtbs from for us.
I would like to propose /boot/dtb/
I think it needs to be in /boot/ to allow for booting from systems
with / and /boot on separate partitions. I believe Ubuntu and hence
Debian put them in /lib somewhere today. Fedora puts them
in /boot/dtb-<kernel version>. im not sure where the other distros put
them.
I think that until they are decoupled from the kernel Fedora would
have /boot/dtb be a symlink to the latest installed kernel. once its
decoupled it would be the location where the dtbs are installed.
Can We all agree to implement ensuing that dtb files will be available
in /boot/dtb/
Regards
Dennis
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
iEYEARECAAYFAlIAG/wACgkQkSxm47BaWfcq2ACdFrw5/68I9+A2ls2htra3fMcN
rN4An1dlK4H8vm4jlJEEYcdp26tRDgJ9
=SXTs
-----END PGP SIGNATURE-----
Hi,
I'm trying to put together a summary of what distributions have hopped on the
Multiarch /lib/<triplet> bandwagon (or plan to), and for those who haven't,
what their solution is to dealing with various instruction sets, system call
interfaces, application binary interfaces, etc. on a single root filesystem.
Here's what I've gathered so far. Links to up-to-date documentation on the
subject, and any corrections or additions to my little list would be greatly
appreciated.
Ubuntu, Debian
Multiarch: /lib/<triplet>
Fedora, OpenSUSE
Multilib A: /lib, /lib64
Arch
Multilib B: /lib32, /lib
Gentoo
Multilib C: /lib32, /lib64, /libx32
OpenEmbedded
Uniarch A: /lib
Android:
Uniarch B: /system/lib
Also, any news on related LSB/FHS or other standardization efforts?
Thanks,
Christopher
--
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by the Linux Foundation.
This is to let you know that the migration of lists.linaro.org has been
successfully completed.
As per the email I sent on Wednesday, it may take some time for the new
address of the server to be seen by your computer. You can check this by
trying to connect to the web site:
http://lists.linaro.org/
If you are able to connect and you do not get an error, this means you are
connecting to the new server and you can send email to the lists.
If you experience any problems after the weekend and you find that you
still cannot connect to the server, please reply to this email to let us
know.
Regards
Philip
IT Services Manager
Linaro
Hello
You are receiving this email because you are subscribed to one or more
mailing lists provided by the lists.linaro.org server.
IT Services are announcing planned maintenance for this server scheduled
for *Friday 15th March 2013, starting at 2pm GMT*. The purpose of the work
is to move the service to another server. There will be some disruption
during this maintenance.
In order to ensure that you do not accidentally try to use the service
while it is being moved, the current server will be shut down at 2pm.
A further email will be sent on Friday afternoon to confirm that the
migration of the service is completed. However, due to the way servers are
found, it may take a while before your computer is able to connect to the
relocated service.
After the old server has been shut down, email sent to any of the lists
will be queued, but it is possible that the sending server will still
trying to deliver the email to the old server rather than the new one when
it is started.
It is therefore *strongly* recommended that you do not send any email to an
@lists.linaro.org email address until you can connect to the new service,
which you will be able to test by trying to use a web browser to connect to
http://lists.linaro.org after you receive the email confirming that the
migration has been completed. Since the old service will be shut down, if
you are able to connect, you can be sure you have connected to the new
service.
If by Monday you are still unable to connect to the service or you are not
able to send email to an @lists.linaro.org email address, please send an
email to its(a)linaro.org.
Thank you.
Regards
Philip
IT Services Manager
Linaro
State of the Debian/Ubuntu arm64 port
=====================================
*** Arm64 lives! ***
Executive summary
-----------------
* There is now a bootable (raring) image to download and run
* Everything has been rebuilt against glibc 2.17 so it works
* A bit more work is needed to make the rootfs useable as a native buildd
* Multiarch crossbuilding and the build-profile mechanism is mature enough to cross-build
a port from scratch (this is a big deal IMHO)
* All packages, sources and tools are in a public repo and this work should be reproducible.
* This image is fully multiarched so co-installing armhf for a
64/32 mix should work nicely, as should multiarch crossbuilding to
legacy x86 architectures. :-) (but I haven't tried that yet...)
* Linaro wants 'the distros' to take this work forward from here so people interested in
Debian and Ubuntu on 64-bit arm hardware need to step up and help out.
Bootable images
---------------
A milestone was reached this week: Enough packages were built for arm64 to debootstrap an
image which booted to a prompt! After a bit of fettling (and switching to multistrap) I got
an image with all the packages configured which boots with upstart to a login prompt (I
admit, I did get quite excited about this, as it represents the coming together of nearly 3
years work on multiarch, crossbuilding, bootstrapping, cyclic dependencies and arm64 :-)
The images are available for download: http://wiki.debian.org/Arm64Port#Pre-built_Rootfs
And there are destructions there for making your own.
All these packages were cross-built on raring, untangling cyclic dependencies with build
profiles (see wiki.debian.org/DebianBootstrap for how that works), making this the first
(non x86) self-bootstrapped debian port ever (so far as I know). All (?) previous ports have
been done using something else like OpenEmbedded (armel, armhf), RedHat/HardHat (arm, alpha,
mips), something IBMy (s390) to get an initial linux rootfs on which debian packages are
built.
The new bootstrap process is (almost) just a list of sbuild commands. In practice there are
still a few rough edges around cross- build-dependencies so of the 140 packages needed for
the bootstrap, 9 had to be built manually with 'dpkg-buildpackage -aarm64 -d' (to skip
build-dep checks) instead of 'sbuild --host arm64 <package>'.
The current bootstrap packageset status is here:
http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
There is no armv8 (arm64/aarch64) hardware available yet, so this image can currently only
be run in a model. ARM provide a free-beer prorietary 'Foundation model' so we do have
someting to test with. It's sluggish but perfectly useable. Booting the images takes a
couple of minutes on my fairly average machine.
The images are using the Linaro OE release kernels which seem to work fine for this purpose.
Thanks to Marcin for modified bootloader lines in .axf files.
Image status
------------
I was impressed that things basically 'just worked' on first boot. There is of course plenty
of breakage, I'm sure, and I haven't looked very hard yet, but it's a lot better than I
expected after months of just building stuff and testing nothing. (Things that are poorly:
nano can't parse it's own syntax-coluring files for example, and multiarch perl has the
wrong @INC path compiled in; I'm sure there is more). Consider this alpha-grade until it's
been used a bit more.
Things that are not yet built which would make the images a lot more useful are apt and a
dhcp client. apt needs gnupg needs curl needs nss. The nss cross-build needs fixing to
unbung that. A debian chroot without apt turns out to be disappointing quite quickly :-)
Expect an updated image with more packages very soon.
Multiarch crossbuilding
-----------------------
It's really nice to have building and crossbuilding using exactly the same mechanisms
and tools, with all files having one canonical location, and dependency mechanisms that
are reliable. The more I've used this, the more I've been impressed by it. There is
still work to do to expand the set of cross-buildable stuff, but it's a solid base to
work from.
Getting this port working has been 'interesting' because it's attempting 4 new things all at
once: multiarch (file layouts and dependencies), crossbuilding (tools and packaging support
in a distro that historically was always natively built), arm64 (aarch64) support in
packages that need it, and build-profiles to linearise the build-order.
The arm64 part of this is a relatively small part as the heavy lifting has been done
upstream (gcc, (e)glibc, binutils, kernel, libffi, autotools and a lot of minor fixes in
various packages). Thanks are due to doko (Matthias Klose) for sterling work getting all
that integrated into the debian and ubuntu toolchain packages, and infinity (Adam Conrad)
for merging various eglibc branches. There were also hordes of very boring patches of the
form 'update config.sub and guess before building'.
Most of the work has been in making things cross-build (exactly the same fixes needed for
armel/hf too so I've had plenty of help there from canonical types who want cross-building
for arm to work nicely), and particular thinks to Neil Williams for taking on the perl
cross-build challenge and creating the debian-perl-cross package to manage the
cross-configury, whilst also working with upstream to make the whole thing a bit less 1996.
Multiarchifying has been going on nicely in libraries and -dev packages, but things like
perl and python needed significant work, along with a lot of boring bugs saying 'mark this
package MA: foreign' and 'build-dep on python:any or perl-base:any'. Thanks are due to doko
for the python multiarching and Niko Tyni for the perl multiarchification. Getting all 3
'aspects' of multiarch perl, cross-built perl and arm64 perl config to work at the same time
was quite hard work, and there are still bugs there. Wider usage of multiarched perl would
no doubt sort this out reasonably quickly. I started a wiki page to track the status of
multiarched cross-buildable perl: http://wiki.debian.org/Multiarch/Perl . Help would be
welcome.
The build-profile work is described on the http://wiki.debian.org/DebianBootstrap page.
Progress has been greatly helped by GSOC projects last year, with good work on the tools
(crossbuild-essential packages, build-profile support) from P.J McDermott and an impressive
contribution from Johannes Schauer on dependency analysis tools around libdose, and apt
build-profile support.
All of this apart from multiarch perl, crossbuildable perl and build-profile stuff (and
a few pending patches) is already in raring.
Building stuff yourself
-----------------------
Setting up an arm64 build environment is very simple. Use sbuild-createchroot or mk-sbuild
and point at the bootstrap repo, with a bit of config and some updated tools packages from
the repo (amd64 only supplied). Details are given on
https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/arm64bootstrap
Once you've created a tarball chroot builds are simply done with
sbuild -c quantal-amd64-sbuild -d quantal --host=arm64 <package.dsc> or
sbuild -c quantal-amd64-sbuild -d quantal --host=arm64 <package>_<version> (I'd love it
if sbuild got smart enough to work out the version itself when given a distro - Roger
says he's working on it)
To deal with the chore of 'find version, run sbuild, sign result, upload to repo, import to
repo, deal with reprepro bitching if you re-upload the same version of something' for every
package build, I wrote 'dimstrap' which is a simple-minded tool to wrap that up and either do
one-off builds or run through a list. It is part of the xbuilder package here:
https://launchpad.net/~linaro-foundations/+archive/cross-build-tools/ It also includes the
logfile-parsing script ('generate html') which generates the nice status pages:
http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
Image building
--------------
The config and instructions provided (in
http://wiki.debian.org/Arm64Port#Building_your_own_rootfs_image ) is
for multistrap. Debootstrap sort-of produces working images too but
takes a lot longer to unpack/configure, and misses out various vital
packages (like libperl5.14). I'm sure it could be kicked into
submission. In theory multistrap (apt really) should have got all the
arch all packages from the main repo, but in practice it refused to do
that so I had to rebuild them or copy them over anyway (grumble).
Any package that installs replaced conffiles seems to generate invalid
dpkg status entries (ifupdown did this to me). I've not got to the
bottom of that yet. Deleting the offending line gets you an image that
works.
Issues
------
General:
The build-profile patches for dpkg and apt need to be pushed into the distro to make
that feature permanent. A thread on debian-devel is working on that
(http://debian.2.n7.nabble.com/Bootstrappable-Debian-proposal-of-needed-chan…).
The main issue is what syntax to use '<>' or '[]' and how to deal with multiple overlapping
profiles. The patches to debian control cannot go in until at least the syntax is agreed and
the tools will parse them without barfing. Johannes ands I will send an updated spec
soonish.
The missing piece of bootstrapping with regard to build-deps is packages that build-dep on
gcc-4.6 or binutils. When cross-building this should be satisfied by <triplet>-gcc-4.6 or
<triplet>-binutils. Nothing makes that happen currently. A scheme has been mooted but
nothing is implemented yet.
There is debate about whether cross-toolchains should build against multiarch libraries
(libgcc, libstdc++) like everything else, or have their own internal copies. Doko and I
disagree on this matter. That will need to be worked out at some point.
We won't get that much further with fixing cross- object-introspection, which is a
non-trivial job.
Image-related:
The images do essentially work but very little has been tested so far.
Multiarch perl still needs work.
nss needs cross-building in order to get apt cross-built
I've not got networking working yet. Info is here:
https://fedoraproject.org/wiki/Architectures/ARM/AArch64/FoundationModel_Ne…
lack of a dhcp client in the image hasn't helped there.
More info
---------
The canonical arm64 port info page is:
http://wiki.debian.org/Arm64Port
Full arm64 cross-build status (i.e everything that has been tried) is here:
http://people.linaro.org/~wookey/buildd/raring-arm64/status.html
All the patches generated so far are here:
http://people.debian.org/~wookey/bootstrap/patches/
(most that can, have been filed as bugs - there is a backlog of stuff
filed in Launchpad but not yet forwarded to the Debian BTS - yes I am
a bad boy - blame the fact that you can't use reportbug or bts from
inside ARM due to their idiotic email policies).
Future work
-----------
Firstly we should say thank you to Linaro for sponsoring this work in various ways over
the last 3 years. We wouldn't be at this point now if it wasn't for that. However
Linaro has a lot of things to do and is trying hard not to do distro's work for them,
concentrating on upstream things. This makes sense for commercially-backed distros like
Red Hat and Ubuntu, but rather less for Debian where we _are_ the distro just as much
as anyone else is, and ultimately someone has to spend the time to get stuff working.
Anyway, I was supposed to stop work on this some time back, but have largely failed to
do so (cross-building is so moreish - there is always one more build to try before
bedtime!) and appreciate being given enough slack to get this to a point of actual
utility. However I expect to have much less time to spend on this from now on, except
insofar as it still co-oncides with things Linaro wants doing. I'd love to hear from
people who actually want to use this, to get more packages built, the Debian
cross-toolchains sorted, build-profiles finalised, and a whole pile of stuff fixed once
Wheezy is released. I'm pretty sure there are quite a lot of people who want multiarch
Debian or Ubuntu on their arm64 machines (or models).
I hear rumours that actual hardware may appear sometime around the middle of the year
with some bagsied for Debian. Setting up the ports infrastructure for that would be
good. I don't know if anyone is interested in building slowly on models in the
meantime, or if we should just carry on crossing and see how far we get. This table
shows that 471 packages in raring can be expected to cross-build already:
http://people.canonical.com/~cjwatson/cross/armhf/raring/
Todo:
Fix up multiarch/cross perl
Fix nss
Build missing packages for apt
Build missing packages for build-essential
Build Debian cross-toolchain
Get all this working in unstable as well as raring
Setup buildds
Build all the other packages
Set up automated bootstraping runs (eventually)
Current setup
-------------
Builds have all been run locally using the sbuild/chroot setup described above and on
the Arm64Port page, which should be easy for anyone to reproduce. The main irritation
is keeping up with raring: out of sync libraries are not MA-installable. Logs are
uploaded to people.linaro.org (rsync). The reprepro repo is on people.debian.org
(dupload). This stuff should probably move to ports.debian.org and ports.ubuntu.com,
but neither of those are set up for cross-building so I'm not quite sure how this will
work.
I could go on at great length about the machinery of profiled bootstrap builds, and
interactions between tools, but it's not very exciting, so will resist. Suffice it to
say that whilst it's all pretty slick I'd still like better buildd tools.
Build-profile changes
---------------------
The build-profile patches are not yet upstreamable so are collecting in the repo.
The patch set so far is here: http://people.debian.org/~wookey/bootstrap/patches/profiles/packages/
Other thanks:
Other people who have helped make this happen in various ways but not got a mention above:
Colin Watson, Dmitry Ledkovs, Steve Langasek, Harry Leibel, Thibaut Girka, Roger Leigh,
Marcus Shawcroft, James Morrisey, Jonathan Austin, Steve McIntyre, Peter Pearse, Aurelien
Jarno, and whoever does sysadmin at people.{linaro,debian}.org
I hope I didn't forget anyone, or any important information.
Feedback from anyone attempting to get this working outside my computer is very
welcome. I have almost certainly forgotten to write down some things, and upload
correct versions of some other things.
Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/
Hi cross-distro readers,
the openSUSE on ARM team was quite busy the last few weeks with
getting openSUSE 12.3 for AArch64 ready. At the time of this post, we
have finished around 4000-4100 packages (out of ~ 6000) of the current
state of the openSUSE 12.3 project for AArch64. With those
successfully built packages, we’re also able to build a regular
openSUSE image for you to try and run in the ARMv8 System emulator
(ARMv8 Foundation Model).
This is a huge achievement and milestone for us, thanks to lots of
helpful hands in openSUSE. Just to put this into context: This is not
a minimal system with a couple of toolchain packages. It is also not
an embedded variant of a Linux environment. No, this is the full
featured, standard openSUSE distribution as you’re used to, ported to
AArch64, up and running. We have built it based on (slightly newer
versions of) standard openSUSE 12.3 packages, and the changes are
mostly already merged back into openSUSE Factory. For all we know it’s
also more successful package builds than any other Linux distribution
has on AArch64! If you’d like to see the status yourself, please check
out the OBS repository we created for this [1].
As an open distribution, it is important to make contributions easy
and we worked hard to enable others to participate in our effort. We
extended OBS (the Open Build Service) to automatically spawn a
Foundation Model virtual machine when you want to build for aarch64.
This works remotely on the OBS server as well as locally using osc
build. Building for AArch64 is therefore as easy as building for any
other architecture, and it feels native to all who are familiar with
the OBS. More information on this is available on the respective wiki
page [2].
So, dive right into it: Get the image and start with openSUSE on
AArch64 by following our wiki page:
https://en.opensuse.org/Portal:ARM/AArch64.
Please contact the openSUSE Team at opensuse-arm(a)opensuse.org
Greetings,
Dirk (openSUSE ARM Team)
[1] https://build.opensuse.org/project/show?project=devel:ARM:AArch64:12.3
[2] https://en.opensuse.org/Portal:ARM/AArch64
Hi Folks,
I propose the following triplet for Fedora on AArch64:
aarch64-redhat-linux-gnu
Skipping the vendor part, that means generally:
aarch64-linux-gnu
There should be no reason to deviate from this. I'm putting it out there
now because I don't want another ARMv7 experience later :)
Jon.
Meant to reply to the list.
Begin forwarded message:
> From: Jon Masters <jonathan(a)jonmasters.org>
> Subject: Re: AArch64 triplet
> Date: November 22, 2012 3:32:45 AM EST
> To: Mike Frysinger <vapier(a)gentoo.org>
>
>
> On Nov 20, 2012, at 1:27 PM, Mike Frysinger <vapier(a)gentoo.org> wrote:
>
>> On Tuesday 20 November 2012 03:25:56 Jon Masters wrote:
>>> The only reason for making a change at this time appears to be cosmetic,
>>> for removing /lib for example. I can understand that, and if we were
>>> discussing this a year ago (or even months ago when I first raised it on
>>> this list), then it might be a reasonable change, but at this time I
>>> cannot find an overwhelming technical justification.
>>
>> i don't think removal of /lib/ is really feasible. that's the path that gets
>> used for firmware (/lib/firmware/) and kernel modules (/lib/modules/<kver>/)
>> regardless of the default ABI on the system.
>>
>> similarly, userspace packages are using that path for supplemental files like
>> the bootloader (grub) or ABI-independent settings (udev rules).
>
> Sorry for the confusion. By "getting rid" what I mean is not having libraries in there. As is obvious, there will always be plenty of stuff in there, and it's mandated by standards anyway. So, really, if it's down to whether we (Fedora) have one thing in /lib that could be in /lib64 vs. not then I would rather just keep the dynamic linker where it is in /lib and move on with life.
>
> By the way, we announced our initial Fedora bootstrap work tonight. The wiki has a lot of information about what is currently being done, along with initial images that will grow:
>
> https://fedoraproject.org/wiki/Architectures/ARM/AArch64
>
> Jon.
>
