[ Please forward this to anyone else who should get a copy - and ask them to signup to cross-distro@linaro because that's where we'll talk ]
Folks,
At the 2011 Linux Plumbers Conference in Santa Rosa, we held an ARM minisummit to discuss a few cross-distribution issues surrounding the "hard float" ABI for v7, and the ARM device and OS ecosystem in general. This mail is intended to summarize the key actions from that meeting.
1). We discussed some of the different distributions (Debian, ChromeOS, Fedora, MeeGo, Mozilla, Ubuntu, etc.). We all agree on a common notion of what "hard float" means: it's the ARM AAPCS hardware floating point stuff in section 6, with the -d16 register set of the VFPv3 being used. Although there is some possibility to standardize v6, we decided we are interested only in the present and future at this point (and that's v7).
Some use Thumb2, some don't, but interworking allows for that (we in Fedora accidentally built a few packages with Thumb2 and will be undoing that without users even noticing). NEON is similarly an optional feature (supported by hwcaps) and not a required part of the core ABI. So it's really just the ABI that we all care about, and we all agree on it.
For toolchain folks, this means we all agree on the following:
--with-float=hard --with-fpu=vfpv3-d16 --with-abi=aapcs-linux
(some of us may target different optimizations, etc. of course but that is down to an application to decide whether it cares only about Cortex+)
2). We agreed on the importance of cross-distro binary compatibility. It seems a number of distributions are excited by Debian's multi-arch approach (as am I), but not all of us will be moving to it any time soon. In the shorter term, if the runtime linker will have a special path, we can work around that with symlinks, but it's a kludge. Our preferred goal (from the meeting) is that everyone use this target:
arm-linux-gnueabihf
(Red Hat toolchains will include the vendor, but it is dropped by autotools when actually generating the paths and comparisons). I will advocate for Fedora ARM to switch to this target when possible, and in the interim will try to ensure we can establish some symlinks for the ability to run binaries compiled against Debian and Ubuntu on Fedora. I don't know what our timeline will be for this, but it's not something that's really landed everywhere yet. We at least agree on paths now!
3). We agree that it is a good idea to do LSB. Adam Conrad (Canonical) and I agreed to represent community interests in driving this. We do know that we'll be preferring OpenGL ES 2.0 over OpenGL (same namespace within the library means one default choice is required). There will be many other things :) We agree that the main benefit to pursuing LSB is that it promotes the idea of working together and being compatible. It may be that we don't have the same level of compliance on ARM, but some level of conformance to some common standard(s) is a good start! :)
4). We decided that we should have a standing Cross Distro meeting at Linaro Connect events, beginning with the next one in November. We will also facilitate a "show and tell" for apps at Connect, and have an opportunity to demo different distributions. We will add a specific session at the next Connect to continue this discussion.
Finally, we agreed we should keep the conversation going. I apologize for being a little tardy in sending this mail, but I want to kickstart further discussion and get this ball rolling again. I'll ping Adam about LSB (and thanks again to Jesse for the deep dive on GL). Meanwhile, David can grab ownership of organizing the Cross Distro meeting at Connect (I would like to assist). Anyone else got actions?
Jon.
2011/9/26 Jon Masters jcm@redhat.com:
Hi, some feedback,
Folks,
I have been doing some work for a Mandriva armv7 port, and I think some feedback is worth...
At the 2011 Linux Plumbers Conference in Santa Rosa, we held an ARM minisummit to discuss a few cross-distribution issues surrounding the "hard float" ABI for v7, and the ARM device and OS ecosystem in general. This mail is intended to summarize the key actions from that meeting.
1). We discussed some of the different distributions (Debian, ChromeOS, Fedora, MeeGo, Mozilla, Ubuntu, etc.). We all agree on a common notion of what "hard float" means: it's the ARM AAPCS hardware floating point stuff in section 6, with the -d16 register set of the VFPv3 being used. Although there is some possibility to standardize v6, we decided we are interested only in the present and future at this point (and that's v7).
Some use Thumb2, some don't, but interworking allows for that (we in Fedora accidentally built a few packages with Thumb2 and will be undoing that without users even noticing). NEON is similarly an optional feature (supported by hwcaps) and not a required part of the core ABI. So it's really just the ABI that we all care about, and we all agree on it.
For toolchain folks, this means we all agree on the following:
--with-float=hard --with-fpu=vfpv3-d16 --with-abi=aapcs-linux
Currently, for mandriva I am using this:
[root@panda2 /]# gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/armv7l-mandriva-linux-gnueabi/4.6.1/lto-wrapper Target: armv7l-mandriva-linux-gnueabi Configured with: ./configure --build=armv7l-mandriva-linux-gnueabi --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib --libexecdir=/usr/lib --localstatedir=/var --sharedstatedir=/usr/com --mandir=/usr/share/man --infodir=/usr/share/info --x-includes=/usr/include --x-libraries=/usr/lib --disable-libjava-multilib --with-java-home=/usr/lib/jvm/java-rpmbuild --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-java-awt=gtk --enable-gtk-cairo --with-cloog --with-ppl --enable-cloog-backend=ppl --disable-libquadmath --disable-libquadmath-support --disable-libssp --disable-libunwind-exceptions --disable-werror --enable-__cxa_atexit --enable-bootstrap --enable-checking=release --enable-gnu-unique-object --enable-languages=c,c++,fortran,java,lto --enable-linker-build-id --enable-plugin --enable-shared --enable-threads=posix --with-system-zlib --with-bugurl=https://qa.mandriva.com/ --with-cpu=cortex-a8 --with-tune=cortex-a8 --with-arch=armv7-a --with-mode=thumb --with-float=softfp --with-fpu=vfpv3-d16 --with-abi=aapcs-linux --host=armv7l-mandriva-linux-gnueabi --target=armv7l-mandriva-linux-gnueabi Thread model: posix gcc version 4.6.1 20110916 (Mandriva) (GCC)
most packages built on armv7 (panda board), but for now using softfp abi, that is, it can "automagically" run armv5 (or earlier) binaries due to using a compatible abi, while still using float registers.
other hopefully useful information:
[root@panda2 stage2]# rpm -q glibc glibc-2.14.90-4-mdv2012.0.armv7l
[root@panda2 stage2]# ld --version GNU gold (Linux/GNU Binutils 2.21.53.0.1.20110716) 1.11 Copyright 2011 Free Software Foundation, Inc. This program is free software; you may redistribute it under the terms of the GNU General Public License version 3 or (at your option) a later version. This program has absolutely no warranty.
Currently I am working with ssh, on a chroot hosted on fedora current work on armv7 and hardfp abi.
(some of us may target different optimizations, etc. of course but that is down to an application to decide whether it cares only about Cortex+)
2). We agreed on the importance of cross-distro binary compatibility. It seems a number of distributions are excited by Debian's multi-arch approach (as am I), but not all of us will be moving to it any time soon. In the shorter term, if the runtime linker will have a special path, we can work around that with symlinks, but it's a kludge. Our preferred goal (from the meeting) is that everyone use this target:
arm-linux-gnueabihf
Call me a troll or whatever you want :-) But for now I am using armv7l-mandriva-linux-gnueabi, mostly because for now I am willing to run armv5 binaries, and would prefer, and considering to, call it just armv7* because I doubt anyone in short term is doing a big endian port, and if doing leave form them adding the "b"...
(Red Hat toolchains will include the vendor, but it is dropped by autotools when actually generating the paths and comparisons). I will advocate for Fedora ARM to switch to this target when possible, and in the interim will try to ensure we can establish some symlinks for the ability to run binaries compiled against Debian and Ubuntu on Fedora. I don't know what our timeline will be for this, but it's not something that's really landed everywhere yet. We at least agree on paths now!
3). We agree that it is a good idea to do LSB. Adam Conrad (Canonical) and I agreed to represent community interests in driving this. We do know that we'll be preferring OpenGL ES 2.0 over OpenGL (same namespace within the library means one default choice is required). There will be many other things :) We agree that the main benefit to pursuing LSB is that it promotes the idea of working together and being compatible. It may be that we don't have the same level of compliance on ARM, but some level of conformance to some common standard(s) is a good start! :)
4). We decided that we should have a standing Cross Distro meeting at Linaro Connect events, beginning with the next one in November. We will also facilitate a "show and tell" for apps at Connect, and have an opportunity to demo different distributions. We will add a specific session at the next Connect to continue this discussion.
Finally, we agreed we should keep the conversation going. I apologize for being a little tardy in sending this mail, but I want to kickstart further discussion and get this ball rolling again. I'll ping Adam about LSB (and thanks again to Jesse for the deep dive on GL). Meanwhile, David can grab ownership of organizing the Cross Distro meeting at Connect (I would like to assist). Anyone else got actions?
More some information on my work on Mandriva, currently a lesser, but somehow active participant, but not really talking on Mandriva behalf, or any legality that may apply :-) Currently most of what I did for Mandriva had as starting point a customized DJDelorie's bootstrap scripts, and so far, the only armv5 rpm binaries I am using are some java-1.6.0-openjdk mandriva 2009.1 rpms I found on the net, a mirror of 2009.1 on russia, that I am using to attempt to get a working java. Call it misscommunication of some kind, that were from an initial work on port of mandriva to armv5, and are now, afaik being continued on Mageia.
Jon.
Thanks, Paulo
On Mon, Sep 26, 2011 at 8:37 PM, Jon Masters jcm@redhat.com wrote:
[ Please forward this to anyone else who should get a copy - and ask them to signup to cross-distro@linaro because that's where we'll talk ]
Folks,
At the 2011 Linux Plumbers Conference in Santa Rosa, we held an ARM minisummit to discuss a few cross-distribution issues surrounding the "hard float" ABI for v7, and the ARM device and OS ecosystem in general. This mail is intended to summarize the key actions from that meeting.
Thanks very much Jon and others for the work towards consensus and a bit of standardization. I'm certain this will help going forward.
Thanks also everyone for the communication on this list, I find it very valuable.
I'd very much like to contribute to the LSB testing on ARM and I'll keep an eye out on this list to see if there is more information on how that work will be organized. (I think Mats Wichman at the Linux Foundation knows a bit more so it ought to be coordinated with him going forward perhaps.)
Regards,
Jeremiah
On Mon, Oct 03, 2011 at 12:33:04PM +0200, Jeremiah Foster wrote:
On Mon, Sep 26, 2011 at 8:37 PM, Jon Masters jcm@redhat.com wrote:
[ Please forward this to anyone else who should get a copy - and ask them to signup to cross-distro@linaro because that's where we'll talk ]
Folks,
At the 2011 Linux Plumbers Conference in Santa Rosa, we held an ARM minisummit to discuss a few cross-distribution issues surrounding the "hard float" ABI for v7, and the ARM device and OS ecosystem in general. This mail is intended to summarize the key actions from that meeting.
Thanks very much Jon and others for the work towards consensus and a bit of standardization. I'm certain this will help going forward.
Thanks also everyone for the communication on this list, I find it very valuable.
I'd very much like to contribute to the LSB testing on ARM and I'll keep an eye out on this list to see if there is more information on how that work will be organized. (I think Mats Wichman at the Linux Foundation knows a bit more so it ought to be coordinated with him going forward perhaps.)
Jon and I have both been talking to Mats and Jeff about LSB already, but things have gone a little quiet while we've been concentrating on bringup in the last few months. They would definitely be the right people to work with going forwards...
Cheers,
2011/10/3 Steve McIntyre steve.mcintyre@linaro.org:
On Mon, Oct 03, 2011 at 12:33:04PM +0200, Jeremiah Foster wrote:
On Mon, Sep 26, 2011 at 8:37 PM, Jon Masters jcm@redhat.com wrote:
[ Please forward this to anyone else who should get a copy - and ask them to signup to cross-distro@linaro because that's where we'll talk ]
Folks,
At the 2011 Linux Plumbers Conference in Santa Rosa, we held an ARM minisummit to discuss a few cross-distribution issues surrounding the "hard float" ABI for v7, and the ARM device and OS ecosystem in general. This mail is intended to summarize the key actions from that meeting.
Thanks very much Jon and others for the work towards consensus and a bit of standardization. I'm certain this will help going forward.
Thanks also everyone for the communication on this list, I find it very valuable.
I'd very much like to contribute to the LSB testing on ARM and I'll keep an eye out on this list to see if there is more information on how that work will be organized. (I think Mats Wichman at the Linux Foundation knows a bit more so it ought to be coordinated with him going forward perhaps.)
I think I am in the "free software zealot" position here :-)
So, a few minutes ago I was pointed to http://wiki.meego.com/Hardware-accelerated_graphics_on_Pandaboard_using_MeeG... that points to http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0041c/Babdhhd...
or for some related information http://groups.google.com/group/pandaboard/browse_thread/thread/cd69411c1cb60...
For now I have been only considering gcc as compiler, and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
/fpregargs Floating-point arguments are passed in floating-point registers. This option is not available for in tcc or tcpp, and is the default for ARM if /hardfp is selected. /nofpregargs Floating-point arguments are not passed in floating-point registers. This option is not available in tcc or tcpp. This option is obsolete and is available for backwards compatibility only.
Note, that "tcc" issues is news to me, as so far I was only considering gcc and its optimizations that make breaking the abi pretty much nonsense, but was not considering an extra compiler, that may be being used to build external, non open source libraries, e.g. libGLes*.
Jon and I have both been talking to Mats and Jeff about LSB already, but things have gone a little quiet while we've been concentrating on bringup in the last few months. They would definitely be the right people to work with going forwards...
Cheers,
Steve McIntyre steve.mcintyre@linaro.org http://www.linaro.org/ Linaro.org | Open source software for ARM SoCs
Paulo
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
So, it's true that we're basically saying as far as we believe Linux distributions go in the ARMv7 timeframe, we assume hardfloat and will all be using the newer ABI. Is that what you meant?
Jon and I have both been talking to Mats and Jeff about LSB already, but things have gone a little quiet while we've been concentrating on bringup in the last few months. They would definitely be the right people to work with going forwards...
So...Adam, we need to sync up on this. I would like to have made a little progress by the time we get to Connect next month.
Jon.
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
So, it's true that we're basically saying as far as we believe Linux distributions go in the ARMv7 timeframe, we assume hardfloat and will all be using the newer ABI. Is that what you meant?
Absolutely not agreeing :-) I have spent some time packaging, and got by myself over 3k Mandriva packages built for softfp, but it is mostly for "clearing the path", and knowing what is ahead; the same way I did go down all the way cross compiling several bootstraps, and starting building rpm, and then a few rpms for all toolchain setups.
One of the major problems I am seeing on this packaging work is because I did choose thumb2 (space saving and compatibility are worth it), so far I did either fetch linaro patches, tweak options, add -marm to CFLAGS and CXXFLAGS, etc, as most packages around expect arm instruction set, and few handle thumb2. There were a few other tricks, like some (very uncommon) code expecting "char" to be always "signed".
Hardfp abi adds a completely new set of incompatibility in the field, that should affect asm in different places, break pretty much any jit/ffi around, does not run armv5 or earlier binaries, and all to have what? Like 3% (with luck) increase in performance in fpu intensive applications, while almost everything else has no benefits. From my understanding, it was wrong from start, and most likely caused by comparing apples to oranges, err, software float vs hardware float, probably also the software float one not using armv6 instructions, not "softfp abi" vs "hardfp abi".
Jon and I have both been talking to Mats and Jeff about LSB already, but things have gone a little quiet while we've been concentrating on bringup in the last few months. They would definitely be the right people to work with going forwards...
So...Adam, we need to sync up on this. I would like to have made a little progress by the time we get to Connect next month.
Jon.
Paulo
On Thu, 2011-10-06 at 15:20 -0300, Paulo César Pereira de Andrade wrote:
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
Yup. We disagree :)
I believe incompatible ABI changes are painful, but occasionally (very occasionally, and with some planning, co-ordination, justification, and a planned execution) can be necessary. The hard float ABI brings enough benefits that the various distributions are standardizing on it for v7. I would hope that we can all get behind this and treat v7 as a new architecture rather than simply an iteration, but I understand if you don't want to do that. Having said that, we do want to do that :)
and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
So, it's true that we're basically saying as far as we believe Linux distributions go in the ARMv7 timeframe, we assume hardfloat and will all be using the newer ABI. Is that what you meant?
Absolutely not agreeing :-) I have spent some time packaging, and got by myself over 3k Mandriva packages built for softfp, but it is mostly for "clearing the path", and knowing what is ahead; the same way I did go down all the way cross compiling several bootstraps, and starting building rpm, and then a few rpms for all toolchain setups.
I understand that you want a forward upgrade path from v5/v6 onwards. If you have a substantive userbase, it might make sense to do "softfp" v7. In our case, we consider it a better use of our (limited) resources to keep a separate v5 build of Fedora around for those on older systems and to build a new v7 version of the distribution for future use. It might be nice to support both ABIs at the same time, but we don't right now.
As far as I know, the vast majority of us are doing hard float on v7. I would like to know who else is in your position, so we can work together to make the experience reasonably good for everyone. You're not the only one doing something different. I just noticed, for example, that Sun's Java bits claim to be "hard float" but they think that means VFPv2 :(
(can we *please* make a collective mental note that we won't take silly terms like "hard float" in the future unless it's abundantly clear these won't be repurposed later on? Hard float should mean precisely one thing. If that is not trusted to be the case, we need a new term).
One of the major problems I am seeing on this packaging work is because I did choose thumb2 (space saving and compatibility are worth it), so far I did either fetch linaro patches, tweak options, add -marm to CFLAGS and CXXFLAGS, etc, as most packages around expect arm instruction set, and few handle thumb2. There were a few other tricks, like some (very uncommon) code expecting "char" to be always "signed".
Fortunately, you can switch in and out, even on a per-package basis.
Hardfp abi adds a completely new set of incompatibility in the field, that should affect asm in different places, break pretty much any jit/ffi around, does not run armv5 or earlier binaries, and all to have what? Like 3% (with luck) increase in performance in fpu intensive applications, while almost everything else has no benefits. From my understanding, it was wrong from start, and most likely caused by comparing apples to oranges, err, software float vs hardware float, probably also the software float one not using armv6 instructions, not "softfp abi" vs "hardfp abi".
I understand what you're saying. The view on our end was that this was effectively a new architecture we could target so we would treat it as such, and in so doing we might as well move to the newer ABI if we're not intending v7 to be backward compatible with v5.
Jon.
Em 6 de outubro de 2011 15:41, Jon Masters jcm@redhat.com escreveu:
On Thu, 2011-10-06 at 15:20 -0300, Paulo César Pereira de Andrade wrote:
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
Yup. We disagree :)
I believe incompatible ABI changes are painful, but occasionally (very occasionally, and with some planning, co-ordination, justification, and a planned execution) can be necessary. The hard float ABI brings enough benefits that the various distributions are standardizing on it for v7. I would hope that we can all get behind this and treat v7 as a new architecture rather than simply an iteration, but I understand if you don't want to do that. Having said that, we do want to do that :)
I understand that the arm ecosystem is not like x86; imagine if suddenly distros did choose to use regparm=n, sseregparm=n, well most distros already expect i686/sse, but do not use sse due to abi constraints.
I think the point here is the "bring enough benefits" that we are not agreeing. Personally, what I am afraid is that who will decide this is not distros, but companies providing binary blobs (e.g. video drivers). Talking again about x86 example, currently Mandriva Brasil makes several OEM images, and for most of them, besides x86_64, we need to make 32 bit images due to binary blobs.
About the abi, for most things it should be ok, but on the jit/ffi side, usually one should not expect any reliability with varargs functions, but printf is so cute :-) Problem is that the hardfp abi uses softfp abi to pass doubles as vararg function arguments.
and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
So, it's true that we're basically saying as far as we believe Linux distributions go in the ARMv7 timeframe, we assume hardfloat and will all be using the newer ABI. Is that what you meant?
Absolutely not agreeing :-) I have spent some time packaging, and got by myself over 3k Mandriva packages built for softfp, but it is mostly for "clearing the path", and knowing what is ahead; the same way I did go down all the way cross compiling several bootstraps, and starting building rpm, and then a few rpms for all toolchain setups.
I understand that you want a forward upgrade path from v5/v6 onwards. If you have a substantive userbase, it might make sense to do "softfp" v7. In our case, we consider it a better use of our (limited) resources to keep a separate v5 build of Fedora around for those on older systems and to build a new v7 version of the distribution for future use. It might be nice to support both ABIs at the same time, but we don't right now.
As far as I know, the vast majority of us are doing hard float on v7. I would like to know who else is in your position, so we can work together to make the experience reasonably good for everyone. You're not the only one doing something different. I just noticed, for example, that Sun's Java bits claim to be "hard float" but they think that means VFPv2 :(
(can we *please* make a collective mental note that we won't take silly terms like "hard float" in the future unless it's abundantly clear these won't be repurposed later on? Hard float should mean precisely one thing. If that is not trusted to be the case, we need a new term).
Lets call it Hardware Freedom :-)
One of the major problems I am seeing on this packaging work is because I did choose thumb2 (space saving and compatibility are worth it), so far I did either fetch linaro patches, tweak options, add -marm to CFLAGS and CXXFLAGS, etc, as most packages around expect arm instruction set, and few handle thumb2. There were a few other tricks, like some (very uncommon) code expecting "char" to be always "signed".
Fortunately, you can switch in and out, even on a per-package basis.
Unlike the abi :-)
Hardfp abi adds a completely new set of incompatibility in the field, that should affect asm in different places, break pretty much any jit/ffi around, does not run armv5 or earlier binaries, and all to have what? Like 3% (with luck) increase in performance in fpu intensive applications, while almost everything else has no benefits. From my understanding, it was wrong from start, and most likely caused by comparing apples to oranges, err, software float vs hardware float, probably also the software float one not using armv6 instructions, not "softfp abi" vs "hardfp abi".
I understand what you're saying. The view on our end was that this was effectively a new architecture we could target so we would treat it as such, and in so doing we might as well move to the newer ABI if we're not intending v7 to be backward compatible with v5.
Jon.
Paulo
Am 06.10.2011 um 22:29 schrieb Paulo César Pereira de Andrade paulo.cesar.pereira.de.andrade@gmail.com:
Em 6 de outubro de 2011 15:41, Jon Masters jcm@redhat.com escreveu:
On Thu, 2011-10-06 at 15:20 -0300, Paulo César Pereira de Andrade wrote:
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
Yup. We disagree :)
I believe incompatible ABI changes are painful, but occasionally (very occasionally, and with some planning, co-ordination, justification, and a planned execution) can be necessary. The hard float ABI brings enough benefits that the various distributions are standardizing on it for v7. I would hope that we can all get behind this and treat v7 as a new architecture rather than simply an iteration, but I understand if you don't want to do that. Having said that, we do want to do that :)
I understand that the arm ecosystem is not like x86; imagine if suddenly distros did choose to use regparm=n, sseregparm=n, well most distros already expect i686/sse, but do not use sse due to abi constraints.
However, nobody compiles x86 binaries without tls support or for < i586 today anymore either. Armv7 seems like a nice time for a clean cut.
I think the point here is the "bring enough benefits" that we are not agreeing. Personally, what I am afraid is that who will decide this is not distros, but companies providing binary blobs (e.g. video drivers).
There are two pieces to the puzzle here:
1) binary video drivers
I don't want them. If you really have to support companies that don't want to work with us in reasonable ways, just put your X server in a chroot for whatever abi/environment they want and call it a day
2) binary application software
Right now there is none. Period. Maybe ARM will get enough market share one day to get ISVs to actually develop for ARM. By then they will check what distros use abi-wise and go with that. If that's hardfp then it's hardfp and everyone's happy.
Alex
On Fri, 2011-10-07 at 00:40 +0200, Alexander Graf wrote:
Am 06.10.2011 um 22:29 schrieb Paulo César Pereira de Andrade paulo.cesar.pereira.de.andrade@gmail.com:
Em 6 de outubro de 2011 15:41, Jon Masters jcm@redhat.com escreveu:
On Thu, 2011-10-06 at 15:20 -0300, Paulo César Pereira de Andrade wrote:
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
Yup. We disagree :)
I believe incompatible ABI changes are painful, but occasionally (very occasionally, and with some planning, co-ordination, justification, and a planned execution) can be necessary. The hard float ABI brings enough benefits that the various distributions are standardizing on it for v7. I would hope that we can all get behind this and treat v7 as a new architecture rather than simply an iteration, but I understand if you don't want to do that. Having said that, we do want to do that :)
I understand that the arm ecosystem is not like x86; imagine if suddenly distros did choose to use regparm=n, sseregparm=n, well most distros already expect i686/sse, but do not use sse due to abi constraints.
It's important to also realize that general purpose ARM systems are a new area, and so we have an opportunity to make some changes. Nobody is going to change the x64 ABI (ok, so x32, but it's not going to be widely adopted by many for this exact reason) because it's so well established with end users. Conversely, at least in the case of Fedora, there is still an opportunity to fix a lot of things while our main user base is very technically inclined and willing to adapt. In the case of Canonical, and others thus far, a lot of vertical integration is happening and so there is still opportunity to shift future systems even in those cases because existing users will stay on older software bases.
In the end there will be many more things we don't like, but hopefully it will be a few decades before we get to that point. I'm all for standards and not throwing things away - once we get to that critical mass beyond which it doesn't make sense to start things over - but I don't think we've reached that point and I want to clean up now :)
However, nobody compiles x86 binaries without tls support or for < i586 today anymore either. Armv7 seems like a nice time for a clean cut.
Exactly. We've done things like that before in x86. Of course it's a little different when no existing software runs afterward (except in the case that you have multi-arch, which I'll mention as a kudos to Debian and friends for doing there). So the x86 equivalent would be more like promoting x32 for widespread use, which would not make sense if we were at a similar level of general purpose adoption in the ARM ecosystem.
I think the point here is the "bring enough benefits" that we are not agreeing. Personally, what I am afraid is that who will decide this is not distros, but companies providing binary blobs (e.g. video drivers).
There are two pieces to the puzzle here:
- binary video drivers
I don't want them. If you really have to support companies that don't want to work with us in reasonable ways, just put your X server in a chroot for whatever abi/environment they want and call it a day
I want to be a bit more pragmatic than that :) However, I think since we're mostly moving to the same ARMv7 base with the same ABI, we have enough mass that there will be new drivers built for the userspace.
- binary application software
Right now there is none. Period. Maybe ARM will get enough market share one day to get ISVs to actually develop for ARM. By then they will check what distros use abi-wise and go with that.
Indeed. I think we will get there, which is why I want to fix things now so that we only have to live with whatever kludges and hacks are developed years from now after it's too late ;) We can at least do cross-distro standards, get LSB rolling, things of that nature.
If that's hardfp then it's hardfp and everyone's happy.
Right. I think one thing we agreed at LPC is that enough of us agree on hardfp that we've got momentum there. Others are free to disagree and go their own way. I'd like it if we'd continue to build consensus as a community around the need to standardize, so that the next time there are big decisions to make even more of us can agree :)
Jon.
On Thu, Oct 06, 2011 at 03:20:16PM -0300, Paulo César Pereira de Andrade wrote:
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
fyi, I found that this page contains a pretty good explanation of why we ended up where we are (at least from Debian's POV): http://wiki.debian.org/ArmHardFloatPort
and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
So, it's true that we're basically saying as far as we believe Linux distributions go in the ARMv7 timeframe, we assume hardfloat and will all be using the newer ABI. Is that what you meant?
Absolutely not agreeing :-) I have spent some time packaging, and got by myself over 3k Mandriva packages built for softfp, but it is mostly for "clearing the path", and knowing what is ahead; the same way I did go down all the way cross compiling several bootstraps, and starting building rpm, and then a few rpms for all toolchain setups.
One of the major problems I am seeing on this packaging work is because I did choose thumb2 (space saving and compatibility are worth it), so far I did either fetch linaro patches, tweak options, add -marm to CFLAGS and CXXFLAGS, etc, as most packages around expect arm instruction set, and few handle thumb2. There were a few other tricks, like some (very uncommon) code expecting "char" to be always "signed".
Hardfp abi adds a completely new set of incompatibility in the field, that should affect asm in different places, break pretty much any jit/ffi around, does not run armv5 or earlier binaries, and all to have what? Like 3% (with luck) increase in performance in fpu intensive applications, while almost everything else has no benefits. From my understanding, it was wrong from start, and most likely caused by comparing apples to oranges, err, software float vs hardware float, probably also the software float one not using armv6 instructions, not "softfp abi" vs "hardfp abi".
Jon and I have both been talking to Mats and Jeff about LSB already, but things have gone a little quiet while we've been concentrating on bringup in the last few months. They would definitely be the right people to work with going forwards...
So...Adam, we need to sync up on this. I would like to have made a little progress by the time we get to Connect next month.
Jon.
Paulo
cross-distro mailing list cross-distro@lists.linaro.org http://lists.linaro.org/mailman/listinfo/cross-distro
On Thu, Oct 06, 2011 at 03:20:16PM -0300, Paulo César Pereira de Andrade wrote:
Em 6 de outubro de 2011 14:38, Jon Masters jcm@redhat.com escreveu:
On Tue, 2011-10-04 at 00:41 -0300, Paulo César Pereira de Andrade wrote:
For now I have been only considering gcc as compiler
I think, realistically, so have we. But I'm open to rethinking that.
The problem is that this compiler apparently only offers two options, softfp (not softfp calling convention) by generating code without fpu support, and hardfp that only supports float arguments on float registers. IMO, the hardfp abi is not something for a distribution, but for a "closed" (not necessarily closed source) image, where the exact hardware is known before hand, and from Linux pretty much only the kernel and a few packages are used.
Nope, not at all (as mentioned in my earlier message).
...
Hardfp abi adds a completely new set of incompatibility in the field, that should affect asm in different places, break pretty much any jit/ffi around, does not run armv5 or earlier binaries, and all to have what? Like 3% (with luck) increase in performance in fpu intensive applications, while almost everything else has no benefits. From my understanding, it was wrong from start, and most likely caused by comparing apples to oranges, err, software float vs hardware float, probably also the software float one not using armv6 instructions, not "softfp abi" vs "hardfp abi".
Konstantinos has benchmarks showing a lot more than "3%" improvements. I'll let him post links and explanations.
Cheers,
On Tue, Oct 04, 2011 at 12:41:46AM -0300, Paulo César Pereira de Andrade wrote:
2011/10/3 Steve McIntyre steve.mcintyre@linaro.org:
I think I am in the "free software zealot" position here :-)
That's fair enough. :-)
So, a few minutes ago I was pointed to http://wiki.meego.com/Hardware-accelerated_graphics_on_Pandaboard_using_MeeG... that points to http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0041c/Babdhhd...
or for some related information http://groups.google.com/group/pandaboard/browse_thread/thread/cd69411c1cb60...
For now I have been only considering gcc as compiler, and the link in infocenter.arm.com appears to suggest that the mapping, to have an armv7 distro capable of running existing armv5 or older software, would be:
gcc -mfloat-abi=softfp -mfpu=vfpv3-d16 -> tcc /hardfp /nofpregargs
possibly replacing vfpv3-d16 with neon
but the problem is, the tcc (arm C compiler!?) calls softfp what gcc calls -mfloat-abi=soft, but "now" only supports hardp with what gcc calls -mfloat-abi=hard calling convention.
*Warning*: You should be aware that this infocenter document is *withdrawn* and should not be relied upon! It's ancient and misleading and not relevant to current ABI discussions.
/fpregargs Floating-point arguments are passed in floating-point registers. This option is not available for in tcc or tcpp, and is the default for ARM if /hardfp is selected. /nofpregargs Floating-point arguments are not passed in floating-point registers. This option is not available in tcc or tcpp. This option is obsolete and is available for backwards compatibility only.
Note, that "tcc" issues is news to me, as so far I was only considering gcc and its optimizations that make breaking the abi pretty much nonsense, but was not considering an extra compiler, that may be being used to build external, non open source libraries, e.g. libGLes*.
Don't worry about it, it's not an issue at all. Modern tools will work just fine.
Cheers,
Jon, tardy but good. I'm interested in LSB; just not sure what effort this is...
Dave
On 26/09/11 14:37, Jon Masters wrote:
[ Please forward this to anyone else who should get a copy - and ask them to signup to cross-distro@linaro because that's where we'll talk ]
Folks,
At the 2011 Linux Plumbers Conference in Santa Rosa, we held an ARM minisummit to discuss a few cross-distribution issues surrounding the "hard float" ABI for v7, and the ARM device and OS ecosystem in general. This mail is intended to summarize the key actions from that meeting.
1). We discussed some of the different distributions (Debian, ChromeOS, Fedora, MeeGo, Mozilla, Ubuntu, etc.). We all agree on a common notion of what "hard float" means: it's the ARM AAPCS hardware floating point stuff in section 6, with the -d16 register set of the VFPv3 being used. Although there is some possibility to standardize v6, we decided we are interested only in the present and future at this point (and that's v7).
Some use Thumb2, some don't, but interworking allows for that (we in Fedora accidentally built a few packages with Thumb2 and will be undoing that without users even noticing). NEON is similarly an optional feature (supported by hwcaps) and not a required part of the core ABI. So it's really just the ABI that we all care about, and we all agree on it.
For toolchain folks, this means we all agree on the following:
--with-float=hard --with-fpu=vfpv3-d16 --with-abi=aapcs-linux
(some of us may target different optimizations, etc. of course but that is down to an application to decide whether it cares only about Cortex+)
2). We agreed on the importance of cross-distro binary compatibility. It seems a number of distributions are excited by Debian's multi-arch approach (as am I), but not all of us will be moving to it any time soon. In the shorter term, if the runtime linker will have a special path, we can work around that with symlinks, but it's a kludge. Our preferred goal (from the meeting) is that everyone use this target:
arm-linux-gnueabihf
(Red Hat toolchains will include the vendor, but it is dropped by autotools when actually generating the paths and comparisons). I will advocate for Fedora ARM to switch to this target when possible, and in the interim will try to ensure we can establish some symlinks for the ability to run binaries compiled against Debian and Ubuntu on Fedora. I don't know what our timeline will be for this, but it's not something that's really landed everywhere yet. We at least agree on paths now!
3). We agree that it is a good idea to do LSB. Adam Conrad (Canonical) and I agreed to represent community interests in driving this. We do know that we'll be preferring OpenGL ES 2.0 over OpenGL (same namespace within the library means one default choice is required). There will be many other things :) We agree that the main benefit to pursuing LSB is that it promotes the idea of working together and being compatible. It may be that we don't have the same level of compliance on ARM, but some level of conformance to some common standard(s) is a good start! :)
4). We decided that we should have a standing Cross Distro meeting at Linaro Connect events, beginning with the next one in November. We will also facilitate a "show and tell" for apps at Connect, and have an opportunity to demo different distributions. We will add a specific session at the next Connect to continue this discussion.
Finally, we agreed we should keep the conversation going. I apologize for being a little tardy in sending this mail, but I want to kickstart further discussion and get this ball rolling again. I'll ping Adam about LSB (and thanks again to Jesse for the deep dive on GL). Meanwhile, David can grab ownership of organizing the Cross Distro meeting at Connect (I would like to assist). Anyone else got actions?
Jon.
-
Alexander Graf -
dann frazier -
David Rusling -
Jeremiah Foster -
Jon Masters -
Jon Masters -
Paulo César Pereira de Andrade -
Steve McIntyre