[Linaro-dev] linaro build system (was Re: Availability of hardware for Linaro development?)
Would it not make more sense to simply standardize on a readily available toolchain such as CodeSourcery's Lite ARM toolchain?
Excellent suggestion! :-)
Hello,
I just subscribed to this maillist and am very interested in build system setup for linaro as a long time embedded developer (half of my time on UI system and half on BSP/distribution) and of course most on ARM :-)
I certainly suggest CodeSourcery toolchain too!
But build system is more than just gcc, binutil and libc and in distributions, it is often bound to package system, and tools to create a clean chroot to install dependencies automatically.
Some tools like scratchbox basically "pretend" the cross compiling to be native to avoid/minimize special patches for cross compiling (we can't guarantee code from upstream are always cross compiling friendly).
OBS(Opensuse Build System) does native compiling through QEMU and it also can be used similar to scratchbox, i.e., native compiling using cross gcc, native(x86)bash/perl/python to speed up the build.
I think Debian/Ubuntu and most other dist are similar to OBS too, while traditional embedded linux doesn't have pkg system and using cross compiling directly.
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
Thanks. Jiandong Zheng
JD Zheng wrote:
But build system is more than just gcc, binutil and libc and in distributions, it is often bound to package system, and tools to create a clean chroot to install dependencies automatically.
Yes, that's correct -- and that is indeed one of the challenges of cross-compilation for Linux targets. The configuration of the compiler determines to some extent where it expects shared libraries to be located at run-time, and where it expects headers to be found (relative to the sysroot) at compile-time. Sometimes, this means that you have to use command-line options to help the poor compiler cope with a Linux distribution that doesn't have things where the compiler expects them to be.
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
Just so that nobody things I'm ignoring this question, I'll answer it: I have no idea. I didn't promise a useful answer! Seriously, this just isn't something I know anything about; I'm sure that the Linaro folks have a much better answer.
Thanks,
On Sun, Jun 6, 2010 at 10:37 AM, Mark Mitchell mark@codesourcery.com wrote:
JD Zheng wrote:
But build system is more than just gcc, binutil and libc and in distributions, it is often bound to package system, and tools to create a clean chroot to install dependencies automatically.
Yes, that's correct -- and that is indeed one of the challenges of cross-compilation for Linux targets. The configuration of the compiler determines to some extent where it expects shared libraries to be located at run-time, and where it expects headers to be found (relative to the sysroot) at compile-time. Sometimes, this means that you have to use command-line options to help the poor compiler cope with a Linux distribution that doesn't have things where the compiler expects them to be.
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
Just so that nobody things I'm ignoring this question, I'll answer it: I have no idea. I didn't promise a useful answer! Seriously, this just isn't something I know anything about; I'm sure that the Linaro folks have a much better answer.
Keep openembedded and bitbake in mind because it is very flexible.
Hello!
Would it not make more sense to simply standardize on a readily available toolchain such as CodeSourcery's Lite ARM toolchain?
Excellent suggestion! :-)
I just subscribed to this maillist and am very interested in build system setup for linaro as a long time embedded developer (half of my time on UI system and half on BSP/distribution) and of course most on ARM :-)
I certainly suggest CodeSourcery toolchain too!
But build system is more than just gcc, binutil and libc and in distributions, it is often bound to package system, and tools to create a clean chroot to install dependencies automatically.
Some tools like scratchbox basically "pretend" the cross compiling to be native to avoid/minimize special patches for cross compiling (we can't guarantee code from upstream are always cross compiling friendly).
OBS(Opensuse Build System) does native compiling through QEMU and it also can be used similar to scratchbox, i.e., native compiling using cross gcc, native(x86)bash/perl/python to speed up the build.
I think Debian/Ubuntu and most other dist are similar to OBS too, while traditional embedded linux doesn't have pkg system and using cross compiling directly.
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
Don't forget about OpenEmbedded: http://www.openembedded.org/ OE can build images for real hardware and for qemu (x86 and qemu-arm). OE can use external toolchain and can build own (gcc-based).
There's a blueprint here - https://blueprints.edge.launchpad.net/ubuntu/+spec/arm-m-xdeb-cross-compilat... - for the cross compilation / build work. There are two distinct strands, getting better cross build tools (I'm personally not keen on scratch box, too hacky to set up and slow) and getting decent build hardware available. The hardware will come.
Dave CTO, Linaro
On 6 Jun 2010, at 07:33, Yuri Bushmelev jay4mail@gmail.com wrote:
Hello!
Would it not make more sense to simply standardize on a readily available toolchain such as CodeSourcery's Lite ARM toolchain?
Excellent suggestion! :-)
I just subscribed to this maillist and am very interested in build system setup for linaro as a long time embedded developer (half of my time on UI system and half on BSP/distribution) and of course most on ARM :-)
I certainly suggest CodeSourcery toolchain too!
But build system is more than just gcc, binutil and libc and in distributions, it is often bound to package system, and tools to create a clean chroot to install dependencies automatically.
Some tools like scratchbox basically "pretend" the cross compiling to be native to avoid/minimize special patches for cross compiling (we can't guarantee code from upstream are always cross compiling friendly).
OBS(Opensuse Build System) does native compiling through QEMU and it also can be used similar to scratchbox, i.e., native compiling using cross gcc, native(x86)bash/perl/python to speed up the build.
I think Debian/Ubuntu and most other dist are similar to OBS too, while traditional embedded linux doesn't have pkg system and using cross compiling directly.
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
Don't forget about OpenEmbedded: http://www.openembedded.org/ OE can build images for real hardware and for qemu (x86 and qemu-arm). OE can use external toolchain and can build own (gcc-based).
-- Yuri Bushmelev
-- С уважением, Бушмелев Юрий
Linaro-dev mailing list Linaro-dev@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-dev
On Sat, Jun 05, 2010, JD Zheng wrote:
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
This is a hard question, as it's still evolving; we currently consider two different buildables: software (packages) and images. For image building, we're currently trying live-helper, it seems flexible enough to cover a lot of use cases. For package builds, we're looking at replacing native builds with some cross-compilers backed solutions (e.g. like scratchbox does), and we're also trying to support cross-compilation, but that wont scale to thousands of packages for sure.
Basically, we need more time to try out/develop the bits we're currently thinking of, the picture is not complete yet. In the mean time, we're doing native builds.
Cheers,
On Sun, Jun 6, 2010 at 2:47 PM, Loïc Minier loic.minier@linaro.org wrote:
On Sat, Jun 05, 2010, JD Zheng wrote:
So my question is what will be the default build system, which, I guess, is part of deliveries? Since linaro is not a distribution, will it be dist-agnostic? (Don't get me wrong as I am a Debian/Ubuntu fan :-) ).
This is a hard question, as it's still evolving; we currently consider two different buildables: software (packages) and images. For image building, we're currently trying live-helper, it seems flexible enough to cover a lot of use cases. For package builds, we're looking at replacing native builds with some cross-compilers backed solutions (e.g. like scratchbox does), and we're also trying to support cross-compilation, but that wont scale to thousands of packages for sure.
Basically, we need more time to try out/develop the bits we're currently thinking of, the picture is not complete yet. In the mean time, we're doing native builds.
OpenEmbedded can handle all usecases and it can generate toolchains for you guys to provide to community incase someone does not want to handle openEmbedded. Scatchbox and buildroot kinda things are too limited. There is a huge community that takes care of openEmbedded.
Hi all
I think people missed one of the points in my email, which is that build is only part of the problem. I'm well aware of cross compilation and emulators for builds, however neither of these environments provides a fully capable *test* environment. This is where a hardware farm is one solution, one which wouldn't require everyone who wants to contribute to purchase their own hardware. From my experience porting the remote GDB backend to a custom JTAG debug architecture it makes everything much easier to debug in an embedded architecture when you can inspect internal state over JTAG. Whilst emulators can provide a good debug environment, they never completely reflect the behaviour of the actual hardware in terms of timing, lockups, undocumented behaviour etc.
Thoughts?
Joel
On Sun, Jun 6, 2010 at 9:51 PM, Joel Crisp cydergoth@gmail.com wrote:
Hi all
I think people missed one of the points in my email, which is that build is only part of the problem. I'm well aware of cross compilation and emulators for builds, however neither of these environments provides a fully capable *test* environment. This is where a hardware farm is one solution, one which wouldn't require everyone who wants to contribute to purchase their own hardware. From my experience porting the remote GDB backend to a custom JTAG debug architecture it makes everything much easier to debug in an embedded architecture when you can inspect internal state over JTAG. Whilst emulators
You mean that this farm be remote to developers? Can you please elaborate ...
can provide a good debug environment, they never completely reflect the behaviour of the actual hardware in terms of timing, lockups, undocumented behaviour etc.
Thoughts?
Joel
Hi Shaz
Yes, a farm remote to developers. I would envision something like:
* Network hub/router * A Linux x86 host(s) with a number of JTAG and USB ports available (via expansion boards if needed, but USB hubs and USB->JTAG adapters should be possible) * A number of ARM boards, with a variety of peripherals. Each ARM board plugged into Ethernet/USB/JTAG as appropriate (or all 3). * Software for a developer to schedule time on a board remotely via the Linux host * Possibly some mechanism to see the output from a LCD, either an adapter board to the Linux host or a webcam pointed at the LCD or something. Other peripherals could be monitored in a similar way. AFAIK ERSA (European Space Agency) do something similar with emulated/real modules hooked together with USB.
Then a developer could develop locally under an emulator, do coarse debugging there, then when ready book time on a real board in the farm for final testing.
I think this should be feasible?
Joel
On Sun, Jun 6, 2010 at 1:42 PM, Shaz shazalive@gmail.com wrote:
On Sun, Jun 6, 2010 at 9:51 PM, Joel Crisp cydergoth@gmail.com wrote:
Hi all
I think people missed one of the points in my email, which is that build
is
only part of the problem. I'm well aware of cross compilation and
emulators
for builds, however neither of these environments provides a fully
capable
*test* environment. This is where a hardware farm is one solution, one
which
wouldn't require everyone who wants to contribute to purchase their own hardware. From my experience porting the remote GDB backend to a custom
JTAG
debug architecture it makes everything much easier to debug in an
embedded
architecture when you can inspect internal state over JTAG. Whilst
emulators
You mean that this farm be remote to developers? Can you please elaborate ...
can provide a good debug environment, they never completely reflect the behaviour of the actual hardware in terms of timing, lockups,
undocumented
behaviour etc.
Thoughts?
Joel
-- Shaz
Joel Crisp wrote:
- A Linux x86 host(s) with a number of JTAG and USB ports available (via
expansion boards if needed, but USB hubs and USB->JTAG adapters should be possible)
We (CodeSourcery) have got a lot of experience with remote board labs -- all of our engineers are distributed around the world, but most of our boards live in an unattended data center.
There tend to be a fair number of headaches for JTAG debug:
* Drivers for various JTAG devices only run on Windows, and Windows machines are somewhat more difficult to access remotely.
* Whether on Windows or Linux, connecting a bunch of devices via USB sometimes leads to conflicts; for example, drivers may only expect *one* instance of a given JTAG device.
* Many new boards are powered over USB, but power-cycling via USB is not always reliable -- and you definitely need to be able to power-cycle them when they get wedged.
It's much easier, however, if you're not too concerned with kernel development. In that case, you probably don't need JTAG. Then, you just connect the boards via ethernet, and as long as you can remotely power-cycle them (which is not bad assuming that they are not USB-powered, but are powered from an AC circuit), you can do that reasonably easily. As long as you're not intent on working with lots of peripherals (display hardware, audio hardware, etc.), it's pretty easy to use boards in this way.
On (06/06/10 16:31), Mark Mitchell wrote:
Joel Crisp wrote:
- A Linux x86 host(s) with a number of JTAG and USB ports available (via
expansion boards if needed, but USB hubs and USB->JTAG adapters should be possible)
I have used BDI to control and debug linux kernel on board remotely. I could telnet into the board as the board is also on network. But BDIs may not work on all kind of boards.
However its hard when it comes to peripherals which need visual inspection.
We (CodeSourcery) have got a lot of experience with remote board labs -- all of our engineers are distributed around the world, but most of our boards live in an unattended data center.
There tend to be a fair number of headaches for JTAG debug:
- Drivers for various JTAG devices only run on Windows, and Windows
machines are somewhat more difficult to access remotely.
- Whether on Windows or Linux, connecting a bunch of devices via USB
sometimes leads to conflicts; for example, drivers may only expect *one* instance of a given JTAG device.
- Many new boards are powered over USB, but power-cycling via USB is not
always reliable -- and you definitely need to be able to power-cycle them when they get wedged.
It's much easier, however, if you're not too concerned with kernel development.
yes thats right.
-Khem
Hi Khem, On Sun, Jun 06, 2010 at 06:30:52PM -0700, Khem Raj wrote:
On (06/06/10 16:31), Mark Mitchell wrote:
Joel Crisp wrote:
- A Linux x86 host(s) with a number of JTAG and USB ports available (via
expansion boards if needed, but USB hubs and USB->JTAG adapters should be possible)
I have used BDI to control and debug linux kernel on board remotely. I could telnet into the board as the board is also on network. But BDIs may not work on all kind of boards.
BDI is also very expensive :(.
baruch
On (07/06/10 07:27), Baruch Siach wrote:
Hi Khem, On Sun, Jun 06, 2010 at 06:30:52PM -0700, Khem Raj wrote:
On (06/06/10 16:31), Mark Mitchell wrote:
Joel Crisp wrote:
- A Linux x86 host(s) with a number of JTAG and USB ports available (via
expansion boards if needed, but USB hubs and USB->JTAG adapters should be possible)
I have used BDI to control and debug linux kernel on board remotely. I could telnet into the board as the board is also on network. But BDIs may not work on all kind of boards.
BDI is also very expensive :(.
there are other less expensive ones like ocdaemon out there but I havent used then on linux.
baruch
-- ~. .~ Tk Open Systems =}------------------------------------------------ooO--U--Ooo------------{=
- baruch@tkos.co.il - tel: +972.2.679.5364, http://www.tkos.co.il -
-
Baruch Siach -
David Rusling -
JD Zheng -
Joel Crisp -
Khem Raj -
Loïc Minier -
Mark Mitchell -
Shaz -
Yuri Bushmelev