Hi asac,

On Wed, Oct 6, 2010 at 2:43 AM, Alexander Sack asac@linaro.org wrote:

On Wed, Oct 6, 2010 at 6:24 AM, Tom Gall tom.gall@linaro.org wrote:

...

The code is located here : lp:~tom-gall/linaro-image-tools/rsync-speedup

I'm still testing so use with caution.

From what I've been able to accomplish thus far, the 67M linaro-headless + hwpack sees the following speed up collected with time:

Normal: real 10m0.596s user 2m2.670s sys 0m9.820s

Parallel: real 7m58.790s user 2m3.170s sys 0m11.950s

nice. do you also have numbers for without hwpack?

I haven't tested without hwpacks. I'm not sure that's a valid test anymore with the removal of flavors and so on from the builds. Outside of using an old linaro image, is it valid anymore to call linaro-media-create without passing hwpacks?

also ...

rsync -a ${DIR}/binary/etc ${DIR}/binary/usr ${DIR}/binary/lib ${DIR}/binary/bin ${DIR}/binary/home ${DIR}/binary/root ${DIR}/binary/media ${DIR}/binary/opt ${DIR}/binary/sbin ${ROOT_DISK}

this should at least include /var as well. how about just synching everything to be safe?

Well things like dev and proc are in use during the hwpack install so I'd rather avoid them during the first rsync.

var you make a good point.

Regards, Tom

"We want great men who, when fortune frowns will not be discouraged." - Colonel Henry Knox w) tom.gall att linaro.org w) tom_gall att vnet.ibm.com h) tom_gall att mac.com

On Wed, Oct 6, 2010 at 9:27 PM, Loïc Minier loic.minier@linaro.org wrote:

[...]

There might be an interesting speedup + bugfix in aligning the rootfs  partition on a power of two, e.g. 1 MiB boundary, as the SD/MMC is  flash internally, and undergoes erase/write cycles for fs meta-data.  I think only OMAP has constraints on the number of heads/sectors, but  we could take a factor of that: 63*255 can't really be factored well,  but we could use LBA addressing and start in the middle of a cylinder.

I had also thought about this as a potential issue.

32 sectors * 16 heads is what I usually use -- this gives a 256KB cylinder size, which should be an integral number of underlying erase blocks on pretty much any flash device, and has divided exactly into the size of every SD card of flash stick I have ever seen. If we really want to be clever, we could use factor /sys/block/<dev>/size, to choose head and sector counts, but I doubt that it's worth it.

Disabling the journal (for ext4, use -onoload / for ext3, mount with -text2 instead) and/or mounting with noatime (-onoatime) would also make sense when mounting the rootfs to populate it, especially when using --mmc; this might improve performance a little.

Of course, since we're not trying to boot an obsolete PC BIOS or operating system on these devices, it's pretty meaningless to think in terms of cylinders; even in PC-land, this convention is now considered obsolete IIUC (how many devices _really_ have 63 sectors and 255 heads, and how much software still can't cope with >1023 cylinders?). In that case, we could just work in multiples of 2^n sectors, for some n (something in the range 19<=n<=21 is probably a good choice). Fortunately, Linux partitioners always support partitioning by sector these days.

I did try to force this configuration manually, but I ran into problems--- the sectors/heads assumption is hard-coded into linaro-media-create in a number of places, and I didn't end up with something bootable... but I probably missed something.

Cheers ---Dave

On Tue, Oct 12, 2010 at 11:15 AM, Loïc Minier loic.minier@linaro.org wrote:

On Tue, Oct 12, 2010, Dave Martin wrote:

...

32 sectors * 16 heads is what I usually use -- this gives a 256KB cylinder size, which should be an integral number of underlying erase blocks on pretty much any flash device, and has divided exactly into the size of every SD card of flash stick I have ever seen.  If we really want to be clever, we could use factor /sys/block/<dev>/size, to choose head and sector counts, but I doubt that it's worth it.

Unfortunately, OMAP's boot ROM imposes the current head/sector counts  we're using

However, Linux never uses CHS, but always LBA; so we could arrange to  use proper CHS and still align the partition properly

Yes, Linux 100% does not care about the CHS junk in the partition table, or the head and sector counts (since when talking to an SD card or similar, the interface has no concept of heads or sectors ... just LBA offsets or something)

So we can make the /boot partition look like it has 63*255 geometry, and we can make it start at sector offset 63.

Everything else can be determined based on sane absolute sector offsets.

I don't know of any tool which allows doing that properly; I trust  parted, but I never managed to force chs geometry with it.  Not sure  whether we can convince fdisk to do it, don't really like driving this  thing

sfdisk seems to works: e.g.

device=<target device or image file> sector_size=512 device_total_sectors=<total sectors, from sysfs, built-in default (for --image_file) or command line> boot_partition_MB=64 # approximate size in MB for the root partition

# number of sectors per partitioning unit; should be a multiple of the rootfs block size (4KiB) and the underlying flash erase block size (256KiB): # 512 (256KiB) is probably a good choice. partition_unit_sectors=512

target_fake_spt=63 # sectors-per-track to fake for the boot partition geometry target_fake_heads=255 # heads to fake for the boot partition geometry

boot_start_sectors=$target_fake_spt boot_size_sectors=$((boot_partition_MB/(sector_size*partition_unit_sectors)*partition_unit_sectors - boot_start_sectors)) boot_type=0xC # FAT32 LBA root_start_sectors=$((boot_start_sectors + boot_size_sectors)) root_size_sectors=$((device_total_sectors - root_start_sectors)) root_type=0x83 # Linux filesystem

cat <<EOF | sfdisk --force -S$target_fake_spt -H$target_fake_heads -us "$device" $boot_start_sectors,$boot_size_sectors,$boot_type,* $root_start_sectors,$root_size_sectors,$root_type EOF

# --force is needed so that sfdisk accepts the non-cylinder aligned partition boundaries.

One advantage is that we can set up the partition table without ever needing to parse the output of any partitioner (which I view as being very fragile...) With a bit of luck, we can also lose the dependencies on parted and fdisk.

I don't yet know whether this approach will actually boot in Beagle though.

...

Disabling the journal (for ext4, use -onoload / for ext3, mount with -text2 instead) and/or mounting with noatime (-onoatime) would also make sense when mounting the rootfs to populate it, especially when using --mmc; this might improve performance a little.

I think relatime is the default already, it's probably good enough;  isn't it?

dunno, you may be right. I guess on atimes will get updated except on directories anyway -- the files are only written, not read.

It would be interesting to benchmark whether disabling the journal makes any difference; I think it might improve things a little, but I haven't measured it.

-obarrier=0 would also be sensible for ext3/4, since the only atomicity we care about it at filesystem unmount.

[...]

Oh ok; so can you arrange for CHS to look like 63 sectors and 255  heads, and use rounded up LBA addresses with some tool of your choice?

Probably the boot partition needs to really start at a CHS address, not  a LBA address

Probably - the CHS fields in the partition table for everything except the boot partition can be (and on many disks actually are) complete garbage.

Cheers ---Dave

On Tue, Oct 12, 2010, Dave Martin wrote:

So we can make the /boot partition look like it has 63*255 geometry, and we can make it start at sector offset 63.

Right; ideally, we'd offset it so that it's still on a nice boundary e.g. we start at sector 512 (1, 8, 8)

Everything else can be determined based on sane absolute sector offsets.

yup, using _only_ sector offsets and not caring about chs

cat <<EOF | sfdisk --force -S$target_fake_spt -H$target_fake_heads -us "$device"

s/-us/-uS/?

# --force is needed so that sfdisk accepts the non-cylinder aligned partition boundaries.

Ok; that seems all good!

On Tuesday 12 October 2010, Loïc Minier wrote:

On Tue, Oct 12, 2010, Dave Martin wrote:

...

So we can make the /boot partition look like it has 63*255 geometry, and we can make it start at sector offset 63.

Right; ideally, we'd offset it so that it's still on a nice boundary e.g. we start at sector 512 (1, 8, 8)

IIRC, Windows uses 1MB partition alignment these days. It's probably a good idea to do the same everywhere because SD card vendors might decide to "optimize" for the common case.

...

Everything else can be determined based on sane absolute sector offsets.

yup, using only sector offsets and not caring about chs

...

cat <<EOF | sfdisk --force -S$target_fake_spt -H$target_fake_heads -us "$device"

s/-us/-uS/?

...

# --force is needed so that sfdisk accepts the non-cylinder aligned partition boundaries.

I think --linux should allow you to do this as well without dropping all the safety nets.

Arnd

On Wednesday 13 October 2010 09:53:17 Dave Martin wrote:

Could be... although the Beagle ROM seems to be very sensitive to the precise layout. We might find that the partition must start at sector 63 (but I'm not sure I understand the behaviour yet).

That would be a serious bug in the firmware, probably serious enough that we could require updating.

It's not the end of the world if the vfat partition is misaligned, since this isn't accessed much, and written only occasionally. If we can have the rootfs partition aligned, that probably brings most of the benefit.

You can also manually align the FAT and the start of data to individual sectors, which is how some SD cards do it when they use the 255/63 geometry.

Arnd

Hi,

On Wed, Oct 13, 2010 at 10:58 AM, Loïc Minier loic.minier@linaro.org wrote:

On Wed, Oct 13, 2010, Arnd Bergmann wrote:

...

You can also manually align the FAT and the start of data to individual sectors, which is how some SD cards do it when they use the 255/63 geometry.

Yup

1 MiB is the value I use myself when creating partitions manually, but  I didn't know Windows used this; sounds like a good idea to use that

Let's go for 255/63 and a boot part starting at a 1 MiB: (130,138,8)

As far as I can make out, the partition must be an exact number of "cylinders", must have enough sectors for a FAT32 filesystem and must start at sector 63. This gives a minumum size is 5 cylinders (80262 sectors), which seems to work OK. Anything else causes the ROM to print out the characters "60" and halt, at least on my board.

Of course, if you follow TI's instructions for setting up the board, you do get the right (if silly) layout.

Conversely, if the FAT partition doesn't start at sector 63, you get this:

Error: reading boot sector u-boot.bin not found or blank nand contents - attempting serial boot . . .

...which suggests that the ROM is maybe making an assumption and reading sector 63 regardless of what the partition table says.

Notwithstanding this, it looks like the rootfs partition can have any configuration you like, so long as it doesn't overlap the FAT partition. This configuration worked for me, for example: Device Boot Start End Blocks Id System /dev/sdg1 * 63 80324 40131 c W95 FAT32 (LBA) /dev/sdg2 81920 7843839 3880960 83 Linux

So, unfortunately to avoid other people wasting a lot of time ... I think the boot partition must start at sector 63, with size 63*(255*n - 1) sectors, and minimum size 80262 sectors (n=5). Perhaps there are some (in)sanity checks and/or assumptions in the ROM which are requiring this. For the filesystem partition, I guess aligning up to the next 1MB boundary is indeed the sensible thing to do (as in the above example).

We should check it works with non-xM boards too...

Cheers ---Dave