Hi,
Our current default root file system, ext3, is proving to be a bottleneck for SD card performance. Not only does it take a long time to format the partitions, but it also takes a long time to write to. This slows down creating images on SD cards a lot. I just did a very simple experiment running linaro-media-create, writing an Ubuntu Desktop image to an SD card:
ext3: 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata 107360maxresident)k 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps
btrfs: 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata 107408maxresident)k 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps
As I understand it, btrfs is considered OK for file systems running on systems that don't suffer from power failure, so for writing an image and testing it this should be fine.
So, what do people think about switching?
On 4 August 2011 12:46, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
Hi,
Our current default root file system, ext3, is proving to be a bottleneck for SD card performance. Not only does it take a long time to format the partitions, but it also takes a long time to write to. This slows down creating images on SD cards a lot. I just did a very simple experiment running linaro-media-create, writing an Ubuntu Desktop image to an SD card:
ext3: 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata 107360maxresident)k 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps
btrfs: 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata 107408maxresident)k 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps
What's dpkg like these days on btrfs - it used to be awfully slow. Some comparisons from the installed running system would really be needed.
As I understand it, btrfs is considered OK for file systems running on systems that don't suffer from power failure, so for writing an image and testing it this should be fine.
It doesn't sound like the description of development systems though where people will crash them and reset them and generally pull the plug on them.
So, what do people think about switching?
To be honest I trust ext* (it's never lost me a byte in many years of use) and btr feels a bit new to me; but that's a personal feeling.
Dave
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/04/2011 01:46 PM, James Tunnicliffe wrote:
Hi,
Our current default root file system, ext3, is proving to be a bottleneck for SD card performance. Not only does it take a long time to format the partitions, but it also takes a long time to write to. This slows down creating images on SD cards a lot. I just did a very simple experiment running linaro-media-create, writing an Ubuntu Desktop image to an SD card:
ext3: 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata 107360maxresident)k 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps
btrfs: 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata 107408maxresident)k 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps
As I understand it, btrfs is considered OK for file systems running on systems that don't suffer from power failure, so for writing an image and testing it this should be fine.
So, what do people think about switching?
IMHO it is not stable enough and I am not sure it is worth having such filesystem as it is mainly used for snapshotting. The last time I played with it, the FS was quickly corrupted but I don't have to complain because the kernel configuration help was explicit enough :)
"Btrfs is highly experimental, and THE DISK FORMAT IS NOT YET FINALIZED. You should say N here unless you are interested in testing Btrfs with non-critical data."
On 4 August 2011 13:07, Daniel Lezcano daniel.lezcano@linaro.org wrote:
IMHO it is not stable enough and I am not sure it is worth having such filesystem as it is mainly used for snapshotting. The last time I played with it, the FS was quickly corrupted but I don't have to complain because the kernel configuration help was explicit enough :)
"Btrfs is highly experimental, and THE DISK FORMAT IS NOT YET FINALIZED. You should say N here unless you are interested in testing Btrfs with non-critical data."
That description is, as far as I can tell, out of date. https://help.ubuntu.com/community/btrfs#Stability
I haven't had any problems with my testing, which includes pulling the power on the board I am using, but I haven't torture tested it.
Paul: It has been file system option for Ubuntu since 10.10.
I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct).
-- James Tunnicliffe
On 4 August 2011 14:52, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct).
What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k should write 4MB chunks out
Dave
On 4 August 2011 14:56, David Gilbert david.gilbert@linaro.org wrote:
On 4 August 2011 14:52, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct).
What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k should write 4MB chunks out
My understanding is they are 4MB chunks, but not aligned to 4MB boundaries.
On 4 August 2011 15:28, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
On 4 August 2011 14:56, David Gilbert david.gilbert@linaro.org wrote:
On 4 August 2011 14:52, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct).
What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k should write 4MB chunks out
My understanding is they are 4MB chunks, but not aligned to 4MB boundaries.
What exactly is your dd line?
Dave
On 4 August 2011 15:37, David Gilbert david.gilbert@linaro.org wrote:
On 4 August 2011 15:28, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
On 4 August 2011 14:56, David Gilbert david.gilbert@linaro.org wrote:
On 4 August 2011 14:52, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
I have seen poor performance when DDing to a card, which I assume is because dd is not writing large aligned chunks. If we can dd the first meg or so of data onto the card, then write in 4MB chunks that are all 4MB aligned that should be quick (at least, if my reading of my flashbench results is correct).
What options to dd are you using? dd if=file of=/dev/thewholedevice bs=4096k should write 4MB chunks out
My understanding is they are 4MB chunks, but not aligned to 4MB boundaries.
What exactly is your dd line?
I haven't used dd in a while for this, so I don't have a dd line as such. I did just try the line you gave and it ran at the highest speed that flashbench showed that my SD card could provide, so thanks for that! Not sure what I did differently last time... no matter, this will speed up testing a lot.
I am slightly concerned having searched around about using ext* on flash - it sounds like ext2 and ext3 at least may be good at wearing out SD cards. Using dd is good for avoiding the file attribute and journal updates that would happen during the initial copy of the root file system to an SD card, so if I am using ext3 it will be my preferred way of getting a card ready. Unfortunately a lot of what I have read should have [citation needed] attached to it.
One post that at least seemed logical was:
http://talk.maemo.org/showthread.php?t=16873 pauljohn32 "In the Fedora linux list, we had a long thread about ext3 on Sd cards. Consensus was that it is bad to do this because it will use up the finite number of writes on the card. They recommend VFAT because it does not make as many separate writes when files are accessed. ext2 is expensive because it tracks so many attributes,especially atime.. ext3 is more costly because of journaling."
I have done a bit of benchmarking, which involves copying the files that are in the root file system of a Nano image plus a few larger files over to the same SD card formatted with different file systems:
btrfs: 0m55.181s ext3: 3m28.145s ext4: 1m20.663s ext4 no journal: 1m17.868s
My guess is that not only does a fast copy keep me happy, it also keeps my SD card happy (faster probably indicates fewer read, modify, write cycles = lasts longer). It looks like we should change our default file system to ext4 so we default to a file system that we can all be confident in and those of us who want to can use btrfs. Disabling the journal may help with SD card longevity, but it doesn't seem to make much difference to speed in this case.
But...
I just tested two a nano images using ext4 and they didn't find the root file system. So, we are at least gated on that bug.
On Fri, Aug 5, 2011 at 1:08 PM, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
I just tested two a nano images using ext4 and they didn't find the root file system. So, we are at least gated on that bug.
I filed https://bugs.launchpad.net/linaro-ubuntu/+bug/822593 to make ext4 our default. Seems that our kernel configs as well as linaro-image-tools have to be updated. For btrfs I remember we have to ensure that btrfs-tools needed are in all rootfs as well.
On Thu, 4 Aug 2011 12:46:58 +0100 James Tunnicliffe james.tunnicliffe@linaro.org wrote:
Hi,
Our current default root file system, ext3, is proving to be a bottleneck for SD card performance. Not only does it take a long time to format the partitions, but it also takes a long time to write to. This slows down creating images on SD cards a lot. I just did a very simple experiment running linaro-media-create, writing an Ubuntu Desktop image to an SD card:
ext3: 139.85user 35.27system 44:03.58elapsed 6%CPU (0avgtext+0avgdata 107360maxresident)k 2876115inputs+7048200outputs (958major+1677659minor)pagefaults 0swaps
btrfs: 146.52user 34.48system 19:57.16elapsed 15%CPU (0avgtext+0avgdata 107408maxresident)k 4417521inputs+6542992outputs (138major+1779874minor)pagefaults 0swaps
As I understand it, btrfs is considered OK for file systems running on systems that don't suffer from power failure, so for writing an image and testing it this should be fine.
So, what do people think about switching?
There were few concerns already expressed, so I just add: is btrfs supported out of the box by kernels in last ~3 Ubuntu releases? Because if one can't look at/change contents produced on an SD card, that undermines purpose of evaluation builds much.
Those ext3 vs btrfs results are very vivid though, but I wonder if it makes sense to try to tweak ext3 & mount options. And after all, we could prepare partition image(s) on the local HDD and dd them to a card...
Hi James
On Thu, Aug 4, 2011 at 6:46 AM, James Tunnicliffe james.tunnicliffe@linaro.org wrote:
Hi,
Our current default root file system, ext3, is proving to be a bottleneck for SD card performance. Not only does it take a long time to format the partitions, but it also takes a long time to write to. This slows down creating images on SD cards a lot. I just did a very simple experiment running linaro-media-create, writing an Ubuntu Desktop image to an SD card:
<snip>
As I understand it, btrfs is considered OK for file systems running on systems that don't suffer from power failure, so for writing an image and testing it this should be fine.
So, what do people think about switching?
There's no need to switch. btrfs vs ext* is simply a choice to be made at the time you call linaro-media-create. There's nothing stopping you from using btrfs if that's what you want to do with all the caveats etc that have been stated by others.
-
Alexander Sack -
Daniel Lezcano -
David Gilbert -
James Tunnicliffe -
Paul Sokolovsky -
Tom Gall