On Friday 29 June 2012, Andrew Bradford wrote:
andrew@bigbox:~$ sudo sfdisk -uS -l /dev/sdc
Disk /dev/sdc: 30436 cylinders, 64 heads, 32 sectors/track Warning: The partition table looks like it was made for C/H/S=*/255/63 (instead of 30436/64/32). For this listing I'll assume that geometry. Units = sectors of 512 bytes, counting from 0
Device Boot Start End #sectors Id System /dev/sdc1 8192 62333951 62325760 c W95 FAT32 (LBA) /dev/sdc2 0 - 0 0 Empty /dev/sdc3 0 - 0 0 Empty /dev/sdc4 0 - 0 0 Empty
andrew@bigbox:~$ sudo fdisk -l /dev/sdc | grep Disk Disk /dev/sdc: 31.9 GB, 31914983424 bytes
andrew@bigbox:~$ factor 31914983424 31914983424: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 103 197
# Maybe another 1.5 MiB or 6 MiB erase block SanDisk card? # If this card has a multiple of 1.5MiB erase blocks, the stock partition map is not aligned according to sfdisk.
103*197 = 20291, so 1.5MB is actually the largest possible size that you can divide 31914983424 by.
align 16777216 pre 1.06ms on 1.32ms post 1.1ms diff 236µs align 8388608 pre 1.09ms on 1.31ms post 1.1ms diff 216µs align 4194304 pre 1.06ms on 1.21ms post 1.07ms diff 150µs align 2097152 pre 1.06ms on 1.07ms post 1.07ms diff 101ns align 1048576 pre 1.06ms on 1.07ms post 1.06ms diff 8.08µs align 524288 pre 1.06ms on 1.17ms post 1.06ms diff 113µs align 262144 pre 1.05ms on 1.06ms post 1.06ms diff 2.25µs align 131072 pre 1.07ms on 1.06ms post 1.07ms diff -10257n align 65536 pre 1.05ms on 1.07ms post 1.06ms diff 16.6µs align 32768 pre 1.06ms on 1.06ms post 1.06ms diff 5.37µs align 16384 pre 1.05ms on 1.06ms post 1.05ms diff 7.25µs align 8192 pre 1.06ms on 1.06ms post 1.06ms diff -179ns align 4096 pre 1.05ms on 1.06ms post 1.06ms diff 8.48µs align 2048 pre 1.06ms on 1.06ms post 1.07ms diff -5206ns
# Looks like 4 MiB erase blocks from that but it's not definite. Page size looks to be in the noise.
Right, this is common for sandisk. You can't really tell much from this.
andrew@bigbox:~/flashbench$ sudo ./flashbench -a /dev/sdc --blocksize=3072 --count=100 align 6442450944 pre 1.11ms on 1.35ms post 1.12ms diff 229µs align 3221225472 pre 1.13ms on 1.42ms post 1.13ms diff 289µs align 1610612736 pre 1.14ms on 1.46ms post 1.13ms diff 329µs align 805306368 pre 1.1ms on 1.42ms post 1.13ms diff 305µs align 402653184 pre 1.15ms on 1.43ms post 1.12ms diff 299µs align 201326592 pre 1.15ms on 1.44ms post 1.16ms diff 280µs align 100663296 pre 1.13ms on 1.44ms post 1.12ms diff 316µs align 50331648 pre 1.13ms on 1.44ms post 1.15ms diff 299µs align 25165824 pre 1.13ms on 1.44ms post 1.14ms diff 311µs align 12582912 pre 1.09ms on 1.28ms post 1.11ms diff 177µs align 6291456 pre 1.14ms on 1.12ms post 1.15ms diff -26264n align 3145728 pre 1.11ms on 1.11ms post 1.09ms diff 9.72µs align 1572864 pre 1.11ms on 1.11ms post 1.11ms diff -738ns align 786432 pre 1.09ms on 1.09ms post 1.14ms diff -27521n align 393216 pre 1.1ms on 1.1ms post 1.13ms diff -9959ns align 196608 pre 1.08ms on 1.08ms post 1.1ms diff -8921ns align 98304 pre 1.08ms on 1.09ms post 1.08ms diff 11.8µs align 49152 pre 1.1ms on 1.1ms post 1.1ms diff -4311ns align 24576 pre 1.09ms on 1.13ms post 1.16ms diff 2.67µs align 12288 pre 1.11ms on 1.12ms post 1.15ms diff -17144n align 6144 pre 1.1ms on 1.11ms post 1.12ms diff -4591ns
# 12 MiB stands out. Interesting.
Right.
# Open-au tests with 6 MiB assumed erase block:
andrew@bigbox:~/flashbench$ sudo ./flashbench /dev/sdc --open-au --erasesize=$[6*1024*1024] --blocksize=$[6*1024] --open-au-nr=1 6MiB 17.4M/s 3MiB 17.5M/s 1.5MiB 17.4M/s 768KiB 17.8M/s 384KiB 17.8M/s 192KiB 19.1M/s 96KiB 19.4M/s 48KiB 18.5M/s 24KiB 15.9M/s 12KiB 12.3M/s 6KiB 8.21M/s
andrew@bigbox:~/flashbench$ sudo ./flashbench /dev/sdc --open-au --erasesize=$[6*1024*1024] --blocksize=$[6*1024] --open-au-nr=2 6MiB 17.2M/s 3MiB 17.3M/s 1.5MiB 17.6M/s 768KiB 17.2M/s 384KiB 16.9M/s 192KiB 17.6M/s 96KiB 15.9M/s 48KiB 14.5M/s 24KiB 10.2M/s 12KiB 4.54M/s 6KiB 2.48M/s
andrew@bigbox:~/flashbench$ sudo ./flashbench /dev/sdc --open-au --erasesize=$[6*1024*1024] --blocksize=$[6*1024] --open-au-nr=6 6MiB 11.4M/s 3MiB 17.3M/s 1.5MiB 17.3M/s 768KiB 17.2M/s 384KiB 16.9M/s 192KiB 17.5M/s 96KiB 16M/s 48KiB 14.2M/s 24KiB 9.89M/s 12KiB 4.49M/s 6KiB 2.42M/s
andrew@bigbox:~/flashbench$ sudo ./flashbench /dev/sdc --open-au --erasesize=$[6*1024*1024] --blocksize=$[6*1024] --open-au-nr=7 6MiB 9.34M/s 3MiB 12.3M/s 1.5MiB 9.58M/s 768KiB 10.2M/s 384KiB 9.54M/s 192KiB 11.6M/s 96KiB 8.54M/s 48KiB 8.13M/s 24KiB 5.68M/s 12KiB 3.56M/s 6KiB 2.12M/s
# 6 open linear erase blocks, not bad.
Ok.
## Different day of testing below, thus different /dev/sdX
andrew@flashy:~/flashbench$ sudo ./flashbench /dev/sdd --open-au --erasesize=$[6*1024*1024] --blocksize=$[12*1024] --open-au-nr=1 --random 6MiB 18.9M/s 3MiB 19.8M/s 1.5MiB 19M/s 768KiB 18.1M/s 384KiB 15M/s 192KiB 9.19M/s 96KiB 4.3M/s 48KiB 2.14M/s 24KiB 966K/s 12KiB 2.13M/s
# Hitting K/s with 1 random open-au, probably can't do random. Things don't fall off going to more open-au but stay this slow.
It's still ok at 384KB, so it can handle random writes to some degree, just not at the smaller units.
andrew@flashy:~/flashbench$ sudo ./flashbench /dev/sdd --findfat --erasesize=$[2*1024*1024] --blocksize=$[16*1024] --fat-nr=7 2MiB 16M/s 18.6M/s 10.2M/s 12.7M/s 9.64M/s 9.96M/s 18.3M/s 1MiB 17.6M/s 18.9M/s 9.25M/s 5.58M/s 7.62M/s 7M/s 19.6M/s 512KiB 18.4M/s 19.5M/s 7.15M/s 8.4M/s 8.91M/s 7.25M/s 19.3M/s 256KiB 18.1M/s 18.2M/s 7.37M/s 7.5M/s 6.59M/s 5.45M/s 18.8M/s 128KiB 17.5M/s 17.4M/s 7.51M/s 7.03M/s 7.4M/s 6.92M/s 18.5M/s 64KiB 17.9M/s 18.9M/s 7.11M/s 7.55M/s 7.3M/s 8.28M/s 18.6M/s 32KiB 16M/s 16.9M/s 6.74M/s 6.87M/s 8.57M/s 6.82M/s 15.7M/s 16KiB 12.9M/s 13M/s 8.74M/s 7.61M/s 6.54M/s 6.75M/s 12.9M/s
# So, the first 12 MiB are special. But the first 4 MiB are "more" special somehow. This possibly explains the sector offset in sfdisk, starting the partition at 4 MiB may be the right thing to do. Special fat area goes from 4 MiB to 12 MiB.
Well, the standard mandates the 4 MB offset for the partition, so they probably took care of that like this.
# Between 39.5 and 40.5 MiB is an erase block boundary. So 40 MiB it is! # 40 - 12 = 28. 28 isn't divisible by 3. So I'm going to go with 4 MiB erase blocks.
andrew@flashy:~/flashbench$ sudo ./flashbench /dev/sdd --open-au --erasesize=$[4*1024*1024] --blocksize=$[16*1024] --open-au-nr=11 --offset=$[40*1024*1024] 4MiB 15.7M/s 2MiB 10.6M/s 1MiB 19.2M/s 512KiB 18.8M/s 256KiB 18M/s 128KiB 16.7M/s 64KiB 15.2M/s 32KiB 11.7M/s 16KiB 8.17M/s
andrew@flashy:~/flashbench$ sudo ./flashbench /dev/sdd --open-au --erasesize=$[4*1024*1024] --blocksize=$[16*1024] --open-au-nr=12 --offset=$[40*1024*1024] 4MiB 16.5M/s 2MiB 12M/s 1MiB 10.3M/s 512KiB 10.4M/s 256KiB 9.96M/s 128KiB 9.68M/s 64KiB 11.3M/s 32KiB 9.42M/s 16KiB 7.21M/s
# Using 4 MiB erase blocks, now 11 open-au linear.
What Sandisk tend to do is to split the erase block into three sub-blocks and handle writes to each of them separately, using a pseudo-SLC mode instead of the regular TLC (3 bit MLC mode). This way they can drastically improve performance for frequently-accessed blocks.
andrew@flashy:~/flashbench$ sudo ./flashbench /dev/sdd --open-au --erasesize=$[4*1024*1024] --blocksize=$[16*1024] --open-au-nr=1 --offset=$[40*1024*1024] --random 4MiB 19.2M/s 2MiB 19.4M/s 1MiB 19.1M/s 512KiB 18.8M/s 256KiB 12.2M/s 128KiB 5.03M/s 64KiB 2.53M/s 32KiB 1.26M/s 16KiB 601K/s
# Still stinks at random.
Only below 256KB accesses.
I'm noting this down as 12 MB erase blocks with 11 SLC blocks of 4 MB open concurrently.
Arnd