Can anyone provide clues to help me get 'sdcard' partition working on Linaro Android KitKat images? Here's where I've got to so far...
After updating the vexpress storage config for KitKat [1] the sdcard partition on Linaro Android images is getting mounted by vold, but apps can't access this because android.process.media keeps crashing in attachVolume because FileUtils.getFatVolumeId is returning -1.
With MediaProvider hacked to provide verbose logging I get:
V/MediaProvider( 2825): /storage/sdcard0 volume ID: -1 E/MediaProvider( 2825): Can't obtain external volume ID even though it's mounted.
and I've verified the -1 comes from the call ioctl(fd, VFAT_IOCTL_GET_VOLUME_ID) in android_os_FileUtils_getFatVolumeId in frameworks/base/core/jni/android_os_FileUtils.cpp.
Running 'mount' gives...
rootfs / rootfs rw,relatime 0 0 tmpfs /dev tmpfs rw,nosuid,relatime,mode=755 0 0 devpts /dev/pts devpts rw,relatime,mode=600 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 tmpfs /mnt/secure tmpfs rw,relatime,mode=700 0 0 tmpfs /mnt/asec tmpfs rw,relatime,mode=755,gid=1000 0 0 tmpfs /mnt/obb tmpfs rw,relatime,mode=755,gid=1000 0 0 /sys/kernel/debug /sys/kernel/debug debugfs rw,relatime 0 0 /dev/block/sda2 /system ext4 ro,relatime,data=ordered 0 0 /dev/block/sda3 /cache ext4 rw,nosuid,nodev,noatime,errors=panic,data=ordered 0 0 /dev/block/sda5 /data ext4 rw,nosuid,nodev,noatime,errors=panic,data=ordered 0 0 nodev /dev/gator gatorfs rw,relatime 0 0 /dev/block/vold/8:6 /mnt/media_rw/sdcard0 vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0 /dev/block/vold/8:6 /mnt/secure/asec vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0 /dev/fuse /storage/sdcard0 fuse rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
So first guess would be that fuse doesn't support this function, however as this seems to be the standard Android way of doing things, you would assume that it should work...?
[1] http://review.android.git.linaro.org/#/c/4563/2
Whilst it appears at first that the following issue could be relevant: https://cards.linaro.org/browse/KWG-73 I believe it isn't.
The old Android IOCTL is still present in the Linaro kernels, and should still work for any volume ID which isn't treated as an error value by the kernel. Just to make sure, I hacked the Android userside to call the upstreamed IOCTL [1] which has a different name a value, and this still returned -1. I also verified that the partition ID really isn't 0xffffffff which would still cause a problem for the Android userside API.
So, the real problem must be elsewhere.
[1] Commit 6e5b93ee55 (fatfs: add FAT_IOCTL_GET_VOLUME_ID)
On Fri, 2013-11-15 at 11:17 +0000, Jon Medhurst (Tixy) wrote:
Can anyone provide clues to help me get 'sdcard' partition working on Linaro Android KitKat images? Here's where I've got to so far...
After updating the vexpress storage config for KitKat [1] the sdcard partition on Linaro Android images is getting mounted by vold, but apps can't access this because android.process.media keeps crashing in attachVolume because FileUtils.getFatVolumeId is returning -1.
With MediaProvider hacked to provide verbose logging I get:
V/MediaProvider( 2825): /storage/sdcard0 volume ID: -1 E/MediaProvider( 2825): Can't obtain external volume ID even though it's mounted.
and I've verified the -1 comes from the call ioctl(fd, VFAT_IOCTL_GET_VOLUME_ID) in android_os_FileUtils_getFatVolumeId in frameworks/base/core/jni/android_os_FileUtils.cpp.
Running 'mount' gives...
rootfs / rootfs rw,relatime 0 0 tmpfs /dev tmpfs rw,nosuid,relatime,mode=755 0 0 devpts /dev/pts devpts rw,relatime,mode=600 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 tmpfs /mnt/secure tmpfs rw,relatime,mode=700 0 0 tmpfs /mnt/asec tmpfs rw,relatime,mode=755,gid=1000 0 0 tmpfs /mnt/obb tmpfs rw,relatime,mode=755,gid=1000 0 0 /sys/kernel/debug /sys/kernel/debug debugfs rw,relatime 0 0 /dev/block/sda2 /system ext4 ro,relatime,data=ordered 0 0 /dev/block/sda3 /cache ext4 rw,nosuid,nodev,noatime,errors=panic,data=ordered 0 0 /dev/block/sda5 /data ext4 rw,nosuid,nodev,noatime,errors=panic,data=ordered 0 0 nodev /dev/gator gatorfs rw,relatime 0 0 /dev/block/vold/8:6 /mnt/media_rw/sdcard0 vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0 /dev/block/vold/8:6 /mnt/secure/asec vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0 /dev/fuse /storage/sdcard0 fuse rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
So first guess would be that fuse doesn't support this function, however as this seems to be the standard Android way of doing things, you would assume that it should work...?
So, hacking getFatVolumeId to return a value other than -1 means the system works and files on 'external' sdcard are seen and usable by apps.
I guess the issue is that the file system is 'fuse' not 'vfat' and the fuse filesystem doesn't implement VFAT_IOCTL_GET_VOLUME_ID.
So it would seem logical that the config is wrong somehow the the non-emulated external media shouldn't be using fuse. However, the description at http://source.android.com/devices/tech/storage/index.html says:
Starting in Android 4.4, the owner, group and modes of files on external storage devices are now synthesized based on directory structure. [...] These synthesized permissions are accomplished by wrapping raw storage devices in a FUSE daemon.
???
On 11/15/2013 09:55 AM, Jon Medhurst (Tixy) wrote:
So, hacking getFatVolumeId to return a value other than -1 means the system works and files on 'external' sdcard are seen and usable by apps.
I guess the issue is that the file system is 'fuse' not 'vfat' and the fuse filesystem doesn't implement VFAT_IOCTL_GET_VOLUME_ID.
So it would seem logical that the config is wrong somehow the the non-emulated external media shouldn't be using fuse. However, the description at http://source.android.com/devices/tech/storage/index.html says:
Starting in Android 4.4, the owner, group and modes of files on external storage devices are now synthesized based on directory structure. [...] These synthesized permissions are accomplished by wrapping raw storage devices in a FUSE daemon.???
So I've just been barely following along here, but that quoted paragraph is a little frightening. Instead of using a filesystem with actual granular permissions, they've added a permissions meta-layer over FAT via fuse? Eww.
As far as the rest of the issue goes, it sounds like you've managed to work it out. But if it ends up being a problem with either of the GET_VOLUME_ID ioctls, let me know.
By the way, this sort of hiccup (and its workaround) is something we probably should be documenting and publicizing.
thanks -john
On Fri, 2013-11-15 at 11:08 -0800, John Stultz wrote:
On 11/15/2013 09:55 AM, Jon Medhurst (Tixy) wrote:
So, hacking getFatVolumeId to return a value other than -1 means the system works and files on 'external' sdcard are seen and usable by apps.
I guess the issue is that the file system is 'fuse' not 'vfat' and the fuse filesystem doesn't implement VFAT_IOCTL_GET_VOLUME_ID.
So it would seem logical that the config is wrong somehow the the non-emulated external media shouldn't be using fuse. However, the description at http://source.android.com/devices/tech/storage/index.html says:
Starting in Android 4.4, the owner, group and modes of files on external storage devices are now synthesized based on directory structure. [...] These synthesized permissions are accomplished by wrapping raw storage devices in a FUSE daemon.???
So I've just been barely following along here, but that quoted paragraph is a little frightening. Instead of using a filesystem with actual granular permissions, they've added a permissions meta-layer over FAT via fuse? Eww.
As far as the rest of the issue goes, it sounds like you've managed to work it out. But if it ends up being a problem with either of the GET_VOLUME_ID ioctls, let me know.
By the way, this sort of hiccup (and its workaround) is something we probably should be documenting and publicizing.
My 'work around' was to hard code the FileUtils.getFatVolumeID function on Android to return 0x12345678 for the volume ID, which is not something that I would want to commit to the Linaro tree. Not very good if there is more than one media partition, or if it isn't mounted yet (which comments in the code seem to anticipate) and I don't know where else that function might get used.
The only reason I tried the hack was to see if there were other issues with getting storage devices to actually work.
I'm still assuming we must have something configured wrong, unless KitKat is just simply broken and I'm the first person to notice.
I'll try an come up with a cleaner hack next week if I find time to understand all the uses of that function...
Hi Tixy,
I am still trying to get sdcard mounted on pandaboard and Arndale. For the current issue you are facing, I think its related to the one discussed in CM. Here is the thread: http://forum.xda-developers.com/showthread.php?t=2524316
Can you please tell if this fixes the issue. If so then we can pick this change: https://github.com/skeevy420/android_frameworks_base/commit/f028b00fbe78aa86...
On 16 November 2013 02:02, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Fri, 2013-11-15 at 11:08 -0800, John Stultz wrote:
On 11/15/2013 09:55 AM, Jon Medhurst (Tixy) wrote:
So, hacking getFatVolumeId to return a value other than -1 means the system works and files on 'external' sdcard are seen and usable by
apps.
I guess the issue is that the file system is 'fuse' not 'vfat' and the fuse filesystem doesn't implement VFAT_IOCTL_GET_VOLUME_ID.
So it would seem logical that the config is wrong somehow the the non-emulated external media shouldn't be using fuse. However, the description at http://source.android.com/devices/tech/storage/index.html says:
Starting in Android 4.4, the owner, group and modes of files on external storage devices are now synthesized based on directory structure. [...] These synthesized permissions are accomplished by wrapping raw storage devices in a FUSE daemon.???
So I've just been barely following along here, but that quoted paragraph is a little frightening. Instead of using a filesystem with actual granular permissions, they've added a permissions meta-layer over FAT via fuse? Eww.
As far as the rest of the issue goes, it sounds like you've managed to work it out. But if it ends up being a problem with either of the GET_VOLUME_ID ioctls, let me know.
By the way, this sort of hiccup (and its workaround) is something we probably should be documenting and publicizing.
My 'work around' was to hard code the FileUtils.getFatVolumeID function on Android to return 0x12345678 for the volume ID, which is not something that I would want to commit to the Linaro tree. Not very good if there is more than one media partition, or if it isn't mounted yet (which comments in the code seem to anticipate) and I don't know where else that function might get used.
The only reason I tried the hack was to see if there were other issues with getting storage devices to actually work.
I'm still assuming we must have something configured wrong, unless KitKat is just simply broken and I'm the first person to notice.
I'll try an come up with a cleaner hack next week if I find time to understand all the uses of that function...
-- Tixy
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
On Mon, 2013-11-18 at 14:56 +0530, Vishal Bhoj wrote:
Hi Tixy,
I am still trying to get sdcard mounted on pandaboard and Arndale. For the current issue you are facing, I think its related to the one discussed in CM. Here is the thread: http://forum.xda-developers.com/showthread.php?t=2524316
Can you please tell if this fixes the issue.
Unfortunately it doesn't fix the error I'm seeing.
Hi Jon,
On 18 November 2013 18:59, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Mon, 2013-11-18 at 14:56 +0530, Vishal Bhoj wrote:
Hi Tixy,
I am still trying to get sdcard mounted on pandaboard and Arndale. For
the
current issue you are facing, I think its related to the one discussed in CM. Here is the thread: http://forum.xda-developers.com/showthread.php?t=2524316
Can you please tell if this fixes the issue.
Unfortunately it doesn't fix the error I'm seeing.
I am seeing the same error on Arndale as well after fixing the sdcard detection issue. I will be trying it on Pandaboard which is using a 3.2 kernel. It will give a better clarity if the issue is related to upstream changes in newer kernel.
-- Tixy
On 16 November 2013 02:02, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Fri, 2013-11-15 at 11:08 -0800, John Stultz wrote:
On 11/15/2013 09:55 AM, Jon Medhurst (Tixy) wrote:
So, hacking getFatVolumeId to return a value other than -1 means
the
system works and files on 'external' sdcard are seen and usable by
apps.
I guess the issue is that the file system is 'fuse' not 'vfat' and
the
fuse filesystem doesn't implement VFAT_IOCTL_GET_VOLUME_ID.
So it would seem logical that the config is wrong somehow the the non-emulated external media shouldn't be using fuse. However, the description at http://source.android.com/devices/tech/storage/index.html says:
Starting in Android 4.4, the owner, group and modes offiles on
external storage devices are now synthesized based ondirectory
structure. [...] These synthesized permissions areaccomplished
by wrapping raw storage devices in a FUSE daemon.???
So I've just been barely following along here, but that quoted
paragraph
is a little frightening. Instead of using a filesystem with actual granular permissions, they've added a permissions meta-layer over FAT via fuse? Eww.
As far as the rest of the issue goes, it sounds like you've managed
to
work it out. But if it ends up being a problem with either of the GET_VOLUME_ID ioctls, let me know.
By the way, this sort of hiccup (and its workaround) is something we probably should be documenting and publicizing.
My 'work around' was to hard code the FileUtils.getFatVolumeID function on Android to return 0x12345678 for the volume ID, which is not something that I would want to commit to the Linaro tree. Not very good if there is more than one media partition, or if it isn't mounted yet (which comments in the code seem to anticipate) and I don't know where else that function might get used.
The only reason I tried the hack was to see if there were other issues with getting storage devices to actually work.
I'm still assuming we must have something configured wrong, unless KitKat is just simply broken and I'm the first person to notice.
I'll try an come up with a cleaner hack next week if I find time to understand all the uses of that function...
-- Tixy
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
On Fri, 2013-11-15 at 11:17 +0000, Jon Medhurst (Tixy) wrote:
Can anyone provide clues to help me get 'sdcard' partition working on Linaro Android KitKat images? Here's where I've got to so far...
After updating the vexpress storage config for KitKat [1] the sdcard partition on Linaro Android images is getting mounted by vold, but apps can't access this because android.process.media keeps crashing in attachVolume because FileUtils.getFatVolumeId is returning -1.
With MediaProvider hacked to provide verbose logging I get:
V/MediaProvider( 2825): /storage/sdcard0 volume ID: -1 E/MediaProvider( 2825): Can't obtain external volume ID even though it's mounted.
and I've verified the -1 comes from the call ioctl(fd, VFAT_IOCTL_GET_VOLUME_ID) in android_os_FileUtils_getFatVolumeId in frameworks/base/core/jni/android_os_FileUtils.cpp.
Running 'mount' gives...
rootfs / rootfs rw,relatime 0 0 tmpfs /dev tmpfs rw,nosuid,relatime,mode=755 0 0 devpts /dev/pts devpts rw,relatime,mode=600 0 0 proc /proc proc rw,relatime 0 0 sysfs /sys sysfs rw,relatime 0 0 tmpfs /mnt/secure tmpfs rw,relatime,mode=700 0 0 tmpfs /mnt/asec tmpfs rw,relatime,mode=755,gid=1000 0 0 tmpfs /mnt/obb tmpfs rw,relatime,mode=755,gid=1000 0 0 /sys/kernel/debug /sys/kernel/debug debugfs rw,relatime 0 0 /dev/block/sda2 /system ext4 ro,relatime,data=ordered 0 0 /dev/block/sda3 /cache ext4 rw,nosuid,nodev,noatime,errors=panic,data=ordered 0 0 /dev/block/sda5 /data ext4 rw,nosuid,nodev,noatime,errors=panic,data=ordered 0 0 nodev /dev/gator gatorfs rw,relatime 0 0 /dev/block/vold/8:6 /mnt/media_rw/sdcard0 vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0 /dev/block/vold/8:6 /mnt/secure/asec vfat rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro 0 0 /dev/fuse /storage/sdcard0 fuse rw,nosuid,nodev,relatime,user_id=1023,group_id=1023,default_permissions,allow_other 0 0
So first guess would be that fuse doesn't support this function, however as this seems to be the standard Android way of doing things, you would assume that it should work...?
So the error comes from fuse_dir_ioctl failing this check:
/* FUSE_IOCTL_DIR only supported for API version >= 7.18 */ if (fc->minor < 18) return -ENOTTY;
because fc->minor == 13. This version matches the fuse implementation in system/core/sdcard/fuse.h, and there doesn't appear to be any code in the sdcard daemon to handle things like faking up a FAT Volume Id.
I therefore think that KitKat is just plain broke, as the MediaProvider assumes removable media is a raw FAT volume, and the new way of doing things makes all access go through a fuse daemon.
So, this begs the question, why don't we just have the 'sdcard' partition of Linaro's images set as internal, i.e. android:removable="false" in storage_list.xml? After all, we can't actually physically remove the SD card as it also contains the system partition. Is there a use case where removable=true is required? (We had this in previous Android versions, I can't remember why)
This obviously isn't a solution for platforms which really do support a separate removable SD card, for that MediaProvider needs to be fixed.
On Mon, 2013-11-18 at 16:56 +0000, Jon Medhurst (Tixy) wrote: [...]
So, this begs the question, why don't we just have the 'sdcard' partition of Linaro's images set as internal, i.e. android:removable="false" in storage_list.xml?
Guess I wasn't explicit in my previous email, the above change works around the whole problem as it means MediaProvider doesn't attempt to call getFatVolumeId at all. It only does that for removable media because, I think, it's trying to keep track of the contents of multiple cards.
On 18 November 2013 22:48, Jon Medhurst (Tixy) tixy@linaro.org wrote:
On Mon, 2013-11-18 at 16:56 +0000, Jon Medhurst (Tixy) wrote: [...]
So, this begs the question, why don't we just have the 'sdcard' partition of Linaro's images set as internal, i.e. android:removable="false" in storage_list.xml?
Guess I wasn't explicit in my previous email, the above change works around the whole problem as it means MediaProvider doesn't attempt to call getFatVolumeId at all. It only does that for removable media because, I think, it's trying to keep track of the contents of multiple cards.
It makes sense now. This seems to be the immediate workaround for the issue. Android 4.4 seems to be working well for devices with emulated partition like Nexus series. This seems to be a genuine issue so it makes sense to raise the issue on android-platform list as well.
-- Tixy
linaro-android mailing list linaro-android@lists.linaro.org http://lists.linaro.org/mailman/listinfo/linaro-android
linaro-android@lists.linaro.org
-
John Stultz -
Jon Medhurst (Tixy) -
Vishal Bhoj