On Mon, Jan 24, 2011 at 12:27:16PM -0000, Pawel Moll wrote:
So - we'll try to enlarge FIFO. For the moment - playing with interrupts affinity seem to be a viable workaround.
I don't think enlarging the FIFO will help too much. The issue is whether the CPU can keep up with the data rate coming off the card. If it can't, then no matter how large the FIFO is, it will eventually overflow.
The real answer is to avoid PIO mode, and use DMA support. However, I've had problems using DMA on the ARM development boards. You can find details my DMA issues internally within ARM by talking to Catalin.
The alternative answer, I believe implemented by some of ARMs silicon partners, is to turn the card clock off when the FIFO becomes full/empty to stop it sending more data. I think this violates some of the MMC/SD requirements, but it seems to work for the silicon partners just fine.
I don't think enlarging the FIFO will help too much. The issue is whether the CPU can keep up with the data rate coming off the card. If it can't, then no matter how large the FIFO is, it will eventually overflow.
Yes, I realize that. But so far the only time when problem happens is the timeout caused by ISP1761 handler. If we substantially increase the FIFO depth, we'll just have much more margin - enough to "mostly work". To my taste, the 1.3ms required to service the USB interrupt is already waaay to long. I'd consider any substantially larger latencies pathological, so making sure that we have margin of 3, maybe even 5 ms sounds good enough to me.
It's not about making this interface perfect. This won't happen, I afraid :-( It's just about making it good enough :-)
The real answer is to avoid PIO mode, and use DMA support. However, I've had problems using DMA on the ARM development boards. You can
Well, using DMA won't be easy on VE, will it? ;-) Besides even with this, in some extreme situation, the bus could be potentially stalled long enough to cause an underrun. Yes, very unlikely, but not impossible.
The alternative answer, I believe implemented by some of ARMs silicon partners, is to turn the card clock off when the FIFO becomes full/empty to stop it sending more data. I think this violates some of the MMC/SD requirements, but it seems to work for the silicon partners just fine.
Oh no, that's absolutely legal - see JEDEC 84-A441, p. 7.7: "The MultiMediaCard bus clock signal can be used by the host to put the card into energy saving mode, or to control the data flow (to avoid under-run or over-run conditions) on the bus."
So this would be the technically correct fix. The problem is that this would require even more MMCI modifications then enlarging FIFO, so it's unlikely to happen.
Paweł
-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
On Mon, Jan 24, 2011 at 04:13:14PM +0000, Pawel Moll wrote:
I don't think enlarging the FIFO will help too much. The issue is whether the CPU can keep up with the data rate coming off the card. If it can't, then no matter how large the FIFO is, it will eventually overflow.
Yes, I realize that. But so far the only time when problem happens is the timeout caused by ISP1761 handler.
That's fixable by having the block driver retry.
I have patches lodged with cjb which rework the MMC block driver error handling, and I now have MMC rootfs entirely usable with the ISP1761 driver. With ISP1761 also fixed (god knows why the driver maintainer isn't doing anything with that patch) then the problem goes away.
So really this is a non-issue.
If we substantially increase the FIFO depth, we'll just have much more margin - enough to "mostly work". To my taste, the 1.3ms required to service the USB interrupt is already waaay to long. I'd consider any substantially larger latencies pathological, so making sure that we have margin of 3, maybe even 5 ms sounds good enough to me.
Well, fixing MMCI because ISP1761 is buggy is not the way forward. The answer is to fix the broken ISP1761 driver.
We know as it currently stands in mainline that it's utterly unusable with certain serial devices. That doesn't mean we go off and fix MMCI.
The real answer is to avoid PIO mode, and use DMA support. However, I've had problems using DMA on the ARM development boards. You can
Well, using DMA won't be easy on VE, will it? ;-) Besides even with this, in some extreme situation, the bus could be potentially stalled long enough to cause an underrun. Yes, very unlikely, but not impossible.
I'm running SD rootfs alongside NFS/ssh/NTP and the buggy ISP1761. With the pile of 'fixes' patches I have here (which are currently stuck with various maintainers) I have a completely stable system.
I've also had it running gstreamer on CLCD with AACI too, again with SD rootfs. Provided the video is already loaded off the SD card (because the transfer rate is too slow) it's fine.
The only remaining problem I have with it is !"£$%% busybox shell which doesn't like readonly rootfs with command history. The last command is /sbin/reboot - I'm sure you can imagine what keeps on happening, particularly at the most annoying points in time.
The alternative answer, I believe implemented by some of ARMs silicon partners, is to turn the card clock off when the FIFO becomes full/empty to stop it sending more data. I think this violates some of the MMC/SD requirements, but it seems to work for the silicon partners just fine.
Oh no, that's absolutely legal - see JEDEC 84-A441, p. 7.7: "The MultiMediaCard bus clock signal can be used by the host to put the card into energy saving mode, or to control the data flow (to avoid under-run or over-run conditions) on the bus."
So this would be the technically correct fix. The problem is that this would require even more MMCI modifications then enlarging FIFO, so it's unlikely to happen.
Well as I see it, it's pointless enlarging the FIFO. ARM Ltd isn't going to give me updated FPGA images for the Integrator/CP, Versatile PB926, Realview EB and Versatile Express.
Given that the problem is already fixed via a set of patches, I see no reason to mess about with the hardware, thereby making the driver more complicated for *no* benefit.
On Mon, Jan 24, 2011 at 04:24:00PM +0000, Russell King - ARM Linux wrote:
On Mon, Jan 24, 2011 at 04:13:14PM +0000, Pawel Moll wrote:
The real answer is to avoid PIO mode, and use DMA support. However, I've had problems using DMA on the ARM development boards. You can
Well, using DMA won't be easy on VE, will it? ;-) Besides even with this, in some extreme situation, the bus could be potentially stalled long enough to cause an underrun. Yes, very unlikely, but not impossible.
I'm running SD rootfs alongside NFS/ssh/NTP and the buggy ISP1761. With the pile of 'fixes' patches I have here (which are currently stuck with various maintainers) I have a completely stable system.
I've also had it running gstreamer on CLCD with AACI too, again with SD rootfs. Provided the video is already loaded off the SD card (because the transfer rate is too slow) it's fine.
The only remaining problem I have with it is !"£$%% busybox shell which doesn't like readonly rootfs with command history. The last command is /sbin/reboot - I'm sure you can imagine what keeps on happening, particularly at the most annoying points in time.
I should also point out that my Versatile Express has been on continuously for the last two to three weeks, running off SD rootfs, and since fixing the ISP1761 driver and MMC block driver error handling there's been zero issues with MMC or ISP1761.
So, MMCI overruns is now a complete non-issue for me.
Given that the problem is already fixed via a set of patches, I see no reason to mess about with the hardware, thereby making the driver more complicated for *no* benefit.
There will be virtually no change in the driver - all required stuff is already there. See the STE UX500 variant - they did exactly the same, enlarged the FIFO. One of the main reason I've immediately abandoned the idea of using FIQ was the need of changing the driver for no good reason.
Well, fixing MMCI because ISP1761 is buggy is not the way forward. The answer is to fix the broken ISP1761 driver.
Totally agree. The thing is that you are the only person right now who doesn't have any problems. The others can quickly benefit from the workaround. I think I made the status of this patch clear enough - let me quote myself: "HACK HACK HACK" ;-)
Well as I see it, it's pointless enlarging the FIFO. ARM Ltd isn't going to give me updated FPGA images for the Integrator/CP, Versatile PB926, Realview EB and Versatile Express.
If we succeed with this, you (and all other users) will get the "improved" FPGA image. But for VE only indeed.
Paweł
Following up on this:
2011/1/24 Russell King - ARM Linux linux@arm.linux.org.uk:
On Mon, Jan 24, 2011 at 12:27:16PM -0000, Pawel Moll wrote:
So - we'll try to enlarge FIFO. For the moment - playing with interrupts affinity seem to be a viable workaround.
I don't think enlarging the FIFO will help too much. The issue is whether the CPU can keep up with the data rate coming off the card. If it can't, then no matter how large the FIFO is, it will eventually overflow.
The real answer is to avoid PIO mode, and use DMA support. However, I've had problems using DMA on the ARM development boards. You can find details my DMA issues internally within ARM by talking to Catalin.
I fully agree with Russell, MMC by nature begs to be used with DMA. Hopefully PL330 does not have all the basic problems found in PL080/PL081, yet Samsung (som version) and ST-Ericsson Nomadik does use the PL080, albeit in modified versions.
The alternative answer, I believe implemented by some of ARMs silicon partners, is to turn the card clock off when the FIFO becomes full/empty to stop it sending more data. I think this violates some of the MMC/SD requirements, but it seems to work for the silicon partners just fine.
One of these fixes does not exclude the other.
We have this "hardware flow control" in U300, Nomadik and Ux500. Basically the clock to the card is simply gated if the FIFO risk to over/underflow.
To be precise, it gates the MCIFBCLK, MCICLKOUT and MCICLK to the card when either RX FIFO is full and DPSM is enabled, or TX FIFO is empty and DPSM is enabled. We do not mess with the internal MCLK clock.
We have some experience in not even DMA being quick enough to avoid overflows under all conditions, making it necessary to clock down the host undesirably low. Increasing FIFO depth will actually help to some extent in this case.
For example: the SD spec permits us to clock the card at something like 23 MHz and with 4 data lines and a standard 64 byte (16 word) FIFO this will fill up in 2.8 microseconds. So that, or rather half of it 1.3 us, is the maximum allowed interupt latency, unless you clock down the card.
The system IRQ latency is a swamp of heuristics unless you have things like FIQ or realtime patches as mentioned.
And as mentioned by Russell one way to mitigate the effect that would also benefit current Versatiles and RealViews would be to dynamically recalibrate the card clock with some error-feedback loop. (Would be pretty cool actually!)
Yours, Linus Walleij