Hi,
we are looking at landing more and more full stack test cases for our automated board support status tracking efforts.
While for some hardware ports it's hard to test whether a port really gets a proper signal etc, we feel for audio this might be relatively straight forward: we got the idea that we could connect a cable from jack out to jack in in the lab and then have a testcase that plays something using aplay and checks that he gets proper input/signal on the jack in.
This could be done on alsa level and later pa level (for ubuntu).
A more advanced idea that came up when discussing options was to use opensource speech recognition like sphinx to even go one step further and see if the output we produce yields roughly the same input. For that we could play one or two words, use speech recognition to parse it and check if the resulting text is stable/expected.
What do you think?
Would MMWG be able to take experimenting and implementing such end-to-end audio test into their 11.10 work list?
Thanks for your support!
On 27 September 2011 09:18, Alexander Sack asac@linaro.org wrote:
Hi,
we are looking at landing more and more full stack test cases for our automated board support status tracking efforts.
While for some hardware ports it's hard to test whether a port really gets a proper signal etc, we feel for audio this might be relatively straight forward: we got the idea that we could connect a cable from jack out to jack in in the lab and then have a testcase that plays something using aplay and checks that he gets proper input/signal on the jack in.
This could be done on alsa level and later pa level (for ubuntu).
A more advanced idea that came up when discussing options was to use opensource speech recognition like sphinx to even go one step further and see if the output we produce yields roughly the same input. For that we could play one or two words, use speech recognition to parse it and check if the resulting text is stable/expected.
What do you think?
These are really good ideas. I had started a discussion with Torez several months ago about an automated test for audio. My idea at the time was to use a sine wav at a particular frequency and use or hack one of the tuner/freq analysis apps to detect the frequency. If it was too garbled or distorted, it wouldnt recognize the frequency.
As you know, sound quality is very subjective and depends on the cables, speakers, amp, etc. I like the speech recognition idea as well, for the same reasons. It might actually be a better test of the quality.
Would MMWG be able to take experimenting and implementing such end-to-end audio test into their 11.10 work list?
I think this is a really good idea to explore. Could we also maybe use camera and face recognition when we hack a pandaboard to do that? Hm...
I'll see if I can get a rough size for this. If not for 11.10, then we should definitely document a card for the next quarter and see what the TSC thinks.
Thanks for your support!
-- Alexander Sack Technical Director, Linaro Platform Teams http://www.linaro.org | Open source software for ARM SoCs http://twitter.com/#%21/linaroorg - http://www.linaro.org/linaro-blog
On Tue, Sep 27, 2011 at 5:16 PM, Kurt Taylor kurt.taylor@linaro.org wrote:
On 27 September 2011 09:18, Alexander Sack asac@linaro.org wrote:
Hi,
we are looking at landing more and more full stack test cases for our automated board support status tracking efforts.
While for some hardware ports it's hard to test whether a port really gets a proper signal etc, we feel for audio this might be relatively straight forward: we got the idea that we could connect a cable from jack out to jack in in the lab and then have a testcase that plays something using aplay and checks that he gets proper input/signal on the jack in.
This could be done on alsa level and later pa level (for ubuntu).
A more advanced idea that came up when discussing options was to use opensource speech recognition like sphinx to even go one step further and see if the output we produce yields roughly the same input. For that we could play one or two words, use speech recognition to parse it and check if the resulting text is stable/expected.
What do you think?
These are really good ideas. I had started a discussion with Torez several months ago about an automated test for audio. My idea at the time was to use a sine wav at a particular frequency and use or hack one of the tuner/freq analysis apps to detect the frequency. If it was too garbled or distorted, it wouldnt recognize the frequency.
As you know, sound quality is very subjective and depends on the cables, speakers, amp, etc. I like the speech recognition idea as well, for the same reasons. It might actually be a better test of the quality.
right. i think it would be hard to measure real audio quality, but if we get speech recognition going we would at least know that the input was similar enough to what we played.
I think some experiments with pocketsphinx would make sense to see how easy that would be. I am happy to create a blueprint for the first investigation steps for your backlog with a quick outline.
Would MMWG be able to take experimenting and implementing such end-to-end audio test into their 11.10 work list?
I think this is a really good idea to explore. Could we also maybe use camera and face recognition when we hack a pandaboard to do that? Hm...
psssst ... i wanted to keep that idea back for a bit :).
Hi Kurt,
On 27 September 2011 20:46, Kurt Taylor kurt.taylor@linaro.org wrote:
These are really good ideas. I had started a discussion with Torez several months ago about an automated test for audio. My idea at the time was to use a sine wav at a particular frequency and use or hack one of the tuner/freq analysis apps to detect the frequency. If it was too garbled or distorted, it wouldnt recognize the frequency.
I think, too distorted sine-wave is an easy kill and most probably wouldn't even leave the developers desk. Usually when a developer calls it done, the driver plays clean enough audio.
In my experience, a commercial quality issue would show up in the form of random/rare phase jumps in the sine wave and/or stability in the long run. While I always tested it manually, I think it should be possible to write a simple "sine-verify" application(or an alsa plugin?) that would analyze the captured stream.
my two paise. -jassi
Hi
I have created in the backlog two blueprints to split the basic case from the ASR case:
https://blueprints.launchpad.net/linaro-multimedia-project/+spec/linaro-mmwg...
https://blueprints.launchpad.net/linaro-multimedia-project/+spec/linaro-mmwg...
We should assign those and fill in the details / specification.
Cheers, Ilias
On 27/09/11 17:18, Alexander Sack wrote:
Hi,
we are looking at landing more and more full stack test cases for our automated board support status tracking efforts.
While for some hardware ports it's hard to test whether a port really gets a proper signal etc, we feel for audio this might be relatively straight forward: we got the idea that we could connect a cable from jack out to jack in in the lab and then have a testcase that plays something using aplay and checks that he gets proper input/signal on the jack in.
This could be done on alsa level and later pa level (for ubuntu).
A more advanced idea that came up when discussing options was to use opensource speech recognition like sphinx to even go one step further and see if the output we produce yields roughly the same input. For that we could play one or two words, use speech recognition to parse it and check if the resulting text is stable/expected.
What do you think?
Would MMWG be able to take experimenting and implementing such end-to-end audio test into their 11.10 work list?
Thanks for your support!
linaro-multimedia@lists.linaro.org
-
Alexander Sack -
Ilias Biris -
Jassi Brar -
Kurt Taylor