risu instruction set test harness now publicly available

[Peter Maydell]

On 8 December 2010 12:57, Dave Martin <dave.martin at linaro.org> wrote:
> On Wed, Dec 8, 2010 at 12:04 PM, Peter Maydell <peter.maydell at linaro.org> wrote:
>> Hi. As part of my work on qemu I've written a simplistic random instruction
>> sequence generator and test harness.

> I'd thought about this sort of thing before, but there were some
> interesting challenges...
>
> Is it random random, or does it try to mould to the architectural envelope?
>
> For example, randomly choosing opcode and shift operation bits in ALU
> instructions will exercise the interpreter much more interestingly
> than randomly choosing the offset bits in a branch instruction; and
> some potentially highly interesting and error-prone instructions like
> barriers, WFE/SEV, and all that kind of thing will get basically no
> testing from a random-number generator, since they tend to map to a
> single, fixed bit pattern.

As I say, it's pretty simplistic: I was aiming for a "good enough" tool
that would let me find and test qemu bugs. The generator works by:
 * pick a pattern at random from all the ones it's using
 * generate a random 32 (or 16) bits
 * mask in the should-be-zero/one bits
 * apply constraints (eg regs never r13 or r15, any user-specified constraint);
   if the constraint is broken then just restart with a new random number
 * force the cond field to "always" a configurable proportion of the time

So the effect is that the testing will be distributed evenly over the
given instructions even if some take a much smaller share of
encoding space.

However for other reasons the tool isn't really suited for testing weird
things like WFE, SEV or barriers -- there's a disclaimer about this
in the README. I think for that sort of thing you'd do much better
with a rather more directed kind of testing.

-- PMM