"Edgar E. Iglesias" <edgar.iglesias@amd.com> writes:

> Hi all,
>
> This the RFC I'm preparing to send to upstream QEMU for initial RFC review.
> I didn't see much precedence for Signed-off-by lines in cover letter so instead
> I'm thinking to explicitely CC Bill and Alex and anyone else who would like to be
> on copy, just let me know.
>
> Changes from v1:
> * Move VIRTIO_MSG_VENDOR_ID to virtio-msg.c
> * Update to match recent spec changes (token + set/get_vqueue padding)
> * Add endian conversion of dev_num and msg_size
> * Add instructions for running on GPEX PCI x86 microVM and ARM virt
> * Add missing spsc_queue.h
>
> -------------------
>
> This adds virtio-msg, a new virtio transport. Virtio-msg works by
> exchanging messages over a bus and doesn't rely on trapping and emulating
> making it a good fit for a number of applications such as AMP, real-time
> and safety applications.
>
> Together with the new transport, this series adds a PCI device that
> implements an AMP setup much like it would look if two SoC's would
> use virtio-msg across a PCI link.
>
> Current limitations:
> We only support a single device per bus (dev_num = 0).
> Shared memory queue layout likely to change in the future.
> Temporarily uses PCI Vendor Xilinx / Device 0x9039.
> Missing documentation.

Why can't the module be built-in?

It hangs in this case:

  ACPI: button: Power Button [PWRB]
  virtio_msg_amp_pci 0000:00:03.0: enabling device (0000 -> 0002)
  virtio_msg_amp_pci 0000:00:03.0: device_name=0000:00:03.0
  virtio_msg_amp_pci 0000:00:03.0: mmr (BAR0) at 0x0000000010040000, size 0x0000000000004000
  virtio_msg_amp_pci 0000:00:03.0: mmr (BAR1) at 0x0000000010044000, size 0x0000000000004000
  ram=(____ptrval____)
  virtio_msg_amp_pci 0000:00:03.0: SHMEM @ 0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  virtio_msg_amp_pci 0000:00:03.0: get bus name for dev_id=0

tracing through we get to:

  transport_msg_prepare (vmdev=0xffffff80c0a9c8c0, msg_id=2 '\002', payload_size=0) at /home/alex/lsrc/linux.git/drivers/virtio/virtio_msg.c:54
  54              msg_prepare(vmdev->request, false, msg_id, vmdev->dev_id, payload_size);
  (gdb) n
  106             transport_msg_prepare(vmdev, VIRTIO_MSG_DEVICE_INFO, 0);
  (gdb)
  ^C
  Thread 4 received signal SIGINT, Interrupt.
  [Switching to Thread 1.4]
  cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
  32              arm_cpuidle_restore_irq_context(&context);
  (gdb) bt
  #0  cpu_do_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:32
  #1  0xffffffc0813a1a40 in arch_cpu_idle () at /home/alex/lsrc/linux.git/arch/arm64/kernel/idle.c:44
  #2  0xffffffc0813a1ba8 in default_idle_call () at /home/alex/lsrc/linux.git/kernel/sched/idle.c:122
  #3  0xffffffc08012baa8 in cpuidle_idle_call () at /home/alex/lsrc/linux.git/kernel/sched/idle.c:190
  #4  do_idle () at /home/alex/lsrc/linux.git/kernel/sched/idle.c:330
  #5  0xffffffc08012bd74 in cpu_startup_entry (state=state@entry=CPUHP_AP_ONLINE_IDLE) at /home/alex/lsrc/linux.git/kernel/sched/idle.c:428
  #6  0xffffffc080048798 in secondary_start_kernel () at /home/alex/lsrc/linux.git/arch/arm64/kernel/smp.c:271
  #7  0xffffffc080056078 in __secondary_switched () at /home/alex/lsrc/linux.git/arch/arm64/kernel/head.S:404
  Backtrace stopped: previous frame identical to this frame (corrupt stack?)

but I can't see where the kernel would trigger a schedule....

>
> The virtio-msg spec:
> https://github.com/Linaro/virtio-msg-spec/
>
> QEMU with these patches:
> https://github.com/edgarigl/qemu/tree/edgar/virtio-msg-rfc
>
> Linux with virtio-msg suppport:
> https://github.com/edgarigl/linux/tree/edgari/virtio-msg-6.17
>
> To try it, first build Linux with the following enabled:
> CONFIG_VIRTIO_MSG=y
> CONFIG_VIRTIO_MSG_AMP=y
> CONFIG_VIRTIO_MSG_AMP_PCI=y
>
> Boot linux in QEMU with a virtio-msg-amp-pci device, in this example
> with a virtio-net device attached to it:
>
> x86/q35 machine:
>  -device virtio-msg-amp-pci
>  -device virtio-net-device,netdev=n1,bus=/q35-pcihost/pcie.0/virtio-msg-amp-pci/vmsg.0
>  -netdev user,id=n1
>
> x86/microvm or ARM virt machines:
>  -device virtio-msg-amp-pci
>  -device virtio-net-device,netdev=n1,bus=/gpex-pcihost/pcie.0/virtio-msg-amp-pci/vmsg.0/virtio-msg/virtio-msg-proxy-bus.0
>  -netdev user,id=n1
>
> Cheers,
> Edgar
>
> Edgar E. Iglesias (4):
>   virtio: Introduce notify_queue
>   virtio: Add virtio_queue_get_rings
>   virtio: Add the virtio-msg transport
>   virtio-msg-bus: amp-pci: Add generic AMP PCI device
>
>  hw/misc/Kconfig                     |   7 +
>  hw/misc/meson.build                 |   1 +
>  hw/misc/virtio-msg-amp-pci.c        | 324 ++++++++++++
>  hw/virtio/Kconfig                   |   4 +
>  hw/virtio/meson.build               |   5 +
>  hw/virtio/virtio-msg-bus.c          |  89 ++++
>  hw/virtio/virtio-msg.c              | 598 ++++++++++++++++++++++
>  hw/virtio/virtio.c                  |  23 +
>  include/hw/virtio/spsc_queue.h      | 213 ++++++++
>  include/hw/virtio/virtio-bus.h      |   1 +
>  include/hw/virtio/virtio-msg-bus.h  |  95 ++++
>  include/hw/virtio/virtio-msg-prot.h | 749 ++++++++++++++++++++++++++++
>  include/hw/virtio/virtio-msg.h      |  45 ++
>  include/hw/virtio/virtio.h          |   2 +
>  14 files changed, 2156 insertions(+)
>  create mode 100644 hw/misc/virtio-msg-amp-pci.c
>  create mode 100644 hw/virtio/virtio-msg-bus.c
>  create mode 100644 hw/virtio/virtio-msg.c
>  create mode 100644 include/hw/virtio/spsc_queue.h
>  create mode 100644 include/hw/virtio/virtio-msg-bus.h
>  create mode 100644 include/hw/virtio/virtio-msg-prot.h
>  create mode 100644 include/hw/virtio/virtio-msg.h

--
Alex Bennée
Virtualisation Tech Lead @ Linaro