This patchset proposes roadtest, a device-driver testing framework. Drivers are tested under User Mode Linux (UML) and interact with mocked/modelled hardware. The tests and hardware models are written in Python, the former using Python's built-in unittest framework.
Drivers are tested via their userspace interfaces. The hardware models allow tests to inject values into registers and assert that drivers control the hardware in the right way and react as expected to stimuli.
Roadtest is meant to be used for relatively simple drivers, such as the ones part of the IIO, regulator and RTC subsystems.
Questions and answers:
= Why do we need this?
There are a large amount of these kind of drivers in the kernel. Most of the hardware is not available in current CI systems so most drivers can only, at best, be build-tested there. Even basic soundness such as a driver successfully probing and binding to the devices it tries to be support cannot be tested. Drivers cannot be easily regression-tested to ensure that bugs fixed once do not get reintroduced.
Many drivers support multiple related hardware variants, and far from all patch submitters have access to all the variants which the driver that they are patching supports, so there is no way for them to easily verify that they haven't broken something basic on a variant which they do not own.
Furthermore, hardware can be used in many different configurations with drivers supporting many different devicetree properties, so even just having access to all the variants would be insufficient.
On top of that, some of the chips measure environmental conditions such as temperature, so testing extreme cases may not be simple even if one has access to the hardware.
All this makes development, modification, maintenance, and reviewing of these drivers harder than it necessarily needs to be. Roadtest hopes to make some of these things slightly easier by providing a framework to create hardware models/mocks and to write testcases which exercise drivers using these models.
= Do you have some specific examples of the kind of code this could be used to test?
Here is an example of a patch which can easily be regression-tested using roadtest (in fact, this series includes such a regression test) but is much harder to do so automatically with real hardware since it requires specific environmental conditions:
iio: light: opt3001: Fixed timeout error when 0 lux https://lore.kernel.org/lkml/20210920125351.6569-1-valek@2n.cz/
Here is another example. This driver has code which correctly parses a documented devicetree property (amstaos,proximity-diodes) but which then fails to actually communicate this setting to the hardware in any way. Such code can be easily tested with roadtest since the framework integrates devicetree support and provides functions to assert that drivers writes expected registers with expected values:
drivers/iio/light/tsl2772.c tsl2772_read_prox_diodes()
(Both the above examples happen to be from the same subsystem but that should in no way be taken to imply that such issues are unique to that subsystem or that that subsystem has more of them.)
= How does this relate to kselftests?
Tests in kselftests also test kernel code using the userspace interfaces, but that's about what's common between the frameworks. kselftests has other goals and does not provide any kind of mechanism for hardware mocking.
= How does this relate to kunit?
Kunit is for unit testing of functions in kernel code, and is not meant for testing kernel code via userspace interfaces. It could in theory be used to test some of the simple drivers too, but that would require (1) a large amount of mocking code in various kernel frameworks, and, more importantly, (2) refactoring of the drivers to be tested.
This can be contrasted with roadtest which works with mostly unmodified drivers and which mocks the hardware at the lowest level without having to change kernel frameworks.
= How do I use it?
See Documentation/dev-tools/roadtest.rst added by the documentation patch for more information about running and writing tests using this framework.
= What's included in the patchset?
The current framework allows developing tests for hardware which uses the I2C bus. Hardware models can also control GPIOs and use them to trigger interrupts.
This series includes tests for some IIO, regulator and RTC drivers. The regulator and RTC tests depend on a few driver patches which are either in review or in linux-next. These are noted in the commit messages.
The entire patch set, including the required dependencies, is also available in a git tree:
https://github.com/vwax/linux/commits/roadtest/rfc-v1
Cc: linux-kernel@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-um@lists.infradead.org
Cc: shuah@kernel.org Cc: brendanhiggins@google.com Cc: linux-kselftest@vger.kernel.org
Cc: jic23@kernel.org Cc: linux-iio@vger.kernel.org
Cc: lgirdwood@gmail.com Cc: broonie@kernel.org
Cc: a.zummo@towertech.it Cc: alexandre.belloni@bootlin.com Cc: linux-rtc@vger.kernel.org
Cc: corbet@lwn.net Cc: linux-doc@vger.kernel.org
Vincent Whitchurch (10): roadtest: import libvhost-user from QEMU roadtest: add C backend roadtest: add framework roadtest: add base config roadtest: add build files roadtest: add documentation iio: light: opt3001: add roadtest iio: light: vcnl4000: add roadtest regulator: tps62864: add roadtest rtc: pcf8563: add roadtest
Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/roadtest.rst | 669 ++++ tools/testing/roadtest/.gitignore | 2 + tools/testing/roadtest/Dockerfile | 25 + tools/testing/roadtest/Makefile | 84 + tools/testing/roadtest/init.sh | 19 + tools/testing/roadtest/pyproject.toml | 10 + tools/testing/roadtest/requirements.txt | 4 + tools/testing/roadtest/roadtest/__init__.py | 2 + .../roadtest/roadtest/backend/__init__.py | 0 .../roadtest/roadtest/backend/backend.py | 32 + .../testing/roadtest/roadtest/backend/gpio.py | 111 + .../testing/roadtest/roadtest/backend/i2c.py | 123 + .../testing/roadtest/roadtest/backend/main.py | 13 + .../testing/roadtest/roadtest/backend/mock.py | 20 + .../roadtest/roadtest/backend/test_gpio.py | 98 + .../roadtest/roadtest/backend/test_i2c.py | 84 + .../testing/roadtest/roadtest/cmd/__init__.py | 0 tools/testing/roadtest/roadtest/cmd/main.py | 146 + tools/testing/roadtest/roadtest/cmd/remote.py | 48 + .../roadtest/roadtest/core/__init__.py | 0 .../testing/roadtest/roadtest/core/control.py | 52 + .../roadtest/roadtest/core/devicetree.py | 155 + .../roadtest/roadtest/core/hardware.py | 94 + tools/testing/roadtest/roadtest/core/log.py | 42 + .../testing/roadtest/roadtest/core/modules.py | 38 + .../testing/roadtest/roadtest/core/opslog.py | 35 + tools/testing/roadtest/roadtest/core/proxy.py | 48 + tools/testing/roadtest/roadtest/core/suite.py | 286 ++ tools/testing/roadtest/roadtest/core/sysfs.py | 77 + .../roadtest/roadtest/core/test_control.py | 35 + .../roadtest/roadtest/core/test_devicetree.py | 31 + .../roadtest/roadtest/core/test_hardware.py | 41 + .../roadtest/roadtest/core/test_log.py | 54 + .../roadtest/roadtest/core/test_opslog.py | 27 + .../roadtest/roadtest/tests/__init__.py | 0 .../roadtest/roadtest/tests/base/config | 84 + .../roadtest/roadtest/tests/iio/__init__.py | 0 .../roadtest/roadtest/tests/iio/config | 1 + .../roadtest/roadtest/tests/iio/iio.py | 112 + .../roadtest/tests/iio/light/__init__.py | 0 .../roadtest/roadtest/tests/iio/light/config | 2 + .../roadtest/tests/iio/light/test_opt3001.py | 95 + .../roadtest/tests/iio/light/test_vcnl4000.py | 132 + .../roadtest/tests/iio/light/test_vcnl4010.py | 282 ++ .../roadtest/tests/iio/light/test_vcnl4040.py | 104 + .../roadtest/tests/iio/light/test_vcnl4200.py | 96 + .../roadtest/tests/regulator/__init__.py | 0 .../roadtest/roadtest/tests/regulator/config | 4 + .../roadtest/tests/regulator/test_tps62864.py | 187 ++ .../roadtest/roadtest/tests/rtc/__init__.py | 0 .../roadtest/roadtest/tests/rtc/config | 1 + .../roadtest/roadtest/tests/rtc/rtc.py | 73 + .../roadtest/tests/rtc/test_pcf8563.py | 348 ++ tools/testing/roadtest/src/.gitignore | 1 + tools/testing/roadtest/src/backend.c | 884 +++++ .../src/libvhost-user/include/atomic.h | 310 ++ .../src/libvhost-user/libvhost-user.c | 2885 +++++++++++++++++ .../src/libvhost-user/libvhost-user.h | 691 ++++ 59 files changed, 8798 insertions(+) create mode 100644 Documentation/dev-tools/roadtest.rst create mode 100644 tools/testing/roadtest/.gitignore create mode 100644 tools/testing/roadtest/Dockerfile create mode 100644 tools/testing/roadtest/Makefile create mode 100755 tools/testing/roadtest/init.sh create mode 100644 tools/testing/roadtest/pyproject.toml create mode 100644 tools/testing/roadtest/requirements.txt create mode 100644 tools/testing/roadtest/roadtest/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/backend.py create mode 100644 tools/testing/roadtest/roadtest/backend/gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/i2c.py create mode 100644 tools/testing/roadtest/roadtest/backend/main.py create mode 100644 tools/testing/roadtest/roadtest/backend/mock.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_i2c.py create mode 100644 tools/testing/roadtest/roadtest/cmd/__init__.py create mode 100644 tools/testing/roadtest/roadtest/cmd/main.py create mode 100644 tools/testing/roadtest/roadtest/cmd/remote.py create mode 100644 tools/testing/roadtest/roadtest/core/__init__.py create mode 100644 tools/testing/roadtest/roadtest/core/control.py create mode 100644 tools/testing/roadtest/roadtest/core/devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/log.py create mode 100644 tools/testing/roadtest/roadtest/core/modules.py create mode 100644 tools/testing/roadtest/roadtest/core/opslog.py create mode 100644 tools/testing/roadtest/roadtest/core/proxy.py create mode 100644 tools/testing/roadtest/roadtest/core/suite.py create mode 100644 tools/testing/roadtest/roadtest/core/sysfs.py create mode 100644 tools/testing/roadtest/roadtest/core/test_control.py create mode 100644 tools/testing/roadtest/roadtest/core/test_devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/test_hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/test_log.py create mode 100644 tools/testing/roadtest/roadtest/core/test_opslog.py create mode 100644 tools/testing/roadtest/roadtest/tests/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/base/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/iio.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/config create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/config create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/rtc.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py create mode 100644 tools/testing/roadtest/src/.gitignore create mode 100644 tools/testing/roadtest/src/backend.c create mode 100644 tools/testing/roadtest/src/libvhost-user/include/atomic.h create mode 100644 tools/testing/roadtest/src/libvhost-user/libvhost-user.c create mode 100644 tools/testing/roadtest/src/libvhost-user/libvhost-user.h
Add the C parts of the roadtest framework. This uses QEMU's libvhost-user to implement the device side of virtio-user and virtio-gpio and bridge them to the Python portions of the backend.
The C backend is also responsible for starting UML after the virtio device implementations are initialized.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- tools/testing/roadtest/src/backend.c | 884 +++++++++++++++++++++++++++ 1 file changed, 884 insertions(+) create mode 100644 tools/testing/roadtest/src/backend.c
diff --git a/tools/testing/roadtest/src/backend.c b/tools/testing/roadtest/src/backend.c new file mode 100644 index 000000000000..d5ac08b20fd9 --- /dev/null +++ b/tools/testing/roadtest/src/backend.c @@ -0,0 +1,884 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright Axis Communications AB + +#define PY_SSIZE_T_CLEAN +#include <Python.h> + +#include <err.h> +#include <getopt.h> +#include <stdlib.h> +#include <sys/epoll.h> +#include <sys/socket.h> +#include <sys/types.h> +#include <sys/uio.h> +#include <sys/un.h> +#include <unistd.h> +#include <stdio.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <linux/virtio_gpio.h> +#include <linux/virtio_i2c.h> +#include <linux/kernel.h> +#include <linux/list.h> + +#include "libvhost-user.h" + +enum watch_type { + LISTEN, + SOCKET_WATCH, + VU_WATCH, +}; + +struct watch { + VuDev *dev; + enum watch_type type; + int fd; + void *func; + void *data; + struct list_head list; +}; + +struct vhost_user_i2c { + VuDev dev; + FILE *control; +}; + +struct vhost_user_gpio { + VuDev dev; + FILE *control; + VuVirtqElement *irq_elements[64]; +}; + +#define dbg(...) \ + do { \ + if (0) { \ + fprintf(stderr, __VA_ARGS__); \ + } \ + } while (0) + +static LIST_HEAD(watches); + +static int epfd; + +static PyObject *py_i2c_read, *py_i2c_write, *py_process_control; +static PyObject *py_gpio_set_irq_type, *py_gpio_unmask; + +static const char *opt_main_script; +static char *opt_gpio_socket; +static char *opt_i2c_socket; + +static struct vhost_user_gpio gpio; +static struct vhost_user_i2c i2c; + +static void dump_iov(const char *what, struct iovec *iovec, unsigned int count) +{ + int i; + + dbg("dumping %s with count %u\n", what, count); + + for (i = 0; i < count; i++) { + struct iovec *iov = &iovec[0]; + + dbg("i %d base %p len %zu\n", i, iov->iov_base, iov->iov_len); + } +} + +static bool i2c_read(struct vhost_user_i2c *vi, uint16_t addr, void *data, + size_t len) +{ + PyObject *pArgs, *pValue; + + dbg("i2c read addr %#x len %zu\n", addr, len); + + pArgs = PyTuple_New(1); + pValue = PyLong_FromLong(len); + PyTuple_SetItem(pArgs, 0, pValue); + + pValue = PyObject_CallObject(py_i2c_read, pArgs); + Py_DECREF(pArgs); + if (!pValue) { + PyErr_Print(); + return false; + } + + unsigned char *buffer; + Py_ssize_t length; + + if (PyBytes_AsStringAndSize(pValue, (char **)&buffer, &length) < 0) { + PyErr_Print(); + errx(1, "invalid result from i2c.read()"); + } + if (length != len) { + errx(1, + "unexpected length from i2c.read(), expected %zu, got %zu", + len, length); + } + + memcpy(data, buffer, len); + + return true; +} + +static bool i2c_write(struct vhost_user_i2c *vi, uint16_t addr, + const void *data, size_t len) +{ + PyObject *pArgs, *pValue; + + dbg("i2c write addr %#x len %zu\n", addr, len); + + pArgs = PyTuple_New(1); + pValue = PyBytes_FromStringAndSize(data, len); + PyTuple_SetItem(pArgs, 0, pValue); + + pValue = PyObject_CallObject(py_i2c_write, pArgs); + Py_DECREF(pArgs); + if (!pValue) { + PyErr_Print(); + return false; + } + + return true; +} + +static void gpio_send_irq_response(struct vhost_user_gpio *gpio, + unsigned int pin, unsigned int status); + +static PyObject *cbackend_trigger_gpio_irq(PyObject *self, PyObject *args) +{ + unsigned int pin; + + if (!PyArg_ParseTuple(args, "I", &pin)) + return NULL; + + dbg("trigger gpio %u irq\n", pin); + + gpio_send_irq_response(&gpio, pin, VIRTIO_GPIO_IRQ_STATUS_VALID); + + Py_RETURN_NONE; +} + +static PyMethodDef EmbMethods[] = { + { "trigger_gpio_irq", cbackend_trigger_gpio_irq, METH_VARARGS, + "Return the number of arguments received by the process." }, + { NULL, NULL, 0, NULL } +}; + +static PyModuleDef EmbModule = { PyModuleDef_HEAD_INIT, + "cbackend", + NULL, + -1, + EmbMethods, + NULL, + NULL, + NULL, + NULL }; + +static PyObject *PyInit_cbackend(void) +{ + return PyModule_Create(&EmbModule); +} + +static void init_python_i2c(PyObject *backend) +{ + PyObject *i2c = PyObject_GetAttrString(backend, "i2c"); + + if (!i2c) { + PyErr_Print(); + errx(1, "Error getting backend.i2c"); + } + + py_i2c_read = PyObject_GetAttrString(i2c, "read"); + if (!py_i2c_read) { + PyErr_Print(); + errx(1, "Error getting i2c.read"); + } + + py_i2c_write = PyObject_GetAttrString(i2c, "write"); + if (!py_i2c_write) { + PyErr_Print(); + errx(1, "Error getting i2c.write"); + } +} + +static void init_python_gpio(PyObject *backend) +{ + PyObject *gpio = PyObject_GetAttrString(backend, "gpio"); + + if (!gpio) { + PyErr_Print(); + errx(1, "error getting backend.gpio"); + } + + py_gpio_set_irq_type = PyObject_GetAttrString(gpio, "set_irq_type"); + if (!py_gpio_set_irq_type) { + PyErr_Print(); + errx(1, "error getting gpio.set_irq_type"); + } + + py_gpio_unmask = PyObject_GetAttrString(gpio, "unmask"); + if (!py_gpio_unmask) { + PyErr_Print(); + errx(1, "error getting gpio.unmask"); + } +} + +static void init_python(void) +{ + PyObject *mainmod, *backend; + FILE *file; + + PyImport_AppendInittab("cbackend", &PyInit_cbackend); + + Py_Initialize(); + + file = fopen(opt_main_script, "r"); + if (!file) + err(1, "open %s", opt_main_script); + + if (PyRun_SimpleFile(file, "main.py") < 0) { + PyErr_Print(); + errx(1, "error running %s", opt_main_script); + } + fclose(file); + + mainmod = PyImport_AddModule("__main__"); + if (!mainmod) { + PyErr_Print(); + errx(1, "error getting __main__"); + } + + backend = PyObject_GetAttrString(mainmod, "backend"); + if (!backend) { + PyErr_Print(); + errx(1, "error getting backend"); + } + + py_process_control = PyObject_GetAttrString(backend, "process_control"); + if (!py_process_control) { + PyErr_Print(); + errx(1, "error getting backend.process_control"); + } + + init_python_i2c(backend); + init_python_gpio(backend); +} + +static void i2c_handle_cmdq(VuDev *dev, int qidx) +{ + struct vhost_user_i2c *vi = + container_of(dev, struct vhost_user_i2c, dev); + VuVirtq *vq = vu_get_queue(dev, qidx); + VuVirtqElement *elem; + + for (;;) { + struct virtio_i2c_out_hdr *hdr; + struct iovec *resultv; + size_t used = 0; + bool ok = true; + + elem = vu_queue_pop(dev, vq, sizeof(VuVirtqElement)); + if (!elem) + break; + + dbg("elem %p index %u out_num %u in_num %u\n", elem, + elem->index, elem->out_num, elem->in_num); + dump_iov("out", elem->out_sg, elem->out_num); + dump_iov("in", elem->in_sg, elem->in_num); + + assert(elem->out_sg[0].iov_len == sizeof(*hdr)); + hdr = elem->out_sg[0].iov_base; + + if (elem->out_num == 2 && elem->in_num == 1) { + struct iovec *data = &elem->out_sg[1]; + + ok = i2c_write(vi, hdr->addr, data->iov_base, + data->iov_len); + resultv = &elem->in_sg[0]; + } else if (elem->out_num == 1 && elem->in_num == 2) { + struct iovec *data = &elem->in_sg[0]; + + ok = i2c_read(vi, hdr->addr, data->iov_base, + data->iov_len); + resultv = &elem->in_sg[1]; + used += data->iov_len; + } else { + assert(false); + } + + struct virtio_i2c_in_hdr *inhdr = resultv->iov_base; + + inhdr->status = ok ? VIRTIO_I2C_MSG_OK : VIRTIO_I2C_MSG_ERR; + + used += sizeof(*inhdr); + vu_queue_push(dev, vq, elem, used); + free(elem); + } + + vu_queue_notify(&vi->dev, vq); +} + +static void i2c_queue_set_started(VuDev *dev, int qidx, bool started) +{ + VuVirtq *vq = vu_get_queue(dev, qidx); + + dbg("queue started %d:%d\n", qidx, started); + + vu_set_queue_handler(dev, vq, started ? i2c_handle_cmdq : NULL); +} + +static bool i2cquit; +static bool gpioquit; + +static void remove_watch(VuDev *dev, int fd); + +static int i2c_process_msg(VuDev *dev, VhostUserMsg *vmsg, int *do_reply) +{ + if (vmsg->request == VHOST_USER_NONE) { + dbg("i2c disconnect"); + remove_watch(dev, -1); + i2cquit = true; + return true; + } + return false; +} +static int gpio_process_msg(VuDev *dev, VhostUserMsg *vmsg, int *do_reply) +{ + if (vmsg->request == VHOST_USER_NONE) { + dbg("gpio disconnect"); + remove_watch(dev, -1); + gpioquit = true; + return true; + } + return false; +} + +static uint64_t i2c_get_features(VuDev *dev) +{ + return 1ull << VIRTIO_I2C_F_ZERO_LENGTH_REQUEST; +} + +static const VuDevIface i2c_iface = { + .get_features = i2c_get_features, + .queue_set_started = i2c_queue_set_started, + .process_msg = i2c_process_msg, +}; + +static void gpio_send_irq_response(struct vhost_user_gpio *gpio, + unsigned int pin, unsigned int status) +{ + assert(pin < ARRAY_SIZE(gpio->irq_elements)); + + VuVirtqElement *elem = gpio->irq_elements[pin]; + VuVirtq *vq = vu_get_queue(&gpio->dev, 1); + + if (!elem) { + dbg("no irq buf for pin %d\n", pin); + assert(status != VIRTIO_GPIO_IRQ_STATUS_VALID); + return; + } + + struct virtio_gpio_irq_response *resp; + + assert(elem->out_num == 1); + assert(elem->in_sg[0].iov_len == sizeof(*resp)); + + resp = elem->in_sg[0].iov_base; + resp->status = status; + + vu_queue_push(&gpio->dev, vq, elem, sizeof(*resp)); + gpio->irq_elements[pin] = NULL; + free(elem); + + vu_queue_notify(&gpio->dev, vq); +} + +static void gpio_set_irq_type(struct vhost_user_gpio *gpio, unsigned int pin, + unsigned int type) +{ + PyObject *pArgs, *pValue; + + pArgs = PyTuple_New(2); + pValue = PyLong_FromLong(pin); + PyTuple_SetItem(pArgs, 0, pValue); + + pValue = PyLong_FromLong(type); + PyTuple_SetItem(pArgs, 1, pValue); + + pValue = PyObject_CallObject(py_gpio_set_irq_type, pArgs); + if (!pValue) { + PyErr_Print(); + errx(1, "error from gpio.set_irq_type()"); + } + Py_DECREF(pArgs); + + if (type == VIRTIO_GPIO_IRQ_TYPE_NONE) { + gpio_send_irq_response(gpio, pin, + VIRTIO_GPIO_IRQ_STATUS_INVALID); + } +} + +static void gpio_unmask(struct vhost_user_gpio *vi, unsigned int gpio) +{ + PyObject *pArgs, *pValue; + + pArgs = PyTuple_New(1); + pValue = PyLong_FromLong(gpio); + PyTuple_SetItem(pArgs, 0, pValue); + + pValue = PyObject_CallObject(py_gpio_unmask, pArgs); + if (!pValue) { + PyErr_Print(); + errx(1, "error from gpio.unmask()"); + } + Py_DECREF(pArgs); +} + +static void gpio_handle_cmdq(VuDev *dev, int qidx) +{ + struct vhost_user_gpio *vi = + container_of(dev, struct vhost_user_gpio, dev); + VuVirtq *vq = vu_get_queue(dev, qidx); + VuVirtqElement *elem; + + while (1) { + struct virtio_gpio_request *req; + struct virtio_gpio_response *resp; + + elem = vu_queue_pop(dev, vq, sizeof(VuVirtqElement)); + if (!elem) + break; + + dbg("elem %p index %u out_num %u in_num %u\n", elem, + elem->index, elem->out_num, elem->in_num); + + dump_iov("out", elem->out_sg, elem->out_num); + dump_iov("in", elem->in_sg, elem->in_num); + + assert(elem->out_num == 1); + assert(elem->in_num == 1); + + assert(elem->out_sg[0].iov_len == sizeof(*req)); + assert(elem->in_sg[0].iov_len == sizeof(*resp)); + + req = elem->out_sg[0].iov_base; + resp = elem->in_sg[0].iov_base; + + dbg("req type %#x gpio %#x value %#x\n", req->type, req->gpio, + req->value); + + switch (req->type) { + case VIRTIO_GPIO_MSG_IRQ_TYPE: + gpio_set_irq_type(vi, req->gpio, req->value); + break; + default: + /* + * The other types couldhooked up to Python later for + * testing of drivers' control of GPIOs. + */ + break; + } + + resp->status = VIRTIO_GPIO_STATUS_OK; + resp->value = 0; + + vu_queue_push(dev, vq, elem, sizeof(*resp)); + free(elem); + } + + vu_queue_notify(&vi->dev, vq); +} + +static void gpio_handle_eventq(VuDev *dev, int qidx) +{ + struct vhost_user_gpio *vi = + container_of(dev, struct vhost_user_gpio, dev); + VuVirtq *vq = vu_get_queue(dev, qidx); + VuVirtqElement *elem; + + for (;;) { + struct virtio_gpio_irq_request *req; + struct virtio_gpio_irq_response *resp; + + elem = vu_queue_pop(dev, vq, sizeof(VuVirtqElement)); + if (!elem) + break; + + dbg("elem %p index %u out_num %u in_num %u\n", elem, + elem->index, elem->out_num, elem->in_num); + + dump_iov("out", elem->out_sg, elem->out_num); + dump_iov("in", elem->in_sg, elem->in_num); + + assert(elem->out_num == 1); + assert(elem->in_num == 1); + + assert(elem->out_sg[0].iov_len == sizeof(*req)); + assert(elem->in_sg[0].iov_len == sizeof(*resp)); + + req = elem->out_sg[0].iov_base; + resp = elem->in_sg[0].iov_base; + + dbg("irq req gpio %#x\n", req->gpio); + + assert(req->gpio < ARRAY_SIZE(vi->irq_elements)); + assert(vi->irq_elements[req->gpio] == NULL); + + vi->irq_elements[req->gpio] = elem; + + gpio_unmask(vi, req->gpio); + } +} + +static void gpio_queue_set_started(VuDev *dev, int qidx, bool started) +{ + VuVirtq *vq = vu_get_queue(dev, qidx); + + dbg("%s %d:%d\n", __func__, qidx, started); + + if (qidx == 0) + vu_set_queue_handler(dev, vq, + started ? gpio_handle_cmdq : NULL); + if (qidx == 1) + vu_set_queue_handler(dev, vq, + started ? gpio_handle_eventq : NULL); +} + +static int gpio_get_config(VuDev *dev, uint8_t *config, uint32_t len) +{ + struct vhost_user_gpio *gpio = + container_of(dev, struct vhost_user_gpio, dev); + static struct virtio_gpio_config gpioconfig = { + .ngpio = ARRAY_SIZE(gpio->irq_elements), + }; + + dbg("%s: len %u\n", __func__, len); + + if (len > sizeof(struct virtio_gpio_config)) + return -1; + + memcpy(config, &gpioconfig, len); + + return 0; +} + +static uint64_t gpio_get_protocol_features(VuDev *dev) +{ + return 1ull << VHOST_USER_PROTOCOL_F_CONFIG; +} + +static uint64_t gpio_get_features(VuDev *dev) +{ + return 1ull << VIRTIO_GPIO_F_IRQ; +} + +static const VuDevIface gpio_vuiface = { + .get_features = gpio_get_features, + .queue_set_started = gpio_queue_set_started, + .process_msg = gpio_process_msg, + .get_config = gpio_get_config, + .get_protocol_features = gpio_get_protocol_features, +}; + +static void panic(VuDev *dev, const char *err) +{ + fprintf(stderr, "panicking!"); + abort(); +} + +static struct watch *new_watch(struct VuDev *dev, int fd, enum watch_type type, + void *func, void *data) +{ + struct watch *watch = malloc(sizeof(*watch)); + + assert(watch); + + watch->dev = dev; + watch->fd = fd; + watch->func = func; + watch->data = data; + watch->type = type; + + list_add(&watch->list, &watches); + + return watch; +} + +static void set_watch(VuDev *dev, int fd, int condition, vu_watch_cb cb, + void *data) +{ + struct watch *watch = new_watch(dev, fd, VU_WATCH, cb, data); + int ret; + + struct epoll_event ev = { + .events = EPOLLIN, + .data.ptr = watch, + }; + + dbg("set watch epfd %d fd %d condition %d cb %p\n", epfd, fd, condition, + cb); + + epoll_ctl(epfd, EPOLL_CTL_DEL, fd, NULL); + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, fd, &ev); + if (ret < 0) + err(1, "epoll_ctl"); +} + +static void remove_watch(VuDev *dev, int fd) +{ + struct watch *watch, *tmp; + + list_for_each_entry_safe(watch, tmp, &watches, list) { + if (watch->dev != dev) + continue; + if (fd >= 0 && watch->fd != fd) + continue; + + epoll_ctl(epfd, EPOLL_CTL_DEL, watch->fd, NULL); + + list_del(&watch->list); + free(watch); + } +} + +static int unix_listen(const char *path) +{ + struct sockaddr_un un = { + .sun_family = AF_UNIX, + }; + int sock; + int ret; + + unlink(path); + + sock = socket(PF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (sock < 0) + err(1, "socket"); + + memcpy(&un.sun_path, path, strlen(path)); + + ret = bind(sock, (struct sockaddr *)&un, sizeof(un)); + if (ret < 0) + err(1, "bind"); + + ret = listen(sock, 1); + if (ret < 0) + err(1, "listen"); + + return sock; +} + +static void dev_add_watch(int epfd, struct watch *watch) +{ + struct epoll_event event = { + .events = EPOLLIN | EPOLLONESHOT, + .data.ptr = watch, + }; + int ret; + + ret = epoll_ctl(epfd, EPOLL_CTL_ADD, watch->fd, &event); + if (ret < 0) + err(1, "EPOLL_CTL_ADD"); +} + +static VuDev *gpio_init(int epfd, const char *path) +{ + struct watch *watch; + VuDev *dev; + int lsock; + bool rc; + + lsock = unix_listen(path); + if (lsock < 0) + err(1, "listen %s", path); + + rc = vu_init(&gpio.dev, 2, lsock, panic, NULL, set_watch, + remove_watch, &gpio_vuiface); + assert(rc == true); + + dev = &gpio.dev; + watch = new_watch(dev, lsock, LISTEN, vu_dispatch, dev); + + dev_add_watch(epfd, watch); + + return dev; +} + +static VuDev *i2c_init(int epfd, const char *path) +{ + static struct vhost_user_i2c i2c = {}; + VuDev *dev = &i2c.dev; + struct watch *watch; + int lsock; + bool rc; + + lsock = unix_listen(path); + if (lsock < 0) + err(1, "listen %s", path); + + rc = vu_init(dev, 1, lsock, panic, NULL, set_watch, + remove_watch, &i2c_iface); + assert(rc == true); + + watch = new_watch(dev, lsock, LISTEN, vu_dispatch, dev); + + dev_add_watch(epfd, watch); + + return dev; +} + +static pid_t run_uml(char **argv) +{ + int log, null, ret; + pid_t pid; + + pid = fork(); + if (pid < 0) + err(1, "fork"); + if (pid > 0) + return pid; + + chdir(getenv("ROADTEST_WORK_DIR")); + + log = open("uml.txt", O_WRONLY | O_TRUNC | O_APPEND | O_CREAT, 0600); + if (log < 0) + err(1, "open uml.txt"); + + null = open("/dev/null", O_RDONLY); + if (null < 0) + err(1, "open null"); + + ret = dup2(null, 0); + if (ret < 0) + err(1, "dup2"); + + ret = dup2(log, 1); + if (ret < 0) + err(1, "dup2"); + + ret = dup2(log, 2); + if (ret < 0) + err(1, "dup2"); + + execvpe(argv[0], argv, environ); + err(1, "execve"); + + return -1; +} + +int main(int argc, char *argv[]) +{ + static struct option long_option[] = { + { "main-script", required_argument, 0, 'm' }, + { "gpio-socket", required_argument, 0, 'g' }, + { "i2c-socket", required_argument, 0, 'i' }, + }; + + while (1) { + int c = getopt_long(argc, argv, "", long_option, NULL); + + if (c == -1) + break; + + switch (c) { + case 'm': + opt_main_script = optarg; + break; + + case 'g': + opt_gpio_socket = optarg; + break; + + case 'i': + opt_i2c_socket = optarg; + break; + + default: + errx(1, "getopt"); + } + } + + if (!opt_main_script || !opt_gpio_socket || !opt_i2c_socket) + errx(1, "Invalid arguments"); + + epfd = epoll_create1(EPOLL_CLOEXEC); + if (epfd < 0) + err(1, "epoll_create1"); + + init_python(); + + gpio_init(epfd, opt_gpio_socket); + i2c_init(epfd, opt_i2c_socket); + + run_uml(&argv[optind]); + + while (1) { + struct epoll_event events[10]; + int nfds; + int i; + + nfds = epoll_wait(epfd, events, ARRAY_SIZE(events), -1); + if (nfds < 0) { + if (errno == EINTR) { + continue; + + err(1, "epoll_wait"); + } + } + + if (!PyObject_CallObject(py_process_control, NULL)) { + PyErr_Print(); + errx(1, "error from backend.process_control"); + } + + for (i = 0; i < nfds; i++) { + struct epoll_event *event = &events[i]; + struct watch *watch = event->data.ptr; + int fd; + + switch (watch->type) { + case LISTEN: + fd = accept(watch->fd, NULL, NULL); + close(watch->fd); + if (fd == -1) + err(1, "accept"); + + watch->dev->sock = fd; + watch->fd = fd; + watch->type = SOCKET_WATCH; + + struct epoll_event event = { + .events = EPOLLIN, + .data.ptr = watch, + }; + + int ret = epoll_ctl(epfd, EPOLL_CTL_ADD, fd, + &event); + if (ret < 0) + err(1, "epoll_ctl"); + + break; + case SOCKET_WATCH: + vu_dispatch(watch->dev); + break; + case VU_WATCH: + ((vu_watch_cb)(watch->func))(watch->dev, POLLIN, + watch->data); + break; + default: + fprintf(stderr, "abort!"); + abort(); + } + } + + if (i2cquit && gpioquit) + break; + } + + vu_deinit(&i2c.dev); + vu_deinit(&gpio.dev); + + Py_Finalize(); + + return 0; +}
Add the bulk of the roadtest framework. Apart from one init shell script, this is written in Python and includes three closely-related parts:
- The test runner which is invoked from the command line by the user and which starts the backend and sends the test jobs and results to/from UML.
- Test support code which is used by the actual driver tests run inside UML and which interact with the backend via a file-based asynchronous communication method.
- The backend which is run by the Python interpreter embedded in the C backend. This part runs the hardware models and is controlled by the tests and the driver (via virtio in the C backend).
Some unit tests for the framework itself are included and these will be automatically run whenever the driver tests are run.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- tools/testing/roadtest/init.sh | 19 ++ tools/testing/roadtest/roadtest/__init__.py | 2 + .../roadtest/roadtest/backend/__init__.py | 0 .../roadtest/roadtest/backend/backend.py | 32 ++ .../testing/roadtest/roadtest/backend/gpio.py | 111 +++++++ .../testing/roadtest/roadtest/backend/i2c.py | 123 ++++++++ .../testing/roadtest/roadtest/backend/main.py | 13 + .../testing/roadtest/roadtest/backend/mock.py | 20 ++ .../roadtest/roadtest/backend/test_gpio.py | 98 ++++++ .../roadtest/roadtest/backend/test_i2c.py | 84 +++++ .../testing/roadtest/roadtest/cmd/__init__.py | 0 tools/testing/roadtest/roadtest/cmd/main.py | 146 +++++++++ tools/testing/roadtest/roadtest/cmd/remote.py | 48 +++ .../roadtest/roadtest/core/__init__.py | 0 .../testing/roadtest/roadtest/core/control.py | 52 ++++ .../roadtest/roadtest/core/devicetree.py | 155 ++++++++++ .../roadtest/roadtest/core/hardware.py | 94 ++++++ tools/testing/roadtest/roadtest/core/log.py | 42 +++ .../testing/roadtest/roadtest/core/modules.py | 38 +++ .../testing/roadtest/roadtest/core/opslog.py | 35 +++ tools/testing/roadtest/roadtest/core/proxy.py | 48 +++ tools/testing/roadtest/roadtest/core/suite.py | 286 ++++++++++++++++++ tools/testing/roadtest/roadtest/core/sysfs.py | 77 +++++ .../roadtest/roadtest/core/test_control.py | 35 +++ .../roadtest/roadtest/core/test_devicetree.py | 31 ++ .../roadtest/roadtest/core/test_hardware.py | 41 +++ .../roadtest/roadtest/core/test_log.py | 54 ++++ .../roadtest/roadtest/core/test_opslog.py | 27 ++ .../roadtest/roadtest/tests/__init__.py | 0 29 files changed, 1711 insertions(+) create mode 100755 tools/testing/roadtest/init.sh create mode 100644 tools/testing/roadtest/roadtest/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/backend.py create mode 100644 tools/testing/roadtest/roadtest/backend/gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/i2c.py create mode 100644 tools/testing/roadtest/roadtest/backend/main.py create mode 100644 tools/testing/roadtest/roadtest/backend/mock.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_i2c.py create mode 100644 tools/testing/roadtest/roadtest/cmd/__init__.py create mode 100644 tools/testing/roadtest/roadtest/cmd/main.py create mode 100644 tools/testing/roadtest/roadtest/cmd/remote.py create mode 100644 tools/testing/roadtest/roadtest/core/__init__.py create mode 100644 tools/testing/roadtest/roadtest/core/control.py create mode 100644 tools/testing/roadtest/roadtest/core/devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/log.py create mode 100644 tools/testing/roadtest/roadtest/core/modules.py create mode 100644 tools/testing/roadtest/roadtest/core/opslog.py create mode 100644 tools/testing/roadtest/roadtest/core/proxy.py create mode 100644 tools/testing/roadtest/roadtest/core/suite.py create mode 100644 tools/testing/roadtest/roadtest/core/sysfs.py create mode 100644 tools/testing/roadtest/roadtest/core/test_control.py create mode 100644 tools/testing/roadtest/roadtest/core/test_devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/test_hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/test_log.py create mode 100644 tools/testing/roadtest/roadtest/core/test_opslog.py create mode 100644 tools/testing/roadtest/roadtest/tests/__init__.py
diff --git a/tools/testing/roadtest/init.sh b/tools/testing/roadtest/init.sh new file mode 100755 index 000000000000..c5fb28478aa3 --- /dev/null +++ b/tools/testing/roadtest/init.sh @@ -0,0 +1,19 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0-only + +mount -t proc proc /proc +echo 8 > /proc/sys/kernel/printk +mount -t sysfs nodev /sys +mount -t debugfs nodev /sys/kernel/debug + +echo 0 > /sys/bus/i2c/drivers_autoprobe +echo 0 > /sys/bus/platform/drivers_autoprobe + +python3 -m roadtest.cmd.remote +status=$? +[ "${ROADTEST_SHELL}" = "1" ] || { + # rsync doesn't handle these zero-sized files correctly. + cp -ra --no-preserve=ownership /sys/kernel/debug/gcov ${ROADTEST_WORK_DIR}/gcov + echo o > /proc/sysrq-trigger +} +exec setsid sh -c 'exec bash </dev/tty0 >/dev/tty0 2>&1' diff --git a/tools/testing/roadtest/roadtest/__init__.py b/tools/testing/roadtest/roadtest/__init__.py new file mode 100644 index 000000000000..dac3ce6976e5 --- /dev/null +++ b/tools/testing/roadtest/roadtest/__init__.py @@ -0,0 +1,2 @@ +ENV_WORK_DIR = "ROADTEST_WORK_DIR" +ENV_BUILD_DIR = "ROADTEST_BUILD_DIR" diff --git a/tools/testing/roadtest/roadtest/backend/__init__.py b/tools/testing/roadtest/roadtest/backend/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/backend/backend.py b/tools/testing/roadtest/roadtest/backend/backend.py new file mode 100644 index 000000000000..bfd19fc363c2 --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/backend.py @@ -0,0 +1,32 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import logging +import os +from pathlib import Path + +from roadtest import ENV_WORK_DIR +from roadtest.core.control import ControlReader + +from . import gpio, i2c, mock + +logger = logging.getLogger(__name__) + +try: + import cbackend # type: ignore[import] +except ModuleNotFoundError: + # In unit tests + cbackend = None + + +class Backend: + def __init__(self) -> None: + work = Path(os.environ[ENV_WORK_DIR]) + self.control = ControlReader(work_dir=work) + self.c = cbackend + self.i2c = i2c.I2CBackend(self) + self.gpio = gpio.GpioBackend(self) + self.mock = mock.MockBackend(work) + + def process_control(self) -> None: + self.control.process({"backend": self}) diff --git a/tools/testing/roadtest/roadtest/backend/gpio.py b/tools/testing/roadtest/roadtest/backend/gpio.py new file mode 100644 index 000000000000..2eaf52b31c72 --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/gpio.py @@ -0,0 +1,111 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import logging +import typing +from typing import Optional + +if typing.TYPE_CHECKING: + # Avoid circular imports + from .backend import Backend + +logger = logging.getLogger(__name__) + + +class Gpio: + IRQ_TYPE_NONE = 0x00 + IRQ_TYPE_EDGE_RISING = 0x01 + IRQ_TYPE_EDGE_FALLING = 0x02 + IRQ_TYPE_EDGE_BOTH = 0x03 + IRQ_TYPE_LEVEL_HIGH = 0x04 + IRQ_TYPE_LEVEL_LOW = 0x08 + + def __init__(self, backend: "Backend", pin: int): + self.backend = backend + self.pin = pin + self.state = False + self.irq_type = Gpio.IRQ_TYPE_NONE + self.masked = True + self.edge_irq_latched = False + + def _level_irq_active(self) -> bool: + if self.irq_type == Gpio.IRQ_TYPE_LEVEL_HIGH: + return self.state + elif self.irq_type == Gpio.IRQ_TYPE_LEVEL_LOW: + return not self.state + + return False + + def _latch_edge_irq(self, old: bool, new: bool) -> bool: + if old != new: + logger.debug(f"{self}: latch_edge_irq {self.irq_type} {old} -> {new}") + + if self.irq_type == Gpio.IRQ_TYPE_EDGE_RISING: + return not old and new + elif self.irq_type == Gpio.IRQ_TYPE_EDGE_FALLING: + return old and not new + elif self.irq_type == Gpio.IRQ_TYPE_EDGE_BOTH: + return old != new + + return False + + def _check_irq(self) -> None: + if self.irq_type == Gpio.IRQ_TYPE_NONE or self.masked: + return + if not self.edge_irq_latched and not self._level_irq_active(): + return + + self.masked = True + self.edge_irq_latched = False + + logger.debug(f"{self}: trigger irq") + self.backend.c.trigger_gpio_irq(self.pin) + + def set_irq_type(self, irq_type: int) -> None: + logger.debug(f"{self}: set_irq_type {irq_type}") + if irq_type == Gpio.IRQ_TYPE_NONE: + self.masked = True + + self.irq_type = irq_type + self.edge_irq_latched = False + self._check_irq() + + def unmask(self) -> None: + logger.debug(f"{self}: unmask") + self.masked = False + self._check_irq() + + def set(self, val: int) -> None: + old = self.state + new = bool(val) + + if old != new: + logger.debug(f"{self}: gpio set {old} -> {new}") + + self.state = new + if self._latch_edge_irq(old, new): + logger.debug(f"{self}: latching edge") + self.edge_irq_latched = True + + self._check_irq() + + def __str__(self) -> str: + return f"Gpio({self.pin})" + + +class GpioBackend: + def __init__(self, backend: "Backend") -> None: + self.backend = backend + self.gpios = [Gpio(backend, pin) for pin in range(64)] + + def set(self, pin: Optional[int], val: bool) -> None: + if pin is None: + return + + self.gpios[pin].set(val) + + def set_irq_type(self, pin: int, irq_type: int) -> None: + self.gpios[pin].set_irq_type(irq_type) + + def unmask(self, pin: int) -> None: + self.gpios[pin].unmask() diff --git a/tools/testing/roadtest/roadtest/backend/i2c.py b/tools/testing/roadtest/roadtest/backend/i2c.py new file mode 100644 index 000000000000..b877c2b76851 --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/i2c.py @@ -0,0 +1,123 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import abc +import importlib +import logging +import typing +from typing import Any, Literal, Optional + +if typing.TYPE_CHECKING: + # Avoid circular imports + from .backend import Backend + +logger = logging.getLogger(__name__) + + +class I2CBackend: + def __init__(self, backend: "Backend") -> None: + self.model: Optional[I2CModel] = None + self.backend = backend + + def load_model(self, modname: str, clsname: str, *args: Any, **kwargs: Any) -> None: + mod = importlib.import_module(modname) + cls = getattr(mod, clsname) + self.model = cls(*args, **kwargs, backend=self.backend) + + def unload_model(self) -> None: + self.model = None + + def read(self, length: int) -> bytes: + if not self.model: + raise Exception("No I2C model loaded") + + return self.model.read(length) + + def write(self, data: bytes) -> None: + if not self.model: + raise Exception("No I2C model loaded") + + self.model.write(data) + + def __getattr__(self, name: str) -> Any: + return getattr(self.model, name) + + +class I2CModel(abc.ABC): + def __init__(self, backend: "Backend") -> None: + self.backend = backend + + @abc.abstractmethod + def read(self, length: int) -> bytes: + return bytes(length) + + @abc.abstractmethod + def write(self, data: bytes) -> None: + pass + + +class SMBusModel(I2CModel): + def __init__( + self, + regbytes: int, + byteorder: Literal["little", "big"] = "little", + *args: Any, + **kwargs: Any, + ) -> None: + super().__init__(*args, **kwargs) + self.reg_addr = 0x0 + self.regbytes = regbytes + self.byteorder = byteorder + + @abc.abstractmethod + def reg_read(self, addr: int) -> int: + return 0 + + @abc.abstractmethod + def reg_write(self, addr: int, val: int) -> None: + pass + + def val_to_bytes(self, val: int) -> bytes: + return val.to_bytes(self.regbytes, self.byteorder) + + def bytes_to_val(self, data: bytes) -> int: + return int.from_bytes(data, self.byteorder) + + def read(self, length: int) -> bytes: + data = bytearray() + for idx in range(0, length, self.regbytes): + addr = self.reg_addr + idx + val = self.reg_read(addr) + logger.debug(f"SMBus read {addr=:#02x} {val=:#02x}") + data += self.val_to_bytes(val) + return bytes(data) + + def write(self, data: bytes) -> None: + self.reg_addr = data[0] + + if len(data) > 1: + length = len(data) - 1 + data = data[1:] + assert length % self.regbytes == 0 + for idx in range(0, length, self.regbytes): + val = self.bytes_to_val(data[idx : (idx + self.regbytes)]) + addr = self.reg_addr + idx + self.backend.mock.reg_write(addr, val) + self.reg_write(addr, val) + logger.debug(f"SMBus write {addr=:#02x} {val=:#02x}") + elif len(data) == 1: + pass + + +class SimpleSMBusModel(SMBusModel): + def __init__(self, regs: dict[int, int], **kwargs: Any) -> None: + super().__init__(**kwargs) + self.regs = regs + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + return val + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + self.regs[addr] = val diff --git a/tools/testing/roadtest/roadtest/backend/main.py b/tools/testing/roadtest/roadtest/backend/main.py new file mode 100644 index 000000000000..25be86ded9ea --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/main.py @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import logging + +import roadtest.backend.backend + +logging.basicConfig( + format="%(asctime)s - %(levelname)s - %(name)s: %(message)s", level=logging.DEBUG +) + +backend = roadtest.backend.backend.Backend() +backend.process_control() diff --git a/tools/testing/roadtest/roadtest/backend/mock.py b/tools/testing/roadtest/roadtest/backend/mock.py new file mode 100644 index 000000000000..8ce33a6bc0f1 --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/mock.py @@ -0,0 +1,20 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import functools +from pathlib import Path +from typing import Any, Callable + +from roadtest.core.opslog import OpsLogWriter + + +class MockBackend: + def __init__(self, work: Path) -> None: + self.opslog = OpsLogWriter(work) + + @functools.cache + def __getattr__(self, name: str) -> Callable: + def func(*args: Any, **kwargs: Any) -> None: + self.opslog.write(f"mock.{name}(*{str(args)}, **{str(kwargs)})") + + return func diff --git a/tools/testing/roadtest/roadtest/backend/test_gpio.py b/tools/testing/roadtest/roadtest/backend/test_gpio.py new file mode 100644 index 000000000000..feffe4fb9625 --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/test_gpio.py @@ -0,0 +1,98 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import unittest +from unittest.mock import MagicMock + +from .gpio import Gpio + + +class TestGpio(unittest.TestCase): + def test_irq_low(self) -> None: + m = MagicMock() + gpio = Gpio(backend=m, pin=1) + + gpio.set_irq_type(Gpio.IRQ_TYPE_LEVEL_LOW) + m.c.trigger_gpio_irq.assert_not_called() + + gpio.unmask() + m.c.trigger_gpio_irq.assert_called_once_with(1) + m.c.trigger_gpio_irq.reset_mock() + + gpio.set(True) + gpio.unmask() + m.c.trigger_gpio_irq.assert_not_called() + + def test_irq_high(self) -> None: + m = MagicMock() + gpio = Gpio(backend=m, pin=2) + + gpio.set_irq_type(Gpio.IRQ_TYPE_LEVEL_HIGH) + gpio.unmask() + + m.c.trigger_gpio_irq.assert_not_called() + + gpio.set(True) + m.c.trigger_gpio_irq.assert_called_once_with(2) + m.c.trigger_gpio_irq.reset_mock() + + gpio.set(False) + gpio.unmask() + m.c.trigger_gpio_irq.assert_not_called() + + def test_irq_rising(self) -> None: + m = MagicMock() + gpio = Gpio(backend=m, pin=63) + + gpio.set_irq_type(Gpio.IRQ_TYPE_EDGE_RISING) + gpio.set(False) + gpio.set(True) + + m.c.trigger_gpio_irq.assert_not_called() + gpio.unmask() + m.c.trigger_gpio_irq.assert_called_once_with(63) + m.c.trigger_gpio_irq.reset_mock() + + gpio.set(False) + gpio.set(True) + + gpio.unmask() + m.c.trigger_gpio_irq.assert_called_once() + + def test_irq_falling(self) -> None: + m = MagicMock() + gpio = Gpio(backend=m, pin=0) + + gpio.set_irq_type(Gpio.IRQ_TYPE_EDGE_FALLING) + gpio.unmask() + gpio.set(False) + gpio.set(True) + m.c.trigger_gpio_irq.assert_not_called() + + gpio.set(False) + m.c.trigger_gpio_irq.assert_called_once_with(0) + m.c.trigger_gpio_irq.reset_mock() + + gpio.set(True) + gpio.set(False) + gpio.set(True) + gpio.unmask() + m.c.trigger_gpio_irq.assert_called_once() + + def test_irq_both(self) -> None: + m = MagicMock() + gpio = Gpio(backend=m, pin=32) + + gpio.set_irq_type(Gpio.IRQ_TYPE_EDGE_BOTH) + gpio.unmask() + gpio.set(False) + gpio.set(True) + m.c.trigger_gpio_irq.assert_called_once_with(32) + + gpio.set(False) + m.c.trigger_gpio_irq.assert_called_once_with(32) + m.c.trigger_gpio_irq.reset_mock() + + gpio.set(True) + gpio.unmask() + m.c.trigger_gpio_irq.assert_called_once_with(32) diff --git a/tools/testing/roadtest/roadtest/backend/test_i2c.py b/tools/testing/roadtest/roadtest/backend/test_i2c.py new file mode 100644 index 000000000000..eda4e1a4b80f --- /dev/null +++ b/tools/testing/roadtest/roadtest/backend/test_i2c.py @@ -0,0 +1,84 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import unittest +from typing import Any +from unittest.mock import MagicMock + +from .i2c import SimpleSMBusModel, SMBusModel + + +class DummyModel(SMBusModel): + def __init__(self, *args: Any, **kwargs: Any) -> None: + super().__init__(*args, **kwargs) + self.regs: dict[int, int] = {} + + def reg_read(self, addr: int) -> int: + return self.regs[addr] + + def reg_write(self, addr: int, val: int) -> None: + self.regs[addr] = val + + +class TestSMBusModel(unittest.TestCase): + def test_1(self) -> None: + m = DummyModel(regbytes=1, backend=MagicMock()) + + m.write(bytes([0x12, 0x34])) + m.write(bytes([0x13, 0xAB, 0xCD])) + + self.assertEqual(m.regs[0x12], 0x34) + self.assertEqual(m.regs[0x13], 0xAB) + self.assertEqual(m.regs[0x14], 0xCD) + + m.write(bytes([0x12])) + self.assertEqual(m.read(1), bytes([0x34])) + + m.write(bytes([0x12])) + self.assertEqual(m.read(3), bytes([0x34, 0xAB, 0xCD])) + + def test_2big(self) -> None: + m = DummyModel(regbytes=2, byteorder="big", backend=MagicMock()) + + m.write(bytes([0x12, 0x34, 0x56, 0xAB, 0xCD])) + self.assertEqual(m.regs[0x12], 0x3456) + self.assertEqual(m.regs[0x14], 0xABCD) + + m.write(bytes([0x12])) + self.assertEqual(m.read(2), bytes([0x34, 0x56])) + + m.write(bytes([0x14])) + self.assertEqual(m.read(2), bytes([0xAB, 0xCD])) + + m.write(bytes([0x12])) + self.assertEqual(m.read(4), bytes([0x34, 0x56, 0xAB, 0xCD])) + + def test_2little(self) -> None: + m = DummyModel(regbytes=2, byteorder="little", backend=MagicMock()) + + m.write(bytes([0x12, 0x34, 0x56, 0xAB, 0xCD])) + self.assertEqual(m.regs[0x12], 0x5634) + self.assertEqual(m.regs[0x14], 0xCDAB) + + m.write(bytes([0x12])) + self.assertEqual(m.read(2), bytes([0x34, 0x56])) + + +class TestSimpleSMBusModel(unittest.TestCase): + def test_simple(self) -> None: + m = SimpleSMBusModel( + regs={0x01: 0x12, 0x02: 0x34}, + regbytes=1, + backend=MagicMock(), + ) + self.assertEqual(m.reg_read(0x01), 0x12) + self.assertEqual(m.reg_read(0x02), 0x34) + + m.reg_write(0x01, 0x56) + self.assertEqual(m.reg_read(0x01), 0x56) + self.assertEqual(m.reg_read(0x02), 0x34) + + with self.assertRaises(Exception): + m.reg_write(0x03, 0x00) + with self.assertRaises(Exception): + m.reg_read(0x03) diff --git a/tools/testing/roadtest/roadtest/cmd/__init__.py b/tools/testing/roadtest/roadtest/cmd/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/cmd/main.py b/tools/testing/roadtest/roadtest/cmd/main.py new file mode 100644 index 000000000000..634c27fe795c --- /dev/null +++ b/tools/testing/roadtest/roadtest/cmd/main.py @@ -0,0 +1,146 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import argparse +import fnmatch +import sys +import unittest +from typing import Optional +from unittest.suite import TestSuite + +assert sys.version_info >= (3, 9), "Python version is too old" + +from roadtest.core.suite import UMLSuite, UMLTestCase + + +def make_umlsuite(args: argparse.Namespace) -> UMLSuite: + return UMLSuite( + timeout=args.timeout, + workdir=args.work_dir, + builddir=args.build_dir, + ksrcdir=args.ksrc_dir, + uml_args_pre=args.uml_prepend, + uml_args_post=args.uml_append, + shell=args.shell, + ) + + +def main() -> None: + parser = argparse.ArgumentParser() + parser.add_argument( + "--timeout", + type=int, + default=60, + help="Timeout (in seconds) for each UML run, 0 to disable", + ) + parser.add_argument("--work-dir", type=str, help="Work directory for UML runs") + parser.add_argument("--build-dir", type=str, required=True) + parser.add_argument("--ksrc-dir", type=str, required=True) + parser.add_argument( + "--uml-prepend", + nargs="*", + default=[], + help="Extra arguments to prepend to the UML command (example: gdbserver :1234)", + ) + parser.add_argument( + "--uml-append", + nargs="*", + default=[], + help="Extra arguments to append to the UML command (example: trace_event=i2c:* tp_printk)", + ) + parser.add_argument( + "--filter", + nargs="+", + default=[], + ) + parser.add_argument("--shell", action="store_true") + parser.add_argument("test", nargs="?", default="roadtest") + args = parser.parse_args() + + if args.shell: + args.timeout = 0 + + if not any(p.startswith("con=") for p in args.uml_append): + print( + "Error: --shell used but no con= UML argument specified", + file=sys.stderr, + ) + sys.exit(1) + + test = args.test + test = test.replace("/", ".") + test = test.removesuffix(".py") + test = test.removesuffix(".") + + loader = unittest.defaultTestLoader + suitegroups = loader.discover(test) + + args.filter = [f"*{f}*" for f in args.filter] + + # Backend tests and the like don't need to be run inside UML. + localsuite = None + + # For simplicity, we currently run all target tests in one UML instance + # since python in UML is slow to start up. This can be revisited if we + # want to run several UML instances in parallel. + deftargetsuite = None + targetsuites = [] + + for suites in suitegroups: + # unittest can in arbitrarily nest and mix TestCases + # and TestSuites, but we expect a fixed hierarchy. + assert isinstance(suites, unittest.TestSuite) + + for suite in suites: + # assert not isinstance(suite, unittest.TestCase) + + # If the import of a test fails, then suite is a + # unittest.loader._FailedTest instead of a suite + if not isinstance(suite, unittest.TestSuite): + suite = [suite] # type: ignore[assignment] + + # Suite at this level contains one TestCase for each + # test method in a particular test class. + # + # All the test functions for one particular test class + # can only be run either in UML or locally, not mixed. + destsuite: Optional[TestSuite] = None + + for t in suite: # type: ignore[union-attr] + # We don't support suites nested at this level. + assert isinstance(t, unittest.TestCase) + + id = t.id() + if args.filter and not any(fnmatch.fnmatch(id, f) for f in args.filter): + continue + + if isinstance(t, UMLTestCase): + if t.run_separately: + if not destsuite: + destsuite = make_umlsuite(args) + targetsuites.append(destsuite) + else: + if not deftargetsuite: + deftargetsuite = make_umlsuite(args) + targetsuites.append(deftargetsuite) + + destsuite = deftargetsuite + else: + if not localsuite: + localsuite = TestSuite() + destsuite = localsuite + + if destsuite: + destsuite.addTest(t) + + tests = unittest.TestSuite() + if localsuite: + tests.addTest(localsuite) + tests.addTests(targetsuites) + + result = unittest.TextTestRunner(verbosity=2).run(tests) + sys.exit(not result.wasSuccessful()) + + +if __name__ == "__main__": + main() diff --git a/tools/testing/roadtest/roadtest/cmd/remote.py b/tools/testing/roadtest/roadtest/cmd/remote.py new file mode 100644 index 000000000000..29c3c6d35c65 --- /dev/null +++ b/tools/testing/roadtest/roadtest/cmd/remote.py @@ -0,0 +1,48 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import importlib +import json +import os +from pathlib import Path +from typing import cast +from unittest import TestSuite, TextTestRunner + +from roadtest import ENV_WORK_DIR +from roadtest.core import proxy + + +def main() -> None: + workdir = Path(os.environ[ENV_WORK_DIR]) + with open(workdir / "tests.json") as f: + testinfos = json.load(f) + + suite = TestSuite() + for info in testinfos: + id = info["id"] + *modparts, clsname, method = id.split(".") + + fullname = ".".join(modparts) + mod = importlib.import_module(fullname) + + cls = getattr(mod, clsname) + test = cls(methodName=method) + + values = info["values"] + if values: + test.dts.values = values + + suite.addTest(test) + + runner = TextTestRunner( + verbosity=0, buffer=False, resultclass=proxy.ProxyTextTestResult + ) + result = cast(proxy.ProxyTextTestResult, runner.run(suite)) + + proxyresult = result.to_proxy() + with open(workdir / "results.json", "w") as f: + json.dump(proxyresult, f) + + +if __name__ == "__main__": + main() diff --git a/tools/testing/roadtest/roadtest/core/__init__.py b/tools/testing/roadtest/roadtest/core/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/core/control.py b/tools/testing/roadtest/roadtest/core/control.py new file mode 100644 index 000000000000..cd74861099b9 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/control.py @@ -0,0 +1,52 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import logging +import os +from pathlib import Path +from typing import Optional + +from roadtest import ENV_WORK_DIR + +CONTROL_FILE = "control.txt" + +logger = logging.getLogger(__name__) + + +class ControlReader: + def __init__(self, work_dir: Optional[Path] = None) -> None: + if not work_dir: + work_dir = Path(os.environ[ENV_WORK_DIR]) + + path = work_dir / CONTROL_FILE + path.unlink(missing_ok=True) + path.write_text("") + + self.file = path.open("r") + + def process(self, vars: dict) -> None: + for line in self.file.readlines(): + cmd = line.rstrip() + + if cmd.startswith("# "): + logger.info(line[2:].rstrip()) + continue + + logger.debug(cmd) + eval(cmd, vars) + + +class ControlWriter: + def __init__(self, work_dir: Optional[Path] = None) -> None: + if not work_dir: + work_dir = Path(os.environ[ENV_WORK_DIR]) + self.file = (work_dir / CONTROL_FILE).open("a", buffering=1) + + def write_cmd(self, line: str) -> None: + self.file.write(line + "\n") + + def write_log(self, line: str) -> None: + self.file.write(f"# {line}\n") + + def close(self) -> None: + self.file.close() diff --git a/tools/testing/roadtest/roadtest/core/devicetree.py b/tools/testing/roadtest/roadtest/core/devicetree.py new file mode 100644 index 000000000000..40876738fb39 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/devicetree.py @@ -0,0 +1,155 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import enum +import subprocess +from pathlib import Path +from typing import Any, Optional + +HEADER = """ +/dts-v1/; + +/ { + #address-cells = <2>; + #size-cells = <2>; + + virtio@0 { + compatible = "virtio,uml"; + socket-path = "WORK/gpio.sock"; + virtio-device-id = <0x29>; + + gpio: gpio { + compatible = "virtio,device29"; + + gpio-controller; + #gpio-cells = <2>; + + interrupt-controller; + #interrupt-cells = <2>; + }; + }; + + virtio@1 { + compatible = "virtio,uml"; + socket-path = "WORK/i2c.sock"; + virtio-device-id = <0x22>; + + i2c: i2c { + compatible = "virtio,device22"; + + #address-cells = <1>; + #size-cells = <0>; + }; + }; + + // See Hardware.kick() + leds { + compatible = "gpio-leds"; + led0 { + gpios = <&gpio 0 0>; + }; + }; +}; +""" + + +class DtVar(enum.Enum): + I2C_ADDR = 0 + GPIO_PIN = 1 + + +class DtFragment: + def __init__(self, src: str, variables: Optional[dict[str, DtVar]] = None) -> None: + self.src = src + if not variables: + variables = {} + self.variables = variables + self.values: dict[str, int] = {} + + def apply(self, values: dict[str, Any]) -> str: + src = self.src + + for var in self.variables.keys(): + typ = self.variables[var] + val = values[var] + + if typ == DtVar.I2C_ADDR: + str = f"{val:02x}" + elif typ == DtVar.GPIO_PIN: + str = f"{val:d}" + + src = src.replace(f"${var}$", str) + + self.values = values + return src + + def __getitem__(self, key: str) -> Any: + return self.values[key] + + +class Devicetree: + def __init__(self, workdir: Path, ksrcdir: Path) -> None: + self.workdir: Path = workdir + self.ksrcdir: Path = ksrcdir + self.next_i2c_addr: int = 0x1 + # 0 is used for gpio-leds for Hardware.kick() + self.next_gpio_pin: int = 1 + self.src: str = "" + + def assemble(self, fragments: list[DtFragment]) -> None: + parts = [] + for fragment in fragments: + if fragment.values: + # Multiple test functions from the same class will use + # the same class instance + continue + + values = {} + + for var, type in fragment.variables.items(): + if type == DtVar.I2C_ADDR: + values[var] = self.next_i2c_addr + self.next_i2c_addr += 1 + elif type == DtVar.GPIO_PIN: + values[var] = self.next_gpio_pin + self.next_gpio_pin += 1 + + parts.append(fragment.apply(values)) + + self.src = "\n".join(parts) + + def compile(self, dtb: str) -> None: + dts = self.workdir / "test.dts" + + try: + subprocess.run( + [ + "gcc", + "-E", + "-nostdinc", + f"-I{self.ksrcdir}/scripts/dtc/include-prefixes", + "-undef", + "-D__DTS__", + "-x", + "assembler-with-cpp", + "-o", + dts, + "-", + ], + input=self.src, + text=True, + check=True, + capture_output=True, + ) + + full = HEADER.replace("WORK", str(self.workdir)) + dts.read_text() + dts.write_text(full) + + subprocess.run( + ["dtc", "-I", "dts", "-O", "dtb", dts, "-o", self.workdir / dtb], + check=True, + capture_output=True, + text=True, + ) + except subprocess.CalledProcessError as e: + raise Exception(f"{e.stderr}") diff --git a/tools/testing/roadtest/roadtest/core/hardware.py b/tools/testing/roadtest/roadtest/core/hardware.py new file mode 100644 index 000000000000..ae81a531d2a2 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/hardware.py @@ -0,0 +1,94 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import contextlib +import functools +import os +from pathlib import Path +from typing import Any, Callable, Optional, Type, cast +from unittest import TestCase +from unittest.mock import MagicMock, call + +from roadtest import ENV_WORK_DIR + +from .control import ControlWriter +from .opslog import OpsLogReader +from .sysfs import write_int + + +class HwMock(MagicMock): + def assert_reg_write_once(self, test: TestCase, reg: int, value: int) -> None: + test.assertEqual( + [c for c in self.mock_calls if c.args[0] == reg], + [call.reg_write(reg, value)], + ) + + def assert_last_reg_write(self, test: TestCase, reg: int, value: int) -> None: + test.assertEqual( + [c for c in self.mock_calls if c.args[0] == reg][-1:], + [call.reg_write(reg, value)], + ) + + def get_last_reg_write(self, reg: int) -> int: + return cast(int, [c for c in self.mock_calls if c.args[0] == reg][-1].args[1]) + + +class Hardware(contextlib.AbstractContextManager): + def __init__(self, bus: str, work: Optional[Path] = None) -> None: + if not work: + work = Path(os.environ[ENV_WORK_DIR]) + + self.bus = bus + self.mock = HwMock() + self.control = ControlWriter(work) + self.opslog = OpsLogReader(work) + self.loaded_model = False + + # Ignore old entries + self.opslog.read_next() + + def _call(self, method: str, *args: Any, **kwargs: Any) -> None: + self.control.write_cmd( + f"backend.{self.bus}.{method}(*{str(args)}, **{str(kwargs)})" + ) + + def kick(self) -> None: + # Control writes are only applied when the backend gets something + # to process, usually because the driver tried to access the device. + # But in some cases, such as when the driver is waiting for a + # sequence of interrupts, the test code needs the control write to take + # effect immediately. For this, we just need to kick the backend + # into processing its control queue. + # + # We (ab)use gpio-leds for this. devicetree.py sets up the device. + write_int(Path("/sys/class/leds/led0/brightness"), 0) + + def load_model(self, cls: Type[Any], *args: Any, **kwargs: Any) -> "Hardware": + self._call("load_model", cls.__module__, cls.__name__, *args, **kwargs) + self.loaded_model = True + return self + + def __enter__(self) -> "Hardware": + return self + + def __exit__(self, *_: Any) -> None: + self.close() + + @functools.cache + def __getattr__(self, name: str) -> Callable: + def func(*args: Any, **kwargs: Any) -> None: + self._call(name, *args, **kwargs) + + return func + + def close(self) -> None: + if self.loaded_model: + self._call("unload_model") + self.control.close() + + def update_mock(self) -> HwMock: + opslog = self.opslog.read_next() + for line in opslog: + eval(line, {"mock": self.mock}) + + return self.mock diff --git a/tools/testing/roadtest/roadtest/core/log.py b/tools/testing/roadtest/roadtest/core/log.py new file mode 100644 index 000000000000..7d73e40eb2d8 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/log.py @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from pathlib import Path + + +class LogParser: + DNF_MESSAGE = "<Test did not finish cleanly>" + + def __init__(self, file: Path): + try: + raw = file.read_text() + lines = raw.splitlines() + except FileNotFoundError: + lines = [] + raw = "" + + self.raw = raw + self.lines = lines + + def has_any(self) -> bool: + return "START<" in self.raw + + def get_testcase_log(self, id: str) -> list[str]: + startmarker = f"START<{id}>" + stopmarker = f"STOP<{id}>" + + try: + startpos = next( + i for i, line in enumerate(self.lines) if startmarker in line + ) + except StopIteration: + return [] + + try: + stoppos = next( + i for i, line in enumerate(self.lines[startpos:]) if stopmarker in line + ) + except StopIteration: + return self.lines[startpos + 1 :] + [LogParser.DNF_MESSAGE] + + return self.lines[startpos + 1 : startpos + stoppos] diff --git a/tools/testing/roadtest/roadtest/core/modules.py b/tools/testing/roadtest/roadtest/core/modules.py new file mode 100644 index 000000000000..5bd2d92a322b --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/modules.py @@ -0,0 +1,38 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import os +import subprocess +from pathlib import Path +from typing import Any + +from roadtest import ENV_BUILD_DIR + + +def modprobe(modname: str, remove: bool = False) -> None: + moddir = Path(os.environ[ENV_BUILD_DIR]) / "modules" + args = [] + if remove: + args.append("--remove") + args += [f"--dirname={moddir}", modname] + subprocess.check_output(["/sbin/modprobe"] + args) + + +def insmod(modname: str) -> None: + modprobe(modname) + + +def rmmod(modname: str) -> None: + subprocess.check_output(["/sbin/rmmod", modname]) + + +class Module: + def __init__(self, name: str) -> None: + self.name = name + + def __enter__(self) -> "Module": + modprobe(self.name) + return self + + def __exit__(self, *_: Any) -> None: + rmmod(self.name) diff --git a/tools/testing/roadtest/roadtest/core/opslog.py b/tools/testing/roadtest/roadtest/core/opslog.py new file mode 100644 index 000000000000..83bb4f525d03 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/opslog.py @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import os +from pathlib import Path + +OPSLOG_FILE = "opslog.txt" + + +class OpsLogWriter: + def __init__(self, work: Path) -> None: + path = work / OPSLOG_FILE + path.unlink(missing_ok=True) + self.file = open(path, "a", buffering=1) + + def write(self, line: str) -> None: + self.file.write(line + "\n") + + +class OpsLogReader: + def __init__(self, work: Path) -> None: + self.path = work / OPSLOG_FILE + self.opslogpos = 0 + + def read_next(self) -> list[str]: + # There is a problem in hostfs (see Hostfs Caveats) which means + # that reads from UML on a file which is extended on the host don't see + # the new data unless we open and close the file, so we can't open once + # and use readlines(). + with open(self.path, "r") as f: + os.lseek(f.fileno(), self.opslogpos, os.SEEK_SET) + opslog = [line.rstrip() for line in f.readlines()] + self.opslogpos = os.lseek(f.fileno(), 0, os.SEEK_CUR) + + return opslog diff --git a/tools/testing/roadtest/roadtest/core/proxy.py b/tools/testing/roadtest/roadtest/core/proxy.py new file mode 100644 index 000000000000..36089e21d7d5 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/proxy.py @@ -0,0 +1,48 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from typing import Any +from unittest import TestCase, TextTestResult + +from . import control + + +class ProxyTextTestResult(TextTestResult): + def __init__(self, stream: Any, descriptions: Any, verbosity: Any) -> None: + super().__init__(stream, descriptions, verbosity) + self.successes: list[tuple[TestCase, str]] = [] + + # Print via kmsg to avoid getting cut off by other kernel prints. + self.kmsg = open("/dev/kmsg", "w", buffering=1) + self.control = control.ControlWriter() + + def addSuccess(self, test: TestCase) -> None: + super().addSuccess(test) + self.successes.append((test, "")) + + def _log(self, test: TestCase, action: str) -> None: + line = f"{action}<{test.id()}>" + self.kmsg.write(line + "\n") + self.control.write_log(line) + + def startTest(self, test: TestCase) -> None: + self._log(test, "START") + super().startTest(test) + + def stopTest(self, test: TestCase) -> None: + super().stopTest(test) + self._log(test, "STOP") + + def _replace_id(self, reslist: list[tuple[TestCase, str]]) -> list[tuple[str, str]]: + return [(case.id(), tb) for case, tb in reslist] + + def to_proxy(self) -> dict[str, Any]: + return { + "testsRun": self.testsRun, + "wasSuccessful": self.wasSuccessful(), + "successes": self._replace_id(self.successes), + "errors": self._replace_id(self.errors), + "failures": self._replace_id(self.failures), + "skipped": self._replace_id(self.skipped), + "unexpectedSuccesses": [t.id() for t in self.unexpectedSuccesses], + } diff --git a/tools/testing/roadtest/roadtest/core/suite.py b/tools/testing/roadtest/roadtest/core/suite.py new file mode 100644 index 000000000000..e99a60b4faba --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/suite.py @@ -0,0 +1,286 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import json +import os +import shlex +import signal +import subprocess +import textwrap +import unittest +from pathlib import Path +from typing import Any, ClassVar, Optional, Tuple, cast +from unittest import TestResult + +from roadtest import ENV_BUILD_DIR, ENV_WORK_DIR + +from . import devicetree +from .log import LogParser + + +class UMLTestCase(unittest.TestCase): + run_separately: ClassVar[bool] = False + dts: ClassVar[Optional[devicetree.DtFragment]] = None + + +class UMLSuite(unittest.TestSuite): + def __init__( + self, + timeout: int, + workdir: str, + builddir: str, + ksrcdir: str, + uml_args_pre: list[str], + uml_args_post: list[str], + shell: bool, + *args: Any, + **kwargs: Any, + ) -> None: + super().__init__(*args, **kwargs) + + self.timeout = timeout + self.workdir = Path(workdir).resolve() + self.builddir = Path(builddir) + self.ksrcdir = Path(ksrcdir) + self.uml_args_pre = uml_args_pre + self.uml_args_post = uml_args_post + self.shell = shell + + self.backendlog = self.workdir / "backend.txt" + self.umllog = self.workdir / "uml.txt" + + # Used from the roadtest.cmd.remote running inside UML + self.testfile = self.workdir / "tests.json" + self.resultfile = self.workdir / "results.json" + + def run( + self, result: unittest.TestResult, debug: bool = False + ) -> unittest.TestResult: + pwd = os.getcwd() + + os.makedirs(self.workdir, exist_ok=True) + workdir = self.workdir + + tests = cast(list[UMLTestCase], list(self)) + + os.environ[ENV_WORK_DIR] = str(workdir) + os.environ[ENV_BUILD_DIR] = str(self.builddir) + + dt = devicetree.Devicetree(workdir=workdir, ksrcdir=self.ksrcdir) + dt.assemble([test.dts for test in tests if test.dts]) + dt.compile("test.dtb") + + testinfos = [] + ids = [] + for t in tests: + id = t.id() + # This fixup is needed when discover is done starting from "roadtest" + if not id.startswith("roadtest."): + id = f"roadtest.{id}" + ids.append(id) + + testinfos.append({"id": id, "values": t.dts.values if t.dts else {}}) + + with self.testfile.open("w") as f: + json.dump(testinfos, f) + + uml_args = [ + str(self.builddir / "vmlinux"), + f"PYTHONPATH={pwd}", + f"{ENV_WORK_DIR}={workdir}", + f"{ENV_BUILD_DIR}={self.builddir}", + # Should be enough for anybody? + "mem=64M", + "dtb=test.dtb", + "rootfstype=hostfs", + "rw", + f"init={pwd}/init.sh", + f"uml_dir={workdir}", + "umid=uml", + # ProxyTextTestResult writes to /dev/kmsg + "printk.devkmsg=on", + "slub_debug", + # For ease of debugging + "no_hash_pointers", + ] + + if self.shell: + # See init.sh + uml_args += ["ROADTEST_SHELL=1"] + else: + # Set by slub_debug + TAINT_BAD_PAGE = 1 << 5 + uml_args += [ + # init.sh increases the loglevel after bootup. + "quiet", + "panic_on_warn=1", + f"panic_on_taint={TAINT_BAD_PAGE}", + "oops=panic", + # Speeds up delays, but as a consequence also causes + # 100% CPU consumption at an idle shell prompt. + "time-travel", + ] + + main_script = (Path(__file__).parent / "../backend/main.py").resolve() + + args = ( + [ + str(self.builddir / "roadtest-backend"), + # The socket locations are also present in the devicetree. + f"--gpio-socket={workdir}/gpio.sock", + f"--i2c-socket={workdir}/i2c.sock", + f"--main-script={main_script}", + "--", + ] + + self.uml_args_pre + + uml_args + + self.uml_args_post + ) + + print( + "Running backend/UML with: {}".format( + " ".join([shlex.quote(a) for a in args]) + ) + ) + + # Truncate instead of deleting so that tail -f can be used to monitor + # the log across runs. + self.backendlog.write_text("") + self.umllog.write_text("") + self.resultfile.unlink(missing_ok=True) + + umlpidfile = workdir / "uml/pid" + umlpidfile.unlink(missing_ok=True) + + newenv = dict(os.environ, PYTHONPATH=pwd) + + try: + process = None + with self.backendlog.open("w") as f: + process = subprocess.Popen( + args, + env=newenv, + stdin=subprocess.PIPE, + stdout=f, + stderr=subprocess.STDOUT, + text=True, + preexec_fn=os.setsid, + ) + process.wait(self.timeout if self.timeout else None) + except subprocess.TimeoutExpired: + pass + finally: + try: + if process: + os.killpg(process.pid, signal.SIGKILL) + except ProcessLookupError: + pass + try: + pid = int(umlpidfile.read_text()) + os.killpg(pid, signal.SIGKILL) + except (FileNotFoundError, ProcessLookupError): + pass + + if process and process.returncode is not None and process.returncode != 0: + with self.backendlog.open("a") as f: + f.write(f"<Backend exited with error code {process.returncode}>\n") + + try: + with self.resultfile.open("r") as f: + proxy = json.load(f) + except FileNotFoundError: + # UML crashed, timed out, etc + proxy = None + + return self._convert_results(proxy, tests, result) + + def _parse_status(self, id: str, proxy: dict) -> Tuple[str, str]: + if not proxy: + return "ERROR", "No result. UML or backend crashed?\n" + + try: + _, tb = next(e for e in proxy["successes"] if e[0] == id) + return "ok", "" + except StopIteration: + pass + + try: + _, tb = next(e for e in proxy["errors"] if e[0] == id) + return "ERROR", tb + except StopIteration: + pass + + try: + _, tb = next(e for e in proxy["failures"] if e[0] == id) + return "FAIL", tb + except StopIteration: + pass + + # setupClass, etc + if proxy["errors"]: + _, tb = proxy["errors"][0] + return "ERROR", tb + + raise Exception("Unable to parse status") + + def _get_log( + self, name: str, parser: LogParser, id: str, full_if_none: bool + ) -> Optional[str]: + testloglines = parser.get_testcase_log(id) + tb = None + if testloglines: + tb = "\n".join([f"{name} log:"] + [" " + line for line in testloglines]) + elif full_if_none and not parser.has_any(): + if parser.raw: + tb = "\n".join( + [f"Full {name} log:", textwrap.indent(parser.raw, " ").rstrip()] + ) + else: + tb = f"\nNo {name} log found." + + return tb + + def _convert_results( + self, + proxy: dict, + tests: list[UMLTestCase], + result: TestResult, + ) -> TestResult: + umllog = LogParser(self.umllog) + backendlog = LogParser(self.backendlog) + + first_fail = True + for test in tests: + assert isinstance(test, unittest.TestCase) + + id = test.id() + if not id.startswith("roadtest."): + id = f"roadtest.{id}" + + status, tb = self._parse_status(id, proxy) + if status != "ok": + parts = [] + + backendtb = self._get_log("Backend", backendlog, id, first_fail) + if backendtb: + parts.append(backendtb) + + umltb = self._get_log("UML", umllog, id, first_fail) + if umltb: + parts.append(umltb) + + # In the case of no START/STOP markers at all in the logs, we include + # the full logs, but only do it in the first failing test case to + # reduce noise. + first_fail = False + tb = "\n\n".join(parts + [tb]) + + if status == "ERROR": + result.errors.append((test, tb)) + elif status == "FAIL": + result.failures.append((test, tb)) + + print(f"{test} ... {status}") + result.testsRun += 1 + + return result diff --git a/tools/testing/roadtest/roadtest/core/sysfs.py b/tools/testing/roadtest/roadtest/core/sysfs.py new file mode 100644 index 000000000000..64228978718e --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/sysfs.py @@ -0,0 +1,77 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import contextlib +from pathlib import Path +from typing import Iterator + + +# Path.write_text() is inappropriate since Python calls write(2) +# a second time if the first one returns an error, if the file +# was opened as text. +def write_str(path: Path, val: str) -> None: + path.write_bytes(val.encode()) + + +def write_int(path: Path, val: int) -> None: + write_str(path, str(val)) + + +def write_float(path: Path, val: float) -> None: + write_str(path, str(val)) + + +def read_str(path: Path) -> str: + return path.read_text().rstrip() + + +def read_int(path: Path) -> int: + return int(read_str(path)) + + +def read_float(path: Path) -> float: + return float(read_str(path)) + + +class I2CDevice: + def __init__(self, addr: int, bus: int = 0) -> None: + self.id = f"{bus}-{addr:04x}" + self.path = Path(f"/sys/bus/i2c/devices/{self.id}") + + +class PlatformDevice: + def __init__(self, name: str) -> None: + self.id = name + self.path = Path(f"/sys/bus/platform/devices/{self.id}") + + +class I2CDriver: + def __init__(self, driver: str) -> None: + self.driver = driver + self.path = Path(f"/sys/bus/i2c/drivers/{driver}") + + @contextlib.contextmanager + def bind(self, addr: int, bus: int = 0) -> Iterator[I2CDevice]: + dev = I2CDevice(addr, bus) + write_str(self.path / "bind", dev.id) + + try: + yield dev + finally: + write_str(self.path / "unbind", dev.id) + + +class PlatformDriver: + def __init__(self, driver: str) -> None: + self.driver = driver + self.path = Path(f"/sys/bus/platform/drivers/{driver}") + + @contextlib.contextmanager + def bind(self, addr: str) -> Iterator[PlatformDevice]: + dev = PlatformDevice(addr) + write_str(self.path / "bind", dev.id) + + try: + yield dev + finally: + write_str(self.path / "unbind", dev.id) diff --git a/tools/testing/roadtest/roadtest/core/test_control.py b/tools/testing/roadtest/roadtest/core/test_control.py new file mode 100644 index 000000000000..a8cf9105eb52 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/test_control.py @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase + +from .control import ControlReader, ControlWriter + + +class TestControl(TestCase): + def test_control(self) -> None: + with TemporaryDirectory() as tmpdir: + work = Path(tmpdir) + reader = ControlReader(work) + writer = ControlWriter(work) + + values = [] + + def append(new: int) -> None: + nonlocal values + values.append(new) + + vars = {"append": append} + writer.write_cmd("append(1)") + + reader.process(vars) + self.assertEqual(values, [1]) + + writer.write_cmd("append(2)") + writer.write_log("append(4)") + writer.write_cmd("append(3)") + + reader.process(vars) + self.assertEqual(values, [1, 2, 3]) diff --git a/tools/testing/roadtest/roadtest/core/test_devicetree.py b/tools/testing/roadtest/roadtest/core/test_devicetree.py new file mode 100644 index 000000000000..db61fd24b39a --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/test_devicetree.py @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import tempfile +import unittest +from pathlib import Path + +from . import devicetree + + +class TestDevicetree(unittest.TestCase): + def test_compile(self) -> None: + with tempfile.TemporaryDirectory() as tmp: + tmpdir = Path(tmp) + # We don't have the ksrcdir so we can't test if includes work. + dt = devicetree.Devicetree(tmpdir, tmpdir) + + dt.assemble( + [ + devicetree.DtFragment( + src=""" +&i2c { + foo = <1>; +}; + """ + ) + ] + ) + dt.compile("test.dtb") + dtb = tmpdir / "test.dtb" + self.assertTrue((dtb).exists()) diff --git a/tools/testing/roadtest/roadtest/core/test_hardware.py b/tools/testing/roadtest/roadtest/core/test_hardware.py new file mode 100644 index 000000000000..eb09b317e258 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/test_hardware.py @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase + +from roadtest.backend.mock import MockBackend + +from .hardware import Hardware + + +class TestHardware(TestCase): + def test_mock(self) -> None: + with TemporaryDirectory() as tmpdir: + work = Path(tmpdir) + + backend = MockBackend(work) + hw = Hardware(bus="dummy", work=work) + + backend.reg_write(0x1, 0xDEAD) + backend.reg_write(0x2, 0xBEEF) + mock = hw.update_mock() + mock.assert_reg_write_once(self, 0x1, 0xDEAD) + + backend.reg_write(0x1, 0xCAFE) + mock = hw.update_mock() + with self.assertRaises(AssertionError): + mock.assert_reg_write_once(self, 0x1, 0xDEAD) + + mock.assert_last_reg_write(self, 0x1, 0xCAFE) + + self.assertEqual(mock.get_last_reg_write(0x1), 0xCAFE) + self.assertEqual(mock.get_last_reg_write(0x2), 0xBEEF) + + with self.assertRaises(IndexError): + self.assertEqual(mock.get_last_reg_write(0x3), 0x0) + + mock.reset_mock() + with self.assertRaises(AssertionError): + mock.assert_last_reg_write(self, 0x2, 0xBEEF) diff --git a/tools/testing/roadtest/roadtest/core/test_log.py b/tools/testing/roadtest/roadtest/core/test_log.py new file mode 100644 index 000000000000..6988ff4419db --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/test_log.py @@ -0,0 +1,54 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from pathlib import Path +from tempfile import NamedTemporaryFile +from unittest import TestCase + +from .log import LogParser + + +class TestLog(TestCase): + def test_parser(self) -> None: + with NamedTemporaryFile() as tmpfile: + path = Path(tmpfile.name) + + path.write_text( + """ +xyz START<finished> +finished1 +finished2 +STOP<finished> +START<empty> +STOP<empty> +START<foo> monkey STOP<foo> +START<unfinished> +unfinished1 +unfinished2""" + ) + + parser = LogParser(path) + self.assertEqual( + parser.get_testcase_log("finished"), ["finished1", "finished2"] + ) + + self.assertEqual( + parser.get_testcase_log("unfinished"), + ["unfinished1", "unfinished2", LogParser.DNF_MESSAGE], + ) + + self.assertEqual( + parser.get_testcase_log("notpresent"), + [], + ) + + self.assertEqual( + parser.get_testcase_log("enpty"), + [], + ) + + # Shouldn't happen since we print from the kernel? + self.assertEqual( + parser.get_testcase_log("foo"), + [], + ) diff --git a/tools/testing/roadtest/roadtest/core/test_opslog.py b/tools/testing/roadtest/roadtest/core/test_opslog.py new file mode 100644 index 000000000000..bd594c587032 --- /dev/null +++ b/tools/testing/roadtest/roadtest/core/test_opslog.py @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from pathlib import Path +from tempfile import TemporaryDirectory +from unittest import TestCase + +from .opslog import OpsLogReader, OpsLogWriter + + +class TestOpsLOg(TestCase): + def test_opslog(self) -> None: + with TemporaryDirectory() as tmpdir: + work = Path(tmpdir) + writer = OpsLogWriter(work) + reader = OpsLogReader(work) + + self.assertEqual(reader.read_next(), []) + + writer.write("1") + writer.write("2") + + self.assertEqual(reader.read_next(), ["1", "2"]) + self.assertEqual(reader.read_next(), []) + + writer.write("3") + self.assertEqual(reader.read_next(), ["3"]) diff --git a/tools/testing/roadtest/roadtest/tests/__init__.py b/tools/testing/roadtest/roadtest/tests/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1
Add the base config options for the roadtest kernel (generated with "savedefconfig").
roadtest uses a single kernel for all tests and the drivers under test are built as modules. Additional config options are added by merging config fragments from each subsystems' test directory.
The kernel is built with several debug options to catch more problems during testing.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- .../roadtest/roadtest/tests/base/config | 84 +++++++++++++++++++ 1 file changed, 84 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/base/config
diff --git a/tools/testing/roadtest/roadtest/tests/base/config b/tools/testing/roadtest/roadtest/tests/base/config new file mode 100644 index 000000000000..c1952d047c8e --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/base/config @@ -0,0 +1,84 @@ +CONFIG_NO_HZ=y +CONFIG_HIGH_RES_TIMERS=y +CONFIG_LOG_BUF_SHIFT=14 +CONFIG_EXPERT=y +CONFIG_HOSTFS=y +CONFIG_UML_TIME_TRAVEL_SUPPORT=y +CONFIG_NULL_CHAN=y +CONFIG_PORT_CHAN=y +CONFIG_PTY_CHAN=y +CONFIG_TTY_CHAN=y +CONFIG_XTERM_CHAN=y +CONFIG_CON_CHAN="pts" +CONFIG_SSL=y +CONFIG_SSL_CHAN="pts" +CONFIG_MAGIC_SYSRQ=y +CONFIG_VIRTIO_UML=y +CONFIG_UML_PCI_OVER_VIRTIO=y +CONFIG_UML_PCI_OVER_VIRTIO_DEVICE_ID=1234 +CONFIG_GCOV_KERNEL=y +CONFIG_MODULES=y +CONFIG_MODULE_UNLOAD=y +CONFIG_BINFMT_MISC=m +# CONFIG_COMPACTION is not set +CONFIG_DEVTMPFS=y +CONFIG_DEVTMPFS_MOUNT=y +CONFIG_OF=y +# CONFIG_INPUT is not set +CONFIG_LEGACY_PTY_COUNT=32 +CONFIG_HW_RANDOM=y +# CONFIG_HW_RANDOM_IXP4XX is not set +# CONFIG_HW_RANDOM_STM32 is not set +# CONFIG_HW_RANDOM_MESON is not set +# CONFIG_HW_RANDOM_CAVIUM is not set +# CONFIG_HW_RANDOM_MTK is not set +# CONFIG_HW_RANDOM_EXYNOS is not set +# CONFIG_HW_RANDOM_NPCM is not set +# CONFIG_HW_RANDOM_KEYSTONE is not set +CONFIG_RANDOM_TRUST_BOOTLOADER=y +CONFIG_I2C=y +# CONFIG_I2C_COMPAT is not set +CONFIG_I2C_CHARDEV=y +CONFIG_I2C_VIRTIO=y +CONFIG_I2C_STUB=m +CONFIG_PPS=y +CONFIG_GPIOLIB=y +CONFIG_GPIO_VIRTIO=y +CONFIG_NET=y +CONFIG_UNIX=y +CONFIG_NEW_LEDS=y +CONFIG_LEDS_CLASS=y +CONFIG_LEDS_GPIO=y +CONFIG_LEDS_TRIGGERS=y +CONFIG_LEDS_TRIGGER_HEARTBEAT=y +CONFIG_RTC_CLASS=y +# CONFIG_RTC_HCTOSYS is not set +# CONFIG_RTC_SYSTOHC is not set +CONFIG_RTC_DEBUG=y +# CONFIG_RTC_NVMEM is not set +CONFIG_VIRTIO_INPUT=y +# CONFIG_BCM_VIDEOCORE is not set +CONFIG_QUOTA=y +CONFIG_AUTOFS4_FS=m +CONFIG_PROC_KCORE=y +CONFIG_TMPFS=y +CONFIG_NLS=y +CONFIG_CRYPTO=y +CONFIG_CRYPTO_CRC32C=y +CONFIG_CRYPTO_JITTERENTROPY=y +CONFIG_CRC16=y +CONFIG_PRINTK_TIME=y +CONFIG_PRINTK_CALLER=y +CONFIG_DYNAMIC_DEBUG=y +CONFIG_DEBUG_INFO=y +CONFIG_FRAME_WARN=1024 +CONFIG_READABLE_ASM=y +CONFIG_DEBUG_FS=y +CONFIG_UBSAN=y +CONFIG_PAGE_EXTENSION=y +CONFIG_DEBUG_OBJECTS=y +CONFIG_DEBUG_OBJECTS_FREE=y +CONFIG_DEBUG_OBJECTS_TIMERS=y +CONFIG_DEBUG_OBJECTS_WORK=y +CONFIG_PROVE_LOCKING=y +CONFIG_ENABLE_DEFAULT_TRACERS=y
Add a Makefile and other miscellaneous build-related files for the roadtest framework.
To make it easier to run the tests on systems which do not have the required libraries or Python version, a Dockerfile is included and the Makefile has built-in support for running the tests in a Docker container.
Targets for code formatting and static checking of the Python code are included.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- tools/testing/roadtest/.gitignore | 2 + tools/testing/roadtest/Dockerfile | 25 ++++++++ tools/testing/roadtest/Makefile | 84 +++++++++++++++++++++++++ tools/testing/roadtest/pyproject.toml | 10 +++ tools/testing/roadtest/requirements.txt | 4 ++ tools/testing/roadtest/src/.gitignore | 1 + 6 files changed, 126 insertions(+) create mode 100644 tools/testing/roadtest/.gitignore create mode 100644 tools/testing/roadtest/Dockerfile create mode 100644 tools/testing/roadtest/Makefile create mode 100644 tools/testing/roadtest/pyproject.toml create mode 100644 tools/testing/roadtest/requirements.txt create mode 100644 tools/testing/roadtest/src/.gitignore
diff --git a/tools/testing/roadtest/.gitignore b/tools/testing/roadtest/.gitignore new file mode 100644 index 000000000000..0cbd00343694 --- /dev/null +++ b/tools/testing/roadtest/.gitignore @@ -0,0 +1,2 @@ +__pycache__ +.py[cod] diff --git a/tools/testing/roadtest/Dockerfile b/tools/testing/roadtest/Dockerfile new file mode 100644 index 000000000000..f2982179c327 --- /dev/null +++ b/tools/testing/roadtest/Dockerfile @@ -0,0 +1,25 @@ +FROM debian:bullseye + +# Kernel build +RUN apt-get update && apt-get -y install \ + bc \ + build-essential \ + flex \ + bison \ + rsync \ + kmod + +# Running roadtests +RUN apt-get update && apt-get -y install \ + python3.9 \ + libpython3.9-dev \ + python3 \ + device-tree-compiler + +# Development and debugging +RUN apt-get update && apt-get -y install \ + uml-utilities \ + telnetd \ + python3-pip +COPY requirements.txt /tmp/ +RUN pip install --requirement /tmp/requirements.txt diff --git a/tools/testing/roadtest/Makefile b/tools/testing/roadtest/Makefile new file mode 100644 index 000000000000..525b26581142 --- /dev/null +++ b/tools/testing/roadtest/Makefile @@ -0,0 +1,84 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +.PHONY: all build-kernel test clean check fmt docker-run + +all: + +KSOURCE := ${PWD} +ROADTEST_DIR = ${CURDIR} +ROADTEST_BUILD_DIR := ${KSOURCE}/.roadtest +KHEADERS := ${ROADTEST_BUILD_DIR}/usr +KMODULES := ${ROADTEST_BUILD_DIR}/modules + +ifeq (${KSOURCE},${ROADTEST_DIR}) +# Make make from the standard roadtest directory work without having to set +# additional variables. +KSOURCE=$(ROADTEST_DIR:/tools/testing/roadtest=) +endif + +CFLAGS += -g -D_GNU_SOURCE=1 -Wall -Werror -std=gnu99 \ + -I${KSOURCE}/tools/include/ \ + -I${KHEADERS}/include/ \ + -I${ROADTEST_DIR}/src/libvhost-user/ \ + $(shell python3-config --embed --includes) -O2 + +${ROADTEST_BUILD_DIR}/roadtest-backend: ${ROADTEST_BUILD_DIR}/backend.o ${ROADTEST_BUILD_DIR}/libvhost-user.o + $(CC) -o $@ $^ $(shell python3-config --embed --libs) + # For the benefit of clangd + echo ${CFLAGS} | tr " " "\n" > ${ROADTEST_DIR}/src/compile_flags.txt + +${ROADTEST_BUILD_DIR}/backend.o: src/backend.c + $(CC) -c -o $@ $(CFLAGS) $< + +${ROADTEST_BUILD_DIR}/libvhost-user.o: src/libvhost-user/libvhost-user.c + $(CC) -c -o $@ $(CFLAGS) $< + +clean: + rm -rf ${ROADTEST_BUILD_DIR} .docker_built + +ifeq ($(DOCKER),1) +.docker_built: Dockerfile requirements.txt + docker build --network=host -t roadtest ${ROADTEST_DIR} + touch $@ + +# --network=host allows UML's con=port:... to work seamlessly +docker-run: .docker_built + mkdir -p ${ROADTEST_BUILD_DIR}/umltmp + docker run --network=host ${DOCKEROPTS} --user $(shell id -u ${USER}):$(shell id -g ${USER}) --interactive --tty --rm -v ${KSOURCE}:${KSOURCE} -w ${KSOURCE} --env TMPDIR=${ROADTEST_BUILD_DIR}/umltmp roadtest sh -c '${MAKE} -C ${ROADTEST_DIR} -${MAKEFLAGS} ${MAKECMDGOALS} DOCKER=0' + +all test build-kernel check fmt: docker-run + @: +else +all: test + +ifneq ($(KBUILD),0) +# Calling make on the kernel is slow even if there is nothing to be rebuilt. +# Allow the user to avoid it with KBUILD=0 +${ROADTEST_BUILD_DIR}/backend.o: build-kernel +${ROADTEST_BUILD_DIR}/libvhost-user.o: build-kernel +test: build-kernel +endif + +build-kernel: + mkdir -p ${ROADTEST_BUILD_DIR} + find ${ROADTEST_DIR}/roadtest/tests/ -type f -name config | xargs cat > ${ROADTEST_BUILD_DIR}/.config + ${MAKE} -C ${KSOURCE} ARCH=um O=${ROADTEST_BUILD_DIR} olddefconfig + ${MAKE} -C ${KSOURCE} ARCH=um O=${ROADTEST_BUILD_DIR} + ${MAKE} -C ${KSOURCE} ARCH=um O=${ROADTEST_BUILD_DIR} INSTALL_HDR_PATH=${KHEADERS} headers_install + ${MAKE} -C ${KSOURCE} ARCH=um O=${ROADTEST_BUILD_DIR} INSTALL_MOD_PATH=${KMODULES} modules_install + +test: ${ROADTEST_BUILD_DIR}/roadtest-backend + python3 -m roadtest.cmd.main --ksrc-dir ${KSOURCE} --build-dir ${ROADTEST_BUILD_DIR} --work-dir ${ROADTEST_BUILD_DIR}/roadtest-work/ ${OPTS} + +check: + mypy --no-error-summary roadtest + pyflakes roadtest + black --check roadtest + isort --profile black --check roadtest + +fmt: + black roadtest + isort --profile black roadtest + +endif diff --git a/tools/testing/roadtest/pyproject.toml b/tools/testing/roadtest/pyproject.toml new file mode 100644 index 000000000000..6b8b05eb3cad --- /dev/null +++ b/tools/testing/roadtest/pyproject.toml @@ -0,0 +1,10 @@ +[tool.isort] +profile = "black" + +[tool.mypy] +disallow_untyped_defs = true +check_untyped_defs = true +no_implicit_optional = true +warn_return_any = true +warn_unused_ignores = true +show_error_codes = true diff --git a/tools/testing/roadtest/requirements.txt b/tools/testing/roadtest/requirements.txt new file mode 100644 index 000000000000..e1ac403d826e --- /dev/null +++ b/tools/testing/roadtest/requirements.txt @@ -0,0 +1,4 @@ +black==22.1.0 +isort==5.10.1 +mypy==0.931 +pyflakes==2.4.0 diff --git a/tools/testing/roadtest/src/.gitignore b/tools/testing/roadtest/src/.gitignore new file mode 100644 index 000000000000..895dab3fe4be --- /dev/null +++ b/tools/testing/roadtest/src/.gitignore @@ -0,0 +1 @@ +compile_flags.txt
Add documentation for the roadtest device driver testing framework. This includes a "how to write your first test" tutorial.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/roadtest.rst | 669 +++++++++++++++++++++++++++ 2 files changed, 670 insertions(+) create mode 100644 Documentation/dev-tools/roadtest.rst
diff --git a/Documentation/dev-tools/index.rst b/Documentation/dev-tools/index.rst index 4621eac290f4..44fea7c50dad 100644 --- a/Documentation/dev-tools/index.rst +++ b/Documentation/dev-tools/index.rst @@ -33,6 +33,7 @@ Documentation/dev-tools/testing-overview.rst kselftest kunit/index ktap + roadtest
.. only:: subproject and html diff --git a/Documentation/dev-tools/roadtest.rst b/Documentation/dev-tools/roadtest.rst new file mode 100644 index 000000000000..114bf822e376 --- /dev/null +++ b/Documentation/dev-tools/roadtest.rst @@ -0,0 +1,669 @@ +======== +Roadtest +======== + +Roadtest is a device-driver testing framework. It tests drivers under User +Mode Linux using models of the hardware. The tests cases and hardware models +are written in Python, the former using the built-in unittest framework. + +Roadtest is meant to be used for relatively simple drivers, such as the ones +part of the IIO, regulator or RTC subsystems. + +Drivers are tested via their userspace interfaces and interact with hardware +models which allow tests to inject values into registers and assert that +drivers control the hardware in the right way and react as expected to stimuli. + +Installing the requirements +=========================== + +Addition to the normal requirements for building kernels, *running* roadtest +requires Python 3.9 or later, including the development libraries: + +.. code-block:: shell + + apt-get -y install python3.9 libpython3.9-dev device-tree-compiler + +There is also support for running the tests in a Docker container without +having to install any packages. + +Running roadtest +================ + +To run the tests, run the following command from the base of a kernel source +tree: + +.. code-block:: shell + + $ make -C tools/testing/roadtest + +Or, if you prefer to use the Docker container: + +.. code-block:: shell + + $ make -C tools/testing/roadtest DOCKER=1 + +Either of these commands will build a kernel and run all roadtests. + +.. note:: + + Roadtest builds the kernel out-of-tree. The kernel build system may instruct + you to clean your tree if you have previously performed an in-tree build. You + can pass the usual ``-jNN`` options to parallelize the build. The tests + themselves are currently always run sequentially. + +Writing roadtests +================= + +Tutorial: Writing your first roadtest +------------------------------------- + +You may find it simplest to have a look at the existing tests and base your new +tests on them, but if you prefer, this section provides a tutorial which will +guide you to write a new basic test from scratch. + +Even if you're not too keen on following the tutorial hands-on, you're +encouraged to skim through it since there are useful debugging tips and notes +on roadtest's internals which could be useful to know before diving in and +writing tests. + +A quick note on the terminology before we begin: we'll refer to the framework +itself as "roadtest" or just "the framework", and we'll call a driver test +which uses this framework a "roadtest" or just a "test". + +Goal for the test +~~~~~~~~~~~~~~~~~ + +In this tutorial, we'll add a basic test for one of the features of the +VCNL4000 light sensor driver which is a part of the IIO subsystem +(``drivers/iio/light/vcnl4000.c``). + +This driver supports a bunch of related proximity and ambient light sensor +chips which communicate using the I2C protocol; we'll be testing the VCNL4000 +variant. The datasheet for the chip is, at the time of writing, available +`here https://cdn-shop.adafruit.com/datasheets/vcnl4000.pdf`_. + +The test will check that the driver correctly reads and reports the illuminance +values from the hardware to userspace via the IIO framework. + +Test file placement +~~~~~~~~~~~~~~~~~~~ + +Roadtests are placed under ``tools/testing/roadtest/roadtest/tests``. (In case +you're wondering, the second ``roadtest`` is to create a Python package, so +that imports of ``roadtest`` work without having to mess with module search +paths.) + +Tests are organized by subsystem. Normally we'd put our IIO light sensor tests +under ``iio/light/`` (below the ``tests`` directory), but since there is +already a VCNL4000 test there, we'll create a new subsystem directory called +``tutorial`` and put our test there in a new file called ``test_tutorial.py``. + +We'll also need to create an empty ``__init__.py`` in that directory to allow +Python to recognize it as a package. + +All the commands in this tutorial should be executed from the +``tools/testing/roadtest`` directory inside the kernel source tree. (To reduce +noise, we won't show the current working directory before the ``$`` in future +command line examples.) + +.. code-block:: shell + + tools/testing/roadtest$ mkdir -p roadtest/tests/tutorial/ + tools/testing/roadtest$ touch roadtest/tests/tutorial/__init__.py + +Building the module +~~~~~~~~~~~~~~~~~~~ + +First, we'll need to ensure that our driver is built. To do that, we'll add +the appropriate config option to built our driver as a module. The lines +should be written to a new file called ``config`` in the ``tutorial`` +directory. Roadtest will gather all ``config`` files placed anywhere under +``tests`` and build a kernel with the combined config. + +.. code-block:: shell + + $ echo CONFIG_VCNL4000=m >> roadtest/tests/tutorial/config + +.. note:: + + This driver will actually be built even if you don't add this config, since + it's already present in the ``roadtest/tests/iio/light/config`` used by the + existing VCNL4000 test. Roadtest uses a single build for all tests. + +Loading the module from the test +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We've set up our module to be built, so we can now start working on the test +case iself. We'll start with the following few lines of code. Tests are +written Python's built-in `unittest +https://docs.python.org/3/library/unittest.html`_ module. This tutorial will +assume familiariy with that framework; see the Python documentation for more +information. + +Test classes should subclass ``roadtest.core.suite.UMLTestCase`` instead of +``unittest.TestCase``. This informs the roadtest core code that the test +should be run inside UML. + +.. note:: + + There are several "real" unit tests for the framework itself; these subclass + ``unittest.TestCase`` directly and are run on the host system. You'll see + these run in the beginning when you run roadtest. + +All this test currently does is insert our driver's module, do nothing, and +then remove our driver's kernel module. (The ``roadtest.core.modules.Module`` +class implements a ``ContextManager`` which automatically cleans up using the +``with`` statement.) + +.. code-block:: python + + from roadtest.core.suite import UMLTestCase + from roadtest.core.modules import Module + + class TestTutorial(UMLTestCase): + def test_illuminance(self) -> None: + with Module("vcnl4000"): + pass + +You can now build the kernel and run roadtest with: + +.. code-block:: shell + + $ make + +.. note:: + + Make sure you have all the dependencies described at the beginning of the + document installed. You can also use a Docker container, append ``DOCKER=1`` + to all the ``make`` commands in this tutorial if you want to do that. + +You should see your new test run and pass in the output of the above command: + +.. code-block:: + + ... + test_illuminance (tests.tutorial.test_tutorial.TestTutorial) ... ok + ... + +Shortening feedback loops +~~~~~~~~~~~~~~~~~~~~~~~~~ + +While just running ``make`` runs your new test, it also runs all the *other* +tests too, and what's more, it calls in to the kernel build system every time, +and that can be relatively slow even if there's nothing to be rebuilt. + +When you're only working on writing tests, and not modifying the driver or the +kernel source, you can avoid calling into Kbuild by passing ``KBUILD=0`` to the +``make`` invocation. For example: + +.. code-block:: shell + + $ make KBUILD=0 + +To only run specific tests, you can use the ``--filter`` option to roadtest's +main script (implemented in ``roadtest.cmd.main``) which takes a wildcard +pattern. Only tests whoses names match the pattern are run. + +Options to the main script are passed via the ``OPTS`` variable. So the +following would both skip the kernel build and only run your test: + +.. code-block:: shell + + $ make KBUILD=0 OPTS="--filter tutorial" + +.. tip:: + + Roadtest builds the kernel inside a directory called ``.roadtest`` in your + kernel source tree. Logs from UML are saved as + ``.roadtest/roadtest-work/uml.txt`` and logs from roadtest's backend (more on + that later) are at ``.roadtest/roadtest-work/backend.txt``. It's sometimes + useful to keep a terminal open running ``tail -f`` on these files while + developing roadtests. + +Adding a device +~~~~~~~~~~~~~~~ + +Our basic test only loads and unloads the module, so the next step is to +actually get our driver to probe and bind to a device. On many systems, +devices are instantiated based on the hardware descriptions in devicetree, and +this is the case on roadtest's UML-based system too. See +:ref:`Documentation/driver-api/driver-model/binding.rst <binding>` and +:ref:`Documentation/devicetree/usage-model.rst <usage-model>` for more +information. + +When working on real harwdare, the hardware design specifies at what address +and on which I2C bus the hardware sensor chip is connected. Roadtest provides +a virtual I2C bus and the test can chose to place devices at any valid address +on this bus. + +In this tutorial, we'll use a hard coded device address of ``0x42`` and set the +``run_separately`` flag on the test, asking roadtest to run our test in a +separate UML instance so that we know that no other test has tried to put a +device at that I2C address. + +.. note:: + + Normally, roadtests use what the framework refers to as *relocatable + devicetree fragments* (unrelated to the fragments used in devicetree + overlays). These do not use fixed addreses for specific devices, but instead + allow the framework to freely assign addresses. This allows several + different, independent tests can be run using one devicetree and one UML + instance (to save on startup time costs), without having to coordinate + selection of device addesses. + + When writing "real" roadtests (after you're done with this tutorial), you too + should use relocatable fragments. See the existing tests for examples. + +The framework's devicetree module (``roadtest.core.devicetree``) includes a +base tree that provides an I2C controller node (appropriately named ``i2c``) +for the virtual I2C, so we will add our new device under that node. + +Unlike on a default Linux system, just adding the node to the devicetree won't +get our I2C driver to automatically bind to the driver when we load the module. +This is because roadtest's ``init.sh`` (a script which runs inside UML after +the kernel boots up) turns off automatic probing on the I2C bus, in order to +give the test cases full control of when things get probed. + +So we'll have ask the ``test_illuminance()`` method to get the ``vcnl4000`` +driver (that's the name of the I2C driver which the module registers, and +that's not necessarily the same as the name of the module) to explicitly bind +to our chosen ``0x42`` I2C device using some of the helper classes in the +framework: + +.. code-block:: python + + from roadtest.core.devicetree import DtFragment + from roadtest.core.devices import I2CDriver + + class TestTutorial(UMLTestCase): + run_separately = True + dts = DtFragment( + src=""" + &i2c { + light-sensor@42 { + compatible = "vishay,vcnl4000"; + reg = <0x42>; + }; + }; + """, + ) + + def test_illuminance(self) -> None: + with ( + Module("vcnl4000"), + I2CDriver("vcnl4000").bind(0x42) as dev, + ): + pass + +You can run this test using the same ``make`` command you used previously. +This time, rather than an "ok", you should see roadtest complain about an error +during your test: + +.. code-block:: + + ====================================================================== + ERROR: test_illuminance (tests.tutorial.test_tutorial.TestTutorial) + ---------------------------------------------------------------------- + Backend log: + Traceback (most recent call last): + File ".../roadtest/backend/i2c.py", line 35, in write + raise Exception("No I2C model loaded") + Exception: No I2C model loaded + Traceback (most recent call last): + File ".../roadtest/backend/i2c.py", line 29, in read + raise Exception("No I2C model loaded") + Exception: No I2C model loaded + + UML log: + [ 1220.410000][ T19] vcnl4000: probe of 0-0042 failed with error -5 + + Traceback (most recent call last): + File ".../roadtest/tests/tutorial/test_tutorial.py", line 21, in test_illuminance + with ( + File "/usr/lib/python3.9/contextlib.py", line 119, in __enter__ + return next(self.gen) + File ".../roadtest/core/devices.py", line 32, in bind + f.write(dev.id.encode()) + OSError: [Errno 5] Input/output error + +To understand and fix this error, we'll have to learn a bit about how roadtest +works under the hood. + +Adding a hardware model +~~~~~~~~~~~~~~~~~~~~~~~ + +Roadtest's *backend* is what allows the hardware to modelled for the sake of +driver testing. The backend runs outside of UML and communication between the +drivers and the models goes via ``virtio-uml``, a shared-memory based +communication protocol. At its lowest level, the backend is written in C and +implements virtio devices for ``virtio-i2c`` and ``virtio-gpio``, both of which +have respective virtio drivers which run inside UML and provide the virtual I2C +bus (and GPIO controller) whose nodes are available in the devicetree. + +The C backend embeds a Python interpreter which runs a Python module which +implements the I2C bus model. It's that Python module which is complaining now +that it does not have any I2C device model to handle the I2C transactions that +it received from UML. This is quite understandable since we haven't +implemented one yet! + +.. note:: + + In the error message above, you'll also notice an error ``printk()`` from the + driver (as part of the *UML log*, which includes kernel console messages), as + well as the exception stacktrace from the test case itself. The ``-EIO`` + seen inside UML is a result of the roadtest backend failing the I2C + transaction due to the exception. + +Models are placed in the same source file as the test cases. The model and +the test cases will however run in two different Python interpreters on two +different systems (the test case inside UML, and the model inside the backend +on your host). + +For I2C, the interface our model needs to implement is specified by the +Abstract Base Class ``roadtest.backend.i2c.I2CModel`` (which can be found, +following Python's standard naming conventions, in the file +``roadtest/backend/i2c.py``). You can see that it expects the model to +implement ``read()`` and ``write()`` functions which transmit and receive the +raw bytes of the I2C transaction. + +Our VCNL4000 device uses the SMBus protocol which is a subset of the I2C +protocol, so we can use a higher-level class to base our implementation off, +``roadtest.backend.i2c.SMBusModel``. This one takes care of doing segmentation +of the I2C requests, and expects subclasses to implement ``reg_read()`` and +``reg_write()`` methods which will handle the register access for the device. + +For our initial model, we'll just going to just make our ``reg_read()`` and +``reg_write()`` methods read and store the register values in a dictionary. +We'll need some initial values for the registers, and for these we use the +values which are specified in the VCNL4000's datasheet. We won't bother with +creating constants for the register addresses and we'll just specify them in +hex: + +.. code-block:: python + + from typing import Any + from roadtest.backend.i2c import SMBusModel + + class VCNL4000(SMBusModel): + def __init__(self, **kwargs: Any) -> None: + super().__init__(regbytes=1, **kwargs) + self.regs = { + 0x80: 0b_1000_0000, + 0x81: 0x11, + 0x82: 0x00, + 0x83: 0x00, + 0x84: 0x00, + 0x85: 0x00, + 0x86: 0x00, + 0x87: 0x00, + 0x88: 0x00, + 0x89: 0x00, + } + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + return val + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + self.regs[addr] = val + +Then we need to modify the test function to ask the backend to load this model: + +.. code-block:: python + :emphasize-lines: 1,6 + + from roadtest.core.hardware import Hardware + + def test_illuminance(self) -> None: + with ( + Module("vcnl4000"), + Hardware("i2c").load_model(VCNL4000), + I2CDriver("vcnl4000").bind(0x42), + ): + pass + +Now run the test again. You should see the test pass, meaning that the driver +successfully talked to and recognized your hardware model. (You can look at +the UML and backend logs mentioned earlier to confirm this.) + +.. tip:: + + You can add arbitrary command line arguments to UML using the + ``--uml-append`` option. For example, while developing tests for I2C + drivers, it could be helpful to turn on the appropriate trace events and + arrange for them to be printed to the console (which you can then access via + the previously mentioned ``uml.txt``.): + + .. code-block:: + + OPTS="--filter tutorial --uml-append tp_printk trace_event=i2c:*" + +Exploring the target +~~~~~~~~~~~~~~~~~~~~ + +Now that we've gotten the driver to probe to our new device, we want to get the +test to read the illuminance value from the driver. However, which file should +the test read the value from? IIO exposes the illuminance value in a sysfs +file, but where do we find this file? + +If you have real hardware with a VCNL4000 chip and already running the vcnl4000 +driver, or are already very familiar with the IIO framework, you likely already +know what sysfs files to read, but in our case, we can open up a shell on UML +to manually explore the system and find the relevant sysfs files before +implementing the rest of the test case. + +Roadtest's ``--shell`` option makes UML start a shell instead of exiting after +the tests are run. However, since our test case cleans up after itself (as +it should) using context managers, neither the module nor the model would +remain loaded after the test exists, which would make manual exploration +difficult. + +To remedy this, we can combine ``--shell`` with temporary code in our test to +_exit(2) after setting up everything: + +.. code-block:: python + :emphasize-lines: 5,7 + + def test_illuminance(self) -> None: + with ( + Module("vcnl4000"), + Hardware("i2c").load_model(VCNL4000), + I2CDriver("vcnl4000").bind(0x42) as dev, + ): + print(dev.path) + import os; os._exit(1) + +.. note:: + + The communication between the test cases and the models uses a simple text + based protocol where the test cases write Python expressions to a file which + the backend reads and evaluates, so it is possible to load a model using only + shell commands, but this is undocumented. See the source code if you need to + do this. + +We'll also need to ask UML to open up a terminal emulator (``con=xterm``) or start a telnet server +and wait for a connection (``con=port:9000``). See +:ref:`Documentation/virt/uml/user_mode_linux_hotwo_v2.rst +<user_mode_linux_hotwo_v2>` for more information about the required packages. +These options can be passed to UML using ``--uml-append``. So the final +``OPTS`` argument is something like the following (you can combine this with +the tracing options): + +.. code-block:: + + OPTS="--shell --uml-append con=xterm" + +.. tip:: + + ``con=xterm doesn``'t work in the Docker container, so use the telnet option + if you're running roadtest inside Docker. ``screen -L //telnet localhost + 9000`` or similar can be used to connect to UML. + + When running *without* using Docker, the telnet option tends to leave UML's + ``port-helper`` running in the background, so you may have to ``kill(1)`` it + yourself after each run. + +Using the shell, you should be able to find the illuminance file under the +device's sysfs path: + +.. code-block:: + + root@(none):/sys/bus/i2c/devices/0-0042# ls -1 iio:device0/in* + iio:device0/in_illuminance_raw + iio:device0/in_illuminance_scale + iio:device0/in_proximity_nearlevel + iio:device0/in_proximity_raw + +You can also attempt to read the ``in_illuminance_raw`` file; you should see +that it fails with something like this (with the trace events enabled): + +.. code-block:: + + root@(none):/sys/bus/i2c/devices/0-0042# cat iio:device0/in_illuminance_raw + [ 151.270000][ T34] i2c_write: i2c-0 #0 a=042 f=0000 l=2 [80-10] + [ 151.270000][ T34] i2c_result: i2c-0 n=1 ret=1 + ... + [ 152.030000][ T34] i2c_write: i2c-0 #0 a=042 f=0000 l=1 [80] + [ 152.030000][ T34] i2c_read: i2c-0 #1 a=042 f=0001 l=1 + [ 152.030000][ T34] i2c_reply: i2c-0 #1 a=042 f=0001 l=1 [10] + [ 152.030000][ T34] i2c_result: i2c-0 n=2 ret=2 + [ 152.070000][ T34] vcnl4000 0-0042: vcnl4000_measure() failed, data not ready + +Controlling register values +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Our next challenge is to get the ``in_illuminance_raw`` file to be read +successfully. From the I2C trace events above, or from looking at the +``backend.txt`` (below), we can see that the driver repeatedly reads a +particular register. + +.. code-block:: + + INFO - roadtest.core.control: START<roadtest.tests.tutorial.test_tutorial.TestTutorial.test_illuminance> + DEBUG - roadtest.core.control: backend.i2c.load_model(*('roadtest.tests.tutorial.test_tutorial', 'VCNL4000'), **{}) + DEBUG - roadtest.backend.i2c: SMBus read addr=0x81 val=0x11 + DEBUG - roadtest.backend.i2c: SMBus write addr=0x80 val=0x10 + DEBUG - roadtest.backend.i2c: SMBus read addr=0x80 val=0x10 + DEBUG - roadtest.backend.i2c: SMBus read addr=0x80 val=0x10 + ... + +To understand this register, we need to take a look at the chip's datasheet and +compare it with the driver code. By doing so, we can see the driver is waiting +for the hardware to signal that the data is ready by polling for a particular +bit to be set. + +One simple way to set the data ready bit, which we'll use for the purpose of +this tutorial, is to simply ensure that the model always returns reads to the +0x80 register with that bit set. + +.. note:: + + This method wouldn't allow a test to be written to test the timeout handling, + but we won't bother with that in this tutorial. You can explore the exising + roadtests for alternative solutions, such as setting the data ready bit + whenever the test injects new data and clearing it when the driver reads the + data. + +.. code-block:: python + :emphasize-lines: 4,5 + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + + if addr == 0x80: + val |= 1 << 6 + + return val + +This should get the bit set and make the read succeed (you can check this using +the shell), but we'd also like to return different values from the data +registers rather the reset values we hardcoded in ``__init__``. One way to do +this is to have the test inject the values into the ALS result registers by +having it call the ``reg_write()`` method of the model. It can do this via the +``Hardware`` object. + +.. note:: + + The test can call methods on the model but it can't receive return values + from these methods, nor can it set attributes on the model. The model and + the test run on different systems and communication between them is + asynchronous. + +We'll combine this with a read of the sysfs file we identified and throw in an +assertion to check that the value which the driver reports to userspace via +that file matches the value which we inject into the hardware's result +registers: + +.. code-block:: python + :emphasize-lines: 6,8,9-13 + + from roadtest.core.sysfs import read_int + + def test_illuminance(self) -> None: + with ( + Module("vcnl4000"), + Hardware("i2c").load_model(VCNL4000) as hw, + I2CDriver("vcnl4000").bind(0x42) as dev, + ): + hw.reg_write(0x85, 0x12) + hw.reg_write(0x86, 0x34) + self.assertEqual( + read_int(dev.path / "iio:device0/in_illuminance_raw", 0x1234) + ) + +And that's it for this tutorial. We've written a simple end-to-end test for +one aspect of this driver with the help of a minimal model of the hardware. + +Verifying drivers' interactions with the hardware +------------------------------------------------- + +The tutorial covered injection of values into hardware registers and how to +check that the driver interprets the value exposed by the hardware correctly, +but another important aspect of testing device drivers is to verify that the +driver actually *controls* the hardware in the expected way. + +For example, if you are testing a regulator driver, you want to test that +driver actually writes the correct voltage register in the hardware with the +correct value when the driver is asked to set a voltage using the kernel's +regulator API. + +To support this, roadtest integrates with Python's built-in `unittest.mock +https://docs.python.org/3/library/unittest.mock.html`_ library. The +``update_mock()`` method on the ``Hardware`` objects results in a ``HwMock`` (a +subclass of ``unittest.mock``'s ``MagicMock``) object which, in the case of +``SMBusModel``, provides access to a log of all register writes and their +values. + +The object can be then used to check which registers the hardware has written +with which values, and to assert that the expect actions have been taken. + +See ``roadtest/tests/regulator/test_tps62864.py`` for an example of this. + +GPIOs +----- + +The framework includes support for hardware models to trigger interrupts by +controlling GPIOs. See ``roadtest/tests/rtc/test_pcf8563.py`` for an example. + +Support has not been implemented yet for asserting that drivers control GPIOs +correctly. See the comment in ``gpio_handle_cmdq()`` in ``src/backend.c``. + +Coding guidelines +----------------- + +Run ``make fmt`` to automatically format your Python code to follow the coding +style. Run ``make check`` and ensure that your code passes static checkers and +style checks. Typing hints are mandatory. + +These two commands require that you have installed the packages listed in +``requirements.txt``, for example with something like the following patch and +then ensuring that ``~/.local/bin`` is in your ``$PATH``. + +.. code-block:: shell + + $ pip3 install --user -r requirements.txt + +Alternatively, you can also run these commands in the Docker container (by +appending ``DOCKER=1`` to the ``make`` commands) which has all the correct +tools installed.
Add a regression test for the problem fixed by the following patch, which would require specific environmental conditions to be able to be reproduced and regression-tested on real hardware:
iio: light: opt3001: Fixed timeout error when 0 lux https://lore.kernel.org/lkml/20210920125351.6569-1-valek@2n.cz/
No other aspects of the driver are tested.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- .../roadtest/roadtest/tests/iio/__init__.py | 0 .../roadtest/roadtest/tests/iio/config | 1 + .../roadtest/tests/iio/light/__init__.py | 0 .../roadtest/roadtest/tests/iio/light/config | 1 + .../roadtest/tests/iio/light/test_opt3001.py | 95 +++++++++++++++++++ 5 files changed, 97 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/iio/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py
diff --git a/tools/testing/roadtest/roadtest/tests/iio/__init__.py b/tools/testing/roadtest/roadtest/tests/iio/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/tests/iio/config b/tools/testing/roadtest/roadtest/tests/iio/config new file mode 100644 index 000000000000..a08d9e23ce38 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/config @@ -0,0 +1 @@ +CONFIG_IIO=y diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/__init__.py b/tools/testing/roadtest/roadtest/tests/iio/light/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/config b/tools/testing/roadtest/roadtest/tests/iio/light/config new file mode 100644 index 000000000000..b9753f2d0728 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/config @@ -0,0 +1 @@ +CONFIG_OPT3001=m diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py new file mode 100644 index 000000000000..abf20b8f3516 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py @@ -0,0 +1,95 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from typing import Any, Final + +from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float + +REG_RESULT: Final = 0x00 +REG_CONFIGURATION: Final = 0x01 +REG_LOW_LIMIT: Final = 0x02 +REG_HIGH_LIMIT: Final = 0x03 +REG_MANUFACTURER_ID: Final = 0x7E +REG_DEVICE_ID: Final = 0x7F + +REG_CONFIGURATION_CRF: Final = 1 << 7 + + +class OPT3001(SMBusModel): + def __init__(self, **kwargs: Any) -> None: + super().__init__(regbytes=2, byteorder="big", **kwargs) + # Reset values from datasheet + self.regs = { + REG_RESULT: 0x0000, + REG_CONFIGURATION: 0xC810, + REG_LOW_LIMIT: 0xC000, + REG_HIGH_LIMIT: 0xBFFF, + REG_MANUFACTURER_ID: 0x5449, + REG_DEVICE_ID: 0x3001, + } + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + + if addr == REG_CONFIGURATION: + # Always indicate that the conversion is ready. This is good + # enough for our current purposes. + val |= REG_CONFIGURATION_CRF + + return val + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + self.regs[addr] = val + + +class TestOPT3001(UMLTestCase): + dts = DtFragment( + src=""" +&i2c { + light-sensor@$addr$ { + compatible = "ti,opt3001"; + reg = <0x$addr$>; + }; +}; + """, + variables={ + "addr": DtVar.I2C_ADDR, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("opt3001") + + @classmethod + def tearDownClass(cls) -> None: + rmmod("opt3001") + + def setUp(self) -> None: + self.driver = I2CDriver("opt3001") + self.hw = Hardware("i2c") + self.hw.load_model(OPT3001) + + def tearDown(self) -> None: + self.hw.close() + + def test_illuminance(self) -> None: + data = [ + # Some values from datasheet, and 0 + (0b_0000_0000_0000_0000, 0), + (0b_0000_0000_0000_0001, 0.01), + (0b_0011_0100_0101_0110, 88.80), + (0b_0111_1000_1001_1010, 2818.56), + ] + with self.driver.bind(self.dts["addr"]) as dev: + luxfile = dev.path / "iio:device0/in_illuminance_input" + + for regval, lux in data: + self.hw.reg_write(REG_RESULT, regval) + self.assertEqual(read_float(luxfile), lux)
On Fri, Mar 11, 2022 at 11:24 AM Vincent Whitchurch vincent.whitchurch@axis.com wrote:
Add a regression test for the problem fixed by the following patch, which would require specific environmental conditions to be able to be reproduced and regression-tested on real hardware:
iio: light: opt3001: Fixed timeout error when 0 lux https://lore.kernel.org/lkml/20210920125351.6569-1-valek@2n.cz/
No other aspects of the driver are tested.
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com
.../roadtest/roadtest/tests/iio/__init__.py | 0 .../roadtest/roadtest/tests/iio/config | 1 + .../roadtest/tests/iio/light/__init__.py | 0 .../roadtest/roadtest/tests/iio/light/config | 1 + .../roadtest/tests/iio/light/test_opt3001.py | 95 +++++++++++++++++++ 5 files changed, 97 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/iio/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py
diff --git a/tools/testing/roadtest/roadtest/tests/iio/__init__.py b/tools/testing/roadtest/roadtest/tests/iio/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/tests/iio/config b/tools/testing/roadtest/roadtest/tests/iio/config new file mode 100644 index 000000000000..a08d9e23ce38 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/config @@ -0,0 +1 @@ +CONFIG_IIO=y diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/__init__.py b/tools/testing/roadtest/roadtest/tests/iio/light/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/config b/tools/testing/roadtest/roadtest/tests/iio/light/config new file mode 100644 index 000000000000..b9753f2d0728 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/config @@ -0,0 +1 @@ +CONFIG_OPT3001=m diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py new file mode 100644 index 000000000000..abf20b8f3516 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py @@ -0,0 +1,95 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB
+from typing import Any, Final
+from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float
+REG_RESULT: Final = 0x00 +REG_CONFIGURATION: Final = 0x01 +REG_LOW_LIMIT: Final = 0x02 +REG_HIGH_LIMIT: Final = 0x03 +REG_MANUFACTURER_ID: Final = 0x7E +REG_DEVICE_ID: Final = 0x7F
+REG_CONFIGURATION_CRF: Final = 1 << 7
+class OPT3001(SMBusModel):
- def __init__(self, **kwargs: Any) -> None:
super().__init__(regbytes=2, byteorder="big", **kwargs)
# Reset values from datasheet
self.regs = {
REG_RESULT: 0x0000,
REG_CONFIGURATION: 0xC810,
REG_LOW_LIMIT: 0xC000,
REG_HIGH_LIMIT: 0xBFFF,
REG_MANUFACTURER_ID: 0x5449,
REG_DEVICE_ID: 0x3001,
}
- def reg_read(self, addr: int) -> int:
val = self.regs[addr]
if addr == REG_CONFIGURATION:
# Always indicate that the conversion is ready. This is good
# enough for our current purposes.
val |= REG_CONFIGURATION_CRF
return val
- def reg_write(self, addr: int, val: int) -> None:
assert addr in self.regs
self.regs[addr] = val
+class TestOPT3001(UMLTestCase):
I am partial to starting with UML since there are a lot of nice easy things about starting there; however, I imagine people will eventually want to use this on other architectures (speaking from experience). How difficult do you think it would be to extend this to support manipulating fake devices in say QEMU?
I also have some colleagues inside of Google that worked on some projects to simulate simple devices on an FPGA to test software and adjacent devices in a conceptually similar way; one of these teams built a Domain Specific Language kind of like roadtest to implement the tests and the environment for the tests. The main reason I mention this here is I am thinking about maybe one day having an API you can implement so you can run your roadtests on UML, QEMU, or on any emulator or hardware testbed that implements the appropriate API.
I'll try to dig up some people who might be interested and add them here.
- dts = DtFragment(
src="""
+&i2c {
- light-sensor@$addr$ {
compatible = "ti,opt3001";
reg = <0x$addr$>;
- };
+};
""",
variables={
"addr": DtVar.I2C_ADDR,
},
- )
- @classmethod
- def setUpClass(cls) -> None:
insmod("opt3001")
- @classmethod
- def tearDownClass(cls) -> None:
rmmod("opt3001")
- def setUp(self) -> None:
self.driver = I2CDriver("opt3001")
self.hw = Hardware("i2c")
self.hw.load_model(OPT3001)
- def tearDown(self) -> None:
self.hw.close()
- def test_illuminance(self) -> None:
data = [
# Some values from datasheet, and 0
(0b_0000_0000_0000_0000, 0),
(0b_0000_0000_0000_0001, 0.01),
(0b_0011_0100_0101_0110, 88.80),
(0b_0111_1000_1001_1010, 2818.56),
]
with self.driver.bind(self.dts["addr"]) as dev:
luxfile = dev.path / "iio:device0/in_illuminance_input"
for regval, lux in data:
self.hw.reg_write(REG_RESULT, regval)
self.assertEqual(read_float(luxfile), lux)
I love the framework; this looks very easy to use.
One nit about this test; it seems like you cover just one test case here - the happy path. Can you cover some other one? Particularly some error paths?
Sorry, I am not trying to be cheeky here; it looks like this driver actually should probably be fully (or very close to fully) testable via roadtest as I understand it. It only looks like there are a handful of cases to cover for the driver: the device is busy, the device returned something invalid, the user requested something invalid, and several SMBus read/write failures - it really only looks like there are a handful of paths and I think they are all accessible via the I2C interface (except for maybe the user requesting something invalid).
On Tue, Mar 15, 2022 at 12:11:50AM +0100, Brendan Higgins wrote:
On Fri, Mar 11, 2022 at 11:24 AM Vincent Whitchurch vincent.whitchurch@axis.com wrote:
+class TestOPT3001(UMLTestCase):
I am partial to starting with UML since there are a lot of nice easy things about starting there; however, I imagine people will eventually want to use this on other architectures (speaking from experience). How difficult do you think it would be to extend this to support manipulating fake devices in say QEMU?
It should be possible, but upstream QEMU doesn't have everything that we need so some work is needed there. Also, of course work is need to provide user space for running the tests and communicating between the virtual machine and the backend:
- We need user space, so build scripts would need to be provided to cross-compile busybox and Python (and whatever libraries it needs) for the target architecture.
- We also use UML's hostfs feature to make things transparent to the user and to avoid having to set up things like networking for communication between the host and the backend. I think QEMU's 9pfs support can be used as a rootfs too but it's not something I've personally tested.
- We use virtio-i2c and virtio-gpio and use virtio-uml which uses the vhost-user API to communicate from UML to the backend. The latest version of QEMU has support for vhost-user-i2c, but vhost-user-gpio doesn't seem to have been merged yet, so work is needed on the QEMU side. This will also be true for other buses in the future, if they are implemented with new virtio devices.
- For MMIO, UML has virtio-mmio which allows implementing any PCIe device (and by extension any platform device) outside of UML, but last I checked, upstream QEMU did not have something similar.
I also have some colleagues inside of Google that worked on some projects to simulate simple devices on an FPGA to test software and adjacent devices in a conceptually similar way; one of these teams built a Domain Specific Language kind of like roadtest to implement the tests and the environment for the tests. The main reason I mention this here is I am thinking about maybe one day having an API you can implement so you can run your roadtests on UML, QEMU, or on any emulator or hardware testbed that implements the appropriate API.
I'll try to dig up some people who might be interested and add them here.
- dts = DtFragment(
src="""
+&i2c {
- light-sensor@$addr$ {
compatible = "ti,opt3001";
reg = <0x$addr$>;
- };
+};
""",
variables={
"addr": DtVar.I2C_ADDR,
},
- )
- @classmethod
- def setUpClass(cls) -> None:
insmod("opt3001")
- @classmethod
- def tearDownClass(cls) -> None:
rmmod("opt3001")
- def setUp(self) -> None:
self.driver = I2CDriver("opt3001")
self.hw = Hardware("i2c")
self.hw.load_model(OPT3001)
- def tearDown(self) -> None:
self.hw.close()
- def test_illuminance(self) -> None:
data = [
# Some values from datasheet, and 0
(0b_0000_0000_0000_0000, 0),
(0b_0000_0000_0000_0001, 0.01),
(0b_0011_0100_0101_0110, 88.80),
(0b_0111_1000_1001_1010, 2818.56),
]
with self.driver.bind(self.dts["addr"]) as dev:
luxfile = dev.path / "iio:device0/in_illuminance_input"
for regval, lux in data:
self.hw.reg_write(REG_RESULT, regval)
self.assertEqual(read_float(luxfile), lux)
I love the framework; this looks very easy to use.
One nit about this test; it seems like you cover just one test case here - the happy path. Can you cover some other one? Particularly some error paths?
Sorry, I am not trying to be cheeky here; it looks like this driver actually should probably be fully (or very close to fully) testable via roadtest as I understand it. It only looks like there are a handful of cases to cover for the driver: the device is busy, the device returned something invalid, the user requested something invalid, and several SMBus read/write failures - it really only looks like there are a handful of paths and I think they are all accessible via the I2C interface (except for maybe the user requesting something invalid).
Yes, there are more things that could be tested in this driver. However, as the commit message says, I only indented this particular test to serve as a regression test for the specific bug fix, which would need an environment where the chip detects 0 lux to be able to test on real hardware. There are a few reasons for this:
- Unlike the other drivers being tested in this series, I don't have access to boards with this chip so my interest in this particular piece of hardware is limited.
- I actually started writing more tests for this driver earlier on (specifically, testing the configuration which uses interrupts), but I quickly discovered that this driver has race conditions which result in unbalanced mutex locking (in brief: the ok_to_ignore_lock stuff is broken). This shows the value of the test framework, but I also didn't want to write non-trivial fixes for drivers where I didn't have real hardware to test.
- Also, some paths in this driver needs a modification to be tested under roadtest. It uses wait_event_timeout() with a fixed value, but we cannot guarantee that this constraint is met in the test environment since it depends on things like CPU load on the host.
(Also, we use UML's "time travel" feature which essentially fast-forwards through idle time, so the constraint can never be met in practice.)
So the timeout parameter would have to be made adjustable via say a module parameter, to be able to make it infinite (to test the normal case) and not (to be able to test timeout handling). I think this could be done fairly cleanly with a one- or two-liner patch to the driver and by hiding the details in a header file behind a roadtest-specific config option, but I wanted to avoid having to patch the kernel proper for the initial version of the framework.
For vcnl4000, I have actually inherited some out-of-tree patches which are in need of mainlining so the tests are a bit more complete since I'm hoping to send some patches to that driver soon. The period mode busy handling is not tested there either though, I can try to add that.
As for I2C API failures, I have not added tests for them in any of the drivers. There's not much the test cases could assert, other than perhaps error propagation, so it's unclear if there is enough value compared to the effort required to implement test cases to make sure that every I2C transaction's failure path is tested.
But I think that we do want to at least make sure the error paths are executed, to check that drivers don't crash or deadlock due to faulty cleanups and the like. A way to solve this could be to implement fault injection support in the I2C framework. The fail-nth feature could be used to systemically trigger each and every I2C transaction failure and check that the driver doesn't blow up, while using the roadtest as a means to hit the various code paths in the driver during each of the iterations of fail-nth. Fault injection support would also be helpful when testing on real hardware.
On Fri, 2022-03-18 at 16:49 +0100, Vincent Whitchurch wrote:
It should be possible, but upstream QEMU doesn't have everything that we need so some work is needed there. Also, of course work is need to provide user space for running the tests and communicating between the virtual machine and the backend:
- We need user space, so build scripts would need to be provided to cross-compile busybox and Python (and whatever libraries it needs) for the target architecture.
You could possibly use some nix recipes for all of this, but that's a fairly arcane thing (we use it, but ...)
- We also use UML's hostfs feature to make things transparent to the user and to avoid having to set up things like networking for communication between the host and the backend. I think QEMU's 9pfs support can be used as a rootfs too but it's not something I've personally tested.
That works just fine, yes. We used to do exactly this in the wireless test suite before we switched to UML, but the switch to UML was due to the "time-travel" feature.
https://w1.fi/cgit/hostap/tree/tests/hwsim/vm
has support for both UML and qemu/kvm.
We use virtio-i2c and virtio-gpio and use virtio-uml which uses the vhost-user API to communicate from UML to the backend. The latest version of QEMU has support for vhost-user-i2c, but vhost-user-gpio doesn't seem to have been merged yet, so work is needed on the QEMU side. This will also be true for other buses in the future, if they are implemented with new virtio devices.
For MMIO, UML has virtio-mmio which allows implementing any PCIe device (and by extension any platform device) outside of UML, but last I checked, upstream QEMU did not have something similar.
I think you have this a bit fuzzy.
The virtio_uml[.c] you speak of is the "bus" driver for virtio in UML. Obviously, qemu has support for virtio, so you don't need those bits.
Now, virtio_uml is actually the virtio (bus) driver inside the kernel, like you'd have virtio-mmio/virtio-pci in qemu. However, virtio_uml doesn't implement the devices in the hypervisor, where most qemu devices are implemented, but uses vhost-user to run the device implementation in a separate userspace. [1]
Now we're talking about vhost-user to talk to the device, and qemu supports this as well, in fact the vhost-user spec is part of qemu: https://git.qemu.org/?p=qemu.git%3Ba=blob%3Bf=docs/system/devices/vhost-user... https://www.qemu.org/docs/master/interop/vhost-user.html
The docs on how to use it are here: https://www.qemu.org/docs/master/system/devices/vhost-user.html
So once you have a device implementation (regardless of whether it's for use with any of the virtio-i2c, arch/um/drivers/virt-pci.c, virtio-gpio, virtio-net, ... drivers) you can actually connect it to virtual machines running as UML or in qemu.
(Actually, that's not strictly true today since it's arch/um/drivers/virt-pci.c and I didn't get a proper device ID assigned etc since it was for experimentation, I guess if we make this more commonly used then we should move it to drivers/pci/controller/virtio- pci.c and actually specify it in the OASIS virtio spec., at the very least it'd have to be possible to compile this and lib/logic_iomem.c on x86, but that's possible. Anyway I think PCI(e) is probably low on your list of things ...)
Also, some paths in this driver needs a modification to be tested under roadtest. It uses wait_event_timeout() with a fixed value, but we cannot guarantee that this constraint is met in the test environment since it depends on things like CPU load on the host.
(Also, we use UML's "time travel" feature which essentially fast-forwards through idle time, so the constraint can never be met in practice.)
Wohoo! This makes me very happy, finally somebody else who uses it :-)
[1] As an aside, you might be interested in usfstl (which you can find at https://github.com/linux-test-project/usfstl) which is one way you could implement the device side - though the focus here is on making a device implementation easy while under "time-travel" mode.
If you ever want to use time-travel with multiple machines or actually with virtio devices, it also contains the necessary controller program to glue the entire simulation together. We use this very successfully to test the (real but compiled for x86) wifi firmware for iwlwifi together with the real driver actually seeing a PCIe device in UML, under time- travel :)
johannes
On Fri, Mar 18, 2022 at 09:09:02PM +0100, Johannes Berg wrote:
On Fri, 2022-03-18 at 16:49 +0100, Vincent Whitchurch wrote:
We use virtio-i2c and virtio-gpio and use virtio-uml which uses the vhost-user API to communicate from UML to the backend. The latest version of QEMU has support for vhost-user-i2c, but vhost-user-gpio doesn't seem to have been merged yet, so work is needed on the QEMU side. This will also be true for other buses in the future, if they are implemented with new virtio devices.
For MMIO, UML has virtio-mmio which allows implementing any PCIe device (and by extension any platform device) outside of UML, but last I checked, upstream QEMU did not have something similar.
I think you have this a bit fuzzy.
The virtio_uml[.c] you speak of is the "bus" driver for virtio in UML. Obviously, qemu has support for virtio, so you don't need those bits.
Now, virtio_uml is actually the virtio (bus) driver inside the kernel, like you'd have virtio-mmio/virtio-pci in qemu. However, virtio_uml doesn't implement the devices in the hypervisor, where most qemu devices are implemented, but uses vhost-user to run the device implementation in a separate userspace. [1]
Now we're talking about vhost-user to talk to the device, and qemu supports this as well, in fact the vhost-user spec is part of qemu: https://git.qemu.org/?p=qemu.git%3Ba=blob%3Bf=docs/system/devices/vhost-user... https://www.qemu.org/docs/master/interop/vhost-user.html
The docs on how to use it are here: https://www.qemu.org/docs/master/system/devices/vhost-user.html
So once you have a device implementation (regardless of whether it's for use with any of the virtio-i2c, arch/um/drivers/virt-pci.c, virtio-gpio, virtio-net, ... drivers) you can actually connect it to virtual machines running as UML or in qemu.
I'm aware of vhost-user, but AFAICS QEMU needs glue for each device type to be able to actually hook up vhost-user implementations to the devices it exposes to the guest via the virtio PCI device. See e.g. hw/virtio/vhost-user-i2c-pci.c and hw/virtio/vhost-user-i2c.c in QEMU.
That is what I meant was missing for virtio-gpio, there seems to be an in-progress patch set for that here though: https://lore.kernel.org/all/cover.1641987128.git.viresh.kumar@linaro.org/
Similarly, glue for something like arch/um/drivers/virt-pci.c does not exist in QEMU.
Or perhaps you are implying that hw/virtio/vhost-user-i2c* in QEMU are not strictly needed?
(Actually, that's not strictly true today since it's arch/um/drivers/virt-pci.c and I didn't get a proper device ID assigned etc since it was for experimentation, I guess if we make this more commonly used then we should move it to drivers/pci/controller/virtio- pci.c and actually specify it in the OASIS virtio spec., at the very least it'd have to be possible to compile this and lib/logic_iomem.c on x86, but that's possible. Anyway I think PCI(e) is probably low on your list of things ...)
PCI is not that interesting, no, but platform devices are. I did some experiments early on with arch/um/drivers/virt-pci.c and a corresponding backend along with a simple PCI driver which probes all devicetree nodes under it, and I was able to use this to get some platform drivers working.
Also, some paths in this driver needs a modification to be tested under roadtest. It uses wait_event_timeout() with a fixed value, but we cannot guarantee that this constraint is met in the test environment since it depends on things like CPU load on the host.
(Also, we use UML's "time travel" feature which essentially fast-forwards through idle time, so the constraint can never be met in practice.)
Wohoo! This makes me very happy, finally somebody else who uses it :-)
Yes, thanks for that feature, it works well to speed up tests and also has a knack for triggering race conditions (the RTC use-after-free for example).
Time travel however sometimes triggers some WARN_ONs from the core timekeeping code. I haven't seen them when running the test suites, but they show up if the system under UML is idle for several (wall time) seconds. I haven't had a chance to investigate it further though, but I can dig up the splats if you are interested.
On Tue, 2022-03-29 at 16:43 +0200, Vincent Whitchurch wrote:
I'm aware of vhost-user, but AFAICS QEMU needs glue for each device type to be able to actually hook up vhost-user implementations to the devices it exposes to the guest via the virtio PCI device. See e.g. hw/virtio/vhost-user-i2c-pci.c and hw/virtio/vhost-user-i2c.c in QEMU.
Oh, I wasn't aware of that.
That is what I meant was missing for virtio-gpio, there seems to be an in-progress patch set for that here though: https://lore.kernel.org/all/cover.1641987128.git.viresh.kumar@linaro.org/
Similarly, glue for something like arch/um/drivers/virt-pci.c does not exist in QEMU.
Or perhaps you are implying that hw/virtio/vhost-user-i2c* in QEMU are not strictly needed?
I _thought_ that was the case, but honestly, that was just from reading about it, not looking at the code. Thinking about it though, I don't need special glue in UML, just passing the device ID on the command line? So not sure what they need the glue for. Looking at the code, it's not really much though? Not sure, I guess you need somebody more familiar with qemu here, sorry.
Wohoo! This makes me very happy, finally somebody else who uses it :-)
Yes, thanks for that feature, it works well to speed up tests and also has a knack for triggering race conditions (the RTC use-after-free for example).
Time travel however sometimes triggers some WARN_ONs from the core timekeeping code. I haven't seen them when running the test suites, but they show up if the system under UML is idle for several (wall time) seconds. I haven't had a chance to investigate it further though, but I can dig up the splats if you are interested.
Oh, I haven't seen that, and I'm pretty sure I've had systems idle for very long periods of time passing inside (think weeks) ...
So yeah, if you have some splats (ideally with corresponding kernel configs), I'd be interested.
johannes
On Tue, 2022-03-29 at 16:50 +0200, Johannes Berg wrote:
Or perhaps you are implying that hw/virtio/vhost-user-i2c* in QEMU are not strictly needed?
I _thought_ that was the case, but honestly, that was just from reading about it, not looking at the code. Thinking about it though, I don't need special glue in UML, just passing the device ID on the command line? So not sure what they need the glue for. Looking at the code, it's not really much though? Not sure, I guess you need somebody more familiar with qemu here, sorry.
So here https://www.qemu.org/docs/master/system/devices/vhost-user.html#vhost-user-d...
the docs say:
These are simple stub devices that ensure the VirtIO device is visible to the guest. The code is mostly boilerplate although each device has a chardev option which specifies the ID of the --chardev device that connects via a socket to the vhost-user daemon.
So maybe if the ID were specified via the command line too, you could have a generic vhost-user stub in qemu?
johannes
Add roadtests for the vcnl4000 driver, testing several of the driver's features including buffer and event handling. Since it's the first IIO roadtest testing the non-sysfs parts, some support code for using the IIO ABI is included.
The different variants supported by the driver are in separate tests and models since no two variants have fully identical register interfaces. This duplicates some of the test code, but it:
- Avoids the tests duplicating the same multi-variant logic as the driver, reducing the risk for both the test and the driver being wrong.
- Allows each variant's test and model to be individually understood and modified looking at only one specific datasheet, making it easier to extend tests and implement new features in the driver.
During development of these tests, two oddities were noticed in the driver's handling of VCNL4040, but the tests simply assume that the current driver knows what it's doing (although we may want to fix the first point later):
- The driver reads an invalid/undefined register on the VCNL4040 when attempting to distinguish between that one and VCNL4200.
- The driver uses a lux/step unit which differs from the datasheet (but which is specified in an application note).
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- .../roadtest/roadtest/tests/iio/iio.py | 112 +++++++ .../roadtest/roadtest/tests/iio/light/config | 1 + .../roadtest/tests/iio/light/test_vcnl4000.py | 132 ++++++++ .../roadtest/tests/iio/light/test_vcnl4010.py | 282 ++++++++++++++++++ .../roadtest/tests/iio/light/test_vcnl4040.py | 104 +++++++ .../roadtest/tests/iio/light/test_vcnl4200.py | 96 ++++++ 6 files changed, 727 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/iio/iio.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py
diff --git a/tools/testing/roadtest/roadtest/tests/iio/iio.py b/tools/testing/roadtest/roadtest/tests/iio/iio.py new file mode 100644 index 000000000000..ea57b28ea9d3 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/iio.py @@ -0,0 +1,112 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import contextlib +import enum +import fcntl +import struct +from dataclasses import dataclass, field +from typing import Any + +IIO_GET_EVENT_FD_IOCTL = 0x80046990 +IIO_BUFFER_GET_FD_IOCTL = 0xC0046991 + + +class IIOChanType(enum.IntEnum): + IIO_VOLTAGE = 0 + IIO_CURRENT = 1 + IIO_POWER = 2 + IIO_ACCEL = 3 + IIO_ANGL_VEL = 4 + IIO_MAGN = 5 + IIO_LIGHT = 6 + IIO_INTENSITY = 7 + IIO_PROXIMITY = 8 + IIO_TEMP = 9 + IIO_INCLI = 10 + IIO_ROT = 11 + IIO_ANGL = 12 + IIO_TIMESTAMP = 13 + IIO_CAPACITANCE = 14 + IIO_ALTVOLTAGE = 15 + IIO_CCT = 16 + IIO_PRESSURE = 17 + IIO_HUMIDITYRELATIVE = 18 + IIO_ACTIVITY = 19 + IIO_STEPS = 20 + IIO_ENERGY = 21 + IIO_DISTANCE = 22 + IIO_VELOCITY = 23 + IIO_CONCENTRATION = 24 + IIO_RESISTANCE = 25 + IIO_PH = 26 + IIO_UVINDEX = 27 + IIO_ELECTRICALCONDUCTIVITY = 28 + IIO_COUNT = 29 + IIO_INDEX = 30 + IIO_GRAVITY = 31 + IIO_POSITIONRELATIVE = 32 + IIO_PHASE = 33 + IIO_MASSCONCENTRATION = 34 + + +@dataclass +class IIOEvent: + id: int + timestamp: int + type: IIOChanType = field(init=False) + + def __post_init__(self) -> None: + self.type = IIOChanType((self.id >> 32) & 0xFF) + + +class IIOEventMonitor(contextlib.AbstractContextManager): + def __init__(self, devname: str) -> None: + self.devname = devname + + def __enter__(self) -> "IIOEventMonitor": + self.file = open(self.devname, "rb") + + s = struct.Struct("L") + buf = bytearray(s.size) + fcntl.ioctl(self.file.fileno(), IIO_GET_EVENT_FD_IOCTL, buf) + eventfd = s.unpack(buf)[0] + self.eventf = open(eventfd, "rb") + + return self + + def read(self) -> IIOEvent: + s = struct.Struct("Qq") + buf = self.eventf.read(s.size) + return IIOEvent(*s.unpack(buf)) + + def __exit__(self, *_: Any) -> None: + self.eventf.close() + self.file.close() + + +class IIOBuffer(contextlib.AbstractContextManager): + def __init__(self, devname: str, bufidx: int) -> None: + self.devname = devname + self.bufidx = bufidx + + def __enter__(self) -> "IIOBuffer": + self.file = open(self.devname, "rb") + + s = struct.Struct("L") + buf = bytearray(s.size) + s.pack_into(buf, 0, self.bufidx) + fcntl.ioctl(self.file.fileno(), IIO_BUFFER_GET_FD_IOCTL, buf) + eventfd = s.unpack(buf)[0] + self.eventf = open(eventfd, "rb") + + return self + + def read(self, spec: str) -> tuple: + s = struct.Struct(spec) + buf = self.eventf.read(s.size) + return s.unpack(buf) + + def __exit__(self, *_: Any) -> None: + self.eventf.close() + self.file.close() diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/config b/tools/testing/roadtest/roadtest/tests/iio/light/config index b9753f2d0728..3bd4125cbb6b 100644 --- a/tools/testing/roadtest/roadtest/tests/iio/light/config +++ b/tools/testing/roadtest/roadtest/tests/iio/light/config @@ -1 +1,2 @@ CONFIG_OPT3001=m +CONFIG_VCNL4000=m diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py new file mode 100644 index 000000000000..16a5bed18b7e --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py @@ -0,0 +1,132 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import errno +import logging +from typing import Any, Final + +from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float, read_int, read_str + +logger = logging.getLogger(__name__) + +REG_COMMAND: Final = 0x80 +REG_PRODUCT_ID_REVISION: Final = 0x81 +REG_IR_LED_CURRENT: Final = 0x83 +REG_ALS_PARAM: Final = 0x84 +REG_ALS_RESULT_HIGH: Final = 0x85 +REG_ALS_RESULT_LOW: Final = 0x86 +REG_PROX_RESULT_HIGH: Final = 0x87 +REG_PROX_RESULT_LOW: Final = 0x88 +REG_PROX_SIGNAL_FREQ: Final = 0x89 + +REG_COMMAND_ALS_DATA_RDY: Final = 1 << 6 +REG_COMMAND_PROX_DATA_RDY: Final = 1 << 5 + + +class VCNL4000(SMBusModel): + def __init__(self, **kwargs: Any) -> None: + super().__init__(regbytes=1, **kwargs) + self.regs = { + REG_COMMAND: 0b_1000_0000, + REG_PRODUCT_ID_REVISION: 0x11, + # Register "without function in current version" + 0x82: 0x00, + REG_IR_LED_CURRENT: 0x00, + REG_ALS_PARAM: 0x00, + REG_ALS_RESULT_HIGH: 0x00, + REG_ALS_RESULT_LOW: 0x00, + REG_PROX_RESULT_HIGH: 0x00, + REG_PROX_RESULT_LOW: 0x00, + REG_PROX_RESULT_LOW: 0x00, + } + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + + if addr in (REG_ALS_RESULT_HIGH, REG_ALS_RESULT_LOW): + self.regs[REG_COMMAND] &= ~REG_COMMAND_ALS_DATA_RDY + if addr in (REG_PROX_RESULT_HIGH, REG_PROX_RESULT_LOW): + self.regs[REG_COMMAND] &= ~REG_COMMAND_PROX_DATA_RDY + + return val + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + + if addr == REG_COMMAND: + rw = 0b_0001_1000 + val = (self.regs[addr] & ~rw) | (val & rw) + + self.regs[addr] = val + + def inject(self, addr: int, val: int, mask: int = ~0) -> None: + old = self.regs[addr] & ~mask + new = old | (val & mask) + self.regs[addr] = new + + +class TestVCNL4000(UMLTestCase): + dts = DtFragment( + src=""" +&i2c { + light-sensor@$addr$ { + compatible = "vishay,vcnl4000"; + reg = <0x$addr$>; + }; +}; + """, + variables={ + "addr": DtVar.I2C_ADDR, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("vcnl4000") + + @classmethod + def tearDownClass(cls) -> None: + rmmod("vcnl4000") + + def setUp(self) -> None: + self.driver = I2CDriver("vcnl4000") + self.hw = Hardware("i2c") + self.hw.load_model(VCNL4000) + + def tearDown(self) -> None: + self.hw.close() + + def test_lux(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + scale = read_float(dev.path / "iio:device0/in_illuminance_scale") + self.assertEqual(scale, 0.25) + + data = [ + (0x00, 0x00), + (0x12, 0x34), + (0xFF, 0xFF), + ] + luxfile = dev.path / "iio:device0/in_illuminance_raw" + for high, low in data: + self.hw.inject(REG_ALS_RESULT_HIGH, high) + self.hw.inject(REG_ALS_RESULT_LOW, low) + self.hw.inject( + REG_COMMAND, + val=REG_COMMAND_ALS_DATA_RDY, + mask=REG_COMMAND_ALS_DATA_RDY, + ) + + self.assertEqual(read_int(luxfile), high << 8 | low) + + def test_lux_timeout(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + # self.hw.set_never_ready(True) + with self.assertRaises(OSError) as cm: + luxfile = dev.path / "iio:device0/in_illuminance_raw" + read_str(luxfile) + self.assertEqual(cm.exception.errno, errno.EIO) diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py new file mode 100644 index 000000000000..929db970405f --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py @@ -0,0 +1,282 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import errno +import logging +from pathlib import Path +from typing import Any, Final, Optional + +from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import ( + I2CDriver, + read_float, + read_int, + read_str, + write_int, + write_str, +) +from roadtest.tests.iio import iio + +logger = logging.getLogger(__name__) + +REG_COMMAND: Final = 0x80 +REG_PRODUCT_ID_REVISION: Final = 0x81 +REG_PROXIMITY_RATE: Final = 0x82 +REG_IR_LED_CURRENT: Final = 0x83 +REG_ALS_PARAM: Final = 0x84 +REG_ALS_RESULT_HIGH: Final = 0x85 +REG_ALS_RESULT_LOW: Final = 0x86 +REG_PROX_RESULT_HIGH: Final = 0x87 +REG_PROX_RESULT_LOW: Final = 0x88 +REG_INTERRUPT_CONTROL: Final = 0x89 +REG_LOW_THRESHOLD_HIGH: Final = 0x8A +REG_LOW_THRESHOLD_LOW: Final = 0x8B +REG_HIGH_THRESHOLD_HIGH: Final = 0x8C +REG_HIGH_THRESHOLD_LOW: Final = 0x8D +REG_INTERRUPT_STATUS: Final = 0x8E + +REG_COMMAND_ALS_DATA_RDY: Final = 1 << 6 +REG_COMMAND_PROX_DATA_RDY: Final = 1 << 5 + + +class VCNL4010(SMBusModel): + def __init__(self, int: Optional[int] = None, **kwargs: Any) -> None: + super().__init__(regbytes=1, **kwargs) + self.int = int + self._set_int(False) + self.regs = { + REG_COMMAND: 0b_1000_0000, + REG_PRODUCT_ID_REVISION: 0x21, + REG_PROXIMITY_RATE: 0x00, + REG_IR_LED_CURRENT: 0x00, + REG_ALS_PARAM: 0x00, + REG_ALS_RESULT_HIGH: 0x00, + REG_ALS_RESULT_LOW: 0x00, + REG_PROX_RESULT_HIGH: 0x00, + REG_PROX_RESULT_LOW: 0x00, + REG_INTERRUPT_CONTROL: 0x00, + REG_LOW_THRESHOLD_HIGH: 0x00, + REG_LOW_THRESHOLD_LOW: 0x00, + REG_HIGH_THRESHOLD_HIGH: 0x00, + REG_HIGH_THRESHOLD_LOW: 0x00, + REG_INTERRUPT_STATUS: 0x00, + } + + def _set_int(self, active: int) -> None: + # Active-low + self.backend.gpio.set(self.int, not active) + + def _update_irq(self) -> None: + selftimed_en = self.regs[REG_COMMAND] & (1 << 0) + prox_en = self.regs[REG_COMMAND] & (1 << 1) + prox_data_rdy = self.regs[REG_COMMAND] & REG_COMMAND_PROX_DATA_RDY + int_prox_ready_en = self.regs[REG_INTERRUPT_CONTROL] & (1 << 3) + + logger.debug( + f"{selftimed_en=:x} {prox_en=:x} {prox_data_rdy=:x} {int_prox_ready_en=:x}" + ) + + if selftimed_en and prox_en and prox_data_rdy and int_prox_ready_en: + self.regs[REG_INTERRUPT_STATUS] |= 1 << 3 + + low_threshold = ( + self.regs[REG_LOW_THRESHOLD_HIGH] << 8 | self.regs[REG_LOW_THRESHOLD_LOW] + ) + high_threshold = ( + self.regs[REG_HIGH_THRESHOLD_HIGH] << 8 | self.regs[REG_HIGH_THRESHOLD_LOW] + ) + proximity = ( + self.regs[REG_PROX_RESULT_HIGH] << 8 | self.regs[REG_PROX_RESULT_LOW] + ) + int_thres_en = self.regs[REG_INTERRUPT_CONTROL] & (1 << 1) + + logger.debug( + f"{low_threshold=:x} {high_threshold=:x} {proximity=:x} {int_thres_en=:x}" + ) + + if int_thres_en: + if proximity < low_threshold: + logger.debug("LOW") + self.regs[REG_INTERRUPT_STATUS] |= 1 << 1 + if proximity > high_threshold: + logger.debug("HIGH") + self.regs[REG_INTERRUPT_STATUS] |= 1 << 0 + + self._set_int(self.regs[REG_INTERRUPT_STATUS]) + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + + if addr in (REG_ALS_RESULT_HIGH, REG_ALS_RESULT_LOW): + self.regs[REG_COMMAND] &= ~REG_COMMAND_ALS_DATA_RDY + if addr in (REG_PROX_RESULT_HIGH, REG_PROX_RESULT_LOW): + self.regs[REG_COMMAND] &= ~REG_COMMAND_PROX_DATA_RDY + + return val + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + + if addr == REG_COMMAND: + rw = 0b_0001_1111 + val = (self.regs[addr] & ~rw) | (val & rw) + elif addr == REG_INTERRUPT_STATUS: + val = self.regs[addr] & ~(val & 0xF) + + self.regs[addr] = val + self._update_irq() + + def inject(self, addr: int, val: int, mask: int = ~0) -> None: + old = self.regs[addr] & ~mask + new = old | (val & mask) + self.regs[addr] = new + self._update_irq() + + def set_bit(self, addr: int, val: int) -> None: + self.inject(addr, val, val) + + +class TestVCNL4010(UMLTestCase): + dts = DtFragment( + src=""" +#include <dt-bindings/interrupt-controller/irq.h> + +&i2c { + light-sensor@$addr$ { + compatible = "vishay,vcnl4020"; + reg = <0x$addr$>; + interrupt-parent = <&gpio>; + interrupts = <$gpio$ IRQ_TYPE_EDGE_FALLING>; + }; +}; + """, + variables={ + "addr": DtVar.I2C_ADDR, + "gpio": DtVar.GPIO_PIN, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("vcnl4000") + + @classmethod + def tearDownClass(cls) -> None: + rmmod("vcnl4000") + + def setUp(self) -> None: + self.driver = I2CDriver("vcnl4000") + self.hw = Hardware("i2c") + self.hw.load_model(VCNL4010, int=self.dts["gpio"]) + + def tearDown(self) -> None: + self.hw.close() + + def test_lux(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + + scale = read_float(dev.path / "iio:device0/in_illuminance_scale") + self.assertEqual(scale, 0.25) + + data = [ + (0x00, 0x00), + (0x12, 0x34), + (0xFF, 0xFF), + ] + luxfile = dev.path / "iio:device0/in_illuminance_raw" + for high, low in data: + self.hw.inject(REG_ALS_RESULT_HIGH, high) + self.hw.inject(REG_ALS_RESULT_LOW, low) + self.hw.set_bit(REG_COMMAND, REG_COMMAND_ALS_DATA_RDY) + + self.assertEqual(read_int(luxfile), high << 8 | low) + + def test_lux_timeout(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + with self.assertRaises(OSError) as cm: + luxfile = dev.path / "iio:device0/in_illuminance_raw" + read_str(luxfile) + self.assertEqual(cm.exception.errno, errno.EIO) + + def test_proximity_thresh_rising(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + high_thresh = ( + dev.path / "iio:device0/events/in_proximity_thresh_rising_value" + ) + write_int(high_thresh, 0x1234) + + mock = self.hw.update_mock() + mock.assert_last_reg_write(self, REG_HIGH_THRESHOLD_HIGH, 0x12) + mock.assert_last_reg_write(self, REG_HIGH_THRESHOLD_LOW, 0x34) + mock.reset_mock() + + self.assertEqual(read_int(high_thresh), 0x1234) + + with iio.IIOEventMonitor("/dev/iio:device0") as mon: + en = dev.path / "iio:device0/events/in_proximity_thresh_either_en" + write_int(en, 1) + + self.hw.inject(REG_PROX_RESULT_HIGH, 0x12) + self.hw.inject(REG_PROX_RESULT_LOW, 0x35) + self.hw.set_bit(REG_COMMAND, REG_COMMAND_PROX_DATA_RDY) + self.hw.kick() + + self.assertEqual(read_int(en), 1) + + event = mon.read() + self.assertEqual(event.type, iio.IIOChanType.IIO_PROXIMITY) + + def test_proximity_thresh_falling(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + high_thresh = ( + dev.path / "iio:device0/events/in_proximity_thresh_falling_value" + ) + write_int(high_thresh, 0x0ABC) + + mock = self.hw.update_mock() + mock.assert_last_reg_write(self, REG_LOW_THRESHOLD_HIGH, 0x0A) + mock.assert_last_reg_write(self, REG_LOW_THRESHOLD_LOW, 0xBC) + mock.reset_mock() + + self.assertEqual(read_int(high_thresh), 0x0ABC) + + with iio.IIOEventMonitor("/dev/iio:device0") as mon: + write_int( + dev.path / "iio:device0/events/in_proximity_thresh_either_en", 1 + ) + + event = mon.read() + self.assertEqual(event.type, iio.IIOChanType.IIO_PROXIMITY) + + def test_proximity_triggered(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + data = [ + (0x00, 0x00, 0), + (0x00, 0x01, 1), + (0xF0, 0x02, 0xF002), + (0xFF, 0xFF, 0xFFFF), + ] + + trigger = read_str(Path("/sys/bus/iio/devices/trigger0/name")) + + write_int(dev.path / "iio:device0/buffer0/in_proximity_en", 1) + write_str(dev.path / "iio:device0/trigger/current_trigger", trigger) + + with iio.IIOBuffer("/dev/iio:device0", bufidx=0) as buffer: + write_int(dev.path / "iio:device0/buffer0/length", 128) + write_int(dev.path / "iio:device0/buffer0/enable", 1) + + for low, high, expected in data: + self.hw.inject(REG_PROX_RESULT_HIGH, low) + self.hw.inject(REG_PROX_RESULT_LOW, high) + self.hw.set_bit(REG_COMMAND, REG_COMMAND_PROX_DATA_RDY) + self.hw.kick() + + scanline = buffer.read("H") + + val = scanline[0] + self.assertEqual(val, expected) diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py new file mode 100644 index 000000000000..f2aa2cb9f3d5 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py @@ -0,0 +1,104 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import logging +from typing import Any + +from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float, read_int + +logger = logging.getLogger(__name__) + + +class VCNL4040(SMBusModel): + def __init__(self, **kwargs: Any) -> None: + super().__init__(regbytes=2, byteorder="little", **kwargs) + self.regs = { + 0x00: 0x0101, + 0x01: 0x0000, + 0x02: 0x0000, + 0x03: 0x0001, + 0x04: 0x0000, + 0x05: 0x0000, + 0x06: 0x0000, + 0x07: 0x0000, + 0x08: 0x0000, + 0x09: 0x0000, + 0x0A: 0x0000, + 0x0A: 0x0000, + 0x0B: 0x0000, + 0x0C: 0x0186, + # The driver reads this register which is undefined for + # VCNL4040. Perhaps the driver should be fixed instead + # of having this here? + 0x0E: 0x0000, + } + + def reg_read(self, addr: int) -> int: + return self.regs[addr] + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + self.regs[addr] = val + + +class TestVCNL4040(UMLTestCase): + dts = DtFragment( + src=""" +&i2c { + light-sensor@$addr$ { + compatible = "vishay,vcnl4040"; + reg = <0x$addr$>; + }; +}; + """, + variables={ + "addr": DtVar.I2C_ADDR, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("vcnl4000") + + @classmethod + def tearDownClass(cls) -> None: + rmmod("vcnl4000") + + def setUp(self) -> None: + self.driver = I2CDriver("vcnl4000") + self.hw = Hardware("i2c") + self.hw.load_model(VCNL4040) + + def tearDown(self) -> None: + self.hw.close() + + def test_illuminance_scale(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + scalefile = dev.path / "iio:device0/in_illuminance_scale" + # The datasheet says 0.10 lux/step, but the driver follows + # the application note "Designing the VCNL4040 Into an + # Application" which claims a different value. + self.assertEqual(read_float(scalefile), 0.12) + + def test_illuminance(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + luxfile = dev.path / "iio:device0/in_illuminance_raw" + + data = [0x0000, 0x1234, 0xFFFF] + for regval in data: + self.hw.reg_write(0x09, regval) + self.assertEqual(read_int(luxfile), regval) + + def test_proximity(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + rawfile = dev.path / "iio:device0/in_proximity_raw" + + data = [0x0000, 0x1234, 0xFFFF] + for regval in data: + self.hw.reg_write(0x08, regval) + self.assertEqual(read_int(rawfile), regval) diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py new file mode 100644 index 000000000000..d1cf819e563e --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py @@ -0,0 +1,96 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import logging +from typing import Any + +from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float, read_int + +logger = logging.getLogger(__name__) + + +class VCNL4200(SMBusModel): + def __init__(self, **kwargs: Any) -> None: + super().__init__(regbytes=2, byteorder="little", **kwargs) + self.regs = { + 0x00: 0x0101, + 0x01: 0x0000, + 0x02: 0x0000, + 0x03: 0x0001, + 0x04: 0x0000, + 0x05: 0x0000, + 0x06: 0x0000, + 0x07: 0x0000, + 0x08: 0x0000, + 0x09: 0x0000, + 0x0A: 0x0000, + 0x0D: 0x0000, + 0x0E: 0x1058, + } + + def reg_read(self, addr: int) -> int: + return self.regs[addr] + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + self.regs[addr] = val + + +class TestVCNL4200(UMLTestCase): + dts = DtFragment( + src=""" +&i2c { + light-sensor@$addr$ { + compatible = "vishay,vcnl4200"; + reg = <0x$addr$>; + }; +}; + """, + variables={ + "addr": DtVar.I2C_ADDR, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("vcnl4000") + + @classmethod + def tearDownClass(cls) -> None: + rmmod("vcnl4000") + + def setUp(self) -> None: + self.driver = I2CDriver("vcnl4000") + self.hw = Hardware("i2c") + self.hw.load_model(VCNL4200) + + def tearDown(self) -> None: + self.hw.close() + + def test_illuminance_scale(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + scalefile = dev.path / "iio:device0/in_illuminance_scale" + self.assertEqual(read_float(scalefile), 0.024) + + def test_illuminance(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + luxfile = dev.path / "iio:device0/in_illuminance_raw" + + data = [0x0000, 0x1234, 0xFFFF] + for regval in data: + self.hw.reg_write(0x09, regval) + self.assertEqual(read_int(luxfile), regval) + + def test_proximity(self) -> None: + with self.driver.bind(self.dts["addr"]) as dev: + rawfile = dev.path / "iio:device0/in_proximity_raw" + + data = [0x0000, 0x1234, 0xFFFF] + for regval in data: + self.hw.reg_write(0x08, regval) + self.assertEqual(read_int(rawfile), regval)
On Fri, 11 Mar 2022 17:24:43 +0100 Vincent Whitchurch vincent.whitchurch@axis.com wrote:
Add roadtests for the vcnl4000 driver, testing several of the driver's features including buffer and event handling. Since it's the first IIO roadtest testing the non-sysfs parts, some support code for using the IIO ABI is included.
The different variants supported by the driver are in separate tests and models since no two variants have fully identical register interfaces. This duplicates some of the test code, but it:
Avoids the tests duplicating the same multi-variant logic as the driver, reducing the risk for both the test and the driver being wrong.
Allows each variant's test and model to be individually understood and modified looking at only one specific datasheet, making it easier to extend tests and implement new features in the driver.
During development of these tests, two oddities were noticed in the driver's handling of VCNL4040, but the tests simply assume that the current driver knows what it's doing (although we may want to fix the first point later):
The driver reads an invalid/undefined register on the VCNL4040 when attempting to distinguish between that one and VCNL4200.
The driver uses a lux/step unit which differs from the datasheet (but which is specified in an application note).
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com
Hi Vincent,
Very interesting bit of work. My current approach for similar testing is to write a qemu model for the hardware, but that currently requires carefully crafted tests. Most of the time I'm only doing that to verify refactoring of existing drivers.
One thing that makes me nervous here is the python element though as I've not written significant python in about 20 years. That is going to be a burden for kernel developers and maintainers... Nothing quite like badly written tests to make for a mess in the long run and I suspect my python for example would be very very badly written :) Cut and paste will of course get us a long way...
I dream of a world where every driver is testable by people with out hardware but I fear it may be a while yet. Hopefully this will get us a little closer!
I more or less follow what is going on here (good docs btw in the earlier patch definitely helped).
So far I'm thoroughly in favour of road test subject to actually being able to review the tests or getting sufficient support to do so. It's a 'how to scale it' question really...
Jonathan
.../roadtest/roadtest/tests/iio/iio.py | 112 +++++++ .../roadtest/roadtest/tests/iio/light/config | 1 + .../roadtest/tests/iio/light/test_vcnl4000.py | 132 ++++++++ .../roadtest/tests/iio/light/test_vcnl4010.py | 282 ++++++++++++++++++ .../roadtest/tests/iio/light/test_vcnl4040.py | 104 +++++++ .../roadtest/tests/iio/light/test_vcnl4200.py | 96 ++++++ 6 files changed, 727 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/iio/iio.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py
diff --git a/tools/testing/roadtest/roadtest/tests/iio/iio.py b/tools/testing/roadtest/roadtest/tests/iio/iio.py new file mode 100644 index 000000000000..ea57b28ea9d3 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/iio.py @@ -0,0 +1,112 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB
+import contextlib +import enum +import fcntl +import struct +from dataclasses import dataclass, field +from typing import Any
+IIO_GET_EVENT_FD_IOCTL = 0x80046990 +IIO_BUFFER_GET_FD_IOCTL = 0xC0046991
+class IIOChanType(enum.IntEnum):
- IIO_VOLTAGE = 0
- IIO_CURRENT = 1
- IIO_POWER = 2
- IIO_ACCEL = 3
- IIO_ANGL_VEL = 4
- IIO_MAGN = 5
- IIO_LIGHT = 6
- IIO_INTENSITY = 7
- IIO_PROXIMITY = 8
- IIO_TEMP = 9
- IIO_INCLI = 10
- IIO_ROT = 11
- IIO_ANGL = 12
- IIO_TIMESTAMP = 13
- IIO_CAPACITANCE = 14
- IIO_ALTVOLTAGE = 15
- IIO_CCT = 16
- IIO_PRESSURE = 17
- IIO_HUMIDITYRELATIVE = 18
- IIO_ACTIVITY = 19
- IIO_STEPS = 20
- IIO_ENERGY = 21
- IIO_DISTANCE = 22
- IIO_VELOCITY = 23
- IIO_CONCENTRATION = 24
- IIO_RESISTANCE = 25
- IIO_PH = 26
- IIO_UVINDEX = 27
- IIO_ELECTRICALCONDUCTIVITY = 28
- IIO_COUNT = 29
- IIO_INDEX = 30
- IIO_GRAVITY = 31
- IIO_POSITIONRELATIVE = 32
- IIO_PHASE = 33
- IIO_MASSCONCENTRATION = 34
+@dataclass +class IIOEvent:
- id: int
- timestamp: int
- type: IIOChanType = field(init=False)
- def __post_init__(self) -> None:
self.type = IIOChanType((self.id >> 32) & 0xFF)
+class IIOEventMonitor(contextlib.AbstractContextManager):
- def __init__(self, devname: str) -> None:
self.devname = devname
- def __enter__(self) -> "IIOEventMonitor":
self.file = open(self.devname, "rb")
s = struct.Struct("L")
buf = bytearray(s.size)
fcntl.ioctl(self.file.fileno(), IIO_GET_EVENT_FD_IOCTL, buf)
eventfd = s.unpack(buf)[0]
self.eventf = open(eventfd, "rb")
return self
- def read(self) -> IIOEvent:
s = struct.Struct("Qq")
buf = self.eventf.read(s.size)
return IIOEvent(*s.unpack(buf))
- def __exit__(self, *_: Any) -> None:
self.eventf.close()
self.file.close()
+class IIOBuffer(contextlib.AbstractContextManager):
- def __init__(self, devname: str, bufidx: int) -> None:
self.devname = devname
self.bufidx = bufidx
- def __enter__(self) -> "IIOBuffer":
self.file = open(self.devname, "rb")
s = struct.Struct("L")
buf = bytearray(s.size)
s.pack_into(buf, 0, self.bufidx)
fcntl.ioctl(self.file.fileno(), IIO_BUFFER_GET_FD_IOCTL, buf)
eventfd = s.unpack(buf)[0]
self.eventf = open(eventfd, "rb")
return self
- def read(self, spec: str) -> tuple:
s = struct.Struct(spec)
buf = self.eventf.read(s.size)
return s.unpack(buf)
- def __exit__(self, *_: Any) -> None:
self.eventf.close()
self.file.close()
diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/config b/tools/testing/roadtest/roadtest/tests/iio/light/config index b9753f2d0728..3bd4125cbb6b 100644 --- a/tools/testing/roadtest/roadtest/tests/iio/light/config +++ b/tools/testing/roadtest/roadtest/tests/iio/light/config @@ -1 +1,2 @@ CONFIG_OPT3001=m +CONFIG_VCNL4000=m diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py new file mode 100644 index 000000000000..16a5bed18b7e --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py @@ -0,0 +1,132 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB
+import errno +import logging +from typing import Any, Final
+from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float, read_int, read_str
+logger = logging.getLogger(__name__)
+REG_COMMAND: Final = 0x80 +REG_PRODUCT_ID_REVISION: Final = 0x81 +REG_IR_LED_CURRENT: Final = 0x83 +REG_ALS_PARAM: Final = 0x84 +REG_ALS_RESULT_HIGH: Final = 0x85 +REG_ALS_RESULT_LOW: Final = 0x86 +REG_PROX_RESULT_HIGH: Final = 0x87 +REG_PROX_RESULT_LOW: Final = 0x88 +REG_PROX_SIGNAL_FREQ: Final = 0x89
+REG_COMMAND_ALS_DATA_RDY: Final = 1 << 6 +REG_COMMAND_PROX_DATA_RDY: Final = 1 << 5
+class VCNL4000(SMBusModel):
- def __init__(self, **kwargs: Any) -> None:
super().__init__(regbytes=1, **kwargs)
self.regs = {
REG_COMMAND: 0b_1000_0000,
REG_PRODUCT_ID_REVISION: 0x11,
# Register "without function in current version"
0x82: 0x00,
REG_IR_LED_CURRENT: 0x00,
REG_ALS_PARAM: 0x00,
REG_ALS_RESULT_HIGH: 0x00,
REG_ALS_RESULT_LOW: 0x00,
REG_PROX_RESULT_HIGH: 0x00,
REG_PROX_RESULT_LOW: 0x00,
REG_PROX_RESULT_LOW: 0x00,
}
- def reg_read(self, addr: int) -> int:
val = self.regs[addr]
if addr in (REG_ALS_RESULT_HIGH, REG_ALS_RESULT_LOW):
self.regs[REG_COMMAND] &= ~REG_COMMAND_ALS_DATA_RDY
if addr in (REG_PROX_RESULT_HIGH, REG_PROX_RESULT_LOW):
self.regs[REG_COMMAND] &= ~REG_COMMAND_PROX_DATA_RDY
return val
- def reg_write(self, addr: int, val: int) -> None:
assert addr in self.regs
if addr == REG_COMMAND:
rw = 0b_0001_1000
val = (self.regs[addr] & ~rw) | (val & rw)
self.regs[addr] = val
- def inject(self, addr: int, val: int, mask: int = ~0) -> None:
old = self.regs[addr] & ~mask
new = old | (val & mask)
self.regs[addr] = new
+class TestVCNL4000(UMLTestCase):
- dts = DtFragment(
src="""
+&i2c {
- light-sensor@$addr$ {
compatible = "vishay,vcnl4000";
reg = <0x$addr$>;
- };
+};
""",
variables={
"addr": DtVar.I2C_ADDR,
},
- )
- @classmethod
- def setUpClass(cls) -> None:
insmod("vcnl4000")
- @classmethod
- def tearDownClass(cls) -> None:
rmmod("vcnl4000")
- def setUp(self) -> None:
self.driver = I2CDriver("vcnl4000")
self.hw = Hardware("i2c")
self.hw.load_model(VCNL4000)
- def tearDown(self) -> None:
self.hw.close()
- def test_lux(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
scale = read_float(dev.path / "iio:device0/in_illuminance_scale")
self.assertEqual(scale, 0.25)
data = [
(0x00, 0x00),
(0x12, 0x34),
(0xFF, 0xFF),
]
luxfile = dev.path / "iio:device0/in_illuminance_raw"
for high, low in data:
self.hw.inject(REG_ALS_RESULT_HIGH, high)
self.hw.inject(REG_ALS_RESULT_LOW, low)
self.hw.inject(
REG_COMMAND,
val=REG_COMMAND_ALS_DATA_RDY,
mask=REG_COMMAND_ALS_DATA_RDY,
)
self.assertEqual(read_int(luxfile), high << 8 | low)
- def test_lux_timeout(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
# self.hw.set_never_ready(True)
with self.assertRaises(OSError) as cm:
luxfile = dev.path / "iio:device0/in_illuminance_raw"
read_str(luxfile)
self.assertEqual(cm.exception.errno, errno.EIO)
diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py new file mode 100644 index 000000000000..929db970405f --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py @@ -0,0 +1,282 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB
+import errno +import logging +from pathlib import Path +from typing import Any, Final, Optional
+from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import (
- I2CDriver,
- read_float,
- read_int,
- read_str,
- write_int,
- write_str,
+) +from roadtest.tests.iio import iio
+logger = logging.getLogger(__name__)
+REG_COMMAND: Final = 0x80 +REG_PRODUCT_ID_REVISION: Final = 0x81 +REG_PROXIMITY_RATE: Final = 0x82 +REG_IR_LED_CURRENT: Final = 0x83 +REG_ALS_PARAM: Final = 0x84 +REG_ALS_RESULT_HIGH: Final = 0x85 +REG_ALS_RESULT_LOW: Final = 0x86 +REG_PROX_RESULT_HIGH: Final = 0x87 +REG_PROX_RESULT_LOW: Final = 0x88 +REG_INTERRUPT_CONTROL: Final = 0x89 +REG_LOW_THRESHOLD_HIGH: Final = 0x8A +REG_LOW_THRESHOLD_LOW: Final = 0x8B +REG_HIGH_THRESHOLD_HIGH: Final = 0x8C +REG_HIGH_THRESHOLD_LOW: Final = 0x8D +REG_INTERRUPT_STATUS: Final = 0x8E
+REG_COMMAND_ALS_DATA_RDY: Final = 1 << 6 +REG_COMMAND_PROX_DATA_RDY: Final = 1 << 5
+class VCNL4010(SMBusModel):
- def __init__(self, int: Optional[int] = None, **kwargs: Any) -> None:
super().__init__(regbytes=1, **kwargs)
self.int = int
self._set_int(False)
self.regs = {
REG_COMMAND: 0b_1000_0000,
REG_PRODUCT_ID_REVISION: 0x21,
REG_PROXIMITY_RATE: 0x00,
REG_IR_LED_CURRENT: 0x00,
REG_ALS_PARAM: 0x00,
REG_ALS_RESULT_HIGH: 0x00,
REG_ALS_RESULT_LOW: 0x00,
REG_PROX_RESULT_HIGH: 0x00,
REG_PROX_RESULT_LOW: 0x00,
REG_INTERRUPT_CONTROL: 0x00,
REG_LOW_THRESHOLD_HIGH: 0x00,
REG_LOW_THRESHOLD_LOW: 0x00,
REG_HIGH_THRESHOLD_HIGH: 0x00,
REG_HIGH_THRESHOLD_LOW: 0x00,
REG_INTERRUPT_STATUS: 0x00,
}
- def _set_int(self, active: int) -> None:
# Active-low
self.backend.gpio.set(self.int, not active)
- def _update_irq(self) -> None:
selftimed_en = self.regs[REG_COMMAND] & (1 << 0)
prox_en = self.regs[REG_COMMAND] & (1 << 1)
prox_data_rdy = self.regs[REG_COMMAND] & REG_COMMAND_PROX_DATA_RDY
int_prox_ready_en = self.regs[REG_INTERRUPT_CONTROL] & (1 << 3)
logger.debug(
f"{selftimed_en=:x} {prox_en=:x} {prox_data_rdy=:x} {int_prox_ready_en=:x}"
)
if selftimed_en and prox_en and prox_data_rdy and int_prox_ready_en:
self.regs[REG_INTERRUPT_STATUS] |= 1 << 3
low_threshold = (
self.regs[REG_LOW_THRESHOLD_HIGH] << 8 | self.regs[REG_LOW_THRESHOLD_LOW]
)
high_threshold = (
self.regs[REG_HIGH_THRESHOLD_HIGH] << 8 | self.regs[REG_HIGH_THRESHOLD_LOW]
)
proximity = (
self.regs[REG_PROX_RESULT_HIGH] << 8 | self.regs[REG_PROX_RESULT_LOW]
)
int_thres_en = self.regs[REG_INTERRUPT_CONTROL] & (1 << 1)
logger.debug(
f"{low_threshold=:x} {high_threshold=:x} {proximity=:x} {int_thres_en=:x}"
)
if int_thres_en:
if proximity < low_threshold:
logger.debug("LOW")
self.regs[REG_INTERRUPT_STATUS] |= 1 << 1
if proximity > high_threshold:
logger.debug("HIGH")
self.regs[REG_INTERRUPT_STATUS] |= 1 << 0
self._set_int(self.regs[REG_INTERRUPT_STATUS])
- def reg_read(self, addr: int) -> int:
val = self.regs[addr]
if addr in (REG_ALS_RESULT_HIGH, REG_ALS_RESULT_LOW):
self.regs[REG_COMMAND] &= ~REG_COMMAND_ALS_DATA_RDY
if addr in (REG_PROX_RESULT_HIGH, REG_PROX_RESULT_LOW):
self.regs[REG_COMMAND] &= ~REG_COMMAND_PROX_DATA_RDY
return val
- def reg_write(self, addr: int, val: int) -> None:
assert addr in self.regs
if addr == REG_COMMAND:
rw = 0b_0001_1111
val = (self.regs[addr] & ~rw) | (val & rw)
elif addr == REG_INTERRUPT_STATUS:
val = self.regs[addr] & ~(val & 0xF)
self.regs[addr] = val
self._update_irq()
- def inject(self, addr: int, val: int, mask: int = ~0) -> None:
old = self.regs[addr] & ~mask
new = old | (val & mask)
self.regs[addr] = new
self._update_irq()
- def set_bit(self, addr: int, val: int) -> None:
self.inject(addr, val, val)
+class TestVCNL4010(UMLTestCase):
- dts = DtFragment(
src="""
+#include <dt-bindings/interrupt-controller/irq.h>
+&i2c {
- light-sensor@$addr$ {
compatible = "vishay,vcnl4020";
reg = <0x$addr$>;
interrupt-parent = <&gpio>;
interrupts = <$gpio$ IRQ_TYPE_EDGE_FALLING>;
- };
+};
""",
variables={
"addr": DtVar.I2C_ADDR,
"gpio": DtVar.GPIO_PIN,
},
- )
- @classmethod
- def setUpClass(cls) -> None:
insmod("vcnl4000")
- @classmethod
- def tearDownClass(cls) -> None:
rmmod("vcnl4000")
- def setUp(self) -> None:
self.driver = I2CDriver("vcnl4000")
self.hw = Hardware("i2c")
self.hw.load_model(VCNL4010, int=self.dts["gpio"])
- def tearDown(self) -> None:
self.hw.close()
- def test_lux(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
scale = read_float(dev.path / "iio:device0/in_illuminance_scale")
self.assertEqual(scale, 0.25)
data = [
(0x00, 0x00),
(0x12, 0x34),
(0xFF, 0xFF),
]
luxfile = dev.path / "iio:device0/in_illuminance_raw"
for high, low in data:
self.hw.inject(REG_ALS_RESULT_HIGH, high)
self.hw.inject(REG_ALS_RESULT_LOW, low)
self.hw.set_bit(REG_COMMAND, REG_COMMAND_ALS_DATA_RDY)
self.assertEqual(read_int(luxfile), high << 8 | low)
- def test_lux_timeout(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
with self.assertRaises(OSError) as cm:
luxfile = dev.path / "iio:device0/in_illuminance_raw"
read_str(luxfile)
self.assertEqual(cm.exception.errno, errno.EIO)
- def test_proximity_thresh_rising(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
high_thresh = (
dev.path / "iio:device0/events/in_proximity_thresh_rising_value"
)
write_int(high_thresh, 0x1234)
mock = self.hw.update_mock()
mock.assert_last_reg_write(self, REG_HIGH_THRESHOLD_HIGH, 0x12)
mock.assert_last_reg_write(self, REG_HIGH_THRESHOLD_LOW, 0x34)
mock.reset_mock()
self.assertEqual(read_int(high_thresh), 0x1234)
with iio.IIOEventMonitor("/dev/iio:device0") as mon:
en = dev.path / "iio:device0/events/in_proximity_thresh_either_en"
write_int(en, 1)
self.hw.inject(REG_PROX_RESULT_HIGH, 0x12)
self.hw.inject(REG_PROX_RESULT_LOW, 0x35)
self.hw.set_bit(REG_COMMAND, REG_COMMAND_PROX_DATA_RDY)
self.hw.kick()
self.assertEqual(read_int(en), 1)
event = mon.read()
self.assertEqual(event.type, iio.IIOChanType.IIO_PROXIMITY)
- def test_proximity_thresh_falling(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
high_thresh = (
dev.path / "iio:device0/events/in_proximity_thresh_falling_value"
)
write_int(high_thresh, 0x0ABC)
mock = self.hw.update_mock()
mock.assert_last_reg_write(self, REG_LOW_THRESHOLD_HIGH, 0x0A)
mock.assert_last_reg_write(self, REG_LOW_THRESHOLD_LOW, 0xBC)
mock.reset_mock()
self.assertEqual(read_int(high_thresh), 0x0ABC)
with iio.IIOEventMonitor("/dev/iio:device0") as mon:
write_int(
dev.path / "iio:device0/events/in_proximity_thresh_either_en", 1
)
event = mon.read()
self.assertEqual(event.type, iio.IIOChanType.IIO_PROXIMITY)
- def test_proximity_triggered(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
data = [
(0x00, 0x00, 0),
(0x00, 0x01, 1),
(0xF0, 0x02, 0xF002),
(0xFF, 0xFF, 0xFFFF),
]
trigger = read_str(Path("/sys/bus/iio/devices/trigger0/name"))
write_int(dev.path / "iio:device0/buffer0/in_proximity_en", 1)
write_str(dev.path / "iio:device0/trigger/current_trigger", trigger)
with iio.IIOBuffer("/dev/iio:device0", bufidx=0) as buffer:
write_int(dev.path / "iio:device0/buffer0/length", 128)
write_int(dev.path / "iio:device0/buffer0/enable", 1)
for low, high, expected in data:
self.hw.inject(REG_PROX_RESULT_HIGH, low)
self.hw.inject(REG_PROX_RESULT_LOW, high)
self.hw.set_bit(REG_COMMAND, REG_COMMAND_PROX_DATA_RDY)
self.hw.kick()
scanline = buffer.read("H")
val = scanline[0]
self.assertEqual(val, expected)
diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py new file mode 100644 index 000000000000..f2aa2cb9f3d5 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py @@ -0,0 +1,104 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB
+import logging +from typing import Any
+from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float, read_int
+logger = logging.getLogger(__name__)
+class VCNL4040(SMBusModel):
- def __init__(self, **kwargs: Any) -> None:
super().__init__(regbytes=2, byteorder="little", **kwargs)
self.regs = {
0x00: 0x0101,
0x01: 0x0000,
0x02: 0x0000,
0x03: 0x0001,
0x04: 0x0000,
0x05: 0x0000,
0x06: 0x0000,
0x07: 0x0000,
0x08: 0x0000,
0x09: 0x0000,
0x0A: 0x0000,
0x0A: 0x0000,
0x0B: 0x0000,
0x0C: 0x0186,
# The driver reads this register which is undefined for
# VCNL4040. Perhaps the driver should be fixed instead
# of having this here?
0x0E: 0x0000,
}
- def reg_read(self, addr: int) -> int:
return self.regs[addr]
- def reg_write(self, addr: int, val: int) -> None:
assert addr in self.regs
self.regs[addr] = val
+class TestVCNL4040(UMLTestCase):
- dts = DtFragment(
src="""
+&i2c {
- light-sensor@$addr$ {
compatible = "vishay,vcnl4040";
reg = <0x$addr$>;
- };
+};
""",
variables={
"addr": DtVar.I2C_ADDR,
},
- )
- @classmethod
- def setUpClass(cls) -> None:
insmod("vcnl4000")
- @classmethod
- def tearDownClass(cls) -> None:
rmmod("vcnl4000")
- def setUp(self) -> None:
self.driver = I2CDriver("vcnl4000")
self.hw = Hardware("i2c")
self.hw.load_model(VCNL4040)
- def tearDown(self) -> None:
self.hw.close()
- def test_illuminance_scale(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
scalefile = dev.path / "iio:device0/in_illuminance_scale"
# The datasheet says 0.10 lux/step, but the driver follows
# the application note "Designing the VCNL4040 Into an
# Application" which claims a different value.
self.assertEqual(read_float(scalefile), 0.12)
- def test_illuminance(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
luxfile = dev.path / "iio:device0/in_illuminance_raw"
data = [0x0000, 0x1234, 0xFFFF]
for regval in data:
self.hw.reg_write(0x09, regval)
self.assertEqual(read_int(luxfile), regval)
- def test_proximity(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
rawfile = dev.path / "iio:device0/in_proximity_raw"
data = [0x0000, 0x1234, 0xFFFF]
for regval in data:
self.hw.reg_write(0x08, regval)
self.assertEqual(read_int(rawfile), regval)
diff --git a/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py new file mode 100644 index 000000000000..d1cf819e563e --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py @@ -0,0 +1,96 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB
+import logging +from typing import Any
+from roadtest.backend.i2c import SMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver, read_float, read_int
+logger = logging.getLogger(__name__)
+class VCNL4200(SMBusModel):
- def __init__(self, **kwargs: Any) -> None:
super().__init__(regbytes=2, byteorder="little", **kwargs)
self.regs = {
0x00: 0x0101,
0x01: 0x0000,
0x02: 0x0000,
0x03: 0x0001,
0x04: 0x0000,
0x05: 0x0000,
0x06: 0x0000,
0x07: 0x0000,
0x08: 0x0000,
0x09: 0x0000,
0x0A: 0x0000,
0x0D: 0x0000,
0x0E: 0x1058,
}
- def reg_read(self, addr: int) -> int:
return self.regs[addr]
- def reg_write(self, addr: int, val: int) -> None:
assert addr in self.regs
self.regs[addr] = val
+class TestVCNL4200(UMLTestCase):
- dts = DtFragment(
src="""
+&i2c {
- light-sensor@$addr$ {
compatible = "vishay,vcnl4200";
reg = <0x$addr$>;
- };
+};
""",
variables={
"addr": DtVar.I2C_ADDR,
},
- )
- @classmethod
- def setUpClass(cls) -> None:
insmod("vcnl4000")
- @classmethod
- def tearDownClass(cls) -> None:
rmmod("vcnl4000")
- def setUp(self) -> None:
self.driver = I2CDriver("vcnl4000")
self.hw = Hardware("i2c")
self.hw.load_model(VCNL4200)
- def tearDown(self) -> None:
self.hw.close()
- def test_illuminance_scale(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
scalefile = dev.path / "iio:device0/in_illuminance_scale"
self.assertEqual(read_float(scalefile), 0.024)
- def test_illuminance(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
luxfile = dev.path / "iio:device0/in_illuminance_raw"
data = [0x0000, 0x1234, 0xFFFF]
for regval in data:
self.hw.reg_write(0x09, regval)
self.assertEqual(read_int(luxfile), regval)
- def test_proximity(self) -> None:
with self.driver.bind(self.dts["addr"]) as dev:
rawfile = dev.path / "iio:device0/in_proximity_raw"
data = [0x0000, 0x1234, 0xFFFF]
for regval in data:
self.hw.reg_write(0x08, regval)
self.assertEqual(read_int(rawfile), regval)
On Sun, Mar 20, 2022 at 06:02:53PM +0100, Jonathan Cameron wrote:
Very interesting bit of work. My current approach for similar testing is to write a qemu model for the hardware, but that currently requires carefully crafted tests. Most of the time I'm only doing that to verify refactoring of existing drivers.
Thank you for taking a look!
One thing that makes me nervous here is the python element though as I've not written significant python in about 20 years. That is going to be a burden for kernel developers and maintainers... Nothing quite like badly written tests to make for a mess in the long run and I suspect my python for example would be very very badly written :)
There's a bunch of static checkers to ensure that the code follows some basic guidelines, and CI can check that the tests work consistently, and also calculate metrics such as test execution time and code coverage, so even non-idiomatic Python in the tests wouldn't be entirely broken.
And unlike driver code, if the tests for a particular driver later do turn out to be bad (in what way?), we could just throw those particular tests out without breaking anybody's system.
Cut and paste will of course get us a long way...
Isn't some amount of copy/paste followed by modification to be expected even if the framework is written in say C (just as there's already copy/paste + modification involved when writing drivers)?
As for the core logic of individual driver tests excluding the framework bits, I have a hard time imagining what Python syntax looks like to someone with no knowledge of Python, so yes, I guess it's going to be harder to review.
I dream of a world where every driver is testable by people with out hardware but I fear it may be a while yet. Hopefully this will get us a little closer!
I more or less follow what is going on here (good docs btw in the earlier patch definitely helped).
So far I'm thoroughly in favour of road test subject to actually being able to review the tests or getting sufficient support to do so. It's a 'how to scale it' question really...
Would rewriting the framework in C and forcing tests to be written in that language mean that maintainers would be able to review tests without external support?
On Tue, 5 Apr 2022 15:48:05 +0200 Vincent Whitchurch vincent.whitchurch@axis.com wrote:
On Sun, Mar 20, 2022 at 06:02:53PM +0100, Jonathan Cameron wrote:
Very interesting bit of work. My current approach for similar testing is to write a qemu model for the hardware, but that currently requires carefully crafted tests. Most of the time I'm only doing that to verify refactoring of existing drivers.
Thank you for taking a look!
One thing that makes me nervous here is the python element though as I've not written significant python in about 20 years. That is going to be a burden for kernel developers and maintainers... Nothing quite like badly written tests to make for a mess in the long run and I suspect my python for example would be very very badly written :)
There's a bunch of static checkers to ensure that the code follows some basic guidelines, and CI can check that the tests work consistently, and also calculate metrics such as test execution time and code coverage, so even non-idiomatic Python in the tests wouldn't be entirely broken.
And unlike driver code, if the tests for a particular driver later do turn out to be bad (in what way?), we could just throw those particular tests out without breaking anybody's system.
True. Though CI test triage folk may disagree ;)
Cut and paste will of course get us a long way...
Isn't some amount of copy/paste followed by modification to be expected even if the framework is written in say C (just as there's already copy/paste + modification involved when writing drivers)?
As for the core logic of individual driver tests excluding the framework bits, I have a hard time imagining what Python syntax looks like to someone with no knowledge of Python, so yes, I guess it's going to be harder to review.
I messed around the other day with writing tests for drivers/staging/iio/cdc/ad7746.c and wasn't "too bad" and was useful for verifying some refactoring (and identified a possible precision problem in some integer approximation of floating point calcs) I'll try and find time to flesh that test set out more in the near future and post it so you can see how bad my python is. It amused my wife if nothing else :)
However a future project is to see if I can use this to hook up the SPDM attestation stack via mctp over i2c - just because I like to live dangerously :)
For IIO use more generally we need a sensible path to SPI (and also platform drivers). For my day job I'd like to mess around with doing PCI devices as well. The PCI DOE support for example would be nice to run against a test set that doesn't involve spinning up QEMU. DOE driver support: https://lore.kernel.org/all/20220330235920.2800929-1-ira.weiny@intel.com/
Effort wise, it's similar effort to hacking equivalent in QEMU but with the obvious advantage of being in tree and simpler for CI systems etc to use.
It would be nice to only have to use QEMU for complex system CI tests like the ones we are doing for CXL.
I dream of a world where every driver is testable by people with out hardware but I fear it may be a while yet. Hopefully this will get us a little closer!
I more or less follow what is going on here (good docs btw in the earlier patch definitely helped).
So far I'm thoroughly in favour of road test subject to actually being able to review the tests or getting sufficient support to do so. It's a 'how to scale it' question really...
Would rewriting the framework in C and forcing tests to be written in that language mean that maintainers would be able to review tests without external support?
I was wondering that. If we stayed in python I think we'd definitely want someone to be the 'roadtester/tests' maintainer (or group of maintainers) and their Ack to be expected for all tests we upstream. Idea being they'd sanity check correct use of framework and just how bad the python code us C developers are writing is ;)
However, we'd still need a good chunk of that 'framework' use review even if doing this in C.
Anyhow, very promising bit of work.
Thanks,
Jonathan
On Wed, Apr 06, 2022 at 03:08:16PM +0200, Jonathan Cameron wrote:
On Tue, 5 Apr 2022 15:48:05 +0200 Vincent Whitchurch vincent.whitchurch@axis.com wrote: I messed around the other day with writing tests for drivers/staging/iio/cdc/ad7746.c and wasn't "too bad" and was useful for verifying some refactoring (and identified a possible precision problem in some integer approximation of floating point calcs)
Good to hear!
I'll try and find time to flesh that test set out more in the near future and post it so you can see how bad my python is. It amused my wife if nothing else :)
However a future project is to see if I can use this to hook up the SPDM attestation stack via mctp over i2c - just because I like to live dangerously :)
For IIO use more generally we need a sensible path to SPI (and also platform drivers).
I have SPI working now. I was able to do this without patching the kernel by have the Python code emulate an SC18IS602 I2C-SPI bridge which has an existing driver. There is a limitation of 200 bytes per transaction (in the SC18IS602 driver/chip) so not all SPI drivers will work, but many will, and the underlying backend can be changed later without having to change the test cases. I used this to implement a test for drivers/iio/adc/ti-adc084s021.c.
Platform devices are going to take more work. I did do some experiments (using arch/um/drivers/virt-pci.c) a while ago but I need to see how well it works with the rest of the framework in place.
For my day job I'd like to mess around with doing PCI devices as well. The PCI DOE support for example would be nice to run against a test set that doesn't involve spinning up QEMU. DOE driver support: https://lore.kernel.org/all/20220330235920.2800929-1-ira.weiny@intel.com/
Effort wise, it's similar effort to hacking equivalent in QEMU but with the obvious advantage of being in tree and simpler for CI systems etc to use.
It would be nice to only have to use QEMU for complex system CI tests like the ones we are doing for CXL.
I dream of a world where every driver is testable by people with out hardware but I fear it may be a while yet. Hopefully this will get us a little closer!
I more or less follow what is going on here (good docs btw in the earlier patch definitely helped).
So far I'm thoroughly in favour of road test subject to actually being able to review the tests or getting sufficient support to do so. It's a 'how to scale it' question really...
Would rewriting the framework in C and forcing tests to be written in that language mean that maintainers would be able to review tests without external support?
I was wondering that. If we stayed in python I think we'd definitely want someone to be the 'roadtester/tests' maintainer (or group of maintainers) and their Ack to be expected for all tests we upstream. Idea being they'd sanity check correct use of framework and just how bad the python code us C developers are writing is ;)
However, we'd still need a good chunk of that 'framework' use review even if doing this in C.
I think this is reasonable, especially for the first tests for each subsystem where there will likely be support code and framework bits missing.
Add a roadtest for the recently-added tps62864 regulator driver. It tests voltage setting, mode setting, as well as devicetree mode translation. It uses the recently-added devicetree support in regulator-virtual-consumer.
All the variants supported by the driver have identical register interfaces so only one test/model is added.
It requires the following patches which are, as of writing, not in mainline:
- regulator: Add support for TPS6286x https://lore.kernel.org/lkml/20220204155241.576342-3-vincent.whitchurch@axis...
- regulator: virtual: add devicetree support https://lore.kernel.org/lkml/20220301111831.3742383-4-vincent.whitchurch@axi...
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- .../roadtest/tests/regulator/__init__.py | 0 .../roadtest/roadtest/tests/regulator/config | 4 + .../roadtest/tests/regulator/test_tps62864.py | 187 ++++++++++++++++++ 3 files changed, 191 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/config create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py
diff --git a/tools/testing/roadtest/roadtest/tests/regulator/__init__.py b/tools/testing/roadtest/roadtest/tests/regulator/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/tests/regulator/config b/tools/testing/roadtest/roadtest/tests/regulator/config new file mode 100644 index 000000000000..b2b503947e70 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/regulator/config @@ -0,0 +1,4 @@ +CONFIG_REGULATOR=y +CONFIG_REGULATOR_DEBUG=y +CONFIG_REGULATOR_VIRTUAL_CONSUMER=y +CONFIG_REGULATOR_TPS6286X=m diff --git a/tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py b/tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py new file mode 100644 index 000000000000..f7db4293d840 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py @@ -0,0 +1,187 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +from typing import Any, Final + +from roadtest.backend.i2c import SimpleSMBusModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod, rmmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import ( + I2CDriver, + PlatformDriver, + read_str, + write_int, + write_str, +) + +REG_VOUT1: Final = 0x01 +REG_VOUT2: Final = 0x02 +REG_CONTROL: Final = 0x03 +REG_STATUS: Final = 0x05 + + +class TPS62864(SimpleSMBusModel): + def __init__(self, **kwargs: Any) -> None: + super().__init__( + # From datasheet section 8.6 Register map + # XXX does not match reality -- recheck + regs={ + REG_VOUT1: 0x64, + REG_VOUT2: 0x64, + REG_CONTROL: 0x00, + REG_STATUS: 0x00, + }, + regbytes=1, + **kwargs, + ) + + +class TestTPS62864(UMLTestCase): + dts = DtFragment( + src=""" +#include <dt-bindings/regulator/ti,tps62864.h> + +&i2c { + regulator@$normal$ { + compatible = "ti,tps62864"; + reg = <0x$normal$>; + + regulators { + tps62864_normal: SW { + regulator-name = "+0.85V"; + regulator-min-microvolt = <400000>; + regulator-max-microvolt = <1675000>; + regulator-allowed-modes = <TPS62864_MODE_NORMAL TPS62864_MODE_FPWM>; + }; + }; + }; + + regulator@$fpwm$ { + compatible = "ti,tps62864"; + reg = <0x$fpwm$>; + + regulators { + tps62864_fpwm: SW { + regulator-name = "+0.85V"; + regulator-min-microvolt = <400000>; + regulator-max-microvolt = <1675000>; + regulator-initial-mode = <TPS62864_MODE_FPWM>; + }; + }; + }; +}; + +/ { + tps62864_normal_consumer { + compatible = "regulator-virtual-consumer"; + default-supply = <&tps62864_normal>; + }; + + tps62864_fpwm_consumer { + compatible = "regulator-virtual-consumer"; + default-supply = <&tps62864_fpwm>; + }; +}; + """, + variables={ + "normal": DtVar.I2C_ADDR, + "fpwm": DtVar.I2C_ADDR, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("tps6286x-regulator") + + @classmethod + def tearDownClass(cls) -> None: + rmmod("tps6286x-regulator") + + def setUp(self) -> None: + self.driver = I2CDriver("tps6286x") + self.hw = Hardware("i2c") + self.hw.load_model(TPS62864) + + def tearDown(self) -> None: + self.hw.close() + + def test_voltage(self) -> None: + with ( + self.driver.bind(self.dts["normal"]), + PlatformDriver("reg-virt-consumer").bind( + "tps62864_normal_consumer" + ) as consumerdev, + ): + maxfile = consumerdev.path / "max_microvolts" + minfile = consumerdev.path / "min_microvolts" + + write_int(maxfile, 1675000) + write_int(minfile, 800000) + + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_CONTROL, 1 << 5) + mock.assert_reg_write_once(self, REG_VOUT1, 0x50) + mock.reset_mock() + + mV = 1000 + data = [ + (400 * mV, 0x00), + (900 * mV, 0x64), + (1675 * mV, 0xFF), + ] + + for voltage, val in data: + write_int(minfile, voltage) + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_VOUT1, val) + mock.reset_mock() + + write_int(minfile, 0) + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_CONTROL, 0) + mock.reset_mock() + + def test_modes(self) -> None: + with ( + self.driver.bind(self.dts["normal"]), + PlatformDriver("reg-virt-consumer").bind( + "tps62864_normal_consumer" + ) as consumerdev, + ): + modefile = consumerdev.path / "mode" + write_str(modefile, "fast") + + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_CONTROL, 1 << 4) + mock.reset_mock() + + write_str(modefile, "normal") + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_CONTROL, 0) + mock.reset_mock() + + def test_dt_force_pwm(self) -> None: + with ( + self.driver.bind(self.dts["fpwm"]), + PlatformDriver("reg-virt-consumer").bind( + "tps62864_fpwm_consumer" + ) as consumerdev, + ): + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_CONTROL, 1 << 4) + mock.reset_mock() + + modefile = consumerdev.path / "mode" + self.assertEquals(read_str(modefile), "fast") + + maxfile = consumerdev.path / "max_microvolts" + minfile = consumerdev.path / "min_microvolts" + + write_int(maxfile, 1675000) + write_int(minfile, 800000) + + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_CONTROL, 1 << 5 | 1 << 4) + mock.reset_mock()
On Fri, Mar 11, 2022 at 05:24:44PM +0100, Vincent Whitchurch wrote:
This looks like it could be useful, modulo the general concerns with mocking stuff. I've not looked at the broader framework stuff in any meanigful way.
- @classmethod
- def setUpClass(cls) -> None:
insmod("tps6286x-regulator")
Shouldn't this get figured out when the device gets created in DT (if it doesn't I guess the tests found a bug...)?
- def setUp(self) -> None:
self.driver = I2CDriver("tps6286x")
self.hw = Hardware("i2c")
self.hw.load_model(TPS62864)
This feels like there could be some syntactic sugar to say "create this I2C device" in one call? In general a lot of the frameworkish stuff feels verbose.
- def test_voltage(self) -> None:
with (
self.driver.bind(self.dts["normal"]),
PlatformDriver("reg-virt-consumer").bind(
"tps62864_normal_consumer"
) as consumerdev,
):
maxfile = consumerdev.path / "max_microvolts"
minfile = consumerdev.path / "min_microvolts"
write_int(maxfile, 1675000)
write_int(minfile, 800000)
mock = self.hw.update_mock()
mock.assert_reg_write_once(self, REG_CONTROL, 1 << 5)
mock.assert_reg_write_once(self, REG_VOUT1, 0x50)
mock.reset_mock()
Some comments about the assertations here would seem to be in order. It's not altogether clear what this is testing - it looks to be verifying that the regulator is enabled with the voltage set to 800mV mapping to 0x50 in VOUT1 but I'm not sure that the idle reader would pick that up.
mV = 1000
data = [
(400 * mV, 0x00),
(900 * mV, 0x64),
(1675 * mV, 0xFF),
]
for voltage, val in data:
write_int(minfile, voltage)
mock = self.hw.update_mock()
mock.assert_reg_write_once(self, REG_VOUT1, val)
mock.reset_mock()
For covering regulators in general (especially those like this that use the generic helpers) I'd be inclined to go through every single voltage that can be set which isn't so interesting for this driver with it's linear voltage control but more interesting for something that's not continuous. I'd also put a cross check in that the voltage and enable state that's reported via the read interface in sysfs is the one that we think we've just set, that'd validate that the framework's model of what's going on matches both what the driver did to the "hardware" and what the running kernel thinks is going on so we're joined up top to bottom (for the regulator framework the read values come from the driver so it is actually covering the driver).
This all feels like it could readily be factored out into a generic helper, much as the actual drivers are especially when they're more data driven. Ideally with the ability to override the default I/O operations for things with sequences that need to be followed instead of just a bitfield to update. Callbacks to validate enable state, voltage, mode and so on in the hardware. If we did that then rather than open coding every single test for every single device we could approach things at the framework level and give people working on a given device a pile of off the shelf tests which are more likely to catch things that an individual driver author might've missed, it also avoids the test coverage being more laborious than writing the actual driver.
This does raise the questions I mentioned about how useful the testing really is of course, even more so when someone works out how to generate the data tables for the test and the driver from the same source, but that's just generally an issue for mocked tests at the conceptual level and clearly it's an approach that's fairly widely used and people get value from.
On Fri, Mar 11, 2022 at 06:06:54PM +0000, Mark Brown wrote:
On Fri, Mar 11, 2022 at 05:24:44PM +0100, Vincent Whitchurch wrote: This looks like it could be useful, modulo the general concerns with mocking stuff. I've not looked at the broader framework stuff in any meanigful way.
Thank you for having a look!
Here's a bit of background story about how I used this particular test, which hopefully shows an example of where I've seen the benefits of mocking hardware:
When I wrote this tps6286x driver a while ago, I tested it as one usually does, checking with i2cdump that the correct register values are written, measuring the voltages with a multimeter, rising and repeating with different devicetree properties, and so on. (This framework didn't exist at that point.)
Later, when preparing the driver for mainline submission, I wanted a quick way to check that any changes or cleanups that I needed to do during that process didn't invalidate all my and others' earlier testing. The easiest way to do that was to ensure that the driver continued to write the same bits in the same registers when given the same set of inputs and devicetree properties, and that is where I found the mocking to be useful.
In this case where there is no external input, the testing could of course have all been done manually with the real hardware, but there was little reason to do so when the hardware was the one factor which had not changed. The abilitly to create multiple devices with different devicetree properties (such as fast mode on/off) was a real time-saver too.
- @classmethod
- def setUpClass(cls) -> None:
insmod("tps6286x-regulator")
Shouldn't this get figured out when the device gets created in DT (if it doesn't I guess the tests found a bug...)?
The system isn't set up to load modules automatically. The reason for this is to give the test cases full control of when the module is loaded and unload, since the tests could want to load the module with specific options.
Also, the framework splits up logs and shows errors that occurs during each specific test if the tests fail, and this would become less useful if all modules for all the devices in the devicetree get loaded on startup when the devicetree is parsed and one of the modules failed to load or crashed when loaded.
- def setUp(self) -> None:
self.driver = I2CDriver("tps6286x")
self.hw = Hardware("i2c")
self.hw.load_model(TPS62864)
This feels like there could be some syntactic sugar to say "create this I2C device" in one call? In general a lot of the frameworkish stuff feels verbose.
Yes, I agree this could be simplified. I think the update_mock/reset_mock dance could also potentially be simplified with a with statement.
Beyond that, yes, there is some boilerplate setup for each test to bind the devices. This can differ between drivers and subsystems so I'm not sure how much could be shared, but I guess some of them could be separated out into a internal function for this particular test.
- def test_voltage(self) -> None:
with (
self.driver.bind(self.dts["normal"]),
PlatformDriver("reg-virt-consumer").bind(
"tps62864_normal_consumer"
) as consumerdev,
):
maxfile = consumerdev.path / "max_microvolts"
minfile = consumerdev.path / "min_microvolts"
write_int(maxfile, 1675000)
write_int(minfile, 800000)
mock = self.hw.update_mock()
mock.assert_reg_write_once(self, REG_CONTROL, 1 << 5)
mock.assert_reg_write_once(self, REG_VOUT1, 0x50)
mock.reset_mock()
Some comments about the assertations here would seem to be in order. It's not altogether clear what this is testing - it looks to be verifying that the regulator is enabled with the voltage set to 800mV mapping to 0x50 in VOUT1 but I'm not sure that the idle reader would pick that up.
Yes, I will add some comments. I also made some of the bit fields use constants in some of the other driver, that could be done here too.
mV = 1000
data = [
(400 * mV, 0x00),
(900 * mV, 0x64),
(1675 * mV, 0xFF),
]
for voltage, val in data:
write_int(minfile, voltage)
mock = self.hw.update_mock()
mock.assert_reg_write_once(self, REG_VOUT1, val)
mock.reset_mock()
For covering regulators in general (especially those like this that use the generic helpers) I'd be inclined to go through every single voltage that can be set which isn't so interesting for this driver with it's linear voltage control but more interesting for something that's not continuous.
That could be useful in some cases, but if going through all the voltages in a loop requires that the test implement the exact same voltage-to-bitfield conversion function as the driver, then the benefit of that part of the test is unclear. That's the reason why for example the OPT3001 test uses known values from the datasheet rather than just copying the conversion function in the driver to Python.
I'd also put a cross check in that the voltage and enable state that's reported via the read interface in sysfs is the one that we think we've just set, that'd validate that the framework's model of what's going on matches both what the driver did to the "hardware" and what the running kernel thinks is going on so we're joined up top to bottom (for the regulator framework the read values come from the driver so it is actually covering the driver).
Makes sense, I can add that.
This all feels like it could readily be factored out into a generic helper, much as the actual drivers are especially when they're more data driven. Ideally with the ability to override the default I/O operations for things with sequences that need to be followed instead of just a bitfield to update. Callbacks to validate enable state, voltage, mode and so on in the hardware. If we did that then rather than open coding every single test for every single device we could approach things at the framework level and give people working on a given device a pile of off the shelf tests which are more likely to catch things that an individual driver author might've missed, it also avoids the test coverage being more laborious than writing the actual driver.
Things could certainly be factored out in the future, but I'm a bit wary of attempting to do that when we have a test for only one regulator driver, and a very minimal regulator driver at that.
This does raise the questions I mentioned about how useful the testing really is of course, even more so when someone works out how to generate the data tables for the test and the driver from the same source, but that's just generally an issue for mocked tests at the conceptual level and clearly it's an approach that's fairly widely used and people get value from.
For the regulator drivers which are purely-data driven such as the ones mostly implemented by setting the various fields in struct regulator_desc along with the helpers in the framework, it could perhaps be useful to implement kunit tests in the regulator subsystem which test that using the various fields actually results in the expected consumer-visible behaviour with the regulator API.
Then, for the indivudal drivers themselves, roadtests could cover things like probe handling, functions implemented without using helpers, checks that the correct variant's registers are used in drivers supporting multiple variants, custom devicetree properties, interrupt handling, and the like.
On Thu, Mar 17, 2022 at 04:13:26PM +0100, Vincent Whitchurch wrote:
On Fri, Mar 11, 2022 at 06:06:54PM +0000, Mark Brown wrote:
- @classmethod
- def setUpClass(cls) -> None:
insmod("tps6286x-regulator")
Shouldn't this get figured out when the device gets created in DT (if it doesn't I guess the tests found a bug...)?
The system isn't set up to load modules automatically. The reason for this is to give the test cases full control of when the module is loaded and unload, since the tests could want to load the module with specific options.
That seems like the uncommon case which could remove the module if it explicitly needed it.
Also, the framework splits up logs and shows errors that occurs during each specific test if the tests fail, and this would become less useful if all modules for all the devices in the devicetree get loaded on startup when the devicetree is parsed and one of the modules failed to load or crashed when loaded.
That sounds like stuff that would be covered already by normal boot testing?
write_int(minfile, voltage)
mock = self.hw.update_mock()
mock.assert_reg_write_once(self, REG_VOUT1, val)
mock.reset_mock()
For covering regulators in general (especially those like this that use the generic helpers) I'd be inclined to go through every single voltage that can be set which isn't so interesting for this driver with it's linear voltage control but more interesting for something that's not continuous.
That could be useful in some cases, but if going through all the voltages in a loop requires that the test implement the exact same voltage-to-bitfield conversion function as the driver, then the benefit of that part of the test is unclear. That's the reason why for example the OPT3001 test uses known values from the datasheet rather than just copying the conversion function in the driver to Python.
That's just a generic problem with mocking though - ultimately you have to type the same values into the mock and the driver somehow, it's just a question of if you type in all the values or some of the values and if you use the same format to type them in. My inclination is to get better coverage since it makes it more likely that the interesting cases will be picked up, and you can make tests that do things like combine multiple settings which might turn something up.
This all feels like it could readily be factored out into a generic helper, much as the actual drivers are especially when they're more data driven. Ideally with the ability to override the default I/O operations for things with sequences that need to be followed instead of just a bitfield to update. Callbacks to validate enable state, voltage, mode and so on in the hardware. If we did that then rather than open coding every single test for every single device we could approach things at the framework level and give people working on a given device a pile of off the shelf tests which are more likely to catch things that an individual driver author might've missed, it also avoids the test coverage being more laborious than writing the actual driver.
Things could certainly be factored out in the future, but I'm a bit wary of attempting to do that when we have a test for only one regulator driver, and a very minimal regulator driver at that.
My thinking here is that since the driver is so minimal and data driven it's clear that any tests for it can also be generalised to cover at the very least all similarly data driven drivers.
This does raise the questions I mentioned about how useful the testing really is of course, even more so when someone works out how to generate the data tables for the test and the driver from the same source, but that's just generally an issue for mocked tests at the conceptual level and clearly it's an approach that's fairly widely used and people get value from.
For the regulator drivers which are purely-data driven such as the ones mostly implemented by setting the various fields in struct regulator_desc along with the helpers in the framework, it could perhaps be useful to implement kunit tests in the regulator subsystem which test that using the various fields actually results in the expected consumer-visible behaviour with the regulator API.
Then, for the indivudal drivers themselves, roadtests could cover things like probe handling, functions implemented without using helpers, checks that the correct variant's registers are used in drivers supporting multiple variants, custom devicetree properties, interrupt handling, and the like.
That would be the more obvious approach than roadtest, but that's what's there in the patch I was reviewing so... There is also the fact that the external pattern for the operations is the same no matter if they're for a simple data driven driver or one using custom ops so you should really be able to get the core coverage for every driver set up in a way that lets you plug in a model of how the I/O performing each operation is expected to work and then have the framework crunch through combinations of actions to make sure that all the corner cases check out.
On Thu, Mar 17, 2022 at 05:53:22PM +0000, Mark Brown wrote:
On Thu, Mar 17, 2022 at 04:13:26PM +0100, Vincent Whitchurch wrote:
On Fri, Mar 11, 2022 at 06:06:54PM +0000, Mark Brown wrote:
- @classmethod
- def setUpClass(cls) -> None:
insmod("tps6286x-regulator")
Shouldn't this get figured out when the device gets created in DT (if it doesn't I guess the tests found a bug...)?
The system isn't set up to load modules automatically. The reason for this is to give the test cases full control of when the module is loaded and unload, since the tests could want to load the module with specific options.
That seems like the uncommon case which could remove the module if it explicitly needed it.
Another reason was to get the tests to test module unloading since I've seen a lot of especially new driver writers forget to test that, but I realise that for most normal drivers that should be mostly covered by the fact that we test device unbinding.
So I went ahead and implemented this and it seems to work. As you hinted earlier, this also means that the modalias stuff gets tested, and as we know that's been broken in the recent past for a bunch of drivers, so that's another advantage to automatic module loading, besides the boilerplate reduction in the tests.
Add a roadtest for the PCF8563 RTC driver, testing many of the features including alarm and invalid time handling. Since it's the first roadtest for RTC, some helper code for handling the ABI is included.
The following fixes were posted for problems identified during development of these tests:
- rtc: fix use-after-free on device removal https://lore.kernel.org/lkml/20211210160951.7718-1-vincent.whitchurch@axis.c...
- rtc: pcf8563: clear RTC_FEATURE_ALARM if no irq https://lore.kernel.org/lkml/20220301131220.4011810-1-vincent.whitchurch@axi...
- rtc: pcf8523: fix alarm interrupt disabling https://lore.kernel.org/lkml/20211103152253.22844-1-vincent.whitchurch@axis.... (not the same hardware/driver, but this was the original target for test development)
Signed-off-by: Vincent Whitchurch vincent.whitchurch@axis.com --- .../roadtest/roadtest/tests/rtc/__init__.py | 0 .../roadtest/roadtest/tests/rtc/config | 1 + .../roadtest/roadtest/tests/rtc/rtc.py | 73 ++++ .../roadtest/tests/rtc/test_pcf8563.py | 348 ++++++++++++++++++ 4 files changed, 422 insertions(+) create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/config create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/rtc.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py
diff --git a/tools/testing/roadtest/roadtest/tests/rtc/__init__.py b/tools/testing/roadtest/roadtest/tests/rtc/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/tools/testing/roadtest/roadtest/tests/rtc/config b/tools/testing/roadtest/roadtest/tests/rtc/config new file mode 100644 index 000000000000..f3654f9d7c19 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/rtc/config @@ -0,0 +1 @@ +CONFIG_RTC_DRV_PCF8563=m diff --git a/tools/testing/roadtest/roadtest/tests/rtc/rtc.py b/tools/testing/roadtest/roadtest/tests/rtc/rtc.py new file mode 100644 index 000000000000..1a2855bfc195 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/rtc/rtc.py @@ -0,0 +1,73 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import contextlib +import fcntl +import struct +import typing +from pathlib import Path +from typing import Any, cast + +RTC_RD_TIME = 0x80247009 +RTC_SET_TIME = 0x4024700A +RTC_WKALM_SET = 0x4028700F +RTC_VL_READ = 0x80047013 + +RTC_IRQF = 0x80 +RTC_AF = 0x20 + +RTC_VL_DATA_INVALID = 1 << 0 + + +class RTCTime(typing.NamedTuple): + tm_sec: int + tm_min: int + tm_hour: int + tm_mday: int + tm_mon: int + tm_year: int + tm_wday: int + tm_yday: int + tm_isdst: int + + +class RTC(contextlib.AbstractContextManager): + def __init__(self, devpath: Path) -> None: + rtc = next(devpath.glob("rtc/rtc*")).name + self.filename = f"/dev/{rtc}" + + def __enter__(self) -> "RTC": + self.file = open(self.filename, "rb") + return self + + def __exit__(self, *_: Any) -> None: + self.file.close() + + def read_time(self) -> RTCTime: + s = struct.Struct("9i") + buf = bytearray(s.size) + fcntl.ioctl(self.file.fileno(), RTC_RD_TIME, buf) + return RTCTime._make(s.unpack(buf)) + + def set_time(self, tm: RTCTime) -> int: + s = struct.Struct("9i") + buf = bytearray(s.size) + s.pack_into(buf, 0, *tm) + return fcntl.ioctl(self.file.fileno(), RTC_SET_TIME, buf) + + def set_wake_alarm(self, enabled: bool, time: RTCTime) -> int: + s = struct.Struct("2B9i") + buf = bytearray(s.size) + s.pack_into(buf, 0, enabled, False, *time) + return fcntl.ioctl(self.file.fileno(), RTC_WKALM_SET, buf) + + def read(self) -> int: + s = struct.Struct("L") + buf = self.file.read(s.size) + return cast(int, s.unpack(buf)[0]) + + def read_vl(self) -> int: + s = struct.Struct("I") + buf = bytearray(s.size) + fcntl.ioctl(self.file.fileno(), RTC_VL_READ, buf) + return cast(int, s.unpack(buf)[0]) diff --git a/tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py b/tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py new file mode 100644 index 000000000000..a9f4c6d92762 --- /dev/null +++ b/tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py @@ -0,0 +1,348 @@ +# SPDX-License-Identifier: GPL-2.0-only +# Copyright Axis Communications AB + +import errno +import logging +from typing import Any, Final, Optional + +from roadtest.backend.i2c import I2CModel +from roadtest.core.devicetree import DtFragment, DtVar +from roadtest.core.hardware import Hardware +from roadtest.core.modules import insmod +from roadtest.core.suite import UMLTestCase +from roadtest.core.sysfs import I2CDriver + +from . import rtc + +logger = logging.getLogger(__name__) + +REG_CONTROL_STATUS_1: Final = 0x00 +REG_CONTROL_STATUS_2: Final = 0x01 +REG_VL_SECONDS: Final = 0x02 +REG_VL_MINUTES: Final = 0x03 +REG_VL_HOURS: Final = 0x04 +REG_VL_DAYS: Final = 0x05 +REG_VL_WEEKDAYS: Final = 0x06 +REG_VL_CENTURY_MONTHS: Final = 0x07 +REG_VL_YEARS: Final = 0x08 +REG_VL_MINUTE_ALARM: Final = 0x09 +REG_VL_HOUR_ALARM: Final = 0x0A +REG_VL_DAY_ALARM: Final = 0x0B +REG_VL_WEEKDAY_ALARM: Final = 0x0C +REG_CLKOUT_CONTROL: Final = 0x0D +REG_TIMER_CONTROL: Final = 0x0E +REG_TIMER: Final = 0x0F + +REG_CONTROL_STATUS_2_AIE: Final = 1 << 1 +REG_CONTROL_STATUS_2_AF: Final = 1 << 3 + +REG_VL_CENTURY_MONTHS_C: Final = 1 << 7 + +REG_VL_ALARM_AE: Final = 1 << 7 + + +class PCF8563(I2CModel): + def __init__(self, int: Optional[int] = None, **kwargs: Any) -> None: + super().__init__(**kwargs) + self.int = int + self._set_int(False) + + self.reg_addr = 0 + # Reset values from Table 27 in datasheet, with X and - bits set to 0 + self.regs = { + REG_CONTROL_STATUS_1: 0b_0000_1000, + REG_CONTROL_STATUS_2: 0b_0000_0000, + REG_VL_SECONDS: 0b_1000_0000, + REG_VL_MINUTES: 0b_0000_0000, + REG_VL_HOURS: 0b_0000_0000, + REG_VL_DAYS: 0b_0000_0000, + REG_VL_WEEKDAYS: 0b_0000_0000, + REG_VL_CENTURY_MONTHS: 0b_0000_0000, + REG_VL_YEARS: 0b_0000_0000, + REG_VL_MINUTE_ALARM: 0b_1000_0000, + REG_VL_HOUR_ALARM: 0b_1000_0000, + REG_VL_DAY_ALARM: 0b_1000_0000, + REG_VL_WEEKDAY_ALARM: 0b_1000_0000, + REG_CLKOUT_CONTROL: 0b_1000_0000, + REG_TIMER_CONTROL: 0b_0000_0011, + REG_TIMER: 0b_0000_0000, + } + + def _set_int(self, active: int) -> None: + # Active-low + self.backend.gpio.set(self.int, not active) + + def _check_alarm(self, addr: int) -> None: + alarmregs = [ + REG_VL_MINUTE_ALARM, + REG_VL_HOUR_ALARM, + REG_VL_DAY_ALARM, + REG_VL_WEEKDAY_ALARM, + ] + timeregs = [ + REG_VL_MINUTES, + REG_VL_HOURS, + REG_VL_DAYS, + REG_VL_WEEKDAYS, + ] + + if addr not in alarmregs + timeregs: + return + + af = all( + self.regs[a] == self.regs[b] + for a, b in zip(alarmregs, timeregs) + if not self.regs[a] & REG_VL_ALARM_AE + ) + self.reg_write(REG_CONTROL_STATUS_2, self.regs[REG_CONTROL_STATUS_2] | af << 3) + + def _update_irq(self) -> None: + aie = self.regs[REG_CONTROL_STATUS_2] & REG_CONTROL_STATUS_2_AIE + af = self.regs[REG_CONTROL_STATUS_2] & REG_CONTROL_STATUS_2_AF + + logger.debug(f"{aie=} {af=}") + self._set_int(aie and af) + + def reg_read(self, addr: int) -> int: + val = self.regs[addr] + return val + + def reg_write(self, addr: int, val: int) -> None: + assert addr in self.regs + self.regs[addr] = val + logger.debug(f"{addr=:x} {val=:x}") + self._check_alarm(addr) + self._update_irq() + + def read(self, len: int) -> bytes: + data = bytearray(len) + + for i in range(len): + data[i] = self.reg_read(self.reg_addr) + self.reg_addr = self.reg_addr + 1 + + return bytes(data) + + def write(self, data: bytes) -> None: + self.reg_addr = data[0] + + for i, byte in enumerate(data[1:]): + addr = self.reg_addr + i + self.backend.mock.reg_write(addr, byte) + self.reg_write(addr, byte) + + +class TestPCF8563(UMLTestCase): + dts = DtFragment( + src=""" +#include <dt-bindings/interrupt-controller/irq.h> + +&i2c { + rtc@$addr$ { + compatible = "nxp,pcf8563"; + reg = <0x$addr$>; + }; + + rtc@$irqaddr$ { + compatible = "nxp,pcf8563"; + reg = <0x$irqaddr$>; + interrupt-parent = <&gpio>; + interrupts = <$gpio$ IRQ_TYPE_LEVEL_LOW>; + }; +}; + """, + variables={ + "addr": DtVar.I2C_ADDR, + "irqaddr": DtVar.I2C_ADDR, + "gpio": DtVar.GPIO_PIN, + }, + ) + + @classmethod + def setUpClass(cls) -> None: + insmod("rtc-pcf8563") + + @classmethod + def tearDownClass(cls) -> None: + # Can't rmmod since alarmtimer holds permanent reference + pass + + def setUp(self) -> None: + self.driver = I2CDriver("rtc-pcf8563") + self.hw = Hardware("i2c") + self.hw.load_model(PCF8563, int=self.dts["gpio"]) + + def tearDown(self) -> None: + self.hw.close() + + def test_read_time_invalid(self) -> None: + addr = self.dts["addr"] + with self.driver.bind(addr) as dev, rtc.RTC(dev.path) as rtcdev: + self.assertEqual(rtcdev.read_vl(), rtc.RTC_VL_DATA_INVALID) + + with self.assertRaises(OSError) as cm: + rtcdev.read_time() + self.assertEqual(cm.exception.errno, errno.EINVAL) + + def test_no_alarm_support(self) -> None: + addr = self.dts["addr"] + with self.driver.bind(addr) as dev, rtc.RTC(dev.path) as rtcdev: + # Make sure the times are valid so we don't get -EINVAL due to + # that. + tm = rtc.RTCTime( + tm_sec=10, + tm_min=1, + tm_hour=1, + tm_mday=1, + tm_mon=0, + tm_year=121, + tm_wday=0, + tm_yday=0, + tm_isdst=0, + ) + rtcdev.set_time(tm) + + alarmtm = tm._replace(tm_sec=0, tm_min=2) + with self.assertRaises(OSError) as cm: + rtcdev.set_wake_alarm(True, alarmtm) + self.assertEqual(cm.exception.errno, errno.EINVAL) + + def test_alarm(self) -> None: + addr = self.dts["irqaddr"] + with self.driver.bind(addr) as dev, rtc.RTC(dev.path) as rtcdev: + tm = rtc.RTCTime( + tm_sec=10, + tm_min=1, + tm_hour=1, + tm_mday=1, + tm_mon=0, + tm_year=121, + tm_wday=5, + tm_yday=0, + tm_isdst=0, + ) + rtcdev.set_time(tm) + + alarmtm = tm._replace(tm_sec=0, tm_min=2) + rtcdev.set_wake_alarm(True, alarmtm) + + mock = self.hw.update_mock() + mock.assert_last_reg_write(self, REG_VL_MINUTE_ALARM, 0x02) + mock.assert_last_reg_write(self, REG_VL_HOUR_ALARM, 0x01) + mock.assert_last_reg_write(self, REG_VL_DAY_ALARM, 0x01) + mock.assert_last_reg_write(self, REG_VL_WEEKDAY_ALARM, 5) + mock.assert_last_reg_write( + self, REG_CONTROL_STATUS_2, REG_CONTROL_STATUS_2_AIE + ) + mock.reset_mock() + + self.hw.reg_write(REG_VL_MINUTES, 0x02) + self.hw.kick() + + # This waits for the interrupt + self.assertEqual(rtcdev.read() & 0xFF, rtc.RTC_IRQF | rtc.RTC_AF) + + alarmtm = tm._replace(tm_sec=0, tm_min=3) + rtcdev.set_wake_alarm(False, alarmtm) + + mock = self.hw.update_mock() + mock.assert_last_reg_write(self, REG_CONTROL_STATUS_2, 0) + + def test_read_time_valid(self) -> None: + self.hw.reg_write(REG_VL_SECONDS, 0x37) + self.hw.reg_write(REG_VL_MINUTES, 0x10) + self.hw.reg_write(REG_VL_HOURS, 0x11) + self.hw.reg_write(REG_VL_DAYS, 0x25) + self.hw.reg_write(REG_VL_WEEKDAYS, 0x00) + self.hw.reg_write(REG_VL_CENTURY_MONTHS, REG_VL_CENTURY_MONTHS_C | 0x12) + self.hw.reg_write(REG_VL_YEARS, 0x21) + + addr = self.dts["addr"] + with self.driver.bind(addr) as dev, rtc.RTC(dev.path) as rtcdev: + tm = rtcdev.read_time() + self.assertEqual( + tm, + rtc.RTCTime( + tm_sec=37, + tm_min=10, + tm_hour=11, + tm_mday=25, + tm_mon=11, + tm_year=121, + tm_wday=0, + tm_yday=0, + tm_isdst=0, + ), + ) + + def test_set_time_after_invalid(self) -> None: + addr = self.dts["addr"] + with self.driver.bind(addr) as dev, rtc.RTC(dev.path) as rtcdev: + self.assertEqual(rtcdev.read_vl(), rtc.RTC_VL_DATA_INVALID) + + tm = rtc.RTCTime( + tm_sec=37, + tm_min=10, + tm_hour=11, + tm_mday=25, + tm_mon=11, + tm_year=121, + tm_wday=0, + tm_yday=0, + tm_isdst=0, + ) + + rtcdev.set_time(tm) + tm2 = rtcdev.read_time() + self.assertEqual(tm, tm2) + + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_VL_SECONDS, 0x37) + mock.assert_reg_write_once(self, REG_VL_MINUTES, 0x10) + mock.assert_reg_write_once(self, REG_VL_HOURS, 0x11) + mock.assert_reg_write_once(self, REG_VL_DAYS, 0x25) + mock.assert_reg_write_once(self, REG_VL_WEEKDAYS, 0x00) + # The driver uses the wrong polarity of the Century bit + # if the time was invalid. This probably doesn't matter(?). + mock.assert_reg_write_once(self, REG_VL_CENTURY_MONTHS, 0 << 7 | 0x12) + mock.assert_reg_write_once(self, REG_VL_YEARS, 0x21) + + self.assertEqual(rtcdev.read_vl(), 0) + + def test_set_time_after_valid(self) -> None: + self.hw.reg_write(REG_VL_SECONDS, 0x37) + self.hw.reg_write(REG_VL_MINUTES, 0x10) + self.hw.reg_write(REG_VL_HOURS, 0x11) + self.hw.reg_write(REG_VL_DAYS, 0x25) + self.hw.reg_write(REG_VL_WEEKDAYS, 0x00) + self.hw.reg_write(REG_VL_CENTURY_MONTHS, REG_VL_CENTURY_MONTHS_C | 0x12) + self.hw.reg_write(REG_VL_YEARS, 0x21) + + addr = self.dts["addr"] + with self.driver.bind(addr) as dev, rtc.RTC(dev.path) as rtcdev: + tm = rtc.RTCTime( + tm_sec=37, + tm_min=10, + tm_hour=11, + tm_mday=25, + tm_mon=11, + tm_year=121, + tm_wday=0, + tm_yday=0, + tm_isdst=0, + ) + + rtcdev.set_time(tm) + tm2 = rtcdev.read_time() + self.assertEqual(tm, tm2) + + mock = self.hw.update_mock() + mock.assert_reg_write_once(self, REG_VL_SECONDS, 0x37) + mock.assert_reg_write_once(self, REG_VL_MINUTES, 0x10) + mock.assert_reg_write_once(self, REG_VL_HOURS, 0x11) + mock.assert_reg_write_once(self, REG_VL_DAYS, 0x25) + mock.assert_reg_write_once(self, REG_VL_WEEKDAYS, 0x00) + mock.assert_reg_write_once( + self, REG_VL_CENTURY_MONTHS, REG_VL_CENTURY_MONTHS_C | 0x12 + ) + mock.assert_reg_write_once(self, REG_VL_YEARS, 0x21)
+Kees Cook - I imagine you have already seen this, but I figured you would be interested because of your recent work on the KUnit UAPI and the mocking discussions. +Dmitry Vyukov - This made me think of the syzkaller/KUnit experiments we did a couple of years back - this would probably work a bit better.
On Fri, Mar 11, 2022 at 11:24 AM Vincent Whitchurch vincent.whitchurch@axis.com wrote:
This patchset proposes roadtest, a device-driver testing framework. Drivers are tested under User Mode Linux (UML) and interact with mocked/modelled hardware. The tests and hardware models are written in Python, the former using Python's built-in unittest framework.
Wow! This sounds awesome! I was hoping to get some kind of hardware modeling with KUnit eventually. I did some experiments, but this looks way more mature.
Drivers are tested via their userspace interfaces. The hardware models allow tests to inject values into registers and assert that drivers control the hardware in the right way and react as expected to stimuli.
I already took a look at the documentation patch - I'll comment there more in detail, but I like the hardware modelling and device tree code; it seems very usable.
Roadtest is meant to be used for relatively simple drivers, such as the ones part of the IIO, regulator and RTC subsystems.
Obviously for an initial version going after simple stuff makes sense, but I would hope there is applicability to any driver stack eventually.
Questions and answers:
= Why do we need this?
There are a large amount of these kind of drivers in the kernel. Most of the hardware is not available in current CI systems so most drivers can only, at best, be build-tested there. Even basic soundness such as a driver successfully probing and binding to the devices it tries to be support cannot be tested. Drivers cannot be easily regression-tested to ensure that bugs fixed once do not get reintroduced.
Many drivers support multiple related hardware variants, and far from all patch submitters have access to all the variants which the driver that they are patching supports, so there is no way for them to easily verify that they haven't broken something basic on a variant which they do not own.
Furthermore, hardware can be used in many different configurations with drivers supporting many different devicetree properties, so even just having access to all the variants would be insufficient.
On top of that, some of the chips measure environmental conditions such as temperature, so testing extreme cases may not be simple even if one has access to the hardware.
All this makes development, modification, maintenance, and reviewing of these drivers harder than it necessarily needs to be. Roadtest hopes to make some of these things slightly easier by providing a framework to create hardware models/mocks and to write testcases which exercise drivers using these models.
Very much agree. I used to do driver development and these difficulties are what prompted me to do KUnit.
= Do you have some specific examples of the kind of code this could be used to test?
Here is an example of a patch which can easily be regression-tested using roadtest (in fact, this series includes such a regression test) but is much harder to do so automatically with real hardware since it requires specific environmental conditions:
iio: light: opt3001: Fixed timeout error when 0 lux https://lore.kernel.org/lkml/20210920125351.6569-1-valek@2n.cz/
Here is another example. This driver has code which correctly parses a documented devicetree property (amstaos,proximity-diodes) but which then fails to actually communicate this setting to the hardware in any way. Such code can be easily tested with roadtest since the framework integrates devicetree support and provides functions to assert that drivers writes expected registers with expected values:
drivers/iio/light/tsl2772.c tsl2772_read_prox_diodes()
(Both the above examples happen to be from the same subsystem but that should in no way be taken to imply that such issues are unique to that subsystem or that that subsystem has more of them.)
= How does this relate to kselftests?
Tests in kselftests also test kernel code using the userspace interfaces, but that's about what's common between the frameworks. kselftests has other goals and does not provide any kind of mechanism for hardware mocking.
I had a question that after thinking about it; I think I know the answer, so I am going to ask the question anyway and attempt to answer it myself:
I agree in regard to mocking, but why not use kselftest for driving tests that check drivers from userspace? I believe there are other kselftest tests implemented in Python, why can't you just run your tests inside of kselftest?
Now, I believe the answer to this question is that you need to control spinning up your own kernel to run inside your test harness because you need to control the environment that the kernel runs in - is this correct?
= How does this relate to kunit?
Kunit is for unit testing of functions in kernel code, and is not meant for testing kernel code via userspace interfaces. It could in theory be used to test some of the simple drivers too, but that would require (1) a large amount of mocking code in various kernel frameworks, and, more importantly, (2) refactoring of the drivers to be tested.
I mostly agree, but I think there is something that is missing here: so roadtest seems to depend on having a user interface to test a driver - for a simple smoke test on a simple driver without a big driver stack on top, that makes sense, but what about testing error paths or a platform driver buried beneath a deep driver stack? I think there is potential for a powerful combination using KUnit to test the low level kernel API and using roadtest to mock the hardware environment and provide configuration.
I am imagining that we could have an in-kernel KUnit/roadtest API that we can use to have an in-kernel test request changes to the environment for creating error cases and the like that can be validated by KUnit test cases.
Going even further, I wonder if we could run kselftests inside of roadtest since roadtest allows us to change the environment on the fly.
This can be contrasted with roadtest which works with mostly unmodified drivers and which mocks the hardware at the lowest level without having to change kernel frameworks.
I think that is both potentially an advantage and a disadvantage.
The advantage is that your test is very general; roadtests would likely be portable across kernel versions.
The disadvantage is that you don't get as much code introspection: I imagine roadtest is not as good as testing error paths for example.
I also think that having to change code to make it more testable is often an advantage as much as a disadvantage.
Still, I think that is a good set of tradeoffs for roadtest to make when set against KUnit and kselftest since roadtest seems to fit in where kselftest and KUnit are weak.
= How do I use it?
See Documentation/dev-tools/roadtest.rst added by the documentation patch for more information about running and writing tests using this framework.
= What's included in the patchset?
The current framework allows developing tests for hardware which uses the I2C bus. Hardware models can also control GPIOs and use them to trigger interrupts.
This series includes tests for some IIO, regulator and RTC drivers. The regulator and RTC tests depend on a few driver patches which are either in review or in linux-next. These are noted in the commit messages.
The entire patch set, including the required dependencies, is also available in a git tree:
https://github.com/vwax/linux/commits/roadtest/rfc-v1
Cc: linux-kernel@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-um@lists.infradead.org
Cc: shuah@kernel.org Cc: brendanhiggins@google.com Cc: linux-kselftest@vger.kernel.org
Cc: jic23@kernel.org Cc: linux-iio@vger.kernel.org
Cc: lgirdwood@gmail.com Cc: broonie@kernel.org
Cc: a.zummo@towertech.it Cc: alexandre.belloni@bootlin.com Cc: linux-rtc@vger.kernel.org
Cc: corbet@lwn.net Cc: linux-doc@vger.kernel.org
Vincent Whitchurch (10): roadtest: import libvhost-user from QEMU roadtest: add C backend roadtest: add framework roadtest: add base config roadtest: add build files roadtest: add documentation iio: light: opt3001: add roadtest iio: light: vcnl4000: add roadtest regulator: tps62864: add roadtest rtc: pcf8563: add roadtest
Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/roadtest.rst | 669 ++++ tools/testing/roadtest/.gitignore | 2 + tools/testing/roadtest/Dockerfile | 25 + tools/testing/roadtest/Makefile | 84 + tools/testing/roadtest/init.sh | 19 + tools/testing/roadtest/pyproject.toml | 10 + tools/testing/roadtest/requirements.txt | 4 + tools/testing/roadtest/roadtest/__init__.py | 2 + .../roadtest/roadtest/backend/__init__.py | 0 .../roadtest/roadtest/backend/backend.py | 32 + .../testing/roadtest/roadtest/backend/gpio.py | 111 + .../testing/roadtest/roadtest/backend/i2c.py | 123 + .../testing/roadtest/roadtest/backend/main.py | 13 + .../testing/roadtest/roadtest/backend/mock.py | 20 + .../roadtest/roadtest/backend/test_gpio.py | 98 + .../roadtest/roadtest/backend/test_i2c.py | 84 + .../testing/roadtest/roadtest/cmd/__init__.py | 0 tools/testing/roadtest/roadtest/cmd/main.py | 146 + tools/testing/roadtest/roadtest/cmd/remote.py | 48 + .../roadtest/roadtest/core/__init__.py | 0 .../testing/roadtest/roadtest/core/control.py | 52 + .../roadtest/roadtest/core/devicetree.py | 155 + .../roadtest/roadtest/core/hardware.py | 94 + tools/testing/roadtest/roadtest/core/log.py | 42 + .../testing/roadtest/roadtest/core/modules.py | 38 + .../testing/roadtest/roadtest/core/opslog.py | 35 + tools/testing/roadtest/roadtest/core/proxy.py | 48 + tools/testing/roadtest/roadtest/core/suite.py | 286 ++ tools/testing/roadtest/roadtest/core/sysfs.py | 77 + .../roadtest/roadtest/core/test_control.py | 35 + .../roadtest/roadtest/core/test_devicetree.py | 31 + .../roadtest/roadtest/core/test_hardware.py | 41 + .../roadtest/roadtest/core/test_log.py | 54 + .../roadtest/roadtest/core/test_opslog.py | 27 + .../roadtest/roadtest/tests/__init__.py | 0 .../roadtest/roadtest/tests/base/config | 84 + .../roadtest/roadtest/tests/iio/__init__.py | 0 .../roadtest/roadtest/tests/iio/config | 1 + .../roadtest/roadtest/tests/iio/iio.py | 112 + .../roadtest/tests/iio/light/__init__.py | 0 .../roadtest/roadtest/tests/iio/light/config | 2 + .../roadtest/tests/iio/light/test_opt3001.py | 95 + .../roadtest/tests/iio/light/test_vcnl4000.py | 132 + .../roadtest/tests/iio/light/test_vcnl4010.py | 282 ++ .../roadtest/tests/iio/light/test_vcnl4040.py | 104 + .../roadtest/tests/iio/light/test_vcnl4200.py | 96 + .../roadtest/tests/regulator/__init__.py | 0 .../roadtest/roadtest/tests/regulator/config | 4 + .../roadtest/tests/regulator/test_tps62864.py | 187 ++ .../roadtest/roadtest/tests/rtc/__init__.py | 0 .../roadtest/roadtest/tests/rtc/config | 1 + .../roadtest/roadtest/tests/rtc/rtc.py | 73 + .../roadtest/tests/rtc/test_pcf8563.py | 348 ++ tools/testing/roadtest/src/.gitignore | 1 + tools/testing/roadtest/src/backend.c | 884 +++++ .../src/libvhost-user/include/atomic.h | 310 ++ .../src/libvhost-user/libvhost-user.c | 2885 +++++++++++++++++ .../src/libvhost-user/libvhost-user.h | 691 ++++ 59 files changed, 8798 insertions(+) create mode 100644 Documentation/dev-tools/roadtest.rst create mode 100644 tools/testing/roadtest/.gitignore create mode 100644 tools/testing/roadtest/Dockerfile create mode 100644 tools/testing/roadtest/Makefile create mode 100755 tools/testing/roadtest/init.sh create mode 100644 tools/testing/roadtest/pyproject.toml create mode 100644 tools/testing/roadtest/requirements.txt create mode 100644 tools/testing/roadtest/roadtest/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/backend.py create mode 100644 tools/testing/roadtest/roadtest/backend/gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/i2c.py create mode 100644 tools/testing/roadtest/roadtest/backend/main.py create mode 100644 tools/testing/roadtest/roadtest/backend/mock.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_i2c.py create mode 100644 tools/testing/roadtest/roadtest/cmd/__init__.py create mode 100644 tools/testing/roadtest/roadtest/cmd/main.py create mode 100644 tools/testing/roadtest/roadtest/cmd/remote.py create mode 100644 tools/testing/roadtest/roadtest/core/__init__.py create mode 100644 tools/testing/roadtest/roadtest/core/control.py create mode 100644 tools/testing/roadtest/roadtest/core/devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/log.py create mode 100644 tools/testing/roadtest/roadtest/core/modules.py create mode 100644 tools/testing/roadtest/roadtest/core/opslog.py create mode 100644 tools/testing/roadtest/roadtest/core/proxy.py create mode 100644 tools/testing/roadtest/roadtest/core/suite.py create mode 100644 tools/testing/roadtest/roadtest/core/sysfs.py create mode 100644 tools/testing/roadtest/roadtest/core/test_control.py create mode 100644 tools/testing/roadtest/roadtest/core/test_devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/test_hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/test_log.py create mode 100644 tools/testing/roadtest/roadtest/core/test_opslog.py create mode 100644 tools/testing/roadtest/roadtest/tests/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/base/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/iio.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/config create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/config create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/rtc.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py create mode 100644 tools/testing/roadtest/src/.gitignore create mode 100644 tools/testing/roadtest/src/backend.c create mode 100644 tools/testing/roadtest/src/libvhost-user/include/atomic.h create mode 100644 tools/testing/roadtest/src/libvhost-user/libvhost-user.c create mode 100644 tools/testing/roadtest/src/libvhost-user/libvhost-user.h
-- 2.34.1
On Mon, Mar 14, 2022 at 11:24:59PM +0100, Brendan Higgins wrote:
+Kees Cook - I imagine you have already seen this, but I figured you would be interested because of your recent work on the KUnit UAPI and the mocking discussions. +Dmitry Vyukov - This made me think of the syzkaller/KUnit experiments we did a couple of years back - this would probably work a bit better.
On Fri, Mar 11, 2022 at 11:24 AM Vincent Whitchurch vincent.whitchurch@axis.com wrote:
This patchset proposes roadtest, a device-driver testing framework. Drivers are tested under User Mode Linux (UML) and interact with mocked/modelled hardware. The tests and hardware models are written in Python, the former using Python's built-in unittest framework.
Wow! This sounds awesome! I was hoping to get some kind of hardware modeling with KUnit eventually. I did some experiments, but this looks way more mature.
Thank you for the comments!
Drivers are tested via their userspace interfaces. The hardware models allow tests to inject values into registers and assert that drivers control the hardware in the right way and react as expected to stimuli.
I already took a look at the documentation patch - I'll comment there more in detail, but I like the hardware modelling and device tree code; it seems very usable.
Roadtest is meant to be used for relatively simple drivers, such as the ones part of the IIO, regulator and RTC subsystems.
Obviously for an initial version going after simple stuff makes sense, but I would hope there is applicability to any driver stack eventually.
Yes, there is no inherent restriction to only simple hardware, but these kinds of subsystem are the ones where it's easier to apply the framework since there's simply less stuff to model/mock in the hardware.
Supporting different busses also requires some work in the framework and potentially some new drivers. For I2C we use virtio-i2c but there's no ready-made virtio-spi for example. For MMIO (PCI / platform drivers), I did some basic experiments with UML's virtio-mmio in the early stages of writing this framework.
= How does this relate to kselftests?
Tests in kselftests also test kernel code using the userspace interfaces, but that's about what's common between the frameworks. kselftests has other goals and does not provide any kind of mechanism for hardware mocking.
I had a question that after thinking about it; I think I know the answer, so I am going to ask the question anyway and attempt to answer it myself:
I agree in regard to mocking, but why not use kselftest for driving tests that check drivers from userspace? I believe there are other kselftest tests implemented in Python, why can't you just run your tests inside of kselftest?
Now, I believe the answer to this question is that you need to control spinning up your own kernel to run inside your test harness because you need to control the environment that the kernel runs in - is this correct?
Yes, that is correct. For example, the devicetree stuff requires that the kernel be booted with the devicetree. For the other tests also it's simpler to have a controlled environment without being affected by other stuff going on on the host. And generally it's of course easier if the kernel which is inevitably going to crash and burn due to buggy drivers isn't the one powering your workstation.
Also, there is no currently way to use virtio drivers such as virtio-i2c and virtio-gpio (which roadtest uses) with the virtio device side implemented in userspace on the same system, so that would have also required a fair bit of work to get running.
(On a side note, I've wondered why kselftest doesn't provide a standard way to run all the tests under kvm or something similar with all the correct configs. For example, the kernels I work with are on embedded systems and I rarely recompile my host kernel, and I assume that there are plenty of others in the same situation.)
= How does this relate to kunit?
Kunit is for unit testing of functions in kernel code, and is not meant for testing kernel code via userspace interfaces. It could in theory be used to test some of the simple drivers too, but that would require (1) a large amount of mocking code in various kernel frameworks, and, more importantly, (2) refactoring of the drivers to be tested.
I mostly agree, but I think there is something that is missing here: so roadtest seems to depend on having a user interface to test a driver - for a simple smoke test on a simple driver without a big driver stack on top, that makes sense, but what about testing error paths or a platform driver buried beneath a deep driver stack? I think there is potential for a powerful combination using KUnit to test the low level kernel API and using roadtest to mock the hardware environment and provide configuration.
Yes, that could be useful. I have previously written some kunit tests for some experimental memory management code which required different devicetree reserved-memory nodes and arm64 (no hardware mocking) to run, and I ran them by having a shell script which ran QEMU several times with appropriate -append kunit.filter_glob=foo and -dtb options and post-processing the logs with kunit.py.
I am imagining that we could have an in-kernel KUnit/roadtest API that we can use to have an in-kernel test request changes to the environment for creating error cases and the like that can be validated by KUnit test cases.
Going even further, I wonder if we could run kselftests inside of roadtest since roadtest allows us to change the environment on the fly.
Sounds interesting, but I would likely need to see concrete examples to understand what kind of environment we'd want to change from within the kernel.
This can be contrasted with roadtest which works with mostly unmodified drivers and which mocks the hardware at the lowest level without having to change kernel frameworks.
I think that is both potentially an advantage and a disadvantage.
The advantage is that your test is very general; roadtests would likely be portable across kernel versions.
The disadvantage is that you don't get as much code introspection: I imagine roadtest is not as good as testing error paths for example.
I also think that having to change code to make it more testable is often an advantage as much as a disadvantage.
Yes, that's true, but I highlighted the unmodified drivers bit because (1) the process of refactoring drivers which don't have tests to make them testable in itself carries it with a risk of breaking stuff, and (2) and there are simply so many existing drivers that it's very unlikely that most of them get refactored, but it should be relatively easy to, for example, add a regression test for a specific bug fix with roadtest.
Still, I think that is a good set of tradeoffs for roadtest to make when set against KUnit and kselftest since roadtest seems to fit in where kselftest and KUnit are weak.
On Fri, 11 Mar 2022 17:24:35 +0100 Vincent Whitchurch vincent.whitchurch@axis.com wrote:
This patchset proposes roadtest, a device-driver testing framework. Drivers are tested under User Mode Linux (UML) and interact with mocked/modelled hardware. The tests and hardware models are written in Python, the former using Python's built-in unittest framework.
Drivers are tested via their userspace interfaces. The hardware models allow tests to inject values into registers and assert that drivers control the hardware in the right way and react as expected to stimuli.
Roadtest is meant to be used for relatively simple drivers, such as the ones part of the IIO, regulator and RTC subsystems.
Hi All,
Just wanted to very briefly report back on my experience of using this framework.
Given I wanted a suitable job to try out it's usefulness when doing refactoring /development, I decided to tidy up one of the remaining IIO drivers in staging and see how things went in developing tests to hit the particular code I was modifying. At some point I might extend this to a more comprehensive test suite for that driver, but for now it does basic channel reading and a few other things + verifies some of the register state changes seen on the hardware side of things.
Whilst my python could be said to be decidedly rusty (last time I recall writing some was for an intern project 20 years back), it was fairly easy to get something working using the docs in this series and the fine engineering tool of cut and paste.
Road test worked very well.
Was it easier than my existing hacked up QEMU board emulation that lets me instantiate minimal emulation pretty quickly?
Pretty similar on balance but big advantage here is I'm not having to ask people to go fetch a tree and build QEMU just to sanity check the driver changes. Also note I'm doing a lot of QEMU work for the day job at the moment, so it's not really a fair comparison if the question is what would most kernel driver developers find useful.
So for now I'll probably mix and match depending on the complexity of the device I'm emulating, but roadtest is definitely a good addition to the toolkit.
Note this is putting aside all the advantages of having tests in tree and the much lighter amount of infrastructure needed to run those over QEMU CI.
If anyone is curious patch set with tests and the staging graduation of the AD7746 CDC driver.
https://lore.kernel.org/all/20220418192907.763933-18-jic23@kernel.org/
Great work Vincent. I'm looking forward to using this more.
Thanks,
Jonathan
Questions and answers:
= Why do we need this?
There are a large amount of these kind of drivers in the kernel. Most of the hardware is not available in current CI systems so most drivers can only, at best, be build-tested there. Even basic soundness such as a driver successfully probing and binding to the devices it tries to be support cannot be tested. Drivers cannot be easily regression-tested to ensure that bugs fixed once do not get reintroduced.
Many drivers support multiple related hardware variants, and far from all patch submitters have access to all the variants which the driver that they are patching supports, so there is no way for them to easily verify that they haven't broken something basic on a variant which they do not own.
Furthermore, hardware can be used in many different configurations with drivers supporting many different devicetree properties, so even just having access to all the variants would be insufficient.
On top of that, some of the chips measure environmental conditions such as temperature, so testing extreme cases may not be simple even if one has access to the hardware.
All this makes development, modification, maintenance, and reviewing of these drivers harder than it necessarily needs to be. Roadtest hopes to make some of these things slightly easier by providing a framework to create hardware models/mocks and to write testcases which exercise drivers using these models.
= Do you have some specific examples of the kind of code this could be used to test?
Here is an example of a patch which can easily be regression-tested using roadtest (in fact, this series includes such a regression test) but is much harder to do so automatically with real hardware since it requires specific environmental conditions:
iio: light: opt3001: Fixed timeout error when 0 lux https://lore.kernel.org/lkml/20210920125351.6569-1-valek@2n.cz/
Here is another example. This driver has code which correctly parses a documented devicetree property (amstaos,proximity-diodes) but which then fails to actually communicate this setting to the hardware in any way. Such code can be easily tested with roadtest since the framework integrates devicetree support and provides functions to assert that drivers writes expected registers with expected values:
drivers/iio/light/tsl2772.c tsl2772_read_prox_diodes()
(Both the above examples happen to be from the same subsystem but that should in no way be taken to imply that such issues are unique to that subsystem or that that subsystem has more of them.)
= How does this relate to kselftests?
Tests in kselftests also test kernel code using the userspace interfaces, but that's about what's common between the frameworks. kselftests has other goals and does not provide any kind of mechanism for hardware mocking.
= How does this relate to kunit?
Kunit is for unit testing of functions in kernel code, and is not meant for testing kernel code via userspace interfaces. It could in theory be used to test some of the simple drivers too, but that would require (1) a large amount of mocking code in various kernel frameworks, and, more importantly, (2) refactoring of the drivers to be tested.
This can be contrasted with roadtest which works with mostly unmodified drivers and which mocks the hardware at the lowest level without having to change kernel frameworks.
= How do I use it?
See Documentation/dev-tools/roadtest.rst added by the documentation patch for more information about running and writing tests using this framework.
= What's included in the patchset?
The current framework allows developing tests for hardware which uses the I2C bus. Hardware models can also control GPIOs and use them to trigger interrupts.
This series includes tests for some IIO, regulator and RTC drivers. The regulator and RTC tests depend on a few driver patches which are either in review or in linux-next. These are noted in the commit messages.
The entire patch set, including the required dependencies, is also available in a git tree:
https://github.com/vwax/linux/commits/roadtest/rfc-v1
Cc: linux-kernel@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-um@lists.infradead.org
Cc: shuah@kernel.org Cc: brendanhiggins@google.com Cc: linux-kselftest@vger.kernel.org
Cc: jic23@kernel.org Cc: linux-iio@vger.kernel.org
Cc: lgirdwood@gmail.com Cc: broonie@kernel.org
Cc: a.zummo@towertech.it Cc: alexandre.belloni@bootlin.com Cc: linux-rtc@vger.kernel.org
Cc: corbet@lwn.net Cc: linux-doc@vger.kernel.org
Vincent Whitchurch (10): roadtest: import libvhost-user from QEMU roadtest: add C backend roadtest: add framework roadtest: add base config roadtest: add build files roadtest: add documentation iio: light: opt3001: add roadtest iio: light: vcnl4000: add roadtest regulator: tps62864: add roadtest rtc: pcf8563: add roadtest
Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/roadtest.rst | 669 ++++ tools/testing/roadtest/.gitignore | 2 + tools/testing/roadtest/Dockerfile | 25 + tools/testing/roadtest/Makefile | 84 + tools/testing/roadtest/init.sh | 19 + tools/testing/roadtest/pyproject.toml | 10 + tools/testing/roadtest/requirements.txt | 4 + tools/testing/roadtest/roadtest/__init__.py | 2 + .../roadtest/roadtest/backend/__init__.py | 0 .../roadtest/roadtest/backend/backend.py | 32 + .../testing/roadtest/roadtest/backend/gpio.py | 111 + .../testing/roadtest/roadtest/backend/i2c.py | 123 + .../testing/roadtest/roadtest/backend/main.py | 13 + .../testing/roadtest/roadtest/backend/mock.py | 20 + .../roadtest/roadtest/backend/test_gpio.py | 98 + .../roadtest/roadtest/backend/test_i2c.py | 84 + .../testing/roadtest/roadtest/cmd/__init__.py | 0 tools/testing/roadtest/roadtest/cmd/main.py | 146 + tools/testing/roadtest/roadtest/cmd/remote.py | 48 + .../roadtest/roadtest/core/__init__.py | 0 .../testing/roadtest/roadtest/core/control.py | 52 + .../roadtest/roadtest/core/devicetree.py | 155 + .../roadtest/roadtest/core/hardware.py | 94 + tools/testing/roadtest/roadtest/core/log.py | 42 + .../testing/roadtest/roadtest/core/modules.py | 38 + .../testing/roadtest/roadtest/core/opslog.py | 35 + tools/testing/roadtest/roadtest/core/proxy.py | 48 + tools/testing/roadtest/roadtest/core/suite.py | 286 ++ tools/testing/roadtest/roadtest/core/sysfs.py | 77 + .../roadtest/roadtest/core/test_control.py | 35 + .../roadtest/roadtest/core/test_devicetree.py | 31 + .../roadtest/roadtest/core/test_hardware.py | 41 + .../roadtest/roadtest/core/test_log.py | 54 + .../roadtest/roadtest/core/test_opslog.py | 27 + .../roadtest/roadtest/tests/__init__.py | 0 .../roadtest/roadtest/tests/base/config | 84 + .../roadtest/roadtest/tests/iio/__init__.py | 0 .../roadtest/roadtest/tests/iio/config | 1 + .../roadtest/roadtest/tests/iio/iio.py | 112 + .../roadtest/tests/iio/light/__init__.py | 0 .../roadtest/roadtest/tests/iio/light/config | 2 + .../roadtest/tests/iio/light/test_opt3001.py | 95 + .../roadtest/tests/iio/light/test_vcnl4000.py | 132 + .../roadtest/tests/iio/light/test_vcnl4010.py | 282 ++ .../roadtest/tests/iio/light/test_vcnl4040.py | 104 + .../roadtest/tests/iio/light/test_vcnl4200.py | 96 + .../roadtest/tests/regulator/__init__.py | 0 .../roadtest/roadtest/tests/regulator/config | 4 + .../roadtest/tests/regulator/test_tps62864.py | 187 ++ .../roadtest/roadtest/tests/rtc/__init__.py | 0 .../roadtest/roadtest/tests/rtc/config | 1 + .../roadtest/roadtest/tests/rtc/rtc.py | 73 + .../roadtest/tests/rtc/test_pcf8563.py | 348 ++ tools/testing/roadtest/src/.gitignore | 1 + tools/testing/roadtest/src/backend.c | 884 +++++ .../src/libvhost-user/include/atomic.h | 310 ++ .../src/libvhost-user/libvhost-user.c | 2885 +++++++++++++++++ .../src/libvhost-user/libvhost-user.h | 691 ++++ 59 files changed, 8798 insertions(+) create mode 100644 Documentation/dev-tools/roadtest.rst create mode 100644 tools/testing/roadtest/.gitignore create mode 100644 tools/testing/roadtest/Dockerfile create mode 100644 tools/testing/roadtest/Makefile create mode 100755 tools/testing/roadtest/init.sh create mode 100644 tools/testing/roadtest/pyproject.toml create mode 100644 tools/testing/roadtest/requirements.txt create mode 100644 tools/testing/roadtest/roadtest/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/__init__.py create mode 100644 tools/testing/roadtest/roadtest/backend/backend.py create mode 100644 tools/testing/roadtest/roadtest/backend/gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/i2c.py create mode 100644 tools/testing/roadtest/roadtest/backend/main.py create mode 100644 tools/testing/roadtest/roadtest/backend/mock.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_gpio.py create mode 100644 tools/testing/roadtest/roadtest/backend/test_i2c.py create mode 100644 tools/testing/roadtest/roadtest/cmd/__init__.py create mode 100644 tools/testing/roadtest/roadtest/cmd/main.py create mode 100644 tools/testing/roadtest/roadtest/cmd/remote.py create mode 100644 tools/testing/roadtest/roadtest/core/__init__.py create mode 100644 tools/testing/roadtest/roadtest/core/control.py create mode 100644 tools/testing/roadtest/roadtest/core/devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/log.py create mode 100644 tools/testing/roadtest/roadtest/core/modules.py create mode 100644 tools/testing/roadtest/roadtest/core/opslog.py create mode 100644 tools/testing/roadtest/roadtest/core/proxy.py create mode 100644 tools/testing/roadtest/roadtest/core/suite.py create mode 100644 tools/testing/roadtest/roadtest/core/sysfs.py create mode 100644 tools/testing/roadtest/roadtest/core/test_control.py create mode 100644 tools/testing/roadtest/roadtest/core/test_devicetree.py create mode 100644 tools/testing/roadtest/roadtest/core/test_hardware.py create mode 100644 tools/testing/roadtest/roadtest/core/test_log.py create mode 100644 tools/testing/roadtest/roadtest/core/test_opslog.py create mode 100644 tools/testing/roadtest/roadtest/tests/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/base/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/iio.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/config create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_opt3001.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4000.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4010.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4040.py create mode 100644 tools/testing/roadtest/roadtest/tests/iio/light/test_vcnl4200.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/config create mode 100644 tools/testing/roadtest/roadtest/tests/regulator/test_tps62864.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/__init__.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/config create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/rtc.py create mode 100644 tools/testing/roadtest/roadtest/tests/rtc/test_pcf8563.py create mode 100644 tools/testing/roadtest/src/.gitignore create mode 100644 tools/testing/roadtest/src/backend.c create mode 100644 tools/testing/roadtest/src/libvhost-user/include/atomic.h create mode 100644 tools/testing/roadtest/src/libvhost-user/libvhost-user.c create mode 100644 tools/testing/roadtest/src/libvhost-user/libvhost-user.h
linux-kselftest-mirror@lists.linaro.org