On Mon, 18 Jan 2016 23:57:09 +0100 Alexander Couzens lynxis@fe80.eu wrote:
On Mon, 18 Jan 2016 18:38:29 +0000 Neil Williams codehelp@debian.org wrote:
You're a bit ahead of the docs at this point. ;-)
;)
Pipeline-only workers would need to be added on the command line currently:
$ sudo lava-server manage shell
from lava_scheduler_app.models import Worker new_worker = Worker.objects.create(hostname='host.domain') new_worker.save()
This should go into docs even it's just an intermediate thing.
This is a standard django shell operation, what we need is a helper.
Rather than changing the admin page, it might be better to provide a command line helper to do this - and the device assignment for that worker.
Now I got the worker, but after adding a job, the lava-server validates the yaml before sending it to the slave.
Correct. There is a submission schema check and there is a full validation of the pipeline. This underpins the descriptive information displayed in the UI for each job.
Is this the intended way? Shouldn't the slave validates the pipeline (calling validate() member functions of actions).
It does. The validation done on the master is for queue processing and scheduling. In order to properly queue and schedule the job, the master has to know that the job is valid before it can split the job into sub_jobs for protocols like multinode and do a bunch more checks on tags, submission privileges etc. There is no point waiting for that to happen on the slave when this can be checked before the job is queued. Instances often have a few dozen jobs in the queue for each device type, so the principle of "fail early" applies - invalid jobs need to be rejected from the queue.
Validation at each end allows the compatibility of each end to be verified as well, if the code on the slave is out of date, this will be picked up by the validation and declared to the master - potentially allowing the job to run on a different slave (that support is pending).
This also have some problems, like checking if a command is present using infrastructure_error() produces wrong results, because it doesn't matter if the file is present on the master it must be present on the client.
It does matter that the file is present on the master - so that the master can properly validate the job as that can include verifying that the commands to be used by the job are valid for those tools.
It's not as if having packages installed is any burden on the master, the master needs many times more capacity for the log files than it ever will for the base packages.