Hi,

Good progress that the adaptive tickless (CONFIG_NO_HZ_FULL) feature have made it to the kernel 3.10. But there's still work to be done, since (according to the LWN article) it is not fully tickless even when there's only one thread on a core. If kernel is still running a 1 HZ tick, it saves cycles but does not improve real-time properties of the system. The worst case response time of a core is still limited to the longest kernel maintenance task it may end up running during a tick (is there an upper limit?). Also throughput will suffer from one or more cores running maintenance and not processing packets. It adds jitter and makes it harder to guarantee maximum throughput figures.

Data plane cores would need to pile up maintenance work and process it only when told so ( application calling yield or some system calls).

About the multithread support, there would be only one truely tickless thread per core. If there are others, time slicing is probably needed, at least as a  backup (swap out data plane thread if it is not yielding).

- Petri





On 20 May 2013 22:38, Ola Liljedahl <Ola.Liljedahl@arm.com> wrote:
Good article at LWN (well they are always good) referenced by the Linaro page you link to. The final comment is correct but not the complete answer. We also need determinism and no-interference for other reasons.

Adaptive tick-less should also be useful when you have multiple threads on the same core but are not using any type of time-sliced scheduling. Threads either wake up each other or are woken up by interrupts (could be the hrtimer with a thread-specific timeout).

-- Ola

Ola Liljedahl, Networking System Architect, ARM
Telephone: +46 706 866 373    Skype: ola.liljedahl


-----Original Message-----
From: Amit Kucheria [mailto:amit.kucheria@linaro.org]
Sent: 20 May 2013 14:49
To: Ola Liljedahl
Cc: Juri Lelli; Mark Orvek; Patrick MacCartee; Mike Holmes; linaro-networking; Lists linaro-dev; linaro-enterprise@linaro.org
Subject: Re: Deadline scheduler inclusion in linux-linaro?

Please refer to my earlier comment about Adaptive Tickless. I think
you'll find it useful for the usecase you outlined below.

On Mon, May 20, 2013 at 6:07 PM, Ola Liljedahl <Ola.Liljedahl@arm.com> wrote:
> Interesting. But SCHED_DEADLINE doesn't depend on the PREEMPT_RT modifications? Are these solutions overlapping in some areas?
>
> For the relevant use cases, I don't know if strict CPU allocation (a fraction of the total CPU cycles)  is needed. Instead these apps (we are talking legacy SW) are designed so that specific threads must start to execute very quickly (within 5-10 microseconds with an almost 100% guarantee, this is also a bound of the overhead of the wakeup and scheduling operations) when some event occurs (timer or possibly some other type of interrupt). The biggest problem to this today in Linux is the kernel itself (e.g. interrupt disabling, locks). How does SCHED_DEADLINE handle this?
>
> -- Ola
>
> Ola Liljedahl, Networking System Architect, ARM
> Telephone: +46 706 866 373    Skype: ola.liljedahl
>
>
> -----Original Message-----
> From: Juri Lelli [mailto:juri.lelli@gmail.com]
> Sent: 20 May 2013 14:19
> To: Ola Liljedahl
> Cc: Amit Kucheria; Mark Orvek; Patrick MacCartee; Mike Holmes; linaro-networking; Lists linaro-dev; linaro-enterprise@linaro.org
> Subject: Re: Deadline scheduler inclusion in linux-linaro?
>
> On 05/20/2013 12:27 PM, Ola Liljedahl wrote:
>> Predictable response times for specific threads. In the use cases I
>> am aware of, these threads would probably belong to the same process
>> and run core-affine on a specific core where as few as possible other
>> (kernel and user) threads and interrupts etc would also run. A number
>> of cores would follow this real-time model while other cores
>> preferably would keep the original Linux scheduling and
>> characteristics (to avoid introducing unexpected behaviour).
>>
>> What benefits does SCHED_DEADLINE give in addition to stuff in
>> PREEMPT_RT?
>>
>
> Predictable response times are what SCHED_DEADLINE is about.
>
> With SCHED_DEADLINE you can provide temporal guarantees up to 100% CPU
> utilization that are not possibile with default real-time policies
> (e.g. SCHED_FIFO/RR). This comes from the fact that SCHED_DEADLINE
> achieves strong temporal isolation between tasks. You can reserve a
> fraction of CPU time to your activities and be assured that no one else
> can interfere with them. And this is critical also in a low loaded
> system, if you want to be really safe against unexpected interferences.
>
> Basically, with SCHED_DEADLINE you can make a feasibility study (no
> deadline will be missed) of the system under development beforehand,
> and be sure, at run-time, that the timing requirements will be met
> under any circumstance. You can't do the same using only stuff already
> in PREEMPT_RT.
>
> Best,
>
> - Juri
>
>> -- Ola
>>
>> Ola Liljedahl, Networking System Architect, ARM Telephone: +46 706
>> 866 373    Skype: ola.liljedahl
>>
>> -----Original Message----- From: Amit Kucheria
>> [mailto:amit.kucheria@linaro.org] Sent: 20 May 2013 12:16 To: Ola
>> Liljedahl Cc: Mark Orvek; Patrick MacCartee; Mike Holmes;
>> linaro-networking; Lists linaro-dev; j.lelli@sssup.it;
>> linaro-enterprise@linaro.org; lng-sc@linaro.org Subject: Re: Deadline
>> scheduler inclusion in linux-linaro?
>>
>> Ola,
>>
>> Are you concerned with lower system-wide scheduling latency or with
>> predictable response times for a particular task?
>>
>> I believe SCHED_DEADLINE with help with the later.
>>
>> /Amit
>>
>> On Mon, May 20, 2013 at 1:10 PM, Ola Liljedahl
>> <Ola.Liljedahl@arm.com> wrote:
>>> In my experience, networking is moving away from realtime
>>> scheduling (and thus RTOS’es). There are some lingering
>>> requirements for some of the processing in the wireless stack but
>>> the main difficulty here is getting the scheduling latency down to
>>> acceptable levels, e.g. five microseconds or so. Will
>>> SCHED_DEADLINE help here?
>>>
>>>
>>>
>>> -- Ola
>>>
>>>
>>>
>>> Ola Liljedahl, Networking System Architect, ARM
>>>
>>> Telephone: +46 706 866 373    Skype: ola.liljedahl
>>>
>>>
>>>
>>> From: Amit Kucheria [mailto:amit.kucheria@linaro.org] Sent: 19 May
>>> 2013 11:51 To: Mark Orvek Cc: Patrick MacCartee; Mike Holmes;
>>> linaro-networking; Lists linaro-dev; j.lelli@sssup.it;
>>> linaro-enterprise@linaro.org; lng-sc@linaro.org Subject: Re:
>>> Deadline scheduler inclusion in linux-linaro?
>>>
>>>
>>>
>>> Similarly, the SCHED_DEADLINE patches shouldn't affect default
>>> runtime scheduler behaviour unless a task uses the DEADLINE
>>> policy.
>>>
>>>
>>>
>>> However, I haven't studied the intersection of the Preempt RT and
>>> SCHED_DEADLINE patches in source form yet. If they touch common
>>> pieces of code, merging both in might be an ongoing effort. Juri,
>>> do you know?
>>>
>>>
>>>
>>> On Fri, May 17, 2013 at 8:11 PM, Mark Orvek <mark.orvek@linaro.org>
>>> wrote:
>>>
>>> The PREEMPT_RT patchset is configurable.  I believe the default is
>>> PREEMPT_DESKTOP which is what most MV CGE customers use.  Another
>>> config options is PREEMPT_NONE but I believe its usage is rare.
>>>
>>>
>>>
>>> Mark
>>>
>>>
>>>
>>> On Fri, May 17, 2013 at 7:25 AM, Patrick MacCartee
>>> <pmaccartee@mvista.com> wrote:
>>>
>>> Will these be added in a way that makes them easy to remove? Many,
>>> >95% don't use Preempt RT in Linux as it impacts network
>>> performance and makes things very temperamental.  You would think
>>> people would just disable this RT, but when trying to isolate
>>> issues it adds another variable to the mix. I believe Yocto has a
>>> way of adding and removing RT patches that is some what straight
>>> forward and preferable based on feedback from OEM's.
>>>
>>>
>>>
>>> Just a thought,
>>>
>>> Patrick
>>>
>>>
>>>
>>> On Fri, May 17, 2013 at 6:34 AM, Mike Holmes
>>> <mike.holmes@linaro.org> wrote:
>>>
>>> In LNG you could end up with an interesting mix of preempt RT and
>>> deadline patches making the analysis and benchmarking interesting
>>> to interpret. In addition there are discussions about adding zero
>>> overhead linux (ZOL) like features.
>>>
>>> Mike
>>>
>>>
>>>
>>>
>>> On Friday, May 17, 2013 6:08:20 AM UTC-4, David Rusling wrote:
>>>
>>> Amit, an interesting proposal.  I think that most of the LNG
>>> steering committee is on this alias, but just in case, I'm adding
>>> them to it... Dave
>>>
>>> Amit Kucheria
>>>
>>> 17 May 2013 10:15
>>>
>>> Hi all,
>>>
>>> As part of our investigations into the Linux scheduler we've
>>> interacted with Juri Lelli at the University of Pisa (cc'ed) who
>>> is part of a group that is working on a DEADLINE scheduler[1] for
>>> Linux[2].
>>>
>>> While we're coming at this from a power managment angle[3], I
>>> suspect that LEG and LNG already have real-world usecases that
>>> would benefit from deadline scheduler found in other RTOSes.
>>>
>>> So I think it makes sense to merge Juri's tree into linux-linaro
>>> going forward to allow easier experimentation. Does LEG and LNG
>>> have any interest in this at this point?
>>>
>>> Juri has expressed an interest in maintaining a current branch of
>>> the code that could be merged into our monthly release. In return,
>>> real world usecases will improve his chances of getting the code
>>> merged into mainline.
>>>
>>> Regards, Amit
>>>
>>> [1] http://retis.sssup.it/?q=node/35 [2]
>>> https://lkml.org/lkml/2013/2/11/373 [3] Mostly involving
>>> discussions at this point, no real engineering effort invested yet
>>>
>>> _______________________________________________ linaro-dev mailing
>>> list
>>>
>>> linar...@lists.linaro.org
>>> http://lists.linaro.org/mailman/listinfo/linaro-dev
>>>
>>>
>>>
>>> -- David Rusling CTO Linaro Ltd
>>>
>>> e.  david....@linaro.org
>>>
>>>
>>> w.  http://www.linaro.org Linaro: The future of Linux on ARM
>>>
>>>
>>>
>>>
>>>
>>> -- Patrick J. MacCartee Director of Product Management MontaVista
>>> Software LLC fone: 408-572-7937 mobile: 415-637-0257
>>> pmaccartee@mvista.com
>>>
>>>
>>>
>>>
>>>
>>> --
>>>
>>> Mark Orvek
>>>
>>>
>>>
>>> mark.orvek@linaro.org
>>>
>>>
>>>
>>> VP, Engineering
>>>
>>>
>>>
>>> M: +1.408.313.6988 IRC: morvek Skype: morvek
>>>
>>> linaro.org │ Open source software for ARM SoCs
>>>
>>>
>>>
>>>
>>>
>>>
>>> -- IMPORTANT NOTICE: The contents of this email and any attachments
>>> are confidential and may also be privileged. If you are not the
>>> intended recipient, please notify the sender immediately and do not
>>> disclose the contents to any other person, use it for any purpose,
>>> or store or copy the information in any medium. Thank you.
>>
>>
>> -- IMPORTANT NOTICE: The contents of this email and any attachments
>> are confidential and may also be privileged. If you are not the
>> intended recipient, please notify the sender immediately and do not
>> disclose the contents to any other person, use it for any purpose, or
>> store or copy the information in any medium.  Thank you.
>>
>
>
> -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.


-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.

--
You received this message because you are subscribed to the Google Groups "linaro-networking@linaro.org" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linaro-networking+unsubscribe@linaro.org.
To post to this group, send email to linaro-networking@linaro.org.