Latency of Raspberry Pi 3 on Standard and Real-Time Linux 4.9 Kernel

Introduction

Cyclic test program provided by Real-Time Linux project provides an easy way to assess the maximum latency of a system. I did a basic experiment measuring the cyclic latency of Raspberry Pi 3 on Standard and Real-Time Linux Kernel.

Cyclic test basically measures how much it takes to respond to an interrupt, and in the case of cyclic test program, the interrupt is generated by the timer. On a real-time system, in other words in a system where interaction with the external world has to be predictable, the interrupt response latency, specifically the distribution or maximum latency is important. Imagine a possibly embedded computing system periodically sampling a fast changing real world quantity by some sensors and responding to it by actuators, it is not acceptable or even dangerous if some data is missed or the action is not performed on time.

Setup

I am using a Raspberry Pi 3 Model B which has a 4-core ARM processor and 1GB RAM.

For the cyclic test program, I simply cloned the rt-tests repo here: rt-tests/rt-tests.git.

and checked out the stable/v1.0 branch and build the project.

The plots below are generated with gnuplot, using the mklatencyplot script of OSADL. The cyclic test is run with the following parameters:

cyclictest -l100000000 -m -S -p90 -i200 -h400 -q

• -l100000000: 100M iterations
• -m: lock current and future memory allocations to prevent being paged out
• -S: SMP support, equivalent to -t -a -n, use clock_nanosleep, create as much thread as number of cores and run these threads on the same core (i.e. set processor affinity). So in this example, we run 4 threads.
• -p90: priority of first thread (is 90). Others are one less, so 90, 89, 88, 87.
• -i200: interval for the first thread (in us).
• -h400: dump histogram for max latency up to 400us.
• -q: print a summary on exit.

See this presentation for more information about the cyclic test program.

A single run of this test takes more than 5 hours.

No additional load was present while running the test.

Test Result 1

First test is with Stock Kernel on Raspberry Pi 3, running the latest (September 2017) Raspbian Stretch Lite image with kernel 4.9.41-v7+.

As we see above, the maximum latency was almost 500us, and there are considerable number of samples between 100us and 150us. The majority of samples are under 50us. Please note y axis has log scale.

Test Result 2

Second test is with Raspberry Pi 3 running a custom kernel build 4.9.47-rt37-v7+ . This is basically kernel 4.9.47 with Real Time patch (PREEMPT_RT patch) applied. Other than Fully Preemptible Kernel (RT), nothing is changed in the Kernel configuration.

At the moment, there is a problem running Real-Time Linux on Raspberry Pi, more info here. Running the cyclic test program above causes instability after a few minutes. So I have applied this patch, and there was no problem.

As we see above, the maximum latency is around 90us, and most of the samples are below 50us.

OSADL provides the latency results for many devices including a Raspberry Pi 2, and the result is here. It is different than above but the device and kernel is different as well.

Conclusion

Not surprisingly Real-Time Linux Kernel decreases the maximum latency of the system. In general, benchmarking a real-time system is not a straightforward task and the results are open to interpretation. This article is meant to just show the result of this basic experiment.