SPI and multi-threading issue

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Gus
Reply | Threaded
Open this post in threaded view
|

SPI and multi-threading issue

Gus
Hi,

I encountered an issue using the spidev driver.  

I used a Linux distribution patch with PREEMPT-RT. I created threads with "pthread_create()" and chosen a "SCHED_FIFO" scheduler policy. I also wrote some functions to determine the execution time of some portions of code.

I want read data from the SPI in a high priority (SIGRTMAX) thread #1 running at 1kHz, while a low priority (SIGRTMAX-1)  thread #2 with high CPU load is running at 100Hz. The SPI thread (#1) is woken up every 1ms using a POSIX interval timer (timer_settime) and this thread woke up the other thread (#2) by sending pthread_kill(th_id, SIGRTMAX-1) if needed.
The spidev device was opened with the following command:

[...]
SPIdev[deviceNum] = open(device[deviceNum], O_RDWR);
[...]

SPI data are read using the following commands:

THREAD #1:
        [...]
        // update SPI transfer structure for transfer
        transferSPI[deviceNum].tx_buf = (unsigned long)ToSend;
        transferSPI[deviceNum].rx_buf = (unsigned long)ToRead;
        transferSPI[deviceNum].len = MAX(NbToRead,NbToSend)*NB_BYTE_INT16;
       
        // send data
       tic("SPI_reception", Thread_ID);
        n_read = ioctl(SPIdev[deviceNum], SPI_IOC_MESSAGE(1), &transferSPI[deviceNum]);
       toc(Thread_ID);
       [...]

The second thread just consists of a long for loop (to simulate high CPU load):

THREAD #2:
      [...]
       tic("Step_1", Thread_ID);
        for (int i=0; i<NB_LOOP; i++){
             temp++;
        }
       toc(Thread_ID);
       [...]

When I run the code I obtained this:

---------------------------   // execution wihtout the thread #2
BEGIN Tic(): SPI_reception
END Toc(): SPI_reception. Elapsed time: 0.793000000 [ms]
BEGIN Tic(): StepBaseRate
END Toc(): StepBaseRate. Elapsed time: 0.061000000 [ms]
TASK 0: Previous Loop execution time: 0.946/1.000 [ms]
---------------------------  // execution with thread #2
BEGIN Tic(): SPI_reception
BEGIN Tic(): Step_1
END Toc(): Step_1. Elapsed time: 4.670000000 [ms]
TASK 1: Previous Loop execution time: 4.883/10.000 [ms]
END Toc(): SPI_reception. Elapsed time: 5.310000000 [ms]
BEGIN Tic(): StepBaseRate
END Toc(): StepBaseRate. Elapsed time: 0.061000000 [ms]
TASK 0: Previous Loop execution time: 5.463/1.000 [ms]

It looks like the "ioctl" function starts reading and, while is waiting for the entire data, the other pending thread is woken up and do not free the CPU when the reading is terminated. As a consequence, the thread #1 is overrunnig.

I did not have the problem when I used I2C device (with the "read" function to read the data):
---------------------------
BEGIN Tic(): I2C_reception
END Toc(): I2C_reception. Elapsed time: 1.403000000 [ms]
BEGIN Tic(): I2C_emission
END Toc(): I2C_emission. Elapsed time: 1.221000000 [ms]
TASK 0: Previous Loop execution time: 2.899/3.000 [ms]
---------------------------
BEGIN Tic(): I2C_reception
BEGIN Tic(): Step_1
END Toc(): I2C_reception. Elapsed time: 1.130000000 [ms]
BEGIN Tic(): I2C_emission
END Toc(): I2C_emission. Elapsed time: 0.916000000 [ms]
TASK 0: Previous Loop execution time: 2.258/3.000 [ms]
[...]
END Toc(): Step_1. Elapsed time: 4.818000000 [ms]

Do you have an idea how to make ioctl to preempt the CPU just after the reading is finished?

Thanks in advance,

Augustin
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Scott Ellis
There is another thread involved with SPIDEV, the SPI worker thread.

From spi.c : spi_init_queue()
    ...
    /*
     * Master config will indicate if this controller should run the
     * message pump with high (realtime) priority to reduce the transfer
     * latency on the bus by minimising the delay between a transfer
     * request and the scheduling of the message pump thread. Without this
     * setting the message pump thread will remain at default priority.
     */
     if (master->rt) {
         dev_info(&master->dev,
             "will run message pump with realtime priority\n");
         sched_setscheduler(master->kworker_task, SCHED_FIFO, &param);
     }
    ...

Something like this should enable RT priority for the SPI worker thread

diff --git a/drivers/spi/spi-omap2-mcspi.c b/drivers/spi/spi-omap2-mcspi.c
index b2fb141..283f382 100644
--- a/drivers/spi/spi-omap2-mcspi.c
+++ b/drivers/spi/spi-omap2-mcspi.c
@@ -1226,6 +1226,8 @@ static int __devinit omap2_mcspi_probe(struct platform_device *pdev)
        if (status || omap2_mcspi_master_setup(mcspi) < 0)
                goto disable_pm;

+       master->rt = 1;
+
        status = spi_register_master(master);
        if (status < 0)
                goto disable_pm;

Just an idea.
Gus
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Gus
Thanks Scott,

I will try this and tell if it work,

Augustin


2014-08-22 13:00 GMT+02:00 Scott Ellis <[hidden email]>:
There is another thread involved with SPIDEV, the SPI worker thread.

>From spi.c : spi_init_queue()


Something like this should enable RT priority for the SPI worker thread



Just an idea.



--
View this message in context: http://gumstix.8.x6.nabble.com/SPI-and-multi-threading-issue-tp4969428p4969434.html
Sent from the Gumstix mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Slashdot TV.
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users



--
Augustin MANECY

tél: 06 11 10 45 97
e-mail: [hidden email]

------------------------------------------------------------------------------
Slashdot TV.  
Video for Nerds.  Stuff that matters.
http://tv.slashdot.org/
_______________________________________________
gumstix-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gumstix-users
Gus
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Gus
In reply to this post by Scott Ellis
Hi Scott,

Sorry I'm new in OE, and I tried to apply the patch you sent to enable rt priority, but I failed. I obtained the following error when I tried to generate the image using bitbake:

ERROR: Command Error: exit status: 1  Output:
Applying patch rt-spi.patch
(Stripping trailing CRs from patch.)
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/drivers/spi/spi-omap2-mcspi.c b/drivers/spi/spi-omap2-mcspi.c
|index b2fb141..283f382 100644
|--- a/drivers/spi/spi-omap2-mcspi.c
|+++ b/drivers/spi/spi-omap2-mcspi.c
--------------------------
No file to patch.  Skipping patch.
patch unexpectedly ends in middle of line
1 out of 1 hunk ignored
Patch rt-spi.patch does not apply (enforce with -f)
ERROR: Function failed: patch_do_patch
ERROR: Logfile of failure stored in: /home/gus/yocto/poky-dylan/build/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/rt-spi/1.0.0-r0/temp/log.do_patch.65193
ERROR: Task 874 (/home/gus/yocto/poky-dylan/meta-pansenti/recipes-custom-user/rt-spi/rt-spi_1.0.0.bb, do_patch) failed with exit code '1'

It seems I didn't copy the patch in the right place. I suppose it is obvious but I do not know where I have to copy the patch in OE. I use your meta-pansenti data, and I cutomized your "pansenti-console-image" for my needs.

Thanks in advance,

Augustin
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Scott Ellis
Here are some instructions (using this spi rt change as an example)

http://www.jumpnowtek.com/overo/overo-kernel-development-simple-patching.html

From the booted system

overo login: root
root@overo:~# dmesg | grep spi
[    1.469177] spi_master spi1: will run message pump with realtime priority
[    1.481201] spi_master spi2: will run message pump with realtime priority
[    1.488769] spi_master spi3: will run message pump with realtime priority
[    1.496368] spi_master spi4: will run message pump with realtime priority
[    4.531738] ads7846 spi1.0: touchscreen, irq 210
[    4.558105] ads7846 spi1.0: no device detected, test read result was 0x00000FFF

I am not saying this is going to help with your original issue.

I made this change once for a Duovero SPI project, but I don't have the
numbers around for the change in performance. I remember there was
an impact on the USB subsystem though. It was a tradeoff. This change
might make the system worse with the single-core Overo.

It's just an idea and (I thought ;-) easy enough to try.

Gus
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Gus
Thanks a lot Scott!

This post was perfect. I successfully generated the new image with your patch. I don't know why I had nothing in my "<TMPDIR>/work/overo-poky-linux-gnueabi/linux-stable/3.5.X-r0/git" directory, in my previous build tree. I followed your instructions here to restart from zero, and it work perfectly. Now I understand better how OE works.

I will make some tests to check if the patch solved my original issue.

Thanks a lot again for your invaluable help!  
Gus
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Gus
Hi,

Thanks Scott for your patch, now I have the rt-priority activated for the spi worker thread, thanks again!

On one hand, the rt-priority solved the issue, so now the spi thread takes the CPU normally when needed.

On the other hand, when reading /writing a big amount of data, the SPI thread blocks the CPU for a long time. This is due to the spidev driver, which makes synchronous operations. So I modified the spidev driver to make it asynchronous, and now it works pretty well. To modify the driver, I followed the instructions on your website (writing a Linux spi driver).

I created a patch to modify spidev.c and spidev.h,  and now I'm facing an issue to install the new spidev.h file in my rootfs image.  The patch is well applied to the ${OVEROTOP}/tmp/.../git/include/linux/spi/spidev.h but is not applied to my final image, i.e. /usr/include/linux/spi/spidev.h
I'm sure there is a dummy solution for that, but I can not find it. I tried to add something like this to the linux recipe (linux-stable_3.5.7.bb), but it failed (saying that files already exist):

do_install_append() {
    do_install_append -m 0644 ${OVEROTOP}/tmp/work/linux-stable/3.5.7-r0/git/include/linux/spi/spidev.h ${includedir}/usr/include/linux/spi/spidev.h
}

PACKAGES =+ "kernel-headers"
FILES_kernel-headers = "${includedir}/usr/linux/spi/spidev.h"

What is wrong with that? What is the best practice to solve this problem?

Thanks in advance.

Gus
Reply | Threaded
Open this post in threaded view
|

Re: SPI and multi-threading issue

Gus
Sorry, I copied and pasted the wrong command.
Here is the one I tried:

do_install_append() {
    do_install -m 0644 ${OVEROTOP}/tmp/work/linux-stable/3.5.7-r0/git/include/linux/spi/spidev.h ${includedir}/usr/include/linux/spi/spidev.h
}

PACKAGES =+ "kernel-headers"
FILES_kernel-headers = "${includedir}/usr/linux/spi/spidev.h"