Read continuous stream of data over SPI

[nordev]

Hello,

I want to read a status register in my SPI peripheral. Once I send my read request, the peripheral will send the status byte over and over, and I want to watch for a change. When it changes, I want to terminate the SPI transfer. 

What is the best way to do this? The complication here is that there isn't a fixed RX buffer length. The status byte might change in 5 bytes or 100 or 1000. It seems suboptimal to allocate memory for a really long transfer buffer just to accommodate, say, the case in which it takes 1000 bytes for the status byte to change, both because of the extra memory, and also because there's no way to know if it'll even be enough. I am using this function:

ret_code_t nrf_drv_spi_transfer(nrf_drv_spi_t const * const p_instance,
uint8_t const * p_tx_buffer,
uint8_t tx_buffer_length,
uint8_t * p_rx_buffer,
uint8_t rx_buffer_length)

Is there a way I can push the RX bytes into a queue, maybe, pop from the queue to read the bytes (which makes room for more bytes to be clocked in on SPI), then abort the spi transfer when I notice the changed byte? If so, is there a way to make sure the dequeueing rate is faster than the spi transfer rate? Or can I read from the transfer buffer even if it is not full?

As a workaround, I guess I could TX my read request over and over and get back a few status bytes each time (buffer length = n+1, TX length = 1, RX = n), but it seems like this would defeat the purpose of SPI. 

Note: I think, based on the below screenshot, that I am using SPI and not SPIM.

Thanks!

[Einar Thorsrud]

Hi,

There are several ways to do this and I am not sure which is better. Using the SPI (and not SPIM) peripheral you will use the CPU for every byte, and you can inspect it. The driver is designed to handle this for you though, but you could for instance do a small hack and add your own callback function that is called from within the transfer_byte() function in nrfx_spi.c so that you can process each byte as it comes in, but still let the driver do it's work as normal.

[nordev]

Thank you Einar. One complication with using this method is that most of the time, I want to read a SPI buffer of known length, and only sometimes, I want to process each byte as it comes in in the manner described above.

I suppose I could create a duplicate transfer_byte() function, as well as a duplicate of all the functions that depend on transfer_byte(), such as spi_xfer(), nrf_drv_spi_xfer(), and nrf_drv_spi_transfer(), and simply call the modified functions (e.g., "my_nrf_drv_spi_transfer") when I need to. But two issues:

1. this still requires me to specify an RX buffer length, which I don't necessarily know in advance.

2. the below function also is dependent on transfer_byte(), but I am not really sure under what circumstances it is called. It seems somewhat redundant with the end of spi_xfer(), except that finish_transfer() is more thorough. Why does the end of spi_xfer() only set the chip select back high and not do the rest of the things in finish_transfer()? If I choose to modify the functions, is there any downside to calling finish_transfer() after while (transfer_byte(p_spi, p_cb)); inside spi_xfer()? 

static void irq_handler_spi(NRF_SPI_Type * p_spi, spi_control_block_t * p_cb)
{
    ASSERT(p_cb->handler);

    nrf_spi_event_clear(p_spi, NRF_SPI_EVENT_READY);
    NRF_LOG_DEBUG("SPI: Event: NRF_SPI_EVENT_READY.");

    if (!transfer_byte(p_spi, p_cb))
    {
        finish_transfer(p_cb);
    }
}

Do you have recommendations for a way to approach the continuous stream read that do not encounter the issues above? Thank you!

[Einar Thorsrud]

Hi,

nordev said:

1. this still requires me to specify an RX buffer length, which I don't necessarily know in advance.

Yes, that is the case. For the SPI peripheral that is not enforced by the HW (which just has double buffered Rx and Tx buffers), so you could write your own driver or modify the driver if that is a problem. Or perhaps simpler, just call nrfx_spi_abort() in the middle of a transfer if you find that you do not need to continue.

nordev said:

2. the below function also is dependent on transfer_byte(), but I am not really sure under what circumstances it is called.

Where transfer_byte() is called from depends on if using the driver in blocking mode or interrupt driven. If blocking, it is called from spi_xfer(). If non-blocking (which is probably what you will use in most cases), it is called from irq_handler(). In either case it is called when a READY event has occurred, which according to the product specification happens in this situation: "The SPI master will generate a READY event every time a new byte is moved to the RXD register.".

nordev said:

Why does the end of spi_xfer() only set the chip select back high and not do the rest of the things in finish_transfer()?

That is because it is a blocking function. So there is no callbck function to call etc.

nordev said:

If I choose to modify the functions, is there any downside to calling finish_transfer() after while (transfer_byte(p_spi, p_cb)); inside spi_xfer()? 

I am not sure why you want to do that? I think you first need to decide to use blocking or non blocking and also understand the driver if you want to make changes to it (other than hooking into transfer_byte() just to read as I suggested, as that does not change the driver behaviour in any way).

[nordev]

Einar Thorsrud said:

Where transfer_byte() is called from depends on if using the driver in blocking mode or interrupt driven. If blocking, it is called from spi_xfer()

I am using blocking mode, and my call stack does include spi_xfer()

I decided to handle this by just polling for the status byte over and over and receiving 1B back each time, instead of modifying the SDK to read a continuous stream.

But I seem to be hitting an infinite loop because this READY event you mention doesn't get generated sometimes:

Einar Thorsrud said:

In either case it is called when a READY event has occurred, which according to the product specification happens in this situation: "The SPI master will generate a READY event every time a new byte is moved to the RXD register.".

Here is my call stack. The highlighted line is where it is stuck. It just so happens here that a separate interrupt happened, so other code executed for a moment.

My code is like this:

uint8_t at_read_status_once(void)
{
    uint8_t spi_tx_cmd[1] = {CMD_READ_STATUS_1};
    uint8_t spi_rx_buf[2];	// 1 B tx, 1 B rx

    nrf_drv_spi_transfer(&m_spi_at, spi_tx_cmd, sizeof(spi_tx_cmd), spi_rx_buf, sizeof(spi_rx_buf));
    return spi_rx_buf[1];	// 2nd byte is the rx response
}

And I call it like this:

while(at_read_status_once() != 0x0);

My purpose is to stay in the while loop and not move onto the next instruction until the status byte turns to 0x0. This works sometimes, but sometimes, this gets stuck in the while loop in the SDK here:

Why is the READY event never getting generated?

[nordev]

As an update, I found out when specifically this "stuck in loop" problem seems to occur:

I have an RTC that generates a COMPARE event every X seconds, upon which it calls my at_read_status_once() fn from above, which TX data to the peripheral. The "stuck in loop" problem happens if the SPI TX command happens while SPI RX is in progress from the same peripheral. The SPI TX and SPI RX were initiated with two separate nrf_drv_spi_transfer(). In theory, SPI is supposed to allow duplex communication, so the simultaneous RX and TX should not be a problem; the issue is probably that at_read_status_once() expects a RX response, but there is already a separate RX in progress. 1) Do you have suggestions for how to mitigate this problem?

Somehow, UART interrupts are able to get through, as you can see from the screenshot above, but other interrupts don't. For example, because the bluetooth device is caught in this while loop, it isn't sending connection packets to the central, and the connection fails -- but no interrupt is generated to process the tasks inside ble_event_handler() case BLE_GAP_EVT_DISCONNECTED. 2) Why do you think some interrupts  are being generated, but others not? (note, RTC interrupt priority = 6. connection related events come from the softdevice, which i thought has the highest interrupt priority)

3) Can you tell me the chain of events that causes the "READY" event to get generated?

[Einar Thorsrud]

Hi,

nordev said:

1) Do you have suggestions for how to mitigate this problem?

SPI is always full duplex. So whenever you transmit x bytes, you also receive x bytes. This is why you always need to provide both an Rx and Tx data buffer in the driver. But you cannot start two transfers at the same time, that will cause problems. Unfortunately the driver does not have a sanity check for this, so it is your responsibility to wait for the first transaction to finish before starting the next. Failing to do so will lead to problems so this should be fixed before anything else.

nordev said:

2) Why do you think some interrupts  are being generated, but others not?

The typical reason for not getting some events is if the CPU is doing some work in the same or higher interrupt priority. I assume that is the case here? Regarding BLE the SoftDevice use two interrupt priorities (0 for mostly for internal usage and 4 for events to the application etc.) If you use the app scheduler and schedule SoftDevice events, then those would be processed in the main loop, so any code executing in an interrupt would block it (that is the case if using scheduler and having configured NRF_SDH_DISPATCH_MODEL to 1 in sdk_config.h).

nordev said:

3) Can you tell me the chain of events that causes the "READY" event to get generated?

The short description is that the READY event every time a new byte is moved to the RXD register. For more details I suggest you refer to the SPI master transaction sequence section in the SPI chapter in the product specification.

[nordev]

Einar Thorsrud said:

it is your responsibility to wait for the first transaction to finish before starting the next

Option 1:The RTC interrupt ultimately triggers the second transfer. Is there a way to temporarily disable the RTC interrupt, such that the event that should trigger the interrupt is registered (an interrupt flag is set, perhaps), but doesn't actually trigger the interrupt until after interrupts are enabled again? In case a RTC interrupt should happen in the middle of an existing SPI transfer, what I would like is for RTC interrupts to be disabled while a transfer is in progress (i.e. while chip select is low) and re-enabled when the transfer completes (chip select goes high), such that any interrupt that tries to happen in the middle of a transfer instead happens immediately after the transfer completes.

Option 2: Is there a way to check if the SPI resource in question is in use? I could place a condition on the SPI transfer called by the RTC interrupt callback so that it only executes if the SPI resource is free and waits otherwise. This would have the same effect as #1 and probably be more elegant.

Also, just to clarify, is this sentence from your link from an outdated specification? "The SPI master does not implement support for chip select directly." I ask because I see that spi_config() includes a way to designate the chip select pin, and the spi driver itself seems to perform operations with this designated pin -- in fact, I rely on the drivers and don't set/clear chip select directly. I am using SDK 14.2.

Einar Thorsrud said:

If you use the app scheduler and schedule SoftDevice events, then those would be processed in the main loop, so any code executing in an interrupt would block it

How do I know if I am using the scheduler? Above I mention two SPI transfers, one initiated by the RTC ("type 1"), and the other initiated by a request by the central to a BLE characteristic ("type 2"). Are you saying that the disconnection event doesn't get processed when it occurs in the middle of the type 2 SPI transfer because BLE disconnect constitutes a SoftDevice interrupt, and the type 2 SPI transfer is already operating in a SoftDevice interrupt? So it seems that you can't interrupt an interrupt with another interrupt of the same kind. But other events, like UART interrupts and the aforementioned RTC interrupt (even though it has a low interrupt priority of 6) are able to execute simply because they are NOT operating in a SoftDevice interrupt? So the SoftDevice doesn't handle things like UART or RTC? And interrupting another interrupt is ok, so long as the second interrupt is from a different source?

Thank you

[Einar Thorsrud]

Hi,

First of all, I was a bit surprised when I checked the driver implementation yesterday and did not find a check for not starting a second transaction before the first finished. I turns out I glanced over it too quickly, though. You should get NRF_ERROR_BUSY if attempting to start a second transaction before the first has finished. So that means you can use this. Sorry for the confusion.

Regarding chip select / slave select it is correct that the SPI peripheral does not handle that. However, the driver implements support for chip select by controlling a GPIO. So when you use the driver you get it handled for you.

nordev said:

How do I know if I am using the scheduler?

Do you call app_sched_execute() from your code? If yes, you are using the scheduler, if not you are not using it. If you also set NRF_SDH_DISPATCH_MODEL to 1 in your sdk_config.h BLE events are using the scheduler (meaning put in the queue and processed when you call app_sched_execute()). As you are not aware of it I suspect you are not using it.

Regarding the rest of the section about interrupts it is actually quite simple. To be specific:

• Any interrupt will be processed immediately as long as it is enabled and no same or higher priority interrupt is currently being serviced.
  • If an interrupt of same or higher priority is being services, the new interrupt will be serviced immediately after the previous is finished (assuming no other higher priority interrupt occurs in the meantime).
  • A interrupt routine that is interrupted by a higher priority interrupt will continue once the higher priority interrupt routine has finished.

Now, there are SW concepts that can be used to make some things easier. The app_scheduler can be used to queue events from any interrupt priority. Then you process the queue in your main loop, so it is not in an interrupt (so basically lowest possible priority). That is useful sometimes, particularly in more complex applications.

Regarding SoftDevice that uses some peripherals that generate interrupts, but you will from the application only see SoftDevice interrupts that you are supposed to handle (in form of SoftDevice events). Interrupts from other peripherals (like for UART and RTC instances you are using) are forwarded to the application directly (it is actually done by the SoftDevice with slightly increases the interrupt latency, but other then that you can think of it as if the SoftDevice plays no role in it).

nordev said:

And interrupting another interrupt is ok, so long as the second interrupt is from a different source?

If you get a lot of the same interrupts before you can process them with the same IRQ number (same source), that will cause you to lose interrupts. This is one of the reasons you should keep interrupts routines short. Other then that, interrupt from the same source cannot preempt another from the same source, as they will always have the same priority. But an interrupt with higher priority then one currently being serviced will interrupt that one.

[nordev]

Thank you Einar. 

Einar Thorsrud said:

You should get NRF_ERROR_BUSY if attempting to start a second transaction before the first has finished.

is the right way to do this APP_ERROR_CHECK()?

The strange thing about checking this way, however, is that I have to actually call nrf_drv_spi_transfer() in order to get the error code. Either with APP_ERROR_CHECK() or something like “if(nrf_drv_spi_transfer() != NRF_ERROR_BUSY)”. 

So sure I might get an error code, but the damage will already have been done: the second transfer will have already disrupted the first transfer, which is what gets me stuck in the infinite while loop. 

is there some way to check without actually calling the SPI transfer function to see if it generates an error?

Einar Thorsrud said:

Do you call app_sched_execute() from your code?

no, I do not. are there other benefits of bundling the BLE event handling like this besides being able to queue interrupts & not lose them?

[Einar Thorsrud]

Hi,

nordev said:

is the right way to do this APP_ERROR_CHECK()?

No. APP_ERROR_CHECK() is used to catch unexpected errors. The default error handler in the SDK will log and break if an error is detected with this and in debug mode. If in release mode it will perform a soft reset - in order to recover from an unexpected and unhandled error. The correct is to handle any expected errors specifically, and then call APP_ERROR_CHECK() if the return value was not an expected/handled error, like you suggest with "if(nrf_drv_spi_transfer() != NRF_ERROR_BUSY)".

nordev said:

So sure I might get an error code, but the damage will already have been done: the second transfer will have already disrupted the first transfer, which is what gets me stuck in the infinite while loop. 

This stems from my incorrect previous answer, I guess. The driver checks if a transaction is ongoing before doing any changes. So a new transaction would not be able to cause problems for an old, but should just result in the second call failing with NRF_ERROR_BUSY. There is a slight possibility for an issue here though, as this is not implemented in thread safe way. So if the two calls are made at the same time with different priorities then it is theoretically possible that the second call with higher priority would interrupt the first after the transfer_in_progress was read but before it is written to. And then you would have problems. This would be solved by always calling from the same priority, or by using a proper check externally (using a mutex of similar).

nordev said:

is there some way to check without actually calling the SPI transfer function to see if it generates an error?

There is no API for that in the driver.

To sum up, if you call from the same priority, you can just rely on handling the NRF_ERROR_BUSY. If not, you should implement your one mutex on the outside. So that one "user" will lock the driver before starting and operation and unlock it when finished. And the same with the second. You could use nrf_atomic_u32_t for that, or just a volatile variable that you read and update in a critical region (CRITICAL_REGION_ENTER() / CRITICAL_REGION_EXIT()).

nordev said:

are there other benefits of bundling the BLE event handling like this besides being able to queue interrupts & not lose them?

There are no extraordinary benefits in the context of BLE events or even nRF5 SDK specifically. It boils down to general concepts of handling interrupts / priorities. If you have a need to move processing of events/interrupts (BLE or other) down from interrupt context to main context, then using the scheduler makes sense. If not, then there is no need. Typically the more complex an application is and the more different interrupts etc or the longer the interrupt routines, the more important it becomes to move priorities down. That way stuff that can wait may wait, and there is time to process other interrupts in in a timely manner. But there is no right or wrong way here, it is application dependent. This is related to the general importance of keeping ISRs short. If you want to do a lot of work because of an interrupt, but only a small portion of it is time critical you can do the small portion in the ISR, and then queue the rest in the scheduler to be processed in the main context.

[nordev]

Einar Thorsrud said:

So a new transaction would not be able to cause problems for an old, but should just result in the second call failing with NRF_ERROR_BUSY.

I don't know if this is actually the case, and I think the following concern still stands:

Einar Thorsrud said:

So sure I might get an error code, but the damage will already have been done: the second transfer will have already disrupted the first transfer, which is what gets me stuck in the infinite while loop. 

Because I changed my code in this function (code in a previous response):

nordev said:

at_read_status_once()

to do this: 

while(nrf_drv_spi_transfer(&m_spi_at, spi_tx_cmd, sizeof(spi_tx_cmd), spi_rx_buf, sizeof(spi_rx_buf)) == NRF_ERROR_BUSY);

so that if I got a NRF_ERROR_BUSY event it would just try again until it isn't busy. But I still get stuck in this while loop forever inside spi_xfer(): 

Here is the full stack. I have highlighted the two calls to nrf_drv_spi_transfer(). 

The first one is called from my main loop, and the second one is called from an RTC interrupt. My RTC is also a low interrupt priority, 6. 

Note that NRF_ERROR_BUSY should be returned from nrf_drv_spi_xfer (second in the call stack above). But this does not happen -- "p_cb->transfer_in_progress" seems to be false, even though there IS a transfer in progress (first highlight on above stack):

What do you think is happening? Is this a bug with SDK 14.2? Is it fixed in later versions? 

-----

As a separate matter, I had to refresh this webpage 5 times to get the "reply" button to show up at the bottom of your latest message (it showed up at the top underneath my original message but not elsewhere). Generally, every time I use the forum, I have to refresh multiple times in order for the reply button to show up. Perhaps you can relay this to your web development team.