NRFX SAADC in Zephyr fast and slow interleaved sampling

[Farhang]

Hi Nordic team,

I have successfully developed an application using nrfx (Timer/PPI/SAADC) combo in NCS 1.5.x to be able to sample 2x differential channels very fast.

Similar to the method advised here

I have a few questions:

Since using advanced mode and PPI,  everything is set to work automatically, very fast (16K sample per sec) for 2x differential channels, and 4x oversample, adding more channels is impossible w/o lowering the overall sample rate.

1- How do I add other channels that I don't need to be sampled very fast? e.g. a few times ( < 20sps) per second. 

does this mean I  have to stop PPI and uninit and reinit  saadc to get the samples for low freq channels, then uninit reinit for the fast sampling? and switch between the two operations constantly? if this is the only way, can I do this in an interrupt context that is initiated by the timer? hence keeping it in sync with the timer/ppi/saadc combo.

Or is there a better way I cannot think of?

Unfortunately I cannot sample the high frequency channels less than 16ksps, so lowering that is not an option.

2- Hardware question: What are the noise/performance figures for the GAIN amplifier inside the nRF52840 SAADC for different gain/attenuate settings? 
Basically we do have an AGC amplifier circuit that feeds those high frequency channels into nRF ADC pins, therefore we're curious to see what gain or attenuate setting inside SAADC results in best performance, and hence adjust our external AGC accordingly.

Thanks,

[Simon]

Since using advanced mode and PPI,  everything is set to work automatically, very fast (16K sample per sec) for 2x differential channels, and 4x oversample, adding more channels is impossible w/o lowering the overall sample rate.

 As mentioned in nRF52840 Product Specification - SAADC - Oversampling, it is not possible to use more than one channel when using oversampling:

"Note: Oversampling should only be used when a single input channel is enabled, as averaging is performed over all enabled channels."

One way of going about this, is to only have one channel enabled at the time, and when you're done oversampling on that channel you reconfigure the SAADC to use the next channel. As Kenneth suggests in this ticket: https://devzone.nordicsemi.com/f/nordic-q-a/71704/right-way-to-oversample-burst-multiple-saadc-input-pins.

You could probably also disable oversampling and sample over several channels and do the averaging in software.

1- How do I add other channels that I don't need to be sampled very fast? e.g. a few times ( < 20sps) per second. 

does this mean I  have to stop PPI and uninit and reinit  saadc to get the samples for low freq channels, then uninit reinit for the fast sampling? and switch between the two operations constantly? if this is the only way, can I do this in an interrupt context that is initiated by the timer? hence keeping it in sync with the timer/ppi/saadc combo.

 "does this mean I have to stop PPI and unitit and reinit saadc...". Yes, this would be the way to go about this, since there is no way of configuring separate sampling rates for separate channels. I will get back to you next week about the specifics of the implementation.

2- Hardware question: What are the noise/performance figures for the GAIN amplifier inside the nRF52840 SAADC for different gain/attenuate settings? 
Basically we do have an AGC amplifier circuit that feeds those high frequency channels into nRF ADC pins, therefore we're curious to see what gain or attenuate setting inside SAADC results in best performance, and hence adjust our external AGC accordingly.

 I will look into this next week as well.

Best regards,

Simon

[Farhang]

Simon Thanks for this info.

I have been facing a new challenge lately with nrfx/ncs where at high sampling rate, very randomly/rarely, the data buffer in RAM would be misaligned.

e.g. if i have 2 channels configured, and doing sampling via Timer/PPI/nrfx_saadc, sometimes instead of the order of channels being, ch1, ch2, ch1, ch2 etc in "RESULT BUFFER" I get ch2,ch1, ch2, ch1.. I can tell when this happens usinga a breakpoint, because I intentionally have ch1 connected to positive potential (differential measurement, V_ch1+ > V_ch1- ) and ch2 is negative. 

Could this be related to interrupt handling delays in zephyr?

I'm sampling at 22KHz which is ~ 44us timer tick that send the SAMPLE TASK to saadc (3us sampling).

My guess is there is a race condition at nrfx_saadc interrupt handling around when EVT_BUF_REQ happens.

e.g. If an extra timer tick happens when the new buffer is being handed over to nrfx/saadc, when the previous buffer has ENDed what is the expected behavior? This isn't explained very clearly in the datasheet of nrf52840.

I increased the IRQ priority of nrfx_saadc from 7 to 0. The problem seems to have gone away, but I'm doing long term testing to see if it ever happens rarely.

another question: Do I need to use IRQ_DIRECT_CONNECT? I'm currently using IRQ_CONNECT. I've read that DIRECT makes the IRQ handling faster potentially.

I have also disabled start_on_end int the nrfx_saadc config, so I can take control of START TASK, I stop the timer after EVT_DONE, to ensure no extra SAMPLE TASKS are triggered to saadc one it is done. Then I send the start task and wait for BUF_EVT and then the timer starts.

If there are any relevant posts that you are aware of please share.

I can share my code if needed in a PM.

Thanks

[Simon]

Farhang said:

I can share my code if needed in a PM.

Yes, this would be nice. 

[Simon]

Hello Farhang, sorry for the delay on this. I got the code from you in PM, but I forgot about it. Since this ticket is in not in my queue of cases (because it's in a waiting state, since I answered last and waited for a reply). My apologies for that. I will test your project the next days.

Best regards,

Simon

[Simon]

Apologies for the late reply, a huge part of the support team are on vacation and you may experience delayed answers.

I was able to reproduce your issue. By setting the differential input of the first channel to positive and vice versa with the second channel. After running it for a while I saw that the check check_ch1_pos_ch2_neg() generated an error. I'm not entirely sure what causing this. I'll try to get a better grasp about it tomorrow, and ask some of my colleagues for possible causes.

It seems like you potentially found a solution:

Farhang said:

I increased the IRQ priority of nrfx_saadc from 7 to 0. The problem seems to have gone away, but I'm doing long term testing to see if it ever happens rarely.

 However, I guess it would be nice to understand the root cause.

[Farhang]

Simon

Thanks for looking into this, 

It seems like I have managed to solve the problem.. Though I am not entirely sure, so it would be really nice to get your team's feedback.

So it seems like since I am triggering the SAADC via PPI via a Timer at 44us, when we hit the EVT_DONE/EVT_BUF_REQ, the IRQ response is not fast enough through Zephyr,

If that is indeed the case, the next SAMPLE TASK could be triggered BEFORE the IRQ has dealt with either of the two events above, the Saadc could be in "end" mode (i.e. not Started), in which case I'd like to know what is the expected behavior of the saadc/ nrfx_saadc driver.

If that is the case, I thought there would be 2 solutions:

1- Increase the IRQ priority of the saadc IRQ and perhaps use IRQ_DIRECT_CONNECT(), I could not find enough info about this, and how to increase the IRQ priority via .dts or .overlay file. I thought I increased it by passing in a lower priority to nrfx_saadc_init() but in Zephyr/nrfx lib this actually doesn't do anything.

Can you please share any info on changing IRQ priorities of peripherals via Zephyr?

2- This solution worked:

Added another Timer (timer 2) and set it to Counter mode. Using this counter I count the number of times SAADC Sample Task has been hit via the PPI/Timer 1, then when it hits a compare value CC0 that I'm interested in, e.g. 1024 in this case, it stops the Timer 1 via another PPI by triggering the STOP task of timer 1. This way I ensure the sample task has been hit exactly N times not more in a timely manner, independent of the IRQ response time of zephyr. I can share this code in PM if needed.

In fact I did confirm the timer 1 was hitting its CC0 too many times, I did confirm this by setting CC1 of timer 2 to 1025 just to see if it ever hits the SAMPLE task too many times, and sure it did! Then when I implemented the STOP via the additional PPI channel explained above, it never did.

Again, I'd really like to know what goes wrong inside the processor when we get the data misalignment (the older code I shared with you), this way I can be more confident I have actually fixed the problem!
In our product application if data gets misaligned it is impossible to detect and product will product bad data without any signs of it... so this is a big deal!

Thanks for looking into this.

[Simon]

Check out this answer https://devzone.nordicsemi.com/f/nordic-q-a/20291/offset-in-saadc-samples-with-easy-dma-and-ble/79053#79053, which explain this issue and why it happens. Also see the two possible solutions.