SOFTDEVICE: ASSERTION FAILED while running mesh.

[bart]

Hi, I'm getting regular crashes with my app: SOFTDEVICE: ASSERTION FAILED.

Using and NRF52832, SDK 15.3.0, SD 132 6.1.1, and Mesh SDK 3.1.0.

With DEBUG defined, I don't really get more info: id=1, pc=148742, info=0.

The last mesh log usually is something with TX (mostly net_beacon.c,  265, BEACON TX).
One crash, I also had a log of getting event NRF_EVT_RADIO_SESSION_IDLE.

I use the mesh SDK to communicate via mesh between nodes, to advertise iBeacon (100ms interval), and I use the scan callback. Next to that, I also use the SD to advertise service data.

Parameters:

	nrf_clock_lf_cfg_t lfclksrc;
	lfclksrc.source = NRF_SDH_CLOCK_LF_SRC;
	lfclksrc.rc_ctiv = NRF_SDH_CLOCK_LF_RC_CTIV;
	lfclksrc.rc_temp_ctiv = NRF_SDH_CLOCK_LF_RC_TEMP_CTIV;
	lfclksrc.accuracy = NRF_CLOCK_LF_ACCURACY_20_PPM;

	mesh_stack_init_params_t init_params;
	init_params.core.irq_priority       = NRF_MESH_IRQ_PRIORITY_THREAD;
	init_params.core.lfclksrc           = lfclksrc;
	init_params.core.p_uuid             = NULL;
	init_params.core.relay_cb           = NULL;

Any idea of what causes this crash?

[Bjørn Kvaale]

Let me see if I can get a hold of a softdevice elf file. Then we could use the addr2line tool to see where the softdevice assert is occuring. I will get back to you soon.

[Bjørn Kvaale]

What happens if you decrease the iBeacon advertising interval to say every 2 seconds? Do you get the assert then?

[Bjørn Kvaale]

Have you tried using the addr2line tool? How are you doing the iBeacon advertising? Could you upload some code so we can double check how you are doing this? Are you using the softdevice to do the iBeacon advertising?

[bart]

I tried, but only get ??:0 Don't you need the softdevice elf for that address?

Here you can see the code, but as i said, i had the crashes even without the mesh doing the advertising. The softdevice is also advertising a different payload.

So far, i didn't get a crash yet with the margin set to 1000, while both mesh and softdevice are advertising.

[Bjørn Kvaale]

Very sorry for the delayed response. As you can see from this blog post, you use the Timeslot API directly to communicate between the Mesh stack & the softdevice. The softdevice is only used for the iBeacon advertising. From your code, it seems there is definitely something wrong with the timeslot of the iBeacon advertisement. 

Have you taken a look at one of the coexist examples between Mesh & BLE UART for example? I will take a closer look tomorrow morning.

[bart]

Hey Bjorn,

I'm aware of the mesh using the timeslot API. And I looked at the ble_app_proximity_coexist example, is the uart one more relevant?

Like I said before, even without the iBeacon advertising, I still got the crashes, so I fail to see how that can be the main cause. Did you maybe mean the advertising i do directly via the SD (which are not iBeacon advertisements)? The ibeacon advertising is done via the mesh api, not directly via the SD.

Also like I said before, ever since I set TIMESLOT_END_SAFETY_MARGIN_US to 1000, I don't get the crash anymore:

I tried setting TIMESLOT_END_SAFETY_MARGIN_US to 1000. In that case, I still quite often get the FDS timeouts on boot, but it doesn't result in the event NRF_EVT_RADIO_SESSION_IDLE, and a softdevice assert anymore.

Does that mean the 100 is too tight? How do you determine the correct value?

[Bjørn Kvaale]

So sorry for the delayed reponse again! I have asked internally & the quick answer is do not use FDS. FDS with Bluetooth Mesh will lead to timeslot issues & SD asserts. Much easier to use the provided flash manager in the mesh sdk. A lot easier to use too from what I have heard.

[Bjørn Kvaale]

Another update: "As I see, they are using FDS. If you see the mesh SDK documentation:
https://infocenter.nordicsemi.com/topic/com.nordic.infocenter.meshsdk.v3.2.0/md_doc_getting_started_how_to_nordicSDK.html?cp=5_2_1_7

We mention that:

nRF5 SDK NVM storage modules

Using nRF5 SDK modules such as fstorage, pstorage, or ble_flash for writing to flash may be problematic due to long timeslot events occupied by the mesh stack. Use the Flash manager module provided by the mesh stack instead.

He can either use flash manager or better, mesh_config, as used in various other internal modules.

Also, for beacon advertising, if possible, use the Mesh advertiser instead of SD advertiser."

[bart]

Ah, I read this as FDS will get in some trouble (write timeouts), not the mesh getting into trouble, and definitely not getting SD asserts. Also, above your quote, the documentation mentions:

To allow safe coexistence of the flash storage module Flash manager in the mesh stack and the flash storage module fstorage in the Nordic nRF5 SDK, add the following code block to nrf_mesh_config_app.h

So that makes one think it's fine to use FDS next to flash manager. Can you maybe explain why FDS would lead to SD asserts?

Regarding the advertising, doesn't the mesh use the timeslot API for that? If so, why would it matter that I also use SD advertiser? Seeing how mesh messages are also just advertisements, you'd expect it doesn't matter for the mesh whether it's advertising mesh messages or iBeacon messages. How would SD advertising result in SD asserts?

Also, how come setting TIMESLOT_END_SAFETY_MARGIN_US to 1000 makes these asserts vanish?

[Bjørn Kvaale]

Very sorry for the delayed response. Received an update: "Using the FDS shouldn't normally cause the mesh to kill the Softdevice, neither should the SD advertiser. Since increasing the safety margin helps, it sounds like there's either some IRQ locking going on in the application or some excessive clock drift that is causing this problem. My bet is on the former.

To rule out any clock drift issues, they can try running the application on a DK if they're on a custom HW platform. They can also try running the normal examples on the same HW with proxy enabled. If this is a xtal HW problem, any softdevice activity on an unedited example application should invoke the same crash. 

Next, I'd profile any and all IRQ locking (including SDK modules). A full IRQ lock preferably shouldn't last for more than 20us, and definitely not for more than 50us. Both the softdevice API for locking (sd_nvic_critical_region_enter) and the hardware API (_DISABLE_IRQS(), __disable_irq()) could be causing issues here. Also make sure that no application interrupts are executing in any of the Softdevice IRQ levels.

If none of this explains it, I think we'd need more details to determine the cause. What happens right before the crash? Is it possible to get a logic analyzer trace of the crash with mesh debug pins enabled and initialized for timeslot? What's the XTAL accuracy passed to mesh_stack_init? Does it fit with the XTAL HW?"