Intermittent problem restarting advertising: ENOMEM, failed to create advertising set

Hi all,

Short problem description: I am troubleshooting an issue with the nRF5340 wherein the device runs for some time, then eventually gets stuck with an err -12 (ENOMEM) when attempting to create an extended advertising set.

This is on a custom board with the Fanstel nRF5340 module. I am using nRF Connect SDK 1.7.1. I cannot upgrade this, because we have already made a declaration with the Bluetooth SIG for this product. I am using hci_rpmsg on the net core. The code I am troubleshooting runs on the app core.

There are two execution paths by which advertising gets restarted:

1) During normal operation, occasionally the app core receives updated manufacturer data for advertisement via SPI. When this happens, advertising is stopped, the advertising set is deleted, a new one created, and advertising is restarted. Normally, this mechanism will work fine, with no problems and runs indefinitely.

2) When a Bluetooth connection is established, advertising stops (by design). After the connection ends, in the callback for the disconnect, a worker task is queued up to restart advertising. this task does not delete or update the advertising set - it merely restarts advertising.

What happens is that if I connect to the device repeatedly - sometimes after 12 times, other times it might take 300 times - eventually I enter a state where the device has disconnected from Bluetooth and is trying to restart advertising and keeps getting the error ENOMEM when attempting to create the advertising set.

I can increase the number of extended advertising sets allowed (CONFIG_BT_EXT_ADV_MAX_ADV_SET), as has been suggested in other posts I have seen, and I did try that but immediately started to see erratic behavior, so I backed off the change, and I wanted to ask here some questions:

  • Is it better for me to, instead of increasing the number of advertising sets, to figure out what causes the problem and address it? The intent of the design is to have only one advertising set.
  • If I catch this exception, could I have code in place that will resolve the issue automatically, perhaps by deleting old advertising sets or somehow re-initializing advertising? What should that look like?  
  • Is it likely that this error occurs as a result of sloppy code on my side and could be prevented?

Thanks!

Scott

Parents
  • I have continued to troubleshoot on this issue, adjusting config options and making tweaks and checks. As it stands right now, I still get err -12 ENOMEM, but now when this occurs, I see the Zephyr boot banner immediately afterward and the IC becomes mostly unresponsive. It still gives the standard "I'm listening" data on the MISO line as the SPI master clocks data in to it, but it does not make any response to commands. Power cycling it, or a reset from the J-Link will revive it.

    Any ideas what could be causing this? My network core config options are below. I will try to attach the app core config as a file, since it's a little longer.

    CONFIG_LOG=n
    CONFIG_SERIAL=n
    CONFIG_BOOT_BANNER=n
    CONFIG_BT_EXT_ADV=y
    CONFIG_BT_CTLR_ADV_EXT=y

    CONFIG_BT_CTLR_CONN_RSSI=y
    CONFIG_BT_CTLR_TX_PWR_DYNAMIC_CONTROL=y

    CONFIG_BT_EXT_ADV_MAX_ADV_SET=3
    CONFIG_BT_CTLR_ADV_SET=3
    CONFIG_BT_MAX_CONN=3
  • Try increasing heap, stack, would also disable printk.

    As for SPI, one of the reasons maybe that CS line is not properly released, did you get a chance to debug inside SPI driver., do you control the other side of the SPI peripheral, if master is stuck with CS selected, the other side should be stuck as well.

    Are using bootloader, haven't seen it in config the file. You can also manually release/set any SPI CS during board initialization before Zephyr runs.


    Thanks,

    Reuven

Reply
  • Try increasing heap, stack, would also disable printk.

    As for SPI, one of the reasons maybe that CS line is not properly released, did you get a chance to debug inside SPI driver., do you control the other side of the SPI peripheral, if master is stuck with CS selected, the other side should be stuck as well.

    Are using bootloader, haven't seen it in config the file. You can also manually release/set any SPI CS during board initialization before Zephyr runs.


    Thanks,

    Reuven

Children
  • Thanks once again, Reuven.  I'm doing some research on increasing stack and heap sizes. When you say increase stack and heap, do you mean for the net core, app core, or both? Or maybe I ought to just give each a shot in turn and see if it helps. Doing some reading online in an effort to find out what are their default and max values, and what config options are used to set the current value.

    I do have control of the SPI master device, though, and I have a logic analyzer on this interface so I can see that the CS line is being asserted and released as expected when communications occur.

    We are not using any bootloader.

    I noticed yesterday afternoon that the reboot and hang event seemed always to be preceded by a failure to restart advertising, and I tried to catch that error and force an NVIC.SystemReset.  I get erratic behavior with that code in place, even before it actually executes, and had to back off the change.

    Anyway, I will get back to research on stack and heap allocation. Thanks for your help.

    Scott

  • For our network core, I have been editing a config file that is under my main application in a folder called child_image.

    But I just noticed, there is a separate application folder for hci_rpmsg and it has its own prj.conf. I didn't do the initial setup on this project or repo, and I'm not sure which of these two config files is actually being used - or maybe settings from both are applied.

    I do know that HCI RPMSG is being built as a child image. Does that imply that this other application folder is vestigial and its config is not used?

  • Hi Scott,

    If it exists under app/child_image, hci_rpmsg.conf  is included in the build, and used for the net core to configure submodules/settings.

Related