FOTA to an encapsulated Bluetooth device that is having "No free slot" error

[Nolan_G]

Hello,

Recently, my team has been struggling with an issue regarding a nRF5340 device that's encapsulated and inaccessible to be flashed with a JTAG connection. Our normal use case is:
1. Connect to the device via the DeviceManager iOS app or equivalent Android app

2. Select the "image" option

3. Select the OTA package.zip

4. Click start and "Confirm only"

This has worked for us up until this point, and we've used this process for nearly a year now. However, we ran into an issue with one of our devices that's fully closed off where the OTA bugged midway through downloading and the app reported an error of "No free slot". My understanding is this:

There is enough space for two builds on the device, each made up of two images, one for the Bluetooth core and one for the main core, and the primary build slot is the firmware that the device is running, and the secondary build slot is for OTAing images. When an image is successfully downloaded into the secondary slot, the primary slot will switch over to the image in the secondary slot and clear the secondary slot, which will allocate space for future OTA images to be placed into the secondary slot.

However, I believe what has occurred is that the OTA process bugged during the uploading stage of the image to the device, and this has left the secondary build slot in a "stuck" state where it cannot accept any new OTA images since it's full with an incomplete one. Thus, the "No free slot" error.

To solve this, my approach would be to connect to the device over Bluetooth and tell it to clear the secondary build slot. This is much easier said than done though. To do this, I tried to communicate with mcuMGR terminal commands, but through many rabbit holes, I found that OTAing is not available over Windows and therefore there's not much support for how to communicate via mcuMGR to the device to get it to clear its secondary build slot.

My question, therefore, is: how can we clear the secondary build slot of the device to solve this "No free slot" error via Bluetooth? We cannot open or unencapsulate the device.

Thanks!

[Edvin]

Hello,

Could it be that your issue is similar to this one?
RE: DFU Issue on NCS 2.6.1: First OTA Succeeds, Subsequent Attempts Fail with 'No Free Slot to Place the Image' Error

Perhaps your device is in "testing mode" of the new application, and hence, it can't delete the old application image before the new one is marked as verified. I understand that the JTAG connection is not available, but is there some way to reset your device? When you do, does it restart the same application that it is running, or the previous version?

If you don't have any way to reset it, I see that you can issue a reset command using SMP. 

Note that I had to press the button where it now says "Basic", where it used to say "Advanced" before I pressed it. Then it should show you the "Send Reset Command" at the bottom. Try this, and see whether the app rolls back to the previous version, or if you are able to do a new DFU after this.

Best regards,

Edvin

[Nolan_G]

Hi Edvin,

For some important additional context, we have many builds of the same device using an encapsulated nRF5340 as we're developing a product where this is the brain of the device. We have many fully-encapsulated devices that have been successful in repetitive FOTAing, but this is a new problem that we're seeing only on one device. Our process has been the same thus far. I believe it's related to the Bluetooth image of the secondary "build" that's not being cleared. 

[Edvin]

You can also try to tap the "Read" from my previous screenshot, and see what it says.

Best regards,

Edvin

[Nolan_G]

Thanks for the reply Edvin!

I actually had already given that a shot, and we found a way to get the application core image to be put into the primary slot. When we clicked "read", we saw that there were two images, the primary and secondary. When I clicked "erase", the secondary image was erased, and only one image was left, the primary image. I then attempted to upload an image, experienced the error, and then clicked "read" again and saw that there were now two images in place. I then clicked "test" to move the secondary image to the primary slot, clicked "confirm" to confirm the secondary image to the primary slot, and then I clicked "reset" to reset the device. The application core image was successfully moved to the primary slot.

However, this didn't solve the issue of the network core image not being able to be FOTA'd. If both images were FOTA-able, then the error wouldn't have happened in the first place. I would like to find a way to erase the secondary network core image so that I can FOTA successfully again.

I believe the answer involves terminal commands, but the terminal commands aren't supported on Windows. I was hoping to find a work-around, or to be able to do it with a Linux VirtualBox.

Do you have any ideas?

[Edvin]

Some of our FOTA samples, such as the zigbee Light_bulb sample has this snippet:

static void confirm_image(void)
{
	if (!boot_is_img_confirmed()) {
		int ret = boot_write_img_confirmed();

		if (ret) {
			LOG_ERR("Couldn't confirm image: %d", ret);
		} else {
			LOG_INF("Marked image as OK");
		}
	}
}

What it does is that it confirms itself. What I suspect is that you were running an image that was not confirmed, and hence it couldn't delete the old application image. I don't know how you usually confirm your images. Whether you do it from the application, like this snippet, or if you do it from the nRF Device Manager app. You didn't try to reboot the device without erasing the secondary image? 

One way to ensure that you are not running an unconfirmed image could be to start a timer if boot_is_img_confirmed() returns false, and have it time out e.g. after one minute. In the timeout handler, if the image is still not confirmed, you can reset the device, and it should boot up the old image. In that case, you should be able to overwrite the new image. 

Alternatively, always confirm the new image on bootup, like it is done in the confirm_image() function here. 

Best regards,

Edvin

[Nolan_G]

Thanks for the help Edvin! That's definitely something I'll look into, I think we're confirming images on the app side, but if we can do it in our code, we definitely will.

How do we reboot our device without erasing the second image? Could you elaborate?

Thanks!

[Edvin]

You can reboot as I described in my initial reply. 

Rebooting will not erase the second image in itself. But it will revert back to the old image, if the new one was not confirmed. Then you can choose whether you want to test the new image again, or if you want to delete it and upload another. 

I believe the issue was that you were running an unconfirmed image, and hence, it couldn't delete the only confirmed image that it had (which was currently placed in slot 1 (the running image is always in slot 0, even though it is not confirmed). 

So if you were to reboot, it would boot up the old confirmed image (after swapping it back to slot 0), and slot 1 would still contain the unconfirmed image, until you either manually erase it using mcumgr, or until you try to upload a new image.

BR,
Edvin