nrf52840 S140 Central DFU Bluetooth Zephyr 2.0.0 nrf52840 peripheral

[DavidKaplan]

We have nrf52840 S140 BLE Central device that currently connects to TI based sensors. I have implemented code that can upgrade those TI sensors' firmware using the TI BLE protocol.

Now we are developing our own BLE peripheral sensors also based on the nrf52840 but now using the newly supported Zephyr 2.0.0. RF connect stuff. I can update the sensors using the Nordic Android program (unauthenticated for now) and now need to implement upgrading BLE DFU using our central device which connects to a cellular gateway.

I do not want to reinvent the wheel, writing existing code, like I had to do with the TI sensors.

Could you point me to code that allows a Central device to update a peripheral device BLE DFU?

If not where can I look to reinvent another wheel?

Thanks David

[Simon]

Have a look at the sample Bluetooth: Central SMP Client

Then use the SMP sever Sample on the Bluetooth Peripheral device

Currently it is only possible to send the echo command from the Central SMP Client, so you would have to implement the other commands yourself, like upload, list, test and so on.. 

Best regards,

Simon

[DavidKaplan]

Now I have gotten back to this issue with our custom sensor hardware to arrive next week.

I have used the Android nRF Toolbox app successfully to update an unsigned image whereas the Nordic DFU tool did not work.

I understand that the Android apps use the SMP protocol that ties in with the img_mgmt libraries.

Where can I find documentation on what the Android app does so I can reproduce it in c in my central device?

Without any documentation it will be very hard for me to reverse engineer the Java somehow.

I do not want to have to add more code in the sensor to handle a custom DFU if possible since it already exists and the Zephyr junk makes a giant image anyway.

I saw a post with my exact request,  but I understand that it is not a priority issue.

 BLE DFU, MCU to MCU (Zephyr), GATT DFU SMP Service Clinet 

I also saw a SoftDevice (not Zephyr) example post of what I need but uses the old DFU stuff I think and SoftDevice APIs

 Mesh DFU: Serial transfer of DFU archive 

I need much more information to implement upload,list & test SMP commands that evidently exist in my sensor using the SMP server.

Could it be that the Android app used some legacy DFU?

Thanks David

[DavidKaplan]

After trying to implement the nRF52840 BLE DFU without SMP & ZCBOR from my nRF52840  central device which worked 2/3 of the time, I am back to trying to get your code to work on my central.

I am using Zephyr ncs v.2.2.0 and have made the two changes in the zcbor_decode.c file and have added the

CONFIG_ZCBOR=y
CONFIG_ZCBOR_STOP_ON_ERROR=y

defines in both my central an my sensors.

At first I was getting an error after the second packet where I got -122 which seems to be a too large of a packet. Maybe the sensor just disconnected.

01-05 10:47 05/01/2023 08:47:49 |  05/01/2023 10:48:04[.ami_vibe.c:1148] #1 DFUpos:129000 len:1000
01-05 10:47 05/01/2023 08:47:49 |  05/01/2023 10:48:04[.ami_vibe.c:1148] #1 DFUpos:130000 len:1000
01-05 10:47 05/01/2023 08:47:49 |  05/01/2023 10:48:04[.ami_vibe.c:1962] ...[ami_ble.c:1703] StartScanConnect#1 FD:2C:12:F9:D6:03
01-05 10:47 05/01/2023 08:47:52 |  05/01/2023 10:48:07[.ami_vibe.c:1962] ...[ami_ble.c:1733] DiscoveryStarted
01-05 10:47 05/01/2023 08:47:57 |  05/01/2023 10:48:12[.ami_vibe.c:1962] ...[ami_ble.c: 609] MainHandlesOK
01-05 10:48 05/01/2023 08:48:00 |  05/01/2023 10:48:15[.ami_vibe.c:1962] ...[ami_ble.c: 636] SetupHandlesOK
01-05 10:48 05/01/2023 08:48:00 |  05/01/2023 10:48:15[.ami_vibe.c:1962] ...[ami_ble.c: 725] DISHandlesOK
01-05 10:48 05/01/2023 08:48:01 |  05/01/2023 10:48:16[.ami_vibe.c:1962] ...[ami_ble.c: 670] TransferHandlesOK
01-05 10:48 05/01/2023 08:48:02 |  05/01/2023 10:48:16[.ami_vibe.c:1962] ...[ami_ble.c: 739] SMPHandlesOK
01-05 10:48 05/01/2023 08:48:12 |  05/01/2023 10:48:26[.ami_vibe.c:1962] ...[ami_smp.c: 671] DFU:0/373928
01-05 10:48 05/01/2023 08:48:22 |  05/01/2023 10:48:36[.ami_vibe.c:1962] ...[ami_smp.c: 671] DFU:50/373928
01-05 10:48 05/01/2023 08:48:22 |  05/01/2023 10:48:37[.ami_vibe.c:1962] ...[ami_smp.c: 780] bt_dfu_smp_command failed with -122
01-05 10:48 05/01/2023 08:48:24 |  05/01/2023 10:48:38[.ami_vibe.c:1962] ...[ami_ble.c:1394] DisconnectTimeout
01-05 10:48 05/01/2023 08:48:24 |  05/01/2023 10:48:39[.ami_vibe.c:1962] ...[ami_ble.c:2492] DFUConnectFinishedErr:5

Now, in my latest code, I must have changed something, since the smp_upload_rsp_proc() is not being called but I do not get an error when calling bt_dfu_smp_command().

01-05 15:43 05/01/2023 13:43:00 |  05/01/2023 15:43:14[.ami_vibe.c:1148] #1 DFUpos:130000 len:1000
01-05 15:43 05/01/2023 13:43:00 |  05/01/2023 15:43:14[.ami_vibe.c:1941] ...[ami_ble.c:1703] StartScanConnect#1 FD:2C:12:F9:D6:03
01-05 15:43 05/01/2023 13:43:03 |  05/01/2023 15:43:17[.ami_vibe.c:1941] ...[ami_ble.c:1733] DiscoveryStarted
01-05 15:43 05/01/2023 13:43:08 |  05/01/2023 15:43:22[.ami_vibe.c:1941] ...[ami_ble.c: 609] MainHandlesOK
01-05 15:43 05/01/2023 13:43:11 |  05/01/2023 15:43:25[.ami_vibe.c:1941] ...[ami_ble.c: 636] SetupHandlesOK
01-05 15:43 05/01/2023 13:43:11 |  05/01/2023 15:43:26[.ami_vibe.c:1941] ...[ami_ble.c: 725] DISHandlesOK
01-05 15:43 05/01/2023 13:43:12 |  05/01/2023 15:43:26[.ami_vibe.c:1941] ...[ami_ble.c: 670] TransferHandlesOK
01-05 15:43 05/01/2023 13:43:12 |  05/01/2023 15:43:27[.ami_vibe.c:1941] ...[ami_ble.c: 739] SMPHandlesOK
01-05 15:43 05/01/2023 13:43:13 |  05/01/2023 15:43:27[.ami_vibe.c:1941] ...[ami_smp.c: 659] DFU:0/373928

Can you think of any reason that the callback is never called?

Is there any reason that your code would not work with the zcbor_decode.c file changes on v2.2.0 ?

I will try to continue next week and compare my code to see what has happened.

I can DFU from my phone with the nRFConnect app on Android.

Thanks David

[DavidKaplan]

This morning I found a few problems:

 a) I am still using v2.0.2 for this project but applied the patch correctly in zcbor_decode.c only in v2.2.0 which I am using for newer projects.

 b) I fixed a problem keeping my ring buffer full. I use a ring buffer with the image streamed from the host to the central instead of from the central's flash.

 c) My custom printf function was evidently causing a problem in smp_upload_rsp_proc() so instead of printing, I just update an error parameter

I was able to finish uploading the send_upload2() function or at least it does not show any error.

Now I understand that I need to call send_smp_test() to set the pending flag but it gives an error.

I then saw that it uses the hash_value_secondary_slot variable as a parameter so I try to call send_smp_list() first so it will be updated.

When I call send_smp_list() the central resets.

I tried to call bt_disable() in send_smp_list() so I could single step to where the problem is, but this did not help as the ble crashes. How can I find the problem?

My sensor boards have the standard nRF52840 image setup.

So, I understand that I need to list,test & reset the sensor board?

Thanks David

[DavidKaplan]

As happens all too often, I awoke in the middle of the night and thought that I could debug the smp_list_rsp_proc() function by disconnecting the sensor 's BLE at the start of the function.

I found the reason for the smp_list_rsp_proc() function's problem in my case.

I found that my sensors are returning what is called the version_key with a length of 7 bytes whereas there are only 5 allocated. I just changed this to 16 for now and no crashes after removing the sensor disconnect lines which I inserted for debugging.

			//Decoding version key 
			char version_key[5]; // my key is 7 ?!?
			ok = zcbor_tstr_decode(zsd, &value);
			if (!ok) {
				printk("Decoding error, version key (err: %d)\n", zcbor_pop_error(zsd));
				return;
			}  /*else if (value.len != 6) {

I am testing my code now, but I have a couple of new questions.

1) I have a MTU of about 491 when uploading. I set the  UPLOAD_CHUNK to 400 in send_upload2().

I would like to calculate the payload at runtime according to the current MTU.

I am a little stuck on figuring out the whole payload size.

send_upload2()
...
  while (!update_complete){
           uint16_t mtu = bt_gatt_get_mtu(dfu_smp.conn);
           uint16_t user_payload_len = ?!?
             ...
			(zse->payload - smp_cmd.payload);
             ...			
           if (user_payload_len>(mtu-header_size)){
             user_payload_len = (mtu-header_size);
           }

2) My sensor code is currently quite large 365 KB (373,888 bytes).

I saw a broken link that might help in the release version to safely cut its size.

 Best practice advice for Zephyr debug and release configurations. 

I will get back on my DFU testing.

Thank you David

[Sigurd Hellesvik]

Hi David,

I will continue helping you with this case.

There are two ways to mark an DFU image as "test".
One is to use the SMP command.
The other is to tag the image before you send it.

For debugging purposes, can you try the second one as well?

I'm thinking that if you upload a "pre-tagged" image to the SMP Server, it should automatically update on its next reboot.

To sign an image manually you use imgtool.
To mark it as "test" you use the "--pad" option.

The command will look something like this:

<NRF_CONNECT_PATH>/bootloader/mcuboot/scripts/imgtool.py sign --header-size 0x200 --align 4 --slot-size 0x79e00  --version 1.0.0 --pad-header --pad --key <SIGNING_KEY_PATH> build/zephyr/app_to_sign.bin app_manually_signed.bin

To find the other variables used for this command (to generate app_update.bin), you can use "west -vvv build -p -b <BOARD_NAME>" and search the log for "imgtool".

Does the image swap if you mark the image as "test" before uploading it?

EDIT: Did not see your previous message before I sent this one.
I will read through it now.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Read through your newest message, and here are my comments for now:

DavidKaplan said:

2) My sensor code is currently quite large 365 KB (373,888 bytes).

I saw a broken link that might help in the release version to safely cut its size.

 Best practice advice for Zephyr debug and release configurations. 

I will get back on my DFU testing.

We have some official docs on optimizing memory usage on Optimizing application.
I would probably start by having a look at the tips there.

DavidKaplan said:

1) I have a MTU of about 491 when uploading. I set the  UPLOAD_CHUNK to 400 in send_upload2().

I would like to calculate the payload at runtime according to the current MTU.

I am a little stuck on figuring out the whole payload size.

I will have to look into this as well, and will let you know what I find out.

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Hi again,

Sorry for the quick succession of answers,
but I asked a colleague who knows a bit more than me about Bluetooth Low Energy, and they say that the MTU is the payload size + 3 bytes of op-code and handle:
payload = mtu - 3

Regards,
Sigurd Hellesvik

[DavidKaplan]

Thanks for your help.

After thinking I was advancing on the DFU, I must have changed something in the central or the sensor since I can't seem to upload now.  I created two images with just my date changed that I wanted to test between them. I do not get a reply from my first packet. My images are not built with the imgtool .

The cellular nRFConnect app behaves differently each time.

Connecting,Starting DFU,Upload,Validating (long time) & Disconnect.

Sometimes there is no Upload.

Very frustrating...

[Sigurd Hellesvik]

DavidKaplan said:

I must have changed something in the central or the sensor since I can't seem to upload now.

Probably not my place to ask, but:

Do you use git?
It would make it easier to track your changes and figure out what changed since last time.

Most people use git, so I assume you do. Just had to check in case not.
However, know that it can definitively be frustrating even with git, especially when I (usually) forget to make commits as frequently as I maybe should.

Regards,
Sigurd Hellesvik

[DavidKaplan]

You are most likely correct that i should use git as now I take a zip snap-shot once or twice a day.

I found that I had recompiled my sensor code deleting my manual BLE image transfer code since I am hoping to get the SMP stuff working and that was causing the problem. Most likely a proj.confg define that I had added for manual BLE image transfer, but I will figure out that later on.

Now I tried without the image tool to upload, list, test & reset and the reset command does not work.

Do I need a special define for It?

01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1885] CentralStatusReply OnLine:0F
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:369200/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:369600/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:370000/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:370400/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:370800/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:371200/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:371600/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:09[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:372000/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:10[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:372400/372827
01-09 13:08 09/01/2023 11:08:42 |  09/01/2023 13:09:10[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:372800/372827
01-09 13:08 09/01/2023 11:08:43 |  09/01/2023 13:09:10[.ami_vibe.c:1941] ...[ami_smp.c: 843] SMP UploadOK
01-09 13:08 09/01/2023 11:08:43 |  09/01/2023 13:09:10[.ami_vibe.c:1941] ...[ami_smp.c:1127] ImageList
01-09 13:08 09/01/2023 11:08:43 |  09/01/2023 13:09:10[.ami_vibe.c:1941] ...[ami_smp.c:1130] ImageTest
01-09 13:08 09/01/2023 11:08:43 |  09/01/2023 13:09:11[.ami_vibe.c:1941] ...[ami_smp.c:1133] ImageReset

Now I tried without the image tool to upload, list & test and after manually resetting the sensor, the uploaded image was not running.

01-09 13:18 09/01/2023 11:18:36 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:370800/372827
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:371200/372827
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:371600/372827
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:372000/372827
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:372400/372827
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 683] DFU:372800/372827
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c: 843] SMP UploadOK
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c:1127] ImageList
01-09 13:18 09/01/2023 11:18:37 |  09/01/2023 13:19:04[.ami_vibe.c:1941] ...[ami_smp.c:1130] ImageTest
01-09 13:18 09/01/2023 11:18:38 |  09/01/2023 13:19:05[.ami_vibe.c:1941] ...[ami_ble.c:2561] DFUConnectFinishedErr:0

I was able to get the image to update at some point yesterday.

I guess I can try the image tool but I would like to get the SMP reset to work also.

I run the tool after I build with VSCode?

static int send_smp_reset(struct bt_dfu_smp *dfu_smp,const char *string)
{
	static struct smp_buffer smp_cmd;
	zcbor_state_t zse[CBOR_ENCODER_STATE_NUM];
	size_t payload_len;

	zcbor_new_encode_state(zse, ARRAY_SIZE(zse), smp_cmd.payload,
			       sizeof(smp_cmd.payload), 0);

	/* Stop encoding on the error. */
	zse->constant_state->stop_on_error = true;

	zcbor_map_start_encode(zse, CBOR_MAP_MAX_ELEMENT_CNT);
	zcbor_tstr_put_lit(zse, "d");
	zcbor_tstr_put_term(zse, string);
	zcbor_map_end_encode(zse, CBOR_MAP_MAX_ELEMENT_CNT);

	if (!zcbor_check_error(zse)) {
		d_printf(LINE_INFO,kDbg_Error|kDbg_Host,"Failed to encode SMP reset packet, err: %d", zcbor_pop_error(zse));
		return -EFAULT;
	}

	payload_len = (size_t)(zse->payload - smp_cmd.payload);

	smp_cmd.header.op = 2; /* Write */
	smp_cmd.header.flags = 0;
	smp_cmd.header.len_h8 = 0;
	smp_cmd.header.len_l8 = 0;
	smp_cmd.header.group_h8 = 0;
	smp_cmd.header.group_l8 = 0; /* OS */
	smp_cmd.header.seq = 0;
	smp_cmd.header.id  = 5; /* RESET */

	return bt_dfu_smp_command(dfu_smp, smp_reset_rsp_proc,
				  sizeof(smp_cmd.header) + payload_len,
				  &smp_cmd);
} // send_smp_reset

Thanks David

[DavidKaplan]

OK.

I flashed by the programmer a sensor with new code.

I then uploaded successfully an unsigned image or at least  smp_upload_rsp_proc() finished without error.

When I performed a list the central again crashed. I disconnected in the code again the sensor and saw yet another place there was an array overflow:

			//decoding version value
			char version_value[16]; // ?!? len=5 was [5]
			ok = zcbor_tstr_decode(zsd, &value);

I also saw that the Listed second slot (10 was not listed so its hash_value_secondary_slot was not retrieved. It is used in both Test and confirm!

I could dynamically check every array value.len before copying or just increase all.

How do I know that tomorrow I will not overflow somewhere?

Another image that I also programmed never returned from the first upload packet.

I need to have bullet proof BLE DFU with maybe oe or two retries.

That is the reason that I put on hold my own image BLE image transfer and write mechanism.

It worked sometimes 2/3 of the time.

Tomorow I may try the signed image, but it does not look trivial to set up.

I may need to work on other projects.

On this project, I am left just with BLE DFU upgrades.

 1) SMP reset did not work.

 2) List gave only slot zero so could not use test or confirm.

Thanks David