Buttonless DFU with bonding

RE: Buttonless DFU with bonding

Einar Thorsrud — Thu, 06 Feb 2020 11:38:14 GMT

Hi Dani,

Yes, I would like the full project folders, so that it includes the toolchain/IDE configuration, main.c file, sdk_config.h file etc. (for example everything within <SDK>/examples/ble_peripheral/ble_app_template/ if that was the example, and same with DFU bootloader).

Can you make a new private case and upload it there if you do not want the code to be public? Please refer to this thread when doing so.

RE: Buttonless DFU with bonding

Dani — Thu, 06 Feb 2020 07:59:54 GMT

Can I send a private message directly to you?

Do you need the full folder?

RE: Buttonless DFU with bonding

Einar Thorsrud — Thu, 06 Feb 2020 07:55:50 GMT

Hi Dani,

That should not be the case when using service changed. Can you upload your complete bootloader project and application project here?

RE: Buttonless DFU with bonding

Dani — Wed, 05 Feb 2020 13:01:59 GMT

Hi Einar,

Is the same: DFU works correctly but if I do not select a 'manual refresh', I'm back to same situation: only DFU services are shown.

Dani

RE: Buttonless DFU with bonding

Einar Thorsrud — Wed, 05 Feb 2020 12:56:02 GMT

Hi Dani,

[quote user="dtarrago"]Before implementing fix suggested by Vidar, I have directly gone to first point you have suggested in last answer: now, nRFToolbox is crashing but I have succeeded with nRF Connect application. After DFU update, the situation was the same (only DFU services were observed) but after selecting and connecting again to the bonded device, all application services appear again after going to options and selecting 'Refresh Services'.[/quote]

That is good. It is expected that the phone can discover all the services when you tell it to do service discovery. The problem is that this should have been automatic when you issue a service changed. What happens if you do DFU again at this point, but not "manually" select refresh service when entering the application again. Do the application's services appear then, or is it back to the old behavior?

RE: Buttonless DFU with bonding

Dani — Wed, 05 Feb 2020 11:28:08 GMT

Hi Einar,

Some new information for you:

Before implementing fix suggested by Vidar, I have directly gone to first point you have suggested in last answer: now, nRFToolbox is crashing but I have succeeded with nRF Connect application. After DFU update, the situation was the same (only DFU services were observed) but after selecting and connecting again to the bonded device, all application services appear again after going to options and selecting 'Refresh Services'.

Dani.

RE: Buttonless DFU with bonding

Einar Thorsrud — Fri, 31 Jan 2020 10:32:19 GMT

Hi Dani,

I did not remember that issue, but yes, this could very well be the problem. You can check if the same thing happens in your case using a sniffer, or simply implement the fix as suggested by Vidar in that thread (which you should do anyway).

With the that fixed I cannot think of any likely reasons for this issue, but if you are still stuck, there are a few things worth doing:

Could it be that the phone has cached the services form before you added service changed characteristic to both the bootloader and application (if ever)? If so, then you should remove the bonding information and toggle BLE off and on on the phone to clear it's cache. Then fully erase the nRF and program the bootloader and application again. In this case, the phone should be aware of the issue
If still not sure if the phone could have cached something from before service changed characteristic was added, you could change the MAC address of the nRF (remember to do so both in app and bootloader in that case), or simply test with a new phone.
If the above also doe snot work, then it would be useful with a sniffer trace to see what is actually happening on-air (is the service change indication sent, ...?)

Einar

RE: Buttonless DFU with bonding

Dani — Thu, 30 Jan 2020 12:51:00 GMT

Hi Einar,

I have implemented what you have suggested:

/**@brief Function for handling Peer Manager events.
 *
 * @param[in] p_evt  Peer Manager event.
 */
static void pm_evt_handler(pm_evt_t const * p_evt)
{
    pm_conn_sec_config_t config;
	
	pm_handler_on_pm_evt(p_evt);
    pm_handler_flash_clean(p_evt);

    switch (p_evt->evt_id)
    {
        case PM_EVT_PEERS_DELETE_SUCCEEDED:
            advertising_start();
            break;
				
		case PM_EVT_CONN_SEC_CONFIG_REQ:
			config.allow_repairing = true;
			pm_conn_sec_config_reply(p_evt->conn_handle, &config);
			break;
				
		case PM_EVT_BONDED_PEER_CONNECTED:
			pm_local_database_has_changed();
			break;

        default:
            break;
    }
}

And the result is the same.

I'm using SDK 15.2.0. I do not know if there is any issue related with this and with also what is commented in this thread (https://devzone.nordicsemi.com/f/nordic-q-a/46633/peer-manager-silently-drop-service-changed-indications-if-an-att-mtu-exchange-was-happening/204968#204968)

Best regards,

Dani

RE: Buttonless DFU with bonding

Einar Thorsrud — Thu, 30 Jan 2020 09:42:29 GMT

Hi dani,

You can for instance call pm_local_database_has_changed() when you get the PM_EVT_BONDED_PEER_CONNECTED event, but the timing is not critical since it is buffered ant sent when the peer connects if called "too" early. See API doc for pm_local_database_has_changed() for details.

(Ideally, you should only call pm_local_database_has_changed() the first time the bonded peer connects after a DFU, though the only consequence of calling every time is that the center will perform service discovery every time, which may not be a problem).

RE: Buttonless DFU with bonding

Dani — Thu, 30 Jan 2020 08:18:56 GMT

Hi Einar,

In my application, at which point should I call 'pm_local_database_has_changed'?

Regards,

Dani

RE: Buttonless DFU with bonding

Einar Thorsrud — Fri, 24 Jan 2020 14:31:33 GMT

Hi Dani,

[quote user="dtarrago"]So, how can we go ahead?[/quote]

Summing up:

The remaining issue is that you only see the DFU service after entering the application after a DFU upgrade.
You have verified that the application and not bootloader is running.

If the application is running and you know it has other BLE services but these are not shown on the central side, this must be because the central has not done service discovery, which it should have if you have service changed. Can you try to call pm_local_database_has_changed() in your application? If you do not have any progress it would be good if you could upload your completed application project and bootloader project so that I can see (you can upload it in a private case if you don't want it public).

Einar

RE: Buttonless DFU with bonding

Dani — Fri, 24 Jan 2020 08:06:32 GMT

Hi Eimar,

You said..

As I understand, you have verified that the device is in the application and not in the bootloader?

Yes, as the device is perfectly working in what regards functionality.

You said..

But you only see the DFU service on the phone, and not any of the services in your application?

Yes, after DFU is performed, device works perfectly but I only see DFU service.

You said...

If so, the problem is that the phone have not done a service discovery, which it should have done automatically if you use service changed properly. Is that enabled in both the application and bootloader?

Yes. I have set NRF_SDH_BLE_SERVICE_CHANGED to 1 in both application and bootloader.

So, how can we go ahead?

All the best,

Dani.

RE: Buttonless DFU with bonding

Einar Thorsrud — Thu, 23 Jan 2020 18:22:46 GMT

Hi Dani,

What we have discussed in the last days should hopefully solve this part of the problem:

But I have more information: up to now I was doing all the tests with the LG K4 (Android 6.0.1). I have just tested the same process with a Xiami Mi6 and now the DFU process is perfectly working. But...

What does it happen with bonding after application update? After DFU, an as it should be, my module is reseted and starts working again but, when I try to connect again with it with BLE (using the same app I normally use), it is not possible to succeed. In order to correctly connect again with it, I have to erase bonding information from the Smartphone, and then everything works. How can I manage bonding in order to avoid this situation to happen after DFU?

But not this part:

I have also tried to check behavior with 'nRF Connect' tool after this succesful DFU process: as the device is already bonded, it appears in bonded list, and when I try to connect, I only am able to see the characteristics related to DFU service (but device is not working in DFU mode, as apart of BLE connection, it is perfectly working).

..which is the same you see now. As I understand, you have verified that the device is in the application and not in the bootloader? But you only see the DFU service on the phone, and not any of the services in your application? If so, the problem is that the phone have not done a service discovery, which it should have done automatically if you use service changed properly. Is that enabled in both the application and bootloader?

Einar

RE: Buttonless DFU with bonding

Dani — Thu, 23 Jan 2020 15:00:29 GMT

Hi Einar,

As described by you, I have done the following settings:

Aplication:

FDS_VIRTUAL_PAGES 6 (bonding info, garbage,...)
Flash (Data):
- //Preserved data blocks address:
- #define FLASH_ADR_FINAL_FLASH (u32) 0x000f2000
- #define FLASH_ADR_INI_PAGE_1 (u32)(FLASH_ADR_FINAL_FLASH-0x1000)
- #define FLASH_ADR_INI_PAGE_2 (u32)(FLASH_ADR_INI_PAGE_1-0x1000)
- #define FLASH_ADR_INI_PAGE_3 (u32)(FLASH_ADR_INI_PAGE_2-0x1000)
- #define FLASH_ADR_INI_PAGE_4 (u32)(FLASH_ADR_INI_PAGE_3-0x1000)
NRF_FSTORAGE_DEF:
- .start_addr = FLASH_ADR_INI_PAGE_4
- .end_addr = FLASH_ADR_FINAL_FLASH

Bootloader:

NRF_DFU_APP_DATA_AREA_SIZE 40960 (10 pages: 6 VIRTUAL + 4 data)

I use fstorage API (as before) in order to manage data writing and reading (preserved data)

DFU process is succesful, but after this, the problem is the same: when I try to connect again with same device (already bonded in my smartphone), it connects but services that are shown belong to DFU service:

Only if I desynchronize device in my smartphone and pair/bond it again is when I can normally connect again.

So, which is the problem? Have I been wrong?

Best regards,

Dani.

RE: Buttonless DFU with bonding

Einar Thorsrud — Thu, 23 Jan 2020 11:48:29 GMT

Hi Dani,

If you have not considered it, you can store other application data using FDS as well. If you do, you do not need to touch any fstorage API. In many cases, FDS is a better choice, but not always. It depends on what you are doing. But it is key to understand what you are doing, so that the memory layout makes sense and there is no overlapping, and that the bootloader is configured accordingly.

A bond is typically about 104 bytes (26 words), but it can vary a bit. So you can fit quite a few bonds per flash page.

RE: Buttonless DFU with bonding

Dani — Thu, 23 Jan 2020 10:18:11 GMT

Hi Einar,

First of all, thank you so much for your answers.

First, I think that I was a bit confused with FDS and fstorage.

Now, I use NRF_FSTORAGE_DEF in my application in the way I sent you. Do I need to do this? Do I need to define this boundaries?

In fact, what I really want is:

- To have 4 pages (or more) in order to preserve my application data: here I have already implemented this in order to write and read equipment parameters using functions as: nrf_fstorage_erase, nrf_fstorage_write, nrf_fstorage_read,...

- To have enough space to save bonding information allowing the device to save bonding information for several devices to which my device is able to connect to. How can I determine how much space is required for a bonding set? How can I know the number of connections (bondings) that can be saved?

Taking into account this information, which implementation do you reccommend me to follow in my application?

Best regards,

Dani.

RE: Buttonless DFU with bonding

Einar Thorsrud — Thu, 23 Jan 2020 09:52:00 GMT

Hi Dani,

The key here is to have a full overview of your memory layout, and once you do, the configuration becomes obvious.

You write that your bootloader starts at 0x000f8000. So everything above there is off limits.
Immediately below the bootloader, you find the FDS pages, (which use fstorage for low-level flash accesss). Is this the NRF_FSTORAGE_DEF you refer to, or do you also use this in your application directly? If so, this overlaps with the FDS pages and will cause problems.

If you use both fstorage directly (4 pages) and FDS (6 pages) via peer manager for storing bonds, then your memory layout should look something like this:

If this is the case and you indeed use fstorage directly, you must modify it so that it does not overlap with FDS by changing your code like this:

NRF_FSTORAGE_DEF(nrf_fstorage_t fstorage) =
{
    /* Set a handler for fstorage events. */
    .evt_handler = fstorage_evt_handler,

    /* These below are the boundaries of the flash space assigned to this instance of fstorage.
    * You must set these manually, even at runtime, before nrf_fstorage_init() is called.
    * The function nrf5_flash_end_addr_get() can be used to retrieve the last address on the
    * last page of flash available to write data. */
    .start_addr = 0x000ee000,
    .end_addr = 0x000f2000,
};

And this means that your bootloader needs to stay away from 10 pages, so NRF_DFU_APP_DATA_AREA_SIZE = 10*1024*4 = 40960.

Regarding the bootloader start address, it is whatever you set it to be in the linker settings, just as the application. And then there are a few tricks that is used to put this into the UICR register that the MBR checks. This is all done for you though, so you normally do not need to care about the details. If you are interested, you can search for "uicr_bootloader_start_address" through the SDK.

RE: Buttonless DFU with bonding

Dani — Wed, 22 Jan 2020 14:47:37 GMT

Hi Einar,

I'm using 4 pages in flash in order to preserve some application data.

In what regards Application I have the following definitions:

- FDS_VIRTUAL_PAGES 6

- Definition of the pages where I'm saving data (I have defined 6 pages but I'm only using 4. And I think there is one page that is used by the system, right?):

NRF_FSTORAGE_DEF(nrf_fstorage_t fstorage) =
{
/* Set a handler for fstorage events. */
.evt_handler = fstorage_evt_handler,

/* These below are the boundaries of the flash space assigned to this instance of fstorage.
* You must set these manually, even at runtime, before nrf_fstorage_init() is called.
* The function nrf5_flash_end_addr_get() can be used to retrieve the last address on the
* last page of flash available to write data. */
.start_addr = 0x000f4000,
.end_addr = 0x000f8000,
};

Should I consider the other pages I'm not using in this NRF_FSTORAGE_DEF call?

And in what regards bootloader I have:

- NRF_DFU_APP_DATA_AREA_SIZE 24576 (6 pages)

I have stated that Bootloader starting address is 0x000f8000. I have fixed this value but, is there any function that gives this address? (This is to have this well managed, as a problem would raise when updating softdevice and Bootloader address changes)

So, where can I have my problem?

Best regards,

Dani

RE: Buttonless DFU with bonding

Einar Thorsrud — Wed, 22 Jan 2020 11:41:37 GMT

Hi Dani,

Based on what you write here, it seems that the bonding information is overwritten when you do DFU. This can happen if the bootloader configuration does not match your applicaiton's configuration.

Bonding information is handled by the peer manager library, which stores it using FDS. The FDS pages are always located immediately below the bootloader in the memory layout. The application's sdk_config.h file has a FDS_VIRTUAL_PAGES, which is typically set to 3, but you may have modified it? A page is 4 kB.

The bootloader on the other hand, needs to be told to stay away from this memory, and not use it for temporarily storing update images (when doing dual bank updates). The bootloader's sdk_config.h file use NRF_DFU_APP_DATA_AREA_SIZE for this. This is in bytes rather than pages, so the default value of 12288 corresponds to 3 pages. If you for insstance use 5 pages in your application, the NRF_DFU_APP_DATA_AREA_SIZE should instead be set to 5*1024*4= 20480.

Einar

RE: Buttonless DFU with bonding

Dani — Tue, 21 Jan 2020 14:30:15 GMT

Hi Einar,

Yes, I have the device bonded first...

Any suggestion?

Best regards,

Dani.

RE: Buttonless DFU with bonding

Einar Thorsrud — Tue, 21 Jan 2020 12:35:37 GMT

Hi Dani,

As you noted, you must check the "bond" button since you are sharing the bond with the bootloader (if just pairing without bonding, the kets etc. are just valid for the current connection).

From where do you get GATT_INVALID_HANDLE when using nRFToolbox on the Android phone? Have you made sure to bond first, not just pair?

Einar

RE: Buttonless DFU with bonding

Dani — Tue, 21 Jan 2020 09:31:27 GMT

Hi again,

Now, with nRFConnect (desktop app) is working but I need to select the following option when pairing:

And with the nRFToolbox (Android app), it does not work: I get the message GATT_INVALID_HANDLE when starting DFU, and device gets stuck in DFU mode...

Hope you can give me any answer with all this information.

Best regards,

Dani.

RE: Buttonless DFU with bonding

Dani — Tue, 21 Jan 2020 08:20:00 GMT

I found I was getting this message because device was in debug mode. Now, it is not in debug mode but I'm getting a different message:

Trying to guess which is the problem....

RE: Buttonless DFU with bonding

Dani — Mon, 20 Jan 2020 16:39:06 GMT

Now, the application is working but when carrying out DFU process, it does not work --> when using 'nRF Connect' desktop application, I get the following message:

Any suggestion?

Best regards,

Dani

RE: Buttonless DFU with bonding

Dani — Mon, 20 Jan 2020 16:26:58 GMT

Hi again!!!!

Seems to be solved by placing 'peer_manager_init' before 'services_init' in the main menu.

Best regards,

Dani.