How to port the SMP Service to the peripheral_uart to support DFU

RE: How to port the SMP Service to the peripheral_uart to support DFU

AHaug — Mon, 06 Nov 2023 13:40:43 GMT

Hi,

Unfortunately BLE DFU on only one slot is not supported in NCS, so the only options are to keep minimizing the application by optimization and/or feature removal or to add an external flash for BLE DFU if you don't wish to do serial recovery

Kind regards,
Andreas

RE: How to port the SMP Service to the peripheral_uart to support DFU

Tom.Fu — Mon, 06 Nov 2023 04:34:08 GMT

Hi AHaug

Many thanks for the detail guiding.

When I built this project for the nRF52840, it got the memory report like the screenshot below. It has exceeded 240kB by 55.79kB.

After I tried, I found that it is difficult to squeeze the size of the firmware to within ~240kB by removing x amount of features.

After I added the below kconfgs in the prj.conf, it could not be built successfully.

#Squeeze 
CONFIG_I2C=n
CONFIG_SPI=n
CONFIG_SIZE_OPTIMIZATIONS=y
CONFIG_ISR_STACK_SIZE=1024
CONFIG_WATCHDOG=n
CONFIG_CONSOLE=n
CONFIG_UART_CONSOLE=n
CONFIG_STDOUT_CONSOLE=n
CONFIG_PRINTK=n
CONFIG_EARLY_CONSOLE=n
CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE=512
CONFIG_IDLE_STACK_SIZE=64
CONFIG_MAIN_STACK_SIZE=256
CONFIG_PRIVILEGED_STACK_SIZE=0
CONFIG_MPSL_WORK_STACK_SIZE=256
CONFIG_ISR_STACK_SIZE=128
 
CONFIG_BT_HCI_TX_STACK_SIZE_WITH_PROMPT=y
CONFIG_BT_HCI_TX_STACK_SIZE=256
CONFIG_BT_RX_STACK_SIZE=256
CONFIG_BT_BUF_EVT_DISCARDABLE_SIZE=13
CONFIG_BT_BUF_EVT_DISCARDABLE_COUNT=1
CONFIG_BT_BUF_EVT_RX_COUNT=1
CONFIG_BT_CONN_TX_MAX=2
CONFIG_BT_L2CAP_TX_BUF_COUNT=3
CONFIG_BT_BUF_ACL_TX_COUNT=1
CONFIG_BT_BUF_ACL_TX_SIZE=64

Below is the bulit log.

-- west build: generating a build system
Loading Zephyr default modules (Zephyr base).
-- Application: C:/ncs/VSC2.4.0/peripheral_uart_smp
-- CMake version: 3.20.5
-- Found Python3: C:/ncs/toolchains/31f4403e35/opt/bin/python.exe (found suitable exact version "3.8.2") found components: Interpreter 
-- Cache files will be written to: C:/ncs/v2.4.0/zephyr/.cache
-- Zephyr version: 3.3.99 (C:/ncs/v2.4.0/zephyr)
-- Found west (found suitable version "1.0.0", minimum required is "0.7.1")
-- Board: nrf52dk_nrf52832
-- Found host-tools: zephyr 0.16.0 (C:/ncs/toolchains/31f4403e35/opt/zephyr-sdk)
-- Found toolchain: zephyr 0.16.0 (C:/ncs/toolchains/31f4403e35/opt/zephyr-sdk)
-- Found Dtc: C:/ncs/toolchains/31f4403e35/opt/bin/dtc.exe (found suitable version "1.4.7", minimum required is "1.4.6") 
-- Found BOARD.dts: C:/ncs/v2.4.0/zephyr/boards/arm/nrf52dk_nrf52832/nrf52dk_nrf52832.dts
-- Found devicetree overlay: c:/ncs/VSC2.4.0/peripheral_uart_smp/app.overlay
-- Generated zephyr.dts: C:/ncs/VSC2.4.0/peripheral_uart_smp/build_1/zephyr/zephyr.dts
-- Generated devicetree_generated.h: C:/ncs/VSC2.4.0/peripheral_uart_smp/build_1/zephyr/include/generated/devicetree_generated.h
-- Including generated dts.cmake file: C:/ncs/VSC2.4.0/peripheral_uart_smp/build_1/zephyr/dts.cmake

warning: BT_RX_STACK_SIZE (defined at C:/ncs/v2.4.0/nrf\subsys\bluetooth\controller/Kconfig:73,
C:/ncs/v2.4.0/zephyr/subsys/bluetooth/host/Kconfig:125, subsys/bluetooth\host/Kconfig:125) was
assigned the value '256' but got the value '2200'. See
http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_BT_RX_STACK_SIZE and/or look up
BT_RX_STACK_SIZE in the menuconfig/guiconfig interface. The Application Development Primer, Setting
Configuration Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful
too.


warning: BT_BUF_EVT_RX_COUNT (defined at C:/ncs/v2.4.0/zephyr/subsys/bluetooth/common/Kconfig:127,
subsys/bluetooth\common/Kconfig:127) was assigned the value '1' but got the value '10'. See
http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_BT_BUF_EVT_RX_COUNT and/or look up
BT_BUF_EVT_RX_COUNT in the menuconfig/guiconfig interface. The Application Development Primer,
Setting Configuration Values, and Kconfig - Tips and Best Practices sections of the manual might be
helpful too.


warning: BT_BUF_EVT_DISCARDABLE_SIZE (defined at
C:/ncs/v2.4.0/zephyr/subsys/bluetooth/common/Kconfig:137, subsys/bluetooth\common/Kconfig:137) was
assigned the value '13' but got the value '43'. See
http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_BT_BUF_EVT_DISCARDABLE_SIZE and/or look up
BT_BUF_EVT_DISCARDABLE_SIZE in the menuconfig/guiconfig interface. The Application Development
Primer, Setting Configuration Values, and Kconfig - Tips and Best Practices sections of the manual
might be helpful too.


warning: BT_CONN_TX_MAX (defined at C:/ncs/v2.4.0/zephyr/subsys/bluetooth/host/Kconfig:293,
Parsing C:/ncs/VSC2.4.0/peripheral_uart_smp/Kconfig
Loaded configuration 'C:/ncs/v2.4.0/zephyr/boards/arm/nrf52dk_nrf52832/nrf52dk_nrf52832_defconfig'
Merged configuration 'c:/ncs/VSC2.4.0/peripheral_uart_smp/prj.conf'
Merged configuration 'c:/ncs/VSC2.4.0/peripheral_uart_smp/overlay-bt.conf'
subsys/bluetooth\host/Kconfig:293) was assigned the value '2' but got the value '3'. See
http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_BT_CONN_TX_MAX and/or look up
BT_CONN_TX_MAX in the menuconfig/guiconfig interface. The Application Development Primer, Setting
Configuration Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful
too.


warning: PRINTK (defined at subsys/debug/Kconfig:202) was assigned the value 'n' but got the value
'y'. See http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_PRINTK and/or look up PRINTK in
the menuconfig/guiconfig interface. The Application Development Primer, Setting Configuration
Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful too.


warning: EARLY_CONSOLE (defined at subsys/debug/Kconfig:221) was assigned the value 'n' but got the
value 'y'. See http://docs.zephyrproject.org/latest/kconfig.html#CONFIG_EARLY_CONSOLE and/or look up
EARLY_CONSOLE in the menuconfig/guiconfig interface. The Application Development Primer, Setting
Configuration Values, and Kconfig - Tips and Best Practices sections of the manual might be helpful
too.


warning: user value 256 on the int symbol BT_RX_STACK_SIZE (defined at C:/ncs/v2.4.0/nrf\subsys\bluetooth\controller/Kconfig:73, C:/ncs/v2.4.0/zephyr/subsys/bluetooth/host/Kconfig:125, subsys/bluetooth\host/Kconfig:125) ignored due to being outside the active range ([1024, 65536]) -- falling back on defaults

warning: user value 1 on the int symbol BT_BUF_EVT_RX_COUNT (defined at C:/ncs/v2.4.0/zephyr/subsys/bluetooth/common/Kconfig:127, subsys/bluetooth\common/Kconfig:127) ignored due to being outside the active range ([2, 255]) -- falling back on defaults

warning: user value 13 on the int symbol BT_BUF_EVT_DISCARDABLE_SIZE (defined at C:/ncs/v2.4.0/zephyr/subsys/bluetooth/common/Kconfig:137, subsys/bluetooth\common/Kconfig:137) ignored due to being outside the active range ([43, 255]) -- falling back on defaults

warning: user value 2 on the int symbol BT_CONN_TX_MAX (defined at C:/ncs/v2.4.0/zephyr/subsys/bluetooth/host/Kconfig:293, subsys/bluetooth\host/Kconfig:293) ignored due to being outside the active range ([3, 255]) -- falling back on defaults

error: Aborting due to Kconfig warnings

CMake Error at C:/ncs/v2.4.0/zephyr/cmake/modules/kconfig.cmake:339 (message):
  command failed with return code: 1
Call Stack (most recent call first):
  C:/ncs/v2.4.0/nrf/cmake/modules/kconfig.cmake:29 (include)
  C:/ncs/v2.4.0/zephyr/cmake/modules/zephyr_default.cmake:115 (include)
  C:/ncs/v2.4.0/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:66 (include)
  C:/ncs/v2.4.0/zephyr/share/zephyr-package/cmake/ZephyrConfig.cmake:92 (include_boilerplate)
  CMakeLists.txt:8 (find_package)


-- Configuring incomplete, errors occurred!
FATAL ERROR: command exited with status 1: 'C:\ncs\toolchains\31f4403e35\opt\bin\cmake.EXE' '-DWEST_PYTHON=C:\ncs\toolchains\31f4403e35\opt\bin\python.exe' '-Bc:\ncs\VSC2.4.0\peripheral_uart_smp\build_1' -GNinja -DBOARD=nrf52dk_nrf52832 -DNCS_TOOLCHAIN_VERSION:STRING=NONE -DBOARD_ROOT:STRING=c:/ncs/vsc2.4.0/peripheral_uart_smp -DCONF_FILE:STRING=c:/ncs/VSC2.4.0/peripheral_uart_smp/prj.conf -DOVERLAY_CONFIG:STRING=c:/ncs/VSC2.4.0/peripheral_uart_smp/overlay-bt.conf -DDTC_OVERLAY_FILE:STRING=c:/ncs/VSC2.4.0/peripheral_uart_smp/app.overlay '-Sc:\ncs\VSC2.4.0\peripheral_uart_smp'

 *  The terminal process terminated with exit code: 1. 
 *  Terminal will be reused by tasks, press any key to close it.

For the cost and the customer requirement reason, we didn't want to used the external flash or use serial recovery. Is it possible to use the one slot for the BLE DFU? If yes, may be we can try it and how to implementation? Many thanks.

RE: How to port the SMP Service to the peripheral_uart to support DFU

AHaug — Tue, 24 Oct 2023 08:38:44 GMT

[quote user="Tom.Fu"]But I could not find the file of 'ninja partition_manager report' like the screenshot below showing. Did you mean the partitions.yml file? Below code is the content of the partitions.yml file．[/quote]

No worries, I will guide you to how to do this, but the partition manager is fine for now

For the partition manager report, you can either use cmd when building and enter 'west build -t partition_manager_report', or navigate to <your_project>/build and enter 'ninja partition_manager report'

The output should be similar to this

The other option which is built in the extension is found under actions and named 'Memory report'

[quote user="Tom.Fu"]Because of the cost of the nRF52832 is cheaper than nRF52840, so we still would like to use the nRF52832 for the project. Is there any other way to integrate the spm service to the peripheral_uart for the nRF52832? Many thanks.[/quote]

That is understandable and I will guide you to some pointers for how to do this, but there are a couple of points that are essential for your understanding to reach the goal. You might already know this, but I'm stating it nonetheless

The error you see is a memory related issue that is caused by that the application with DFU support is too large for the device you're developing with
When you add DFU support to your application, you will in practice reduce the available flash you have for an application to less than half of the total flash on the device.

Why do you half the available flash?

This is explained in https://github.com/hellesvik-nordic/samples_for_nrf_connect_sdk/tree/main/bootloader_samples/smp and https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/config_and_build/bootloaders_and_dfu/index.html

As seen in this section, what you do is adding a secondary application slot where you will temporary hold the new firmware. This needs to be atleast as large as the primary application slot, + the swap area illustrated

To resolve item 2

You will have to

Optimize the application w.r.t ROM and RAM, see the following links for some resources for how to do this
- https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/test_and_optimize.html
- Debunking misconceptions: A technical analysis of nRF5 SDK and nRF Connect SDK
Optimize MCUboot:
- Investigate the nRF Machine Learning application to see how this is done:https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/applications/machine_learning/README.html
Optimize by further removing other features from the sample
Add an external flash where you have your secondary mcuboot image
- See this sample for how to do this https://github.com/hellesvik-nordic/samples_for_nrf_connect_sdk/tree/main/bootloader_samples/smp/mcuboot_smp_uart_feat_external_flash
And the final method which looses quite a lot of safety w.r.t redundancy is to use serial recovery without a secondary slot
- See these samples for how to do this https://github.com/hellesvik-nordic/samples_for_nrf_connect_sdk/tree/main/bootloader_samples/serial_recovery which uses one application slot instead of two slots which the smp server uses

Recommended steps:

See how large the application is when you build for the nRF52840. Is it possible to fit this within ~240kB? (This is just a rough number, and not necessarily accurate)
- If thats an "absoluete no", then move on to either add external flash or use serial recovery
- If it is a "yes, I can remove x amount of features" by using the guides I've linked, then try to squeeze the size of the firmware down to fit
I also recommend that you use static partitioning https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/scripts/partition_manager/partition_manager.html#adding-a-static-partition

This is a lot of resources and information, so please take the time to go through the samples and docs to properly understand them and see if you can apply them to your project

If you have any other questions after doing this, then please reach out and I'll do my best to answer them

Kind regards,
Andreas

RE: How to port the SMP Service to the peripheral_uart to support DFU

Tom.Fu — Tue, 24 Oct 2023 03:10:03 GMT

Hi AHaug

Many thanks for supporting and guiding.

It could be compiled successfully for nRF52840DK.

But I could not find the file of 'ninja partition_manager report' like the screenshot below showing. Did you mean the partitions.yml file? Below code is the content of the partitions.yml file．

app:
  address: 0xc200
  end_address: 0x42000
  region: flash_primary
  size: 0x35e00
littlefs_storage:
  address: 0x7a000
  end_address: 0x80000
  placement:
    before:
    - end
  region: flash_primary
  size: 0x6000
mcuboot:
  address: 0x0
  end_address: 0xc000
  placement:
    before:
    - mcuboot_primary
  region: flash_primary
  size: 0xc000
mcuboot_pad:
  address: 0xc000
  end_address: 0xc200
  placement:
    align:
      start: 0x1000
    before:
    - mcuboot_primary_app
  region: flash_primary
  size: 0x200
mcuboot_primary:
  address: 0xc000
  end_address: 0x42000
  orig_span: &id001
  - app
  - mcuboot_pad
  region: flash_primary
  sharers: 0x1
  size: 0x36000
  span: *id001
mcuboot_primary_app:
  address: 0xc200
  end_address: 0x42000
  orig_span: &id002
  - app
  region: flash_primary
  size: 0x35e00
  span: *id002
mcuboot_secondary:
  address: 0x42000
  end_address: 0x78000
  placement:
    after:
    - mcuboot_primary
    align:
      start: 0x1000
  region: flash_primary
  share_size:
  - mcuboot_primary
  size: 0x36000
settings_storage:
  address: 0x78000
  end_address: 0x7a000
  placement:
    align:
      start: 0x1000
    before:
    - end
  region: flash_primary
  size: 0x2000
sram_primary:
  address: 0x20000000
  end_address: 0x20010000
  region: sram_primary
  size: 0x10000

Sorry, I also could not find the icon in the below screenshot for the built in tool in the extension.

Because of the cost of the nRF52832 is cheaper than nRF52840, so we still would like to use the nRF52832 for the project. Is there any other way to integrate the spm service to the peripheral_uart for the nRF52832? Many thanks.

RE: How to port the SMP Service to the peripheral_uart to support DFU

AHaug — Mon, 23 Oct 2023 12:16:19 GMT

Hi,

It looks like you've done the merging of the samples correct, but the issue comes from ROM and RAM restrictions from your hardware. I assume you're using a 512kB ROM / 64kB RAM version of the nRF52832?

Could you try to compile it for the nRF52840DK and see if the errors are still present? i.e build target 'nrf52840dk_nrf528440'
Could you either
1. Navigate to the build folder of your project in ncs and enter 'ninja partition_manager report' and post the result here, or
2. Use the built in tool in the extension to show me the partitioning?

Kind regards,
Andreas