how to allocate the MCUmgr buffers dynamically from within the application

RE: how to allocate the MCUmgr buffers dynamically from within the application

Vidar Berg — Wed, 27 Aug 2025 15:29:14 GMT

I'm not sure I understand what approach you took for your solution, especially when you mention having to make changes to the network buffer (are you referring to the actual network buffer implementation)? In any case, if you want to make a contribution, it would better to explain your proposal to the code owners to get their feedback: https://docs.zephyrproject.org/latest/contribute/guidelines.html#pull-requests-and-issues

RE: how to allocate the MCUmgr buffers dynamically from within the application

ziv123 — Wed, 27 Aug 2025 13:40:39 GMT

well i did manage to make it work by making some changes in the trasfer/smp.c and in the net buf

i was thinking of contribution but i think the following is a bit of an issue

i manage to path a pointer from my own pool to replace the static net buff.. but in that way the user might give wrong buff size (not NETBUF_SIZE*NETBUF_COUNT) .. another option is that the user will give an function pointer (according to zephyr required api, like in a callbacks) and then zephyr pools the correct size and it informs the user of the size (though user can check how much is left on his pool by his own) .. does the second approach looks nicer, is it zephyr valid in your view ?

RE: how to allocate the MCUmgr buffers dynamically from within the application

ziv123 — Mon, 21 Jul 2025 07:30:05 GMT

RE: how to allocate the MCUmgr buffers dynamically from within the application

ziv123 — Sat, 12 Jul 2025 16:18:48 GMT

Hi Vidar

[quote userid="4240" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541930"]If you need to make changes to the SDK sources, you can also propose your changes upstream in Zephyr by submitting a pull request. If accepted you would not have to maintain a patch for every new release. You may check Zephyr's contribution guidelines for more details about this process.[/quote]

contribution is one option but what i think of at first is maybe wrap the smp module with my own implementation (we did something similar with spi ) but i wonder which internal mcumgr module should i wrap as my_smp.c and my_smp.h .. if you have any ideas i would love to read your thoughts

[quote userid="4240" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541930"]If you were referring to a application defined network buffer (created with NET_BUF_POOL_DEFINE), then I think I understand what you meant by allowing the application to provide a buffer pointer.[/quote]

what i meant is a buffer i allocate in the app and then i give the net buf the pointer for this buf . is this possible using NET_BUF_POOL_DEFINE macro to achieve what i need ? .. and do i need to call it 4 times for each of the 4 buffers or give it a pointer for a buff of size = 4_buffer size )??

[quote userid="4240" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541930"]

you may try with the following configuration and check if that gives you an acceptable upload speed.

CONFIG_BT_L2CAP_TX_MTU=247

CONFIG_BT_BUF_ACL_RX_SIZE=251

CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE=1220

[/quote]

on one of my central devices i only have ble 4.0/2 so mtu=251 will not work ..

if i change my mcumgr net buf from 704 bytes to 2048 bytes i get between 20-40% increase in throughput (using fs_download we transfer 100kbytes-500kbytes , but we only receive small commands and upload rarely).. so if i could set large buffer for tx and use the space on the expense of the rx buffers that could really help. i don't think i'll get the same affect even f i could enable the MTU= 251

hope to read you soon

best regards

Ziv

RE: how to allocate the MCUmgr buffers dynamically from within the application

Vidar Berg — Wed, 09 Jul 2025 11:44:44 GMT

If you need to make changes to the SDK sources, you can also propose your changes upstream in Zephyr by submitting a pull request. If accepted you would not have to maintain a patch for every new release. You may check Zephyr's contribution guidelines for more details about this process.

[quote userid="90084" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application"]free for other uses of the app (i manage it with my own heap pool).[/quote]

I may have misunderstood what you meant by "heap" here. If you were referring to a application defined network buffer (created with NET_BUF_POOL_DEFINE), then I think I understand what you meant by allowing the application to provide a buffer pointer. However, I believe this solution would be quite specific to your requirements.

[quote userid="90084" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541799"]

and last related question .. i found out there are 4 buffers actually .. and i wonder of all of them have a footprint in memory at the same time (looks like that by ram size indication when i build) so..

is there a way they can share the memory (cause incoming and out coming are usually not parallel operations i think (might be wrong) )?

[/quote]

Yes, it's 4x by default.

[quote userid="90084" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541799"]why is it better in your view ?, what risks or problem you see in the alternative ?[/quote]

I don't see any risks, but as with many other modules in Zephyr, mcumgr initialization is handled outside of the application before reaching main(). If you were using other shared memory, such as the system heap, you could keep this separation. Whether that’s a good solution I’m not sure.

I would rather try to optimize the buffer configuration. For example, you may try with the following configuration and check if that gives you an acceptable upload speed.

CONFIG_BT_L2CAP_TX_MTU=247

CONFIG_BT_BUF_ACL_RX_SIZE=251

CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE=1220

RE: how to allocate the MCUmgr buffers dynamically from within the application

ziv123 — Tue, 08 Jul 2025 12:33:15 GMT

Hey

[quote userid="4240" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541766"]If you want these file(s) to use system heap, etc, you need to modify the implementation to use k_malloc/k_free and provide new configuration symbols to select whether you want to use dynamic or static allocation[/quote]

i don't want to change ncs internally cause we are trying to stay align with updated versions and this is not a wise practice i think

did you see my lat question (i edit it while you where answering):

is there a way they can share the memory (cause incoming and out coming are usually not parallel operations i think (might be wrong) )?

also, since my incoming messages are usually small commands and my out messages are actual data is it possible to give different size for incoming and out coming?

or maybe there is a way to give larger buffs only in case of fs_download etc. ?

[quote userid="4240" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541766"]I believe the buffer management should remain decoupled from the application.[/quote]why is it better in your view ?, what risks or problem you see in the alternative ?

RE: how to allocate the MCUmgr buffers dynamically from within the application

Vidar Berg — Tue, 08 Jul 2025 09:42:43 GMT

Hi Ziv,

You can search through the source files in the SDK tree for CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE to see where this symbol is used. If you want these file(s) to use system heap, etc, you need to modify the implementation to use k_malloc/k_free and provide new configuration symbols to select whether you want to use dynamic or static allocation.

[quote user="ziv123"] attempted to use that CONFIG_MCUMGR_TRANSPORT_BT_DYNAMIC_SVC_REGISTRATION=y[/quote]

As I said in my previous response, this option enables you to add/remove the SMP service from the attribute table at runtime. I don't see how it is relevant to what you are trying to do.

[quote user="ziv123"]i am a bit confused by terminology you used here, as i understand the buffers size i give here

CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE has a footprint in memory through all the application run and are not just allocated when smp is used .. else this ram would probably be free for use when ble is off no ?

[/quote]

The buffer allocated at build time and not freed after use, like a regular static variable.

[quote user="ziv123"]

is there a plane to add to the smp service api an option to give the buffers as pointers from the app layer ?

it seems kind of wastefull to hold that much memory when it may be used only 50% of app runtime

hope to read you soon

[/quote]

I believe the buffer management should remain decoupled from the application.

RE: how to allocate the MCUmgr buffers dynamically from within the application

ziv123 — Tue, 08 Jul 2025 09:15:59 GMT

Hey Vidar

i attempted to use that CONFIG_MCUMGR_TRANSPORT_BT_DYNAMIC_SVC_REGISTRATION=y

with that #define NET_BUF_POOL_HEAP_DEFINE(smp_pool, SMP_BUF_COUNT, SMP_BUF_SIZE, NULL);

and try register and unregister to smp each time i connect and disconect but it seems that if i set CONFIG_MCUMGR_TRANSPORT_NETBUF_SIZE=0

then i have a problem with fs_download -> this CONFIG_MCUMGR_GRP_FS_DL_CHUNK_SIZE is not defined

[quote userid="4240" url="~/f/nordic-q-a/122791/how-to-allocate-the-mcumgr-buffers-dynamically-from-within-the-application/541712"]use statically allocated memory [/quote]

i am a bit confused by terminology you used here, as i understand the buffers size i give here

is there a plane to add to the smp service api an option to give the buffers as pointers from the app layer ?

it seems kind of wasteful to hold that much memory when it may be used only 50% of app runtime

is there a way they can share the memory (cause incoming and out coming are usually not parallel operations i think (might be wrong) )?

also, since my incoming messages are usually small commands and my out messages are actual data is it possible to give different size for incoming and out coming?

hope to read you soon

best regards

Ziv

RE: how to allocate the MCUmgr buffers dynamically from within the application

Vidar Berg — Tue, 08 Jul 2025 06:42:22 GMT

Hello,

If you don't want to use statically allocated memory for the mcumgr transport, you would need to modify the implementation itself. There is no configuration option to disable the use of the statically allocated memory pool. The CONFIG_MCUMGR_TRANSPORT_BT_DYNAMIC_SVC_REGISTRATION config symbol is not relevant for this. It can be enabled if you want the ability to dynamically add/remove the SMP service in your attribute table at runtime and will likely increase the memory usage.

Best regards,

Vidar