using smp to download/upload files from fs is extremely slow in compare to custom service

RE: using smp to download/upload files from fs is extremely slow in compare to custom service

ziv123 — Mon, 27 Nov 2023 09:05:28 GMT

figured it out myself eventually

so one of the reason for the slow transmission using the SMP service with file system is cause every event that generates a fs_mgmt_file_download/upload there is a file open operation which reads the data base in search for the file path. when working with external flash, which is much bigger then internal flash, it can take a lot of read operations. so solutions:

1. there is this config CONFIG_MCUMGR_BUF_SIZE (in ncs v.2.3.0, in v2.5.0 there is a different config for that) as large as we can make it the less events calling for open file operations we will have so larger buff will make transmission much faster.

2. in ncs v.2.5.0 there was a change that avoid the multiplication of fs_open operation, so fs_open is called once in the start of transmission and another time before the end (not sure yet why it is called the second time) which is much more efective

hope it helps someone, good luck all

best regards

Ziv

RE: using smp to download/upload files from fs is extremely slow in compare to custom service

ziv123 — Thu, 09 Nov 2023 12:29:58 GMT

hi AHaug

i am referring to BLE only

i am not taking about dfu but uploading/downloading from fs with no regards to dfu (which probably works with internal flash and not external flash or fs).

i add a log file from rtt on the nrf52840. the prints "read flash:" log appear once in the code under the flash_read implementation in the driver.

one file is for upload and one is for download (using the mcumgr cli)

/cfs-file/__key/communityserver-discussions-components-files/4/canary_5F00_upload.txt

/cfs-file/__key/communityserver-discussions-components-files/4/canary_5F00_download.txt

this are the commands i used in cli:

 sudo ./mcumgr --conntype ble --connstring 'peer_id=cb:73:d2:ed:41:24' fs download /lfs/slim_shady.txt ~/workspace/slim_return.txt

sudo ./mcumgr --conntype ble --connstring 'peer_id=cb:73:d2:ed:41:24' fs upload ~/workspace/slim_shady.txt /lfs/slim_shady.txt

it does not make sense to me that i get so many reads operation while uploading a file to the nrf52840.

also it does not make sense to me why same addresses are read over and over, a random address i looked in the file of the download read 118 time (this could definitely explain the delay but why is it behave like this ????

also even looking for a segment of read that correlates to the size of the entire file are read 5 times (which is only 5 probably because of the time change at the start of each line), as can be seen in the download file and in this picture:

https://devzone.nordicsemi.com/home/ziv/Pictures/Screenshots/Screenshot%20from%202023-11-12%2015-37-23.png

reading the flash so many extra time, is the reason it is extremely slow ?? maybe it does that because it reads the whole file every time but sends only a small part of it (which seems too wasteful for me to think this is how lfs and smp work together) any idea why i see this behaviour of too many readings (if i am talking about downloading a file of 100kb i see reads, from the log of external flash driver, of several mega bytes from flash which does not make sense) ?

is there some example or guide for working with smp and fs (not dfu) ?

hope to read you soon

best regards

Ziv

RE: using smp to download/upload files from fs is extremely slow in compare to custom service

AHaug — Thu, 09 Nov 2023 10:24:32 GMT

Hi,

Could you clarify what transport you're using for the image transport? Is it BLE or UART? If it is BLE you could check out this to see https://github.com/nrfconnect/sdk-nrf/blob/788b23cf8e20e9dcb1c5b1325c2142db947efb3b/samples/common/mcumgr_bt_ota_dfu/Kconfig#L122-L134 to see how you can speed up the transport

Kind regards,
Andreas