using smp to download/upload files from fs is extremely slow in compare to custom service

[ziv123]

hello Nordic

i am using nrf52832 +and nrf52840 with ncs v2.3.0

i am using little fs in the nrfs (which are the servers), i use SMP service for DFU. 

when implementing smp service on my client and try to upload or download files to the servers i see it takes a very long time (~100kb /1min)

i wonder why it is so, what configuration might i miss ?

also when i add prints to every read of my external flash under the lfs, i see that it reads much more the 100kb, looks like it reads the same area over and over, so i wonder why does it behave like this and how can i fix it 

another issue that is less related to smp and more to fs, it looks like file system asks the flash driver to read area that is not designated to fs in the .yml fils .. any ideas on that ?

hope to read from you soon

best regards

Ziv

[AHaug]

Hi,

Could you clarify what transport you're using for the image transport? Is it BLE or UART? If it is BLE you could check out this to see https://github.com/nrfconnect/sdk-nrf/blob/788b23cf8e20e9dcb1c5b1325c2142db947efb3b/samples/common/mcumgr_bt_ota_dfu/Kconfig#L122-L134  to see how you can speed up the transport

Kind regards,
Andreas

[ziv123]

hi AHaug

i am referring to BLE only

i am not taking about dfu but uploading/downloading from fs with no regards to dfu (which probably works with internal flash and not external flash or fs).

i add a log file from rtt on the nrf52840. the prints "read flash:" log appear once in the code under the flash_read implementation in the driver.

one file is for upload and one is for download (using the mcumgr cli)

/cfs-file/__key/communityserver-discussions-components-files/4/canary_5F00_upload.txt

/cfs-file/__key/communityserver-discussions-components-files/4/canary_5F00_download.txt

this are the commands i used in cli:

 sudo ./mcumgr --conntype ble --connstring 'peer_id=cb:73:d2:ed:41:24' fs download /lfs/slim_shady.txt ~/workspace/slim_return.txt

sudo ./mcumgr --conntype ble --connstring 'peer_id=cb:73:d2:ed:41:24' fs upload ~/workspace/slim_shady.txt /lfs/slim_shady.txt  

it does not make sense to me that i get so many reads operation while uploading a file to the nrf52840.

also it does not make sense to me why same addresses are read over and over, a random address i looked in the file of the download read 118 time (this could definitely explain the delay but why is it behave like this ????

also even looking for a segment of read that correlates to the size of the entire file are read 5 times (which is only 5 probably because of the time change at the start of each line), as can be seen in the download file and in this picture:

https://devzone.nordicsemi.com/home/ziv/Pictures/Screenshots/Screenshot%20from%202023-11-12%2015-37-23.png

reading the flash so many extra time, is the reason it is extremely slow ?? maybe it does that because it reads the whole file every time but sends only a small part of it (which seems too wasteful for me to think this is how lfs and smp work together) any idea why i see this behaviour of too many readings (if i am talking about downloading a file of 100kb i see reads, from the log of external flash driver, of several mega bytes from flash which does not make sense) ? 

is there some example or guide for working with smp and fs (not dfu) ?

hope to read you soon

best regards

Ziv

[ziv123]

figured it out myself eventually

so one of the reason for the slow transmission using the SMP service with file system is cause every event that generates a fs_mgmt_file_download/upload there is a file open operation which reads the data base in search for the file path. when working with external flash, which is much bigger then internal flash, it can take a lot of read operations. so solutions:

1. there is this config CONFIG_MCUMGR_BUF_SIZE (in ncs v.2.3.0, in v2.5.0 there is a different config for that) as large as we can make it the less events calling for open file operations we will have so larger buff will make transmission much faster.

2. in ncs v.2.5.0 there was a change that avoid the multiplication of fs_open operation, so fs_open is called once in the start of transmission and another time before the end (not sure yet why it is called the second time) which is much more efective 

hope it helps someone, good luck all 

best regards

Ziv