Store variable in external flash at compile time

Hello, I'm working with nRF5340DK, and need to initialize some variables - long arrays of approximately 16 KB each

  • they must be included at compile time (without depending on any external serial transmission on runtime)
  • they must be stored in QSPI external flash MX25R64, already available in the kit
  • if it is possible, I would prefer not to use any storage system (LittleFS, NVS, FAT), but straight linker description and variables locating on specified memory regions

This post is highly related, with the only difference that it refers to internal instead of external flash, but nevertheless that post does not seem to be solved yet.

I have already enabled external flash from proj.conf :

CONFIG_NORDIC_QSPI_NOR=y

CONFIG_FLASH=y
CONFIG_FLASH_PAGE_LAYOUT=y

CONFIG_NVS=y
CONFIG_NVS_LOG_LEVEL_DBG=y
CONFIG_MPU_ALLOW_FLASH_WRITE=y

I have also enabled external flash from overlay file:

/ {
	chosen {
			nordic,pm-ext-flash = &mx25r64;
	};

};

I have created a custom .ld file and declared a memory region located in external_flash:

SECTION_PROLOGUE(my_external_flash, 0x00000000 (NOLOAD),)
{
    __my_external_flash_start = 0x00000000;
	KEEP(*(".my_external_flash*"));
    __my_external_flash_end = 0x000E0000;
} GROUP_LINK_IN(external_flash)

I have added the instruction in CMakeLists.txt to use that linker file:

zephyr_linker_sources(SECTIONS custom-sections.ld)

I have declared a variable in my source file to be located in the previously defined region:

uint32_t  __attribute__((section ("my_external_flash"))) testArray4[1024]    = {0} ;

When the size of the variable is below or equal to 4096 bytes (1024 uint32_t), I can read and write values to that variable.

But, if I raise the size by one, I get a bus fault error during run time.

Is there some obvious mistake that I am making in any of the steps?

I have also tried locating the storage partition on external flash, through my pm_static.yml file, and even after verifying the partition was correctly created and located on both partitions.yml and zephyr.dts, I keep getting the same error described above when I declare the variable in "storage" region/partition (?).

Would you have any recomendation on how to achieve this? How to locate a variable on external flash at compile time?

Another acceptable alternative, no so efficient as what we are looking for above, but still acceptable, would be to be able to create arrays to be stored at NVS located on external flash, at compile time. (which means that this initialized values must be defined and stored during compilation, without taking any primary flash or ram space). Is there a way to do this?

Thanks in advance!

Leopoldo

Parents
  • Hi Leopoldo

    It should be possible to modify the code_relocation_nocopy example to store const variables in external flash that can be easily accessed from the application, but I don't think this will work so well if you need to write to the variable as well. 

    How are you writing to the variable in your example?

    How often are you planning to write to the variable during normal operation?

    Would you be willing to share your project with me so I can try to reproduce the issue? 

    Best regards
    Torbjørn

  • Hi Torbjørn, thanks for the quick response!

    Actually the variables will be declared as const, they won't need to be written during runtime, just defined at compile time and being read at runtime. The arrays represent short sound samples, which will be read and fed to I2S buffers, and they are just part of a sound bank that won't be modified (unless in firmware updates, which will bring all the new arrays in the new binary file, but that's a sepparate issue).

    For example, our sounds.h header file (or sounds.c source file) will have multiple arrays as the next one:

    #include <stdio.h>
    
    const int16_t sound0[16000] = {-7, 4, 2, -2, -1, 9, 4, 9, 11, 10, 10, 10, 13, 
    0, 0, 1, -1, 0, -6, -3, 3, 10, 11, 9, 12, 15, 9, 13, 17, 17, 7, 2, 1, 0, 6, -4,
    5, 8, 13, 10, 14, 15, 14, 16, 13, 16, 6, 8, 7, 1, 3, 6, 8, 3, 13, 5, 6, 5, 6, 10, 
    10, 13, 12, 14, 8, 13, 4, 7, 15, 11, 7, 24, 13, 21, 17, 11, 7, 2, 5, 3, 0, 0, 2, 
    -5, -3, -3, -1, 5, 9, 11, 16, 17, 7, 11, 10, 6, 2, -2, -1, 6, -8, -1, 3, -2, -4,
    2, 6, 18, -11, 27, -42, 72, -114, 194, -957, -286, -2293, -4880, -8960, -10727,
    -11669, -14586, -15099, -17268, -18830, -20997, -22886, -22972, -22922, -25502, 
    -26030, -25494, -25707, -25618 ...

    So I will take a close look at the code_relocation_nocopy you suggested. The only annoying detail is that we are working with NRF Connect SDK 1.7.1, which is based on zephyr 2.6. Do you know by any chance if there is any incompatibility between that example developed for the latest zephyr version, and older ones?

    I'm not allowed to share the complete project, but I will prepare a simplified version that I can post here, and try to modify it based on your suggested example. I'll be back in a few days with the code, thank you very much!

    Leopoldo

  • Hi 

    lbudde said:
    If it is not too much to ask, could you give it a quick look just to make sure that what I have just said is correct regarding data location?

    Thanks for sharing the code. I tested it here and it seems to work very well in v2.2.0.

    I also verified that the data is written to the right place by reading the variables using nrfjprog:

    C:\WINDOWS\system32>nrfjprog --memrd 0x1000004e --n 32
    0x1000004E: 0001 0002 0003 0000 0000 0000 0000 0000   |................|
    0x1000005E: 0000 0000 0000 0000 0000 0000 0000 0000   |................|
    
    C:\WINDOWS\system32>nrfjprog --memrd 0x10030d8e --n 32
    0x10030D8E: 000B 000C 000D 0000 0000 0000 0000 0000   |................|
    0x10030D9E: 0000 0000 0000 0000 0000 0000 0000 0000   |................|

    lbudde said:
    I look forward to hear any usefull info regarding this, and also mark your answers as verified, as code_relocation_nocopy seems to be working perfectly!

    I still haven't heard back, and will need to follow this up next week. 

    I tried to build the sample in v1.8.0, which was the oldest version I had readily available, but was unable to get it working here unfortunately. 

    I believe part of the issue is down to the ext_mem_init.c implementation, which configures the QSPI interface correctly for XIP. I tried to change this to remove the references to pinctrl, and configure the pins in the config struct instead, but I was still unable to get the variables included in the flash (confirmed by reading the flash content using nrfjprog, showing that nothing was changed). 

    I also get some linker warnings during the build, so it might be more work to get this running in v1.7.1 than I hoped:

    c:/ncs/v1.8.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe:\zephyr\linker_zephyr_prebuilt.cmd:77: warning: memory region `EXTFLASH' not declared
    c:/ncs/v1.8.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe:\zephyr\linker_zephyr_prebuilt.cmd:404: warning: redeclaration of memory region `EXTFLASH'

    Again, I will have to investigate this further next week. If you have any progress on your end in the mean time just let me know. 

    Best regards
    Torbjørn

  • Hi Torbjørn,

    Thanks a lot for verifying the correct location of the variables in code_relocation example sent.

    I believe part of the issue is down to the ext_mem_init.c implementation, which configures the QSPI

    I got the same problem when trying to adapt the code for SDK 1.7.1. I took a different approach for trying to skip that issue by enabling the QSPI flash at prj.conf with CONFIG_NORDIC_QSPI_NOR=y, and then getting device binding in main.c with flash_dev = device_get_binding(FLASH_DEVICE) through Device Tree, I might messed up things a little bit, but basically I think I'm using the flash driver.

    I attach this code, named 1_7_1_code_relocation inside the zip file, and basically project compiles, but variables are not correctly located in external flash, and I also get warning: memory region `EXTFLASH' not declare

    On the other hand, I took a different approach removing code_relocation usage, still using a custom LD file, and declaring const variables in the defined memory region with __attribute__((section ("EXTFLASH"))), which according to the documentation is basically the same in a more "handmade" way, if I didn't get it wrong (without considering XIP). This code is named 1_7_1_no_code_relocation in the attached zip.

    With this approach, variables seem to be correctly mapped, both at the zephyr.map file, and at the variables addresses being printed at console. BUT, the values declared for the variables are apparently not being stored. Should I be using the flash_read() API for getting this values, instead of just trying to access to them? Is this approach somehow wrong and shouldn't be considered for declaring variables?

    Here are both projects attached, in case you want to try them there, I think there may not be many differences between using SDK 1.7.1 and 1.8.0. One detail to consider is that I did a little mod in the QSPI Kconfig inside zephyr directory, which was recommended in a different and older thread here at the forum.

    1_7_1.rar

    Thanks a lot for your time and patience, I'll keep you updated if I make any progress.

    Leopoldo

  • Hi Leopoldo

    Me and a colleague had a look at your code for a long while and in the end I think we got to a working stage, with a couple of caveats (I'll get to that later). 

    The code we ended up with is this one:

    1_7_1_code_relocation_modified_by_nordic.rar

    I went back to the prj.conf configuration, and removed the zephyr_linker_sources line from CMakeLists.txt (only keeping the zephyr_code_relocate line). 

    If you look at the linker script I put back in the MEMORY portion, and also added a custom section below it that I named sound_data, in order to allow you to define these variables by referencing the section, rather than putting them in a specific file. 

    Then I changed the variables to be declared using the section attribute, like this:

    const uint16_t __attribute__ ((section(".sound_data"))) sound0[SOUND_LEN] = {1, 2, 3};

    Now you should technically be able to read these variables using XIP, like in the original example, but I would not recommend this as using XIP for reading data comes with some drawbacks. In particular you are unlikely to get very good power consumption in this case, since you won't be able to disable the QSPI interface in between reads. Secondly, if you want to use DFU at any point to read and write DFU images to the external flash you won't be able to combine this with XIP in the same project. 

    In other words I would strongly recommend just using the flash API to read out the variables instead. 

    In order to do this you simply need to declare the variables in a similar way, but subtract the 0x10000000 XIP offset before issuing a read over the flash interface (since the local address in the flash starts at 0, not 0x10000000). 

    I wish I could provide a bit more context on some of the changes we ended up with, but as you might have picked up on we are sort of learning as we go here Slight smile
    Still, if you have any questions I will do my best to answer. 

    Can you give the modified example a go and see if it works better? 

    Please note that I didn't implement the flash read mechanism in the example, you would need to add this. 

    Also, it seems that flashing the XIP directly from VSCode does not work in v1.7.1. This means you need to manually flash the code from the command line using nrfjprog:

    nrfjprog --program zephyr.hex --sectorerase --verify

    You might also need to manually erase the flash device:

    nrfjprog --qspieraseall

    Best regards
    Torbjørn 

  • Hi Torbjørn,

    I've built your uploaded code, and added flash_read functions as you recomended, and everything works perfectly on SDK 1.7.1!

    Thanks a lot for all your help, this is exactly what we were looking for, and also thanks on the suggestion on using flash API instead of XIP on variables reading.

    I leave here a .zip with my final version, which is exactly the same code you uploaded, with flash_read added in main.c, in case there is someone else who also needs this in the future.1_7_1_code_relocation_t.rar

    Thanks again for your - and your collegue's - time, this has been really usefull for us.

    Regards,

    Leopoldo

  • Hi Leopoldo

    Great to hear that you got it working well Slight smile

    Thanks for sharing the final code with the community, I am sure this can be useful for others. Like I said this was a good learning experience for us as well, I can foresee many applications where it is handy to store data in external flash in this way. 

    The best of luck with your project!

    Best regards
    Torbjørn

Reply
  • Hi Leopoldo

    Great to hear that you got it working well Slight smile

    Thanks for sharing the final code with the community, I am sure this can be useful for others. Like I said this was a good learning experience for us as well, I can foresee many applications where it is handy to store data in external flash in this way. 

    The best of luck with your project!

    Best regards
    Torbjørn

Children
No Data
Related