Store variable in external flash at compile time

RE: Store variable in external flash at compile time

ovrebekk — Tue, 10 Jan 2023 09:16:22 GMT

Hi Leopoldo

Great to hear that you got it working well 🙂

Thanks for sharing the final code with the community, I am sure this can be useful for others. Like I said this was a good learning experience for us as well, I can foresee many applications where it is handy to store data in external flash in this way.

The best of luck with your project!

Best regards
Torbjørn

RE: Store variable in external flash at compile time

lbudde — Mon, 09 Jan 2023 21:14:57 GMT

Hi Torbjørn,

I've built your uploaded code, and added flash_read functions as you recomended, and everything works perfectly on SDK 1.7.1!

Thanks a lot for all your help, this is exactly what we were looking for, and also thanks on the suggestion on using flash API instead of XIP on variables reading.

I leave here a .zip with my final version, which is exactly the same code you uploaded, with flash_read added in main.c, in case there is someone else who also needs this in the future.devzone.nordicsemi.com/.../1_5F00_7_5F00_1_5F00_code_5F00_relocation_5F00_t.rar

Thanks again for your - and your collegue's - time, this has been really usefull for us.

Regards,

Leopoldo

RE: Store variable in external flash at compile time

ovrebekk — Mon, 09 Jan 2023 14:00:40 GMT

Hi Leopoldo

Me and a colleague had a look at your code for a long while and in the end I think we got to a working stage, with a couple of caveats (I'll get to that later).

The code we ended up with is this one:

devzone.nordicsemi.com/.../1_5F00_7_5F00_1_5F00_code_5F00_relocation_5F00_modified_5F00_by_5F00_nordic.rar

I went back to the prj.conf configuration, and removed the zephyr_linker_sources line from CMakeLists.txt (only keeping the zephyr_code_relocate line).

If you look at the linker script I put back in the MEMORY portion, and also added a custom section below it that I named sound_data, in order to allow you to define these variables by referencing the section, rather than putting them in a specific file.

Then I changed the variables to be declared using the section attribute, like this:

const uint16_t __attribute__ ((section(".sound_data"))) sound0[SOUND_LEN] = {1, 2, 3};

Now you should technically be able to read these variables using XIP, like in the original example, but I would not recommend this as using XIP for reading data comes with some drawbacks. In particular you are unlikely to get very good power consumption in this case, since you won't be able to disable the QSPI interface in between reads. Secondly, if you want to use DFU at any point to read and write DFU images to the external flash you won't be able to combine this with XIP in the same project.

In other words I would strongly recommend just using the flash API to read out the variables instead.

In order to do this you simply need to declare the variables in a similar way, but subtract the 0x10000000 XIP offset before issuing a read over the flash interface (since the local address in the flash starts at 0, not 0x10000000).

I wish I could provide a bit more context on some of the changes we ended up with, but as you might have picked up on we are sort of learning as we go here 🙂
Still, if you have any questions I will do my best to answer.

Can you give the modified example a go and see if it works better?

Please note that I didn't implement the flash read mechanism in the example, you would need to add this.

Also, it seems that flashing the XIP directly from VSCode does not work in v1.7.1. This means you need to manually flash the code from the command line using nrfjprog:

nrfjprog --program zephyr.hex --sectorerase --verify

You might also need to manually erase the flash device:

nrfjprog --qspieraseall

Best regards
Torbjørn

RE: Store variable in external flash at compile time

lbudde — Sat, 07 Jan 2023 22:24:55 GMT

Hi Torbjørn,

Thanks a lot for verifying the correct location of the variables in code_relocation example sent.

[quote userid="2116" url="~/f/nordic-q-a/95302/store-variable-in-external-flash-at-compile-time/403613"]I believe part of the issue is down to the ext_mem_init.c implementation, which configures the QSPI[/quote]

I got the same problem when trying to adapt the code for SDK 1.7.1. I took a different approach for trying to skip that issue by enabling the QSPI flash at prj.conf with CONFIG_NORDIC_QSPI_NOR=y, and then getting device binding in main.c with flash_dev = device_get_binding(FLASH_DEVICE) through Device Tree, I might messed up things a little bit, but basically I think I'm using the flash driver.

I attach this code, named 1_7_1_code_relocation inside the zip file, and basically project compiles, but variables are not correctly located in external flash, and I also get warning: memory region `EXTFLASH' not declare

On the other hand, I took a different approach removing code_relocation usage, still using a custom LD file, and declaring const variables in the defined memory region with __attribute__((section ("EXTFLASH"))), which according to the documentation is basically the same in a more "handmade" way, if I didn't get it wrong (without considering XIP). This code is named 1_7_1_no_code_relocation in the attached zip.

With this approach, variables seem to be correctly mapped, both at the zephyr.map file, and at the variables addresses being printed at console. BUT, the values declared for the variables are apparently not being stored. Should I be using the flash_read() API for getting this values, instead of just trying to access to them? Is this approach somehow wrong and shouldn't be considered for declaring variables?

Here are both projects attached, in case you want to try them there, I think there may not be many differences between using SDK 1.7.1 and 1.8.0. One detail to consider is that I did a little mod in the QSPI Kconfig inside zephyr directory, which was recommended in a different and older thread here at the forum.

devzone.nordicsemi.com/.../1_5F00_7_5F00_1.rar

Thanks a lot for your time and patience, I'll keep you updated if I make any progress.

Leopoldo

RE: Store variable in external flash at compile time

ovrebekk — Fri, 06 Jan 2023 14:30:53 GMT

[quote user="lbudde"]If it is not too much to ask, could you give it a quick look just to make sure that what I have just said is correct regarding data location?[/quote]

Thanks for sharing the code. I tested it here and it seems to work very well in v2.2.0.

I also verified that the data is written to the right place by reading the variables using nrfjprog:

C:\WINDOWS\system32>nrfjprog --memrd 0x1000004e --n 32
0x1000004E: 0001 0002 0003 0000 0000 0000 0000 0000   |................|
0x1000005E: 0000 0000 0000 0000 0000 0000 0000 0000   |................|

C:\WINDOWS\system32>nrfjprog --memrd 0x10030d8e --n 32
0x10030D8E: 000B 000C 000D 0000 0000 0000 0000 0000   |................|
0x10030D9E: 0000 0000 0000 0000 0000 0000 0000 0000   |................|

[quote user="lbudde"]I look forward to hear any usefull info regarding this, and also mark your answers as verified, as code_relocation_nocopy seems to be working perfectly![/quote]

I still haven't heard back, and will need to follow this up next week.

I tried to build the sample in v1.8.0, which was the oldest version I had readily available, but was unable to get it working here unfortunately.

I believe part of the issue is down to the ext_mem_init.c implementation, which configures the QSPI interface correctly for XIP. I tried to change this to remove the references to pinctrl, and configure the pins in the config struct instead, but I was still unable to get the variables included in the flash (confirmed by reading the flash content using nrfjprog, showing that nothing was changed).

I also get some linker warnings during the build, so it might be more work to get this running in v1.7.1 than I hoped:

c:/ncs/v1.8.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe:\zephyr\linker_zephyr_prebuilt.cmd:77: warning: memory region `EXTFLASH' not declared
c:/ncs/v1.8.0/toolchain/opt/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe:\zephyr\linker_zephyr_prebuilt.cmd:404: warning: redeclaration of memory region `EXTFLASH'

Again, I will have to investigate this further next week. If you have any progress on your end in the mean time just let me know.

Best regards
Torbjørn

RE: Store variable in external flash at compile time

lbudde — Thu, 05 Jan 2023 22:50:19 GMT

Hi Torbjørn,

[quote userid="2116" url="~/f/nordic-q-a/95302/store-variable-in-external-flash-at-compile-time/403346"]Would using the flash interface be an option?[/quote]

That sounds good, I haven't tried it yet because I got a bit messed up between linker files and code relocation. I will check it out, thanks for the advice.

[quote userid="2116" url="~/f/nordic-q-a/95302/store-variable-in-external-flash-at-compile-time/403346"]For playing back data over I2S you will probably have to store the data temporarily in RAM anyway, since the I2S EasyDMA controller can not read data from external flash directly. [/quote]

Yes there is no problem in storing temporarily some data in buffers located in RAM, we can afford that between I2S interrupts.

I downloaded nRF Connect SDK 2.1.1 and tested the code_relocation_nocopy, modified it a little bit to declare some const big arrays located in external flash, and everything seems to work fine, their address show they are effectively located in ext_flash, and they are accessible from main() application located in primary_flash. If it is not too much to ask, could you give it a quick look just to make sure that what I have just said is correct regarding data location?

If that's the case, I will stick to this approach as you suggested in your first response, and try to addapt it to our current nRF connect SDK 1.7.1 (otherwise I would have to migrate the complete project, which may take longer).

[quote userid="2116" url="~/f/nordic-q-a/95302/store-variable-in-external-flash-at-compile-time/403346"]I see that the example has changed, but I doubt there are any changes to the build system that would make it impossible to do something similar. I will check with the developers to be sure, and get back to you.[/quote]

I look forward to hear any usefull info regarding this, and also mark your answers as verified, as code_relocation_nocopy seems to be working perfectly!

Thanks a lot for your help! 🙌

devzone.nordicsemi.com/.../code_5F00_relocation_5F00_nocopy_5F00_v2.rar

RE: Store variable in external flash at compile time

ovrebekk — Thu, 05 Jan 2023 09:59:56 GMT

Hi Leopoldo

Would using the flash interface be an option?

This is a relatively low level interface, but for reading arrays from external flash it should be more than sufficient.

For playing back data over I2S you will probably have to store the data temporarily in RAM anyway, since the I2S EasyDMA controller can not read data from external flash directly.

What you would lose compared to the custom linker script method is that you would need to find some other way to determine at which address the various sounds files are stored.

[quote user="lbudde"]Do you know by any chance if there is any incompatibility between that example developed for the latest zephyr version, and older ones?[/quote]

I see that the example has changed, but I doubt there are any changes to the build system that would make it impossible to do something similar. I will check with the developers to be sure, and get back to you.

Best regards
Torbjørn

RE: Store variable in external flash at compile time

lbudde — Tue, 03 Jan 2023 19:04:32 GMT

Hi Torbjørn, thanks for the quick response!

Actually the variables will be declared as const, they won't need to be written during runtime, just defined at compile time and being read at runtime. The arrays represent short sound samples, which will be read and fed to I2S buffers, and they are just part of a sound bank that won't be modified (unless in firmware updates, which will bring all the new arrays in the new binary file, but that's a sepparate issue).

For example, our sounds.h header file (or sounds.c source file) will have multiple arrays as the next one:

#include <stdio.h>

const int16_t sound0[16000] = {-7, 4, 2, -2, -1, 9, 4, 9, 11, 10, 10, 10, 13, 
0, 0, 1, -1, 0, -6, -3, 3, 10, 11, 9, 12, 15, 9, 13, 17, 17, 7, 2, 1, 0, 6, -4,
5, 8, 13, 10, 14, 15, 14, 16, 13, 16, 6, 8, 7, 1, 3, 6, 8, 3, 13, 5, 6, 5, 6, 10, 
10, 13, 12, 14, 8, 13, 4, 7, 15, 11, 7, 24, 13, 21, 17, 11, 7, 2, 5, 3, 0, 0, 2, 
-5, -3, -3, -1, 5, 9, 11, 16, 17, 7, 11, 10, 6, 2, -2, -1, 6, -8, -1, 3, -2, -4,
2, 6, 18, -11, 27, -42, 72, -114, 194, -957, -286, -2293, -4880, -8960, -10727,
-11669, -14586, -15099, -17268, -18830, -20997, -22886, -22972, -22922, -25502, 
-26030, -25494, -25707, -25618 ...

So I will take a close look at the code_relocation_nocopy you suggested. The only annoying detail is that we are working with NRF Connect SDK 1.7.1, which is based on zephyr 2.6. Do you know by any chance if there is any incompatibility between that example developed for the latest zephyr version, and older ones?

I'm not allowed to share the complete project, but I will prepare a simplified version that I can post here, and try to modify it based on your suggested example. I'll be back in a few days with the code, thank you very much!

Leopoldo

RE: Store variable in external flash at compile time

ovrebekk — Tue, 03 Jan 2023 13:41:54 GMT

Hi Leopoldo

It should be possible to modify the code_relocation_nocopy example to store const variables in external flash that can be easily accessed from the application, but I don't think this will work so well if you need to write to the variable as well.

How are you writing to the variable in your example?

How often are you planning to write to the variable during normal operation?

Would you be willing to share your project with me so I can try to reproduce the issue?

Best regards
Torbjørn