FDS wear-leveling not working

[Adrian Eggenberger]

Hi

I've just done a few tests with the FDS system of the SDK12.2. My observation is, that the FDS doesn't properly use all assigned flash pages to reach a maximal lifetime.

I have a configuration with the space for the FDS is set to 4 pages. The FDS will use one page as SWAP and the other three pages as DATA pages. My code is writting approximatly 30 records a ~30Bytes to the FDS and randomly updates these records. Once one page is nearly full the garbage collector copies the valid data to the SWAP page and converts the previous SWAP page into a DATA page and the erased page becomes the new SWAP page. Once the new data page is nearly full the valid records are copied back to the SWAP page and so on. In fact only two of the four pages are used.

In my opinion a good flash manager should use all assigned flash pages to reach the maximal possible lifetime. This will allow the developer to add more space to the FDS to increase lifetime.

The problem is based in the fds.c in the function "write_space_reserve" this function always starts at the same position in the flash page list.

Any thoughts how this could be improved?

Adrian

[emdi]

Hi,

if you run garbage collection when the flash is full then the swap page will be moved around as you would expect and wear leveling will be maximized.

[Adrian Eggenberger]

It could be correct that the wear leveling is ok if the GC is only called if the FDS is full. In my case I modified the FDS to follow a different strategy. I've added a function to the FDS that can be called if the system is idle. This function will check if it makes sense to cleanup one of the pages based on various thresholds. If the function returns true the classic fds_gc() may be called to cleanup the page.

[Hung Bui]

Hi Adrian,

Could you explain a little bit more on what you want to achieve with your strategy ? I guess you want to take advantage of the CPU idle time so you can do gb collection instead of waiting till the fds system is full and flash storing will have to wait for gb collection?

[Adrian Eggenberger]

Hi Hung Your guess is correct. Our systems have mostly a lot of idle time. I personally prefer the strategy to clean up to flash storage when idle time is available to avoid the FDS is full case normally. I'm aware that it can happen, but it shouldn't be the regular case.

[Hung Bui]

I see. This could be an use case we should think about. When we make our library, we usually choose to take generic solution just to make the library simple and easy to use. Other customer may expect a solution to only call gb when it's full to avoid unexpected CPU occupation from the fds.

My suggestion for you is maybe only do gb when there is no free page . Meaning when you first moved to the last free page, you start the gb collection, this will avoid using only 2 pages for swap page.

[Adrian Eggenberger]

Your suggestion was my first though as well. I have a simulation of our device where I can see the flash usage. This allows a good view to how the FDS behaves. The problem with your proposed way is that the GC process of the FDS doesn't create empty pages in most cases. If the cleaned page has still some valid records in it the cleaned up page will contain the same record. This would end in too much garbage collection.

We used previously an own implementation for the FDS management that had a different approach for GC. The records of the page to clean up were moved to normal valid pages one by one. As soon as all records are moved the page was erased. A swap page was not necessary with this approach. With this implementation the strategy was to wait until no full page was available. The GC process checked what page has most wasted space and clean it up.

I understand and like the generic thinking when creating your libraries. With the FDS this might be a bit tricky as the generic way should include a lot of possible usages of the FDS in term of file size and other parameters. A guideline or a recommendation on "How to use" the FDS incl. its garbage collection would be really helpful. The documentation in the fds.h doesn't provide the needed information.