USB MSC Corruption

RE: USB MSC Corruption

Edvin — Wed, 15 Jan 2020 06:45:41 GMT

Please create a new ticket if you have this issue.

Best regards,

Edvin

RE: USB MSC Corruption

Nandi — Wed, 15 Jan 2020 03:26:27 GMT

This is my previous question link

devzone.nordicsemi.com/.../the-fat32-fs-isn-t-working-properly-when-writeback-mode-is-turned-on

RE: USB MSC Corruption

Nandi — Wed, 15 Jan 2020 02:22:06 GMT

Jeff, Edvin,

My test here is like NRF_BLOCK_DEV_QSPI_FLAG_CACHE_WRITEBACK This function has not been implemented well. I have mentioned a problem before, and fae told me that there is no problem and I will not follow up. You can help me to see if this Flag is the cause

tks,

Nandi

RE: USB MSC Corruption

Serhii B. — Mon, 02 Dec 2019 09:09:45 GMT

Hi, has the issue resolved?

RE: USB MSC Corruption

junpu — Tue, 25 Jun 2019 00:20:17 GMT

HI, I got the same problem,

sdk :nRF5_SDK_15.2.0_9412b96

revision of the nRF52840 ： cortex-m4 nrf52840_xxAA 1024KB+4KB 256KB PCA1005

I took up the problem on

https://devzone.nordicsemi.com/f/nordic-q-a/48535/problem-about-fatfs-and-qspi-on-nrf52840

RE: USB MSC Corruption

JeffW — Mon, 24 Jun 2019 14:06:27 GMT

Edvin,
Thanks for continuing to look at this.

Yes, you're right. Currently I've implemented FAT_FS on an external flash chip, with the USB stripped out and have not experienced corruption issues. I am using NUS to grab data from the file system and send it over the air as a replacement for reading the file over USB. This method is working quite well.

I haven't tried QSPI only with USB, but just normal QSPI did not have this corruption.

With your testing were you able to reproduce a similar corruption using the method I've described?

--Jeff

RE: USB MSC Corruption

Edvin — Mon, 24 Jun 2019 13:47:22 GMT

Hello Jeff,

I am really struggling to get consistent behavior on the data written using the FAT_FS system on the external flash chip.

You mentioned that you tested with BLE instead of the USB, right? And then the issue was not present?

Have you tried to use the QSPI example (SDK\examples\qspi) together with the USB? It is a bit more hazzle, because it doesn't contain this file system. The issue is that the fatfs is an external library, and most of our developers are currently busy working on the nRF Connect SDK for the nRF91.

But you said that the issue was not present while using BLE + FAT_FS? Only when using the USB?

Best regards,

Edvin

RE: USB MSC Corruption

JeffW — Wed, 19 Jun 2019 14:46:28 GMT

Edvin,

No worries, glad you made it back safe!
My one change to the SDK is ffconf.h which is included in the directory I shared. This is the only change to the otherwise fresh SDK 15.2 to make this work.
The changes are:
_USE_STRFUNC 1
_USE_CHMOD 1
_USE_LFN 1
_FS_EXFAT 1

These are the settings I have been testing on between FATFS and EX-FAT experimentation throughout this thread in demo and actual product. I assume I could get rid of EXFAT/LFN/CHMOD for this example, but all my testing is for this configuration in ffconfig.h
Thanks,
Jeff

RE: USB MSC Corruption

Edvin — Wed, 19 Jun 2019 14:23:01 GMT

Hello Jeff,

Nice to meet you too! Glad you enjoyed the Tech Tour.

Sorry for the late reply. This is my first day in the office after the travel.

So it has been a while since I looked into this. Using the project folder that you provided it didn't compile because fprintf() was not defined, which is because _USE_STRFUNC was set to 0 in ffconf.h. I assume that you have set this to 1 (outside the project folder that you zipped). But just to make sure, can you try to unzip the project that you sent now in a fresh copy of the SDK, and see if there are any other changes that are needed in order to reproduce the issue? I've had some catching up to do today, but I will give it a go tomorrow. I just want to make sure that there are no other settings that are different as well 🙂

Best regards,

Edvin

RE: USB MSC Corruption

JeffW — Tue, 04 Jun 2019 14:51:59 GMT

Edvin,

It was a pleasure learning from you and the rest of the Nordic team yesterday at the Nordic Tech Tour!

Attached is a git project of a modified usb_msc example that can be run on the nRF52840 dev-kit.
Notable changes from the example project (shown in git):
Only uses QSPI device.
Changed to 4096 byte allocation size.
Added app_timer module.
Added file write supplied by nordic engineers.
Added timer that sets a flag to write a string to a file every 500ms.
Timer will stop on USB power detect event, and resume on power removed.

The project is relative to: nRF5_SDK_15.2.0_9412b96\examples\peripheral

The procedure I followed to get the corruption is as follows:
Device is formatted using button 3 while USB is unplugged, 1000 files are written [1 per 500ms] (Can be much less, but it seems like there is a lower boundary before corruption happens)

Multiple "log_data.txt file reads (opens using Notepad) on Windows 7 work without signs of corruption.
The same ~1000 log file is then read using Windows 10 Notepad, the file becomes corrupt on the 105th character of the file in this case. Both files are saved and included as text documents.

When rerunning the test, it is important to reformat from the nRF52840 device (Button 3) as both windows 7/10 re-format will cause a similar corruption when reading on Windows 7 AND Windows 10. (With settings as discussed earlier in this thread)

I'm still unsure of reason for corruption but in our discussion we proposing looking at a USB sniffer while opening the file on Windows 10.

Again, thank you for the continued support,
Jeff
devzone.nordicsemi.com/.../usb_5F00_msc_5F00_git.zip

RE: USB MSC Corruption

Edvin — Wed, 15 May 2019 06:48:22 GMT

Hello Jeff,

It has been a while since you sent the previous project. Is it possible to send the updated project, so that I can try to replicate it with the same setup that you have now?

RE: USB MSC Corruption

JeffW — Fri, 10 May 2019 11:05:53 GMT

Edvin,

I ran a few more tests across multiple computers. It seems like opening a large log file on windows 10 corrupts the file by deleting some data by deleting and inserting non-ascii data as we've seen before.

I can work with a file without corruption for hundreds of logs on windows 7, then plugging it into windows 10 will immediately corrupt the data (not sure if it happens before the file is opened). The data remains corrupted when opening back on windows 7, and through internal micro fatfs file reads.

I'm going to try to compare a capture using the windows Process Monitor Tool across the different operating systems. I will let you know if this leads to any conclusion.

I assume you would be able to corrupt one of the demo projects by either method outlined. I'd be curious if the engineers have input on my findings if it is repeatable on your end.

--Jeff

RE: USB MSC Corruption

Edvin — Fri, 10 May 2019 07:14:41 GMT

Hello,

[quote user="JeffWelder"]our team has a working theory that the file does not corrupt when being accessed on a windows 7 PC, but a windows 10 PC does somehow corrupt the file in the reading process.[/quote]

Does that mean that reading from a windows 10 PC corrupts the file, or that it reads it wrong?

If you read a corrupted file with a windows 10, unplug the device, and plugs it into a windows 7, is the file still corrupted?

RE: USB MSC Corruption

JeffW — Thu, 09 May 2019 12:51:56 GMT

Edvin,
Thank you for the continued support.

Currently in my application using an 8MB QSPI Flash Chip, I experience corruption very early in the log when I format using the following settings:

Using these settings, any PC will corrupt the file (Unsure if the mount/read or write corrupts the data). I suspect if you run the same tests as previous after formating this way, you will experience corruption in the test project you shared.

However, when I format the chip using: ff_result = f_mkfs("", FM_FAT, 4096, buf, sizeof(buf)); The file seems less likely to corrupt:
Being formatted by the microcontroller, our team has a working theory that the file does not corrupt when being accessed on a windows 7 PC, but a windows 10 PC does somehow corrupt the file in the reading process.

I'm not sure if there is any merit to this conclusion related to the OS, but these are just the very high level observations I've experienced.
Unfortunately all of my testing was done on my custom board's software, instead of the examples we've been sharing back and forth.

From our previous conversations, I think I can relate the corruption from our previous tests that worked for you, but I experienced corruption to the chip being formatted by windows on one dev kit, and by the nRF52840 on the other working (no corruption) dev-kit.

Thanks for the continued support,
Jeff

RE: USB MSC Corruption

Edvin — Thu, 09 May 2019 07:07:09 GMT

Hello Jeff,

I hope you are doing fine 🙂

There hasn't been any updates on this front from our side, no.

So you say that you can see the issue when you format using:
FAT with unit size of 4096 when formatting the SD card, like this:

And also, in the application, do you still make the file system using f_mkfs("", FM_FAT, 4096, buf, sizeof(buf)); ?

BR,

Edvin

RE: USB MSC Corruption

JeffW — Wed, 08 May 2019 16:58:30 GMT

Edvin,
Unfortunately I am still having issues with these implemented changes. Certainly it has gotten better but all of my field units still experience corruption at various points in their life.
In another recent thread there has been discussion of allocation unit size matching sector size of the flash part, which seemed to possibly resolve an issue.

However, I can force the issue when the part is formatted from a Windows PC with FAT, Allocation unit size 4096, with quick format off.

I hope to be able to run a few more experiments on this test project, but I am not sure how much of a conclusion I will be able to reach.

Have there been any internal updates on this issue in the past 3 months?

Thanks,
Jeff

RE: USB MSC Corruption

Edvin — Mon, 28 Jan 2019 09:11:11 GMT

Hello Jeff,

Can you try the attached file:

devzone.nordicsemi.com/.../main_5F00_test.c

Changed: ff_result = f_mkfs("", FM_FAT, 4096, buf, sizeof(buf));

Then format the device as FAT (not exFAT), and allocation unit size 4096 bytes.

I ran this test now for 5004 records (each record 50 bytes long), and did not encounter any corrupted data.

BR,

Edvin

RE: USB MSC Corruption

JeffW — Thu, 24 Jan 2019 13:33:31 GMT

Jakub,

PCA10056 1.0.0 2018.20

SDK 15.2 w/ soft device 6.1.

--Jeff

RE: USB MSC Corruption

Jakub Rzeszutko — Thu, 24 Jan 2019 08:29:50 GMT

JeffW: Please let us know what is version of your DK?

RE: USB MSC Corruption

JeffW — Wed, 23 Jan 2019 13:52:20 GMT

Edvin,

On the first nRF52840 development kit I was using the corruption still occurred when using the provided noirq example. A second development kit seemed to work up to the 5000 records I tested, but it still seems pretty flakey. When re-running the test (deleting the file and restarting) on the 2nd board, I sometimes did see corruption of a very long string of ÿ's in the beginning/middle of the log.

When re-testing the first development kit, I noticed the fatfs_ls function returned this:

[00:00:00.000,000] <info> app: 
Listing directory: /
   <DIR>   ÿÿÿÿÿÿÿÿ.ÿÿÿ
   // Truncated for Readability
   <DIR>   ÿÿÿÿÿÿÿÿ.ÿÿÿ
   120812  log_data.txt
   <DIR>   System Volume Information
Entries count: 259

After re-formatting the drive using Windows it seemed to get rid of this, and the corruption didn't happen the next test.

It seems that removing the writes from an interrupt base has sometimes helped us pass over the first discovered 13825th byte boundary, but this solution does not work all of the time.

Attached is a corrupted file with the noirq code:devzone.nordicsemi.com/.../test_5F00_log1.txt

I appreciate the help, It looks like we are making some progress but this still needs a little more attention,
Jeff

RE: USB MSC Corruption

Edvin — Tue, 22 Jan 2019 12:56:34 GMT

Hello Jeff. Did you try the main.c file in your project, or an unmodified usbd_msc example from SDK15.2.0? Can you try with an unmodified version of this example? And to be completely sure, unzip a fresh copy of SDK15.2.0

I did the same test, and I do not get the corrupted messages with this example. Please note that this example doesn't by default delete the old file, so connect the device with the usbd, and delete the LOG_DATA.txt file. Also note that some .txt readers cache the old files, so make sure to close this file (e.g. if you use notepad/notepad++), and then re-open it to see the latest version.

Best regards,

Edvin

RE: USB MSC Corruption

JeffW — Mon, 21 Jan 2019 13:21:27 GMT

Edvin,

I'm still get corruption running the SDK engineers code. Near the 1905th byte this time (~132nd record). Ran the test twice from my end with the same corrupted results.

Regards,
Jeff

RE: USB MSC Corruption

Edvin — Mon, 21 Jan 2019 08:12:58 GMT

Hello Jeff,

I got an answer from one of our SDK engineers. He wrote the following:

----------------------

I've tested it using attached code, but I did not manage to get it working. In provided code f_printf is called from irq context, while internally it is waiting for interrupt from qspi and without proper irq priorities it hangs.

I modified example to write to file from main loop, and it seems to be working on nRF52840 DK 1.0.0, I don't get any corrupted data tested up to 4500 records. Please check example with attached code (nonirq_main.c) if data still gets corrupted.

----------------------

Attached is the noirq_main.c file that he used for testing. Can you please try it and see if it works better?

devzone.nordicsemi.com/.../noIRQ_5F00_main.c

Best regards,

Edvin

RE: USB MSC Corruption

JeffW — Wed, 16 Jan 2019 14:04:30 GMT

Edvin,

Nothing fruitful on my end from this, as the serial flash should be able to deal with single byte accesses.

Any update from Nordic? Surely I am not the only one running into issues with file system implementations.

Thanks,
Jeff

RE: USB MSC Corruption

JeffW — Wed, 02 Jan 2019 16:30:17 GMT

In that thread rc766437 suggested:
"I think it is the same problem, just modify the QSPI block driver code<nrf_block_dev_qspi.c>, let the write/read buf pointer address aligned 4 bytes."

I have come across a few examples of people running into this issue, but haven't been able to find the correct spot to 4 byte align the buffer.
https://devzone.nordicsemi.com/f/nordic-q-a/38156/qspi-read-fail/

Any insight on this?