[nRF Connect SDK](Urgent)Assert after DFU via OTA

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Mon, 05 Sep 2022 02:15:43 GMT

I could avoid assertion

1. Increase stack size
2. I update NCS 2.0.0. Ringbuffer assertion disappeared.

Thank you Simon.

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Simon — Tue, 16 Aug 2022 11:28:01 GMT

[quote user="Tim Hwang"]I found lots of changes. so please review my stack size is affordable in my attached config.
It it is fine, I will use NCS2.0 instead.[/quote]

It is hard to comment on the stack sizes just by looking at it. As mentioned in Memory footprint Optimizations-->General recommendations, you can analyze the stack usage in each thread of your application by using the Thread analyzer. Reduce the stack sizes where possible.

Best regards,

Simon

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Mon, 08 Aug 2022 00:30:32 GMT

Aug 08- 2022

Hello.

In my test I found I already had ring buffer(provided by zephyr) assert issue.
Recently I update NCS version to 2.0.0. And it looks good in my test.
I found lots of changes. so please review my stack size is affordable in my attached config.
It it is fine, I will use NCS2.0 instead.

My SW	NCS	OTA assert	Ringbuffer test	Result
V1.3.6:	1.9.1	NG(sometime happened)	assert (within 48 hours)	NG
V1.3.7	1.9.1	Increase stack size So far so good	assert (within 48 hours)	NG
V1.4.0	2.0	Increase stack size So far so good	No Ringbuffer assert (so far so good)	Need your review

devzone.nordicsemi.com/.../NCS2.0_5F00_build.7z

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Simon — Fri, 29 Jul 2022 14:49:36 GMT

[quote user="Tim Hwang"]What I asked you was if "Used 99.8%" itself is fine.[/quote]

I think it should be fine.

[quote user="Tim Hwang"]On the other hand I change CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE from 2048 to 512.
I could this issue immediately.

I'm not sure if this issue is from stack over flow.[/quote]

Okay, so right after you change CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE from 2048 to 512 the issue happens, then it's likely a stack overflow.

I'm not sure if I understood your questions correctly, let me know if not.

However, I will go on vacation the next two weeks, so I'm not able to reply until 15th of Aug. If that is too long to wait, could you open a new ticket and point to this one?

Best regards,

Simon

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Simon — Thu, 28 Jul 2022 23:39:12 GMT

Hello, sorry for the delay on this. In July, most of the support team are on vacation and you can expect slower responses. I will reply tomorrow (Friday)

Best regards,

Simon

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Thu, 28 Jul 2022 05:58:19 GMT

I have to tell you another view of this issue.
When this issue was made, BLE was reset repeatedly with those logs.

I have no choice but to flash same binary to recover it.
After then I cannot made this issue again easily. ( I could not make same issue again)

On the other hand I change CONFIG_SYSTEM_WORKQUEUE_STACK_SIZE from 2048 to 512.
I could this issue immediately.

I'm not sure if this issue is from stack over flow.

Soon, one of my customer will test our product to buy.
I'm expecting you give me any opinion.

Thank you.

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Wed, 27 Jul 2022 00:18:10 GMT

Hello.

I did stress test with two product. And one of device was reset without any log last night. (Stress test means receiving data from UART and send it to Android)
I already did lots of times it was fine so far.

[rtt log] silent reset
00> connect_done secmode(4)
00> auth_handler 1
00> *** Booting Zephyr OS build v2.7.99-ncs1-1 ***
00>
00> Starting Bluetooth Peripheral HIDS keyboard example for RFR
00> ## build time: Jul 26 2022 09:49:04
00> ver V1.3.7

[Changes & SRAM]
V1.3.6
CONFIG_MAIN_STACK_SIZE=1024
CONFIG_BT_MAX_PAIRED=6
RINGBUFFER size: 24KB
[264/272] Linking C executable zephyr\zephyr.elf
Memory region Used Size Region Size %age Used
FLASH: 214636 B 232960 B 92.13%
SRAM: 64672 B 64 KB 98.68%
IDT_LIST: 0 GB 2 KB 0.00%
[272/272] Generating zephyr/merged.hex

v1.3.7
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_BT_MAX_PAIRED=4
RINGBUFFER size: 24KB
[264/272] Linking C executable zephyr\zephyr.elf
Memory region Used Size Region Size %age Used
FLASH: 214636 B 232960 B 92.13%
SRAM: 65408 B 64 KB 99.80%
IDT_LIST: 0 GB 2 KB 0.00%
[272/272] Generating zephyr/merged.hex

V1.3.8 <- Under Testing
CONFIG_MAIN_STACK_SIZE=2048
CONFIG_BT_MAX_PAIRED=6
RINGBUFFER size: 24KB -> 22KB
[264/272] Linking C executable zephyr\zephyr.elf
Memory region Used Size Region Size %age Used
FLASH: 214636 B 232960 B 92.13%
SRAM: 63360 B 64 KB 96.68%
IDT_LIST: 0 GB 2 KB 0.00%

Stress test requires lots of time(a day). So I hope you give me your opinion current my project situation.

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Tue, 26 Jul 2022 00:10:07 GMT

Thanks for good information. it is very helpful.

My project requires maximum performance. We read UART from host and send it to Android via BLE.
(bandwidth 60 KBytes/sec). So that it need much ram to run.
What I asked you was if "Used 99.8%" itself is fine.
2048 bytes of CONFIG_MAIN_STACK_SIZE is fine?
0.02% of ram is left. Is it okay?
If we need more margin(0.02%), I have to reduce my buffer for ring buffer or so.
What is your opinion?

I will test "CONFIG_MAIN_STACK_SIZE=2048" until today.
Thanks.

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Simon — Mon, 25 Jul 2022 09:24:11 GMT

Check out https://developer.nordicsemi.com/nRF_Connect_SDK/doc/2.0.1/nrf/app_memory.html#bluetooth to see how you can reduce the FLASH and RAM footprint for a Bluetooth sample.

This prj.conf specifically may be useful: https://github.com/nrfconnect/sdk-nrf/blob/v2.0.1/samples/bluetooth/peripheral_uart/prj_minimal.conf

Best regards,

Simon

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Mon, 25 Jul 2022 08:22:52 GMT

Thanks for updating.
I'm testing it too.

Plus.
I change CONFIG_MAIN_STACK_SIZE from 1024 to 2048.

Memory region Used Size Region Size %age Used
FLASH: 214972 B 232960 B 92.28%
SRAM: 65408 B 64 KB 99.80%
IDT_LIST: 0 GB 2 KB 0.00%

Used 99.8% is it okay?

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Simon — Mon, 25 Jul 2022 08:18:51 GMT

Try increasing CONFIG_MAIN_STACK_SIZE. You can check the file hello_world\build\zephyr\.config to see what the actual value is.

Best regards,

Simon

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Tim Hwang — Mon, 25 Jul 2022 00:03:01 GMT

Thanks for updating.
Please find attached.

1) I found z_main_thread is located 0x20002350

zephyr.map
.bss.z_main_thread
0x20002350 0x80 zephyr/kernel/libkernel.a(init.c.obj)
0x20002350 z_main_thread

2) According to other case, I call boot_write_img_confirmed() every time. Is this normal?

(+) [nRF Connect SDK]Sometimes DFU failed to update binary without confirm. - Nordic Q&A - Nordic DevZone - Nordic DevZone (nordicsemi.com)
void main(void)
{
boot_write_img_confirmed();
...
}

3) CONFIG_THREAD_NAME=y
I'll chage it then test it again.

Once this situation happened, I have to flash via jtag again. it is real problem.
We have no choice but to open product's cover and flash firmware via flash.

RE: [nRF Connect SDK](Urgent)Assert after DFU via OTA

Simon — Fri, 22 Jul 2022 13:00:20 GMT

It seems like a stack overflow happens. Could you set CONFIG_THREAD_NAME=y, so we can see what thread that caused the overflow. As you can see now the thread is unknown:

00> E: Current thread: 0x20002350 (unknown)

Then rebuild, trigger the fault again and upload the new log. Then I will tell you how to increase the stack size of that thread.

Also the uploaded .elf file is corrupted. Could you upload it again?

Best regards,

Simon