debug nrf52840 zephyr app with threads using JLINK + gdb

[Eyal_Gal]

hi.

I have a zephyr app that includes threads, 3 to be exact.

I'm compiling thread info using -D param passed on to the west command (well vs code does it for me, i'm just verifying it's there).

When i want to de bug the app i connect j-link to the target and do the following:

1. run JLinkGDBServer -if swd -device NRF52840_XXAA -speed 1000  -rtos /opt/SEGGER/JLink_V798g/GDBServer/RTOSPlugin_Zephyr.so  -singlerun -nogui
2. worth mentioning that i tried with and without -singlerun and in both cases it's the same result...
3. the west command line to compile includes: -DCONFIG_DEBUG_OPTIMIZATIONS=y -DCONFIG_DEBUG_THREAD_INFO=y 
4. the gdb command: arm-zephyr-eabi-gdb app/build/app/zephyr/zephyr.elf (ncs toolchain is on the path)
5. in gdb i use `target remote :2331`.
6. i set a breakpoints and use monitor reset and 'c' to continue. And the breakpoints is reached , however i cannot debug.
7. when using info threads i get just 1 thread and it's not true.
8. when trying 's' or 'n' it seems to stop the thread and restart it and not step in step out, i do not see the command executed and i do not get to the next command.

since you have disabled in VScode extension the debugger, can you please explain how i can debug a threaded app in GDB ?

my JLink version is: V7.98g, ncs version including toolchain is v2.7.0
I have tried advises from different places, but non actually helped, the only option currently is turn off the other threads...

[Håkon Alseth]

Hi,

 

Here's a way to use gdb + threadaware plugin:

 RE: gdb commands to list Zephyr app threads? 

 

Based on your commands, it seems that you are following this.

What I have seen is that you have to do a single instruction "si" for the gdb server / client to get the thread info.

 

Could you try to reproduce this in a simpler sample, like the zephyr/samples/basic/threads?

 

Kind regards,

Håkon

[Eyal_Gal]

thanks Hakon.

Naturally i can try to use a simpler app, i didn't try si though just 's' and 'n' so it might work.

You think maybe something in our thread causes an issue to debug it ?

[Håkon Alseth]

Hi,

 

Are you debugging a bluetooth sample? If so, try to break first on main, then add your actual break point and continue.

There are assertions/ISRs that will run inside the stack, and those will cause problems if you step too far with the debug client.

 

Eyal_Gal said:

You think maybe something in our thread causes an issue to debug it ?

It could be, but running a simpler sample is basically just to verify the process itself, ie. that the rtos plugin is loaded correctly and that it can detect the threads properly. If that works, we know it is not a host setup issue.

 

Kind regards,

Håkon

[Eyal_Gal]

no i don't have BLE yet in my code, but i do have several timers, i2c devices, UART0 and UART1 used in async mode, so definitely there are interrupts running around. 

Also my app is not "alone" we used b0 bootloader with mcubootloader (immutable bootloader and mcu bootloader that can be upgraded and can upgrade the app as well.) so i have to use the same struct along with a new app, more basic.

[Håkon Alseth]

Hi,

 

I tested quickly in a standard application, with "west build ...args -- -DCONFIG_DEBUG__THREAD_INFO=y", and setup the GDBServer similar to you (just different SoC + macos):

JLinkGDBServer -if swd -device NRF9160_XXAA -speed 1000  -rtos /Applications/SEGGER/JLink/GDBServer/RTOSPlugin_Zephyr.dylib -singlerun -nogui 
SEGGER J-Link GDB Server V7.98i Command Line Version

JLinkARM.dll V7.98i (DLL compiled Sep 18 2024 15:49:01)

Command line: -if swd -device NRF9160_XXAA -speed 1000 -rtos /Applications/SEGGER/JLink/GDBServer/RTOSPlugin_Zephyr.dylib -singlerun -nogui
-----GDB Server start settings-----
GDBInit file:                  none
GDB Server Listening port:     2331
SWO raw output listening port: 2332
Terminal I/O port:             2333
Accept remote connection:      yes
Generate logfile:              off
Verify download:               off
Init regs on start:            off
Silent mode:                   off
Single run mode:               on
Target connection timeout:     0 ms
------J-Link related settings------
J-Link Host interface:         USB
J-Link script:                 none
J-Link settings file:          none
------Target related settings------
Target device:                 NRF9160_XXAA
Target device parameters:      none
Target interface:              SWD
Target interface speed:        1000kHz
Target endian:                 little

Connecting to J-Link...
J-Link is connected.
Firmware: J-Link OB-nRF5340-NordicSemi compiled Sep 11 2024 13:35:00
Hardware: V1.00
S/N: 1051242634
Feature(s): RDI, FlashBP, FlashDL, JFlash, GDB
Checking target voltage...
Target voltage: 3.30 V
Listening on TCP/IP port 2331
Connecting to target...
Halting core...
Core security extensions: Implemented
Connected to target
...rest

 

And setup arm-zephyr-eabi-gdb:

(gdb) target remote :2331
Remote debugging using :2331
__enable_irq () at /opt/ncs/modules/hal/cmsis/CMSIS/Core/Include/cmsis_gcc.h:951
951	  __ASM volatile ("cpsie i" : : : "memory");
(gdb) si
[New Thread 536871920]
[New Thread 536871712]
[New Thread 536871536]
[New Thread 536871360]

 info threads:

(gdb) info threads
  Id   Target Id                                     Frame 
* 2    Thread 536871920 (idle UNKNOWN PRIO 15)       arch_cpu_idle () at /opt/ncs/zephyr/arch/arm/core/cortex_m/cpu_idle.c:97
  3    Thread 536871712 (blink0_id SUSPENDED PRIO 7) arch_swap (key=key@entry=0) at /opt/ncs/zephyr/arch/arm/core/cortex_m/swap.c:49
  4    Thread 536871536 (blink1_id SUSPENDED PRIO 7) arch_swap (key=key@entry=0) at /opt/ncs/zephyr/arch/arm/core/cortex_m/swap.c:49
  5    Thread 536871360 (uart_out_id PENDING PRIO 7) arch_swap (key=0) at /opt/ncs/zephyr/arch/arm/core/cortex_m/swap.c:49

 

Did you try any other sample and get the threads info?

 

Kind regards,

Håkon

[Eyal_Gal]

Hi.

Seems like you are right,

i have copied my workspace to another location, removed everything but main.c in my /app directory, updated cmake and main.c to run an endless while loop with sleep and led 1 toggle, another thread is started and toggles another led and sleeps a different amount of time.

started the gdb server like before and the arm-gdb as well.
This time using info threads yielded 

(gdb) info threads
  Id   Target Id                                    Frame 
* 2    Thread 536877336 (main RUNNING PRIO 0)       main () at /home/eyal/develop/scms/staging_area/west_ws_1/nrf-connect-tondo/app/src/main.c:51
  3    Thread 536877120 (idle UNKNOWN PRIO 15)      0x00031222 in arch_cpu_idle () at /home/eyal/develop/scms/staging_area/west_ws_1/zephyr/arch/arm/core/cortex_m/cpu_idle.c:98
  4    Thread 536872464 (logging PENDING PRIO 14)   arch_swap (key=key@entry=0) at /home/eyal/develop/scms/staging_area/west_ws_1/zephyr/arch/arm/core/cortex_m/swap.c:49
  5    Thread 536877616 (sysworkq PENDING PRIO 255) arch_swap (key=key@entry=0) at /home/eyal/develop/scms/staging_area/west_ws_1/zephyr/arch/arm/core/cortex_m/swap.c:49
  6    Thread 536876840 (mcumgr smp QUEUED PRIO 3)  z_thread_entry (entry=0x2c209 <work_queue_main>, p1=0x20001728 <smp_work_queue>, p2=0x0, p3=0x0)
    at /home/eyal/develop/scms/staging_area/west_ws_1/zephyr/lib/os/thread_entry.c:37
  7    Thread 536872248 (thread1_id QUEUED PRIO 7)  z_thread_entry (entry=0x23bdd <thread1>, p1=0x0, p2=0x0, p3=0x0) at /home/eyal/develop/scms/staging_area/west_ws_1/zephyr/lib/os/thread_entry.c:37
(gdb) si
0x00023c36	51		ret = gpio_pin_configure_dt(&led, GPIO_OUTPUT_ACTIVE);
(gdb) si
0x00023c38	51		ret = gpio_pin_configure_dt(&led, GPIO_OUTPUT_ACTIVE);
(gdb) si

So the general setup seems to work, why i have an SMP thread, i have no idea, related to mcuboot i assume.

SI and NI instructions work nicely, so i'm not sure what's causing an issue with our app.
I'll keep this request open after i'll try again (we are in the middle of merges of features...)
I also have noticed (dunno if it's new or not) that now if in the nrf-connect extension page under applications -> app -> build i do not have debug, but if i go further to applications -> app -> build ->app (it's sys-build based west ws) then i do have both debug and debug with ozone, didn't test it yet but i have a hunch it will work, so if i'll update the runners yaml to include the zephyr rtos plugin it will work, ozone loads it automatically if it identifies rtos, so no need to update.

[Håkon Alseth]

Hi,

 

Glad to hear that the setup itself works as it should.

Let me know if you still run into issues with your current firmware, and please provide a "bt" and "mon regs" output if that still does not give you your thread info.

Eyal_Gal said:

SI and NI instructions work nicely, so i'm not sure what's causing an issue with our app.
I'll keep this request open after i'll try again (we are in the middle of merges of features...)
I also have noticed (dunno if it's new or not) that now if in the nrf-connect extension page under applications -> app -> build i do not have debug, but if i go further to applications -> app -> build ->app (it's sys-build based west ws) then i do have both debug and debug with ozone, didn't test it yet but i have a hunch it will work, so if i'll update the runners yaml to include the zephyr rtos plugin it will work, ozone loads it automatically if it identifies rtos, so no need to update.

It is similar to this, right?

 

This is because sysbuild will append the image name inside the build-folder, as opposed to the partition manager.

So now it will be build/$(IMAGE_NAME)/zephyr/ where the build files are located, as without sysbuild it is build/zephyr/

 

Kind regards,

Håkon