Flashing custom nRF52811 board with softdevice using OpenOCD

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Fri, 31 Mar 2023 21:09:48 GMT

So after a few more hours of debugging, I finally found a solution. It all came down to not decreasing the RAM segment length in the linker script despite increasing the segment start address.

After incrementally adding back custom features to the ble_app_template, at one point I saw the project was working when building with SEGGER, but not when building with make - despite me only increasing the RAM segment start address and not decreasing the segment length in both SEGGER and linker script used in make. After some digging, I found that SEGGER was configured to calculate the segment length on-the-fly and appeared to completely ignore the configured segment length. This lead me to decreasing the segment length in the linker script, and now the make build works as well.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Fri, 31 Mar 2023 18:41:48 GMT

Great news. I attempted the same exercise with the ble_app_template example and got it working. Now it's just a matter of going through the process of elimination by incrementally adding custom features until it breaks again. Hopefully it'll be more clear what's going wrong after that.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Fri, 31 Mar 2023 16:01:16 GMT

So I tried flashing the unmodified ble_app_beacon (pca10056e) using SEGGER embedded studio and got the same result - firmware starts, but fails when initializing BLE. This time I used hardware without the 32.768 kHz crystal, so I followed the instructions to use LFRC instead, found here.

This is where I end up when running the firmware.

Using breakpoints, I was able to determine the abort happens at this point during ble initialization.

Setting a breakpoint at any point after this one (e.g. line 231) is never reached. However, nrf_sdh_enable_request() returns NRF_SUCCESS.

I'll verify the same happens on hardware with the 32.768 kHz crystal and update this post.

EDIT: So I just made an interesting discovery. For some reason, NRF_SDH_CLOCK_LF_SRC option change wasn't saved to sdk_config.h, which caused the abort. After fixing the configuration, firmware seems to run just fine (both on device with crystal and device with LFRC). However, I still can't see the BLE device. So at this point I'm investigating potential hardware issues. Would love to know, if there are still any potential software causes for this issue.

EDIT 2: Guess the option did save, but it appears there are three versions of the same option? Why is that?

NRF_SDH_CLOCK_LF_SRC
NRFX_CLOCK_CONFIG_LF_SRC
CLOCK_CONFIG_LF_SRC

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Einar Thorsrud — Thu, 30 Mar 2023 07:19:16 GMT

Hi,

Yes, 0xdeadbeee clearly indicate an error, but I am not sure what. Regarding debugger, any GUI debugger usually works well for me :D You can for instance use Segger Ozone which works with any elf file. I don't have strong opinion about which debugger you use as long as you understand it, it is just that I am not able to help you with explaining why you see 0xdeadbeee.

Regarding the comment about stepping with a SoftDevice, that is because it seemed to me like you were stepping in the code, and that will not work with the SoftDevice most of the time. It will use RTC0 and TIMER0 to keep track of time, and if it detects that it has lost a event of some sort (because the timer elapsed while the CPU was halted), it will assert. So stepping (or continuing from a breakpoint for that matter) while using a SoftDevice will cause problems.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Wed, 29 Mar 2023 13:06:24 GMT

To me 0xdeadbeee is one off 0xdeadbeef which is often used to indicate some sort of failure. I'm not sure however, if that's read off the device or if it's just a way for the GDB server to report an error communicating with the device or something of that sort.

Which debugger are you more familiar with? I'll see if I can get that setup instead.

Not sure I understand what you mean by the following:

The SoftDevice will assert when stepping in most cases, so instead when using a SoftDevice, you can mimmic it by using a breakpoint, and move the breakpoint a bit further and reset to let it run to the next breakpoint, and so on.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Einar Thorsrud — Wed, 29 Mar 2023 13:00:40 GMT

I hust admit I am more used to GUI debuggers and not GDB so I am not sure what it means when you the address is 0xdeadbeee, but that said, it seems like you are stepping? The SoftDevice will assert when stepping in most cases, so instead when using a SoftDevice, you can mimmic it by using a breakpoint, and move the breakpoint a bit further and reset to let it run to the next breakpoint, and so on.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Wed, 29 Mar 2023 11:56:49 GMT

Yes, app was built with correct headers. I started with the emulated version of ble_app_template example as a starting point which already uses S112, so I would assume everything is already correctly configured (except for the two defines which are mentioned here under "Transferring emulated project").

By "stuck" I mean that GDB hangs at that command and never returns, preventing me from entering any further commands. As mentioned in my previous post above, I let it run for a while and after a few minutes I ended up at address 0xdeadbeee.

During all tests I was also monitoring for the expected BLE device to show up on my phone, but it never appeared.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Einar Thorsrud — Wed, 29 Mar 2023 11:50:16 GMT

Hi,

Did you build your application with the correct SoftDevice header files (for S112 7.2.0), or do you still have the headers for S140 in your include path instead?

When you write "get stuck", what does that mean?

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Wed, 29 Mar 2023 10:54:28 GMT

Jumping to `main` got me stuck after asking for the current frame.

EDIT: After restarting the device through nrfconnect and restarting GDB and GDB server, I get stuck after the following:

EDIT 2: After a few minutes, I ended up at address 0xdeadbeee.

EDIT 3: Here are logs from GDB bridge

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Wed, 29 Mar 2023 10:48:39 GMT

Here's some more info I was able to get from my BLE app. I set the PC to the application start address and after a few steps, I again end up at 0x000189cc.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Wed, 29 Mar 2023 10:30:00 GMT

So today, the DK works again. Odd. I'm starting to suspect this USB hub I got recently.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Tue, 28 Mar 2023 20:51:44 GMT

Also, in case this helps, here's the MEMORY section from my linker script. Looks like start address is correct and if I understand .hex files right, they should already contain the location in flash where the contents need to be placed. I think the only change I had to make here was increase the RAM origin by 0x10 (or something like that) to give softdevice more ram to handle a custom BLE service. Again, worked fine on DK, so I would assume it should be the same with nRF52811.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Tue, 28 Mar 2023 20:14:43 GMT

A few more interesting observations. I tried debugging both my BLE app as well as blinky and wasn't able to get far with either one.

I used JLinkGDBServer -USB -device cortex-m4 to start a GDB server and arm-none-eabi-gdb to try and debug the firmware.

In GDB I ran into the first issue, not knowing which file to load. I first tried file nRF5_SDK_17.1.0_ddde560/examples/peripheral/blinky/pca10056e/blank/armgcc/_build/nrf52811_xxaa.hex, but always got the response that this file doesn't contain any debug symbols. I could only step between two instructions for the blinky app and one instruction for the BLE app.

# Blikly app
(gdb) target remote localhost:2331
Remote debugging using localhost:2331
0x00000710 in ?? ()
(gdb)
(gdb) ni
0x00000712 in ?? ()
(gdb)
0x00000710 in ?? ()
(gdb)
0x00000712 in ?? ()

# BLE app
(gdb) ni
0x000189cc in ?? ()
(gdb) ni
0x000189cc in ?? ()
(gdb) ni
0x000189cc in ?? ()
(gdb) ni
0x000189cc in ?? ()
(gdb) ni
0x000189cc in ?? ()
(gdb) ni
0x000189cc in ?? ()
(gdb) ni
0x000189cc in ?? ()

I later found, through trial and error, that the `.out` file contains debug symbols, so I decided to load that one instead. When loading the blinky example, here's what I got:

1st try:

(gdb) target remote localhost:2331
Remote debugging using localhost:2331
0x00000712 in delay_machine_code ()
(gdb) n
Single stepping until exit from function delay_machine_code.0,
which has no line number information.
0xdeadbeee in ?? ()
(gdb) n
Cannot find bounds of current function
(gdb) n
Cannot find bounds of current function
(gdb) ni
0xdeadbeee in ?? ()
(gdb)
0xdeadbeee in ?? ()
(gdb)
0xdeadbeee in ?? ()
(gdb)
0xdeadbeee in ?? ()

2nd try:

(gdb) target remote localhost:2331
Remote debugging using localhost:2331
0x00000712 in delay_machine_code ()
(gdb) ni
0x00000710 in delay_machine_code ()
(gdb)
0x00000712 in delay_machine_code ()
(gdb)
0x00000710 in delay_machine_code ()
(gdb)
0x00000712 in delay_machine_code ()
(gdb)
0x00000710 in delay_machine_code ()
(gdb) n
Single stepping until exit from function delay_machine_code.0,
which has no line number information.
nrf_delay_ms (ms_time=<optimized out>) at ../../../../../../components/libraries/delay/nrf_delay.h:73
73 } while (--ms_time);
(gdb) n
72 nrf_delay_us(1000);
(gdb) n
73 } while (--ms_time);
(gdb) n
72 nrf_delay_us(1000);
(gdb) n
73 } while (--ms_time);
(gdb) n
72 nrf_delay_us(1000);
(gdb) n
73 } while (--ms_time);
(gdb) n
72 nrf_delay_us(1000);
(gdb)
73 } while (--ms_time);
(gdb)
72 nrf_delay_us(1000);
(gdb)
73 } while (--ms_time);
(gdb)
72 nrf_delay_us(1000);
(gdb)
73 } while (--ms_time);
(gdb)
72 nrf_delay_us(1000);
(gdb)
73 } while (--ms_time);

This seemed to go into infinity

(gdb) frame
#0 nrf_delay_ms (ms_time=<optimized out>) at ../../../../../../components/libraries/delay/nrf_delay.h:73
73 } while (--ms_time);
(gdb) s
72 nrf_delay_us(1000);
(gdb)
nrfx_coredep_delay_us (time_us=1000) at ../../../../../../modules/nrfx/soc/nrfx_coredep.h:173
173 delay_cycles(cycles);
(gdb)
nrf_delay_ms (ms_time=<optimized out>) at ../../../../../../components/libraries/delay/nrf_delay.h:73
73 } while (--ms_time);
(gdb)
72 nrf_delay_us(1000);
(gdb)
nrfx_coredep_delay_us (time_us=1000) at ../../../../../../modules/nrfx/soc/nrfx_coredep.h:173
173 delay_cycles(cycles);
(gdb)
nrf_delay_ms (ms_time=<optimized out>) at ../../../../../../components/libraries/delay/nrf_delay.h:73
73 } while (--ms_time);
(gdb)
72 nrf_delay_us(1000);
(gdb)

I was curious if this would ever end, so I let it run until it's past the current line. This however, turned out to be a bad idea, because after maybe a minute or two of it running through the loop, the target stopped responding. Even trying to power cycle the entire setup had no effect.

nrf_delay_ms (ms_time=<optimized out>) at ../../../../../../components/libraries/delay/nrf_delay.h:73
73 } while (--ms_time);
(gdb)
72 nrf_delay_us(1000);
(gdb) u
73 } while (--ms_time);
(gdb) u

Program received signal SIGTRAP, Trace/breakpoint trap.
0xdeadbeee in ?? ()
(gdb) frame
#0 0xdeadbeee in ?? ()
(gdb) target remote localhost:2331
A program is being debugged already. Kill it? (y or n) y
Remote debugging using localhost:2331

Program received signal SIGTRAP, Trace/breakpoint trap.
<signal handler called>
(gdb) k
Kill the program being debugged? (y or n) y
Remote connection closed
(gdb) target remote localhost:2331Quit
(gdb) q

Now the JLink on the DK appears to be in some weird state - when I try running JLinkGDBServer -USB -device cortex-m4, I get a popup, asking for the remote IP address, because it can't find a USB device. nrfconnect also doesn't detect any devices. Very weird. It's as if the DK is completely bricked. I also tried disconnecting the custom board completely to see, if I can flash the onboard nRF52840, but no luck. In all cases the LED next to JLink starts blinking really fast.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Tue, 28 Mar 2023 18:38:56 GMT

I think the VTG connection was the issue. Or at least "SWD SEL" as it's marked on my DK. Now reading/writing flash shows the correct total size. However, when I flash both softdevice and application, the application gets merged into the softdevice for some reason. Not sure if this is just a visual glitch or could have something to do with the issue I'm experiencing. See flash layout below as shown in nRFConnect software.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Einar Thorsrud — Tue, 28 Mar 2023 13:08:07 GMT

Hi,

It does not matter much if you are using the ST-Link or a J-link like the DK, but it is important that you are able to debug (not just program). Are you? If so, you can continue with the ST-Link. If not, I would work a bit more on getting debugging working. If it does not work, perhaps you did not connect VTG (see this post). Also, the supply voltage of your board must be about 3V in order to use the DK as a debugger.

As you get blinky to work the basics are there, so I would make sure you can debug blinky. When you do, you can move on to try to debug a SoftDevice application. You should at least be able to see that you enter main(). If that does not happen, I would double check that you have programmed the SoftDevice, and that the application start address is correct. That needs to be the same as the size of the SoftDevice given in the release notes for the SoftDevice you are using. So when using S112 7.2.0, the application start address must be 0x19000.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Tadej — Tue, 28 Mar 2023 11:24:49 GMT

Our board does have a 32.768 kHz crystal. However, I still tried to use LFRC, in case that would change anything, but had no luck. I also followed the instructions on how to transfer the project to nRF52811 - I started with an emulated project (examples/ble_peripheral/ble_app_template/pca10056e/s112/armgcc), so I only had to remove DEVELOP_IN_NRF52840 and NRFX_COREDEP_DELAY_US_LOOP_CYCLES.

I'm now trying to debug using the DK, but am also having issues. I shorted SB47 and connected our custom board to P20 (I'm powering our board from VDD nRF), but when I try to write/read flash, I end up writing/reading the nRF52840 that's on the DK. I know this, because I can still read/write the flash even when our custom board is disconnected and because the firmware works as expected (despite the two compiler defines being removed - DEVELOP_IN_NRF52840, NRFX_COREDEP_DELAY_US_LOOP_CYCLES).

I'm pretty suck at this point - not sure how to get the DK to flash our custom board. I saw SB19 next to SB47, so I'm wondering if I should short it as well? But SB19 appears to be undocumented, so I'm not even sure what it's for.

I can also add that the basic blinky example (examples/peripheral/blinky) flashed using ST-LINK works fine. It's only when I try to use BLE which requires the softdevice I start running into issues. I've also made sure to always flash the softdevice (S112), so that's definitely not missing.

RE: Flashing custom nRF52811 board with softdevice using OpenOCD

Einar Thorsrud — Thu, 23 Mar 2023 13:39:31 GMT

Hi,

The first thing that comes to mind as your application worked on a DK but not on your custom board is if your custom board has a 32.768 kHz crystal? If not, you need to configure the SoftDevice to use the LFRC instead, as explained in this post. And you need to make sure that you have followed the instructions under Transferring the project to nRF52811 hardware.

If that does not help, the next step is to start debugging to see what happens on the device.

As you have a DK, I would recommend that you use that as a debugger instead of the ST-Link, as that way you will be able to use all the tools we provide which works with J-Link out of the box. See Debug output in the DK documentation for details on how to use the DK as a debugger for an external device.

Regarding NRF_LOG_ output, you can get that either via UART or Segger RTT. If you have routed out pins for UART you can hook that up, and configure the logger module for use with UART. Or you can use RTT. In that case, you just need to use a Segger debugger (like a DK). See Logger module for details.