undefined reference to _open when using libc with NCS 2.4.2 and above

[pnc2]

Hello,

I am developing an application for a custom nrf52840 board. This application has been building and running without problems for nrf Connect SDK v 2.3.0

However, I recently tried to upgrade to NCS v2.5.0 and I get the below link-time error:

D:/ncs2/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi/lib/thumb/v7e-m/nofp\libc.a(lib_a-openr.o): in function `_open_r':
openr.c:(.text._open_r+0x14): undefined reference to `_open'

Note that 

CONFIG_NEWLIB_LIBC=y

CONFIG_FILE_SYSTEM=y

CONFIG_FILE_SYSTEM_LITTLEFS=y

is set in my prj.conf. I also see these lines in autoconf.h: 

#define CONFIG_FULL_LIBC_SUPPORTED 1
#define CONFIG_MINIMAL_LIBC_SUPPORTED 1
#define CONFIG_NEWLIB_LIBC_SUPPORTED 1
#define CONFIG_PICOLIBC_SUPPORTED 1
#define CONFIG_NEWLIB_LIBC 1
#define CONFIG_HAS_NEWLIB_LIBC_NANO 1

I tried to downgrade to NCS 2.4.2, but it seems to have the same issue. However, no problems with NCS v2.3.0

Has something changed since NCS 2.4.2? Am I supposed to enable some other config flag as well, in order to use full newlib libc?

thanks

regards

pnc

[pnc2]

Hello,

following on from above query, I have tested the following setting also:

CONFIG_NEWLIB_LIBC=y

CONFIG_NEWLIB_LIBC_NANO=n

However, there is no change in the situation.

Further, I also tried 

CONFIG_NEWLIB_LIBC=y

CONFIG_NEWLIB_LIBC_NANO=y

In this case, the build goes through, but I get run-time errors (divide by zero error somewhere in the Kernel code) at the startup , during Kernel boot. This might be because my application code requires the full LIBC, and not the Nano version. I am unable to debug, as the debugger also throws an exception.

Just to repeat: this code works perfectly with NCS 2.3.0, so it is definitely some issue with 2.5.0 

Appreciate any help, as I am badly stuck.

[AHaug]

Hi, 

CONFIG_NEWLIB_LIBC=y should not be broken in NCS 2.5.0 as far as I can see as long as all the dependencies are met. Have you also gone through the changelogs in between the releases as well as checked on the SDK main to see if there are any commits that changes this?

I will look into this during the week, but here are at least some starting points

Kind regards,
Andreas

[pnc2]

Hi Ahaug,

any updates on this? I am still unable to use Newlib. The only way I can get my app to build is by setting 

CONFIG_NEWLIB_LIBC=n

CONFIG_NEWLIB_LIBC_NANO=n

In this case, I presume the default Pico clib gets included. My app seems to be working. I had activated Newlib at one time for a specific reason, but probably that reason is no longer valid. But I would certainly like to have the option to use Newlib in future, and I am sure other users would too. So I request you to please look into this. Regarding the links sent by you, I had a look, but it appears impossible to figure out which change might or might not be responsible for this, there are so many of them, many of which I don't understand.

thanks

regards

pnc2

[AHaug]

Hi,

Apologies for the long response time. I was out of office at the end of last week and before that I was waiting for a reply from you regarding my questions in my previous reply. 

So to continue from here:

1. There are changes in between NCS v2.5.0 and v2.3.0 w.r.t dependencies of this configuration. I assume you've already investigated this and checked if you have met all the dependencies?

2. The error is also described in this Zephyr issue https://github.com/zephyrproject-rtos/zephyr/issues/41694. Have you had a look at the Zephyr forums to see if this report solves the issue?

3. Could you provide me with the your prj.conf, the .conf from build/zephyr as well as the full build error log from the faulty compilation?

Kind regards,
Andreas

[pnc2]

Hi AHaug,

I checked out your references, they are all OK in my case (I have verified by looking into autoconf.h). 

I have managed to reproduce the problem with a NORDIC board, instead of my custom board, with a tiny program. Please follow these steps to reproduce:

clone the Hello world sample.

Modify main.c like so:

#include <stdio.h>
int main(void)
{
printf("Hello World!\n");
FILE *f = fopen("posix.txt","a");
return 0;
}

modify prj.conf like so:

CONFIG_NEWLIB_LIBC=y

CONFIG_NEWLIB_LIBC_NANO=n

CONFIG_FILE_SYSTEM=y

CONFIG_FILE_SYSTEM_LITTLEFS=y

CONFIG_FLASH=y

CONFIG_FLASH_MAP=y

CONFIG_FLASH_PAGE_LAYOUT=y

CONFIG_POSIX_API=y

Create a build configuration for nrf52840dk_nrf52840

Build, you will get this error:

D:/ncs2/toolchains/c57af46cb7/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi/lib/thumb/v7e-m/nofp\libc.a(lib_a-openr.o): in function `_open_r':
openr.c:(.text._open_r+0x14): undefined reference to `_open'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

HOWEVER, set 

CONFIG_NEWLIB_LIBC=n

And it will build.

Also, downgrade NCS to 2.3.0, and it will build.

Hope you will be able to reproduce it now

NOTE: the fopen is required, as that seems to be the source of the problem. In my code, I use posix calls for file IO. This is a technical requirement, in my case, I cannot avoid it. 

regards

[pnc2]

By the way, I have also now tried it with NCS2.5.99-dev1, and the problem is still there.

[Håkon Alseth]

Hi,

 

fopen() is a std lib function that is provided by the libc that is chosen, and for newlib (and newlib-nano), this is provided by the toolchain that you use.

I would recommend that you use the zephyr fs subsys.

 

Kind regards,

Håkon

[pnc2]

Hi Hakon,

as you will see from the above messages, I already have 

CONFIG_FILE_SYSTEM=y

I use the posix file API for a particular reason (as I mentioned in my first message) It is a project requirement. Also, as previously mentioned, my code works fine with NCS 2.3.0 and below. Something seems to have broken with NCS 2.5.0. As for toolchain, I use the toolchain that your Toolchain Manager installs along with NCS 2.5.0

I have also mentioned above that I have reproduced this problem above with a Nordic board and a standard sample. I request you to please look into this.

Thanks

[Håkon Alseth]

Hi,

 

The POSIX support in zephyr is partial for DEVICE_IO, as shown here:

https://docs.zephyrproject.org/latest/services/portability/posix/option_groups/index.html#posix-device-io

When adding CONFIG_POSIX_FS=y, I see that this compiles in ncs v2.3.0 (zephyr v3.2.99-ncs1) and fails to compile in ncs v2.5.0.

If we fallback to picolib (CONFIG_PICOLIB=y), the default selected libc in zephyr, it does compile as expected.

 

I would recommend that you open a issue on zephyr github wrt. POSIX compatibility:

https://github.com/zephyrproject-rtos/zephyr/issues

 

Kind regards,

Håkon

[pnc2]

Hi Hakon,

as suggested by you, I posted this to Zephyr issues forum. Here is the link:

https://github.com/zephyrproject-rtos/zephyr/issues/66132

This was the "helpful" response I got:

====================================================

Thank you for reporting this. However - unless you are able to reproduce this issue with upstream Zephyr main - please report issues with the nRF Connect SDK (NCS) on the Nordic Semiconductor DevZone.

==================================================

As mentioned previously, I already have tried it out with the latest development version of NCS 2.5.99-dev1 ( Zephyr 3.4.99). Unfortunately, you do not use upstream Zephyr, but a fork of Zephyr currently at 3.4.99, and it is clear that the Zephyr folk won't help me unless I can reproduce this with upstream Zephyr main (currently at 3.5.99).

As my organization's hardware vendor, it is certainly your responsibility to fix this issue, rather than batting this down to Zephyr. 

[Håkon Alseth]

I checked out zephyr HEAD (commit 3d3b287b7908625dc8356ac3532d9b6943edbb50), and tried to compile this sample for board nrf52dk_nrf52832:
hello_world_posix.zip

 

It will fail with the following output:

$ ninja
[1/174] Preparing syscall dependency handling

[3/174] Generating include/generated/version.h
-- Zephyr version: 3.5.99 (/opt/repos/upstream-zephyr/zephyr), build: zephyr-v3.5.0-2646-g3d3b287b7908
[10/174] Building C object CMakeFiles/app.dir/src/main.c.obj
/opt/repos/upstream-zephyr/zephyr/samples/hello_world_posix/src/main.c: In function 'main':
/opt/repos/upstream-zephyr/zephyr/samples/hello_world_posix/src/main.c:12:15: warning: unused variable 'f' [-Wunused-variable]
   12 |         FILE *f = fopen("posix.txt", "a");
      |               ^
[168/174] Linking C executable zephyr/zephyr_pre0.elf
FAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map /opt/repos/upstream-zephyr/zephyr/samples/hello_world_posix/build/zephyr/zephyr_pre0.map 
: && ccache /opt/zephyr-sdk/zephyr-sdk-0.16.0/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -fuse-ld=bfd  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=/opt/repos/upstream-zephyr/zephyr/samples/hello_world_posix/build/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/newlib/liblib__libc__newlib.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/lib/posix/liblib__posix.a  zephyr/soc/soc/arm/nordic_nrf/nrf52/libsoc__arm__nordic_nrf__nrf52.a  zephyr/subsys/fs/libsubsys__fs.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/flash/libdrivers__flash.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  modules/littlefs/libmodules__littlefs.a  modules/segger/libmodules__segger.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L"/opt/zephyr-sdk/zephyr-sdk-0.16.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp"  -L/opt/repos/upstream-zephyr/zephyr/samples/hello_world_posix/build/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -mfp16-format=ieee  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -lm  -Wl,-lc  -L"/opt/zephyr-sdk/zephyr-sdk-0.16.0/arm-zephyr-eabi/arm-zephyr-eabi"/lib/thumb/v7e-m/nofp  -Wl,-lgcc  -lc && cd /opt/repos/upstream-zephyr/zephyr/samples/hello_world_posix/build/zephyr && /usr/bin/cmake -E true
/opt/zephyr-sdk/zephyr-sdk-0.16.0/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /opt/zephyr-sdk/zephyr-sdk-0.16.0/arm-zephyr-eabi/arm-zephyr-eabi/lib/thumb/v7e-m/nofp/libc.a(lib_a-openr.o): in function `_open_r':
openr.c:(.text._open_r+0x14): undefined reference to `_open'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

If you use picolib instead of newlib, it will compile as expected.

 

You can share this in the github issue if you'd like.

 

Kind regards,

Håkon