I'm working on a project using CBOR, and Nordic's zcbor library in particular, to help with the serialization of data structures that need to be transmitted or stored. We're working with NCS SDK 2.3.0 on Windows with SDK, VS code, etc installed via the nRF Connect for Desktop toolchain manager. In case it matters the target is nRF5340. The zcbor library is found in the SDK at modules/lib/zcbor/. This ticket is focused on using the library's zcbor.py for the purpose of ingesting a CDDL file and generating C source and headers which can be included in an NCS SDK project to encode data into CBOR format.
The zcbor library includes some tests which also serve as examples of how the library could be used. Referring to the test found in modules/lib/zcbor/tests/encode/test2_simple/, there is a CMakeLists.txt example showing how the process of generating C source and header files can be automated in the firmware build process. The file includes the following -- it executes the zcbor.py script with certain arguments; the script generates a file called pet.cmake; and finally that generated pet.cmake file is included so it'll be processed by cmake. I've adapted it to work with the appropriate names for my project.
set(py_command
zcbor
code
-c ${CMAKE_CURRENT_LIST_DIR}/../../cases/pet.cddl
--output-cmake ${PROJECT_BINARY_DIR}/pet.cmake
-t Pet
-e
${bit_arg}
--short-names
)
execute_process(
COMMAND ${py_command}
COMMAND_ERROR_IS_FATAL ANY
)
include(${PROJECT_BINARY_DIR}/pet.cmake)
I've encountered two problems in trying to run the zcbor.py script as part of the build. It isn't clear to me whether the problems are with the library itself or with something about the NCS compilation environment, nor whether it's appropriate to report zcbor problems through this forum.
The first problem is in a call to python's makedirs(). The call includes a path element "./". When zcbor.py is executed as part of the NCS build there's a failure: this makedirs() call gets an argument like ./C:\ncs\src\nrf5340\build\eventlog and obviously that's not a valid path -- but when I run zcbor.py myself in a cmd.exe or powershell shell this doesn't occur. If I remove the "./" from the makedirs() call the script works correctly both in the shell and in the NCS build. It's not clear whether this breakdown when run through the NCS build environment could be a problem with the SDK/build environment (or my configuration).
def render(self, modes, h_files, c_files, type_file, include_prefix, cmake_file=None,
output_c_dir=None, output_h_dir=None):
for mode in modes:
h_name = Path(include_prefix, Path(h_files[mode].name).name)
# Create and populate the generated c and h file.
makedirs("./" + path.dirname(c_files[mode].name), exist_ok=True)
Traceback (most recent call last):
File "C:/ncs/v2.3.0/zephyr/../modules/lib/zcbor/zcbor/zcbor.py", line 3085, in <module>
main()
File "C:/ncs/v2.3.0/zephyr/../modules/lib/zcbor/zcbor/zcbor.py", line 3081, in main
args.process(args)
File "C:/ncs/v2.3.0/zephyr/../modules/lib/zcbor/zcbor/zcbor.py", line 3015, in process_code
renderer.render(modes, output_h, output_c, output_h_types, args.include_prefix,
File "C:/ncs/v2.3.0/zephyr/../modules/lib/zcbor/zcbor/zcbor.py", line 2690, in render
makedirs("./" + path.dirname(c_files[mode].name), exist_ok=True)
File "os.py", line 213, in makedirs
File "os.py", line 213, in makedirs
File "os.py", line 213, in makedirs
[Previous line repeated 4 more times]
File "os.py", line 223, in makedirs
OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: './C:'
CMake Error at CMakeLists.txt:57 (execute_process):
execute_process failed command indexes:
1: "Child return code: 1"
The second problem is in the zcbor.py render_cmake_file() method. This method writes the generated cmake file to disk. Because I'm working on Windows the Path() calls in this method generate file paths using the Windows separator: the single backslash. Apparently Cmake doesn't like single backslash, even on Windows. It raises an error because it thinks these are intended to be escape sequences.
def render_cmake_file(self, target_name, h_files, c_files, type_file,
output_c_dir, output_h_dir):
return \
f"""\
#
# Generated using zcbor version {self.version}
# https://github.com/NordicSemiconductor/zcbor{'''
# at: ''' + datetime.now().strftime('%Y-%m-%d %H:%M:%S') if self.print_time else ''}
#
add_library({target_name})
target_sources({target_name} PRIVATE
{Path(output_c_dir, "zcbor_decode.c")}
{Path(output_c_dir, "zcbor_encode.c")}
{Path(output_c_dir, "zcbor_common.c")}
{(linesep + " ").join((c.name for c in c_files.values()))}
)
target_include_directories({target_name} PUBLIC
{(linesep + " ").join(set((str(Path(output_h_dir)),
str(Path(type_file.name).parent),
*(str(Path(h.name).parent) for h in h_files.values()))))}
)
"""
Writing to C:\ncs\src\nrf5340\build\eventlog\src\eventlog_encode.c
Writing to C:\ncs\src\nrf5340\build\eventlog\include\eventlog_encode.h
Writing to C:\ncs\src\nrf5340\build\eventlog\include\eventlog_types.h
Writing to C:\ncs\src\nrf5340\build\eventlog\eventlog.cmake
CMake Error at build/eventlog/eventlog.cmake:7 (target_sources):
Syntax error in cmake code at
C:/ncs/src/nrf5340/build/eventlog/eventlog.cmake:8
when parsing string
C:\ncs\v2.3.0\zephyr\..\modules\lib\zcbor\src\zcbor_decode.c
Invalid character escape '\v'.
Following is the cmake file that generated the above error.
#
# Generated using zcbor version 0.6.0
# https://github.com/NordicSemiconductor/zcbor
#
add_library(eventlog)
target_sources(eventlog PRIVATE
C:\ncs\v2.3.0\zephyr\..\modules\lib\zcbor\src\zcbor_decode.c
C:\ncs\v2.3.0\zephyr\..\modules\lib\zcbor\src\zcbor_encode.c
C:\ncs\v2.3.0\zephyr\..\modules\lib\zcbor\src\zcbor_common.c
C:\ncs\src\nrf5340\build\eventlog\src\eventlog_encode.c
)
target_include_directories(eventlog PUBLIC
C:\ncs\v2.3.0\zephyr\..\modules\lib\zcbor\include
C:\ncs\src\nrf5340\build\eventlog\include
)
I modified this render_cmake_file() method so that it always uses pathlib's as_posix() method -- with this change the generated cmake file uses forward slash as the directory separator (C:/ncs/src/<directory>).
def render_cmake_file(self, target_name, h_files, c_files, type_file,
output_c_dir, output_h_dir):
return \
f"""\
#
# Generated using zcbor version {self.version}
# https://github.com/NordicSemiconductor/zcbor{'''
# at: ''' + datetime.now().strftime('%Y-%m-%d %H:%M:%S') if self.print_time else ''}
#
add_library({target_name})
target_sources({target_name} PRIVATE
{Path(output_c_dir, "zcbor_decode.c").as_posix()}
{Path(output_c_dir, "zcbor_encode.c").as_posix()}
{Path(output_c_dir, "zcbor_common.c").as_posix()}
{(linesep + " ").join((Path(c.name).as_posix() for c in c_files.values()))}
)
target_include_directories({target_name} PUBLIC
{(linesep + " ").join(set((str(Path(output_h_dir).as_posix()),
str(Path(type_file.name).parent.as_posix()),
*(str(Path(h.name).parent.as_posix()) for h in h_files.values()))))}
)
"""
As explained I've worked out modifications to zcbor.py so that I can move forward. But modifications to the SDK are highly undesirable. I'm looking for your advice as to whether this all should have worked without any change to zcbor.py, and if so, what changes my project or NCS build environment might need. On the other hand, if the changes to zcbor.py appear necessary, please indicate whether the report here is good enough or whether I should re-post in Nordic's zcbor github project.
