Compare sdk_config files

RE: Compare sdk_config files

Kenneth — Wed, 20 Jul 2022 07:52:16 GMT

Thanks for sharing!

RE: Compare sdk_config files

Yohann — Wed, 20 Jul 2022 04:59:47 GMT

I made a python script to pretty print the differences between sdk_config.h files.

It looks like this:

$ ./compare_sdkconfig.py examples/sdk_config_1.h examples/sdk_config_2.h
╒════════════════════════════════════╤═══════════════════╤═══════════════════╕
│ Define                             │   examples/sdk_co │ examples/sdk_co   │
│                                    │          nfig_1.h │ nfig_2.h          │
╞════════════════════════════════════╪═══════════════════╪═══════════════════╡
│ NRF_RADIO_ANTENNA_COUNT            │                12 │ 3                 │
├────────────────────────────────────┼───────────────────┼───────────────────┤
│ DTM_ANOMALY_172_TIMER_IRQ_PRIORITY │                 2 │ None              │
╘════════════════════════════════════╧═══════════════════╧═══════════════════╛

You can find the script here: github.com/.../nrf_sdk_sdkconfig_diff

RE: Compare sdk_config files

Bruno Randolf — Wed, 19 Jan 2022 14:25:37 GMT

Here is a Python script to show the difference between two sdk_config.h files:

import re
from argparse import ArgumentParser
from pprint import pprint

arg_parser = ArgumentParser(description="Nordic SDK sdk_config.h diff utility")
arg_parser.add_argument('-o', '--output', default="def")
arg_parser.add_argument('-i2', '--ignore2', action='store_true')
arg_parser.add_argument('file1')
arg_parser.add_argument('file2')
args = arg_parser.parse_args()

pattern = re.compile(r'\s?#define\s+(\w*)\s+(\w*)\s*')

def read_file(name):
	defs = {}
	file = open(name, "r")
	for line in file:
		match = pattern.match(line)
		if match:
			if match.group(1) == "SDK_CONFIG_H":
				continue
			defs[match.group(1)] = match.group(2)
	return defs

d1 = read_file(args.file1)
d2 = read_file(args.file2)

diff = {}

# find items in d1 which are different in d2, including non existent
for k, v1 in d1.items():
	v2 = d2.get(k)
	if v2 != v1:
		diff[k] = [v1, v2]

# find items which are in d2 but not d1
for k, v2 in d2.items():
	v1 = d1.get(k)
	if not v1:
		diff[k] = [v1, v2]

# output
if args.output == "pprint":
	pprint(diff)
elif args.output == "list":
	for k in sorted(diff):
		print(k, diff[k])
elif args.output == "def":
	for k in sorted(diff):
		if diff[k][0] and diff[k][1]:
			print("#define", k, diff[k][1], "// was:", diff[k][0])
		elif not diff[k][0]:
			print("#define", k, diff[k][1], "// not set in file 1")
		elif not diff[k][1] and diff[k][0] != "0":
			if not args.ignore2:
				print("#define", k, "0 // not set in file 2, was:", diff[k][0])

RE: Compare sdk_config files

spekary — Thu, 27 Aug 2020 15:27:32 GMT

Your post should be required reading for all Nordic users. This is still a problem in SDK 17. Absolutely horrible architecture. And telling people that they can use an app_config.h file when they really can't?

Nordic, if you really want to support legacy systems, you should have a separate include file for that, and document it.

Also, this architecture makes it extremely difficult to port a project to a newer SDK.

RE: Compare sdk_config files

mtsunstrum — Thu, 14 May 2020 21:54:08 GMT

Yes, at some point in the future we can only hope a major SDK update will enforce users having to upgrade their apps to non-legacy ...

RE: Compare sdk_config files

RayFoulkes — Sat, 02 May 2020 20:46:42 GMT

PPS, I failed to identify the other coders having problems. Well done. The advice needs to be "only put into your own version of sdk_config.h what you absolutely need for your application" the remainder is dangerous.

RE: Compare sdk_config files

RayFoulkes — Sat, 02 May 2020 20:31:10 GMT

I should have added: what did I do? I started with minimal sdk_config.h My code failed to compile. I looked at the first thing that was missing, searched all sdk_config until I found it, copied and pasted the entire chunk in which it appeared. Thereafer rinse and repeat until it compiles... A bit crude but it works ;-)

RE: Compare sdk_config files

RayFoulkes — Sat, 02 May 2020 20:26:13 GMT

Hi mtsunstrum, I had sort-of given up on anyone responding - Phew, I have a fellow coder who understands the issue. Unfortunately there is a BUT. The BUT is that I suspect any fix to this is likely to make some end-user legacy code fail to compile. If it uses the original nrf stuff then anything that triggers nrfx code is likely to screw up the nrf stuff. I suspect that is why they tried to default back to the nrf code - albeit with the consequence that sdk_config cannot be used both by the legacy AND the nrfx stuff.

I really don't have concrete evidence of this, not having tested it, but surely there must be a reason other than failure to separate legacy and new.

In any case, thanks for the support, I wasted quite a bit of time to decide it was a mess. Perhaps others will read and understand this, and thereby avoid wasting their time.

My wish is that Nordic abandon the documentation encouraging the use of app_config.h until (and if) the problem with legacy code is resolved. I don't like the "copy and edit" model for sdk_config.h but at least it works and is simple.

RE: Compare sdk_config files

mtsunstrum — Sat, 02 May 2020 18:21:40 GMT

Oh Ray you hit the nail on the head.

In nrfx_glue.h this tiny line:

#include <legacy/apply_old_config.h>

is the source of all EVIL :} and days of wasted time

This is a very poorly explained legacy issue I think that Nordic should (must) obliterate.

What is killing me is seeing that a fresh download of SDK16.0.0 applies this by default, and many (or possibly all) example projects are based on these legacy nrf_drv implementations. But elsewhere, I see the nrfx_ are the preferred route, so why they have defaulted to legacy is unknown.

Many others are getting tripped up by this issue ...

https://devzone.nordicsemi.com/f/nordic-q-a/59960/how-to-use-the-nrf5_sdk/244087#244087

https://devzone.nordicsemi.com/f/nordic-q-a/60536/do-we-need-to-define-nrfx_timer1_enabled-inorder-to-use-timer1/246209#246209

We should start a Nordic bounty to remove all EVIL legacy stuff in SDK16 … I would throw in $100 … :}

It is never explained properly anywhere, and the 16K line sdk_config.h is full of surprises like this.

RE: Compare sdk_config files

RayFoulkes — Fri, 24 Apr 2020 11:36:29 GMT

Why it seems that using "$NRF_SDK/config/nrf52840/config/sdk_config.h" unchanged from the SDK and overriding with my own "app_config.h" can never work.

This discussion is based on SDK 15.2 and using SES to develop for nrf52840 on both the DK and the dongle. Development environment on both Windows 10 and Linux.
I developed my application using my own version of sdk_config.h edited from an existing one, adding chunks as needed from others until all the packages I needed behaved well. I do not consider this good practice despite all the Nordic provided examples doing exactly that.

I relied on the following from the Nordic infocenter:

Quote[
Overriding the sdk_config.h configuration

Each define in the sdk_config.h file is a conditional define that is added only if it has not been defined previously so it can be easily overriden.

Configuration in sdk_config.h can be overriden in two ways:

    Adding a project flag/define.
    Adding the USE_APP_CONFIG flag to the project settings and creating an app_config.h file which is included at the top of the sdk_config.h file.
]

Therefore I added the path $NRF_SDK/config/nrf52840/config/sdk_config.h at the top of the include search list, defined USE_APP_CONFIG and, sure enough, it accessed my app_config.h file. For the purpose of this explanation I will concentrate on the first problem that this generated. There may be others, but I cannot find an elegant way of avoiding this problem.

Amongst others, I had the lines:
#define NRFX_CLOCK_ENABLED 1
#define NRFX_TIMER_ENABLED 1
#define NRFX_TIMER1_ENABLED 1
#define APP_PWM_ENABLED 1

which had previously been at the top of my own version of sdk_config.h and worked.

Attempting to compile my application, I had a compilation error:

At the line "APP_PWM_INSTANCE(analogue_out,1); // Create the instance "analogue_out" using TIMER1."
I got the error:
error: 'NRFX_TIMER1_INST_IDX' undeclared here (not in a function);

In the crumb-trail back to the problem was the area of Nordic code that creates the value NRFX_TIMER1_INST_IDX which is from nrfx_timer.h:

enum {
#if NRFX_CHECK(NRFX_TIMER0_ENABLED)
    NRFX_TIMER0_INST_IDX,
#endif
#if NRFX_CHECK(NRFX_TIMER1_ENABLED)
    NRFX_TIMER1_INST_IDX,
#endif
#if NRFX_CHECK(NRFX_TIMER2_ENABLED)
    NRFX_TIMER2_INST_IDX,
#endif
#if NRFX_CHECK(NRFX_TIMER3_ENABLED)
    NRFX_TIMER3_INST_IDX,
#endif
#if NRFX_CHECK(NRFX_TIMER4_ENABLED)
    NRFX_TIMER4_INST_IDX,
#endif
    NRFX_TIMER_ENABLED_COUNT
};

This is an anonymous enum which generates an index for enabled timers. Essentially the code NRFX_CHECK(NRFX_TIMER1_ENABLED) turned out to be returning "false" and therefore failed to generate TIMER1 index. But just a minute, I had defined it right at the beginning of includes to be 1 - what gives???

The culprit.

It has to be said that it took me a few Sherlock Holmes moments to get to the bottom of this.

In $NRF_SDK/config/nrf52840/config/sdk_config.h
I found:
#ifndef NRFX_TIMER1_ENABLED
#define NRFX_TIMER1_ENABLED 0
#endif

but that is fine, since I had defined it already as 1 so the first line returned false - I checked - it did. There are no other references to
NRFX_TIMER1_ENABLED in $NRF_SDK/config/nrf52840/config/sdk_config.h so the culprit for undefining or zeroing it is not there.

But there is a culprit and they are the following lines in $NRF_SDK/config/nrf52840/config/sdk_config.h:

// <e> TIMER_ENABLED - nrf_drv_timer - TIMER periperal driver - legacy layer
//==========================================================
#ifndef TIMER_ENABLED
#define TIMER_ENABLED 0
#endif
and
#ifndef TIMER1_ENABLED
#define TIMER1_ENABLED 0
#endif

Now, I had not defined TIMER_ENABLED so the ifndef returned TRUE so TIMER_ENABLED and TIMER1_ENABLED were defined as zero.
"So what?", I hear you ask, "0 means 'not enabled' so that is fine". Well it would be but for some code lurking elsewhere.
Read now the file "apply_old_config.h" which arrives via nrfx_clock.h via nrfx.h via nrfx_glue.h

That file contains the code:

#if defined(TIMER_ENABLED)
#undef NRFX_TIMER_ENABLED
#define NRFX_TIMER_ENABLED TIMER_ENABLED

#if defined(TIMER0_ENABLED)
#undef NRFX_TIMER0_ENABLED
#define NRFX_TIMER0_ENABLED TIMER0_ENABLED
#endif
#if defined(TIMER1_ENABLED)
#undef NRFX_TIMER1_ENABLED
#define NRFX_TIMER1_ENABLED TIMER1_ENABLED
#endif
...ETC.

This file DOESN'T CARE what TIMER_ENABLED is set to, just that it has been defined. It then quite happily sets
NRFX_TIMER1_ENABLED to TIMER1_ENABLED which, of course, $NRF_SDK/config/nrf52840/config/sdk_config.h set to zero - disabled!

TA-DAH Dr Watson, all is revealed! We now know who is the guilty party for the murder of NRFX_TIMER1_ENABLED.

If only $NRF_SDK/config/nrf52840/config/sdk_config.h had not fiddled with TIMER_ENABLED.
Worse, there is nothing you can do in app_config because that it called before the evil deed is done by defining TIMER_ENABLED
essentially we cannot AFAIK "undefine" it in advance. I did think about defining APPLY_OLD_CONFIG_H__ which would block
"apply_old_config.h" from doing anything, but that file is a catch-all for lots of other stuff as well so I cannot predict the side
effects.

For the future - This was only the first of perhaps many problems that direct use of $NRF_SDK/config/nrf52840/config/sdk_config.h
might cause so I conclude that it is just too dangerous to try to use it without modification OR changing the legacy code to
correspond to the later protocol of the value of a definition determining the yes/no and not simply the act of defining.

So I conclude that the entry in the documentation of Nordic Infocenter is essentially useless. Sure, pre-defining things might work, but using a standard sdk_config certainly doesn't. I have checked with SDK 16 : It has the same problem as far as I can tell.

I hope that this explanation will help others in the future - it sure wasted a lot of my time.

RE: Compare sdk_config files

RayFoulkes — Fri, 24 Apr 2020 09:21:47 GMT

Hello, your question seems to lack answers so here is a bit of moral support from Ray. First, I have just been trying to do the same for my project i.e. use "$NRF_SDK/config/nrf52840/config/sdk_config.h" and provide an "app_config.h" to enable the parts of the SDK that I actually need. I previously simply edited an existing sdk_config.h and placed that in the include tree before all the others. I wanted to change to what you propose because, for professonal development, copying huge chunks of an SDK and modifying them for your project is not a good idea since Nordic might decide to change things in the sdk_config.h original thereby making it tricky to upgrade to new versions of the SDK.

Like you in my case it fails to work and, as far as I can tell, can never work. I will try to explain why but first I will try to help you with what I think is a question you asked.

You don't say what OS you are using for development, but you seem to have access to diff which is OK but tough to use for looking for reasons for failure. I think that you are asking what better tool to use for text comparisons. I am developing on Windows 10 and normally use a paid-for tool, UltraCompare, part of the Ultraedit package. If you are using linux, I have used Kompare previously which is a gui file compare system which shows two files side by side and easy to see differences. It is part of KDE but can be used on other distributions if you install it.

If you are developing on Windows 10 then it is possible to install the Linux subsystem, install Kompare and an X server - it runs although it will show an error in the console. Tell me if file comparison is still a problem for you.

I will explain why app_config.h fails to work in my application and why it can never work (I think) in another answer to your original question.

RE: Compare sdk_config files

lybrus — Tue, 14 Apr 2020 12:13:16 GMT

Suddenly, using app_config with diff and replacing sdk_config with base config dosn't work 😞

It's some definitions in base config ($NRF_SDK/config/nrf52840/config/sdk_config.h) aren't neutral. And I have Fourth step, to make it neutral.

RE: Compare sdk_config files

lybrus — Tue, 14 Apr 2020 09:24:14 GMT

I continue to work on it. Trying to make complete config.

First step - find all definitions.

Second - default values.

Third - add definitions to sdk_config.h to correct sections.

I parsed all sdk_config.h from examples, made full list of definitions.
I parsed all /config/**/config/sdk_config.h, made full list of know definition.
Calculate all unknown definition. Total 585.
and tried to find them in SDK folder except 'examples' (grep -Rlwn [defintion] $NRF_SDK --exclude-dir=examples). Some of definitions like '_ENABLED' ended weren't found, I added to search this definition without this ending. Total 374.
Looked into all found definitions, there are seem a lot of trash.

I have assumption, that all sdk libraries must contain '[library_name]_ENABLED' definition. Is it correct? If it's true, I can found all libraries by 'library_name', and left definitions which starts with found 'library_name's.

Regarding second and third step - I don't have any ideas.

RE: Compare sdk_config files

lybrus — Mon, 13 Apr 2020 17:51:44 GMT

Currently it works only with $NRF_SDK/config/nrf52840/config/sdk_config.h as base configuration.
It's not complete configuration, if somebody suggest another one - great.

Any improvements are welcome. Also share your experience from the topic.

RE: Compare sdk_config files

lybrus — Mon, 13 Apr 2020 17:43:06 GMT

I wrote simple npm package, which could be used in command line

https://github.com/lybrus/nrf-config-diff-cli

For blinky_freertos (in $NRF_SDK/examples/peripheral/blinky_freertos/pca10056/blank/config/) it creates file with content:

/* Generated by nrf-config-diff-cli */

#ifndef APP_CONFIG_H__
#define APP_CONFIG_H__

#define USE_APP_CONFIG

/* Changed variables */
#define GPIOTE_ENABLED 1
#define GPIOTE_CONFIG_NUM_OF_LOW_POWER_EVENTS 4
#define NRFX_CLOCK_ENABLED 1
#define NRFX_GPIOTE_ENABLED 1
#define NRF_CLOCK_ENABLED 1
#define APP_TIMER_ENABLED 1
#define BUTTON_ENABLED 1

#endif // APP_CONFIG_H__