continuous integration and automated testing results of nRF SDK and tooling releases

[jahess]

For something as complicated as the nRF Connect SDK and Zephyr I would hope there is continous integration running during development to make sure host platforms, samples, build targets, and etc keep working.

If the results of those tests were ran for releases and made available, as a newbie I would like to be able to dig into that information.  Having such a references would:

1. convince me the samples do work and that it is my own learning curve causing me issues, and
2. if I could find and recognize the steps the automation took to success, I would have a proven reference of steps to learn how to do things from.  Things that aren't always obvious from the sample or reference documentation.

Just an idea.

[Sigurd Hellesvik]

Hi,

The nRF Connect SDK is made up of multiple different repositories, some of which are forks. See https://github.com/nrfconnect.

You can see a list of worklows for each under  "actions", for example https://github.com/nrfconnect/sdk-nrf/actions.

We  also have support for  Testing with Unity and CMock.

Is this what you were looking for?

Regards,
Sigurd Hellesvik

[jahess]

Those are the links I was looking for.   When you originally replied I spent some time clicking around and eventually found links to nordic build machines that are not available from the internet.  I perfectly understand that nordic could and would be working on hardware on projects that they don't want in the open.  But for, build and test log results for samples, ya, I would like to see even that.

As a newbie I'm guessing what I'm asking for is...

Assuming I finally found a sample of interest in documentation.  From a few clicks I would like to see a 2d grid of target boards vs host build platforms.  Cells would have build and test results (pass/fail) for that pair of host and board.  If I click on that cell, I would be taken to the logs for that build and test.  I could then see precisely how it was built and tested and reason about how what I did or was having issue with wasn't the same.

Ideally there would be no difference between build hosts but... we all know there can be exceptions to that goal.  And, if a developer is on one of those platforms, they need to know in advance that they are in store for trouble -- perhaps they can adjust or plan accordingly.

So, the links you provided are more tailored for building pipelines and engineers doing primary development.  Yes, you can see results of those pipelines but... for a developer that is a user of those efforts and that is focused on a single sample, they kind of need to start from the grand summary of sample test results that went into a successful attempt at a release.

Ya, this is kind of a pie in the sky dream.  But if the test logs are there... others might find having simple access to them useful.

[Sigurd Hellesvik]

As you say, this is not public at the time.

I will forward your suggestion.
Thanks for feedback on our nRF Connect SDK!

Regards,
Sigurd Hellesvik

[Sigurd Hellesvik]

Hi,

This is not something that we will make public.

Here are some more resources which might help you instead:

See Protocol Support.

Each sample has a list in its sample.yaml file, showing which platforms have been tested.
For example sample.yaml for Central and Peripheral HR.
integration_platforms: are tested on every commit.
platform_allow: should work but are not tested on every commit.

Lastly, Circuit Dojo have a video on how you can set up CI yourself:
https://www.youtube.com/watch?v=h8Y_anpPrcI

Regards,
Sigurd Hellesvik