Need to track nRF Connect SDK repos in org specific repo

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Mon, 14 Nov 2022 16:03:42 GMT

Feel free to close this ticket at your earliest convenience. Thank you again for your time with this issue!

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Mon, 14 Nov 2022 16:03:41 GMT

Feel free to close this ticket at your earliest convenience. Thank you again for your time with this issue!

RE: Need to track nRF Connect SDK repos in org specific repo

Jared — Wed, 09 Nov 2022 09:57:24 GMT

Alin,

That was actually going to be my next suggestion if the previous one wouldn't work out for you.. So I guess I don't have anything better in mind. Unfortunately..

regards

Jared

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Tue, 08 Nov 2022 15:45:55 GMT

Hello Jared,

We decided to create a zip file with the NCS version we used for this initial development, and committed that in a separate repository. That seemed like the most simple solution to the issue we were having. Feel free to comment if you can think of any issues with this approach.

Thank you for you time in this matter!

Alin

RE: Need to track nRF Connect SDK repos in org specific repo

Jared — Mon, 07 Nov 2022 13:27:02 GMT

Hi,

What about skipping the fork altogether, and simply adding a backup remote to all the repositories that you host on your own git server, or on github. Then you just have to add a second remote to all the repositories and push them once after you have installed NCS. If ever one of the original repositories where to go offline you could alter the west manifest to point to your backup copy rather than the original.

regards

Jared

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Thu, 03 Nov 2022 19:19:26 GMT

Hi Jared,

If we fork the NCS repo from https://github.com/orgs/nrfconnect we will end up in the same situation we are currently in where we will not be able to track the SDK in our private repository. That is because the forked repos will still be just that, git repositories that are not trackable by Sourcetree. So it seems that there is no difference between obtaining NCS through Toolchain Manager which is what we did and forking each repo as you suggested. In fact the latter is much more cumbersome. I will restate our situation with some more detail in case I was not clear about our needs before.

We have a project that contains two applications and we are trying to add a simple BLE application based on NUS central_uart that will be upgradable over serial DFU. We will release our project soon and we plan to update the BLE application at a later time with a version that supports BLE functionality. Since we are releasing our project with enough functionality to perform DFU but we will not be developing the BLE functionality until later down the road, we need to have access to the nRF Connect SDK version X used for this minimal development. Since there are no guarantees that NCS version X will be available when we will start development for the formal BLE application, we have to check the SDK in our private repository.

Below is the project file structure for your reference:

C:\local_repo\.git - project level

C:\local_repo\.gitignore - project level

C:\local_repo\app1

C:\local_repo\app2

C:\local_repo\BLE\nRF_Connect\v2.1.0 - nRF Connect SDK version 2.1.0 obtained with Toolchain Manager

C:\local_repo\BLE\nRF_Connect\toolchains - Toolchain version 2.1.0 obtained with Toolchain Manager

C:\local_repo\BLE\nRF_Connect\ble_app\ble_app_freestanding\central_uart - Freestanding application based on NUS central uart

Thanks for your time,

Alin

RE: Need to track nRF Connect SDK repos in org specific repo

Jared — Thu, 03 Nov 2022 13:52:15 GMT

Hi,

West use Git , I think that's why it's complaining when you remove it. I'm not sure exactly what Simon was referring to.

[quote user="screen_name"]Can you elaborate a little more what you mean by "...this is a bit up to the user"?[/quote]

By "up to the user" I meant to say that Nordic doesn't give any guarantees as far as I know that any of the repos will be available for the next seven year, thus it will be up to the user to make sure that they have a local copy of the repos that they would want to have guaranteed access to later.

I think you have to fork the NCS repo, in addition to every other repo that is pointed to in the west.yml file.

regards

Jared

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Thu, 03 Nov 2022 03:04:17 GMT

Hello Jared,

Regarding your suggestion to create a fork of each repo within NCS for later access...

Simon had shared a post in his response where it was mentioned that we could remove the .git folders from the entire NCS directory. Would that achieve the same as creating a fork of each repository in NCS in order to track it in our private repo?

[quote userid="72692" url="~/f/nordic-q-a/93028/need-to-track-nrf-connect-sdk-repos-in-org-specific-repo/392358"]

Open the command line in C:\local_repo\BLE\nRF_Connect\v2.1.0 and execute the following command https://stackoverflow.com/a/32022157, then run git init. Does this achive the workflow you're trying to implement?

The reason you need to delete all .git* related files is because all of the subfolders inside nRF_Connect\v2.1.0 are associated with a git repository.

[/quote]

We did not run the command to remove all .git folders from NCS since we cannot track changes in that directory and the result seemed risky, but we did remove a single .git from C:\local_repo\BLE\nRF_Connect\v2.1.0\zephyr and when we tried to build the application in VS Code we noticed that it failed.

Once we added the .git folder back, we were able to build the application again. FYI, our project is based on the central_uart freestanding app.

Can you elaborate a little more what you mean by "...this is a bit up to the user"?

[quote userid="73165" url="~/f/nordic-q-a/93028/need-to-track-nrf-connect-sdk-repos-in-org-specific-repo/393611"]As far as I know we don't provide any guarantees of the longevity of any of our NCS repos, this is a bit up to the user. [/quote]

We are trying to save a version of NCS to our private repo in order to guarantee that we can build the project in the future but we are unable to track its contents. If we remove the .git folders from NCS, we are able to track it but we are unable to build the application anymore because there seems to be a build dependency on the .git folders.

Regards,

Alin

RE: Need to track nRF Connect SDK repos in org specific repo

Jared — Wed, 02 Nov 2022 10:13:45 GMT

Hi,

[quote user="screen_name"]We are concerned that since NCS is comprised of multiple repos that we could run into an issue in the future where a library is no longer supported. Thus, if we use West to manage NCS, we would not be able to retrieve those files and we would be unable to create the firmware to match what was originally released. We noticed it is stated in the NCS documentation that both nRF and OSS repos are created and maintained by Nordic. What does this mean in terms of having access to previous versions (multiple years old) of NCS and availability of any of the libraries over time?[/quote]

As far as I know we don't provide any guarantees of the longevity of any of our NCS repos, this is a bit up to the user. It might be a bit cumbersome , but couldn't you fork the repos that you would want to have access to later?

regards

Jared

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Mon, 31 Oct 2022 15:57:15 GMT

Hello Simon,

Thank you for your reply! I want to take a step back and provide some additional information about the reason why we are interested in managing the nRF Connect SDK in our private repository.

Our organization is required to support a version of software for up to seven years after its release and we must ensure we have access to the exact code (Nordic SDK included) as it was used in the original release. If we release our firmware today to include NCS version 2.1.0 as an example, we would need to be able to have access to all its libraries as they are available today at any time for the next seven years.

We are concerned that since NCS is comprised of multiple repos that we could run into an issue in the future where a library is no longer supported. Thus, if we use West to manage NCS, we would not be able to retrieve those files and we would be unable to create the firmware to match what was originally released. We noticed it is stated in the NCS documentation that both nRF and OSS repos are created and maintained by Nordic. What does this mean in terms of having access to previous versions (multiple years old) of NCS and availability of any of the libraries over time?

For the record, when we release an update to our firmware, assuming it makes sense to update and we don't encounter any issues with the latest version of NCS, we may update to that latest version but we will likely not need to modify any of the SDK files.

Thank you in advance,

Alin

RE: Need to track nRF Connect SDK repos in org specific repo

Simon — Tue, 25 Oct 2022 11:14:22 GMT

At the bottom of this reply I have provided a potential solution to your issue.

I'm sure you have thought about your workflow and why you want to go for it, but in case you haven't, I'll try to give some explanations of how the nRF Connect SDK works regarding this.

I would like to know why you want to manage the whole NCS folder and all it's sub-projects as one entity? The nRF Connect SDK and the nRF5 SDK is quite different. The nRF Connect SDK is a compilation of a bunch of independent modules/repositories, like MCUboot, Zephyr, nrfxlib, hal_nordic, nrf itself etc.. All of them are associated with a git repository. The nrf repository is the manifest repository, which means that if you go to C:\local_repo\BLE\nRF_Connect\v2.1.0/nrf and run

git checkout v2.1.1
west update

All the other sub repositories will automatically be updated to the correct version. For example mcuboot v1.9.99-ncs2 is associated with NCS v2.1.1, and will be checked out when running west update.

If you use your approach, you won't be able to easily upgrade and manage NCS that easily.

Of course you can install a new version through the Toolchain Manager each time you want to do an NCS upgrade, but then why bother to manage it yourself at all through git? If you first want to manage it yourself, then you should consider doing it through git/west as explained above

Let me propose another workflow

Check out the workflow Application as the manifest repository. Check out the sample https://github.com/simon-iversen/application_as_manifest for a demonstration.

In this workflow, your application is the manifest repo instead of nrf. All you need to do to update NCS version is to change the west.yml nrf revision appropriately and run west update from your sample. This workflow is easier to have control over than your workflow. For example if you want to share your project with a colleague, they can just get your project and run west update, and it will automatically check out the correct NCS (nrf) version (and the correct mcuboot version, Zephyr version and so on)..

If you want to modify some drivers inside Zephyr for example, you can create a custom Zephyr fork, modify it, and add it to the west.yml file (if you run west update, your custom Zephyr fork will be checked out instead).

If you want to add your own drivers/boards, you can add it directly to your project like done in the Out Of Tree Driver sample and the Out Of Tree Board sample.

[quote user="screen_name"]We would like to commit nRF Connect SDK to our remote repository but have been unable to track any changes in the directory where it is located. [/quote]

The reason you need to delete all .git* related files is because all of the subfolders inside nRF_Connect\v2.1.0 are associated with a git repository.

Best regards,

Simon

RE: Need to track nRF Connect SDK repos in org specific repo

screen_name — Mon, 24 Oct 2022 13:25:47 GMT

Hi Simon,

Sorry about the delayed reply!

We would like to commit nRF Connect SDK to our remote repository but have been unable to track any changes in the directory where it is located. We use Sourcetree git GUI client for Windows to track changes in C:\local_repo for app1 and app2, and we would like to do the same with C:\local_repo\BLE\nRF_Connect\v2.1.0 and C:\local_repo\BLE\nRF_Connect\toolchains. This directory contains the nRF Connect SDK version 2.1.0 which we obtained along with the appropriate toolchain using Toolchain Manager.

Local repository file structure:

C:\local_repo\app1

C:\local_repo\app2

C:\local_repo\BLE\nRF_Connect\v2.1.0

C:\local_repo\BLE\nRF_Connect\toolchains

C:\local_repo\BLE\nRF_Connect\ble_app\ble_app_freestanding\central_uart

Best,

Alin

RE: Need to track nRF Connect SDK repos in org specific repo

Simon — Thu, 20 Oct 2022 09:57:57 GMT

[quote user=""]We followed workflow 1 and 2 which seem to most closely fit our need but we were unsuccessful in either instance. Is there a way to proceed with our approach? If so, can you provide documentation or the necessary steps to help us do so?[/quote]

Can you elaborate more on this. Why were you unsuccessful with workflow 1 and 2?

Best regards,

Simon