nRF53: What files to add to project Git Repo

RE: nRF53: What files to add to project Git Repo

AHaug — Wed, 27 Apr 2022 10:29:10 GMT

[quote user="bbbakke"]This sounds like a useful document, unfortunately I can't access it. Is this document public? Can you provide a link that is not in your "support-private" URL path?[/quote]

The wrong url was pasted in the hyperlink. This should be the correct url to managing code base. Does it work?

[quote user="bbbakke"] turns out the thing you and I did not know is, git does automatically ignore any sub-folder that contains a .git folder (i.e., the sub-folder is already a repo so no need to track any files contain in it within the parent repo). [/quote]

Then I learned something new aswell. Thanks for sharing!

[quote user="bbbakke"]So the solution is to: [/quote]

1-2 sounds right.

I agree that 3.a needs to be checked. It could be containing (just speculation about what could go wrong) some info about how the environment/paths is set up (similar to appdata for vscode) but this needs to be verified.

3b sounds like a plan.

4-6 I agree.

[quote user="bbbakke"]Would be nice if the Nordic Team had this documented somewhere, maybe in the document link listed above that didn't work for me[/quote]

I'll ask around the docs/product marketing team if we should make a guide/blog about this. Seems like its nice to have after you clarified some of the misunderstandings we had in this thread.

Kind regards,
Andreas

RE: nRF53: What files to add to project Git Repo

bbbakke — Tue, 26 Apr 2022 17:14:28 GMT

[quote userid="107683" url="~/f/nordic-q-a/87025/nrf53-what-files-to-add-to-project-git-repo/364850#364850"]If you investigate the folders from your workspace setup and compare it to the SDK installation folder from the automatic installation you will find that they contain the same files, e.g you clone the entire SDK with the exception of the west.yml file within your application folder.[/quote]

Of course it clones of the nRF SDK, Zephyr, bootloadeer, etc., git repos, that is exactly the purpose of the "west" tool. But, more importantly, it clones exact and specific versions. That is the benefit of using west workspaces. When you have a multi-developer team, you don't want one on SDK ver 1.7.0, another on 1.9.99, and a third on the newest release made(hypothetically) this week. You want everyone on the same version and mange upgrades to newer version if and when appropriate.

[quote userid="107683" url="~/f/nordic-q-a/87025/nrf53-what-files-to-add-to-project-git-repo/364850#364850"]Tech support at Nordic Semiconductor are a part of the engineering team, so you've already reached us.[/quote]

But we were not making any progress in our discuss so I hope you would reach out to others.

[quote userid="107683" url="~/f/nordic-q-a/87025/nrf53-what-files-to-add-to-project-git-repo/364850#364850"]Managing the code base should give you some tips and tricks if you want to work with git based development and how the SDK folders are related through git and west as well as more info about how to instance [/quote]

This sounds like a useful document, unfortunately I can't access it. Is this document public? Can you provide a link that is not in your "support-private" URL path?

[quote userid="107683" url="~/f/nordic-q-a/87025/nrf53-what-files-to-add-to-project-git-repo/364850#364850"]Brute force through deleting folders in the SDK, upload to git, clone back, pristine build and flash to see if it works. If it does not work, then there are some dependencies that causes you to require that folder.[/quote]

This is what I tried yesterday.

I initialized git with "git init" in the "PeripheralUART" directory.
I added a .gitignore with the build directory so it would be ignored.
I look at the files git lists as untracked.

Guess what, none of the clone repos, nRF SDK, Zephyr, bootloadeer, etc., where listed. It was as if git was ignoring them even though they were not in the .gitignore. After some google searches and StackOverflow reading, turns out the thing you and I did not know is, git does automatically ignore any sub-folder that contains a .git folder (i.e., the sub-folder is already a repo so no need to track any files contain in it within the parent repo).

So the solution is to:

Create the initial project from the sample of choice in VS Code as a "Workspace" (a product development team should not use freestanding IMHO, and if you do, make sure the version of all other repos used in the build and toolchain are well documented if you want to be able to build it a year later for maintenance).
Initialize your git repo in the parent directory of the project folder (where the .west folder exists).
Add a .gitignore file:
1. Ignore ".vscode" folders (Is this correct? Need to check.)
2. Ignore "build" folders (I have a problem with this because ignoring the build directory means that the build configuration is not tracked in git, but I will take that up in a different thread).
Add all of the project folders and files, including the .west subdirectory which has the west manifest (config file).
Do the commit and push it (you will need to setup your remote URL).
When you or a team member clones the project:
1. Open the app project folder in VS Code (with the nRF Connect plugin installed of course)
2. Open a nRF terminal (it has the correct env to run west commands)
3. Run "west update" and it will pull in all the other required repos with the exact version everyone else is using.

Hope this helps others. Would be nice if the Nordic Team had this documented somewhere, maybe in the document link listed above that didn't work for me. Hopefully that link will be fixed / made public.

-Brad

RE: nRF53: What files to add to project Git Repo

AHaug — Tue, 26 Apr 2022 08:09:05 GMT

Hi Brad,

[quote user="bbbakke"]I suggest that you try creating a project from a sample as I have with the Nordic tools, specifically in VS Code with the nRF Connect Extension Pack. [/quote]

This is what I based my previous answer on. If you investigate the folders from your workspace setup and compare it to the SDK installation folder from the automatic installation you will find that they contain the same files, e.g you clone the entire SDK with the exception of the west.yml file within your application folder.

[quote user="bbbakke"]but the nRF Connect VS Code plug-in (through the use of "west workspace") cloned all these additional git repos.[/quote]

Yes, this is what "Application type workspace" does. It "sets up" the SDK in a new instance where you get your instance verified with the addition of a west.yml file that instances which version this application is for. In a freestanding this is not done and instancing and organizing where to put the application is left to the developer. You could compare the west.yml in peripheral_uart in your peripheralUART workspace and consult with the resources below if instancing the project fixes your issue.

[quote user="bbbakke"]The guidance I need from your engineering team[/quote]

Tech support at Nordic Semiconductor are a part of the engineering team, so you've already reached us.

[quote user="bbbakke"]But, as you said, I do not want to add these to my git repo as it is unnecessary[/quote]

Without restating my previous reply here are some new suggestions and resources I recommend you try and/or read up on.

Brute force through deleting folders in the SDK, upload to git, clone back, pristine build and flash to see if it works. If it does not work, then there are some dependencies that causes you to require that folder.
Managing the code base should give you some tips and tricks if you want to work with git based development and how the SDK folders are related through git and west as well as more info about how to instance
NCS Structure and content as well as this older, but still valid NCS tutorial should tell you more about how the SDK is set and maybe give you more clues about the dependencies between the SDK folders.

- Andreas

RE: nRF53: What files to add to project Git Repo

bbbakke — Mon, 25 Apr 2022 21:38:39 GMT

[quote userid="107683" url="~/f/nordic-q-a/87025/nrf53-what-files-to-add-to-project-git-repo/364634#364634"]To summarize, if I've understood your configuration properly, you will have to add the whole SDK to the git repo to get it to work as you describe. I would instead recommend you to install the SDK locally either through the automatic install in nRF Connect for Desktop or the manual installation guide linked above, and to only have your application in the repository. Every project I've worked on with NCS has used this "workspace philosophy", since what you want to change is on the application level of folders, and extremely rarely in the SDK files. [/quote]

Actually, I don't think you have understood this correctly. I suggest that you try creating a project from a sample as I have with the Nordic tools, specifically in VS Code with the nRF Connect Extension Pack. If you follow the steps I did, creating the project as a "Workspace" (not Freestanding) you will find that I did not "add the whole SDK to the git repo", but the nRF Connect VS Code plug-in (through the use of "west workspace") cloned all these additional git repos. As you can see in the following screenshot, the Nordic tool tells you the "west workspace" is used to manage the SDK version and it will create its own instance.

This is not bad, it is actually good. The "west" pulls in the correct version of every thing needed to build the app. But, as you said, I do not want to add these to my git repo as it is unnecessary. I do not need a copy in my git repo because the "west" tool will pull in the correct version of the other git repos; that is its purpose.

The guidance I need from your engineering team (please reach out to them for their guidance) is clear definition of what items need to be in my git repo so, when cloned and "west update" is executed, the west tool has the correct information to pull in everything else correctly.

Again, these are the initial steps I did in VS Code (please, try it yourself):

[quote userid="94567" url="~/f/nordic-q-a/87025/nrf53-what-files-to-add-to-project-git-repo/364559#364559"]As I stated I’m used VSCode with the nRF Connect Extension Pack plugin. From the “nRF Connect for VS Code” welcome screen I selected “Create a new application from sample…”. In the next screen, for Application type I selected “Workspace”. [/quote]

If you try this and look in the director above your project you will see all the other repos were cloned as the workspace was created from the sample.

RE: nRF53: What files to add to project Git Repo

AHaug — Mon, 25 Apr 2022 08:45:48 GMT

Hi Brad,

Thanks for filling in with more details on your configuration. I will do my best to answer them in turn!

1) In this case it would be where the SDK is installed, I.E the root workspace folder PeripheralUART. If you initialize the git repo at periphehral_uart, it will only take the application project folder in a similar manner to what I suggested in my previous reply.

Although this might seem like a optimal solution, I believe this will and may cause some possible time consuming issues down the line.

One possible issue is that the SDK- and your project folder is over 4GB large, which will require quite a lot of files to both commit, pull and push whenever you're swapping devices or your colleagues clones the repository.

A second issue I foresee is that you will spend some time to set the SDK up properly due to either some missing paths or that requirements such as nrfjprog has not been installed. The best way to guarantee this is to perform the SDK setup through either the toolchain manager in nRF Connect for Desktop, or through this manual installation guide. Both of these installation methods allows you to quickly swap between west workspaces or SDK versions if desired so after the installation procedure.

2) This will be trickier. If you end up with going for your desired configuration where the SDK is a part of the git-repo, the only files I can guarantee that it would be ok to omit without diving deep into the SDK structure is the sample folders in zephyr and nrf. This will come with a drawback of having to look up the samples either online or in a separate NCS SDK workspace if you want to investigate how a sample such as central_uart sample works.

3) The sample you are suggesting would not require you to change any of the files in SDK or Zephyr, but rather through configuring your application files either within prj.conf in the peripheral_uart folder above or by defined macros in the .c or .h files. All these files can be found within properly initialized application folder. But in case you modify a SDK or Zephyr file, the changes are handled by git! Git only tracks changes in your files, relative to the origin repository. So lets say you have a local repo on your computer that only has the UUID changed (1 code line) relative to the origin, the commit that will be pushed will only be that one line of code.

To summarize, if I've understood your configuration properly, you will have to add the whole SDK to the git repo to get it to work as you describe. I would instead recommend you to install the SDK locally either through the automatic install in nRF Connect for Desktop or the manual installation guide linked above, and to only have your application in the repository. Every project I've worked on with NCS has used this "workspace philosophy", since what you want to change is on the application level of folders, and extremely rarely in the SDK files.

Let me know if this helps you in any way, and feel free to follow up with more questions if you have any regarding this issue/topic!

Kind regards,
Andreas

RE: nRF53: What files to add to project Git Repo

bbbakke — Sat, 23 Apr 2022 21:38:34 GMT

Hello AHaug,

Thank you for that very thorough explanation. It was helpful and I think it will be helpful to many others as well.

I have been developing on nRF528xx devices for some time now, but just getting start with the Zephyr based nRF Connect SDK for a nRF5340 project. Could you help me a little more with a specific case? Sorry for repeating the scenario, but I want to give a little more detail.

As I stated I’m used VSCode with the nRF Connect Extension Pack plugin. From the “nRF Connect for VS Code” welcome screen I selected “Create a new application from sample…”. In the next screen, for Application type I selected “Workspace”. I made this selection because, as the description states,

“Workspace applications use west workspace to version nRF Connect SDK. They will have their own nRF Connect SDK instance”.

This is a great option because it means if I clone the git repo on a different machine, or a colleague clones the repo, we don’t have to worry about what SDK / tool chain version to install, as “west” will pull in everything needed, with the correct versions.

So, when I complete the process, this is what my VS Code project looks like:

And it builds without error, and I can flash it to my nRF5340 dk, step through code, see variables and memory space, etc. Alls great so far but notice in the image above, none of the zephyr or nRF Connect files are in the workspace. So where are they? One level up as you can see in this image:

So, my questions are with respect to this configuration...

1. Where do I initialize the git repo in this directory structure so when it is cloned, and west invoked, the entire project is replicated, which will build, flash, and run without error? Looking at the second image, should it be at the PeripheralUART level? Or the peripheral_uart level?

2. What should the contents of the .gitignore be so only the necessary files are added and tracked in git? I know how to create the .gitignore, but in need to know, for the zephyr based project that uses west workspace, what should be ignored in addition to the build directory.

A follow up question that has come up since my first post

3. If changes are needed to the nRF SDK or zephyr files, how do those get included and tracked in git. For example, if NUS is used but the UUID needs to be changed how would that be done so change would be in the git commit? Would he whole SDK have to be added to the git repo?

Thank you,

Brad

RE: nRF53: What files to add to project Git Repo

AHaug — Tue, 19 Apr 2022 10:27:46 GMT

Hi Brad,

Correct me if I'm wrong, but as far as I'm understanding you're getting started developing and want to upload a NCS-projects that you've written such as a modified peripheral_uart to to your own github repository. Also, in case you're either new or intermediary to developing with nRF Connect for Desktop in VSCode, the Nordic Academy page has a great getting started guide you might want to check out!

[quote user=""]So now I want to take what I have created and get it into a new project Git repo that I will manage on Github. But I don't know what directory to init my git repo,[/quote]

In general you only want your application uploaded to git and to have the SDK (nRF Connect for Desktop) and its toolchain set up locally on the computers. Doing it this way allows you to build and flash the application on your local device where you clone your github-repo to. If you're doing it this way, then I would add the build folder to the .gitignore file.

There are multiple ways to create and work with your application, but I personally prefer to write my projects in a "my_projects"-folder on the same level as the toolchain. The application you're writing can still be created from existing samples through the nRF Connect extension and developed in VSCode.

Where you may organize your applications according to what toolchain you're working with (1.9.1 or 1.8.0 etc), where "my_git_sample" is the sample app for demonstration purposes. This folder contains a "hello_world"-sample with an additional .gitignore.

From here, you must have installed git on your Windows machine through a tutorial such as this step by step guide from pheonixnap.com (or other similar tutorials. This was just the first search I found).

[quote user=""]So now I want to take what I have created and get it into a new project Git repo that I will manage on Github[/quote]

The first step is to create a empty git-repo on your github.

After this, you will be prompted from a quick-setup to either add something to the repo through bash/cmd/desktop. Choose the method suited for you based on if its a new or existing project.

If you choose method "existing repository", open up a git bash window to that application folder (in my case the path is C:\Nordic\SDKs\ncs\my_projects\1.9.1\my_git_sample), and write the lines from the quick setup. You might have to set up a git token or two-factor on your git to allow you to set up remotes from bash, but that should be relatively quick to set up if you have not done so already.

[quote user=""]how to define my .gitignore[/quote]

The .gitignore can be defined by simply creating a new file in the application folder (either from explorer or adding a new file in vscode), and naming it ".gitignore". Here you write what folders you want to ignore, such as "build/", meaning all files in the folder named build.

I hope this gets you started, and let me know if you have any additional questions!

Kind regards,
Andreas