ccache no longer works for NCS in Windows with newer versions of ccache

RE: ccache no longer works for NCS in Windows with newer versions of ccache

Burt — Thu, 28 Nov 2024 14:46:11 GMT

Thanks, Charlie, at least now I know the reason for no built in ccache in the Windows Toolchain. I'll say that this is one of those cases where a well-intentioned and generally smart policy ends up giving a non-optimal result. Anyway, it's too bad that some remarks could not be included in the nRF Connect SDK documentation (installation section); a helpful hint like, "although not officially supported, some users may wish to install ccache externally, and we recommend version 4.8.2 (rather than the latest version) for maximum compatibility with Zephyr" maybe adding "Use at your own risk."

I'm only making such an issue out of this because:

1. At first, I didn't realize I needed to add my own ccache, but when I did, I always would add the latest version.
2. Strange but I didn't realize for a long time that my ccache wasn't doing anything. (Maybe it was when I first installed ccache.)
3. It was time consuming figuring out the trick to make the latest version work.
4. My bad, I didn't look at what version was in the Linux toolchain. I actually thought you folks had used a customized version.
5. Windows builds are slow enough that any extra help is a good thing.
6. My colleague never takes my advice if it is an addition to what you provide!

The good news is that this ticket has 86 views so some people will know all their options (if they use Windows!)

I will close the ticket now. Thanks again for giving me the official word from the Toolchain developers.

Burt

RE: ccache no longer works for NCS in Windows with newer versions of ccache

Charlie — Thu, 28 Nov 2024 11:52:36 GMT

Hi Burt,

After discussing with our toolchain maintainers, I have the following information to share:

The reason ccache is not included in the NCS toolchain for Windows is that it has never fully supported Windows, including 4.8.2 and latest version(4.10.2 currently).

Probably works, may work or is partially supported. Not part of the test procedure before a new release. Bug fixing and testing largely depend on contributions from the community.

While developers are welcome to manually include and test ccache on Windows, we encourage using the same version included in the Linux toolchain for consistency. The specific software versions included in the toolchain can be found here:

Let me know if you have additional questions or need further clarification.

Best regards,

Charlie

RE: ccache no longer works for NCS in Windows with newer versions of ccache

Burt — Fri, 22 Nov 2024 14:54:15 GMT

Hi Charlie,

I agree with you up to where you say that this relates to zephyr/scripts/west_commands/build.py. Perhaps you are conflating ccache with the CMake cache that is referenced in build.py.

A more interesting question would be why we find for example ncs/toolchains/e9dba88316/usr/bin/ccache in the Linux case but do not find for example a C:\ncs\toolchains\c57af46cb7\usr\bin\ccache.exe in the Windows case. Are not the contents of the various ncs/toolchains put together by NS based on input from zephyr but customized by NS? I see this as a primarily NS issue at this time, as you could include the same version of ccache in the Windows toolchain as the one which works for Linux; granted if you put the latest version of ccache into either Linux or Windows then we'd have to ask Zephyr to fix zephyr/cmake/modules/ccache.cmake. There is an argument for either method, I'd say, but currently your Windows users are at a disadvantage because most probably are not looking so closely to see that ccache is either absent or not doing its job.

Burt

RE: ccache no longer works for NCS in Windows with newer versions of ccache

Charlie — Fri, 22 Nov 2024 14:02:38 GMT

Hi Burt,

Thank you for sharing your knowledge. I found that Windows indeed does not have ccache installed by default.

I’m not entirely sure why Linux-like systems include it while Windows does not. However, I believe the Issues · zephyrproject-rtos/zephyr would be a better place to discuss this topic, as it relates to zephyr/scripts/west_commands/build.py.

Best regards,

Charlie

RE: ccache no longer works for NCS in Windows with newer versions of ccache

Burt — Thu, 21 Nov 2024 15:21:00 GMT

Hi Charlie,

I believe you are referring to the Linux version of NCS 2.8.0, which includes the ccache version you mention. Maybe Mac, too; I forget.

The Windows version of the toolchains do not include ccache. I don't recall where in the documentation for installing NCS on Windows do they explicitly mention ccache, but in general I believe Chocolatey is recommended for bringing in some tools, and I believe Chocolatey would bring in the 4.10.2.

Can you verify that ccache is pulled in when following the documentation instructions to install NCS on Windows? Does it happen automatically when using VS Code and otherwise does not get pulled in? I just always remember to make certain I have it, whether or not I found it in the instructions.

In Windows, if you execute the 2 command sequence:

set VERBOSE=1
west build ...

you can look at some of the C file compilations to see if ccache is being invoked.

If ccache is invoked, you can see if it is caching by running a pristine build, then issuing "ccache -z" to zero out statistics, then run the pristine build again, then run "ccache -s -v" and you should see hundreds of cache hits in the statistics printed out; otherwise, you know it is not working. You shouldn't see any instances of "not cached due to bad parameter" or something to that effect

Thanks, Charlie.

Burt

RE: ccache no longer works for NCS in Windows with newer versions of ccache

Charlie — Thu, 21 Nov 2024 12:52:53 GMT

Hi Burt,

Thank you for reporting your observation. I apologize, as I am not very familiar with ccache, but I am trying to better understand your issue.

I see that NCS 2.8.0 uses ccache version 4.8.2 by default. Are you using an environment set up with the latest ccache version 4.10.2?

If that’s the case, I encourage you to use the default toolchain whenever possible. We will ensure that the build process is compatible with the newer toolchain and ccache 4.10.2 in future updates.

Best regards,

Charlie