What is the trace protocol used on the trace DIAG interface for LTE-M?

[EAB]

Our project requires a more efficient way to collect live trace Diagnistic trace data from LTE-M RRC and NAS, on a Thingy:91.

However, using the "Cellular Monitor" app, is not an option with it's poor and cumbersome way to collect trace data. And also its neither open source nor are we able to easily decipher the resulting files without using your conversion tools, making live collection impossible and unstable.

We can of course connect a terminal to the diag COM port (on windows) but we only get a lot of garbage, as we don't know what protocol you are using on that interface. 

We are mainly using python for our testing/development tools and would like to see a python app that can either read the COM port directly, or read the COM port via some pipe to tshark before inputting the stream into python. 

This brings up several questions: 

1. What is the live protocol used on COM/diag port?

2. One used to use GSM-TAP in wireshark to read cellular modem network protocols, but I don't see you using this here?

3. There are a bunch of DB files that seem to be what you use to decipher the modem diag logs, can you tell us how we can use python with that database?

4. How complete is that DB? Does it show detailed LTE NAS and L1 messages?

[Håkon Alseth]

Hi,

 

1. What is the live protocol used on COM/diag port?

2. One used to use GSM-TAP in wireshark to read cellular modem network protocols, but I don't see you using this here?

3. There are a bunch of DB files that seem to be what you use to decipher the modem diag logs, can you tell us how we can use python with that database?

The modem trace data format is not open source, but it can be obtained/converted to pcapng via our tooling.

You can use "nrfutil trace" as a CLI option:

https://docs.nordicsemi.com/bundle/nrfutil/page/nrfutil-trace/nrfutil-trace_4.0.0.html

 

4. How complete is that DB? Does it show detailed LTE NAS and L1 messages?

As listed in the above link, it will be able to give NAS related messaging, but not down to L1.

 

Kind regards,

Håkon

[EAB]

Unfortunately your answer is wrongly assuming I didn't read the manual. 
The trace function is not working correct as I recently stated in a different Q/A.

The trace *.mtrace files created from CM app, are 10-100x times larger than the pcapng produced, either directly or by conversion. In one file there are only 2 lines. 1 from RRC and the other from AT. There is no NAS. 

So you guys need to fix and be more transparent about your traces. AFAICT, Nordic is the ONLY company trying to keep their trace data and function secret! Why? 

I've been working with Qualcomm products for years, and their SW used to be locked down, but there was never a problem getting proper traces and documents about it...

To be blunt, your "nrfutil trace" program simply sucks, and its unreliable and very poorly documented function, prevents any type of automation where live traces need to be collected. 

[Håkon Alseth]

Hi,

 

My apologies that our products does not live up to your expectations.

I will forward your comments internally to our LTE team.

 

Kind regards,

Håkon

[EAB]

Thank you Håkon,
  Looking forward to hear back from your team. 

I understand, from looking at code and documents (you already shared with me), that your DIAG trace output is based on the MIPI STP v2 protocol. So nothing strange there, however, without having the proper decoders for mapping the output to your many trace DB's and version, there is little hope making progress. 

I have no time, interest or capacity for trying to reverse engineer your (nrfutil trace) tool. I'm pretty sure you have your own unified/working tools like everyone else does. 

The "MIPI System Trace Protocol (STP) v2" specs can be obtained from MIPI's website, and the Zephyr project have a decoder header, but not sure if it can be used with your implementation. However, they also mention "Sys-T" which is something different topic, AFAICT.

You can find this here:

• https://docs.nordicsemi.com/bundle/ncs-latest/page/zephyr/services/debugging/mipi_stp_decoder.html
• https://github.com/zephyrproject-rtos/mipi-sys-t
• https://github.com/zephyrproject-rtos/zephyr/blob/main/include/zephyr/debug/mipi_stp_decoder.h
• https://www.mipi.org/specifications/sys-t 
• https://www.mipi.org/mipi-stp-download
• https://github.com/MIPI-Alliance/public-mipi-sys-t
• https://www.mipi.org/knowledge-library/whitepapers/mipi-architecture-overview-for-debug-v1.3
• # ^^ PDF
• https://www.mipi.org/articles/enhancing-debug-and-trace-capabilities-for-system-developers
• https://www.mipi.org/specifications/stp
• https://www.mipi.org/mipi-stp-download
• https://www.mipi.org/download-mipi-architecture-overview-for-debug-white-paper
• https://www.mipi.org/current-specifications

---

And spec overview: 

As you can tell, it's well documented, but poorly implemented in terms publicly available tools for using it. 

Anything Nordic could do to help, would be greatly appreciated.