Using high precision measurement in Nordic Distance Toolbox

[Dymek117]

Hey,

I watched webinar with presentation of NDT and a Lidar as a reference data and if looks quite impressive for high precision calc method.
We are using 7002 DK and we would like to really see how it works. Currently we did some testing for IFFT method from nrf-sdk samples, but as it was said in webinar these result are not trustworthy for wide measurement ranges.

However, i haven't seen example usage of nrf_dm_high_precision_calc() function in sdk, or github.
I know that there is a need of calling nrf_dm_populate_report but i still fail using this with sdk 2.5.0.
Could you maybe share a piece of code, where you present how to properly use these tools for high precision distance measurement?
It could be for other target or SDK version, i just need a start point.It could be the code, that you use for webinar measurement with roll camera :)

We are eager to test it and try to auto-map 5x5 nrf device matrix in large space with coodrdinates, but for this purpose we need as precise result as possible.
Of course, we could share our results of this experiments.

Thanks!

[Dymek117]

Maybe useful input is when currently we use DM sample, our measurement looks like this:
Distance estimates: mcpd: high_precision=nan ifft=0.29 phase_slope=2.60 rssi_openspace=2.51 best=0.29

It's nrf 2.5.0 sdk version
These NaNs are present in high_precision in 95% of measurements. 5% it seems to measure probable value.

We use DM sample from nrf-sdk, and:
NRF5340-7002DK board is a initiator
NRF52840DK is a Reflector.

I deliver more precise data, as im not really sure what is the cause. Sample code wasn't modified except this device role parameter.
Please share with me if you have any idea.
Thanks :)

[Elfving]

Hello,

However, i haven't seen example usage of nrf_dm_high_precision_calc() function in sdk, or github.

Could you maybe share a piece of code, where you present how to properly use these tools for high precision distance measurement?

I think it should be added to the rest of the estimates as long as you enable CONFIG_DM_HIGH_PRECISION_CALC, which is enabled in the sample by default. You can see how nrf_dm_high_precision_calc() is used in nrf/subsys/dm/dm.c.

Dymek117 said:

These NaNs are present in high_precision in 95% of measurements. 5% it seems to measure probable value.

Hmm I see. nrf_dm_high_precision_calc can sometimes return NaN when it is not able to do an estimation with the measurements provided. The algorithm uses these tone_sinr_indicators to determine the quality of collected IQ measurements. If multiple of the tones are of quality worse than medium then the nrf_dm_high_precision_calc will likely return NaN. 

You could look into the quality of these on your side, but if your main goal is to try it out then lets not start out with that. I've only personally tried this out on my desk, and I've never seen the high precision mode returning NaN before myself. Could you expand a bit on how you are testing it and what the environment looks like?

I've heard of another customer running into this issue in a building with a lot of concrete structures, which might've made the reflections strong, and little line of sight.

Regards,

Elfving

[Dymek117]

Hey, thank you for answer. 

You explained my fear that I’m not actually using precise calc. But it seems that sample use this by default. 

All these NaNs are present during test in very stable conditions - same room, no walls between, two different desks or devices lying on the floor. But the building is made in concrete and steel. 
On Monday i will proceed test in open space and also try to change the hardware

Anyway for now we know, that there is no problems in software. 

[Elfving]

Dymek117 said:

But the building is made in concrete and steel. 

Im not saying that concrete and steel should imply that you get bad results, though that a lot of concrete structures (and maybe no line of sight) was the reasoning behind why a previous customer had similar results. 

Dymek117 said:

On Monday i will proceed test in open space and also try to change the hardware

Great, give that a go. Im surprised that a very controlled environment like what you describing should get this bad results. I've never experienced that myself.

Any reason why you are not simply running the default sample, just for the test?

Regards,

Elfving

[Dymek117]

Actually I am, but should I define in code Reflector and Initiator roles for each devices?
Or rather both of them can be Initiators?

[Elfving]

Dymek117 said:

Actually I am, but should I define in code Reflector and Initiator roles for each devices?

They have different roles, but I think what role is selected automatically. Just flash the same code to both.

Regards,

[Dymek117]

Hey, I did some measurements today. I used nrf7002 and nrf52840 devkits in different combinations. Unfortunatelly results are pretty much the same not matter the hardware. I used same sample code build on linux with sdk 2.5.0. Here are the observations:

Up to 2m distance high precision seems to work fine (only sometimes NaN happens). 
Increasing distance, increases also NaN ratio very fast. With 4m its almost 100%. I got only few right measurements during the test.
As you can see below, IFTT and phase slope are quite ok. Especially the ifft seems to give very probable results.
RSSI_Openspace is completely off earth. It gives few hundreds meters when im 10m away from second device. But maybe this is expected as this is rssi measurement, im not sure :)

Measurement result:
Addr: D1:3F:48:6B:7E:92 (random)
Quality: ok
Distance estimates: mcpd: high_precision=nan ifft=7.91 phase_slope=8.77 rssi_openspace=19.95 best=7.91

Test was performed in open space, outside the building to minimize interferations.
Do you have any insights? Are you sure that i should set anything in sample code? This one wasn't modified, just build from sdk directory.

Thanks!

[Dymek117]

Confirmed also on 2.4.0 and 2.6.1 sdk fresh install on windows.
For sure hardware and sdk installation are not a case here.

[Elfving]

Dymek117 said:

Up to 2m distance high precision seems to work fine (only sometimes NaN happens). 
Increasing distance, increases also NaN ratio very fast. With 4m its almost 100%. I got only few right measurements during the test.

This does sound worse than what I've gotten on my side, though to be fair I haven't tried anything longer than 4m. Though I've never seen a single Nan myself.

According to the R&D team responsible for the distance toolbox, it has been tested to about 18 meters, though at 18 meters we have seen that fewer rangings are successful compared to close range. We have also seen that the high precision algorithm is less robust than other algorithms when some of the IQ measurements are missing or corrupted. If you want to get more results from the high accuracy algorithm you could try overwriting the sinr indicators with NRF_DM_SINR_INDICATOR_HIGH in the  nrf_dm_report_t before you run nrf_dm_high_precision_calc. Though I wouldn't say that this solution is recommended. Its returning Nan because multiple tones are of a quality worse than medium, so it wouldn't really be a high precision result.

I would rather recommend this fallback approach: if the high precision does not give a valid result you can still call the regular calculate.

Having a look at the levels of these tone_sinr_indicators to figure out what the quality of the measurements are might either way be interesting to you.

Dymek117 said:

RSSI_Openspace is completely off earth. It gives few hundreds meters when im 10m away from second device. But maybe this is expected as this is rssi measurement, im not sure :)

Maybe that is a bit much.., but RSSI is expected to perform awfully, especially so over longer distances. Have a look at this webinar in case you haven't yet. Though I guess it can be somewhat useful at very short distances. I've gotten the impression that if you want to get something that performs well you'd not just try to add a Kalman filter on top of these, but for instance use the different ways of measuring the distance in the situations in which they are trustworthy. Eg. not trusting RSSI over distances larger than a meter or two, or using the phase slope as a maximum.

Maybe you'll be able to create an algorithm based on the regular results that you prefer. And there is also always the option of creating everything regarding the algorithm on your own using the IQ samples.

Regards,

Elfving

[Dymek117]

Hey Elfving,

As we need to have NRF DM Toolbox measured i am going to perform detailed testing anyway. What I'm gonna do is exactly what you described - use high precision if available and IFFT in other case.
I'm going to do some openspace test and some other with walls in between. Hopefully it will be good enough. 
But anyway i'm wondering why my measurements are so bad quality, which is the reason of these NaNs. Is it regular situations on devkits? I tried to move away from any other signal sources. Maybe there is something wrong in my setup, shouldn't I use some external antenna, etc?
It really does not look good - high precision works only on desk setup, moving anywhere blocks any results.

[Dymek117]

Hey, I spent 2 days of rebuilding, changing hardware, conditions, SDK versions and even build platforms. Not matter what i did high preciosion measurements stopped working at about 2 meters. I'm afraid you've got some issue here :(

Below i present my measurements based on IFFT method. Not quite good unfortunatelly, but like you have said, probably some Kalman filtering would do the job here. But being honest i really thought such filtering or calculation will be done inside a driver 

Open space measurements

Measurements done in predictable conditions - open office space, antennas placed 1m above floor, no obstacles in between.

Distance	Measured value
1m	1.2m
2m	2.1m
3m	3.2m
4m	4.2m
5m	5.2m
6m	6.1m
7m	7.2m
8m	9m
9m	14m
10m	9.5m
11m	14.7m
12m	13.6m

Walls measurements

Real conditions test. DUTs placed in different rooms - up to 3 concrete walls in between, glass or wooden doors.

Distance	Measured value
2m	2.5m
4m	4.4m
9m	9.6m
12m	13.5m
14m	14.1m

These results are after "visual selection", as there is even 1-2m variations in each measurement. I would suggest to take a look at this sample guys and make some adjustments here.