9160/NCS: GPS accuracy value, how to interpret, and inconsistency

Hi,

I'm experimenting the GPS on nrf9160 by using the ncs/samples/nrf9160/gps project. I set the GPS mode to be continuous mode. The 9160DK board is placed in a relative open area.

The initial result is encouraging. I let it run for 60s after getting the first fix. I can see the accuracy reported by GPS goes down quickly. Usually after 10~15s, the accuracy is around or below 3m and stay consistent thereafter. I understand the accuracy is a range with probability. So there's chance the true location is outside the radius as indicated by the accuracy. But if I get consistent accuracy reading that stays below 3m that probably indicates higher accuracy and/or higher confidence. Then I plot the coordinates (60 seconds) and calculated the distance from them to the average of this group of positions. It turns out that majority (>90%) of them fall in the 3m radius to the average. So that's very good result.

Now here's the inconsistent part that I don;t understand. If I stop the gps (through the api: nrf_modem_gnss_stop()) and then start over again, I'll get another set of 60s GPS data. Everything looks just like the first set: reported accuracy drops down to 3m or less quickly and the coordinates mostly fall into the radius. Except the average of the coordinates (hence the whole cluster of positions) are at a point quite some distance from the first test. It could be 6~8 meters away, or even more sometime.

I attached the data and the plot. The blue dots are from test1. Each dot is from the 1 second fix data. The size of the dot is proportional to the accuracy value of that fix. So the largest dot is the first fix. Then it get smaller and smaller. And eventually it all stays in a relatively small area. The dark blue dot (1A) is the average of all blue dots. Same situation for the yellow dots which is from the second test. The dark yellow dot(2A) is the average of all yellow dots. (the X is latitude, Y is longitude)

So my questions is, if you look at either blue or yellow individually, it all looks pretty good. If you keep the GPS in continuous mode you'll get more and more dost around the cluster and thinking that's roughly where the true position would be. But if you run a separate test, it will generate a different cluster and as we can see here from 1A to 2A it's about 6.2 meter. So how should I understand this? Why they are different? Did I miss something here?

(I uses formula here to calculate distance: https://www.movable-type.co.uk/scripts/latlong.html )

A further questions is, if I need to track position to the accuracy as data sheet specified (ie 3meter), it seems I can only keep the GPS running in continuous mode? But even so I'm not sure if the position I get is accurate or not.

Thanks!

 9160 - GPS-accuracy-static-continuous-psm0.csv

Parents
  • Hello, 

    The 9160DK board is placed in a relative open area.

    Can you please describe the area where you are testing? Please note that buildings and other elements around may cause reflections, which can impact the results.

    Everything looks just like the first set: reported accuracy drops down to 3m or less quickly and the coordinates mostly fall into the radius. Except the average of the coordinates (hence the whole cluster of positions) are at a point quite some distance from the first test. It could be 6~8 meters away, or even more sometime.

    The numbers sound correct as described in the nRF9160 Product Specification

    What version of the nRF9160 DK are you running? Are you running an external antenna? What modem version and nRF Connect SDK version are you running?

    Kind regards,
    Øyvind

  • That's in the middle of backyard. It's generally open, with trees and houses at least 30~40 feet away in all directions. But there're some tall trees on one side. The device definitely won't be able to see the horizon. Not sure how important that is. It's open straight up.  This is probably not ideal for GPS testing, but quite comparable to actual usage scenario , which will mostly be dense urban/suburban area.

    Hardware setups:

    no external antenna

    9160DK: PCA10090 1.0.0 / 2021.7

    modem firmware:  v1.2.2-9-g854050c28061

    NCS: 1.7.1

    Now I realize there's new modem firmware 1.3. I probably try it to see if that makes any difference.

  • Ok the modem firmware version is not v1.2.2-9.... That's the modem library build string. I believe the modem firmware version is 1.2.3 when I use the SLM.

    (I wonder if there's an API to read the modem firmware version? I can't find one)

    Now I upgraded the modem firmware to the latest 1.3.1. I did the test again. I don't see much difference in the results. Maybe that's limited by the environment I have. I plan to find an open area and test this again to see if I can get consistent results regarding the accuracy.

    Can you clarify if this is the right expectation. If I can find an ideal environment (open sky, what else?), then the fixes should converge to the true position and should be within the "accuracy" range?

  • Hello, and thanks for providing these details. I will need to forward to our nRF91 GPS experts. 

    Kind regards,
    Øyvind

Reply Children
  • Hello, my apologies for the late reply. Our R&D team have replied with the following:

    Q: A further questions is, if I need to track position to the accuracy as data sheet specified (ie 3meter), it seems I can only keep the GPS running in continuous mode?

    A: Yes, in order to get to 3 m accuracy, the device should be running in continuous mode.

    • Accuracy is reported with 2D 1-sigma (= with 68% confidence) as said in doc
    • Averaging data partly is dangerous. In plot all positions has been treated equally when first are reported with low accuracy value.
    • in 60 second case first fix is cold/hot fix and accuracy improves over time. We don't get 3m accuracy with the first fix even it is started in continuous mode.

    This picture tries to illustrate the case:

    2D errors is distance between positions in Test A and Test B. So after 55 seconds distance between position reported in Test A and Test B is less than reported accuracy.. If test would have continue little bit further we may have discovered the true location.

    Please note that the DevZone team is currently entering Christmas holidays with lower staff count. Delayed answer / longer response time must be expected

  • Thanks! I'll need to collect more data after the holiday...

  • Yes, please do. It would be good to have more data to compare with what the R&D team has. 

  • Ok, I did more tests and here's the results.

    First chart is from the same setup at my back yard with everything exactly the same. Device is not moving. I did 3 run, each lasts 5 minutes. Then plot the reported accuracy and the 2D-error between the tests. As you can see, the reported accuracy drops to below 2 meter pretty quickly. But the 2D-error bounces up and down in a wide range through out the 5 minutes. It does not converge as you R&D team implied earlier. But it doesn't run off either. So I conclude that in my specific environment the real accuracy is more like 7~8 meter instead of 3 meter. I consider this 'typical' scenario for my project's use cases.

    The second chart is from a test done on a roof top. It's the top floor of a public parking garage without cover. There's no taller building nearby. So it's really wide open. The result is much better. Not only the reported accuracy is lower (about 1.5 meter), the 2D-error is generally below the reported accuracy most of the time. Even the part that's elevated is generally under 3 meter. So I'm convinced the device can achieve the specification under ideal condition.

    Back to my project's perspective, the remaining question is, is there any way to improve GPS performance under an non-ideal environment? Particularly it the device is moving, ie walking or running but not driving. Does a better antenna help? (this is done with 9160DK with the onboard antenna)

    Thanks!

  • Thanks for providing these details. I will need to forward this to the team and discuss possible solutions. 
    If you have not heard back from me by Monday, please ping me here Slight smile

Related