9160/NCS: GPS accuracy value, how to interpret, and inconsistency

Hi,

I'm experimenting the GPS on nrf9160 by using the ncs/samples/nrf9160/gps project. I set the GPS mode to be continuous mode. The 9160DK board is placed in a relative open area.

The initial result is encouraging. I let it run for 60s after getting the first fix. I can see the accuracy reported by GPS goes down quickly. Usually after 10~15s, the accuracy is around or below 3m and stay consistent thereafter. I understand the accuracy is a range with probability. So there's chance the true location is outside the radius as indicated by the accuracy. But if I get consistent accuracy reading that stays below 3m that probably indicates higher accuracy and/or higher confidence. Then I plot the coordinates (60 seconds) and calculated the distance from them to the average of this group of positions. It turns out that majority (>90%) of them fall in the 3m radius to the average. So that's very good result.

Now here's the inconsistent part that I don;t understand. If I stop the gps (through the api: nrf_modem_gnss_stop()) and then start over again, I'll get another set of 60s GPS data. Everything looks just like the first set: reported accuracy drops down to 3m or less quickly and the coordinates mostly fall into the radius. Except the average of the coordinates (hence the whole cluster of positions) are at a point quite some distance from the first test. It could be 6~8 meters away, or even more sometime.

I attached the data and the plot. The blue dots are from test1. Each dot is from the 1 second fix data. The size of the dot is proportional to the accuracy value of that fix. So the largest dot is the first fix. Then it get smaller and smaller. And eventually it all stays in a relatively small area. The dark blue dot (1A) is the average of all blue dots. Same situation for the yellow dots which is from the second test. The dark yellow dot(2A) is the average of all yellow dots. (the X is latitude, Y is longitude)

So my questions is, if you look at either blue or yellow individually, it all looks pretty good. If you keep the GPS in continuous mode you'll get more and more dost around the cluster and thinking that's roughly where the true position would be. But if you run a separate test, it will generate a different cluster and as we can see here from 1A to 2A it's about 6.2 meter. So how should I understand this? Why they are different? Did I miss something here?

(I uses formula here to calculate distance: https://www.movable-type.co.uk/scripts/latlong.html )

A further questions is, if I need to track position to the accuracy as data sheet specified (ie 3meter), it seems I can only keep the GPS running in continuous mode? But even so I'm not sure if the position I get is accurate or not.

Thanks!

 9160 - GPS-accuracy-static-continuous-psm0.csv

Parents
  • Hello, 

    The 9160DK board is placed in a relative open area.

    Can you please describe the area where you are testing? Please note that buildings and other elements around may cause reflections, which can impact the results.

    Everything looks just like the first set: reported accuracy drops down to 3m or less quickly and the coordinates mostly fall into the radius. Except the average of the coordinates (hence the whole cluster of positions) are at a point quite some distance from the first test. It could be 6~8 meters away, or even more sometime.

    The numbers sound correct as described in the nRF9160 Product Specification

    What version of the nRF9160 DK are you running? Are you running an external antenna? What modem version and nRF Connect SDK version are you running?

    Kind regards,
    Øyvind

  • That's in the middle of backyard. It's generally open, with trees and houses at least 30~40 feet away in all directions. But there're some tall trees on one side. The device definitely won't be able to see the horizon. Not sure how important that is. It's open straight up.  This is probably not ideal for GPS testing, but quite comparable to actual usage scenario , which will mostly be dense urban/suburban area.

    Hardware setups:

    no external antenna

    9160DK: PCA10090 1.0.0 / 2021.7

    modem firmware:  v1.2.2-9-g854050c28061

    NCS: 1.7.1

    Now I realize there's new modem firmware 1.3. I probably try it to see if that makes any difference.

  • Hello, and thanks for providing these details. I will need to forward to our nRF91 GPS experts. 

    Kind regards,
    Øyvind

  • Hello, my apologies for the late reply. Our R&D team have replied with the following:

    Q: A further questions is, if I need to track position to the accuracy as data sheet specified (ie 3meter), it seems I can only keep the GPS running in continuous mode?

    A: Yes, in order to get to 3 m accuracy, the device should be running in continuous mode.

    • Accuracy is reported with 2D 1-sigma (= with 68% confidence) as said in doc
    • Averaging data partly is dangerous. In plot all positions has been treated equally when first are reported with low accuracy value.
    • in 60 second case first fix is cold/hot fix and accuracy improves over time. We don't get 3m accuracy with the first fix even it is started in continuous mode.

    This picture tries to illustrate the case:

    2D errors is distance between positions in Test A and Test B. So after 55 seconds distance between position reported in Test A and Test B is less than reported accuracy.. If test would have continue little bit further we may have discovered the true location.

    Please note that the DevZone team is currently entering Christmas holidays with lower staff count. Delayed answer / longer response time must be expected

  • Thanks! I'll need to collect more data after the holiday...

  • Yes, please do. It would be good to have more data to compare with what the R&D team has. 

  • Ok, I did more tests and here's the results.

    First chart is from the same setup at my back yard with everything exactly the same. Device is not moving. I did 3 run, each lasts 5 minutes. Then plot the reported accuracy and the 2D-error between the tests. As you can see, the reported accuracy drops to below 2 meter pretty quickly. But the 2D-error bounces up and down in a wide range through out the 5 minutes. It does not converge as you R&D team implied earlier. But it doesn't run off either. So I conclude that in my specific environment the real accuracy is more like 7~8 meter instead of 3 meter. I consider this 'typical' scenario for my project's use cases.

    The second chart is from a test done on a roof top. It's the top floor of a public parking garage without cover. There's no taller building nearby. So it's really wide open. The result is much better. Not only the reported accuracy is lower (about 1.5 meter), the 2D-error is generally below the reported accuracy most of the time. Even the part that's elevated is generally under 3 meter. So I'm convinced the device can achieve the specification under ideal condition.

    Back to my project's perspective, the remaining question is, is there any way to improve GPS performance under an non-ideal environment? Particularly it the device is moving, ie walking or running but not driving. Does a better antenna help? (this is done with 9160DK with the onboard antenna)

    Thanks!

Reply
  • Ok, I did more tests and here's the results.

    First chart is from the same setup at my back yard with everything exactly the same. Device is not moving. I did 3 run, each lasts 5 minutes. Then plot the reported accuracy and the 2D-error between the tests. As you can see, the reported accuracy drops to below 2 meter pretty quickly. But the 2D-error bounces up and down in a wide range through out the 5 minutes. It does not converge as you R&D team implied earlier. But it doesn't run off either. So I conclude that in my specific environment the real accuracy is more like 7~8 meter instead of 3 meter. I consider this 'typical' scenario for my project's use cases.

    The second chart is from a test done on a roof top. It's the top floor of a public parking garage without cover. There's no taller building nearby. So it's really wide open. The result is much better. Not only the reported accuracy is lower (about 1.5 meter), the 2D-error is generally below the reported accuracy most of the time. Even the part that's elevated is generally under 3 meter. So I'm convinced the device can achieve the specification under ideal condition.

    Back to my project's perspective, the remaining question is, is there any way to improve GPS performance under an non-ideal environment? Particularly it the device is moving, ie walking or running but not driving. Does a better antenna help? (this is done with 9160DK with the onboard antenna)

    Thanks!

Children
  • Thanks for providing these details. I will need to forward this to the team and discuss possible solutions. 
    If you have not heard back from me by Monday, please ping me here Slight smile

  • Hi,

    Any more update? I'll be nice if there's more info or suggestion.

    On the other hand I'm continuing more tests with the cellular radio enabled. I basically took the agps sample project and put the GPS in continuous mode, and run GPS for 1 minute and then send the data over LTE to the NRF cloud portal.

    I did not intentionally stop the GPS but I think the cellular radio will block GPS when it needs to transfer data. So there's a few seconds gap for GPS data. It's probably equivalent to a hot start.

    What I can see is it seems the accuracy is a lot worse than when there's only GPS turned on. Is this something expected? Does the cellular radio affect the GPS operation in anyway (other than a small gap in time)? In the agps sample code, I see the PSM mode is requested. I did not enable GPS's PSM mode. Other conditions for testing are the same as before.

    Any thoughts?

    Thanks!

  • Hello, 

    Sorry, the response from the team was not sent for some reason. 


    In customer's use case when device is moving and GNSS is running on continuous tracking mode, changing dynamic mode could improve accuracy.

    See here: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrfxlib/nrf_modem/doc/gnss_interface.html#changing-the-dynamics-mode

    Dynamic mode can be one of these: https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrfxlib/nrf_modem/doc/api.html#dynamics-modes

    Regarding antenna, 9160DK onboard antenna is rather good, but using good quality active antenna could improve accuracy. Also it often enables placing the antenna on better location in challenging environment, example higher.

    bluebeam said:
    I did not intentionally stop the GPS but I think the cellular radio will block GPS when it needs to transfer data. So there's a few seconds gap for GPS data. It's probably equivalent to a hot start.

    The GPS can not run when LTE radio is active. As described in the GPS receiver chapter of the nR9160 Product Specification:

    Operation is time multiplexed with LTE modem, and it is possible to use the GPS and QZSS receiver to obtain position either while the LTE is in RRC Idle mode or power saving mode (PSM), or when the LTE modem is completely deactivated.

    bluebeam said:
    What I can see is it seems the accuracy is a lot worse than when there's only GPS turned o

    Are you comparing with AGPS?

  • Hello, 

    bluebeam said:

    On the other hand I'm continuing more tests with the cellular radio enabled. I basically took the agps sample project and put the GPS in continuous mode, and run GPS for 1 minute and then send the data over LTE to the NRF cloud portal.

    I did not intentionally stop the GPS but I think the cellular radio will block GPS when it needs to transfer data. So there's a few seconds gap for GPS data. It's probably equivalent to a hot start.

    What I can see is it seems the accuracy is a lot worse than when there's only GPS turned on. Is this something expected? Does the cellular radio affect the GPS operation in anyway (other than a small gap in time)? In the agps sample code, I see the PSM mode is requested. I did not enable GPS's PSM mode. Other conditions for testing are the same as before.

    Our modem responds with the following:

    In that case LTE will interfere with GNSS operation. If PSM is not enabled, the modem needs to periodically listen to pagings from the network. Unless eDRX is used, the paging cycle is usually 1.28 seconds.

    If both LTE and GNSS are enabled at the same time, it's recommended to use eDRX or/and PSM. eDRX can be used to increase the paging cycle much longer. And when in PSM, modem doesn't need to listen to pagings at all. It should be noted, though, that the device can only be reached by the network when it's listening to pagings. So, the suitable configuration depends on the use case (and what the network supports).

  • Thanks, enabling PSM and eDRX does help. Now the accuracy is as good as before when I ran GPS only.

    Now I'm looking at these PSM and eDRX parameters. I wonder if there's any guidance as to how to properly set them and what they mean. Or should I always let the cell network decide that? I print out some logs and looking at the PSM active time which is set at 180s which is way too long for my case. I need to send data every 60s. So that seem indicate that the device will never be in PSM mode. I attempt to set the value to something smaller like 10s or 20s. But I still get the 180s. Does that mean the network won;t allow me to change it? Is there example to for how to set PSM and eDRX parameters?

    part of my logs when start to connect to LTE network:

    ===============

    *** Booting Zephyr OS build v2.6.99-ncs1-1  ***
    I: APP thread running
    I: modem init
    D: lte_lc_init
    I: AT+CGMR: 0, mfw_nrf9160_1.3.1
    , 0
    %XMODEMSLEEP: 4
    D: lte_lc_psm_param_set
    D: lte_lc_psm_req
    D: lte_lc_edrx_req
    D: lte_lc_connect
    %XMODEMSLEEP: 4
    D: LTE_LC_EVT_MODEM_SLEEP_ENTER
    %XMODEMSLEEP: 4,0
    D: LTE_LC_EVT_MODEM_SLEEP_EXIT
    +CEREG: 2,"910F","066E4D0F",7
    D: LTE_LC_EVT_NW_REG_STATUS
    D: LTE_LC_EVT_CELL_UPDATE: id=107891983 tac=37135
    D: LTE_LC_EVT_LTE_MODE_UPDATE
    +CSCON: 1
    D: LTE_LC_EVT_RRC_UPDATE
    +CEREG: 1,"910F","066E4D0F",7,,,"00100011","00011111"
    I: modem init done
    I: cloud init
    D: LTE_LC_EVT_NW_REG_STATUS
    D: LTE_LC_EVT_PSM_UPDATE: tau=18600 act_time=180(s)
    +CEDRXP: 4,"1001","1001","0001"
    D: LTE_LC_EVT_EDRX_UPDATE: mode=7 edrx=163.839996(s) ptw=2.560000(s)

    =================================

Related