Problem with ADC precision

[Louis Hatte]

Hello everyone,

We are using the nrf52840 Dev Kit (PCA10056 1.1.0 2019.19 683465548)

SDK Version: 15.3

OS: Fedora

When we are generating a 1V voltage our adc returns values between [260, 280]. We use this function to convert it into µV:

#define ADC_REF_VOLTAGE_IN_MILLIVOLTS     600
#define ADC_RES_10BIT                     1024
#define ADC_PRE_SCALING_COMPENSATION      6

int32_t adc_result_to_uv(int16_t adc_value) {
    int32_t   adc_uv = ((((adc_value) * 600000) / 1024) * 6);
    return adc_uv;
}

We get values around 940 000 µV instead of 1 000 000µV.

We tried to change the resolution into 14 BIT like above:

#define ADC_REF_VOLTAGE_IN_MILLIVOLTS     600
#define ADC_RES_14BIT                     16384
#define ADC_PRE_SCALING_COMPENSATION      6

int32_t adc_result_to_uv(int16_t adc_value) {
    int32_t   adc_uv = ((((adc_value) * 600000) / 16384) * 6);
    return (int32_t) adc_uv;
}

But we get strange results:

- adc_value between [4200, 4400]

- adc_uv around -631554 µV

We don't know what we are doing wrong and how to get a better precision...

Any ideas?

[p_t]

(600000/16384) * 6  = 219.7265625

try,

adc_uv = adc_value * 219.7265625;

or make adc_uv unsigned32

shouldnt get negative values after that

[awneil]

p_t said:

(600000/16384) * 6  = 219.7265625

Louis Hatte: remember that 'C' will do this in integer arithmetic

So it will first calculate 600000/16384 = 36 and then multiply by 6 to give 192 - which is a significant error!

With integer maths, always do the multiplication first;

ie, 600000 * 6 = 3600000 and then dividing by 16384 gives 219 - which is much closer.

Of course, you have to be careful of overflow ...

You might want to note this thread about using the FPU:

https://devzone.nordicsemi.com/f/nordic-q-a/49062/fpu-power-consumption

[p_t]

In here multiplication was the reason which overflowed the int32, thats why i suggested doing division first . but thats true that if result is int type then it will lose its fraction part.  Thanks for the correction.

[awneil]

You should be able to avoid the int32 overflow by casting to 64-bit ...

Or, as it's all compile-time constants, just work it out before hand.