Why is the result of my float-number operation incorrect?

[duxinglang]

I developed a product for customers using ncs2.3.0 on nrf9160, which requires the use of floating-point numbers to record temperature sensor data and display it on the screen through images. When displaying it, I need to enlarge the temperature of the floating-point number by 10 times and display it through digital images. However, during testing, I found that in some cases, the floating-point number calculation results are different from what was expected, such as 36.7 * 10.0, but the result shows 366. What is the reason for this?

    

[AHaug]

Hi,

The short answer is precision and rounding errors when working with float numbers, multiplication of float numbers and outputting them with 1 decimal precision.

You're trying to multiply a float number by 10 and cast it into an integer. Rounding errors occur when working with float numbers, arithmetics and changing types. Your value 3.7 is probably closer to for instance 36.698, which you print with 1 decimal precision, which then becomes 36.7. When you multiply that by 10, cast to an int and print with 1 decimal precision you get 366.

There are multiple resources online teaching you how floating point numbers work in programming, and I recommend that you have a look at those.

Kidn regards,
Andreas