Excel addition bug?

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Excel addition bug?

Shroud: May 11, 2009

I've been noticing something strange going on with Excel 2007. Sometimes, when I have a column of numbers set to the Accounting format, when I sum them in a new cell I get a wrong amount.

For example, I'll see something like this:

code:

Col. A     Col. B

Cell 1      1.01
Cell 2      2.02
Cell 3      3.03
Cell 4      4.04
Cell 5      5.05

Total      15.17 (formula - sum(B3:B7))

I've googled and seen Microsoft say it's a result of floating point stuff (https://support.microsoft.com/en-us/kb/214118), but don't calculators do the same thing while getting the correct result?

Is there a fix, other than to use the ROUND functions or the display precision option?

# ¿ Jul 29, 2015 16:22

Adbot: ADBOT LOVES YOU

# ¿ Apr 28, 2024 15:31

Shroud: May 11, 2009

Alereon posted:

Are you actually seeing this with the example you gave? Because floating point precision issues should only be affecting really small numbers, or causing a very small difference from what you expect, nothing like cents out of ten dollars. And yes, calculators have the exact same issue, there are certain math operations you can do to reveal prevision limitations.

Not with those specific numbers. The ones I deal with are in the high hundreds and low thousands, but I didn't want to put the exact ones because they're payroll numbers.

# ¿ Jul 29, 2015 17:45

Shroud: May 11, 2009

sarehu posted:

If they're using IEEE 754, the numbers you input, if they have fractional parts, have an inexact representation with about 15 or 16 decimal digits of precision. If they're in the high hundreds or low thousands, you're looking at about 42 binary digits right of the decimal place, 36 binary digits right of the cents place.

The maximum error caused by rounding to the nearest binary representation on an initial value you've input would be about 2^-43, if it's between 1024 and 2048.

How much rounding error you could get by computations combining these values depends on what computations are done, and also on how Excel carries them out. (There's more than one way to add a list of numbers and they have different characteristics about how much error could possibly accumulate, in terms of maximum error and expected value.)

This is not something you'd see affecting results with 5 numbers or 1000 numbers.

I think it's quite likely that you're truncating a value somewhere it should be rounded, or something like that.

If you want to eliminate errors when adding dollars-and-cents values, you can operate in terms of cents. Multiply by 100, round to the nearest integer, add the numbers, divide by 100. There is no loss of precision when working with integers (unless they're more than 2^53).

Would it be affected by the fact that I use the ROUNDUP function in previous cells that are then summed, or that some cells are the result of a copy/paste?

The errors are happening in a column whose sum is less than $40,000.

# ¿ Jul 29, 2015 18:14

Shroud: May 11, 2009

Found the problem, but it makes no sense.

This column:

633.50 is the correct amount (manually adding the column on a calculator), 633.48 is the amount from using SUM.

The numbers in this column (aside from the sum) were copied/pasted from another worksheet in the same spreadsheet.

When I delete the 44.55 (2nd row) and enter it manually, the 633.48 changes to 633.49.

When I delete the first 159.48 (8th row) and enter it manually, the 633.49 changes to 633.50, and now everything adds up correctly.

I guess that means I enter everything manually from now on, but how does a copy/paste actually change a value?

# ¿ Jul 29, 2015 19:16

Shroud: May 11, 2009

sarehu posted:

It's displaying rounded values. When you type in "44.55" manually, you're replacing a value that's quite far from 44.55 with the closest possible representation.

I had no idea. Thanks for the info!

# ¿ Jul 29, 2015 21:10

Adbot: ADBOT LOVES YOU

# ¿ Apr 28, 2024 15:31

Shroud: May 11, 2009

Yes - I have the cells format set to "accounting".

# ¿ Aug 1, 2015 13:50

The Something Awful Forums > Discussion > Serious Hardware/Software Crap > Excel addition bug?