Decimal calculation error

[arekowczarek]

In one of my macros I have three decimals:

%PROBBEFORE[3]%=33,4

%PROBAFTER[3]%=23,8

%DIFFERENCE[3]%=%PROBBEFORE[3]%-%PROBAFTER[3]%

 

The problem occured only for the values above. I run it about a hundred times, different values each time.

 

The %DIFFERENCE[3]% should equal "-9,6".

Instead of this I am getiing "-9,60000000000001".

It's not just a matter of rounding the result, because 33,4-23,8 ≠ -9,60000000000001

 

Now, I tried to create a new macro with the decimals preset to the values above - and the error didn't occur.

So, somehow it only appears in the original macro file. My first thought was that I didn't round the two decimals before the subtraction. But I debugged it and watched the values of decimals. And they equal 33,4 and 23,8 just before the subtraction. and then the subtraction result becomes -9,60000000000001.

 

 

If anyone wanna give it a quick check I attached the macro file. The macro doesn't interfere with your mouse or keybord (no text type). All you will get are two text boxes. The last three lines of macro is where it happens.

 

Unless I've made some dumb error myself and somebody can point it out, I'll report it as a bug to ISS.

MOVE PROBABILITY.mex

[paul]

This rings so many alarm bells! I think you'll find that floating point operations (which I assume the Decimal variable is prone to) often result in these types of inexact results - I can remember similar problems in Microsoft Access and VB. If I'm right, then there's nothing Insight ca do for you.

 

For more information, see:

Floating Point 1

Floating Point 2

[arekowczarek]

This rings so many alarm bells! I think you'll find that floating point operations (which I assume the Decimal variable is prone to) often result in these types of inexact results - I can remember similar problems in Microsoft Access and VB. If I'm right, then there's nothing Insight ca do for you.

 

For more information, see:

Floating Point 1

Floating Point 2

 

I read about problems with floating point operations, thanks for the links. I haven't actually understood, yet, why the errors occur (I will get down to it some other time, I know the answer's there, I'm just a lil too busy now and it w'd take me some time to comprehend all that info).

 

Now, having stated the above, this can be a stupid question, but:

Shouldn't the error reoccur for the same values in another macro? As mentioned, it doesn't.

 

 

Thanks for your input Paul

[Cory]

33.4, 23.8, 0.0 then 33.4, 23.8, -9.6 was my result.

 

I'm not sure what you're trying to do and I don't have time to research this but I have had problems with floating point calculations before. I think if you search my posts you will find at least one. The big one I remember was with a date which are actually decimals. I remember two dates which were theoretically equal did not pass an IF condition. I don't think it's a bug so much as a limitation of computers.

[arekowczarek]

33.4, 23.8, 0.0 then 33.4, 23.8, -9.6 was my result.

Now that's hell of a surprise!

I am getting:

33.4, 23.8, 0.0 then 33.4, 23.8, -9,60000000000001

 

If the problem is connected with the floating point, shouldn't we both receive EXACT results running this macro? How can the calculation result be machine dependent??

Unless of course I am missing here the big picture and floating point calculations are machine dependent. If anyone can confirm they are I'll get down to studying the floating point before bothering you guys anymore.

 

I'm not sure what you're trying to do and I don't have time to research

This is just a part of a bigger macro counting the probability of showing desired card combinations. I.E. What is the chance that next two cards taken from the deck are gonna be a "King" and a "9". That sorta stuff.

 

Thanks Cory

[Cory]

http://support.microsoft.com/kb/42980

 

I don't know why it would be different for sure but it might be hardware and OS. I have a Windows 7 64 bit machine and that might have an effect.

 

Floating point precision is a huge theoretical problem in computers and it's not easily understood but here's a good example. Two divided by three results in .6 (repeating) which is slightly different from .666666666666666666666666666666666666666666666666667. Right? You can not ever store an exact 2/3 value in a computer because all computer memory has a denominator of 2 whereas in this example one of the denominators is 3. So there's a huge area of computer science dedicated to this and astrophysicists and such actually write their calculations in such a way to recognize when these differences will potentially cause them problems and deal with it. Stats is a perfect example!

 

Now I seem to remember that when programming that floating point calculator that's 'built in' is fast and handy but there was also often a precision calculator available as well. The difference being that the floating point calculator was much faster. I suggest you either look for such a precision calculator or do like the scientist do and accept that in some cases you're going to get these errors.

 

Just please realize this is not a problem with MEP as much as it is a problem with computers in general. ME is probable using the single data type and could use double instead and have a better tolerance in logic comparisons but really none of these fundamentally solve the problem.

 

 

 

 

[arekowczarek]

I don't know why it would be different for sure but it might be hardware and OS. I have a Windows 7 64 bit machine and that might have an effect.

I have Win XP 32 bit, so this might be it.

 

I understood the problem is not connected to ME itself. I am accepting the fact that sometimes, for some values I might get the slight error and thanks to your throughout explanation I do know what it's caused by.

 

I'm probably being a little to inquisitive now and should probably let go, but:

Why the calculation performed on same values in two different macros would give different results?

The macro I attached always returns the -9,60000000000001 (on my machine). If I preset the values of the two decimals in a brand new macro the result I get is "-9,6".

 

Normally I would just round the output to 1 decimal place. I only need 1 decimal place accuracy anyway. But it gets tricky here.

-9,60000000000001 rounded to 1 decimal place returns -9,5 instead of -9,6. I know because the number is negative ME will round it to the larger value but I would expect the choice should be made between -9,6 and -9,7 (because -9,60000000000001 is somwhere between the two numbers). Odd.

I did some testing and found that ME would even round -9,6 to -9,5 ! In this case rounding negative numbers is just a bad bad idea

[paul]

MEP produces -9.5 when asked to round -9.60000000000001 to a single decimal point. This answer is simply wrong - the correct response is -9.6.

You should report this as a bug.

[arekowczarek]

MEP produces -9.5 when asked to round -9.60000000000001 to a single decimal point. This answer is simply wrong - the correct response is -9.6.

You should report this as a bug.

I reported it, but haven't been assigned a tracking number yet.

 

OT: Is the tracking number only for using as reference when contacting ISS or I can enter it somewhere on their website to see what progress has been made on a particular issue?

[Cory]

You can't use their tracking number for any other purpose that your correspondence with them. A long time ago I created a web page to track all of mine but I don't keep it up anymore.

 

I do seem to remember an issue where negatives were rounding the wrong direction some time ago. Might want to search the forum for posts about rounding and ask ISS to do the same.

 

 

[kevin]

MEP produces -9.5 when asked to round -9.60000000000001 to a single decimal point. This answer is simply wrong - the correct response is -9.6.

You should report this as a bug.

While we might think that the method of rounding a number is a simple decision there are actually many definitions of how to round a number (as illustrated by spending some time searching the Internet). Macro Express supports two rounding methods, Asymetric Arithmetic Rounding and Bankers Rounding. By default, Asymetric Arithmetic rounding is used. You may set Macro Express to use Bankers Rounding with a Registry entry.

 

In either case, Macro Express always rounds up. That is, it rounds to a higher number. In the case of -9.600001, the value -9.5 is the next higher number. Rounding to -9.6 would be rounding to a smaller number.

 

There are different methods of rounding and different opinions about which is the 'correct' one. An argument could be made for rounding to the next higher absolute value but the decision was made to always round up. This decision was actually made for us. Macro Express is written in the Delphi development environment. This is the way that Delphi (and other development environments) round by default.

 

You can use macro commands to alter the rounding method. These commands round to the next higher absolute value:

// Perform absolute rounding
If Variable %Dec% Is Less Than "0"
 Extended Math%Dec%=Abs(%Dec%)
 Variable Modify Decimal: Round %Dec% to 4 decimal places
 Variable Modify Decimal: %Dec% = %Dec% * -1
Else
 Variable Modify Decimal: Round %Dec% to 5 decimal places
End If

[paul]

While we might think that the method of rounding a number is a simple decision there are actually many definitions of how to round a number (as illustrated by spending some time searching the Internet). Macro Express supports two rounding methods, Asymetric Arithmetic Rounding and Bankers Rounding. By default, Asymetric Arithmetic rounding is used. You may set Macro Express to use Bankers Rounding with a Registry entry.

 

In either case, Macro Express always rounds up. That is, it rounds to a higher number. In the case of -9.600001, the value -9.5 is the next higher number. Rounding to -9.6 would be rounding to a smaller number.

I'd be very interested to see any authority where rounding -9.600001 could ever produce -9.5!

[kevin]

I'd be very interested to see any authority where rounding -9.600001 could ever produce -9.5!

As I said, Delphi and other development environments round up. Java rounds up.

 

In Excel "Fix() rounds towards 0 (up in the absolute sense, but down in terms of absolute magnitude). Fix(-3.5) is -3.5." But "Int() rounds away from 0 (up in terms of absolute magnitude, but down in the absolute sense). Int(-3.5) is -4."

 

Open office's calc rounds up.

Spend some time with an Internet search and you will see all kinds of differing opinions. At any rate, Macro Express rounds negative numbers up.

[paul]

As I said, Delphi and other development environments round up. Java rounds up.

I don't understand what you are saying. An hour on Google has yielded the statement that "Delphi uses Banker's rounding", which means that an exact half value causes a rounding to an even number. Please see this URL for some examples: My link

 

Open office's calc rounds up.

Spend some time with an Internet search and you will see all kinds of differing opinions. At any rate, Macro Express rounds negative numbers up.

The very first example given in this example exactly demonstrates my point!

-0.5 -0.6 -0.7 -0.8 -0.9 -1.0 -1.1 -1.2 -1.3 -1.4 will all round to -1

implying that, say, -1.5 will round to -2 (with which I entirely concur).

[kevin]

Paul and Arek, my apologies. What was I thinking? You are correct, there is a rounding error with negative numbers. I was getting confused with how Macro Express rounds numbers that end in 5 (1.5, 1.05, 1.005, etc.).

 

After taking a few days to look at this issue further, I finally realized what is going on. We are working on a fix. In the meantime, enabling Bankers Rounding will fix the issue with rounding -9.600001. It will, however, round numbers that end in 5 differently.

 

To enable Bankers Rounding create this Windows Registry setting:

HKLM\Software\Insight Software Solutions\Macro Express 4\AdvOptions\Bankers Rounding

and give it a DWORD value of 1.

[paul]

Paul and Arek, my apologies. What was I thinking?

At times we are all faced with questions we are unable to answer. I believe yours is an example of this!