arekowczarek Posted November 21, 2010 Report Share Posted November 21, 2010 In one of my macros I have three decimals: %PROBBEFORE%=33,4 %PROBAFTER%=23,8 %DIFFERENCE%=%PROBBEFORE%-%PROBAFTER% The problem occured only for the values above. I run it about a hundred times, different values each time. The %DIFFERENCE% should equal "-9,6". Instead of this I am getiing "-9,60000000000001". It's not just a matter of rounding the result, because 33,4-23,8 ≠ -9,60000000000001 Now, I tried to create a new macro with the decimals preset to the values above - and the error didn't occur. So, somehow it only appears in the original macro file. My first thought was that I didn't round the two decimals before the subtraction. But I debugged it and watched the values of decimals. And they equal 33,4 and 23,8 just before the subtraction. and then the subtraction result becomes -9,60000000000001. If anyone wanna give it a quick check I attached the macro file. The macro doesn't interfere with your mouse or keybord (no text type). All you will get are two text boxes. The last three lines of macro is where it happens. Unless I've made some dumb error myself and somebody can point it out, I'll report it as a bug to ISS. MOVE PROBABILITY.mex Quote Link to comment Share on other sites More sharing options...
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.