@Cashtoe: The 'problem value', as it were, is encountered when using an implementation of the Cumulative Density Function which has been pretty much industry standard for the past 20 years. If you're interested we can arrange to get together on TS or some such to have a look at the actual blurb. What I find suggests a CPU issue is that this exact code has been tested and found to work 100% of the time on numerous machines (MrC, Terr, my server, my workstation etc etc... ASGS being the one exception). Thanks for doing the legwork and checking the hex for the posted values, though.
Since I'm pretty certain that the precision issue is limited to ASGS, there is no need to provide additional results to the original post.
Granary Sergeant Baker - Special Bread Service (Wurf - 13th Oct 2011)
It's definitely the non-associativity of IEEE floating-point addition that is causing this. It's not a CPU or platform issue. To test it, I wrote a little program (in Erlang) that summed the numbers in every possible permutation (11! ~= 40 million ways). I got five distinct answers:
(Note that I didn't get *exactly* the same numbers as you or everyone else, probably because I was using Erlang, which uses 64-bit doubles in memory but may or may not keep intermediate results at 80-bit precision. Also I have my doubts about the accuracy of the floating-point-to-decimal routine. I should try C and maybe strict FP see what I get).
Basically, every time the running sum goes over a power of two (32, 64, 128, 256) -- that's an opportunity to lose a bit of precision at the end.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
So the error you are seeing is 3 ulps. That could easily be accounted for due to the non-associative property of machine floating-point.
In English, everyone's elses compiler is doing the math exactly as you asked for it -- left to right. But since you're using SQL, under the hood the database might be fetching and/or computing the numbers in some random order.
My suggestion is that if an error as small as 3 ulps in an intermediate calculation results in a 50% change in the final result, then whatever algorithm you are using is highly succeptible to rounding error; you need to do some numerical analysis (for example, using interval arithmetic) to determine where you are losing precision.
Edit: I meant non-associative, not non-commutative.
I think cashto wins the cookie
--TE
The Allegiance community currently hates their sysadmin because he is doing: [Too Much] [____________|] [Too Little]
Current reason: Removing the PayPal contribute page. Send Bitcoin instead: 1EccFi98tR5S9BYLuB61sFfxKqqgSKK8Yz. This scale updates regularly.
Globemaster_III wrote:QUOTE (Globemaster_III @ Jan 11 2018, 11:27 PM) as you know i think very little of cashto, cashto alway a flying low pilot, he alway flying a trainer airplane and he rented
cashto wrote:QUOTE (cashto @ Dec 16 2008, 01:30 AM) It's definitely the non-associativity of IEEE floating-point addition that is causing this. It's not a CPU or platform issue. To test it, I wrote a little program (in Erlang) that summed the numbers in every possible permutation (11! ~= 40 million ways). I got five distinct answers:
(Note that I didn't get *exactly* the same numbers as you or everyone else, probably because I was using Erlang, which uses 64-bit doubles in memory but may or may not keep intermediate results at 80-bit precision. Also I have my doubts about the accuracy of the floating-point-to-decimal routine. I should try C and maybe strict FP see what I get).
Basically, every time the running sum goes over a power of two (32, 64, 128, 256) -- that's an opportunity to lose a bit of precision at the end.
I was just going to sudggest doing that, `ym on pre-emptivly fufilling my sudggestion
Deschain wrote:QUOTE (Deschain @ Dec 16 2008, 05:38 AM) Result = 10
Using: fingers
Nothing made me laugh more than this response. And I really needed a GOOD LAUGH
[indent][/indent]Former Squad leader and Assitant Squad Leader BLACKSHADOW™ "Retired" "Democracy is two wolves and a sheep voting on what's for dinner" "Liberty is a well-armed sheep contesting the vote".
