开发者

ruby: converting from float to integer in ruby produces strange results

ree-1.8.7-2010.02 :003 > (10015.8*100.0).to_i
 => 1001579 
ree-1.8.7-2开发者_如何学Go010.02 :004 > 10015.8*100.0
 => 1001580.0 
ree-1.8.7-2010.02 :005 > 1001580.0.to_i
 => 1001580 

ruby 1.8.7 produces the same. Does anybody knows how to eradicate this heresy? =)


Actually, all of this make sense.

Because 0.8 cannot be represented exactly by any series of 1 / 2 ** x for various x, it must be represented approximately, and it happens that this is slightly less than 10015.8.

So, when you just print it, it is rounded reasonably.

When you convert it to an integer without adding 0.5, it truncates .79999999... to .7

When you type in 10001580.0, well, that has an exact representation in all formats, including float and double. So you don't see the truncation of a value ever so slightly less than the next integral step.

Floating point is not inaccurate, it just has limitations on what can be represented. Yes, FP is perfectly accurate but cannot necessarily represent every number we can easily type in using base 10. (Update/clarification: well, ironically, it can represent exactly every integer, because every integer has a 2 ** x composition, but "every fraction" is another story. Only certain decimal fractions can be exactly composed using a 1/2**x series.)

In fact, JavaScript implementations use floating point storage and arithmetic for all numeric values. This is because FP hardware produces exact results for integers, so this got the JS guys 52-bit math using existing hardware on (at the time) almost-entirely 32-bit machines.


Due to truncation error in float calculation, 10015.8*100.0 is actually calculated as 1001579.999999... So if you simply apply to_i, it cuts off the decimal part and returns 1001579


http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

>> sprintf("%.16f", 10015.8*100.0)
=> "1001579.9999999999000000"

And Float#to_i truncates this to 1001579.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜