开发者

multiplication error

If I multiply 12 x 25.4 i get 304.7 whereas I expect 304.8. I see that using odd numbers x 25.4 I get the c开发者_开发问答orrect answers but using even numbers always seem to be off by 0.1. I am writing a scientific app that is heavily based on formulas so any insight would be helpful.

if (((m1_sqs1_spinner.getSelectedItem().toString().equals("in"))))  

    { // start square in inches
         double m1_sqs1_eng = new Double(m1_sqs1.getText().toString()); 

         double square_effective_dia_inch = m1_sqs1_eng;

         double square_effective_dia_mm = square_effective_dia_inch * 25.4;
         m1_ed_mm.setText(Double.toString(square_effective_dia_mm));

    } // end square in inches


Floating point number types in Java are approximations. Try adding 1 + 2 + 3 + 4 as float type and you might end up with 10.00000003.

To ensure accurate math, try using the java.math.BigDecimal type. It is a memory hog, but is accurate.


I don't know what programming language you're using, but 12 * 25.4 gives me 304.8 when I do this using Java:

public static void main(String[] args) {
   double result = 12 * 25.4;

   System.out.printf("Result: %.1f%n", result );
}

But seriously -- how are you displaying the result obtained? Are you using one of the many numeric formatting options available in Java (one of which is displayed above)? Also do you understand the limits to precision when using 64-bit IEEE 754 floating point (double) variables? They are pretty accurate and useful for most floating-point applications, but not applications that have strict precision requirements such as financial calculations.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜