ASP.Net conversion on using float numbers
I have a conversion or some kind of type casting problem in my asp.net/C# math app. A contractor wrote some code to calculate a relative standard error in an assembly. But when I run it, apparently doesn't work right.
The data types are as follows:
public double dSwx = 0.0, dSw = 100.0;
public double dTx = 0.0, dTwn=0.0;
public float fEstimateVal = 0.0F; // For Estimate Value, it should be same as Swx/Sw
//i did a system.out on this in my web app and they came out as:
//zeroDataCell.dSwx = 0.0;
//zeroDataCell.dSw = 100.0;
zeroDataCell.fEstimateVal = (float)(zeroDataCell.dSwx / zeroDataCell.dSw) * 100.0f;
//so now zeroDataCell.fEstimateVal should be 0.0, but my code blows
//throug开发者_StackOverflow社区h the if statement below, is there some conversion problem?
//should i use EqualsTo?
if (zeroDataCell.fEstimateVal != 0.0f)
zeroDataCell.fRse = zeroDataCell.fTwx / zeroDataCell.fEstimateVal * 100.0f;0
Why does fEstimateVal that eqauls zero stil run through the if loop?
Dealing with float/double numbers is always precision balancing. If you want to compare it against clear 0, convert it to integer and compare after.
Or add some precisio stuff like
((int)(floatnumber * 100)) == 0.
The following code works fine here:
if (zeroDataCell.fEstimateVal != 0)
zeroDataCell.fRse = zeroDataCell.fTwx / zeroDataCell.fEstimateVal * 100.0f;
精彩评论