Floating point comparison in VB6
What's the best way to test two Singles for equality in VB6?
I want to test two Single values for equality to 7 significant figures.
This MSDN article recommends using something like
If Abs(a - b) <= Abs(a / 10 ^ 7) Then
valuesEqual = True
End If
However, that can fail for certain values, e.g.
Public Sub Main()
Dim a As Single
Dim b As Single
a = 0.50000005
b = 0.50000014
Debug.Print "a = " & a
Debug.Print开发者_如何学C "b = " & b
Debug.Print "a = b: " & (a = b)
Debug.Print "SinglesAreEqual(a, b): " & SinglesAreEqual(a, b)
// Output:
// a = 0.5000001
// b = 0.5000001
// b = b: False
// SinglesAreEqual(a, b): False
End Sub
Private Function SinglesAreEqual(a As Single, b As Single) As Boolean
If Abs(a - b) <= Abs(a / 10 ^ 7) Then
SinglesAreEqual = True
Else
SinglesAreEqual = False
End If
End Function
The simplest way I've found of getting the result I need is to convert the values to strings, but seems horribly ugly:
Private Function SinglesAreEqual(a As Single, b As Single) As Boolean
SinglesAreEqual = (Str$(a) = Str$(b))
End Function
Are there any better ways?
I maintain a CAD/CAM application and I have to deal with floating point numbers all the time. I have a function that I call fComp
that I pass a floating point value when I need to test for equality. fComp
call a rounding function set to a certain level of precision. For our system I round to 6 decimal places. Yours may need higher or get away with lower it depends on the application.
The fComp
Function exists so I have one spot to change the rounding factor used in these calculations. This proved handy a couple of years back when we started manufacturing higher precision machines.
Public Function pRound(ByVal Value As Double, ByVal Power As Double) As Double
Dim TempValue As Double
Dim tSign As Double
TempValue = Value
tSign = TempValue
TempValue = Abs(TempValue)
TempValue = TempValue * 10 ^ (Power * -1)
TempValue = Fix(TempValue + 0.5)
TempValue = TempValue / 10 ^ (Power * -1)
pRound = TempValue * Sign(tSign)
End Function
To round to the 6th decimal place you go
RoundedNumber = pRound(MyValue, -6)
Negative is to the right of the decimal place positive to the left.
Instead if rounding and testing for equality, you can take the difference of two numbers and compare that with a factor
If Abs(a - b) < 0.000001 Then
You can adjust the 0.000001
to whatever resolution you need
I don't believe you can use the single
data type to that many significant figures. You would need to use double
instead:
Dim a As Single
Dim s As String
s = "0.50000005"
a = 0.50000005
Debug.Print s & " " & a
The above outputs:
0.50000005
0.5000001
精彩评论