bizzare nullable decimal precision behavior
I have some columns in a data store using 12 decimal precision.
I want to round it to 4 places for the purpose of sending it off via text feed, and the consumers of that feed need x precision.
I am using a custom built workflow engine, and don't have direct access to the code at any point of execution, but can inject code on any field using conditons for example:
if (obj.myValue.HasValue)
obj.myValue = System.Math.Round(obj.MyValue.Value, 4)
the above example works in doing the rounding, but in SOME cases does not remove the trailing 0's. The end result automatically generated using obj.myValue.ToString() (and so on, for all the fields)
This seems to happen only on nullable decimal fields.
I am not sure what is different but in about 10% of the rows, the output retains trailing 0's. The actual value IS rounded, but 0s stay:
134.402100000000
I also tried doing it this way (just to be sure and remove the initial precision of the decimal)
if (obj.myValue.HasValue)
obj.myValue = decimal.Parse(obj.MyValue.Value.ToString("#.####"))
again, SOME rows remain with appended trailing 0s.
It almost looks like the property is initialized to x decimal places, and final ToString.
Considering i do not have direct access of this code, but can inject any c# to be performed on that property, are there any other things I could try? (I also tried hard coding all rows to a value, and that works fine w/out extra 0s)
also interesting is that i can't reproduce this behavior in my tests..
decimal d1 = 160236194.3900000000000001M;
Console.WriteLine(d1);
d1 = Math.Round(d1, 4);
Console.WriteLine(d1);
160236194.3900000000000001
160236194.3900
Press any key to continue . . .
edit
tried another approach thanks to e.James (testing to see if truncate would remove the 0s)
if (obj.myValue.HasValue)
obj.myValue = decimal.Truncate(obj.MyValue.Value)
and.. truncati开发者_Go百科on works on all rows, but 0s still stay on those that had the previously! The difference is that this time it is only 0s.. before truncating it was .8900000000 etc.. This is wild!
Some decimal numbers can not be perfectly represented in binary format. Your obj.myValue
property is a decimal number. For certain decimals, it simply will not be able to store anything other than (for example) 1234.4021000000000001, which is as close as it can be with a limited number of bits.
I think Domenic has the right idea with his answer. Let decimal numbers remain un-rounded in storage, and only display* them rounded off.
If you absolutely must have the numbers rounded in storage, you will have to use an integer format, which you then divide by 1000 prior to display. 1234.4021 would be stored as 12344021.
*note: by "display" I mean "output to text", not necessarily "show to a user"
Why not let the display logic handle the, uh, display logic? That is, only when you go to display the decimal
as a string
should you care about the number of trailing 0
s displayed, in which case you should use theDecimal.ToString("#.####")
.
精彩评论