开发者

Problem with truncation of floating point values in DBSlayer

When I run a query through DBslayer http://code.nytimes.com/projects/dbslayer the floating point results are truncated to a total of six digits (plus decimal point and negative sign when needed).

{ ... "lat":52.2228,"lng":-2.19906, ... }

When I run the same query in MySQL, the results are as expected.

| 52.22280884 | -2.19906425 |

Firstly, am I correct in identifying DBSlayer as the cause of this effect? (Or the JSON library it uses, etc.)

Secondly, is this floating point precision configurable within DBSlayer?

Thanks,

Chris.

P.S. Ubuntu 9.10, x86_64

Path: . URL: http://dbslayer.googlecode.com/svn/trunk Repository Root: http://dbslayer.googlecode.com/svn Repository UUID: 5df2be84-4748-0410-afd4-f777a056bd0c Revision: 65 Node Kind: directory Schedule: normal Last Changed Author: dgottfrid Last Changed Rev: 65 Last Ch开发者_JS百科anged Date: 2008-03-28 22:52:46 +0000 (Fri, 28 Mar 2008)


I eventually solved this problem.

There are several places where DBSlayer uses printf's %g to format doubles. This gives only 6 significant digits.

Replacing all of these occurrences with %.15g, and recompiling, has solved the problem.

0

上一篇:

下一篇:

精彩评论

暂无评论...
验证码 换一张
取 消

最新问答

问答排行榜