rounding problems with milliseconds

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

rounding problems with milliseconds

Ivo Kulms
Hello,

It seems that at least under windows there are rounding problems with the
milliseconds:

sqlite> select strftime("%f", "2005-01-01 12:34:55.122");
55.121

sqlite> select strftime("%f", "2005-01-01 12:34:55.123");
55.122

sqlite> select strftime("%f", "2005-01-01 12:34:55.124");
55.124

Similar with Julian dates:

sqlite> select strftime("%J", "2005-08-02 08:30:00");
2453584.854166667

Which should be 2453584.85417 as I think.


I've used Sqlite version 3.2.2 (windows).

Is this maybe a problem of the underlying c-library?

Has anyboby an idea how to workaround this?


ivo

Reply | Threaded
Open this post in threaded view
|

Re: rounding problems with milliseconds

Doug Currie-2
Wednesday, August 10, 2005, 5:57:50 AM, Ivo wrote:

IK> Hello,

IK> It seems that at least under windows there are rounding problems with the
IK> milliseconds:

IK> sqlite> select strftime("%f", "2005-01-01 12:34:55.122");
IK> 55.121

IK> [...]

IK> Is this maybe a problem of the underlying c-library?

IK> Has anyboby an idea how to workaround this?

Use a better C library ;-).  At least some parts of sqlite use an
internal printf() which does not implement the dragon algorithm.

See:
http://portal.acm.org/citation.cfm?id=989431
http://portal.acm.org/citation.cfm?id=93557

implementations:
http://www.cs.indiana.edu/~burger/fp/
http://netlib.bell-labs.com/netlib/fp/

e