Tuesday, January 12, 2016

Re: How to avoid rounding problems when working with doubles

Try BigDecimal if you need precision.

/Stefan

Den tisdag 12 januari 2016 kl. 12:28:31 UTC+1 skrev Frank:
Javascript has some serious problems when working with floating numbers.

Like if you enter the following into a javascript in any browser : 7.3 - 7
These will all give the result : 0.2999999999999998


This problems is also visible in GWT : 

int test = 7;
double dTest = 7.3;
double result = dTest - test;
GWT.log("test result = " + result);

will also print out 0.2999999999999998


How does one avoid these problems in GWT, so that you can trust on the results you receive when calculating with floating point numbers ?



FYI : The specific use case that I have is that users enter a time in decimals hours in a textbox. And I must recalculate this to minutes.
For example when the user enters 7.5, this should give 450 minutes.
But with all these rounding problems this often goes of by one minute...

--
You received this message because you are subscribed to the Google Groups "GWT Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-web-toolkit+unsubscribe@googlegroups.com.
To post to this group, send email to google-web-toolkit@googlegroups.com.
Visit this group at https://groups.google.com/group/google-web-toolkit.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment