[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Floating point precision (or lack thereof)



PureBytes Links

Trading Reference Links

 >> IIF(L<38.78,X,Y)

the code will yield X when in fact it should yield Y because the low is 
not less than 38.78 (it is exactly 38.78). This occurs simply because of 
the imprecise  representation of the value in AB.

Does anyone know how to get around this, and if its normal for the value 
to be imprecise in AB? <<

Comparing floating point numbers can be a problem if you are not 
careful. Fortunately financial price data is quantized so this problem 
is easy to work around. I your case, lets say the next legal price below 
38.78 is 38.77. You can use any number between 38.78 and 38.77 for your 
test. L<38.779 will work because it is below the computers 
representation of 38.78 but above 38.77. Also L<38.778 will work so will 
L<38.777, etc.

Bill



------------------------ Yahoo! Groups Sponsor --------------------~--> 
<font face=arial size=-1><a href="http://us.ard.yahoo.com/SIG=12h9kadq5/M=362343.6886682.7839641.1493532/D=groups/S=1705632198:TM/Y=YAHOO/EXP=1124896937/A=2894354/R=0/SIG=11qvf79s7/*http://http://www.globalgiving.com/cb/cidi/c_darfur.html";>Help Sudanese refugees rebuild their lives through GlobalGiving</a>.</font>
--------------------------------------------------------------------~-> 

Please note that this group is for discussion between users only.

To get support from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com

For other support material please check also:
http://www.amibroker.com/support.html

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/