|
PureBytes Links
Trading Reference Links
|
It seems that the base operating system that is used on the computer for
storing numbers has a rounding error when the number exceeds 7 digits. You
can experiment with this by doing an explore with the result column
containing 8 decimal places. Here I have used number 1.1 with various
decimal places
Filter = 1;
x=1.1;
AddColumn(x,"num",1.1);
AddColumn(x,"num",1.7);
AddColumn(x,"num",1.8);
AddColumn(x,"num",1.9);
And the results columns
UNI 04/04/2003 1.1 1.1000000 1.10000002 1.100000024
And the same with large numbers
Filter = 1;
x = 1000123; //7 digits
y = 10000123; //8 digits
z = 100000123; //9 digits
AddColumn(x,"num",1);
AddColumn(y,"num",1);
AddColumn(z,"num",1);
Results are
UNI 04/04/2003 1000123 10000123 100000120
Cheers,
Graham
http://groups.msn.com/ASXShareTrading
http://groups.msn.com/FMSAustralia
-----Original Message-----
From: mmqp [mailto:mmqp@xxxxxxxxx]
Sent: Saturday, 5 April 2003 5:38 AM
To: amibroker@xxxxxxxxxxxxxxx
Subject: [amibroker] strange "cum(volume) result compared to Excel.
Hi,
Has anyone seen this strange results from cum(volume) as it compared
to same formula in Excel spread sheet. I have following lines in
Auto-Analysis.
AddColumn(Volume ,"Volume",1);
AddColumn(Cum(volume) ,"CumVol",1);
filter = 1;
"Explore" on one stock; copy result to an Excel sheet. insert
cumulative formula for volume and the results are different in some
lines moving forward.
I would expect I get no 1's and 10's column in AB "cum" function
because volume datas are reported with 100's as smallest size.
However I see some 2,4,6,8 in the least significant column. Has
anyone seen this before and have an explaination for it?
TIA
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at:
http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
------------------------ Yahoo! Groups Sponsor ---------------------~-->
Make Money Online Auctions! Make $500.00 or We Will Give You Thirty Dollars for Trying!
http://us.click.yahoo.com/yMx78A/fNtFAA/i5gGAA/GHeqlB/TM
---------------------------------------------------------------------~->
Send BUG REPORTS to bugs@xxxxxxxxxxxxx
Send SUGGESTIONS to suggest@xxxxxxxxxxxxx
-----------------------------------------
Post AmiQuote-related messages ONLY to: amiquote@xxxxxxxxxxxxxxx
(Web page: http://groups.yahoo.com/group/amiquote/messages/)
--------------------------------------------
Check group FAQ at: http://groups.yahoo.com/group/amibroker/files/groupfaq.html
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
|