[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: TJ: tick charts, historical data



PureBytes Links

Trading Reference Links

Hello,

> I have the realtime version but not with a eSignal feed. With eSignal 
> can you request historical tick data from their servers? If so, how 
> much can you load into AB?
>
Yes eSignal plugin provides tick histories. There is a limit of 10 days
of tick data and 100000 bars currently. The 10 days limit is imposed
by eSignal tick server, 100000 bars limit is imposed by Database settings
window in AmiBroker. Since I have been asked to increase it
- I will do that in the next beta. 


> Given the ascii import limitation how is a person supposed to go 
> about realistically backtesting longer amounts of tick data, 
> especially if eSignal doesn't provide access to large amounts of 
> historical tick data? For running the system, eSignal would be fine, 
> but unless you can request severals months or longer worth of tick 
> data from eSignal how could you possibly back test?

As written above eSignal provides max. 10 days of tick-by-tick data.
Frankly I am not big fan of backtesting on single tick data.
There are several reasons for this: 
a) tick data for one day can be 1MB or more. Several months
of tick data will sum up to more than 100MB of data. This
will slow down backtesting very much.
b) there is much less probability that system backtested on tick
data will ever work in reality. This is due to the fact that tick data
represent much more "noise" than interval data that represent some kind
of "smoothed" market response. Backtesting and optimization
systems on tick data is much more prone to curve-fitting and over-optimization.
Also backtesting system on tick data is bad idea because 
it gives you false feeling that you can trade on every tick.
In reality we have to face slippage, order routing delays and other problems
that make backtesting on single tick data much less reliable
than testing interval data. 

> 
> I also don't understand why their is a difference between the way 
> ascii importer works and the eSignal plugin. Is the eSignal data 
> compressed in the same way as the ascii importer data or not?
No. 
The difference comes from the fact that plugin architecture is completely
different from what ASCII importer does.
Plugin cares about the data and provides ready-to-use 
quote array to AmiBroker. 
AmiBroker just uses it without any transformation.

ASCII importer is quite different - it reads the input file line by line
and matches date and time values read to records already present in the AmiBroker
database - if a matching record (for the same date/time) is found
new data overwrite existing record and no new record is added.

Now consider tick data. Tick data very often contain multiple records
for the same hour:minute:second. No data feed provides more resolution
than ONE second so multiple trade records occur for the same second.
With plugin architecture - such records are passed directly to AmiBroker
and it accepts multiple records with the same date/time.

Using ASCII importer - you can not import multiple records for the same
symbol, date and time because once data because matching record
overwrites existing one so if you try to import 5 records having the same time
one will overwrite another and only one (the last one) will remain.

This behaviour is required by 99.9% users because people want
to re-import their data (with probable data fixes) and want new data to
overwrite old data.
Also ASCII importer does not imply any ordering of data thanks
to this matching mechanism. You can import year 2001 then 1998 then
2002 then 1999 and everything will work OK.

Allowing multiple records per same date/time will force you to
"assume" some rule to distinguish and order such records.
You would need to assume that new record is newer or older than
the previous one with the same date/time. 

Now with eSignal plugin this is not a problem because
eSignal builds up entire quote array - it is aware of multiple
records per same second and nows about the correct order since
eSignal servers return tick histories in correct order.

> 
> I understand that you don't officially support tick data charting 
> without the eSignal plugin, but please appreciate that I am 
> attempting to backtest here. I really am not currently concerned with 
> realtime tick charting. When I get to the point of running my 
> systems, then maybe, but not now. I also understand that you only 
> have so much time and that you can't go supporting every crazy 
> project of every person that buys your software. I am ALL FOR going 
> about this myself, but please appreciate the fact that there is 
> nothing at all in the help files about tick charting or the 5 second 
> issue with ascii import.
I agree that I didn't write that exactly but I did write that 
only RT version supports tick, 5-sec, 15-sec and I did write that
you need eSignal subscription. 

I will be publishing data feed plugin API soon with a sample
code and I will think about adding a section how to implement
plugin that reads ASCII tick data.

As I explained above - the special nature of tick data (multiple records
for the same second) - make it different story to import it from ASCII file

Best regards,
Tomasz Janeczko
amibroker.com