|
PureBytes Links
Trading Reference Links
|
Hello Tomasz,
Thank you for elaborating on this matter. Gives us some more insight
on the special nature of tick data. Thanks Sean for bringing it up.
One more thing about it:
Although I'm rather sceptical about backtesting and especially
optimizing in general, I don't quite see why backtesting on tick data
would be less reliable than on interval data. Well, if you (could)
optimize for an exact number of ticks in a bar, yes that would be a
clear example of "curve fitting". But other than that I would say
that the boundaries (O,H,L,C) of any bar are always a bit arbitrary:
if a bar had closed three seconds later (e.g. due to a slow computer
clock), the bar would be different, but it would not make much
difference for the whole picture of trend and momentum.
When using tick bars, the same thing happens (as you know): in an
equally *arbitrary* way, the beginning and ending of a bar are
determined. The only difference is that the criterium for determining
is not the amount of time elapsed, but a particular # of bars having
occurred. As soon as you have chosen your tick #, the noise is taken
out. Only the "smoothing" is done in a different way, but it's being
done just as much as in an interval chart.
I'm not talking about bars with very low # of bars of course or very
low liquidity stocks, then it may be a different matter. But working
with say, 50, 100, 200 and 500 tick charts on liquid securities,
should not be less reliable in backtesting (or chart study) than say,
1, 5, 15 min bars charts.
At least I fail to see why it could be.
So for the near future, I'm happy with your promise of raising the
100000 bar limit. On a longer time frame you might consider a way
for "true" tick importing from ASCII. Despite huge file sizes and
resource problems, I think it would be appreciated by more people.
Regards,
Ralph
--- In amibroker@xxxx, "Tomasz Janeczko" <amibroker@xxxx> wrote:
> Hello,
>
> > I have the realtime version but not with a eSignal feed. With
eSignal
> > can you request historical tick data from their servers? If so,
how
> > much can you load into AB?
> >
> Yes eSignal plugin provides tick histories. There is a limit of 10
days
> of tick data and 100000 bars currently. The 10 days limit is imposed
> by eSignal tick server, 100000 bars limit is imposed by Database
settings
> window in AmiBroker. Since I have been asked to increase it
> - I will do that in the next beta.
>
>
> > Given the ascii import limitation how is a person supposed to go
> > about realistically backtesting longer amounts of tick data,
> > especially if eSignal doesn't provide access to large amounts of
> > historical tick data? For running the system, eSignal would be
fine,
> > but unless you can request severals months or longer worth of
tick
> > data from eSignal how could you possibly back test?
>
> As written above eSignal provides max. 10 days of tick-by-tick data.
> Frankly I am not big fan of backtesting on single tick data.
> There are several reasons for this:
> a) tick data for one day can be 1MB or more. Several months
> of tick data will sum up to more than 100MB of data. This
> will slow down backtesting very much.
> b) there is much less probability that system backtested on tick
> data will ever work in reality. This is due to the fact that tick
data
> represent much more "noise" than interval data that represent some
kind
> of "smoothed" market response. Backtesting and optimization
> systems on tick data is much more prone to curve-fitting and over-
optimization.
> Also backtesting system on tick data is bad idea because
> it gives you false feeling that you can trade on every tick.
> In reality we have to face slippage, order routing delays and other
problems
> that make backtesting on single tick data much less reliable
> than testing interval data.
>
> >
> > I also don't understand why their is a difference between the way
> > ascii importer works and the eSignal plugin. Is the eSignal data
> > compressed in the same way as the ascii importer data or not?
> No.
> The difference comes from the fact that plugin architecture is
completely
> different from what ASCII importer does.
> Plugin cares about the data and provides ready-to-use
> quote array to AmiBroker.
> AmiBroker just uses it without any transformation.
>
> ASCII importer is quite different - it reads the input file line by
line
> and matches date and time values read to records already present in
the AmiBroker
> database - if a matching record (for the same date/time) is found
> new data overwrite existing record and no new record is added.
>
> Now consider tick data. Tick data very often contain multiple
records
> for the same hour:minute:second. No data feed provides more
resolution
> than ONE second so multiple trade records occur for the same second.
> With plugin architecture - such records are passed directly to
AmiBroker
> and it accepts multiple records with the same date/time.
>
> Using ASCII importer - you can not import multiple records for the
same
> symbol, date and time because once data because matching record
> overwrites existing one so if you try to import 5 records having
the same time
> one will overwrite another and only one (the last one) will remain.
>
> This behaviour is required by 99.9% users because people want
> to re-import their data (with probable data fixes) and want new
data to
> overwrite old data.
> Also ASCII importer does not imply any ordering of data thanks
> to this matching mechanism. You can import year 2001 then 1998 then
> 2002 then 1999 and everything will work OK.
>
> Allowing multiple records per same date/time will force you to
> "assume" some rule to distinguish and order such records.
> You would need to assume that new record is newer or older than
> the previous one with the same date/time.
>
> Now with eSignal plugin this is not a problem because
> eSignal builds up entire quote array - it is aware of multiple
> records per same second and nows about the correct order since
> eSignal servers return tick histories in correct order.
>
> >
> > I understand that you don't officially support tick data charting
> > without the eSignal plugin, but please appreciate that I am
> > attempting to backtest here. I really am not currently concerned
with
> > realtime tick charting. When I get to the point of running my
> > systems, then maybe, but not now. I also understand that you only
> > have so much time and that you can't go supporting every crazy
> > project of every person that buys your software. I am ALL FOR
going
> > about this myself, but please appreciate the fact that there is
> > nothing at all in the help files about tick charting or the 5
second
> > issue with ascii import.
> I agree that I didn't write that exactly but I did write that
> only RT version supports tick, 5-sec, 15-sec and I did write that
> you need eSignal subscription.
>
> I will be publishing data feed plugin API soon with a sample
> code and I will think about adding a section how to implement
> plugin that reads ASCII tick data.
>
> As I explained above - the special nature of tick data (multiple
records
> for the same second) - make it different story to import it from
ASCII file
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
|