|
PureBytes Links
Trading Reference Links
|
--- In amibroker@xxxx, "Tomasz Janeczko" <tj@xxxx> wrote:
> During my own research I checked the performance of the very
> same system on different groups of stocks and the results
> was from loosing money on most of the trades to
> outperforming B&H by 400%. The system clearly preferred
> less volatile, blue chip stocks.
How do you really know there is a true cause-and-effect relationship
between the system and the different groups? It might clearly *seem*
to be true but without rigorous statistical analysis you don't know
if your results are significant. Also, the results by group could be
dynamic -- perhaps in 2002 blue chips will be at the bottom instead
of on top.
> 1. yes, comparing backtesting results can be misleading but
> does it mean we shouldn't do that at all?
> I guess giving backtesting results for different markets/groups
> will give valuable information on the behaviour of the system
Well, statistically rigorous backtesting is a lot of work and
requires processing data over a lot of time frames. Plus analysis to
guard against skew, overparameterization, bias, seasonal effects,
etc.
"Steve" <slwiserr@xxxx> wrote:
> Almost my whole Amibroker database is based upon specific stock
> selection criteria. Therefore my complete Amibroker database is
> very specific
No real difference between this and using Auto Analysis to filter.
The same danger applies, that 'apparently obvious' results from
backtesting may or may not be real.
Anyway, I don't mean to pour cold water over backtesting. I'm just
suggesting that there's a lot to think about regarding
what "backtesting" implies in terms of effort to get meaningful
results.
Regards,
Jim Varney
|