Yeah, ive had my share of curve fitting problems. I just thought I
had sufficiently avoided it this time.
In case you or anyone is interested, i now understand why this flaw
caused such a bump in performance. In essence, this bug caused a form
of "future peeking" that substantially increased backtest performance.
My system is a long/short system based on daily signals. The problem
with this flaw is was not just that it placed trades when funds weren't
yet available, its that it tended to place trades that had a high rate
of success over those first few hours (the time between market open and
existing positions getting stopped out).
Anyway, I won't go into all the details as some of it has to do with
the logic of my system, but suffice to say that this problem of funds
being made available too early can actually cause inordinately
inaccurate backtest results due to future peeking.
Thanks for your help!
--- In amibroker@xxxxxxxxxps.com,
Mark Hike <markhike@xx.> wrote:
>
> Curve fitting is one of the two main pitfalls in stock system
development
> (the other is survivorship bias).
> They come in many forms and have cost me a lot of $$$.
> I would be very cautious if there are more than 3 degrees of
freedom
> (variables). Forward walking and 3-D analysis would help, but it
is hard to
> eliminate, especially when getting all too excited about those
phenomenal
> performance numbers.
>
>
> On Wed, Sep 16, 2009 at 5:00 PM, Brandon_Ridenour <
> brandon_ridenour@...> wrote:
>
> >
> >
> > Its a fair point...i really need to look into switching over
to the CBI. I
> > need to quit my job so i can do this full time.
> >
> > I have had my share of experiences with curve fitting, but i
really thought
> > i had a solid system this time. 10+ years of backtesting,
1000's of trades,
> > consistent profit year after year...and then this slight
change, which only
> > impacted a few trades per year, radically altered the
performance.
> >
> > Its really crazy since I had been tweaking positionscore with
this system
> > previously, causing it to take different sequences of trades,
and the
> > performance was staying solid.
> >
> > Not sure what to make of it...
> >
> >
> > --- In amibroker@xxxxxxxxxps.com
<amibroker%40yahoogroups.com>, Mark Hike
> > <markhike@> wrote:
> > >
> > > I read Graham's note as that you can use the value in
sell array to tell
> > > what has happened, probably for reporting purpose.
Obviously there is
> > some
> > > undocumented effect of setting the value. I would be
wary of this kind of
> > > "hack", because sometimes one doesn't really know what's
going on. I like
> > > the low level CBI because you can look at all the
signals, positions,
> > > trades, equity etc bar by bar and have total control of
what's going on.
> > >
> > > As to your bigger problem, been there done that :-)
> > >
> > > On Wed, Sep 16, 2009 at 4:04 PM, Brandon_Ridenour <
> > > brandon_ridenour@> wrote:
> > >
> > > >
> > > >
> > > > I had reached the same conclusion, that i would
need to use the CBI to
> > work
> > > > around this, but I'm very happy to have avoided it.
> > > >
> > > > This problem only happens if you are managing your
stops manually --
> > here
> > > > is how you can fake it so it thinks you are using
applystop(), and it
> > solves
> > > > this problem. Its in the notes at the bottom of the
link.
> > > >
> > > > Now, my bigger problem is a system I have been
using and had backtested
> > > > over 10 years no longer performs adequately after
making this change.
> > :-/
> > > >
> > > > -------
> > > >
> > > > http://www.amibroker.com/guide/afl/afl_view.php?id=20
> > > >
> > > > Graham Kavanagh
> > > > gkavanagh [at] e-wire.net.au
> > > > 2004-09-30 21:55:42 from equity comments
> > > >
> > > > Depending on kind of the stop various values
> > > > are written back to sell/cover array to enable you
to distinguish if
> > given
> > > > signal was generated by regular rule or by stop.
> > > >
> > > > 1 - regular exit
> > > > 2 - max. loss
> > > > 3 - profit target
> > > > 4 - trailing
> > > > 5 - n-bar stop
> > > > 6 - ruin stop
> > > >
> > > > --- In amibroker@xxxxxxxxxps.com
<amibroker%40yahoogroups.com><amibroker%
> > 40yahoogroups.com>,
> > > > "sdkingman" <markhike@> wrote:
> > > > >
> > > > > Interesting. I could not find anything in the
user guide saying that
> > you
> > > > can set sell/cover signal to 4 or other value to
affect the way stops
> > > > behave. (The only thing I found was in Equity()
function, sell/cover
> > arrays
> > > > would be filled with some number to indicate what
kind of exit has
> > taken
> > > > place)
> > > > >
> > > > > To handle the complex situation you described,
I think that low level
> > CBT
> > > > would be the right way to go.
> > > > >
> > > > > --- In amibroker@xxxxxxxxxps.com
<amibroker%40yahoogroups.com><amibroker%
> > 40yahoogroups.com>,
> > > > "Brandon_Ridenour" <brandon_ridenour@>
wrote:
> > > > > >
> > > > > > I may have found a solution to this. When
I manually set the
> > sell/cover
> > > > signal, I am now setting it to a value of '4',
rather than '1'. When
> > setting
> > > > it this way, ActivateStopsImmediately seems to
apply properly.
> > > > > >
> > > > > > Basically, this fakes out as if I were
using the ApplyStops
> > function (i
> > > > think)
> > > > > >
> > > > > >
> > > > > > --- In amibroker@xxxxxxxxxps.com
<amibroker%40yahoogroups.com><amibroker%
> > 40yahoogroups.com>,
> > > > "Brandon_Ridenour" <brandon_ridenour@>
wrote:
> > > > > > >
> > > > > > >
> > > > > > > I noticed this additional note:
> > > > > > >
> > > > > > > "a) (if you want to have stops
executed AFTER regular signals, so
> > > > cash from stopped out positions is NOT available to
enter trades the
> > same
> > > > day)
> > > > > > > ActivateStopsImmediately turned
ON "
> > > > > > >
> > > > > > >
> > > > > > > It seems to suggest that
ActivateStopsImmediately addresses this
> > > > issue of making sure the freed up equity is not
made available the same
> > day.
> > > > > > >
> > > > > > >
> > > > > > > In my case, because I have custom
stop logic and am not using the
> > > > ApplyStops function, I am using regular Sell/Cover
signals to close
> > > > positions. I set tradedelays(1,0,1,0).
> > > > > > >
> > > > > > > If I understand the logic correctly,
the backtester is improperly
> > > > using my capital in this scenario, and because my
stop logic is custom,
> > > > activatestopsimmediately is not effective.
> > > > > > >
> > > > > > > Is there any way to prevent this
issue from occurring?
> > > > > > >
> > > > > > > Thanks
> > > > > > >
> > > > > > > --- In amibroker@xxxxxxxxxps.com
<amibroker%40yahoogroups.com><amibroker%
> > 40yahoogroups.com>,
> > > > "sfclimbers" <sfclimbers@> wrote:
> > > > > > > >
> > > > > > > > Read the notes provided for
ActiveStops and see if you
> > recognized
> > > > your strategy.
> > > > > > > >
> > > > > > > > http://www.amibroker.com/guide/afl/afl_view.php?id=20
> > > > > > > >
> > > > > > > > You may find that changing the
setting (from AA window) for
> > > > > > > >
> > > > > > > > ActivateStopsImmediately
will fix your problem.
> > > > > > > >
> > > > > > > > Mike
> > > > > > > >
> > > > > > > > --- In amibroker@xxxxxxxxxps.com
<amibroker%40yahoogroups.com><amibroker%
> > 40yahoogroups.com>,
> > > > "Brandon_Ridenour" <brandon_ridenour@>
wrote:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > By the way, i have trade
delays for Buy and Short set to 1,
> > > > meaning it buys on the morning after the signal is
generated. I just
> > need it
> > > > to use the available balance from either the day
the signal was
> > generated,
> > > > or at the exact market open. It seems to be
calculating available
> > > > funds/balance based on the close of the next day.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --- In amibroker@xxxxxxxxxps.com<amibroker%40yahoogroups.com><amibroker%
> > 40yahoogroups.com>,
> > > > "Brandon_Ridenour" <brandon_ridenour@>
wrote:
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > I have a system that
opens positions only at the market
> > open of
> > > > a trading day. At market open, it should calculate
the position size
> > based
> > > > on the funds currently available at that time.
> > > > > > > > > >
> > > > > > > > > > Today there was a
signal at market open but no funds
> > available.
> > > > Later during the day, a trailing stop was hit on
one of my open
> > positions,
> > > > causing that position to be closed.
> > > > > > > > > >
> > > > > > > > > > Tonight when I
updated all quotes and ran the backtest,
> > > > Amibroker shows the position from market open being
opened, even there
> > are
> > > > no funds available. I can only assume that it
believes funds were
> > available
> > > > because of the position which was exited later in
the day.
> > > > > > > > > >
> > > > > > > > > > Obviously, the system
should not be able to use funds which
> > > > have not actually come available yet. How can I
ensure that it will
> > only
> > > > open positions if the funds are available at market
open?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Brandon
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> >
> >
> >
>