|
PureBytes Links
Trading Reference Links
|
Hi Doug --
The objective function is not selected on a theoretical basis. It is selected by you by looking at a variety of trading results and selecting the objective function that best describes the alternatives that best fit your own style and personality. For a start, print out some equity curve charts. Lay them on the floor and look at them. Rearrange them and rank them in order from best to worst according to your own personal preferences. Select or design an objective function that scores highest on those charts you like and lowest on those you reject.
I think there are several criteria that must be met:
Having a trading system with a positive expectancy.
Having an objective function that fits your trading style and personality.
Having a trading system that passes walk forward validation.
My experience is that objective functions that reward fairly short holding periods and smooth equity curves both fit my style and personality and perform reasonably well out-of-sample.
Your experience with the tests you ran agree pretty well with mine. In particular, using net profit as the objective function usually (but not always) selects systems that do not perform well out-of-sample. But notice that net profit is often the default objective function for trading system development platforms. In fact, some platforms default to net profit and give no opportunity to use anything else. But those also usually do not provide the tools for system validation, such as automatic walk forward testing, that AmiBroker does.
On the subject of adding data to the in-sample period. More is not always better. Determination of the length of the in-sample period is critical to getting good results. If there is too little data, the model will not detect enough patterns or enough of the pattern, and it will fail to recognize the pattern both in-sample and out-of-sample. If there is too much data, the extraneous / old / stale data will distract the model and the signal to noise ratio will drop. Experiment to find the right length. That length is very dependent on the model and the data it is processing.
Thanks,
Howard
On Fri, May 8, 2009 at 1:36 PM, dloyer123 <dloyer123@xxxxxxxxx> wrote:
This raises a interesting point.
If some objective functions are poor predictors of out of sample performance, are there other objective functions that are good predictors of out of sample performance? Or at least better?
It seems like this is the kind of problem where there should be a optimal answer. I wish that I had the math background to find it.
The best that I could come up with is that there needs to be a measure of performance and a measure of risk to balance it. There are several such measures in common use and they will each tend to favor different solutions.
Since I am not able to pick one on a theoretical basis, I had to resort to empirical testing using Ami's walk forward feature. As a result, I found that net profit performed the worst of all of the objective functions I tried in terms of out of sample profit. In my simple test, UPI performed the best. CAR/MDD also performed well. I don't recall how the others did. I believe that Sharpe was somewhere in the middle and k-ratio didnt do well at all.
However, this is just one simple system and I really wish that I had a theoretical basis to hang my hat on it. Results for other systems may be very different and it might be different if I repeated the test now, with a extra year of test data.
-Doug
--- In amibroker@xxxxxxxxxxxxxxx, Howard B <howardbandy@xxx> wrote:
> It is also interesting to note that some objective functions tend to reward
> / select values for the logic and parameters that result in trading systems
> that do not trade well out-of-sample. In particular, be careful using net
> profit, sharpe ratio, and so forth. By all means, define your own custom
> objective function metric, program it, and tell AmiBroker to use it.
>
> Thanks for listening,
> Howard
>
>
__._,_.___
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
![]()
__,_._,___ |