|
PureBytes Links
Trading Reference Links
|
Hi Herman, and all --
The issue determining the length of the in-sample period is not the number of trades, but the number of data points it takes for the model (afl) and the data (OHLC) to become synchronized. Imagine a model that is looking for cyclic behavior in the data. If the cycle completes in one month it takes many fewer data points to sync than if the cycle completes in one year.
As for out-of-sample, the statistics can be run on any number of data points. The data points can be daily equity changes or closed trades -- whatever time granularity the trader will be making position adjustments. Providing the relationship between the mean of the metric being studied, its standard deviation, and the number of data points is reasonable, control charts can used to indicate when the system is operating well and when it is beginning to fail.
Assume the metric is expectancy. If the mean expectancy is positive and the ratio of the mean to the standard deviation is high (high would be a ratio of 0.5, extremely high would be 1.0, normal would be 0.2), it takes fewer than 20 trades to have very high confidence that the system is performing well. Since systems go out of sync -- either periodically or forever -- once there are enough OOS data points to give the confidence that the developer requires, there is no benefit in waiting for further confirmation. Trade now, while the system is in sync.
Thanks for listening,
Howard
On Fri, Feb 19, 2010 at 6:51 AM, Herman vandenBergen <psytek@xxxxxxxx> wrote:
Hello Howard,
I cannot help but wonder why the OOS periods in optimizations aren't based on the number of trades, instead of some arbitrary time period. Could it be that this is just another example of how TA methodology is based on convenience, i.e., a reluctance to redesign tools and/or a reluctance to change :-)
I am no statistician, but I would guess that if one wanted some specific OOS performance the first thing to do would be to figure out how many trades would be needed to do the calculations. If 100 trades gives you the confidence why would you run a OOS period that gives you 200 trades?
Best regards,
herman
Friday, February 19, 2010, 8:18:45 AM, you wrote:
|
Hi SpaceBass --
The only way to determine the correct length for the in-sample period is by running experiments. The length needs to be long enough for the model to synchronize with the data and learn to recognize the signal. But not so long that the signal has changed significantly, making it hard to identify. And not so short that there is not enough signal to learn, resulting in a system that has synced to the noise.
In a few words -- the length of the in-sample period should be as short as is practical and effective.
Once the length of the in-sample period has been determined, the length of the out-of-sample period is easy. It is the length of time that the system remains profitable.
There is no general relationship between the anything and the length of the in-sample period. There is no relationship between the length of the in-sample period and the length of the out-of-sample period.
There is some controversy in the modeling and simulation field about whether the in-sample length is a legitimate variable in an optimization. While it can be dangerous to system validity to run an optimization on in-sample length, some trials using different lengths are necessary.
Thanks for listening,
Howard
On Thu, Feb 18, 2010 at 7:23 PM, spacebass5000 <spacebass5000@xxxxxxxxx> wrote:
I was wondering if there was a way to optimize the In-Sample and Step time periods within AB. If not, can someone point me in the direction of a good resource on this topic?
|
__._,_.___
**** IMPORTANT PLEASE READ ****
This group is for the discussion between users only.
This is *NOT* technical support channel.
TO GET TECHNICAL SUPPORT send an e-mail directly to
SUPPORT {at} amibroker.com
TO SUBMIT SUGGESTIONS please use FEEDBACK CENTER at
http://www.amibroker.com/feedback/
(submissions sent via other channels won't be considered)
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
![]()
__,_._,___ |