|
PureBytes Links
Trading Reference Links
|
I'm not sure I completely understand your question.
If you are asking if the backtester in MS is worth the prize in a cracker jack box, it's not. Basically, on the food chain of backtesting programs, it's somewhere around goat's ass.
I've posted info and written articles about backtesting. I realize that a lot of people really love backtesting and believe they are doing it correctly. I've ran 1000's of back tests over the years using a variety of testers and basically none of them are completely accurate, none of them compare to how I do live trading and none of them produce anything more than relative results. Basically, the results of one test can be compared to another test on a relative basis, if the same tester is used, the settings are the same and the same historical data was used.
Even if the tester was perfect, the data most likely wouldn't have been. Most historical testing relies on data and techniques that have multiple biases. Survivorship bias is the one most often talked about. But there are data mining bias, pre-test bias, trading cost bias, look ahead bias, and cherry picking bias. Survivorship bias comes in a dozen different flavors and is very expensive and difficult to eliminate.
In addition, even bias free testing faces the obstacles of proper statistical testing so that the statistical significance between tests is determinable. The results of most backtests are stated as X% profitable trades, with Y% made per trade, etc. Actually, those are the mean results for the tests. That's misleading. The test results should give a confidence interval that puts and high and low bands around the X% and the other statistics.
Once there are decent test results, and one test is compared to another, statistical inference can be derived. The key word is inference. While it is possible to infer a certain trading system is going to earn between $A and $Z dollars 90% of the time, $A and $Z is a very broad number, and 90% means that 10% of the time the real earnings are going to fall outside of the band.
Ninety percent is either really good odds or really bad odds depending on the context. If you have a 10% chance of dying during a surgery, you're going to be really reluctant to make the wager. If you are going to win more than a dime on every dollar bet, you be in within a second.
So basically all testing results are relative. In other words, if one system makes significantly more money under the same conditions as another system, if those conditions repeat themselves, then it's rational to conclude the good performing system will out perform the lessor one. However, it is not possible to conclude how much money is going to be made.
Since history, especially trading history, has never repeated itself and it's not likely to start now. That's another problem.
The bottom line is if you've been making money with a particular system, then keep using it. However, understand that every system has a specific life and that eventually, it's performance will deteriorate to the point where it is no longer worth using. You have to analyze live current data to figure out when that is happening. In addition, every system has a specific set of market conditions it works best under. You better know what those are. When the system performance deteriorates, or the conditions change, move on to something else.
I wish there were perfect testers. I would own one. I wish there were perfect data sets to test on. I would buy them. I wish tests were specific and not just relative. Then I could tell my wife how much income we're likely to have to offset our expenses every year. She would like to know that.
Perfect testers using perfect data would make trading easy because everyone would know what worked, when it worked and how much they were going to make. Sweet!
Super
--- In equismetastock@xxxxxxxxxxxxxxx, trader3cnd <no_reply@xxx> wrote:
>
> I read a paper about backtesting flaws and I am really concerned since one of the programs my partner and I have been using for several years is Metastock and it appears it is one of the programs shown to have flaws in that paper.
>
> Does anyone know anything about those particular flaws/bugs discussed in the paper and in which versions of Metastock were they present?
>
> http://www.tradingpatterns.com/About_Us/articles/backtesting/Backtesting.pdf
>
------------------------------------
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/equismetastock/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/equismetastock/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:equismetastock-digest@xxxxxxxxxxxxxxx
mailto:equismetastock-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
equismetastock-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|