[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: System evaluation



PureBytes Links

Trading Reference Links

I don't agree with this post at all. Here's why...

First, I'm not clear whether this rule applies to the purchase of a system
or the purchase of "education". They are very different. A system means
that there is a set of rules for when/how you enter and when/how you exit a
trade. Results will never differ. For example, if I tell you my system (and
if indeed it is a system), you will get exactly the same results as I do
(except for varying slippage) if you follow its rules exactly.

MOST important in a system is how it is tested. You can't just toss it into
daily bars on TradeStation and have it throw out one of those one page
summaries. You must dig through time and sales to get REALISTIC assumptions
about where fills would occur...and test loads of out of sample data to get
a true idea of drawdown and see the smoothness of the equity curve.

A system, being a set of rules on paper, doesn't know if it's rules are
being traded or not. Those rules would exist regardless of whether you told
a broker to make buy and sell orders based on them. Therefore, a system
performs the same whether it's traded or not provided you make realistic
expectations on fills and verify them.

What's more, a system's performance -- whether hypothetical (tested
carefully) or real -- is only a measure of performance in the past. It does
not predict future performance. Just because you have a brokerage statement
doesn't mean that markets won't change and degrade the system performance.

So let's say someone develops a system that after being tested *carefully*
over a long time period on out of sample data returns 35 percent per year
on a fairly smooth equity curve with a $50,000 drawdown. Let's say I
developed it. The best thing you can say about my system is that it
performed okay in the past. There are no guarantees for the future.

Now I think we all agree that one should only use risk capital for
speculation on  futures contracts. To me, risk capital means that if I took
the $50,000 I need to absorb the drawdown -- and just flushed it down the
toilet, I'd be making no sacrifice in my lifestyle. Let's say by my
definition I do not have risk capital. Is my system less worthy?

As far as systems are concerned, I would be more cautious about how it is
tested, what assumptions were made about fills and how smooth the equity
curve is than whether it was actually used. A system is just an algorithm
-- nothing more, nothing less. If my algorithm for predicting the weather
has a great hypotetical track record, does the fact that Willard Scott
never used it a ding on its record?

Now... if you're talking about methodology...that's another story.
Methodology means I will "teach" you a method. But it also means that your
results and my results will, by definition, vary based on how you interpret
my work. Let's say I'm a guru and I say I'll teach you how to use Andrew's
Pitchforks.

Sounds great. But how would you know whether or not my results had anything
to do with Pitchforks? The answer is you don't. In fact, even a trading
statement with a rule-based system means nothing because who knows whether
the system generated the results or whether discretionary trades were made?
And furthermore, unless you really "internalized" what I know about
Pitchforks, you'll never be able to trade as well as me and you probably
won't have the confidence to let you trade in the first place.

Another observation. I used to teach as an adjunct professor. Did you ever
have a college professor who was an expert in his field but couldn't teach
at all? Doesn't matter if he knew his stuff, if he couldn't communicate it,
he was worthless. Ever try to read Tom DeMark's stuff? I rest my case.
(sorry Tom) And one of my neighbors teaches the oboe. He's taught some of
the world-class symphony oboeists -- but he really can't play all that well
himself. Sometimes it is true that those who can't do (as well as others)
can teach (better than others)

The best solution is -- as many have pointed out -- to do your own
research. Put enough time to it and you can "internalize" what you need to
know about trading. No matter how much money you spend on a teacher or
course, it can never compare to the confidence of developing your own
methods.

Another solution is finding a mentor but that shouldn't cost anything. I've
found in life that if you are nice to people -- and things click -- you can
usually find a mentor in virtually any field who wouldn't dream of charging
you a nickel.






>Chris, that was a very helpful post.  I would like to emphasize one part of it.
>
>Question # 7: Asking a guru, mentor, or trainer to show you consecutive
>brokerage statements for a "minimum" of a few months (or "at least" 3 months
>of monthly statements showing all trades for those months), preferably
>longer,  is the absolute "best" way of initially screening a trainer.
>
>First determine if they are profitable with their own method or system.
>
>This question should be moved to the top of the list as to save time.  Most
>trading system vendors or teachers (if you actually want to call them that)
>will fall short with this request.
>
>You will find that 99% of the time they will come up with every excuse in
>the book not to show you there own trading statements.  Some may offer to
>show you a spreadsheet or trading log but that could very easily be fabricated!
>
>Would you want to take a pilot's training course from somebody that can't
>fly???
>
>Absolutely not and you should not let anybody's hype related to selling
>their training/system persuade you to part with you money unless they can
>prove to you via actual trading account statements that they are successful
>at trading!
>
>The rest of the questions are great for evaluating whether they have
>something you might want to pursue.  But if they can't pass good old
>question # 7 (re-label it to question # 1), move on and save you money, time
>and effort!
>
>Good trading,
>
>Steve
>stevess@xxxxxxx
>
>
>>Chris Jackson Wrote:
>
>>Here is a checklist i saved from a newsgroup. Credit goes to the author.
>
>>I've evolved an "interview" sieve process for evaluating trading
>>systems, basically my hands won't part with any money until a vendor
>>can answer a majority of these questions directly, honestly, without
>>evasion or obfuscation.
>
>>>>>>>>>
>>7.) Are you willing to show me your personal brokerage trading record
>>(not a computer generated record) of trades using the "System" or
>>"Method"?
>>>>>>>>