|
PureBytes Links
Trading Reference Links
|
V.K. wrote:
>
> Well Mark,
>
> this is getting really long, I can not see any hindsight if I develop a
> system today that is changing the parameters every 12 month. If i store the
> result for this month and publish it and change/or do not change the
> rules/parameters in 12 month and publish the new results then that is
> totally legal to me! If you understand what i mean!!!!
> Simulated real time numbers or results is what i call the above!
>
Test results that cover an "out of sample" (after tweaking) period are
supposedly more meaningful bec. you havent fitted the system to those
prices. It's intended to be a simulation to see if the system will hold
up in the "real" out of sample test - the future.
Therefore readers of FT would like to see results that occur after the
system was fixed in stone and submitted to FT. It's been a while since
I looked at an issue, but I think the master performance tables only
list one year anyway. The real useful info is the detailed reports on a
specific system. These show data from two periods - before and after
"release" which I believe is the date the system was submitted to FT.
The before release may have been curve fit by the system author. The
implication of the "post release" period is that this is a period of
testing *without adjustments being made*. If adjustments were permitted
at all to a system that was tanking, then at a minimum the release date
should be revised to reflect that. And presumably this would make the
vendor think twice about tinkering since a longer post-release period
should impress people more.
The question then arises, if a vendor does a small improvement to the
system, what happens. Supposedly he cant' change the FT version. In
some cases I bet the vendor sells the better system but FT is testing
the older. If I was a purchaser what I would want is *both* versions.
Then i could chose between the more tested version or the "better"
version. I wonder if this isn't the source of confusion. Someone may
have seen a different version of a system yet FT is still testing the
earlier version in accordance with their rules? Now you could complain
a bit if the vendor is giving results of the newer version in an ad,
along with stating that it's tested by FT. At least in their literature
they should explain this.
I emailed FT and asked for an explanation of their procedures.
Conrad Bowers
> -----Ursprüngliche Nachricht-----
> Von: Mark Brown [mailto:markbrown@xxxxxxxxxxxxx]
> Gesendet: Sonntag, 29. August 1999 17:51
> An: Omega List
> Betreff: Re: AW: Wanted EUREX daytraders with TS4 (eurex trading system)
>
> Well,i guess i have to quote from page 3 of the last issue of the "Master
> Performance Table":
>
> It says:
>
> "Futures Truth does not permit any "tweaking" of numbers in our reports.
>
> ***that sounds like a disclaimer that clears them from responsibility of
> vendors wanting to change their results after the fact. it does not cover
> the before publication tweaking that takes place.
>
> Not one number in out report is with the benefit of hindsight, with the one
> exception on the detailed reports (that is because of understandable
> reasons, so to speak that no software so far does what i believe is the only
> real testing method, not even yours Mark, but i am willing to try your
> software). The only numbers we show are after the vendor' s most recent
> modification. That is clearly pointed out." End of quote.
> I feel that is fair, and should be clear.
>
> ***i have some news for you and futures truth - EVERYTHING THAT THEY
> PUBLISH - has the benefit of HINDSIGHT! and when hindsight doesn't suit the
> vendors needs any longer then they change the rules and ignore previous
> hindsight and on to the next one.
>
> In easy english that means, the numbers in the "Master Performance Table"
> are the real-time simulated numbers.
>
> ***what the heck is a simulated REAL TIME numbers? either its real time or
> its not.
>
|