[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Trying to store Optimized Variables using AddToComposite



PureBytes Links

Trading Reference Links

--- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@xxx> wrote:
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown see3d@ wrote:
> >
> > The way I keep them straight is to think of IF( condition) as a flow
> > control that will only take a True/False scaler, and IIF as an
> > operator that returns an array or scaler value.
> >
> > I sometimes use IIF(condition, 1, null) as a special type of
boolean.
> >
> > You could write IF( IIF( condition1, condition2, condition3 ) )
> > instead of writing IF( ( condition1 AND condition2 ) OR
> > ( NOT( condition1 ) AND condition3 ) )
> >
> > I prefer the first one for readability of intent.
> >
> > IIF can be very useful with scalers.
> >
> > BR,
> > Dennis
> >
> >
> > On Jan 9, 2009, at 3:16 PM, ozzyapeman wrote:
> >
> > > Mike,
> > >
> > > Thank you yet again! The code works, and your insights are very
> > > helpful.
> > >
> > > Each time I think I've coded something correctly, seems I actually
> > > have a number of basic bugs or sloppy implementation. Getting the
> hang
> > > of when to use array functions (iif) vs scalar (if) can also be
> > > confusing sometimes.
> > >
> > >
> > > --- In amibroker@xxxxxxxxxxxxxxx, "Mike" sfclimbers@ wrote:
> > >>
> > >>
> > >> Ozzy,
> > >>
> > >> You've got a few things going on:
> > >>
> > >> 1. You should not be using IIf in this scenario. Because all
values
> > >> are
> > >> scalers, you should just use a simple if statement.
> > >>
> > >> 2. Since you are initializing WinTest to 0, no WinPct will ever
be
> > >> less
> > >> than it and FinalFast will never be changed from its initial
value
> > >> of 0.
> > >> You must instead initialize WinTest to greater than 100.
> > >>
> > >> 3. You need to do some rudimentary validation on the input data
> > >> before
> > >> blindly trying to work with it. Even after correcting the 2
issues
> > >> above, the last line of your file may be a blank line. This
causes
> > >> StrExtract to return an empty string "" which StrToNum then
> > >> converts to
> > >> a 0. Thus, your lowest %Win would be found to be 0 as would the
> > >> associated fast.
> > >>
> > >> Some additional thoughts:
> > >>
> > >> 1. No need to seperate out the first line read, do it all in the
> same
> > >> while loop.
> > >>
> > >> 2. Move the fclose(fh) inside the if (fh) block.
> > >>
> > >> 3. Good practice to move variables into the smallest scope within
> > >> which
> > >> they will be used.
> > >>
> > >> Have a try with the following:
> > >>
> > >> FinalFast = 0;
> > >>
> > >> fh = fopen( "C:\\logFile2.txt", "r" );
> > >>
> > >> if ( fh )
> > >> {
> > >>     WinTest = 101;
> > >>
> > >>     while ( !feof( fh ) )
> > >>     {
> > >>         Line1 = fgets( fh );
> > >>         WinsPctStr = StrExtract( Line1, 2 );
> > >>
> > >>         if ( WinsPctStr == "" )
> > >>         {
> > >>             continue;
> > >>         }
> > >>
> > >>         WinsPct = StrToNum( WinsPctStr );
> > >>
> > >>         if ( WinsPct < WinTest )
> > >>         {
> > >>             FinalFast = StrToNum( StrExtract( Line1, 0 ) );
> > >>             WinTest = WinsPct;
> > >>         }
> > >>     }
> > >>
> > >>     fclose( fh );
> > >> }
> > >>
> > >> _TRACE( NumToStr( FinalFast ) );
> > >>
> > >>
> > >> Mike
> > >>
> > >>
> > >> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@> wrote:
> > >>>
> > >>> Having a bit of trouble designing a simple parser to extract
> values
> > >> from
> > >>> a csv file. My code only seems to work for some cases, but not
> > >>> others,
> > >>> and I can't seem to figure out why.
> > >>>
> > >>> I have a csv log file with three columns: FastMa, SlowMA,
> %Winners:
> > >>>
> > >>> 11,100,32
> > >>> 22,100,35
> > >>> 31,100,36
> > >>> 43,100,37
> > >>> 52,100,38
> > >>> 23,125,32
> > >>> 32,125,35
> > >>> 44,125,36
> > >>> 53,125,48
> > >>> 12,150,31
> > >>> 25,150,33
> > >>> 34,150,32
> > >>> 46,150,35
> > >>> 54,150,36
> > >>> 26,175,31
> > >>> 36,175,34
> > >>> 47,175,34
> > >>> 55,175,36
> > >>> 27,200,29
> > >>> 37,200,33
> > >>> 48,200,34
> > >>> 57,200,36
> > >>>
> > >>> If I want to extract the value of the FastMA that corresponds to
> the
> > >>> largest %Winner, the following code works correctly and prints
the
> > >> value
> > >>> "53":
> > >>>
> > >>>
> > >>
> > >
>
//----------------------------------------------------------------------\
\
> \
> > >> \
> > >>> --------
> > >>> // PARSER VERSION 1: Find value of FastMA that corresponds to
> > >>> largest
> > >>> %Winners
> > >>>
> > >>
> > >
>
//----------------------------------------------------------------------\
\
> \
> > >> \
> > >>> --------
> > >>>
> > >>>
> > >>> fh = fopen( "C:\\logFile2.txt", "r" );
> > >>>
> > >>> FinalFast = WinsPct = WinTest = fast = slow = 0;
> > >>>
> > >>> if(fh)
> > >>> {
> > >>> Line1 = fgets(fh);
> > >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> > >>> WinTest = WinsPct; // initialize WinTest
> > >>>
> > >>> while( !feof(fh) )
> > >>> {
> > >>> Line1 = fgets(fh);
> > >>>
> > >>> fast = StrToNum( StrExtract(Line1,0) );
> > >>> slow = StrToNum( StrExtract(Line1,1) );
> > >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> > >>>
> > >>> FinalFast = IIf(WinsPct > WinTest, fast, FinalFast);
> > >>>
> > >>> _TRACE("FinalFast"+FinalFast);
> > >>>
> > >>> WinTest = IIf(WinsPct > WinTest, WinsPct, WinTest);
> > >>>
> > >>> _TRACE("WinTest"+WinTest);
> > >>>
> > >>> }
> > >>> }
> > >>> fclose(fh);
> > >>>
> > >>> printf(NumToStr(FinalFast) );
> > >>>
> > >>>
> > >>> But if I want to extract the value of the FastMA that
corresponds
> to
> > >> the
> > >>> smallest %Winners, and simply change all ">" to "<", the code
> should
> > >>> work. But it doesn't. Intead of printing "27", it prints "0".
Why
> > >>> are
> > >>> the values being reset to initialization, when it did not happen
> > >>> with
> > >>> the case above? Below is the code. As mentioned, the only
trivial
> > >>> difference are the inequalities. Otherwise, it's identical to
the
> > >> above.
> > >>>
> > >>> Any feedback much appreciated.
> > >>>
> > >>>
> > >>>
> > >>
> > >
>
//----------------------------------------------------------------------\
\
> \
> > >> \
> > >>> --------
> > >>> // PARSER VERSION 2: Find value of FastMA that corresponds to
> > >>> smallest
> > >>> %Winners
> > >>>
> > >>
> > >
>
//----------------------------------------------------------------------\
\
> \
> > >> \
> > >>> --------
> > >>>
> > >>>
> > >>> fh = fopen( "C:\\logFile2.txt", "r" );
> > >>>
> > >>> FinalFast = WinsPct = WinTest = fast = slow = 0;
> > >>>
> > >>> if(fh)
> > >>> {
> > >>> Line1 = fgets(fh);
> > >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> > >>> WinTest = WinsPct; // initialize WinTest
> > >>>
> > >>> while( !feof(fh) )
> > >>> {
> > >>> Line1 = fgets(fh);
> > >>>
> > >>> fast = StrToNum( StrExtract(Line1,0) );
> > >>> slow = StrToNum( StrExtract(Line1,1) );
> > >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> > >>>
> > >>> FinalFast = IIf(WinsPct < WinTest, fast, FinalFast);
> > >>>
> > >>> _TRACE("FinalFast"+FinalFast);
> > >>>
> > >>> WinTest = IIf(WinsPct < WinTest, WinsPct, WinTest);
> > >>>
> > >>> _TRACE("WinTest"+WinTest);
> > >>>
> > >>> }
> > >>> }
> > >>> fclose(fh);
> > >>>
> > >>> printf(NumToStr(FinalFast) );
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>
> > >>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" zoopfree@ wrote:
> > >>>>
> > >>>> My bad. I had changed the location of the file in one area of
the
> > >> code
> > >>>> and neglected the other.
> > >>>>
> > >>>> It works.
> > >>>>
> > >>>> Thanks a bunch!!
> > >>>>
> > >>>>
> > >>>>
> > >>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" zoopfree@ wrote:
> > >>>>>
> > >>>>> Awesome, Mike. That's exactly the type of thing I was looking
> for.
> > >>>>>
> > >>>>> One problem, though. I try to run your code and only the
column
> > >>>>> titles, "Fast,Slow,PCTWin" get written to the file. The actual
> > >>> values
> > >>>>> are not getting written, despite the fact that my optimization
> > >>> reports
> > >>>>> shows many values corresponding to % winners > 60.
> > >>>>>
> > >>>>> Looking at the trace output indicates the number of triggers
are
> > >>> being
> > >>>>> properly hit. Your code also appears to be quite
> straightforward,
> > >>> and
> > >>>>> I cannot spot any obvious bug.
> > >>>>>
> > >>>>> Did you try running this? Does it work for you? Any idea where
> the
> > >>> bug
> > >>>>> might be?
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
> > >>>>>>
> > >>>>>> AddToComposite is for storing values on a bar by bar basis. I
> > >>> don't
> > >>>>>> think that this would be the right approach for your goals,
> > >> since
> > >>> you
> > >>>>>> are looking for a single scaler value representing total
> > >>>> performance for
> > >>>>>> a given period. Yes, you could write scripts to dig out the
> > >>>> performance
> > >>>>>> metrics and persist the subset of interest. However, it would
> be
> > >>>> easier
> > >>>>>> to just track the interesting ones as they occur. The
following
> > >>>> AFL can
> > >>>>>> be run through the Optimizer and persist in a separate file
the
> > >>>>>> parameter values for only those backtests that were of
> interest.
> > >>> I'll
> > >>>>>> leave the digging out of values in the second AFL to you, as
it
> > >>> sounds
> > >>>>>> like you have an idea already of how you want to do that.
Mike
> > >>>>>> fast = Optimize("Fast", 10, 10, 50, 10); slow =
> Optimize("Slow",
> > >>> 100,
> > >>>>>> 100, 200, 25);
> > >>>>>> Buy = Cross(MA(Close, fast), MA(Close, slow)); Sell =
> > >>> Cross(MA(Close,
> > >>>>>> slow), MA(Close, fast));
> > >>>>>> SetCustomBacktestProc("");
> > >>>>>> if (Status("ActionEx") == actionExOptimizeSetup) {
> > >>>>>> _TRACE("Optimize Begin"); fh =
> > >>>>>> fopen("C:\\temp\\logFile.csv", "w");
> > >>>>>> if (fh) { fputs("Fast,Slow,PCTWin\n", fh);
> > >>>>>> fclose(fh); } }
> > >>>>>> if (Status("action") == actionPortfolio) { _TRACE("Portfolio
> > >>>>>> Backtest"); bo = GetBacktesterObject(); bo.Backtest();
> > >>>>>> stats = bo.GetPerformanceStats(0); pctWinners =
> > >>>>>> stats.getValue("WinnersPercent");
> > >>>>>> if (pctWinners > 60) { _TRACE("Trigger"); fh =
> > >>>>>> fopen("C:\\temp\\logFile.csv", "a");
> > >>>>>> if (fh) {
> > >>>>>> fputs(StrFormat("%f,%f,%f\n", fast, slow, pctWinners),
> > >>> fh);
> > >>>>>> fclose(fh);
> > >>>>>> } }
> > >>>>>> } --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@>
> > >>> wrote:
> > >>>>>>>
> > >>>>>>> Sorry if I was not clear enough. I will try to give more
> > >>> details.
> > >>>>>>>
> > >>>>>>> Basically I want to run an optimization on two variables,
then
> > >>>>>>> somehow, automatically, store all values of optimized
> > >> variables
> > >>> that
> > >>>>>>> correspond to a certain metric range (let's say %Winners >
> > >> 60).
> > >>>>>>> Another AFL would then pull that range of optimized
variables
> > >>> for a
> > >>>>>>> backtest.
> > >>>>>>>
> > >>>>>>> I would use the Walk Forward feature of AmiBroker for this,
> > >>>> except it
> > >>>>>>> only uses the 'best' value, not a range of values. My system
> > >>>> requires
> > >>>>>>> a range of values, not just the best one. I also want to
> > >>> permanently
> > >>>>>>> store the best values.
> > >>>>>>>
> > >>>>>>> I am stuck trying to figure out how to automatically pull
the
> > >>>>>>> optimized variables from the optimization report. The custom
> > >>>>>>> backtester only allows me to pull the built-in metrics.
> > >>>>>>>
> > >>>>>>> So the basic question is - how do I extract the optimized
> > >>> variables
> > >>>>>>> after an optimization is run? Do I have to write some
vbscript
> > >>> that
> > >>>>>>> exports the report to csv, then opens that report, then
> > >> somehow
> > >>>> parses
> > >>>>>>> through that report to find the correct column and range? Or
> > >> is
> > >>>> there
> > >>>>>>> (hopefully) a simpler way of extracting the values?
> > >>>>>>>
> > >>>>>>> Once the values are extracted, it is then fairly easy to
> > >> either
> > >>>> write
> > >>>>>>> them to a file or store them in static variables. But I am
> > >>> aiming to
> > >>>>>>> store them in composite symbols, as (a) they are 'permanent'
> > >>> like
> > >>>>>>> external files and (b) my sense is that it is more efficient
> > >> to
> > >>> pull
> > >>>>>>> values from a composite symbol than from an external file.
> > >>>>>>>
> > >>>>>>> As I run through optimizations across different historical
> > >>>> periods, I
> > >>>>>>> want to build a number of composite symbols that contain the
> > >>> 'best
> > >>>>>>> values' of optimized variables for use in 'walk forward'
> > >>> backtests,
> > >>>>>>> and then eventual live trading.
> > >>>>>>>
> > >>>>>>> Hopefully the above is clearer now. Please let me know if
not.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" sfclimbers@ wrote:
> > >>>>>>>>
> > >>>>>>>> You would have to be a little more clear on exactly what it
> > >> is
> > >>> you
> > >>>>>> are
> > >>>>>>>> trying to accomplish. Though, writing to a file directly,
or
> > >>> using
> > >>>>>>>> static variables might be areas to explore.
> > >>>>>>>>
> > >>>>>>>> Mike
> > >>>>>>>>
> > >>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@>
> > >>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Ah. Well that would explain that. Thanks.
> > >>>>>>>>>
> > >>>>>>>>> Any ideas for a possible workaround?
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@>
> > >>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Ozzy,
> > >>>>>>>>>>
> > >>>>>>>>>> The Stats object only contains values for built in
> > >> metrics
> > >>> as
> > >>>>>>>>>> described here (scroll to bottom):
> > >>>>>>>>>>
> > >>>>>>>>>> http://www.amibroker.com/guide/a_custombacktest.html
> > >>>>>>>>>>
> > >>>>>>>>>> Mike
> > >>>>>>>>>>
> > >>>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman"
> > >> <zoopfree@>
> > >>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hello, I've read Herman's excellent doc, "IntroToATC".
> > >>>>>>>>>>>
> > >>>>>>>>>>> I am trying to run an optimization, and then store the
> > >>>> values
> > >>>>>> of
> > >>>>>>>> the
> > >>>>>>>>>>> optimized variables in some composite symbols. I later
> > >>>> want to
> > >>>>>>>> pull
> > >>>>>>>>>>> values of a certain range and input them automatically
> > >>> in
> > >>>>>>>> another
> > >>>>>>>>>> AFL.
> > >>>>>>>>>>> However, I keep getting a syntax error that the fields
> > >>>> are not
> > >>>>>>>>>>> available, even though they clearly are.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hoping someone can point out my mistake, or give me
> > >> some
> > >>>>>>>>>> suggestions on
> > >>>>>>>>>>> what else to try.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Here is the code. Any ideas? :
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>
> //--------------------------------------------------------------
> > >>>>>>>> ----
> > >>>>>>>>>> --
> > >>>>>>>>>>> // TRADING SYSTEM
> > >>>>>>>>>>>
> > >>>>>>
> //--------------------------------------------------------------
> > >>>>>>>> ----
> > >>>>>>>>>> --
> > >>>>>>>>>>>
> > >>>>>>>>>>> FastMALength = Optimize("FastMALength", 10, 1,
> > >>>> 10,
> > >>>>>>>
> > >>>>>>>>>> 1);
> > >>>>>>>>>>> SlowMALength = Optimize("SlowMALength", 20, 20,
> > >>> 50,
> > >>>>>>>>>> 10);
> > >>>>>>>>>>>
> > >>>>>>>>>>> FastMA = MA( C, FastMALength );
> > >>>>>>>>>>> SlowMA = MA( C, SlowMALength );
> > >>>>>>>>>>> Buy = Cross( FastMA, SlowMA );
> > >>>>>>>>>>> Sell = Cross( SlowMA, FastMA );
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>
> //--------------------------------------------------------------
> > >>>>>>>> ----
> > >>>>>>>>>> --
> > >>>>>>>>>>> // CUSTOM OPTIMIZATION PROCEDURE (Store opt vars in
> > >>>> composite
> > >>>>>>>>>> symbols)
> > >>>>>>>>>>>
> > >>>>>>
> //--------------------------------------------------------------
> > >>>>>>>> ----
> > >>>>>>>>>> --
> > >>>>>>>>>>>
> > >>>>>>>>>>> SetCustomBacktestProc( "" );
> > >>>>>>>>>>>
> > >>>>>>>>>>> if ( Status( "action" ) == actionPortfolio )
> > >>>>>>>>>>> {
> > >>>>>>>>>>> bo = GetBacktesterObject();
> > >>>>>>>>>>>
> > >>>>>>>>>>> // run default backtest procedure
> > >>>>>>>>>>> bo.Backtest( 1 );
> > >>>>>>>>>>>
> > >>>>>>>>>>> st = bo.getperformancestats( 0 );
> > >>>>>>>>>>>
> > >>>>>>>>>>> // iterate through closed trades first
> > >>>>>>>>>>> for ( trade = bo.GetFirstTrade(); trade; trade =
> > >>>>>>>> bo.GetNextTrade
> > >>>>>>>>>> () )
> > >>>>>>>>>>> {
> > >>>>>>>>>>> FastMALength = st.getvalue( "FastMALength" );
> > >>>>>>>>>>> SlowMALength = st.getvalue( "SlowMALength" );
> > >>>>>>>>>>>
> > >>>>>>>>>>> AddToComposite( FastMALength, "~OptFastMA", "X",
> > >>>>>>>> 1+2+8+16+64 );
> > >>>>>>>>>>> AddToComposite( SlowMALength, "~OptSlowMA", "X",
> > >>>>>>>> 1+2+8+16+64 );
> > >>>>>>>>>>> }
> > >>>>>>>>>>> bo.ListTrades();
> > >>>>>>>>>>> }
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> > >
> > >
> > > ------------------------------------
> > >
> > > **** IMPORTANT ****
> > > This group is for the discussion between users only.
> > > This is *NOT* technical support channel.
> > >
> > > *********************
> > > TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail
> > > directly to
> > > SUPPORT {at} amibroker.com
> > > *********************
> > >
> > > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > > http://www.amibroker.com/devlog/
> > >
> > > For other support material please check also:
> > > http://www.amibroker.com/support.html
> > >
> > > *********************************
> > > Yahoo! Groups Links
> > >
> > >
> > >
> >
>



------------------------------------

**** IMPORTANT ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

*********************
TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com
*********************

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html

*********************************
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/