[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[amibroker] Re: Trying to store Optimized Variables using AddToComposite



PureBytes Links

Trading Reference Links

--- In amibroker@xxxxxxxxxxxxxxx, Dennis Brown <see3d@xxx> wrote:
>
> The way I keep them straight is to think of IF( condition) as a flow
> control that will only take a True/False scaler, and IIF as an
> operator that returns an array or scaler value.
>
> I sometimes use IIF(condition, 1, null) as a special type of boolean.
>
> You could write IF( IIF( condition1, condition2, condition3 ) )
> instead of writing IF( ( condition1 AND condition2 ) OR
> ( NOT( condition1 ) AND condition3 ) )
>
> I prefer the first one for readability of intent.
>
> IIF can be very useful with scalers.
>
> BR,
> Dennis
>
>
> On Jan 9, 2009, at 3:16 PM, ozzyapeman wrote:
>
> > Mike,
> >
> > Thank you yet again! The code works, and your insights are very
> > helpful.
> >
> > Each time I think I've coded something correctly, seems I actually
> > have a number of basic bugs or sloppy implementation. Getting the
hang
> > of when to use array functions (iif) vs scalar (if) can also be
> > confusing sometimes.
> >
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Mike" sfclimbers@ wrote:
> >>
> >>
> >> Ozzy,
> >>
> >> You've got a few things going on:
> >>
> >> 1. You should not be using IIf in this scenario. Because all values
> >> are
> >> scalers, you should just use a simple if statement.
> >>
> >> 2. Since you are initializing WinTest to 0, no WinPct will ever be
> >> less
> >> than it and FinalFast will never be changed from its initial value
> >> of 0.
> >> You must instead initialize WinTest to greater than 100.
> >>
> >> 3. You need to do some rudimentary validation on the input data
> >> before
> >> blindly trying to work with it. Even after correcting the 2 issues
> >> above, the last line of your file may be a blank line. This causes
> >> StrExtract to return an empty string "" which StrToNum then
> >> converts to
> >> a 0. Thus, your lowest %Win would be found to be 0 as would the
> >> associated fast.
> >>
> >> Some additional thoughts:
> >>
> >> 1. No need to seperate out the first line read, do it all in the
same
> >> while loop.
> >>
> >> 2. Move the fclose(fh) inside the if (fh) block.
> >>
> >> 3. Good practice to move variables into the smallest scope within
> >> which
> >> they will be used.
> >>
> >> Have a try with the following:
> >>
> >> FinalFast = 0;
> >>
> >> fh = fopen( "C:\\logFile2.txt", "r" );
> >>
> >> if ( fh )
> >> {
> >>     WinTest = 101;
> >>
> >>     while ( !feof( fh ) )
> >>     {
> >>         Line1 = fgets( fh );
> >>         WinsPctStr = StrExtract( Line1, 2 );
> >>
> >>         if ( WinsPctStr == "" )
> >>         {
> >>             continue;
> >>         }
> >>
> >>         WinsPct = StrToNum( WinsPctStr );
> >>
> >>         if ( WinsPct < WinTest )
> >>         {
> >>             FinalFast = StrToNum( StrExtract( Line1, 0 ) );
> >>             WinTest = WinsPct;
> >>         }
> >>     }
> >>
> >>     fclose( fh );
> >> }
> >>
> >> _TRACE( NumToStr( FinalFast ) );
> >>
> >>
> >> Mike
> >>
> >>
> >> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@> wrote:
> >>>
> >>> Having a bit of trouble designing a simple parser to extract
values
> >> from
> >>> a csv file. My code only seems to work for some cases, but not
> >>> others,
> >>> and I can't seem to figure out why.
> >>>
> >>> I have a csv log file with three columns: FastMa, SlowMA,
%Winners:
> >>>
> >>> 11,100,32
> >>> 22,100,35
> >>> 31,100,36
> >>> 43,100,37
> >>> 52,100,38
> >>> 23,125,32
> >>> 32,125,35
> >>> 44,125,36
> >>> 53,125,48
> >>> 12,150,31
> >>> 25,150,33
> >>> 34,150,32
> >>> 46,150,35
> >>> 54,150,36
> >>> 26,175,31
> >>> 36,175,34
> >>> 47,175,34
> >>> 55,175,36
> >>> 27,200,29
> >>> 37,200,33
> >>> 48,200,34
> >>> 57,200,36
> >>>
> >>> If I want to extract the value of the FastMA that corresponds to
the
> >>> largest %Winner, the following code works correctly and prints the
> >> value
> >>> "53":
> >>>
> >>>
> >>
> >
//----------------------------------------------------------------------\
\
> >> \
> >>> --------
> >>> // PARSER VERSION 1: Find value of FastMA that corresponds to
> >>> largest
> >>> %Winners
> >>>
> >>
> >
//----------------------------------------------------------------------\
\
> >> \
> >>> --------
> >>>
> >>>
> >>> fh = fopen( "C:\\logFile2.txt", "r" );
> >>>
> >>> FinalFast = WinsPct = WinTest = fast = slow = 0;
> >>>
> >>> if(fh)
> >>> {
> >>> Line1 = fgets(fh);
> >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> >>> WinTest = WinsPct; // initialize WinTest
> >>>
> >>> while( !feof(fh) )
> >>> {
> >>> Line1 = fgets(fh);
> >>>
> >>> fast = StrToNum( StrExtract(Line1,0) );
> >>> slow = StrToNum( StrExtract(Line1,1) );
> >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> >>>
> >>> FinalFast = IIf(WinsPct > WinTest, fast, FinalFast);
> >>>
> >>> _TRACE("FinalFast"+FinalFast);
> >>>
> >>> WinTest = IIf(WinsPct > WinTest, WinsPct, WinTest);
> >>>
> >>> _TRACE("WinTest"+WinTest);
> >>>
> >>> }
> >>> }
> >>> fclose(fh);
> >>>
> >>> printf(NumToStr(FinalFast) );
> >>>
> >>>
> >>> But if I want to extract the value of the FastMA that corresponds
to
> >> the
> >>> smallest %Winners, and simply change all ">" to "<", the code
should
> >>> work. But it doesn't. Intead of printing "27", it prints "0". Why
> >>> are
> >>> the values being reset to initialization, when it did not happen
> >>> with
> >>> the case above? Below is the code. As mentioned, the only trivial
> >>> difference are the inequalities. Otherwise, it's identical to the
> >> above.
> >>>
> >>> Any feedback much appreciated.
> >>>
> >>>
> >>>
> >>
> >
//----------------------------------------------------------------------\
\
> >> \
> >>> --------
> >>> // PARSER VERSION 2: Find value of FastMA that corresponds to
> >>> smallest
> >>> %Winners
> >>>
> >>
> >
//----------------------------------------------------------------------\
\
> >> \
> >>> --------
> >>>
> >>>
> >>> fh = fopen( "C:\\logFile2.txt", "r" );
> >>>
> >>> FinalFast = WinsPct = WinTest = fast = slow = 0;
> >>>
> >>> if(fh)
> >>> {
> >>> Line1 = fgets(fh);
> >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> >>> WinTest = WinsPct; // initialize WinTest
> >>>
> >>> while( !feof(fh) )
> >>> {
> >>> Line1 = fgets(fh);
> >>>
> >>> fast = StrToNum( StrExtract(Line1,0) );
> >>> slow = StrToNum( StrExtract(Line1,1) );
> >>> WinsPct = StrToNum( StrExtract(Line1,2) );
> >>>
> >>> FinalFast = IIf(WinsPct < WinTest, fast, FinalFast);
> >>>
> >>> _TRACE("FinalFast"+FinalFast);
> >>>
> >>> WinTest = IIf(WinsPct < WinTest, WinsPct, WinTest);
> >>>
> >>> _TRACE("WinTest"+WinTest);
> >>>
> >>> }
> >>> }
> >>> fclose(fh);
> >>>
> >>> printf(NumToStr(FinalFast) );
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" zoopfree@ wrote:
> >>>>
> >>>> My bad. I had changed the location of the file in one area of the
> >> code
> >>>> and neglected the other.
> >>>>
> >>>> It works.
> >>>>
> >>>> Thanks a bunch!!
> >>>>
> >>>>
> >>>>
> >>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" zoopfree@ wrote:
> >>>>>
> >>>>> Awesome, Mike. That's exactly the type of thing I was looking
for.
> >>>>>
> >>>>> One problem, though. I try to run your code and only the column
> >>>>> titles, "Fast,Slow,PCTWin" get written to the file. The actual
> >>> values
> >>>>> are not getting written, despite the fact that my optimization
> >>> reports
> >>>>> shows many values corresponding to % winners > 60.
> >>>>>
> >>>>> Looking at the trace output indicates the number of triggers are
> >>> being
> >>>>> properly hit. Your code also appears to be quite
straightforward,
> >>> and
> >>>>> I cannot spot any obvious bug.
> >>>>>
> >>>>> Did you try running this? Does it work for you? Any idea where
the
> >>> bug
> >>>>> might be?
> >>>>>
> >>>>>
> >>>>>
> >>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
> >>>>>>
> >>>>>> AddToComposite is for storing values on a bar by bar basis. I
> >>> don't
> >>>>>> think that this would be the right approach for your goals,
> >> since
> >>> you
> >>>>>> are looking for a single scaler value representing total
> >>>> performance for
> >>>>>> a given period. Yes, you could write scripts to dig out the
> >>>> performance
> >>>>>> metrics and persist the subset of interest. However, it would
be
> >>>> easier
> >>>>>> to just track the interesting ones as they occur. The following
> >>>> AFL can
> >>>>>> be run through the Optimizer and persist in a separate file the
> >>>>>> parameter values for only those backtests that were of
interest.
> >>> I'll
> >>>>>> leave the digging out of values in the second AFL to you, as it
> >>> sounds
> >>>>>> like you have an idea already of how you want to do that. Mike
> >>>>>> fast = Optimize("Fast", 10, 10, 50, 10); slow =
Optimize("Slow",
> >>> 100,
> >>>>>> 100, 200, 25);
> >>>>>> Buy = Cross(MA(Close, fast), MA(Close, slow)); Sell =
> >>> Cross(MA(Close,
> >>>>>> slow), MA(Close, fast));
> >>>>>> SetCustomBacktestProc("");
> >>>>>> if (Status("ActionEx") == actionExOptimizeSetup) {
> >>>>>> _TRACE("Optimize Begin"); fh =
> >>>>>> fopen("C:\\temp\\logFile.csv", "w");
> >>>>>> if (fh) { fputs("Fast,Slow,PCTWin\n", fh);
> >>>>>> fclose(fh); } }
> >>>>>> if (Status("action") == actionPortfolio) { _TRACE("Portfolio
> >>>>>> Backtest"); bo = GetBacktesterObject(); bo.Backtest();
> >>>>>> stats = bo.GetPerformanceStats(0); pctWinners =
> >>>>>> stats.getValue("WinnersPercent");
> >>>>>> if (pctWinners > 60) { _TRACE("Trigger"); fh =
> >>>>>> fopen("C:\\temp\\logFile.csv", "a");
> >>>>>> if (fh) {
> >>>>>> fputs(StrFormat("%f,%f,%f\n", fast, slow, pctWinners),
> >>> fh);
> >>>>>> fclose(fh);
> >>>>>> } }
> >>>>>> } --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@>
> >>> wrote:
> >>>>>>>
> >>>>>>> Sorry if I was not clear enough. I will try to give more
> >>> details.
> >>>>>>>
> >>>>>>> Basically I want to run an optimization on two variables, then
> >>>>>>> somehow, automatically, store all values of optimized
> >> variables
> >>> that
> >>>>>>> correspond to a certain metric range (let's say %Winners >
> >> 60).
> >>>>>>> Another AFL would then pull that range of optimized variables
> >>> for a
> >>>>>>> backtest.
> >>>>>>>
> >>>>>>> I would use the Walk Forward feature of AmiBroker for this,
> >>>> except it
> >>>>>>> only uses the 'best' value, not a range of values. My system
> >>>> requires
> >>>>>>> a range of values, not just the best one. I also want to
> >>> permanently
> >>>>>>> store the best values.
> >>>>>>>
> >>>>>>> I am stuck trying to figure out how to automatically pull the
> >>>>>>> optimized variables from the optimization report. The custom
> >>>>>>> backtester only allows me to pull the built-in metrics.
> >>>>>>>
> >>>>>>> So the basic question is - how do I extract the optimized
> >>> variables
> >>>>>>> after an optimization is run? Do I have to write some vbscript
> >>> that
> >>>>>>> exports the report to csv, then opens that report, then
> >> somehow
> >>>> parses
> >>>>>>> through that report to find the correct column and range? Or
> >> is
> >>>> there
> >>>>>>> (hopefully) a simpler way of extracting the values?
> >>>>>>>
> >>>>>>> Once the values are extracted, it is then fairly easy to
> >> either
> >>>> write
> >>>>>>> them to a file or store them in static variables. But I am
> >>> aiming to
> >>>>>>> store them in composite symbols, as (a) they are 'permanent'
> >>> like
> >>>>>>> external files and (b) my sense is that it is more efficient
> >> to
> >>> pull
> >>>>>>> values from a composite symbol than from an external file.
> >>>>>>>
> >>>>>>> As I run through optimizations across different historical
> >>>> periods, I
> >>>>>>> want to build a number of composite symbols that contain the
> >>> 'best
> >>>>>>> values' of optimized variables for use in 'walk forward'
> >>> backtests,
> >>>>>>> and then eventual live trading.
> >>>>>>>
> >>>>>>> Hopefully the above is clearer now. Please let me know if not.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" sfclimbers@ wrote:
> >>>>>>>>
> >>>>>>>> You would have to be a little more clear on exactly what it
> >> is
> >>> you
> >>>>>> are
> >>>>>>>> trying to accomplish. Though, writing to a file directly, or
> >>> using
> >>>>>>>> static variables might be areas to explore.
> >>>>>>>>
> >>>>>>>> Mike
> >>>>>>>>
> >>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@>
> >>> wrote:
> >>>>>>>>>
> >>>>>>>>> Ah. Well that would explain that. Thanks.
> >>>>>>>>>
> >>>>>>>>> Any ideas for a possible workaround?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@>
> >>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Ozzy,
> >>>>>>>>>>
> >>>>>>>>>> The Stats object only contains values for built in
> >> metrics
> >>> as
> >>>>>>>>>> described here (scroll to bottom):
> >>>>>>>>>>
> >>>>>>>>>> http://www.amibroker.com/guide/a_custombacktest.html
> >>>>>>>>>>
> >>>>>>>>>> Mike
> >>>>>>>>>>
> >>>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman"
> >> <zoopfree@>
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hello, I've read Herman's excellent doc, "IntroToATC".
> >>>>>>>>>>>
> >>>>>>>>>>> I am trying to run an optimization, and then store the
> >>>> values
> >>>>>> of
> >>>>>>>> the
> >>>>>>>>>>> optimized variables in some composite symbols. I later
> >>>> want to
> >>>>>>>> pull
> >>>>>>>>>>> values of a certain range and input them automatically
> >>> in
> >>>>>>>> another
> >>>>>>>>>> AFL.
> >>>>>>>>>>> However, I keep getting a syntax error that the fields
> >>>> are not
> >>>>>>>>>>> available, even though they clearly are.
> >>>>>>>>>>>
> >>>>>>>>>>> Hoping someone can point out my mistake, or give me
> >> some
> >>>>>>>>>> suggestions on
> >>>>>>>>>>> what else to try.
> >>>>>>>>>>>
> >>>>>>>>>>> Here is the code. Any ideas? :
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
//--------------------------------------------------------------
> >>>>>>>> ----
> >>>>>>>>>> --
> >>>>>>>>>>> // TRADING SYSTEM
> >>>>>>>>>>>
> >>>>>>
//--------------------------------------------------------------
> >>>>>>>> ----
> >>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>> FastMALength = Optimize("FastMALength", 10, 1,
> >>>> 10,
> >>>>>>>
> >>>>>>>>>> 1);
> >>>>>>>>>>> SlowMALength = Optimize("SlowMALength", 20, 20,
> >>> 50,
> >>>>>>>>>> 10);
> >>>>>>>>>>>
> >>>>>>>>>>> FastMA = MA( C, FastMALength );
> >>>>>>>>>>> SlowMA = MA( C, SlowMALength );
> >>>>>>>>>>> Buy = Cross( FastMA, SlowMA );
> >>>>>>>>>>> Sell = Cross( SlowMA, FastMA );
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
//--------------------------------------------------------------
> >>>>>>>> ----
> >>>>>>>>>> --
> >>>>>>>>>>> // CUSTOM OPTIMIZATION PROCEDURE (Store opt vars in
> >>>> composite
> >>>>>>>>>> symbols)
> >>>>>>>>>>>
> >>>>>>
//--------------------------------------------------------------
> >>>>>>>> ----
> >>>>>>>>>> --
> >>>>>>>>>>>
> >>>>>>>>>>> SetCustomBacktestProc( "" );
> >>>>>>>>>>>
> >>>>>>>>>>> if ( Status( "action" ) == actionPortfolio )
> >>>>>>>>>>> {
> >>>>>>>>>>> bo = GetBacktesterObject();
> >>>>>>>>>>>
> >>>>>>>>>>> // run default backtest procedure
> >>>>>>>>>>> bo.Backtest( 1 );
> >>>>>>>>>>>
> >>>>>>>>>>> st = bo.getperformancestats( 0 );
> >>>>>>>>>>>
> >>>>>>>>>>> // iterate through closed trades first
> >>>>>>>>>>> for ( trade = bo.GetFirstTrade(); trade; trade =
> >>>>>>>> bo.GetNextTrade
> >>>>>>>>>> () )
> >>>>>>>>>>> {
> >>>>>>>>>>> FastMALength = st.getvalue( "FastMALength" );
> >>>>>>>>>>> SlowMALength = st.getvalue( "SlowMALength" );
> >>>>>>>>>>>
> >>>>>>>>>>> AddToComposite( FastMALength, "~OptFastMA", "X",
> >>>>>>>> 1+2+8+16+64 );
> >>>>>>>>>>> AddToComposite( SlowMALength, "~OptSlowMA", "X",
> >>>>>>>> 1+2+8+16+64 );
> >>>>>>>>>>> }
> >>>>>>>>>>> bo.ListTrades();
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
> >
> >
> > ------------------------------------
> >
> > **** IMPORTANT ****
> > This group is for the discussion between users only.
> > This is *NOT* technical support channel.
> >
> > *********************
> > TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail
> > directly to
> > SUPPORT {at} amibroker.com
> > *********************
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> >
> > *********************************
> > Yahoo! Groups Links
> >
> >
> >
>



------------------------------------

**** IMPORTANT ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

*********************
TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com
*********************

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html

*********************************
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/