[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [amibroker] Re: Trying to store Optimized Variables using AddToComposite



PureBytes Links

Trading Reference Links

The way I keep them straight is to think of IF( condition) as a flow  
control that will only take a True/False scaler, and IIF as an  
operator that returns an array or scaler value.

I sometimes use IIF(condition, 1, null) as a special type of boolean.

You could write IF( IIF( condition1, condition2, condition3 ) )
instead of writing IF( ( condition1 AND condition2 ) OR  
( NOT( condition1 ) AND condition3 ) )

I prefer the first one for readability of intent.

IIF can be very useful with scalers.

BR,
Dennis


On Jan 9, 2009, at 3:16 PM, ozzyapeman wrote:

> Mike,
>
> Thank you yet again! The code works, and your insights are very  
> helpful.
>
> Each time I think I've coded something correctly, seems I actually
> have a number of basic bugs or sloppy implementation. Getting the hang
> of when to use array functions (iif) vs scalar (if) can also be
> confusing sometimes.
>
>
> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@xxx> wrote:
>>
>>
>> Ozzy,
>>
>> You've got a few things going on:
>>
>> 1. You should not be using IIf in this scenario. Because all values  
>> are
>> scalers, you should just use a simple if statement.
>>
>> 2. Since you are initializing WinTest to 0, no WinPct will ever be  
>> less
>> than it and FinalFast will never be changed from its initial value  
>> of 0.
>> You must instead initialize WinTest to greater than 100.
>>
>> 3. You need to do some rudimentary validation on the input data  
>> before
>> blindly trying to work with it. Even after correcting the 2 issues
>> above, the last line of your file may be a blank line. This causes
>> StrExtract to return an empty string "" which StrToNum then  
>> converts to
>> a 0. Thus, your lowest %Win would be found to be 0 as would the
>> associated fast.
>>
>> Some additional thoughts:
>>
>> 1. No need to seperate out the first line read, do it all in the same
>> while loop.
>>
>> 2. Move the fclose(fh) inside the if (fh) block.
>>
>> 3. Good practice to move variables into the smallest scope within  
>> which
>> they will be used.
>>
>> Have a try with the following:
>>
>> FinalFast = 0;
>>
>> fh = fopen( "C:\\logFile2.txt", "r" );
>>
>> if ( fh )
>> {
>>     WinTest = 101;
>>
>>     while ( !feof( fh ) )
>>     {
>>         Line1 = fgets( fh );
>>         WinsPctStr = StrExtract( Line1, 2 );
>>
>>         if ( WinsPctStr == "" )
>>         {
>>             continue;
>>         }
>>
>>         WinsPct = StrToNum( WinsPctStr );
>>
>>         if ( WinsPct < WinTest )
>>         {
>>             FinalFast = StrToNum( StrExtract( Line1, 0 ) );
>>             WinTest = WinsPct;
>>         }
>>     }
>>
>>     fclose( fh );
>> }
>>
>> _TRACE( NumToStr( FinalFast ) );
>>
>>
>> Mike
>>
>>
>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@> wrote:
>>>
>>> Having a bit of trouble designing a simple parser to extract values
>> from
>>> a csv file. My code only seems to work for some cases, but not  
>>> others,
>>> and I can't seem to figure out why.
>>>
>>> I have a csv log file with three columns: FastMa, SlowMA, %Winners:
>>>
>>> 11,100,32
>>> 22,100,35
>>> 31,100,36
>>> 43,100,37
>>> 52,100,38
>>> 23,125,32
>>> 32,125,35
>>> 44,125,36
>>> 53,125,48
>>> 12,150,31
>>> 25,150,33
>>> 34,150,32
>>> 46,150,35
>>> 54,150,36
>>> 26,175,31
>>> 36,175,34
>>> 47,175,34
>>> 55,175,36
>>> 27,200,29
>>> 37,200,33
>>> 48,200,34
>>> 57,200,36
>>>
>>> If I want to extract the value of the FastMA that corresponds to the
>>> largest %Winner, the following code works correctly and prints the
>> value
>>> "53":
>>>
>>>
>>
> //----------------------------------------------------------------------\
>> \
>>> --------
>>> // PARSER VERSION 1: Find value of FastMA that corresponds to  
>>> largest
>>> %Winners
>>>
>>
> //----------------------------------------------------------------------\
>> \
>>> --------
>>>
>>>
>>> fh = fopen( "C:\\logFile2.txt", "r" );
>>>
>>> FinalFast = WinsPct = WinTest = fast = slow = 0;
>>>
>>> if(fh)
>>> {
>>> Line1 = fgets(fh);
>>> WinsPct = StrToNum( StrExtract(Line1,2) );
>>> WinTest = WinsPct; // initialize WinTest
>>>
>>> while( !feof(fh) )
>>> {
>>> Line1 = fgets(fh);
>>>
>>> fast = StrToNum( StrExtract(Line1,0) );
>>> slow = StrToNum( StrExtract(Line1,1) );
>>> WinsPct = StrToNum( StrExtract(Line1,2) );
>>>
>>> FinalFast = IIf(WinsPct > WinTest, fast, FinalFast);
>>>
>>> _TRACE("FinalFast"+FinalFast);
>>>
>>> WinTest = IIf(WinsPct > WinTest, WinsPct, WinTest);
>>>
>>> _TRACE("WinTest"+WinTest);
>>>
>>> }
>>> }
>>> fclose(fh);
>>>
>>> printf(NumToStr(FinalFast) );
>>>
>>>
>>> But if I want to extract the value of the FastMA that corresponds to
>> the
>>> smallest %Winners, and simply change all ">" to "<", the code should
>>> work. But it doesn't. Intead of printing "27", it prints "0". Why  
>>> are
>>> the values being reset to initialization, when it did not happen  
>>> with
>>> the case above? Below is the code. As mentioned, the only trivial
>>> difference are the inequalities. Otherwise, it's identical to the
>> above.
>>>
>>> Any feedback much appreciated.
>>>
>>>
>>>
>>
> //----------------------------------------------------------------------\
>> \
>>> --------
>>> // PARSER VERSION 2: Find value of FastMA that corresponds to  
>>> smallest
>>> %Winners
>>>
>>
> //----------------------------------------------------------------------\
>> \
>>> --------
>>>
>>>
>>> fh = fopen( "C:\\logFile2.txt", "r" );
>>>
>>> FinalFast = WinsPct = WinTest = fast = slow = 0;
>>>
>>> if(fh)
>>> {
>>> Line1 = fgets(fh);
>>> WinsPct = StrToNum( StrExtract(Line1,2) );
>>> WinTest = WinsPct; // initialize WinTest
>>>
>>> while( !feof(fh) )
>>> {
>>> Line1 = fgets(fh);
>>>
>>> fast = StrToNum( StrExtract(Line1,0) );
>>> slow = StrToNum( StrExtract(Line1,1) );
>>> WinsPct = StrToNum( StrExtract(Line1,2) );
>>>
>>> FinalFast = IIf(WinsPct < WinTest, fast, FinalFast);
>>>
>>> _TRACE("FinalFast"+FinalFast);
>>>
>>> WinTest = IIf(WinsPct < WinTest, WinsPct, WinTest);
>>>
>>> _TRACE("WinTest"+WinTest);
>>>
>>> }
>>> }
>>> fclose(fh);
>>>
>>> printf(NumToStr(FinalFast) );
>>>
>>>
>>>
>>>
>>>
>>>
>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" zoopfree@ wrote:
>>>>
>>>> My bad. I had changed the location of the file in one area of the
>> code
>>>> and neglected the other.
>>>>
>>>> It works.
>>>>
>>>> Thanks a bunch!!
>>>>
>>>>
>>>>
>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" zoopfree@ wrote:
>>>>>
>>>>> Awesome, Mike. That's exactly the type of thing I was looking for.
>>>>>
>>>>> One problem, though. I try to run your code and only the column
>>>>> titles, "Fast,Slow,PCTWin" get written to the file. The actual
>>> values
>>>>> are not getting written, despite the fact that my optimization
>>> reports
>>>>> shows many values corresponding to % winners > 60.
>>>>>
>>>>> Looking at the trace output indicates the number of triggers are
>>> being
>>>>> properly hit. Your code also appears to be quite straightforward,
>>> and
>>>>> I cannot spot any obvious bug.
>>>>>
>>>>> Did you try running this? Does it work for you? Any idea where the
>>> bug
>>>>> might be?
>>>>>
>>>>>
>>>>>
>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@> wrote:
>>>>>>
>>>>>> AddToComposite is for storing values on a bar by bar basis. I
>>> don't
>>>>>> think that this would be the right approach for your goals,
>> since
>>> you
>>>>>> are looking for a single scaler value representing total
>>>> performance for
>>>>>> a given period. Yes, you could write scripts to dig out the
>>>> performance
>>>>>> metrics and persist the subset of interest. However, it would be
>>>> easier
>>>>>> to just track the interesting ones as they occur. The following
>>>> AFL can
>>>>>> be run through the Optimizer and persist in a separate file the
>>>>>> parameter values for only those backtests that were of interest.
>>> I'll
>>>>>> leave the digging out of values in the second AFL to you, as it
>>> sounds
>>>>>> like you have an idea already of how you want to do that. Mike
>>>>>> fast = Optimize("Fast", 10, 10, 50, 10); slow = Optimize("Slow",
>>> 100,
>>>>>> 100, 200, 25);
>>>>>> Buy = Cross(MA(Close, fast), MA(Close, slow)); Sell =
>>> Cross(MA(Close,
>>>>>> slow), MA(Close, fast));
>>>>>> SetCustomBacktestProc("");
>>>>>> if (Status("ActionEx") == actionExOptimizeSetup) {
>>>>>> _TRACE("Optimize Begin"); fh =
>>>>>> fopen("C:\\temp\\logFile.csv", "w");
>>>>>> if (fh) { fputs("Fast,Slow,PCTWin\n", fh);
>>>>>> fclose(fh); } }
>>>>>> if (Status("action") == actionPortfolio) { _TRACE("Portfolio
>>>>>> Backtest"); bo = GetBacktesterObject(); bo.Backtest();
>>>>>> stats = bo.GetPerformanceStats(0); pctWinners =
>>>>>> stats.getValue("WinnersPercent");
>>>>>> if (pctWinners > 60) { _TRACE("Trigger"); fh =
>>>>>> fopen("C:\\temp\\logFile.csv", "a");
>>>>>> if (fh) {
>>>>>> fputs(StrFormat("%f,%f,%f\n", fast, slow, pctWinners),
>>> fh);
>>>>>> fclose(fh);
>>>>>> } }
>>>>>> } --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@>
>>> wrote:
>>>>>>>
>>>>>>> Sorry if I was not clear enough. I will try to give more
>>> details.
>>>>>>>
>>>>>>> Basically I want to run an optimization on two variables, then
>>>>>>> somehow, automatically, store all values of optimized
>> variables
>>> that
>>>>>>> correspond to a certain metric range (let's say %Winners >
>> 60).
>>>>>>> Another AFL would then pull that range of optimized variables
>>> for a
>>>>>>> backtest.
>>>>>>>
>>>>>>> I would use the Walk Forward feature of AmiBroker for this,
>>>> except it
>>>>>>> only uses the 'best' value, not a range of values. My system
>>>> requires
>>>>>>> a range of values, not just the best one. I also want to
>>> permanently
>>>>>>> store the best values.
>>>>>>>
>>>>>>> I am stuck trying to figure out how to automatically pull the
>>>>>>> optimized variables from the optimization report. The custom
>>>>>>> backtester only allows me to pull the built-in metrics.
>>>>>>>
>>>>>>> So the basic question is - how do I extract the optimized
>>> variables
>>>>>>> after an optimization is run? Do I have to write some vbscript
>>> that
>>>>>>> exports the report to csv, then opens that report, then
>> somehow
>>>> parses
>>>>>>> through that report to find the correct column and range? Or
>> is
>>>> there
>>>>>>> (hopefully) a simpler way of extracting the values?
>>>>>>>
>>>>>>> Once the values are extracted, it is then fairly easy to
>> either
>>>> write
>>>>>>> them to a file or store them in static variables. But I am
>>> aiming to
>>>>>>> store them in composite symbols, as (a) they are 'permanent'
>>> like
>>>>>>> external files and (b) my sense is that it is more efficient
>> to
>>> pull
>>>>>>> values from a composite symbol than from an external file.
>>>>>>>
>>>>>>> As I run through optimizations across different historical
>>>> periods, I
>>>>>>> want to build a number of composite symbols that contain the
>>> 'best
>>>>>>> values' of optimized variables for use in 'walk forward'
>>> backtests,
>>>>>>> and then eventual live trading.
>>>>>>>
>>>>>>> Hopefully the above is clearer now. Please let me know if not.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" sfclimbers@ wrote:
>>>>>>>>
>>>>>>>> You would have to be a little more clear on exactly what it
>> is
>>> you
>>>>>> are
>>>>>>>> trying to accomplish. Though, writing to a file directly, or
>>> using
>>>>>>>> static variables might be areas to explore.
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman" <zoopfree@>
>>> wrote:
>>>>>>>>>
>>>>>>>>> Ah. Well that would explain that. Thanks.
>>>>>>>>>
>>>>>>>>> Any ideas for a possible workaround?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "Mike" <sfclimbers@>
>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Ozzy,
>>>>>>>>>>
>>>>>>>>>> The Stats object only contains values for built in
>> metrics
>>> as
>>>>>>>>>> described here (scroll to bottom):
>>>>>>>>>>
>>>>>>>>>> http://www.amibroker.com/guide/a_custombacktest.html
>>>>>>>>>>
>>>>>>>>>> Mike
>>>>>>>>>>
>>>>>>>>>> --- In amibroker@xxxxxxxxxxxxxxx, "ozzyapeman"
>> <zoopfree@>
>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello, I've read Herman's excellent doc, "IntroToATC".
>>>>>>>>>>>
>>>>>>>>>>> I am trying to run an optimization, and then store the
>>>> values
>>>>>> of
>>>>>>>> the
>>>>>>>>>>> optimized variables in some composite symbols. I later
>>>> want to
>>>>>>>> pull
>>>>>>>>>>> values of a certain range and input them automatically
>>> in
>>>>>>>> another
>>>>>>>>>> AFL.
>>>>>>>>>>> However, I keep getting a syntax error that the fields
>>>> are not
>>>>>>>>>>> available, even though they clearly are.
>>>>>>>>>>>
>>>>>>>>>>> Hoping someone can point out my mistake, or give me
>> some
>>>>>>>>>> suggestions on
>>>>>>>>>>> what else to try.
>>>>>>>>>>>
>>>>>>>>>>> Here is the code. Any ideas? :
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> //--------------------------------------------------------------
>>>>>>>> ----
>>>>>>>>>> --
>>>>>>>>>>> // TRADING SYSTEM
>>>>>>>>>>>
>>>>>> //--------------------------------------------------------------
>>>>>>>> ----
>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> FastMALength = Optimize("FastMALength", 10, 1,
>>>> 10,
>>>>>>>
>>>>>>>>>> 1);
>>>>>>>>>>> SlowMALength = Optimize("SlowMALength", 20, 20,
>>> 50,
>>>>>>>>>> 10);
>>>>>>>>>>>
>>>>>>>>>>> FastMA = MA( C, FastMALength );
>>>>>>>>>>> SlowMA = MA( C, SlowMALength );
>>>>>>>>>>> Buy = Cross( FastMA, SlowMA );
>>>>>>>>>>> Sell = Cross( SlowMA, FastMA );
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>> //--------------------------------------------------------------
>>>>>>>> ----
>>>>>>>>>> --
>>>>>>>>>>> // CUSTOM OPTIMIZATION PROCEDURE (Store opt vars in
>>>> composite
>>>>>>>>>> symbols)
>>>>>>>>>>>
>>>>>> //--------------------------------------------------------------
>>>>>>>> ----
>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> SetCustomBacktestProc( "" );
>>>>>>>>>>>
>>>>>>>>>>> if ( Status( "action" ) == actionPortfolio )
>>>>>>>>>>> {
>>>>>>>>>>> bo = GetBacktesterObject();
>>>>>>>>>>>
>>>>>>>>>>> // run default backtest procedure
>>>>>>>>>>> bo.Backtest( 1 );
>>>>>>>>>>>
>>>>>>>>>>> st = bo.getperformancestats( 0 );
>>>>>>>>>>>
>>>>>>>>>>> // iterate through closed trades first
>>>>>>>>>>> for ( trade = bo.GetFirstTrade(); trade; trade =
>>>>>>>> bo.GetNextTrade
>>>>>>>>>> () )
>>>>>>>>>>> {
>>>>>>>>>>> FastMALength = st.getvalue( "FastMALength" );
>>>>>>>>>>> SlowMALength = st.getvalue( "SlowMALength" );
>>>>>>>>>>>
>>>>>>>>>>> AddToComposite( FastMALength, "~OptFastMA", "X",
>>>>>>>> 1+2+8+16+64 );
>>>>>>>>>>> AddToComposite( SlowMALength, "~OptSlowMA", "X",
>>>>>>>> 1+2+8+16+64 );
>>>>>>>>>>> }
>>>>>>>>>>> bo.ListTrades();
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
>
> ------------------------------------
>
> **** IMPORTANT ****
> This group is for the discussion between users only.
> This is *NOT* technical support channel.
>
> *********************
> TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail  
> directly to
> SUPPORT {at} amibroker.com
> *********************
>
> For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> http://www.amibroker.com/devlog/
>
> For other support material please check also:
> http://www.amibroker.com/support.html
>
> *********************************
> Yahoo! Groups Links
>
>
>


------------------------------------

**** IMPORTANT ****
This group is for the discussion between users only.
This is *NOT* technical support channel.

*********************
TO GET TECHNICAL SUPPORT from AmiBroker please send an e-mail directly to 
SUPPORT {at} amibroker.com
*********************

For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/

For other support material please check also:
http://www.amibroker.com/support.html

*********************************
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/amibroker/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/amibroker/join
    (Yahoo! ID required)

<*> To change settings via email:
    mailto:amibroker-digest@xxxxxxxxxxxxxxx 
    mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx

<*> To unsubscribe from this group, send an email to:
    amibroker-unsubscribe@xxxxxxxxxxxxxxx

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/