Re: [amibroker] Re: Questions for people using automatic transactions with IB, AmiBroker Email List Archive

2008/6/23 Louis Préfontaine <rockprog80@xxxxxxxxx>:

Hi Dennis,

Thank you again for your reply and explanations.

First, I was curious about the order limit you talk about. I was always very skeptical about those kind of orders because it is difficult to backtest them. But I think your idea to have a 5-second window is interesting. I was trying to do that and set a 5-minute window in 15-minute bars, and I'd like to use what I learned (thanks to Graham) to work something with 1-minute bars. But so fars 1-minute seems like a tough nut to crack anyway so I don't know if I'll go into AB professional edition yet... Do you think it is worth all the headache? I mean: you HAVE to have it in AT at this speed, right?

About the rollover problem, I will try to do as you said (try to see if newer contract have a longer history), but the problem I see is this one: not only the new ticker is exactly as the February ticker, but the volume and liquidity is also. Maybe what I say is not clear enough; I'll try to explain it the best I can.   When I look at the January contract, it is very liquid with lot of action at the beginning of December, two weeks before the end of the contract. My idea of a rollover is somehow to merge the two contracts so it can stay liquid all the time. But the new ticker is absolutely not liquid at the beginning of December, so this is why I thought the rollover process did not work correctly. The new ticker is exactly as is the February contract ticker: not liquid before the end of the January contract. Not liquid before December 20 or so.

I will try on more recent tickers, but I am skeptical about this. Anyway, if it works that would be wonderful!   Did you learn this kind of code all by yourself in the last two years? I am trying to learn a bit more about AFL; I've read the manual (well, not entirely) and read Howard's book. I hope I could find some way to get better at this.

Thank you,

Louis

2008/6/23 Dennis Brown <see3d@xxxxxxxxxxx>:

Louis,

When you go through the whole process, the rolled contract becomes the new Feb ticker. The Jan ticker is price adjusted to match the Feb ticker on the rollover date and pre-pended to it. The point is to get a longer history than you can get with the newest contract.   The Feb ticker database has been altered at this point, so you can't compare it to itself and see any difference.

Essentially, if you do this process for every contract on the rollover date, you end up with a rolling continuous contract price much like adjusting a stock for splits or dividends. The test is to see if the history goes back all the way to the beginning of the Jan ticker which would be a month longer than the Feb ticker (unless they are both history limited to the same date by the supplier because they are both a ways back in time now).

Try it on some contracts where you are sure that the older contract has a longer history the the new one for a test.

BR,
Dennis

On Jun 23, 2008, at 8:58 PM, Louis Préfontaine wrote:

Hi Dennis,

I experienced some trouble trying to merge the futures with your rollover code.

I followed everything step by step but it didn't work as it was suppose to.

To try the feature, I used Miny Crude Oil January 2008 (QMF8) and February 2008 (QMG8).

// 1. Create a new ticker for the new contract
// 2. Fill in the information section of the new ticker
// 3. Backfill the new ticker with as much data as available

I created both and backfilled both, and filled the information section.

// 4. Run this AFL on the old contract ticker and backfill it if needed

I inserted the chart on the old contract (January).

// 5. Turn off all filtering in the database settings
// 6. Set the chart timeframe to match the database base timeframe

(Note sure what those mean, but I am on 24 hours trading and in 1-minute bars)

// 7. Fill in the parameter for the NEW contract ticker
// 8. Select the rollover date desired

I clicked on "parameters" on the February ticker and chose the rollover date to be December 17, 2007.

// 9. Verify higher volume for new contract on that date in the chart
//10. Click the parameter to write out the new merged contract text file

Done.

//11. Use Import Wizard to import the data into the database

Done.
//12. If you mess up, delete the new ticker from the database and start over
//

The result is a file which is exactly the same as the February contract. I ran the February contract and just below the new file, and both were absolutely the same, even before the December 2007 rollover.

What do you think I could have done wrong?

Thanks,

Louis
2008/6/23 Louis Préfontaine <rockprog80@xxxxxxxxx>:

Dennis,

After looking carefully at your AFL code, I can say only two words: it rocks! Really, that is what I was looking for.   I am trying to make it work for 1-minute data, but I wonder: what is the best rollover date?

You seem to have some experience with futures. Would you say it is better to trade under 1-minute for the, or 1-minute is ok? Cause of course if you trade 5-seconds, you absolutely need to set it automatically to your broker. I can't imagine someone standing there all day long and trying to fill a trade in 5 seconds... You say you use it to anticipate, but isn't 1 -minute doing it the same way?

Thanks,

Louis

2008/6/23 Dennis Brown <see3d@xxxxxxxxxxx>:

Louis,

I handle rollover via an AFL chart I wrote last week to make the problem simple for me. I uploaded it to the AFL library. Some data suppliers provide a Continuous Contract ticker for futures. However, unless they give you as much history as you want for backtesting, it will be a problem. That is why I had to write my own CC rollover. I want months of 5 second data, and DTN only provides a week of history. However, I must make sure I do not let a week go by without updating my database of all futures contracts I want to save. I also have to collect data for months before I have a history available.

If you only want 1 minute bars, many offer a lot longer history. I could not get good volume or range bar charts with more than 5 second database. Tick is better and I will experiment with that at some future date. However, 1 minute is the shortest chart timeframe I run for the time bars.

I do not have an IB account. I will not run auto-trading for quite a while into the future if ever. Right now, AB just tells me (audio and on chart) when to anticipate an order so I can set it up in my broker account screen, then tells me when to send the order (if it gets triggered).

I still want to have final say so on any order sent, because I am aware of more things than the program, like economic announcements, breaking news events, patterns I have seen before, but not programmed into the charts, etc.

I will take things one step at a time. First thing is to have a backtested edge that is not over fitting the data.

Next is to test trade it with a small position and discover how human emotions play into execution --very important!

I am at the third stage of cleaning up my trading platform to make it easier on myself. The computer is an extension of my brain, and good at nagging me about opportunities I might miss because of too many bits of information at the same time.

I am also working on expanding my AFL to ever complex systems that take over more functions that I do mentally with pattern recognition.

I have been working on this system for two years already. I have come a long way with my AFL skills. I think I have started over twice already as I understood the problems better and how to organize the system to make it more general and easier for me to maintain. It is a never ending evolutionary process.

Next, I will start having the AFL start filling out the fields on my Broker Interface with things like the position size, limit price, etc. All I have to do is press the send button at the right time. Then, I will have AFL also press the send button, after I pre-approve the trade setup. I will always maintain an override on trading.

Eventually, I may sign up with a specialized Futures broker and use their data feed and API to generate the trades. It all just depends on how things progress.

You see, one step at a time. If you can't make money the simple manual way, then no point in getting fancy with programming fast ways to lose money. Of course I am referring to Day Trading, not longer term swing trading or investing where the execution timing is not so critical. If you want to make 5 or more trades a day, one second delay in execution can make the difference between a good profit or a loss overall.

BR,
Dennis

On Jun 23, 2008, at 4:05 PM, Louis Préfontaine wrote:

Thanks Dennis for your reply. I have traveled a long path since January with AB, but now I feel like this is a very long way too.

If there are a lot of errors on stocks, I might go with futures in the future. However, I am kind of impressed by the rollover problem and how to handle the end date for each contract. I am not sure how to deal with this.

On a more practical level, do you know if there is any information on how to set AB to run automatic trading on a demo account on IB? I searched but didn't find out how. I am really starting from scratch here.

Thanks,

Louis

2008/6/23 Dennis Brown <see3d@xxxxxxxxxxx>:

Louis,

It is too complicated for me also. I try to simplify everything I can, because there are so many complicated things to deal with. So I went to Futures trading which simulates very close to actual executions if you build your trading algorithms carefully. Once you are aware of the potential problems, it is easier to avoid getting trapped by them. For a final test, I trade it live for a couple of days and compare results realtime from the backtest (which runs live in indicator mode on my system) and what my broker account is doing. I limit my backtesting to a couple of days when I am trading realtime to keep things running fast. I tweak my trading algorithms until my backtests perform a little worse than my real trading. That way I have a bit of a cushion for the inevitable failures due to data feed problems, computer system malfunctions, and human error.

BR,
Dennis

On Jun 23, 2008, at 2:56 PM, Louis Préfontaine wrote:

OMG that seems very complicated. How do you do to do that?

Louis

2008/6/23 Dennis Brown <see3d@xxxxxxxxxxx>:

Something to consider is that stocks have a large number of bad tick data. Suppliers are trying to correct these as quickly as possible. Futures data has very few bad ticks. That is the main reason I switched to only trading e-mini futures instead of stocks for many trades per day, and why I limit my trading to just one future at a time.

To backtest a HF system, you actually need two tickers, the raw data ticker and the corrected data ticker. You generate trading signals from the raw data, and execute the order prices on the corrected data. And don't forget to take into account the time lag between the raw data showing up, and the time of execution on the corrected data.

BR,
Dennis

On Jun 23, 2008, at 1:37 PM, Louis Préfontaine wrote:

Hi Herman,

I read this one http://www.amibroker.org/userkb/2007/12/24/high-frequency-automated-trading-hfat-part-2/ but it seemed to me like it was a bit complicated to get an uncorected database. I mean, having to manually check where backfill ends and raw begins, saving two different databases, etc. Seems like a lot of work... and doing it everyday! Isn't there any easier way?

Do you think this is an issue for 1-minute intraday, or only for tick, or 5 seconds? Because it does not make sense to waste a lot of time on corrected 1-minute data only to see it fails miserabily with raw data!

Thanks,

Louis

2008/6/23 Herman <psytek@xxxxxxxx>:

I remember at least one case where I developed a HG system using historical 1-Minute bars that didn't work with real-time (uncorrected) data.

Any system that trades frequently intraday can easily be verified by running it for a few days with life data. Also, the 2nd article on the UKB tell you how to save uncorrected (raw) data for later comparison with backfilled data.

Of course different systems have different sensitivities to qualities. You can try and design so that your system is insensitive to the problems outlined on the UKB.

best regards,
herman

For tips on developing Real-Time Auto-Trading systems visit:
http://www.amibroker.org/userkb/

Monday, June 23, 2008, 10:04:25 AM, you wrote:

>

Hi,

Well, I'm not really sure but it seems that one-minute bars have so many trades (assuming your contract is at least somewhat liquid) that these errors should pretty much cancel out and stop being a problem.

----- Original Message -----
From: Louis Préfontaine

To: amibroker@xxxxxxxxxxxxxxx
Sent: Monday, June 23, 2008 9:40 AM
Subject: Re: [amibroker] Re: Questions for people using automatic transactions with IB

Hi,

I wonder if the problems with HFAT can happen in « normal » trading. I mean: if I trade with 1-minute bars, I will build a system with data that has been revised at the end of the day. E.g. the data errors will not be there anymore. However, in real-time trading, those errors will be there.. So how can one deal with this, even without automatic trading?

Louis

2008/6/23 reinsley <reinsley@xxxxxxxx>:

thank you, I discovered the use of static var on that occasion and now
this point is much better.

Too many things are missing in my formula, I can't ask help as it
could seem that i'am asking for a ready-made solution. I have to keep
my cards when I'll be trapped.

I need to scambled for the archive first.

Regards