|
PureBytes Links
Trading Reference Links
|
Tomasz,
I think that when "Import Files" is sued, it sorts my selection (I
select an entire years worth of data files at once). My file names
are like this: "nasdaq_20010201.txt" (that is with yyyymmdd in the
name). Is AmiBroker sorting those file names? or does it import in
the order that was selected?
--- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@xxx>
wrote:
>
> Hello,
>
> 5 years of data on my end imports in *minutes* not hours.
>
> a) you have small size of "in-memory" cache so it requires a lot
of disk mangling
> b) the data you import are not sorted or sorted improperly (in
reverse order) and AB needs to re-sort.
>
> For fast import It is advised to use data that is sorted in OLDEST
QUOTES FIRST - NEWEST LAST
> and to have one file per symbol.
>
> The worst possible case (it's probably yours) is having sorted in
opposite direction
> (newest first) and different symbol in every line.
>
> Best regards,
> Tomasz Janeczko
> amibroker.com
> ----- Original Message -----
> From: "jacklweinberg" <jacklweinberg@xxx>
> To: <amibroker@xxxxxxxxxxxxxxx>
> Sent: Tuesday, May 15, 2007 2:59 PM
> Subject: [amibroker] Re: Massive data loading?
>
>
> > Tomasz,
> > Perhaps my original message wasn't clear.
> > I have the data in txt files, and am using AmiBroker "Import
Files"
> > to load them into the database. That is the process that is
taking
> > over 32 hours!
> >
> > --- In amibroker@xxxxxxxxxxxxxxx, "Tomasz Janeczko" <groups@>
> > wrote:
> >>
> >> Hello,
> >>
> >> It is MUCH quicker if you change the settings.
> >> Default settings are fail-safe. You can however
> >> change "number of simultaneous downloads" to 5 or more.
> >> BY setting it say to 10 you can get 10x speed up,
> >> however Yahoo may block you after some excessive downloading
> >> and you will need to revert to defaults and wait a couple of
> > minutes
> >> for Yahoo to unblock you.
> >>
> >> Best regards,
> >> Tomasz Janeczko
> >> amibroker.com
> >> ----- Original Message -----
> >> From: "jacklweinberg" <jacklweinberg@>
> >> To: <amibroker@xxxxxxxxxxxxxxx>
> >> Sent: Monday, May 14, 2007 7:33 PM
> >> Subject: [amibroker] Massive data loading?
> >>
> >>
> >> > Does anyone have experience at loading up an entirely new
> > database
> >> > with AmiBroker?
> >> > For example, loading AB with all of Nasdaq with 5 years of
data?
> >> > I used the file import command, and selected a series of .txt
> > files,
> >> > and it takes around 32 hours of elapsed time per year!!! (On
a
> > fast
> >> > machine, no less!)
> >> > Does anyone know of a quicker method?
> >> >
> >> >
> >> >
> >> >
> >> > Please note that this group is for discussion between users
only.
> >> >
> >> > To get support from AmiBroker please send an e-mail directly
to
> >> > SUPPORT {at} amibroker.com
> >> >
> >> > For NEW RELEASE ANNOUNCEMENTS and other news always check
DEVLOG:
> >> > http://www.amibroker.com/devlog/
> >> >
> >> > For other support material please check also:
> >> > http://www.amibroker.com/support.html
> >> >
> >> > Yahoo! Groups Links
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
> >
> >
> >
> > Please note that this group is for discussion between users only.
> >
> > To get support from AmiBroker please send an e-mail directly to
> > SUPPORT {at} amibroker.com
> >
> > For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
> > http://www.amibroker.com/devlog/
> >
> > For other support material please check also:
> > http://www.amibroker.com/support.html
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
> >
>
Please note that this group is for discussion between users only.
To get support from AmiBroker please send an e-mail directly to
SUPPORT {at} amibroker.com
For NEW RELEASE ANNOUNCEMENTS and other news always check DEVLOG:
http://www.amibroker.com/devlog/
For other support material please check also:
http://www.amibroker.com/support.html
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/amibroker/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/amibroker/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:amibroker-digest@xxxxxxxxxxxxxxx
mailto:amibroker-fullfeatured@xxxxxxxxxxxxxxx
<*> To unsubscribe from this group, send an email to:
amibroker-unsubscribe@xxxxxxxxxxxxxxx
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|