bulk data retrieval

P

peidor

New Member
Hi,

can anyone enlighten me on the following topic:

Trying to find the best way to retrieve data from large tables (>1M recs) from a v8.3 database for datamining. The problem is the processing time.

Options tested so far are:
1) unload
2) ODBC (Merant) to MS-Access on client

Now considering:
1) Running a v9 server install next to the v8.3 with nighttime mirrorring/copy
2) upgrading the slow network (10M)

unfortunately we have the following restrictions:
1) no SQL server allowed
2) no Access on Server allowed (is still being discussed however)
3) Win NT (should be W2003 by 2005)
4) upgrading the v8.3 install will result in extensive (i.e. expensive) rebuild of the app. Will not be an option

Is there any other useable option?
 
You may want to look at Enterprise Cyberquery from Cyberscience as a way of extracting your data. I received a letter from them recently stating that it has native access to Progress DBs. They also apparently have 200+ QAD MFG/PRO customers. It also claims full-feature ad-hoc query and BI functionality - so you may not need Data Mining tools on top.

Hope that helps.
Jacob.
 
just saw eCQ demo yesterday

I just saw an eCQ demo yesterday and it was impressive. The company I work for currently uses CQ, but the enhancements to the enterprise version are impressive enhancements (Web Access interface, Reports direct to email, Scheduled reporting from eCQ's own scheduler).
 
peidor said:
Hi,

can anyone enlighten me on the following topic:

Trying to find the best way to retrieve data from large tables (>1M recs) from a v8.3 database for datamining. The problem is the processing time.

....

4) upgrading the v8.3 install will result in extensive (i.e. expensive) rebuild of the app. Will not be an option
Click to expand...

It is probably way too late now (the original post is 2 years old) but just in case anyone is in a similar boat...

Upgrading does not require you to do anything to the app. It does provide opportunities to do so but you can just leave the application "as is" (many people do). And there are enormous performance benefits available between 8.3 and 10.1. You should seriously reconsider the upgrade option.

There may also, of course, be significant tuning opportunities within the 8.3 framework that is in place. Some of the likely areas include server configuration, database parameters, database block size, and the retrieval process itself.
 
You must log in or register to reply here.
Back
Top