V
Valeriy Bashkatov
Guest
Hi, >>Is it so that Pro2 is using ROWID when checking for new records? Yes. >>if so, when doing a full restore, will rowid change? The rowid is changed only after Dump&Load on the source database. >> Should I start one bulkload thread for each of the large tables, and use one thread for the rest? We use one thread for each small table (30-40 million records is a small table for us) But for really large tables (~15 billion records), we wrote our own primary synchronization template and process for multithreaded primary synchronization of such a table.The number of threads can be adjusted. We usually use 25-30 threads per table. The main thing here is to have enough server power. We usually synchronize such tables in 3-5 days. >>Are you saying that Pro2 could bulkload without indexing on SQL ? and then do a rebuild of index on SQL ? Yes, it's possible. But to do this, you need to remove unnecessary code from the primary synchronization template. We use Pro2Oracle and in our templates we removed the check for records in Oracle before copying (FIND FIRST bfrTgt >). This check significantly reduces performance. However, should always remember that no one work with the target table at this time. At the same time, we still had two main indexes active by PRROWID and PROGRESS_RECID fields. This allows us to avoid wasting time after the initial synchronization is completed and start ETL processes immediately (replbatch.sh). All other indexes that are needed by a third party system are rebuilt online in oracle.
Continue reading...
Continue reading...