Progress Replication?

tylerhardison

New Member
We've got two progress databases here, one is our development server and the other is our live system. Currently to synchronize the two, were using proback and then using rsync to move the backups to the devel server where we use prorest. We've tried NFS, and other methods. Our database is over 21 gigs now and it's taking upwards 10 hours to complete it all.

Does anyone have a better way? Is there a replication utility like Oracle has? It seems a waste of bandwidth and time to copy data over thats not changing....

Thanks.

Tyler.
 
I don't know if you can do that on windows but on unix we do have a special method using pipes. We simply do a Progress backup (probkup) online on the production database and pipe the result to another machine where we restore (prorest) through another pipe.

Users in production don't see any difference.
However, we do that by the end of the day.

Total time: 45 min for 30 GB.

J. Cossette
Talvest Fund Management
DBA
 
progress does actually have remote database replication tools, but i beleive that you have to pay extra for them. These are live changes, so that you have two databases,e xactly the same on different servers.

surely you are not updating live every day? would not one weekend a month be better for updating live, then you can do loads of testing before the changes go live?

you cold also create delta's of data and tables, then apply them... I don't touch that side ( my collegue does)

You can also do incremental backups everynight, but not sure how this affects separate regular backups...

also, compressing while doing an rsync causes much more data to be transferred - i did a test today, as we are looking at daily replication for our customers...
 
Do you update the data on the development server? I'm guessing that you do. In which case log based replication (ai files or oe replication) won't help you. If I'm wrong about that and you never change anything on the development server stop reading and we'll discuss a different strategy.

There are several likely bottlenecks that you're dealing with. The first is the disk subsystem. I wouldn't expect a probkup of 20GB or so to take more than about an hour -- if it is taking longer then I'd be deeply suspicious that your disk subsystem is (very) poorly configured. (On a well configured disk array I'd expect such a backup to take more like 15 minutes...)

Make sure that you are reading from and writing to different disks when doing the probkup. Also enable compression. Use a command something like this:

Code:
probkup /dbpath/dbname -vs 125000 -com /backup/dbname.bk1 < /backup/backup.list

/dbpath and /backup should be on a different set of disks -- ideally they are striped and mirrored. The worst case is that there is a single RAID5 (or just one disk) and both paths are on the same disks. Such a setup will indeed take a long time to run.

If you have 8k db blocks the "125000" will create 1GB backup extents (the -vs parameter is in db block units). You will need 20 or so extents listed in /backup/backup.list. The -com parameter will skip empty blocks and do some other things to avoid writing useless data.

The next bottleneck is the transfer process. First compress the backup extents using gzip or whatever your favorite file compressor may be. When you transfer them you can use multiple threads -- you'll need to monitor your network to see how heavily you are utilizing it and a great deal depends on the hardware resources available but, as a rule of thumb, a single thread rarely uses more than 10% of the available network bandwidth. So starting 10 threads to do the transfers in parallel will probably speed things up. Especially if the disks (on both ends) are fast.

Make sure that the target disks that you are transferring to are different from those that you are planning to restore into -- on the restore you want to be reading the backup extents from different disks than you are writing the database to just as you read from different database disks when writing the backup extents.

While the transfer is taking place (re)build an empty database to restore into using prostrct create. If you don't do this then prorest will do it as its first step. So save some time and do it while you're waiting.

After the transfer finishes unzip the backup extents and restore them into the waiting empty database.

The whole thing really shouldn't take long at all.
 
Back
Top