Cim Load

[AndyH]

I have been using CIM to load our test data. I create the Cim file then
load using 36.15.2 option. I have been loading to 1.2.5(PPPLPMT.p) batches of aroung 750 records. This process takes hours for these many records.
I'm new to Progress so not sure where to start looking for the problem.

AIX
Progress 9.1.D


Thanks for any help

 

[toby.Harman]

You may want to take this over to the QAD section of the list - there may be some better resources checking there!

As to the general "It doesn't go fast enough", here are a couple of things to check out.

When you are running the CIM load, try to get some hardware/OpSys statistics regarding CPU utilisation and Disk utilistation. Also find out how much memory your database is using and how much the system has to spare.

Rule 1 of making it go faster. More memory will almost always help.

Rule 2 of making it go faster. More disk will usually help.

Rule 3 of making it go faster. More CPU will sometimes help.

It may be that resources are not available to Progress at the correct time. More memory may mean cutting back on some other application or tweaking a system parameter, not just rushing off to your local IBM office!

Promon also has some good output. When you are running the load, go into Promon and select R&D (not a displayed option but supported). Then select option 2, 1 and press S to get a fresh sample (rather than Data from the last time the server was brought up). That gives you a 10 second sample. Wait a couple of minutes and then press U which will update the statistics on the screen with the activity since you began your sample.

Things to look for
Buffer Hits - Less than 95% is poor. - If below 95% then make more memory available to the database using -B
BI Buf Waits - As small as you can get it - Startup parameter can be set to help this -bibufs
Checkpoints - 1 per minute should be about right! - This one is harder to tune - To decrease the frequency of checkpoints you need to increase your BI Cluster size (as part of a Truncate bi command). However there is a tradeoff here since if you get it too big you may end up getting lumpy performance where the whole database seems to "freeze" fro a few moments. This is due to the Flushed at Checkpoint stuff.
Flushed at Checkpoint - small as possible. 0 is good!

I don't know if you have Enterprise or Workgroup Database, but if it is Enterprise then make sure that
-spin is set to at least 5000 (probably more if you have more than 1 CPU)
APW is started - probably 2
BIW is started - you can only have one.
If you are running AI then AIW should be started.

 

[cecsno]

I have been using CIM to load our test data. I create the Cim file then
load using 36.15.2 option. I have been loading to 1.2.5(PPPLPMT.p) batches of aroung 750 records. This process takes hours for these many records.
I'm new to Progress so not sure where to start looking for the problem.

AIX
Progress 9.1.D


Thanks for any help

A CIM load is streaming keyboard input from a file. So you are automating keystrokes and it takes only slightly less time than if you are doing it by hand.

What version of MFG/PRO is this, I can't find ppplpmt.p or 1.2.5 in either 9.0 or eB. If this is a mod you could look at to code and possibly find the problem.

 

[toby.Harman]

Eeeww.

I take back all my previous tuning advice since it is totally irrelevant here!

Tell me this isn't running client server. Please......

 

[roettgermann]

Flushed at Checkpoint - small as possible. 0 is good!

Checkpoints - 1 per minute should be about right! - This one is harder to tune - To decrease the frequency of checkpoints you need to increase your BI Cluster size (as part of a Truncate bi command). However there is a tradeoff here since if you get it too big you may end up getting lumpy performance where the whole database seems to "freeze" fro a few moments. This is due to the Flushed at Checkpoint stuff.
Flushed at Checkpoint - small as possible. 0 is good!

=> HOW CAN I REACH THIS VALUE 0?
=> WITH A STARTUP PARAMETER??
Can anyone help me?
Kind regards
Matthias Röttgermann

 

[Casper]

Bigger bi cluster size means longer time between checkpoints, this is only usefull if you have enterprise database because then you have biw, apw and aiw which you should start.

Another advantage of bigger clustersize is that the bigger the cluster the more time for the biw (and apw) to write buffers to disk.
To decrease the checkpoint time or in other words decrease # of flushed buffers at checkpoint start one or more apw's until no more buffers are flushed at checkpoint.

Look in promon for empty buffer waits, keep this number as low as possible by increasing number of bi buffers.

Put bi on different disk then the database.

Make sure bi blockzsize is same as ai blocksize.

And, btw, -spin is only usefull if you have more then one cpu. So if you only have one cpu then you can omit -spin.

HTH,


Casper