How progress populates data into the database?

T

triqt

New Member
Hi,

How progress populates data into the database? I afraid of the 2GB limit from the Linux file systems. Is it going to populate the data into the timeware_8.d5 until it hit the 2GB limit and the database will stop? Or it will hit the 2GB limit from timeware_8.d5 and then it will write to the other timeware_8.dx?

The current size of the database:
-rw-r--r-- 1 root root 209715200 Jun 17 15:20 timeware_7.d1
-rw-r--r-- 1 root root 637272064 Jun 17 15:20 timeware_7.d2
-rw-r--r-- 1 root root 419430400 Jun 17 15:20 timeware_8.d1
-rw-r--r-- 1 root root 419430400 Jun 14 19:31 timeware_8.d2
-rw-r--r-- 1 root root 419430400 Jun 14 19:31 timeware_8.d3
-rw-r--r-- 1 root root 209715200 Jun 14 19:31 timeware_8.d4
-rw-r--r-- 1 root root 1323433984 Jun 17 15:20 timeware_8.d5
-rw-r--r-- 1 root root 209715200 Jun 17 15:20 timeware_9.d1
-rw-r--r-- 1 root root 209715200 Jun 17 15:20 timeware_9.d2
-rw-r--r-- 1 root root 209715200 Jun 17 15:20 timeware_9.d3
-rw-r--r-- 1 root root 625999872 Jun 17 15:20 timeware_9.d4
-rw-r--r-- 1 root root 209715200 Jun 17 15:25 timeware.b1


Below is the content of the timeware.st

d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d1 f 204800
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d2
#
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d1 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d2 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d3 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d4 f 204800
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d5
#
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d1 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d2 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d3 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d4
 
I don't think there is a 2GB filesize limit on Linux systems nowadays.
Just make sure you have EnableLargeFiles enabled for that database and ulimit is set to unlimited.
But to anwer your question: if there would be a maximum file size limit then the database would stop once that limit is reached.You can avoid that by monitoring growth and adding extents online when necessary.

Casper.
 
It would help if you mentioned what release of Progress this is... The structure file is a v9 structure file but it could just be that the db has been upgraded to oe10 and not yet converted to using type 2 storage areas.

If it is v9 then largefile support depends on which minor version and patch release of v9 it is.

However, as Casper says, the 2GB limit only applies to individual extents (files). The total amount of data can be well over 2GB by being spread through multiple extents. Variable extents (those without an "f" in the .st file, for instance timeware_7.d2) will grow as data is added to them. If unmanaged and if largefiles is not enabled and/or if the filesystem does not support large files they may hit a 2GB limit.

Check your .lg file to see if largefiles is enabled. If it is there will be a message similar to this:

Code: 
Large database file access has been enabled. (9426)

Use:

Code: 
proutil dbname -C enablelargefiles

to turn on large file support.

Also, your structure file shows a "functional" approach to assigning tables and indexes to storage areas. This is naive. You will have much better results if you assign objects to storage areas based on their technical characteristics (average row size, frequency of use, frequency of growth & changes).
 
Hi,

We are running version Progress 9.1D, and Linux versionRed Hat Linux release 7.3 (Valhalla)

I guess the best option for us is to add more extends. Please help to verify the .st file and the procedure to add new extends.

The new timeware.st file would be

#
b . f 204800
b .
#
d "Schema Area":6,64 .
#
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d1 f 204800
#d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d2
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d2 f 204800
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d3 f 204800
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d4 f 204800
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d5
#
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d1 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d2 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d3 f 409600
#
# CAN I CHANGE THE SIZE OF THE EXISTNG EXTEND FROM 204800?
# TO 409600. I THINK I SHOULD NOT DO THIS. NEW DATA WILL
# WRITE TO THIS EXTEND WHICH WE DO NOT WANT.
#d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d4 f 204800
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d4 f 409600
#UPDATE THE SIZE OF THE LAST EXISTING EXTEND
#d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d5
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d5 f 409600
# THREE NEW EXTENDS
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d6 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d7 f 409600
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d8
#
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d1 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d2 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d3 f 204800
# UPDATE THE SIZE OF THE LAST EXISTING EXTEND
#d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d4
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d4 f 204800
# THREE NEW EXTENDS
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d5 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d6 f 204800
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d7

Procedure to add new extends
  • Update the timeware.st file as above
  • Shutdown the database and do a backup by probkup
  • prostrct add timeware timeware.st
  • start the database again (done)
 
Obviously you should be planning to upgrade to a supported release ASAP.

To add extents you only specify the new extents in the .st file. I usually name this file "add.st".

You can only change the size of an extent by recreating the database. One way to do that would be to create your desired structure, use "prostrct create" to build the structure and then use prorest to restore a backup into it.

Ideally you would want to do this in a non-destructive manner -- that is do not delete the original database until the new database is validated. That means that you would need disk space for two copies of the db (as well as space for the backup files). For example (assuming a sports2000 db installed in /data/s2k):

Code: 
cd /data/s2k
probkup sports2000 /backup/sports2000.pbk -com
cd ..
mv s2k s2k.old  # preserve the old database just in case
mkdir s2k
cd s2k
cp ../s2k.old/sports2000.st.new sports2000
prostrct create sports2000 -blocksize 4096  # match blocksize to original
prorest sports2000 /backup/sports2000.pbk

Using "mv" to rename the old db dir avoids having to run prostrct repair to fix up path names.
 
Tom,

If I add new extends, it's ok for me to change those variable extends without the "f" to fix size with the "f" right?

These extends.
d "General Data":7,64 /apps/tempus/timeware/database/timeware_7.d2
d "Labor Data":8,64 /apps/tempus/timeware/database/timeware_8.d5
d "All Index":9,64 /apps/tempus/timeware/database/timeware_9.d4
 
Yes, that is what you do.

The following experiment with sports2000 demonstrates:

Code: 
$ cat add.st
d "Misc":12,32;1 . f 1024
d "Misc":12,32;1 .
$ prostrct add sports2000 add.st

Converting relative path database to absolute path database. (8461)

Formatting extents:
   size                area name   path name
    256                     Misc /data/tmp/sports2000_12.d3 00:00:00
     16                     Misc /data/tmp/sports2000_12.d4 00:00:00
 
Before you start adding extents, read a little more about the prostruct utility. Do a prostruct list and produce a current version of the .st file (it may have changed during the life of the db), and do a prostruct statistics to find out what extents are currently being populated, I have not seen anything in the post that says the variable extents are growing yet.
 
You must log in or register to reply here.
Back
Top