insufficient disk space or write access denied

C

ctoman

New Member

Situation​


  • OS: Ubuntu 16.04 (getting old, unsupported — worth noting)
  • Progress OE: 10.2B (32-bit database on a 64-bit OS)
  • Database storage: NFS mount — known risk for any database, especially Progress.
  • Error 291 multiple times a day since NetApp head move/reboot.


First, What is Error (291)?​


In Progress 4GL documentation:


(291) Cannot find or open file for read or update.
(sometimes shown as SYSTEM ERROR 291)
Click to expand...

In a database context, it typically means:


  • DB is trying to access a file (e.g., BI file, AI file, db file)
  • But the NFS mount isn't 100% available — even for just a fraction of a second.
  • Result: DB process throws 291, might retry, and (worse) might eventually crash if too many I/O errors pile up.

In short: I/O error or "momentary" unavailability on NFS = 291.



Now your hints are extremely important:​


  • No OS changes
  • No Progress version changes
  • Only infrastructure change: NetApp moved from head 1 → head 2 → reboot
  • Now frequent 291 errors.
 
no 6072.

I enabled logging, and found the 291 error

AS -- (Procedure: 'get_stock_detail inv/inv310spsx3p.p' Line:6684) Insufficient disk space or Write access denied. (291)

[25/04/29@06:11:54.781-0400] P-002712 T-005864 1 AS -- (Procedure: 'get_stock_detail inv/inv310spsx3p.p' Line:6684) Insufficient disk space or Write access denied. (291)
 
yes, i dd xref, sending developer.
just for each statement.
6684 FOR EACH cis.mfg-tagm NO-LOCK
6685 WHERE cis.mfg-tagm.tag-bsn = axs.facl.plant_bsn AND
6686 ((cis.mfg-tagm.tag-status = 'I' OR cis.mfg-tagm.tag-status = 'L')
6687 OR
6688 (cis.mfg-tagm.tag-status = 'S' AND cis.mfg-tagm.tag-bsn = '01') /*Used to pull all inventory in consagnment status for atlanta*/
6689 OR
6690 (cis.mfg-tagm.tag-status = 'S' AND SUBSTRING(cis.mfg-tagm.wctr-nbr,3) = 'CS')
6691 OR
6692 (cis.mfg-tagm.tag-status = 'S' AND SUBSTRING(cis.mfg-tagm.wctr-nbr,3) = 'IT')
6693 )
6694 AND
6695 NOT (cis.mfg-tagm.wctr-nbr MATCHES '*TP' or cis.mfg-tagm.wctr-nbr MATCHES '*RW') /*requested by David S to be ignored in the report*/
6696 AND
6697 NOT (LOOKUP(cis.mfg-tagm.wctr-nbr, iv_wipdone_stkrms, "|":U) > 0)
6698 BY cis.mfg-tagm.part-nbr
6699 BY cis.mfg-tagm.wctr-nbr:
6700
 
Are there a lot of records in the mfg-tagm table?
It looks like it's a sorting of BY phrase that causes the error.
What is the -T directory?
 
yes, around 19 million records, however, small as far as bytes.
the -T /tmp - there is enough space too.
 
Insufficient disk space or Write access denied with error 291 when lots of disk space is available

Progress Customer Community

community.progress.com community.progress.com
Cause
There are many "old" temporary files present in Progress working directory, e.g. srt*, lbi*, DBI*, rcd*, etc, which are "leftovers" from previous sessions that crashed, so those temp files were never cleaned (which should have been deleted automatically after a clean exit from a Progress session).
When Progress tries to use the same (existing) filename for a temp file, it fails with error (291) because that file already exists (from some previous crashed session).

Do you get just one error for a while? Not the series of the errors followed by a quiet period?
 
The servers were throwing low virtual memory errors, IG added memory and expanded the drives, all seem to be good for now.

Thanks,
 
You must log in or register to reply here.
Back
Top