Server Or Has No More Resources

Jack@dba

Member
Hi All,

We had issue with our Mfg/pro application "server has no more resources" .Database is up and running fine but application we are not able to connect.

Some times user able to connect to application for 5 mins after that we are receiving same issue.
In promon it shows user 108 users.

Promon > 5:Activity> 25 Servers, 108 Users (1 Local, 107 Remote, 1 Batch),4 Apws

.pf file

-n 1050 # Number of users
-Ma 20 # Maximum Clients per Server
-Mn 50 # Maximum servers
-Mi 5 # Min processes on a client server

Progress verison : 9.1E
Unix O/S: Aix 5.3

In database log file we found below error.

10:44:19 BROKER 0: No servers are available. Try again later. (1154)
10:44:36 BROKER 0: No servers are available. Try again later. (1154)
10:44:59 BROKER 0: No servers are available. Try again later. (1154)

I would like to know whether we need to Change -Ma -Mn parameter values?
Are we good with these values?


upload_2017-8-10_23-2-20.png
 

TomBascom

Curmudgeon
Progress 9.1E is ancient, obsolete and unsupported. So is AIX 5.3. You should upgrade. Your company is taking a grave risk continuing to operate these releases.

Progress KB - Corrective Measures for Error 748

-n 1080 but you really only have 108 users? I'm a fan of having -n larger than it needs to be but that is kind of "excessive".

Are you using -minport and -maxport? You should be.

Do you have SQL-92 connections? If you do then you should have a dedicated login brokers for both SQL and 4GL users. You should NOT use -ServerType BOTH. This means that you will also need -Mpb for each of them.

4 APWs is also excessive. One is probably plenty.
 

Jack@dba

Member
Thanks Tom for info.

I have checked all this conditions we are good
Error 748 can be caused when:
  • the -n setting is not high enough,
  • the kernel parameters for semaphores might not be set correctly, === We not checked this kernel parameters.
  • the services file might not be set up properly, or
  • all servers have reached maximum number for remote clients per server. === How to check this condition.
  • the system has no more available resources. In this case, a full system restart may be required (to free all system resources that are currently in use).
After checking all the above conditions we have taken decision to reboot the server and database.

-n 1080 but you really only have 108 users? Yes still why we are facing resource issue.

Are you using -minport and -maxport? You should be.

Of course

11:43:54 BROKER 0: Minimum Port for Auto Servers (-minport): 13100.
Maximum Port for Auto Servers (-maxport): 14999.

Do you have SQL-92 connections? If you do then you should have a dedicated login brokers for both SQL and 4GL users.
02:01:58 BROKER 0: Multi-user session begin. (333)
02:01:58 BROKER 0: Started for prdcrn using tcp, pid 69372. (5644)
02:01:59 BROKER 0: Progress OpenEdge Release 9.1E on AIX. (4234)
02:01:59 BROKER 0: Server started by pgresdba on batch. (4281)
02:01:59 BROKER 0: Started using pid: 3836. (6574)
02:01:59 BROKER 0: Physical Database Name (-db): /progdata/prd/crndb1/prdcrn. (4235)
02:01:59 BROKER 0: Database Type (-dt): PROGRESS. (4236)
02:01:59 BROKER 0: Force Access (-F): Not Enabled. (4237)
02:01:59 BROKER 0: Direct I/O (-directio): Not Enabled. (4238)
02:01:59 BROKER 0: Number of Database Buffers (-B): 70000. (4239)
02:01:59 BROKER 0: Maximum private buffers per user (-Bpmax): 64. (9422)
02:01:59 BROKER 0: Excess Shared Memory Size (-Mxs): 33554432. (4240)
02:01:59 BROKER 0: The shared memory segment is not locked in memory. (10014)
02:01:59 BROKER 0: Current Size of Lock Table (-L): 1024000. (4241)
02:01:59 BROKER 0: Hash Table Entries (-hash): 18289. (4242)
02:01:59 BROKER 0: Current Spin Lock Tries (-spin): 12000. (4243)
02:01:59 BROKER 0: Number of Semaphore Sets (-semsets): 256. (6526)
02:01:59 BROKER 0: Crash Recovery (-i): Enabled. (4244)
02:01:59 BROKER 0: Database Blocksize (-blocksize): 8192. (6573)
02:01:59 BROKER 0: Delay of Before-Image Flush (-Mf): 3. (4245)
02:01:59 BROKER 0: Before-Image File I/O (-r -R): Reliable. (4247)
02:01:59 BROKER 0: Before-Image Truncate Interval (-G): 60. (4249)
02:01:59 BROKER 0: Before-Image Cluster Size: 524288. (4250)
02:01:59 BROKER 0: Before-Image Block Size: 8192. (4251)
02:01:59 BROKER 0: Number of Before-Image Buffers (-bibufs): 200. (4252)
02:01:59 BROKER 0: BI File Threshold size (-bithold): 0.0 Bytes. (9238)
02:01:59 BROKER 0: BI File Threshold Stall (-bistall): Disabled. (6552)
02:01:59 BROKER 0: After-Image Stall (-aistall): Not Enabled. (4254)
02:01:59 BROKER 0: After-Image Block Size: 8192. (4255)
02:01:59 BROKER 0: Number of After-Image Buffers (-aibufs): 300. (4256)
02:01:59 BROKER 0: Storage object cache size (-omsize): 1024 (8527)
02:01:59 BROKER 0: Maximum Number of Clients Per Server (-Ma): 15. (4257)
02:01:59 BROKER 0: Maximum Number of Servers (-Mn): 36. (4258)
02:01:59 BROKER 0: Minimum Clients Per Server (-Mi): 5. (4259)
02:01:59 BROKER 0: Maximum Number of Users (-n): 1001. (4260)
02:01:59 BROKER 0: Host Name (-H): p5lp5. (4261)
02:01:59 BROKER 0: Service Name (-S): prdcrn. (4262)
02:01:59 BROKER 0: Network Type (-N): tcp. (4263)
02:01:59 BROKER 0: Character Set (-cpinternal): iso8859-1. (4264)
02:01:59 BROKER 0: Parameter File: /prd/dba/etc/pf/db_prdcrn.pf. (4282)
02:01:59 BROKER 0: Minimum Port for Auto Servers (-minport): 13100. (5648)
02:01:59 BROKER 0: Maximum Port for Auto Servers (-maxport): 14999. (5649)
02:01:59 BROKER 0: This broker supports both 4GL and SQL server groups. (8865)
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
-n 1050 # Number of users
-Ma 20 # Maximum Clients per Server
-Mn 50 # Maximum servers
-Mi 5 # Min processes on a client server

02:01:59 BROKER 0: Maximum Number of Clients Per Server (-Ma): 15. (4257)
02:01:59 BROKER 0: Maximum Number of Servers (-Mn): 36. (4258)
02:01:59 BROKER 0: Minimum Clients Per Server (-Mi): 5. (4259)
02:01:59 BROKER 0: Maximum Number of Users (-n): 1001. (4260)
There are discrepancies between what you claim and what the database reports for these parameter values. Perhaps someone has recently changed the configuration, or there is more than one .pf in use and these parameters are being specified more than once.


all servers have reached maximum number for remote clients per server. === How to check this condition.
Look at promon R&D 1, 17: Servers By Broker. Check how many servers have actually been spawned, how many connections each has, and how many pending connections each has. Pending connections take up a connection slot just like actual connections, preventing servers from being fully utilized. If you have many pending connections, you may want to try using the -PendConnTime parameter to prevent this.
Knowledge Article
Progress KB - What is the -PendConnTime parameter?

02:01:59 BROKER 0: This broker supports both 4GL and SQL server groups. (8865)
Follow Tom's advice: keep 4GL and SQL clients and servers segregated on separate brokers. At a minimum, for client/server each one needs -ServerType, -S, -H, -Mpb, -Ma, -Mi, -minport, and -maxport.

-n 1080 but you really only have 108 users? Yes still why we are facing resource issue.
You need to look at more than just -n. It can be a limiting factor but in your case clearly it is not. Other limiting factors are the product of -Mpb (or -Mn) and -Ma, the number of servers that actually get spawned (as opposed to the number that should) and whether there are long-term pending connections. With a shared broker you can also have the problem that you have too many servers of one type (e.g. SQL) and therefore can't later spawn enough servers of the other type (4GL) to serve all the clients that want to connect.

Also:
  • Make sure your -minport/-maxport ranges are unique and separate for each broker; don't re-use or overlap ranges.
  • Make sure that none of the ports in those ranges are in use or even defined in /etc/services. The 4GL broker will not spawn a server on a port if it has a mapping in /etc/services, even if no process is currently bound to it. You may need to clear out some unused "junk" from /etc/services to free up contiguous port ranges that are large enough for your needs. I like to add my -minport/-maxport ranges as comments in /etc/services so that unwitting admins don't later take ports that I want to use for database servers.
  • Make sure each broker's -minport/-maxport range is at least as large as its -Mpb value.
  • Make sure your firewall permits connections from client to server zone on all broker ports (-S) and all possible server ports (-minport/-maxport ranges).
  • Make sure you keep your -minport/-maxport ranges above 1024 and below 32000.
 

Cringer

ProgressTalk.com Moderator
Staff member
IIRC 9.1e suffered from the issue that once a server on a shared broker has been connected to by a SQL process it is reserved for SQL, once a server has been connected to by 4GL it is reserved for 4GL, so it's possible that if your broker is shared that too many servers are reserved for SQL, meaning your Progress application can't connect. This is one of the biggest reasons for segregating the brokers.
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
IIRC 9.1e suffered from the issue that once a server on a shared broker has been connected to by a SQL process it is reserved for SQL, once a server has been connected to by 4GL it is reserved for 4GL
4Gl server is _mprosrv, SQL server is _sqlsrv2. Was this not the case in 9.x?
 

Jack@dba

Member
How to set -Mpb startup parameter?

if i want to setup new connection for 500 users.Can i set the parameters as below.

-n 500 -Mn 100 -Ma 5 -Mi 1.
 

Rob Fitzpatrick

ProgressTalk.com Sponsor
if i want to setup new connection for 500 users.Can i set the parameters as below.

-n 500 -Mn 100 -Ma 5 -Mi 1.
What does "setup new connection" mean? Are you trying to start a new broker on the database you were discussing above? Are you trying to reconfigure the existing broker above? It isn't clear what you are trying to do. Please provide a more detailed explanation.

Note also that -n and -Mn are database-wide settings; they go on the primary broker only.

Client/server parameters like -Mpb, -Ma, -Mi, -S, -minport, -maxport, -ServerType (this is not a complete list) go on each broker that will be spawning remote servers.
 

TomBascom

Curmudgeon
Code:
#!/bin/sh
#

DB=mydb
Mm=8192
HOST=myhost

proserve ${DB} -n 1000 -Mn 200 -spin 10000 -B 1000000 -L 100000 -tablerangesize 1250 -inderangesize 2500 -Mm ${Mm} -Mn 500
probiw ${DB}
proaiw ${DB}
prowdog ${DB}
proapw ${DB}

# 4GL login broker for remote 4gl connections

proserve ${DB} -m3 -Mm ${Mm} -Mi 1 -Ma 5 -Mbp 100 -H ${HOST} -S 9990 -ServerType 4GL -minport 20000 -maxport 21999

# SQL login broker

proserve ${DB} -m3 -Mm ${Mm} -Mi 5 -Ma 5 -Mpb 50 -H ${HOST} -S 9991 -ServerType SQL -minport 22000 -maxport 22999

This will allow 1,000 total connections. 500 are potentially remote 4gl connections, 250 can be SQL. (-Mpb * -Ma for that "proserve") The remaining 250 could be self-service users and other local connections (like the BIW etc).

Note: -Mi 5 for SQL. SQL brokers are multi-threaded so you don't usually want to use -Mi 1 -- that would waste resources. Old releases have bugs if too many connections are made to a SQL broker. So keep it down to 5 or less with -Ma 5.

Notre 2: I somehow left out -Mn 200 in the first iteration of this. -Mn is the total number of servers across all login brokers. It needs to be larger than the sum of all the -Mpb values.
 
Last edited:
Top