performance issue with the webspeed reports

[Raj]

Hi All,

I have a performance issue with the webspeed reports.

User is requesting the information (which return's some 1500 rows)

It behaves very strangly like
Some time same report comes with in 6sec

when i try the same report it take me more 5mins and there is a error along the result saying CGI Timeout.

All the row are retrieved on a single request to to WebSpeed in the Unix from the IE Client.

some times we are getting a message saying the agents are busy.
lasttime i remember when we got the error saying that the user is not able to access the webreports.

So we found that the agent is busy not released automatically. we restart the transaction server deamon.

After this event later 3days the user's started saying we are having performance issue on the reports. there reports were working fine for a long time without any modification.

Now Current setup is something like

The WebSpeed Messenger is running on IIS under w2k server.
and the remaining are runing under the Sun Solaris Box.
How do i find out how many broker/agents were licenced to us.

Version Information:
The Progress version 9.1A patch 08.
The Webspeed version 3.1A patch 05.
No of Brokers:3 (wSspn,wSInter,wSExter)
No of Agents :2 Agent Attached to each broker.
my obervation while monitoring NS2.ns.log: is keep on adding the line when agent become busy and we get the cgi timeout error.
some of the log informations from unix( /data/webspeed/wrk):
-rw-rw-r-- 1 root system 24714 Nov 30 15:56 spnExter.server.log
-rw-rw-r-- 1 root system 265131 Nov 30 15:56 spnExter.broker.log
-rw-rw-r-- 1 cedric system 142930 Nov 30 15:57 AdminServerSystemOut.log
-rw-rw-r-- 1 root system 9133402 Nov 30 15:57 spnInter.broker.log
-rw-rw-r-- 1 root system 273177 Nov 30 15:57 spnInter.server.log
-rw------- 1 root system 162979840 Nov 30 16:17 core
-rw-rw-r-- 1 root system 219751011 Nov 30 18:03 wsSpn.server.log
-rw-rw-r-- 1 root system 468610973 Dec 1 04:06 wsSpn.broker.log
-rw-rw-r-- 1 cedric system 3760321 Dec 1 04:17 admserv.log
-rw-rw-r-- 1 cedric system 34823 Dec 1 04:17 cmdplugin.log
-rw-rw-r-- 1 root system 828269962 Dec 1 06:51 NS2.ns.log
wsSpn.broker.log:
C-0001>(01-Dec-05 04:00:42:485) EOFException received on client connection : C-0001. (8084)
C-0001>(01-Dec-05 04:00:42:489) Client disconnected : . (8534)
L-3058>(01-Dec-05 04:04:58:013) Got a connection:: (8125)
C-0001>(01-Dec-05 04:04:58:019) Client connected : . (8533)
C-0001>(01-Dec-05 04:04:58:027) EOFException received on client connection : C-0001. (8084)
C-0001>(01-Dec-05 04:04:58:031) Client disconnected : . (8534)
L-3058>(01-Dec-05 04:06:47:779) Got a connection:: (8125)
C-0001>(01-Dec-05 04:06:47:785) Client connected : . (8533)
C-0001>(01-Dec-05 04:06:47:883) EOFException received on client connection : C-0001. (8084)
C-0001>(01-Dec-05 04:06:47:887) Client disconnected : . (8534)
wsSpn.server.log
S-0003>(30-Nov-05 17:07:36:350) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 5. (6404)
S-0003>(30-Nov-05 17:07:37:351) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 6. (6404)
S-0003>(30-Nov-05 17:07:38:350) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 7. (6404)
S-0003>(30-Nov-05 17:07:39:350) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 8. (6404)
S-0003>(30-Nov-05 17:07:40:350) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 9. (6404)
S-0003>(30-Nov-05 17:07:41:350) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 10. (6404)
S-0003>(30-Nov-05 17:07:42:350) [26594] WTA: WebWrite() failed during npp_send with error: -70 size: 67 sentbytes 0 retrycount 11. (6404)
S-0003>(30-Nov-05 17:07:52:351) [26594] WTA: npp_send() failed while sending terminate message! Error: -70. (6397)
S-0001>(30-Nov-05 18:03:14:951) [35] X-NODEREF or X-DOCUMENT GET-CHILD got an error: Invalid child index. (9082)
S-0001>(30-Nov-05 18:03:14:954) [35] X-NODEREF must be associated with a valid X-DOCUMENT in order to use it in method NODE-VALUE. (9102)
NS2.ns.log:
Thread-2>(01-Dec-05 06:49:32:502) Known Broker 135.121.245.29:6312a:df6e325bcf:-8000 WS.wsSpn astdec/135.121.245.29 3058. (8209)
Thread-2>(01-Dec-05 06:49:50:460) Known Broker 135.121.245.29:6312a:df5ddaf8f3:-8000 WS.spnExter astdec/135.121.245.29 3059. (8209)
Thread-2>(01-Dec-05 06:50:08:928) Known Broker 135.121.245.29:6312a:df5ddb3ae9:-8000 WS.spnInter astdec/135.121.245.29 3060. (8209)
Thread-2>(01-Dec-05 06:50:32:503) Known Broker 135.121.245.29:6312a:df6e325bcf:-8000 WS.wsSpn astdec/135.121.245.29 3058. (8209)
Thread-2>(01-Dec-05 06:50:50:461) Known Broker 135.121.245.29:6312a:df5ddaf8f3:-8000 WS.spnExter astdec/135.121.245.29 3059. (8209)
Thread-2>(01-Dec-05 06:51:08:929) Known Broker 135.121.245.29:6312a:df5ddb3ae9:-8000 WS.spnInter astdec/135.121.245.29 3060. (8209)
Thread-2>(01-Dec-05 06:51:32:504) Known Broker 135.121.245.29:6312a:df6e325bcf:-8000 WS.wsSpn astdec/135.121.245.29 3058. (8209)
Thread-2>(01-Dec-05 06:51:50:462) Known Broker 135.121.245.29:6312a:df5ddaf8f3:-8000 WS.spnExter astdec/135.121.245.29 3059. (8209)
Thread-2>(01-Dec-05 06:52:08:929) Known Broker 135.121.245.29:6312a:df5ddb3ae9:-8000 WS.spnInter astdec/135.121.245.29 3060. (8209)
Thread-2>(01-Dec-05 06:52:32:506) Known Broker 135.121.245.29:6312a:df6e325bcf:-8000 WS.wsSpn astdec/135.121.245.29 3058. (8209)

please let me you need any other log or any information on my configuration..

i am new bee to this area almost 1yr exp in this language. tell me where to pull the infromation too...

Thanks in Advance..
Raj

 

[Lee Curzon]

I can't help you (at present) with your central (performance/agent) questions - but here's why your Timeout error is probably occurring, although altering it doesn't address your problem, I think:

when i try the same report it take me more 5mins and there is a error along the result saying CGI Timeout.

KB P152
Title: "Low CGI Timeout value causes WebWrite errors (errors 6404) within WebSpeed server log"

http://tinyurl.com/8nkdl


KB 20842
Title: "How to Change CGI Timeout Setting for IIS 4.0 & 5.0 and 6.0?"

http://tinyurl.com/7ggew


HTH

 

[gcampbell]

But Lee, you ignored the actual issue .....

Any reports should be spun off and processed in a batch process. Never run reports interactively on a web platform app.

Later,
Gordon

 

[Lee Curzon]

Gordon,

But Lee, you ignored the actual issue .....

Which I acknowledged:

I can't help you (at present) with your central (performance/agent) questions - but here's why your Timeout error is probably occurring, although altering it doesn't address your problem, I think:

[bold added]

Any reports should be spun off and processed in a batch process. Never run reports interactively on a web platform app.

True for 'normal' reports, but its quite common for web apps to return data 'interactively' if they are quick to process:

Some time same report comes with in 6sec

But I might be misunderstanding you on that point.

However you are correct, Raj's problem is very probably a design/deployment issue, and my knowledgebase pointers didn't have anything to do with that.

Lee

 

[Raj]

gcampbell/lee

Here my main concern is abt the performance issue. the same is working fine for 6mins and the same is taking more than 5 mins. and i agree our timeout is set to 300 sec.

i got the confirmation saying we can user 25 agents.

is that the agents will be created automaticall once the existing are busy?

are it has be set in the configuration saying it should start with x many no of agents attached to the broker?

where to monitor the exact problem?

basically the user retrieves the information of the transaction no ,trans date and last modifed (there is a for loop which builds hyper link on the transaction # )

once the user clicks the link he can modify the transaction.

i just wanted to how many rows we can restrict for the intractive web reports is there any limits are there?

your expertise should help me

Thank you again.

 

[Casper]

There is no thumb rule for maximum numbers of rows to retrieve. It all depends on the time it consumes. Finding one record in a very large table with no index can take ages....

My thumb rule for reports I make (for example with pdfinclude is that a request should take no more time then 500 ms on the slowest server of customers of ours. Within this time it is very good possible to create a descent report.
Reports which (are likely to) take more time are better of running in the batch.

HTH,

Casper.

 

[Raj]

Casper,

good point,

Here my report is good enought to come by 6 sec, why this talking more 5mins some time?

Below is my broker settings on ubroker.properties on unix box. the initialSrvrInstance server instance is two. when it will release the busy agent and when it will create the new agent on need basis?

do i need to explicity specify the maxSrvrInstance = 25?

what could be the reason or requirnment for initializating the new agent?


[UBroker.WS.wsSpn]
initialSrvrInstance=2
userName=
brokerLogFile=$WRKDIR/wsSpn.broker.log
defaultService=0
uuid=135.121.245.29:6312a:df6e325bcf:-8000
appserviceNameList=wsSpn
srvrMaxPort=2219
srvrStartupParam=-p web/objects/web-disp.p -weblogerror -pf /spn/src/web/wsSpn.pf
description=A sample WebSpeed Transaction Server
portNumber=3058
autoStart=0
srvrMinPort=2216
controllingNameServer=NS2
groupName=
srvrLogFile=$WRKDIR/wsSpn.server.log
environment=wsSpn

[UBroker]
hostName=
registrationRetry=60
initialSrvrInstance=3
userName=
brokerLogFile=$WRKDIR/broker.log
maxSrvrInstance=3
PROPATH=${PROPATH}:${WRKDIR}
defaultService=0
jvmArgs=$JVMARGS
brkrLogAppend=1
registrationMode=Register-IP
uuid=
appserviceNameList=
srvrMaxPort=3502
description=
minSrvrInstance=1
portNumber=3050
brkrLoggingLevel=3
autoStart=0
srvrLogAppend=1
autoTrimTimeout=0
infoVersion=9010
srvrMinPort=3202
priorityWeight=0
srvrLoggingLevel=3
controllingNameServer=
workDir=$WRKDIR
groupName=
maxClientInstance=512
srvrLogFile=$WRKDIR/server.log
environment=
srvrStartupTimeout=10
requestTimeout=30
Thank You,
Raj

 

[Casper]

I don't have too much time right now, but looking quick at your configuration I have a few remarks:

• If you have licenties for 25 agents then you should set maxSrvrInstance = 25
• Furhtermore you srvrMinPort and srvrMaxport only has room for 4 agents to start up. You should set that range to the maximum number of agents which you can use.

Another remark is that 6 seconds is IMHO much much too long for a request on a webspeed agent.

As soon as the request is finished, the agent is released.
An agent is started as soon as there are no more available servers to serve the request.

By the way: Do you use Progress Explorer to set up the webspeedbroker? If not I think you should use that.

I only used WebSpeed from V9.1D06 and up and I don't know if there are any issues with WebSpeed in 9.1A which where fixed in later releases.

HTH,

Casper.