Hi Casper,
Please find the answers below. Thanks for your time.
1- which platform you work (on windows probably regarding the progress.exe)? Java version?
***** App Server and Web Server are in Windows and Database in Linux.
2- why do you use 5 brokers?
***** We have configured it that way to have dedicated brokers for certain applications.
3- What changes are made to the report?
***** Along with existing data, we have added few rows and columns of data. Technically, from the same table few more columns were fetched for data. This table will have close to 2 million records and is growing.
4- Where is the database residing (OS plus OS-level)
***** Database is residing in Linux server.
5- which SP level do you have with Progress?
***** What do you mean by this?
6- Are messenger and WTA on the same machine?
***** They are in the same machine.
7- Do you use IIS or Apache?
***** We use IIS.
9- Is there a certain point in the report where this happens?
***** We tried logging the time taken for the report and messages to track the point where the error comes. But unfortunately could not make out anything. A report taken for a specific criteria has gone through well at a point of time, where as the same report has taken triple the time at a different time.(at the time the server went down).
10- If in your test environment you generate many reports at the same time, what happens?
***** In test environment, the report gets over without any issues. If more requests are given than the available agents, then the excess requests are put in queue and processed subsequently.
11-Do you have webspeed configured as stateless?
***** Yes, we have configured the operating mode as Stateless.
12- Do you use shared variables?
***** Yes we used shared variables.
13- Does the broker hang or do the agents hang?
***** Agents hang and at that time, we could see many Progress.exe processes running with 100% CPU usage.
14- anything in the log files?
***** No help with logs.
15- Are there shared locks involved? Did you check with promon (with -F option perhaps: record locking table)?
***** As this is a report, we have looped in with NO-LOCK only.