Performance issue

We have been experiencing rather lack lustre performance over the past year or so since upgrading our server. During some recent research on Progress Talk I noticed an article pointing out the running the DB in a Raid config is pretty much the worst thing you can do for performance and recommended switching to a pair of mirrored SSDs. We really thought this was our pot of gold fix for our non performance headaches so immedaitely set about installing a pair of SSDs in the server and moving the Progress DB server across from the Raid 5 array.

But.... performance seems just as bad , so now I throw myself on the mercy of you Progress experts for some hopefully sagious advice on where a solution might lie :)

Performance averages show Disk I/O, Network I/O, CPU and Memory usage all consistently in the lowest quartile.

To give some idea of how bad this is, the same application run on an exact copy of the database running on a local Windows PC seems to run at least twice as fast as when run in the production environment over the LAN.

Thanks in advance!

Configuration
VMWare 4.1.0
Progress DB Server running on mirrored SSD drives
Server 2004 Enterprise
4 CPUs allocated
3GB RAM allocated
Progress OpenEdge Release 10.2A build 1390 SP03 on WINNT
Workgroup DB
DB around 4GB in size
After Imaging enabled

.st file
#
b e:\am.b1 f 61440
b e:\am.b2 f 61440
b e:\am.b3 f 61440
b e:\am.b4 f 61440
b e:\am.b5 f 61440
b e:\am.b6 f 61440
b e:\am.b7
#
d "Schema Area":6,32 h:\am.d1
#
a d:\am.a1
#
a d:\am.a2
#
a d:\am.a3
#
d "Data Area":10,32 f:\am_10.d1 f 512000
d "Data Area":10,32 f:\am_10.d2 f 512000
d "Data Area":10,32 f:\am_10.d3 f 512000
d "Data Area":10,32 f:\am_10.d4 f 512000
d "Data Area":10,32 f:\am_10.d5 f 512000
d "Data Area":10,32 f:\am_10.d6 f 512000
d "Data Area":10,32 f:\am_10.d7 f 512000
d "Data Area":10,32 f:\am_10.d8 f 512000
d "Data Area":10,32 f:\am_10.d9
#
a d:\am.a4
#
d "Index Area":20,32 i:\am_20.d1 f 512000
d "Index Area":20,32 i:\am_20.d2 f 512000
d "Index Area":20,32 i:\am_20.d3 f 512000
d "Index Area":20,32 i:\am_20.d4 f 512000
d "Index Area":20,32 i:\am_20.d5
Latest db log file

Fri Dec 30 06:37:52 2011
[2011/12/30@06:37:52.685+1300] P-2420 T-920 I BACKUP : (451) Probkup session begin for administrator on CON:.
[2011/12/30@06:37:52.716+1300] P-2420 T-920 I BACKUP : (5326) Begin Physical Redo Phase at 3008 .
[2011/12/30@06:37:52.763+1300] P-2420 T-920 I BACKUP : (7161) Physical Redo Phase Completed at blk 3131 off 2525 upd 0.
[2011/12/30@06:37:52.778+1300] P-2420 T-920 I BACKUP : (13547) At end of Physical redo, transaction table size is 256.
[2011/12/30@06:37:52.888+1300] P-2420 T-920 I BACKUP : (3777) Switched to ai extent d:\am.a3.
[2011/12/30@06:37:52.888+1300] P-2420 T-920 I BACKUP : (3778) This is after-image file number 3507 since the last AIMAGE BEGIN
[2011/12/30@06:37:52.888+1300] P-2420 T-920 I BACKUP : (6686) 1372026 active blocks out of 1572833 blocks in h:\am will be dumped.
[2011/12/30@06:37:52.888+1300] P-2420 T-920 I BACKUP : (6688) 0 BI blocks will be dumped.
[2011/12/30@06:37:52.903+1300] P-2420 T-920 I BACKUP : (9285) Backup requires an estimated 5.2 GBytes of media.
[2011/12/30@06:37:52.919+1300] P-2420 T-920 I BACKUP : (9286) Restore would require an estimated 1372026 db blocks using 5.2 GBytes of media.
[2011/12/30@06:37:52.919+1300] P-2420 T-920 I BACKUP : (12850) Backup blocks will be written to j:\am-offline.bck.
[2011/12/30@06:37:52.935+1300] P-2420 T-920 I BACKUP : (1362) Full backup started.
[2011/12/30@06:37:52.950+1300] P-2420 T-920 I BACKUP : (5461) Begin backup of Data file(s).
[2011/12/30@06:46:02.792+1300] P-2420 T-920 I BACKUP : (5462) End backup of Data file(s).
[2011/12/30@06:46:03.104+1300] P-2420 T-920 I BACKUP : (13625) Wrote a total of 39142 backup blocks using 5.1 GBytes of media.
[2011/12/30@06:46:03.151+1300] P-2420 T-920 I BACKUP : (1364) Full backup successfully completed.
[2011/12/30@06:46:03.510+1300] P-2420 T-920 I BACKUP : (334) Probkup session end.
Fri Dec 30 13:30:40 2011
[2011/12/30@13:30:40.953+1300] P-2108 T-2112 I BROKER 0: (333) Multi-user session begin.
[2011/12/30@13:30:40.984+1300] P-2108 T-2112 I BROKER 0: (5326) Begin Physical Redo Phase at 3008 .
[2011/12/30@13:30:41.171+1300] P-2108 T-2112 I BROKER 0: (7161) Physical Redo Phase Completed at blk 3131 off 2605 upd 0.
[2011/12/30@13:30:41.171+1300] P-2108 T-2112 I BROKER 0: (13547) At end of Physical redo, transaction table size is 256.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (452) Login by Administrator on CON:.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (5644) Started for amadeus using tcp IPV4 address 0.0.0.0, pid 2108.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4234) Progress OpenEdge Release 10.2A build 1390 SP03 on WINNT .
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4281) Server started by Administrator on CON:.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (6574) Started using pid: 2108.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4235) Physical Database Name (-db): h:\am.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4236) Database Type (-dt): PROGRESS.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4237) Force Access (-F): Not Enabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4238) Direct I/O (-directio): Not Enabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (-----) LRU mechanism enabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4239) Number of Database Buffers (-B): 120000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (9422) Maximum private buffers per user (-Bpmax): 64.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4240) Excess Shared Memory Size (-Mxs): 32.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (10014) The shared memory segment is not locked in memory.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4241) Current Size of Lock Table (-L): 1000000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (13953) Maximum Area Number (-maxArea): 32000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4242) Hash Table Entries (-hash): 35863.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4243) Current Spin Lock Tries (-spin): 1.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (6526) Number of Semaphore Sets (-semsets): 3.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (13924) Maximum Shared Memory Segment Size (-shmsegsize) 1024 Mb.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4244) Crash Recovery (-i): Enabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (6573) Database Blocksize (-blocksize): 4096.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4245) Delay of Before-Image Flush (-Mf): 3.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4247) Before-Image File I/O (-r -R): Reliable.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4249) Before-Image Truncate Interval (-G): 0.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4250) Before-Image Cluster Size: 524288.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4251) Before-Image Block Size: 8192.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4252) Number of Before-Image Buffers (-bibufs): 20.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (-----) Record free chain search depth factor 5 (-recspacesearchdepth)
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (9238) BI File Threshold size (-bithold): 0.0 Bytes.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (6552) BI File Threshold Stall (-bistall): Disabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4254) After-Image Stall (-aistall): Not Enabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4255) After-Image Block Size: 8192.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4256) Number of After-Image Buffers (-aibufs): 20.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (8527) Storage object cache size (-omsize): 1024
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4257) Maximum Number of Clients Per Server (-Ma): 12.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4258) Maximum Number of Servers (-Mn): 5.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4259) Minimum Clients Per Server (-Mi): 1.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4260) Maximum Number of Users (-n): 51.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4261) Host Name (-H): nzam-prog.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4262) Service Name (-S): amadeus.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (14268) TCP/IP Version (-ipver) : IPV4
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4263) Network Type (-N): tcp.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4264) Character Set (-cpinternal): ISO8859-1.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4282) Parameter File: Not Enabled.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (5648) Minimum Port for Auto Servers (-minport): 3000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (5649) Maximum Port for Auto Servers (-maxport): 5000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (8865) This broker supports both 4GL and SQL server groups.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (9336) Created shared memory with segment_id: 10092544
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12813) Allowed index cursors (-c): 204.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12814) Group delay (-groupdelay): 10.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12815) Lock table hash table size (-lkhash): 137743
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12816) Maxport (-maxport): 5000
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12817) Minport (-minport): 3000
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12818) Message Buffer Size (-Mm): 4096
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12821) Use muxlatches (-mux): 1
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12823) Semaphore Sets (-semsets): 3
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (13870) Database Service Manager - IPC Queue Size (-pica) : 64.0 KBytes.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (13896) TXE Commit lock skip limit (-TXESkipLimit): 10000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (10471) Database connections have been enabled.
 

cj_brandt

Active Member
You can find some info on progresstalk or other places about workgroup and a performance issue with servers that have multiple cores or cpu's. Not sure if that is your issue or not. I have never used workgroup so I don't follow that info very closely.

What exactly is slow - reads or writes or both ? Can you reproduce the slowness on demand ? I would watch promon screen 5 to see db activity during slowdown - may show if you are performing a large amount of reads from disk.
If you have memory available on your server - I would suggest increasing the -B from 120,000 to 300,000. Help reduce disk reads.
Do you use Windows Performance Monitor and do you use it to monitor the physical disks ? What is the Disk Queue length for the 2 disks that the database is on ? Where are the client's temp files located ?
Do clients connect over the network ? If the same job runs on a client's machine or on the server is there a large performance difference ?
What processes are using the most CPU time on the server when the slowdown occurs ? It isn't some virus scanner right ?

Well that should be enough to get you started.
 

TomBascom

Curmudgeon
We have been experiencing rather lack lustre performance over the past year or so since upgrading our server.

It is always good to specify what sorts of things illustrate a performance problem. The specific can go a long ways towards identifying a solution.

During some recent research on Progress Talk I noticed an article pointing out the running the DB in a Raid config is pretty much the worst thing you can do for performance

RAID 5 or it's evil step-brother RAID 6. Or any flavor of parity based RAID (because they provide a pardoy of performance.)

And most especially any RAID that a vendor invents -- particularly those which contain letters. Like "RAID S" or "RAID DP". When a vendor starts making up lettered RAID levels they are selling snake oil.

See http://www.baarf.com/ for details.

and recommended switching to a pair of mirrored SSDs.

I plead guilty.

We really thought this was our pot of gold fix for our non performance headaches so immedaitely set about installing a pair of SSDs in the server and moving the Progress DB server across from the Raid 5 array.

But.... performance seems just as bad

Perhaps I wasn't clear -- SSDs are a miraculous solution to an IO problem. If you aren't bottlenecked on IO they won't make much difference.

, so now I throw myself on the mercy of you Progress experts for some hopefully sagious advice on where a solution might lie :)

As I mentioned above a bit more detail on what it is that embodies the problem would be helpful. Some specific measurements of specific tasks would go a long ways.

Performance averages show Disk I/O, Network I/O, CPU and Memory usage all consistently in the lowest quartile.

A wise man once said "trust, but verify". So what metrics are you looking at that tell you this? Averages over what period of time?

To give some idea of how bad this is, the same application run on an exact copy of the database running on a local Windows PC seems to run at least twice as fast as when run in the production environment over the LAN.

If you are running it as a shared memory connection locally and as a client/server connection over the LAN then you already have your answer. Even if it is client/server on the local machine a local connection would be expected to be quite a lot faster than running traffic over a network connection.

Thanks in advance!

You're welcome :)

Configuration
VMWare 4.1.0
Progress DB Server running on mirrored SSD drives
Server 2004 Enterprise

Why not Windows Server 2008?

4 CPUs allocated
3GB RAM allocated

Can you actually boot windows in 3GB? ;)

That seems like a very small amount of RAM.

Progress OpenEdge Release 10.2A build 1390 SP03 on WINNT

32 bit?

All database servers should be running 64 bit Progress on 64 bit operating systems.

Workgroup DB

Your performance tuning options are severely constrained.

The workgroup database is not intended for demanding workloads. You lack many important tools for improving performance. The most important of which are -spin and asynchronous page writers. You cannot expect to have excellent performance while sticking to workgroup.

Workgroup also has big problems running on multi-CPU systems. These were partially fixed with 10.1B but it is still an issue under load. It is unlikely that that will ever change -- WG isn't intended to be used where performance is critical.

DB around 4GB in size

Get to 64 bits and the whole thing will fit in -B :)

After Imaging enabled

Excellent!

.st file
#
b e:\am.b1 f 61440
b e:\am.b2 f 61440
b e:\am.b3 f 61440
b e:\am.b4 f 61440
b e:\am.b5 f 61440
b e:\am.b6 f 61440
b e:\am.b7

This is silly. Make a single fixed bi extent plus a variable extent. Or just go with a single variable extent. There is no practical use for 6 tiny bi extents.
#
d "Schema Area":6,32 h:\am.d1

Excellent! No data in the schema area! +1

#
a d:\am.a1
#
a d:\am.a2
#
a d:\am.a3

3 after-image extents is a suspiciously small number. You should always have at least 4 and I prefer 8.

Oops, just found the 4th one. Ok, you've got 4.

+1 that they are all variable.

+1 that they are on a different drive than the data.

#
d "Data Area":10,32 f:\am_10.d1 f 512000
d "Data Area":10,32 f:\am_10.d2 f 512000
d "Data Area":10,32 f:\am_10.d3 f 512000
d "Data Area":10,32 f:\am_10.d4 f 512000
d "Data Area":10,32 f:\am_10.d5 f 512000
d "Data Area":10,32 f:\am_10.d6 f 512000
d "Data Area":10,32 f:\am_10.d7 f 512000
d "Data Area":10,32 f:\am_10.d8 f 512000
d "Data Area":10,32 f:\am_10.d9
#
a d:\am.a4
#
d "Index Area":20,32 i:\am_20.d1 f 512000
d "Index Area":20,32 i:\am_20.d2 f 512000
d "Index Area":20,32 i:\am_20.d3 f 512000
d "Index Area":20,32 i:\am_20.d4 f 512000
d "Index Area":20,32 i:\am_20.d5

You have 2 storage areas. That's a start but:

1) They are type 1 areas. All data and indexes should be in type 2 areas. This is a small db so a cluster size of 8 is probably fine if you don't want to work out better values.

2) 32 rows per block? You're probably wasting a lot of space and thus your IO ops are "fluffy" and you -B has a lot of air in it.

3) I would run a dbanalys and break this out into a lot more storage areas. See Storage Optimization Strategies for some ideas.

I'm going to strip the noise out of the .lg file...

Latest db log file

Fri Dec 30 06:37:52 2011
[2011/12/30@06:37:52.685+1300] P-2420 T-920 I BACKUP : (451) Probkup session begin for administrator on CON:.

You do backups +1

Is there a reason that you do them offline vs online?

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (6573) Database Blocksize (-blocksize): 4096.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4239) Number of Database Buffers (-B): 120000.

Seems low. 120,000 4k blocks is only 480MB

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4241) Current Size of Lock Table (-L): 1000000.

Wow! That's high. Is there really a need for that?

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4243) Current Spin Lock Tries (-spin): 1.

This is a WG database so that cannot be changed. But if it could be that would be something to target. I'd go with 5,000 for starters.

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4250) Before-Image Cluster Size: 524288.

Since this is a WG database -- if your performance problem is that checkpoints are taking too long then you could try decreasing this. As a result you would have more, but shorter, checkpoints. Right now we have no evidence that you have such a problem -- so don't change it.

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4257) Maximum Number of Clients Per Server (-Ma): 12.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4258) Maximum Number of Servers (-Mn): 5.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4259) Minimum Clients Per Server (-Mi): 1.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (4260) Maximum Number of Users (-n): 51.

How many users do you really have? (This is set up for 60 although -n says 51.)

You are running client/server connections with a default configuration. That's probably a bad idea.

I prefer to have no more than 2 users sharing a server - 4 or 5 at the very most. And I've been known to go with a 1 to 1 mapping. For 60 users and 2:1 use -Mn 30 -Ma 2

-Mi 1 is fine -- that means "round robin".

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (5648) Minimum Port for Auto Servers (-minport): 3000.
[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (5649) Maximum Port for Auto Servers (-maxport): 5000.

+1

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (8865) This broker supports both 4GL and SQL server groups.

Are you allowing any SQL-92 access?

[2011/12/30@13:30:41.421+1300] P-2108 T-2112 I BROKER 0: (12818) Message Buffer Size (-Mm): 4096

I would strongly consider doubling this. If you change it for the db you must also change it for the clients. Everything that connects to this db client/server must have the same value.

Without specific problems this advice is all generic. So don't expect miracles ;)

BTW, I'd be happy to visit NZ and explore it in detail :)
 
Tom, thanks for the advice, could we contract your services remotely to investigate this in detail and provide us a recommended resolution path going forward. Note quite as good as being here in NZ in person, though frankly you wouldn't want to be here at the moment anyway as the summer weather is plain awful! Email me directly at chris DOT rutter AT amadeusnz DOT co DOT nz if it's easier. Thanks Tom. Chris
 
Thanks for the great advice, quite a bit to check out but as you say, enough to get started! Thanks again, I will report back on results.
 
Do you use Windows Performance Monitor and do you use it to monitor the physical disks ? What is the Disk Queue length for the 2 disks that the database is on ?
F: data extents
I: index extents

Capture.gif
As you can see the processors are hardly ticking over...
Capture2.PNG
Where are the client's temp files located ?
c:\temp
Do clients connect over the network ?
Yes
If the same job runs on a client's machine or on the server is there a large performance difference ?
Huge difference. On client's machine is usually at least twice as fast as the same job on the server.
What processes are using the most CPU time on the server when the slowdown occurs ? It isn't some virus scanner right ?

Well that should be enough to get you started.
 
Some initial performance monitoring results:

Promon shows Buffer Hits in the 99-100% range. Never drops below 98%.

Windows Performance Monitor on DB Server
Disks 99% idle
Processors max out at 19.5%
Physical Disks Current Queue Length 0. Does not seem to move from 0.

VMWare Performance Monitor on DB Server
CPU% < 7.5%
CPU Usage Mhz < 750
Memory% 5-35%
Disk KBps < 2500
Network MBps < 15

As a footnote, lack lustre performance is across the board, no particular process or table stands out. Everything is s l u g g i s h...
 
Top