How did you know about our work on the mersenne prime? ;p

Actually, the Oracle DB that our middleware utilizes is on an RX5670 (Itanium) box and that box is between 80-100% idle. So I think that Oracle is, expectedly, well behaved and the Itanium performs well. I've also been looking at the %CPU figure for processes and nothing seems to be grabbing a lot of CPU. What we do have is a lot of processes (over half [800+] are for the client PC connections). But they still only utilize a small portion of CPU (0.02% typically) so I'm not seeing anything that adds up to the high user CPU percentage figures I'm seeing. I've used 'ps -eo user,args,pcpu' there don't appear to be any major CPU hogs. The largest processes take up about 14-20% CPU but only briefly throughout the day.

Now that I look at things a little deeper though... Where I think the issue lies is in the architecture of the middleware and how it uses RAM. Looking at the RAM used by all process images for the middleware user account, it is constantly over the amount of physical RAM in the box. The vendor was particularly freaked out when they realized how large our client data set is which affects the size of lookup tables that they run for each process. It looks like we have (at this moment):

857 processes using 6M each ~10 Gigs
159 processes at 1.1M each ~180 Megs
43 processes at about 150M each ~6.5 Gigs
14 processes at about 111M each ~1.2 Gigs
Total = ~17 to 18 Gigs

Our box is maxed at 16 Gigs of physical RAM right now. So it would appear that we must be swapping a bit. When I tried a total of the vsz output of ps for the application user account, I saw 36 Gigs worth of process RAM usage. So... maybe the CPU user/sys/idle percentages are more a reflection of RAM usage? Does this sound like a possibility? I've already checked for i/o bottlenecks on our disk array and have cleared that as a possible problem point.


I believe we've populated the box's memory slots fully, so I don't think there is much we can do there until we get a bigger box.