Login or Register to Ask a Question and Join Our Community


AIX

Any Additional Steps After Adding New RAM To Sever?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Operating Systems AIX Any Additional Steps After Adding New RAM To Sever?
# 1  
Old 05-25-2011
Any Additional Steps After Adding New RAM To Sever?
Hi All,

We have a server at a client site running AIX 5.3, which we just up the RAM to 32GB, from initially 16GB (if I'm not mistaken).

This server is our Application server running J2EE applications on top of Oracle Internet Application Server. Recently we encountered one of the batch jobs hitting out of memory error.

From NMON I see that on average, there's 5GB of free memory available. %MAXPERM is set to '15' and %MINPERM is '5'. lru_file_repage is '0'. %comp is around '70' and %numperm is '15'.

I would like to ask
a) after adding real memory to the server, is there any additional changes we need to do to the VMO?
b) is the %MAXPERM too low?

Thanks.
# 2  
Old 05-25-2011
a)
Usually not - many parameters regarding memory are percentage based. Others have fixed values. When doubling the memory it should usually run smoother than before if there was a memory shortage or no tuning.

b)
Not easy to say this way. When using lru_file_repage=0, it is recommended to have maxperm% back to 80 which is default and depending on memory usage, some do even set 85 or 90.
A box running out of memory will usually start to page out, before anything crashes, and sowith become very slow. I heared once that Java dislikes to be paged out - not sure if this is true or still a problem though. It could also be that you can increase memory usage inside your application server (I am not very familiar with java but heapsize stuff etc. come to my mind), especially after having added real memory.

Post a vmstat -vs and a svmon -G. It could also be, that the ulimits for the user where this application is running, has too tight limits. A ulimit -a issued from that user will show the current limits.

That as a start.
# 3  
Old 05-25-2011
What does the memory error look like?
# 4  
Old 05-26-2011
Quote:
Originally Posted by garethr
What does the memory error look like?
The error looks like below
Code:
01:37:30:911   java.lang.OutOfMemoryError
	at oracle.jdbc.driver.OracleStatement.prepareAccessors(OracleStatement.java(Compiled Code))
	at oracle.jdbc.driver.OracleStatement.execute_maybe_describe(OracleStatement.java(Compiled Code))
	at oracle.jdbc.driver.T4CPreparedStatement.execute_maybe_describe(T4CPreparedStatement.java(Compiled Code))
	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java(Compiled Code))
	at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java(Compiled Code))
	at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java(Compiled Code))
	at sqlj.runtime.ExecutionContext$OracleContext.oracleExecuteQuery(ExecutionContext.java(Inlined Compiled Code))

---------- Post updated at 11:39 AM ---------- Previous update was at 11:36 AM ----------

@zaxxon: below are the output of the commands you suggested me to run

Code:
$vmstat -vs
           6773583119 total address trans. faults
             85402374 page ins
            179082270 page outs
                    0 paging space page ins
                    0 paging space page outs
                    0 total reclaims
           1568460598 zero filled pages faults
            437557684 executable filled pages faults
           1598879605 pages examined by clock
                  247 revolutions of the clock hand
            176110235 pages freed by the clock
             29164084 backtracks
                 1067 free frame waits
                    0 extend XPT waits
              6849007 pending I/O waits
            264484781 start I/Os
             56911924 iodones
          43584605859 cpu context switches
          15545396044 device interrupts
           1335402576 software interrupts
           3604933076 decrementer interrupts
               212611 mpc-sent interrupts
               212611 mpc-receive interrupts
             19173376 phantom interrupts
                    0 traps
         101258980092 syscalls
              8134656 memory pages
              7786454 lruable pages
              1862642 free pages
                    1 memory pools
              1772455 pinned pages
                 80.0 maxpin percentage
                  5.0 minperm percentage
                 15.0 maxperm percentage
                 14.9 numperm percentage
              1166944 file pages
                  0.0 compressed percentage
                    0 compressed pages
                 14.9 numclient percentage
                 15.0 maxclient percentage
              1166944 client pages
                    0 remote pageouts scheduled
                19863 pending disk I/Os blocked with no pbuf
                    0 paging space I/Os blocked with no psbuf
                 2228 filesystem I/Os blocked with no fsbuf
                   47 client filesystem I/Os blocked with no fsbuf
                 8856 external pager filesystem I/Os blocked with no fsbuf

$ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        4194304
memory(kbytes)       32768
coredump(blocks)     unlimited
nofiles(descriptors) 2000

#svmon -G
               size      inuse       free        pin    virtual
memory      8134656    6435521    1699135    1772857    5254449
pg space    3276800      13467

               work       pers       clnt      other
pin         1499599          0          0     273258
in use      5254449          0    1181072

PageSize   PoolSize      inuse       pgsp        pin    virtual
s   4 KB          -    6315233      13467    1674265    5134161
m  64 KB          -       7518          0       6162       7518

Last edited by a_sim; 05-26-2011 at 02:59 AM..
# 5  
Old 05-26-2011
Your box has plenty of memory, I'd say too much Smilie

This looks like a Java memory error. Maybe check with your application server if you can set parameters for memory pools/heapsize (whatever there is and they are called) etc. to a higher value. This does not look like a problem with the OS.

Here is a general link about that java error:
Unveiling the java.lang.Out OfMemoryError | JAVA Developer's Journal

Best open up a call via Metalink and/or browse OTN or open up a thread there additionally.
# 6  
Old 05-26-2011
Quote:
Originally Posted by a_sim
Code:
$ulimit -a
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        4194304
memory(kbytes)       32768
coredump(blocks)     unlimited
nofiles(descriptors) 2000

Hmm... Is there a reason why memory usage for the application is limited?

Looking at the other data it seems to be quite ok, but the tuning parameters would be interesting to know to effectively assess the situation. Please post the output of "vmo -a", "ioo -a" and "schedo -a".

The numbers in the "vmstat -vs" output are collected since last reboot. What was the uptime of the system when you issued the command? Would it be possible to reboot the system to start over these statistics?

I hope this helps.

bakunin
# 7  
Old 05-26-2011
Quote:
memory(kbytes) 32768
Good point - I should sometimes read what my eyes see Smilie If possible and nothing else running on this box, set this for the user to -1 (unlimited) and logoff/logon and restart the application with the new limits. That could be the problem indeed.
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

LAMP Sever

How do I install a LAMP server on a new installation of Debian 9 using the lalest versions of AMP? Here is what I have. Corrections please. MYSQL apt-get install mysql-server mysql-client You can verify the MySQL server status using command: systemctl status mysql ------- PHP7... (0 Replies)
Discussion started by: Meow613
0 Replies

2. Red Hat

Adding Additional Capacity with megacli

I'm trying to add 6 more hard drives to my RAID array, none of the drives are foreign, they won't be replacing any drives either. I just need to add them to the RAID array. I can't seem to get them added, what am I missing? ---------- Post updated 08-03-12 at 12:28 PM ---------- Previous... (0 Replies)
Discussion started by: eccentricson
0 Replies

3. Shell Programming and Scripting

Adding an additional blank field to a file

Hi, I have the following file, I'd like to add an additional blank field to this file This is a tab delimited file, I have tried the same thing on excel, but looking for a unix solution. Here is my input: Country Postal Admin4 StreetBaseName StreetType HUN 2243 Kóka Dózsa György ... (3 Replies)
Discussion started by: ramky79
3 Replies

4. Shell Programming and Scripting

Adding additional characters while preserving original text

Hi Forum. I'm struggling on this relatively easy request to add additional 4 0's to an existing text in a file (whenever I see the pattern -# where # represents any number) using sed command while preserving the rest of the text in the files. Original Text: $DBConnection_EDW=SAS2EDW... (5 Replies)
Discussion started by: pchang
5 Replies

5. Shell Programming and Scripting

Adding additional column at the end

I have a file like below. The separator between reconds is ">" Each record consists of 2 numbers separated by a space. I want to write an awk script that copies the second number and puts it in the third column. :rolleyes: > 10 0 13 5.92346 16 10.3106 19 13.9672 22 16.9838 25... (5 Replies)
Discussion started by: kristinu
5 Replies

6. Red Hat

red hat Linux 5.0 is detecting 3gb ram but physical ram is 16gb

Hi, On server 64bit Hw Arch , Linux 5.0(32bit) is installed it is showing only 3gb of ram though physical is 16gb can u give me idea why? (4 Replies)
Discussion started by: manoj.solaris
4 Replies

7. UNIX for Advanced & Expert Users

adding additional drive to sco the is xenix

I am taking an old xenix drive and installing it in a recent SCO Build Server. I have gone through the process of running mkdev hd twice since the drive is a SCSI then proceed to run mkdev fs and when I attempt to add one of the shown partitions I get the following: fsck: cannot determine... (1 Reply)
Discussion started by: justenglabs
1 Replies

8. UNIX for Dummies Questions & Answers

Adding an additional harddrive in solaris 9

Hello, I have a system which a new harddrive was installed for additional space. I now need to mount the drive and transfer data from /home to the new drive with a mount point named /home. How do I go about doing this? Thanks in advance. (5 Replies)
Discussion started by: GLJ@USC
5 Replies
Login or Register to Ask a Question