http://2.bp.blogspot.com/-twY6HY3lXx...0/Untitled.jpg
Other parameters like
* AIO enable
* AIXTHREAD_SCOPE=S
*
all have been enabled and oracle documentation has been followed
Code
lsattr -El aio0
autoconfig available STATE to be configured at system restart True
fastpath enable State of fast path True
kprocprio 39 Server PRIORITY True
maxreqs 4096 Maximum number of REQUESTS True
maxservers 10 MAXIMUM number of servers per cpu True
minservers 1 MINIMUM number of servers True
root@clodb:/>ps -kf | grep aio
root 86172 1 0 14:00:14 - 0:00 posix_aioserver
root 90168 1 0 14:00:14 - 0:00 aioserver
root 135390 1 0 14:08:42 - 0:00 aioserver
root 282784 1 0 14:08:45 - 0:00 aioserver
root 319648 1 0 14:08:42 - 0:00 aioserver
root 323742 1 0 14:08:42 - 0:00 aioserver
root 331944 1 0 14:08:45 - 0:00 aioserver
root 336036 1 0 14:08:46 - 0:00 aioserver
root 340134 1 0 14:08:46 - 0:00 aioserver
root 344232 1 0 14:08:46 - 0:00 aioserver
root 348330 1 0 14:08:46 - 0:00 aioserver
root 352428 1 0 14:08:47 - 0:00 aioserver
root 356526 1 0 14:08:47 - 0:00 aioserver
root 360624 1 0 14:08:47 - 0:00 aioserver
root 364722 1 0 14:08:47 - 0:00 aioserver
root 368820 1 0 14:08:47 - 0:00 aioserver
root 372918 1 0 14:08:47 - 0:00 aioserver
root 377016 1 0 14:08:47 - 0:00 aioserver
root 381114 1 0 14:08:47 - 0:00 aioserver
root 385212 1 0 14:08:47 - 0:00 aioserver
root 389310 1 0 14:08:48 - 0:00 aioserver
root@clodb:/>iostat -A
System configuration: lcpu=2 drives=3 ent=0.10
aio: avgc avfc maxg maif maxr avg-cpu: % user % sys % idle % iowait physc % entc
0 0 945 0 4096 8.8 0.1 36.8 54.4 0.0 9.2
" Disk history since boot not available. "
in the above you can see maxreqs= 4096
Code
root@clodb:/>lsattr -El aio0 -a maxreqs
maxreqs 4096 Maximum number of REQUESTS True
Before running the report:
Code
root@clodb:/>vmstat -v
2097152 memory pages
2017797 lruable pages
1704079 free pages
2 memory pools
117776 pinned pages
80.0 maxpin percentage
5.0 minperm percentage
90.0 maxperm percentage
0.4 numperm percentage
9366 file pages
0.0 compressed percentage
0 compressed pages
0.9 numclient percentage
90.0 maxclient percentage
18919 client pages
0 remote pageouts scheduled
0 pending disk I/Os blocked with no pbuf
0 paging space I/Os blocked with no psbuf
2740 filesystem I/Os blocked with no fsbuf
10349 client filesystem I/Os blocked with no fsbuf
0 external pager filesystem I/Os blocked with no fsbuf
After running the report Parts Potential Report - Order Management
Code
root@clodb:/>vmstat -v
2097152 memory pages
2017797 lruable pages
1695656 free pages
2 memory pools
118544 pinned pages
80.0 maxpin percentage
5.0 minperm percentage
90.0 maxperm percentage
0.5 numperm percentage
11262 file pages
0.0 compressed percentage
0 compressed pages
1.0 numclient percentage
90.0 maxclient percentage
21501 client pages
0 remote pageouts scheduled
0 pending disk I/Os blocked with no pbuf
0 paging space I/Os blocked with no psbuf
2740 filesystem I/Os blocked with no fsbuf
10349 client filesystem I/Os blocked with no fsbuf
0 external pager filesystem I/Os blocked with no fsbuf
root@clodb:/>
root@clodb:/>svmon -G
size inuse free pin virtual
memory 2097152 401985 1695167 118642 380315
pg space 131072 2627
work pers clnt lpage
pin 118642 0 0 0
in use 380315 0 21670 0
After running the report topas shows
Code
Topas Monitor for host: clodb EVENTS/QUEUES FILE/TTY
Mon Jan 27 16:05:45 2014 Interval: 2 Cswitch 649 Readch 1740.2K
Syscall 2005 Writech 14514
Kernel 1.0 |# | Reads 241 Rawin 0
User 94.5 |########################### | Writes 30 Ttyout 122
Wait 4.5 |## | Forks 0 Igets 0
Idle 0.0 | | Execs 0 Namei 0
Physc = 1.00 %Entc= 999.3 Runqueue 1.0 Dirblk 0
Waitqueue 0.0
Network KBPS I-Pack O-Pack KB-In KB-Out
en0 34.5 155.0 182.0 43.9 25.9 PAGING MEMORY
lo0 3.4 84.0 84.0 3.5 3.5 Faults 3 Real,MB 8191
Steals 0 % Comp 19.1
Disk Busy% KBPS TPS KB-Read KB-Writ PgspIn 0 % Noncomp 1.0
hdisk7 100.0 1272.0 169.0 1496.0 1048.0 PgspOut 0 % Client 1.5
hdisk6 0.0 0.0 0.0 0.0 0.0 PageIn 0
hdisk8 0.0 0.0 0.0 0.0 0.0 PageOut 0 PAGING SPACE
Sios 0 Size,MB 512
Name PID CPU% PgSp Owner % Used 2.5
rmcd 123114 50.0 10.0 root NFS (calls/sec) % Free 97.4
oracle 680012 0.0 5.2 oraprod ServerV2 0
dtterm 671820 0.0 1.3 root ClientV2 0 Press:
topas 520302 0.0 1.6 root ServerV3 0 "h" for help
oracle 495862 0.0 4.3 oraprod ClientV3 0 "q" to quit
VMO VMM Parameters optimized to the max
Code
root@clodb:/>vmo -a
cpu_scale_memp = 8
data_stagger_interval = 161
defps = 1
force_relalias_lite = 0
framesets = 2
htabscale = n/a
kernel_heap_psize = 4096
large_page_heap_size = 0
lgpg_regions = 0
lgpg_size = 0
low_ps_handling = 1
lru_file_repage = 0
lru_poll_interval = 0
lrubucket = 131072
maxclient% = 90
maxfree = 1088
maxperm = 1816016
maxperm% = 90
maxpin = 1677722
maxpin% = 80
mbuf_heap_psize = 4096
memory_affinity = 1
memory_frames = 2097152
memplace_data = 2
memplace_mapped_file = 2
memplace_shm_anonymous = 2
memplace_shm_named = 2
memplace_stack = 2
memplace_text = 2
memplace_unmapped_file = 2
mempools = 1
minfree = 960
minperm = 100889
minperm% = 5
nokilluid = 0
npskill = 1024
npsrpgmax = 8192
npsrpgmin = 6144
npsscrubmax = 8192
npsscrubmin = 6144
npswarn = 4096
num_spec_dataseg = 0
numpsblks = 131072
page_steal_method = 1
pagecoloring = n/a
pinnable_frames = 1978524
pta_balance_threshold = n/a
relalias_percentage = 0
rpgclean = 0
rpgcontrol = 2
scrub = 0
scrubclean = 0
soft_min_lgpgs_vmpool = 0
spec_dataseg_int = 512
strict_maxclient = 1
strict_maxperm = 0
v_pinshm = 0
vm_modlist_threshold = -1
vmm_fork_policy = 1
root@clodb:/>
root@clodb:/>vmo -L
NAME CUR DEF BOOT MIN MAX UNIT TYPE
DEPENDENCIES
--------------------------------------------------------------------------------
cpu_scale_memp 8 8 8 1 64 B
--------------------------------------------------------------------------------
data_stagger_interval 161 161 161 0 0 4KB pages D
lgpg_regions
--------------------------------------------------------------------------------
defps 1 1 1 0 1 boolean D
--------------------------------------------------------------------------------
force_relalias_lite 0 0 0 0 1 boolean D
--------------------------------------------------------------------------------
framesets 2 2 2 1 10 B
--------------------------------------------------------------------------------
htabscale n/a -1 -1 -4 0 B
--------------------------------------------------------------------------------
kernel_heap_psize 4K 4K 4K 4K 16M bytes B
lgpg_regions
--------------------------------------------------------------------------------
large_page_heap_size 0 0 0 0 8E-1 bytes B
lgpg_regions
--------------------------------------------------------------------------------
lgpg_regions 0 0 0 0 D
lgpg_size
--------------------------------------------------------------------------------
lgpg_size 0 0 0 0 16M bytes D
lgpg_regions
--------------------------------------------------------------------------------
low_ps_handling 1 1 1 1 2 D
--------------------------------------------------------------------------------
lru_file_repage 0 1 0 0 1 boolean D
--------------------------------------------------------------------------------
lru_poll_interval 0 0 0 0 60000 milliseconds D
--------------------------------------------------------------------------------
lrubucket 128K 128K 128K 64K 4KB pages D
--------------------------------------------------------------------------------
maxclient% 90 80 90 1 100 % memory D
maxperm%
--------------------------------------------------------------------------------
maxfree 1088 1088 1088 8 200K 4KB pages D
minfree
memory_frames
--------------------------------------------------------------------------------
maxperm 1773K 1773K S
--------------------------------------------------------------------------------
maxperm% 90 80 90 1 100 % memory D
minperm%
maxclient%
--------------------------------------------------------------------------------
maxpin 1638K 1638K S
--------------------------------------------------------------------------------
maxpin% 80 80 80 1 99 % memory D
pinnable_frames
memory_frames
--------------------------------------------------------------------------------
mbuf_heap_psize 4K 4K 4K 4K 16M bytes B
--------------------------------------------------------------------------------
memory_affinity 1 1 boolean d
memplace_text
memplace_data
memplace_stack
memplace_shm_named
memplace_shm_anonymous
memplace_mapped_file
memplace_unmapped_file
--------------------------------------------------------------------------------
memory_frames 2M 2M 4KB pages S
--------------------------------------------------------------------------------
memplace_data 2 2 2 1 2 D
--------------------------------------------------------------------------------
memplace_mapped_file 2 2 2 1 2 D
--------------------------------------------------------------------------------
memplace_shm_anonymous 2 2 2 1 2 D
--------------------------------------------------------------------------------
memplace_shm_named 2 2 2 1 2 D
--------------------------------------------------------------------------------
memplace_stack 2 2 2 1 2 D
--------------------------------------------------------------------------------
memplace_text 2 2 2 1 2 D
--------------------------------------------------------------------------------
memplace_unmapped_file 2 2 2 1 2 D
--------------------------------------------------------------------------------
mempools 1 1 1 1 2 B
--------------------------------------------------------------------------------
minfree 960 960 960 8 200K 4KB pages D
maxfree
memory_frames
--------------------------------------------------------------------------------
minperm 100889 100889 S
--------------------------------------------------------------------------------
minperm% 5 20 5 1 100 % memory D
maxperm%
--------------------------------------------------------------------------------
nokilluid 0 0 0 0 4G-1 uid D
--------------------------------------------------------------------------------
npskill 1K 1K 1K 1 128K-1 4KB pages D
--------------------------------------------------------------------------------
npsrpgmax 8K 8K 8K 0 128K-1 4KB pages D
npsrpgmin
--------------------------------------------------------------------------------
npsrpgmin 6K 6K 6K 0 128K-1 4KB pages D
npsrpgmax
--------------------------------------------------------------------------------
npsscrubmax 8K 8K 8K 0 128K-1 4KB pages D
npsscrubmin
--------------------------------------------------------------------------------
npsscrubmin 6K 6K 6K 0 128K-1 4KB pages D
npsscrubmax
--------------------------------------------------------------------------------
npswarn 4K 4K 4K 0 128K-1 4KB pages D
--------------------------------------------------------------------------------
num_spec_dataseg 0 0 0 0 B
--------------------------------------------------------------------------------
numpsblks 128K 128K 4KB pages S
--------------------------------------------------------------------------------
page_steal_method 1 1 1 0 1 boolean B
--------------------------------------------------------------------------------
pagecoloring n/a 0 0 0 1 boolean B
--------------------------------------------------------------------------------
pinnable_frames 1932K 1932K 4KB pages S
--------------------------------------------------------------------------------
pta_balance_threshold n/a 1 1 0 99 % pta segment D
--------------------------------------------------------------------------------
relalias_percentage 0 0 0 0 32K-1 D
--------------------------------------------------------------------------------
rpgclean 0 0 0 0 1 boolean D
--------------------------------------------------------------------------------
rpgcontrol 2 2 2 0 3 D
--------------------------------------------------------------------------------
scrub 0 0 0 0 1 boolean D
--------------------------------------------------------------------------------
scrubclean 0 0 0 0 1 boolean D
--------------------------------------------------------------------------------
soft_min_lgpgs_vmpool 0 0 0 0 90 % D
lgpg_regions
--------------------------------------------------------------------------------
spec_dataseg_int 512 512 512 0 B
--------------------------------------------------------------------------------
strict_maxclient 1 1 1 0 1 boolean D
--------------------------------------------------------------------------------
strict_maxperm 0 0 0 0 1 boolean D
--------------------------------------------------------------------------------
v_pinshm 0 0 0 0 1 boolean D
--------------------------------------------------------------------------------
vm_modlist_threshold -1 -1 -1 -2 2G-1 D
--------------------------------------------------------------------------------
vmm_fork_policy 1 1 1 0 1 boolean D
--------------------------------------------------------------------------------
n/a means parameter not supported by the current platform or kernel
Parameter types:
S = Static: cannot be changed
D = Dynamic: can be freely changed
B = Bosboot: can only be changed using bosboot and reboot
R = Reboot: can only be changed during reboot
C = Connect: changes are only effective for future socket connections
M = Mount: changes are only effective for future mountings
I = Incremental: can only be incremented
d = deprecated: deprecated and cannot be changed
Value conventions:
K = Kilo: 2^10 G = Giga: 2^30 P = Peta: 2^50
M = Mega: 2^20 T = Tera: 2^40 E = Exa: 2^60
root@clodb:/>
root@clodb:/>vmo -L minperm% -L maxperm% -L maxclient% -L lru_file_repage
NAME CUR DEF BOOT MIN MAX UNIT TYPE
DEPENDENCIES
--------------------------------------------------------------------------------
lru_file_repage 0 1 0 0 1 boolean D
--------------------------------------------------------------------------------
maxclient% 90 80 90 1 100 % memory D
maxperm%
--------------------------------------------------------------------------------
maxperm% 90 80 90 1 100 % memory D
minperm%
maxclient%
--------------------------------------------------------------------------------
minperm% 5 20 5 1 100 % memory D
maxperm%
--------------------------------------------------------------------------------
root@clodb:/>
PRODUCTION MACHINE PARAMETERS
Code
root@oradb:/>vmo -a
memory_frames = 4194304
pinnable_frames = 3976053
maxfree = 128
minfree = 120
minperm% = 20
minperm = 800049
maxperm% = 80
maxperm = 3200197
strict_maxperm = 0
maxpin% = 80
maxpin = 3355444
maxclient% = 80
lrubucket = 131072
defps = 1
nokilluid = 0
numpsblks = 2228224
npskill = 17408
npswarn = 69632
v_pinshm = 0
pta_balance_threshold = 50
pagecoloring = 0
framesets = 2
mempools = 1
lgpg_size = 0
lgpg_regions = 0
num_spec_dataseg = n/a
spec_dataseg_int = n/a
memory_affinity = 1
htabscale = -1
force_relalias_lite = 0
relalias_percentage = 0
data_stagger_interval = 161
large_page_heap_size = n/a
kernel_heap_psize = n/a
soft_min_lgpgs_vmpool = 0
vmm_fork_policy = 0
low_ps_handling = 1
mbuf_heap_psize = n/a
strict_maxclient = 1
cpu_scale_memp = 8
lru_poll_interval = 0
lru_file_repage = 1
root@oradb:/>
root@oradb:/>vmo -L minperm% -L maxperm% -L maxclient% -L lru_file_repage
NAME CUR DEF BOOT MIN MAX UNIT TYPE
DEPENDENCIES
--------------------------------------------------------------------------------
minperm% 20 20 20 1 100 % memory D
maxperm%
--------------------------------------------------------------------------------
maxperm% 80 80 80 1 100 % memory D
minperm%
maxclient%
--------------------------------------------------------------------------------
maxclient% 80 80 80 1 100 % memory D
maxperm%
--------------------------------------------------------------------------------
lru_file_repage 1 1 1 0 1 boolean D
--------------------------------------------------------------------------------
root@oradb:/>
My only suspect here is SAN Disk vs Internal Disk, this is what I could zoom into. Anyone think the problem could be other than HDD , please let me know.
