P4 with 1 processor and 16 GB RAM and SAN HDD = Oracle report takes 25 minutes
P5 with 2 processors and 16 GB RAM internall HDD with LPAR = Oracle Report takes 1 hour 15 minutes ( please note I have assigned all the max processors and memory to this LPAR and this is the only LPAR active )
* How can I measure and troubleshoot to find out what is causing the delay ?
We have seen differences in performance of a similar scale and nature.
Internal/local disk is managed by your server and it is responsible for all the real IO, writing the bits to the disk controllers and getting the acknowledgement when the update is made to the brown-spinning part.
With SAN storage, the request by your OS to write data is accepted and confirmed by the SAN before the data is really written to disk. It's written to the SAN cache, but as far as your OS is concerned, then transaction is complete and processing can continue.
A SAN disk array will have a large cache dedicated to holding write requests and flushing them to real disk in large IO writes. This is why all SAN disk arrays will have batteries to keep the cache alive in the event of a power failure. This is in addition to any UPS you may have in place. A SAN shutdown can also be done in two ways roughly described as immediate and controlled. In the controlled shutdown, the cache is flushed and the SAN disk is consistent. An Immediate shutdown does not flush the cache and is quicker. It has no difference for the actual data stored unless the battery fails or is replaced.
As to a way to measure it, I suppose you are into creating a large complex file and copying it about a few times. By complex, I mean something that compression software will not easily deal with. My way to do this (mainly for testing FTP speed) is to tar up some large files and compress them, then append them a few times to get the size big enough.
If your hardware can support it, then it can be beneficial to have everything as SAN storage, including the OS and paging space. The limitation then becomes the connection speed to the SAN.
If you replicate your SAN to another site, then this takes care of updates to the boot disks / rootvg / vg00 (whatever you want to call it) that may affect the DR position. Of course, the IP address will be replicated so you have to handle that along with a process to use perhaps different hardware at the remote site.
It is, however no substitute for backups as errors / corruption will also be replicated honestly.
I am using P550 VIOS so it cannot have a dedicated SAN HBA card,
Have to first assign the HDISK from SAN to I/O VIOS server and then from VIOS to LPAR
#2 IBMTECH and Bakunin
Yes, both database size are exactly same, infact it is a clone of the production environment, the min max paging space and other parameters are also same.... everything is same. DATA, DATABASE, Parameters, OS,
only difference
ORADB = Production Environment runs on P4 and Oracle is on SAN = report take 25 minutes
CloneDB = Clone Environment runs on P5 and Oracle is on internal Disks = report takes 1 hour and 15 minutes
AIX P4 = PRODUCTION
Code:
root@oradb:/>pagesize -a
4096
root@oradb:/>vmstat -v
4194304 memory pages
4000247 lruable pages
9961 free pages
1 memory pools
218074 pinned pages
80,1 maxpin percentage
20,0 minperm percentage
80,0 maxperm percentage
80,3 numperm percentage
3214895 file pages
0,0 compressed percentage
0 compressed pages
0,0 numclient percentage
80,0 maxclient percentage
0 client pages
0 remote pageouts scheduled
53 pending disk I/Os blocked with no pbuf
0 paging space I/Os blocked with no psbuf
80406 filesystem I/Os blocked with no fsbuf
0 client filesystem I/Os blocked with no fsbuf
0 external pager filesystem I/Os blocked with no fsbuf
AIX P5 ON LPAR = CLONE
Code:
pagesize -a
4096
root@clodb:/>vmstat -v
2097152 memory pages
2017797 lruable pages
1713122 free pages
2 memory pools
121734 pinned pages
80.0 maxpin percentage
5.0 minperm percentage
90.0 maxperm percentage
0.7 numperm percentage
14208 file pages
0.0 compressed percentage
0 compressed pages
1.2 numclient percentage
90.0 maxclient percentage
24760 client pages
0 remote pageouts scheduled
0 pending disk I/Os blocked with no pbuf
0 paging space I/Os blocked with no psbuf
2740 filesystem I/Os blocked with no fsbuf
0 client filesystem I/Os blocked with no fsbuf
0 external pager filesystem I/Os blocked with no fsbuf
ORACLE ON CLONE AIX ENV P5 LPAR
Code:
oraprod@clodb:/oratech/proddb/9.2.0/>./startup.sh
You are running addbctl.sh version 115.8
Starting the database PROD ...
SQL*Plus: Release 9.2.0.3.0 - Production on Wed Jan 15 14:16:00 2014
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Connected to an idle instance.
ORACLE instance started.
Total System Global Area 615482512 bytes
Fixed Size 742544 bytes
Variable Size 436207616 bytes
Database Buffers 167772160 bytes
Redo Buffers 10760192 bytes
Database mounted.
Database opened.
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.3.0 - 64bit Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.3.0 - Production
addbctl.sh: exiting with status 0
You are running addlnctl.sh version 115.6
Logfile: /oratech/proddb/9.2.0/appsutil/log/PROD_clodb/addlnctl.txt
Starting listener process PROD ...
addlnctl.sh: exiting with status 0
oraprod@clodb:/oratech/proddb/9.2.0/>
oraprod@clodb:/oratech/proddb/9.2.0/>sqlplus
SQL*Plus: Release 9.2.0.3.0 - Production on Wed Jan 15 14:19:04 2014
Copyright (c) 1982, 2002, Oracle Corporation. All rights reserved.
Enter user-name: sys
Enter password:
Connected to:
Oracle9i Enterprise Edition Release 9.2.0.3.0 - 64bit Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.3.0 - Production
SQL> connect as sysdba
Enter user-name: sys
Enter password:
Connected.
SQL> show parameters area_size
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
bitmap_merge_area_size integer 1048576
create_bitmap_area_size integer 8388608
hash_area_size integer 131072
sort_area_size integer 65536
workarea_size_policy string AUTO
SQL> set pages 999; column pga_size format 999,999,999;
SQL> select
1048576+a.value+b.value pga_size
from
v$parameter a,
v$parameter b
where
a.name = 'sort_area_size'
and
b.name = 'hash_area_size'
;
PGA_SIZE
------------
1,245,184
Yes, both database size are exactly same, infact it is a clone of the production environment, the min max paging space and other parameters are also same.... everything is same. DATA, DATABASE, Parameters, OS,
only difference
ORADB = Production Environment runs on P4 and Oracle is on SAN = report take 25 minutes
CloneDB = Clone Environment runs on P5 and Oracle is on internal Disks = report takes 1 hour and 15 minutes
This way it sounds more logical, because SAN disks are (in most cases) much faster than physical disks. They have usually a lot more IOPS and more throughput bandwidth, because a lot of different reasons: stripesets, caching controllers, buses with higher bandwidth, ....
Your systems seem not to have the same amount of memory at all:
root@oradb:/>vmstat -v
[...]
80406 filesystem I/Os blocked with no fsbuf
[...]
whereas:
Code:
root@clodb:/>vmstat -v
[...]
2740 filesystem I/Os blocked with no fsbuf
[...]
Not the value in itself is problematic, but the huge difference. Monitor the value closely over time, if it increases heavily you have found a potential bottleneck. If it stays at this level it is perhaps an artefact of some temporary memory shortness.
Finally, the tuning parameters seem to be different (compare "maxperm" and "minperm" in the different outputs, probably others are different too). You should run "vmo -a" (and the other tuning utilities, "schedo", "ioo", "no") on both machines to investigate other differences. What good values for maxperm and minperm would be is hard to suggest because it depends heavily on the (detailed) OS version which we do not know.
Another supposition (which would have to be proven by facts) of mine is that the faster processor is not needed in the special kind of report you run and therefore contributes nothing to some faster execution, whereas the disks contribute heavily.
And to add more to bakunin's finding, have you enable AIO? ( I assume you are using AIX5.3, given the specification of hardware). Did you cross verify that CIO option for DB file systems is enabled on Prod system (if AIX5.3 or less).
Can you provide us the VMM parameters that are currently configured in both systems?
Greeting All
Diag tool HDD clone SCO 5 + Oracle 7 DB ( IBM T30 )
I am new to this forum and my knowledge on computers OS is average .
I have just acquired a factory diag tool for BMW/RR/MINI from a retired mechanic. Its runs on a IBM T30 laptop with a Unix/Oracle DB system. Sco 5... (8 Replies)
Hi,
This is follow up to the post https://www.unix.com/aix/233361-san-disk-appearing-double-aix.html
When I connected Pseries Machine HBA Card ( Dual Port ) directly to the SAN Storage DS4300 , I was able to see Host Port Adapter WWN numbers , although I was getting this message... (2 Replies)
I need to get an external HDD for a SUN server running Solaris 10. The Western Digital that I have will not recognize and when I went looking for drivers WD only has them for MAC and Windows. Is there a External HDD that is known to work with Unix? (24 Replies)
All,
I am a complete grasshopper when it comes to Unix, so here goes. I have a Solaris 9 server, running two 36GB HDD in a mirrored configuration. I am running low on disk space, and have purchased an additional 145GB HDD. Can anyone point me in the direction to some documentation on how to... (1 Reply)
Over the last few months the HDD spins louder and louder, so I fiqured its time to replace the HDD. Its been running 24/7/365 since 98 :eek:. yes i said since 98 :D
I have an IBM system 43P Model 240. 233 MHz. running AIX Version 4.
The current HDD is an IBM DGHS COMP IEC -950
FRU PN#... (5 Replies)
Hi All,
I would like to share this incident that happened the other day.
I have a question with this, https://www.unix.com/aix/64921-create-new-vg-san-rename-fs.html
And I thought it's related to the above link but the problem was the ibm san 4300 cache battery was dead and I need to click... (2 Replies)
I am adding a new HDD to a Unix Sco Release 5 webserver.
I consider myself a windows pro. However, growing up in the late 90's means I have little Unix knowledge. I know the HDD has to be mounted and formatted correctly. Can anyone give me any advice on this?
A dummy's guide to installing a... (5 Replies)
Hi theres
I am quite new to solaris, I have 40GB HDD in which I have created only 10 GB partition & installed solaris 10. Now I want to add another 10GB from remaining 30GB space. I tried this with format utility but I get stuck after I create fdsik partition. After creating this I cant... (1 Reply)
Hi,
My HP-UX has some errors to wake up and one of them are:
/dev/vg00/rlvol4 BAD CYLINDER GROUPS
I have thought that my hdd has physical problems, my question is:
Exists a program to check the hdd physically and to repair it...
Thanks.... (0 Replies)
