AIX

Hi Guys,

This is the situation I am in. Provide your views and input where should I start?

I have one P7 test server and a p520 production server. the job is taking pretty long on the P7 test server when compared to the P5 production server. below is the full detail.

Informix transaction activity and archive processing are taking
noticeably longer on the Power7 8233-E8B 12way/3.3GHz processor with
128GB of memory running AIX Version 6.1. The Informix LPAR in
question has 3 cores and 16GB of memory assigned. The system has been
fully configured to be virtualized with 3-4 LPARs and 2 x VIOS LPARS
supporting all I/O. The SAN storage consists of a 4Gbps Fibre
Channel SAN with an IBM SAN Volume Controller supporting an IBM
nSeries N5600 and an n6040 plus a fully populated DS4800. This is
the first time the customer has had any LPAR environments in their
data center. The systems were initially configured by 2 x IBM
Advanced Technical Support specialists due to the complexity of the
environment.

The current Informix environment and ATM processing application is
running on an IBM 8203-E4A model 520 with 4 x 4.2GHz processors, 8GB
of memory, and Version 5.3 of AIX, attached via fibre Channel to a
shared EMC CX3-40C Clariion storage unit. On the model 520, the
archive process is taking 25 minutes to archive 96GB of data while
still supporting user activity. But on the model 750 it is taking
around 45 minutes to complete. The archive process is a single
threaded process and, we believe, is not being able to take full
advantage of the additional processors and advanced architectural
design of the Power 7. This could be due to lack of proper tuning of
the OS or other undiscovered factors. The Informix environment and
the ATM application code are running the 32bit version of their code
on the model 520. But on the model 750, the Informix software is
running the 64bit version but the software vendor's application is
still running the 32bit version of the application.

Thank you.

a single threaded process will be by nature slower on a slower cpu - and as you mentioned above - 4.2 GHz is faster than 3.3 GHz - so it will be obviously slower on the new box ? In addition if your vio servers have not enough network resources to support the traffic of all the lpars attached to them, you will loose more time compared to the 520 with fibre and as I assume physical network adapters?
In addition your queue depth on the virtualized SAN might be very low - I assume as low as 3 - compared to the clariion where the queue depth is usually 32.

If your tuning is proper or not is something we cannot say as long as we do not get any data from your box but I would guess that you have the default settings from 5.3 on the new system which are horrible.

If you like, show us the outputs of vmo -a, ioo -a, vmstat -Iwt 2 10 and similar output and we maybe able to tell you what you might want to change.

On top of that by default 32bit applications have only a very limited shared memory area which you can open up for your application to become dynamic but that most likely had not been done either.

Hope that helps
regards
zxmaus

Regards
zxmaus

Hi zxmus,

I checked the queue depth which is 8 on the performance issue box when compared to 32 on production box.

---------- Post updated at 02:43 PM ---------- Previous update was at 09:53 AM ----------

the queue depth is 8 on vio and client server, what should be the correct queue depth?

Thanks

I am sorry but I only know the queue depths for the EMC family (symmetrix, clariion, hds) but I would think that 16 would be a good point to start - extend it carefully up to 32 if you like to see if your performance is improving. Another thought ... the default for hba's on 5.3 is as far as I know 1 gig - if you are sure your entire infrastructure (switches and so on) supports 4 gig, you can try and open up your adapters to 4 gig using chdev command.

Hope that helps
regards
zxmaus

p7 is not a slower CPU. the architecture is different than p5. the p520 is small compared to his new setup



did this ever perform well or is this a new issue? is it CPU bound, memory, disk, etc...?? we need more technical details to diagnose. check out perfpmr if you have not already. consider calling the people that did the setup.

4.2 GHz is p6, not p5 ...

Apart from that - A notable difference from POWER6 is that the POWER7 executes instructions out-of-order instead of in-order like p5 and predecessors did - and p7 has 12 instruction units per core instead of 8 and a lot more cache. There are certain workloads - particularly single threaded ones that do a particular task, that are significantly slower on p7 - we have in our lab tested the p7 in any possible circumstance before we have decided to go with them - for normal multithreaded webserver and DB workloads they are significantly better in throughput, for stupid single threaded tasks like backups with compression, they take significantly longer - and the differences are bigger when you run 5.3 than when you run 6.1 on p7.