Application running too slow on Sun SPARC T5440 but run normal on sun M3000


 
Thread Tools Search this Thread
Operating Systems Solaris Application running too slow on Sun SPARC T5440 but run normal on sun M3000
# 1  
Old 06-13-2012
Application running too slow on Sun SPARC T5440 but run normal on sun M3000

Hi all,
I have application running on sun server T5440 4x8x1.4 GHz, 64 GB RAM, application running very slow though load average too low. when I install my application on another server SUN M3000 (One CPU 1x8x2.5GHz, 8GB RAM), application run smoothly.

Here is my server T5440 info:

Code:
bash-3.00# uname -a
    SunOS 5.10 Generic_142909-17 sun4v sparc SUNW,T5440

Psrinfo -pv

Code:
bash-3.00# psrinfo -pv
    The physical processor has 64 virtual processors (0-63)
    UltraSPARC-T2+ (chipid 0, clock 1414 MHz)
    The physical processor has 64 virtual processors (64-127)
    UltraSPARC-T2+ (chipid 1, clock 1414 MHz)
    The physical processor has 64 virtual processors (128-191)
    UltraSPARC-T2+ (chipid 2, clock 1414 MHz)
    The physical processor has 64 virtual processors (192-255)
    UltraSPARC-T2+ (chipid 3, clock 1414 MHz)

/etc/system

Code:
set semsys:seminfo_semopm=10000
    set semsys:seminfo_semmsl=10000
    set semsys:seminfo_semmni=1024
    set max_nprocs=50000
    set msgsys:msginfo_msgmnb=2097152
    set msgsys:msginfo_msgmni=2560
    set msgsys:msginfo_msgtql=2560
    set shmsys:shminfo_shmmax=1073741824
    set shmsys:shminfo_shmmni=1000
    set rlim_fd_cur=10000

Ipcs

Code:
bash-3.00# ipcs -a
    IPC status from as of Wed Jun 13 12:31:01 ICT 2012
    T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME
    Message Queues:
    q 62 0x1388 --rw-rw-rw- topx topx topx topx 0 0 2097152 0 0 no-entry no-entry 8:36:22
    q 61 0x1387 --rw-rw-rw- topx topx topx topx 0 0 2097152 0 0 no-entry no-entry 8:36:22
    q 60 0x1318 --rw-rw-rw- topx topx topx topx 0 0 2097152 0 0 no-entry no-entry 8:36:22
    q 59 0x12a9 --rw-rw-rw- topx topx topx topx 0 0 2097152 9069 9071 10:25:38 10:25:38 8:36:22
    q 58 0x100f --rw-rw-rw- topx topx topx topx 0 0 2097152 9069 9070 10:06:16 10:06:16 8:36:22
    q 57 0xfa0 -Rrw-rw-rw- topx topx topx topx 0 0 2097152 9071 9069 10:25:38 10:25:38 8:36:22
    q 56 0xf30 --rw-rw-rw- topx topx topx topx 0 0 2097152 0 0 no-entry no-entry 8:36:20
    q 55 0xc27 -Rrw-rw-rw- topx topx topx topx 0 0 2097152 0 0 no-entry no-entry 8:28:14
    q 54 0xbb8 -Rrw-rw-rw- topx topx topx topx 0 0 2097152 172 9066 10:47:08 10:47:08 8:28:14
    T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME
    Shared Memory:
    T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME
    Semaphores:
    s 104 0x9c4 --ra-ra-ra- topx topx topx topx 3 12:31:00 8:36:20
    s 103 0xa8c --ra-ra-ra- topx topx topx topx 1 8:36:20 8:36:20
    s 102 0xd48 --ra-ra-ra- topx topx topx topx 1 8:36:20 8:36:20
    s 101 0xce4 --ra-ra-ra- topx topx topx topx 1 8:36:20 8:36:20
    s 100 0x26de --ra-ra-ra- topx topx topx topx 8 10:19:34 8:28:14
    s 99 0xc1c --ra-ra-ra- topx topx topx topx 1 8:36:20 8:28:14
    s 98 0xbb8 --ra-ra-ra- topx topx topx topx 1 8:43:50 8:28:14
    s 97 0xaf0 --ra-ra-ra- topx topx topx topx 500 8:36:20 8:28:14
    s 96 0xa28 --ra-ra-ra- topx topx topx topx 500 8:36:20 8:28:14
    s 95 0x7ee --ra-ra-ra- topx topx topx topx 1 8:36:20 8:28:14
    s 94 0x7e4 --ra-ra-ra- topx topx topx topx 1001 8:36:20 8:28:14
    s 93 0x7da --ra-ra-ra- topx topx topx topx 220 8:36:20 8:28:14
    s 92 0x7d5 --ra-ra-ra- topx topx topx topx 200 12:05:15 8:28:14
    s 91 0x7d0 --ra-ra-ra- topx topx topx topx 200 8:56:42 8:28:14
    s 90 0xc80 --ra-ra-ra- topx topx topx topx 2 12:31:01 8:27:52 -a

Prstat

Code:
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP

    9202 topx 1460M 893M sleep 59 0 0:16:37 0.2% integrity_check/1
    12455 root 3880K 3600K cpu65 59 0 0:00:00 0.0% prstat/1
    9080 topx 142M 130M sleep 59 0 0:00:52 0.0% intf_resp_low/1
    9125 topx 102M 78M sleep 59 0 0:00:36 0.0% interface_vsc/1
    9121 topx 88M 80M sleep 59 0 0:00:32 0.0% interface_tcu/1
    9120 topx 89M 80M sleep 59 0 0:00:29 0.0% interface_rcm/1
    9124 topx 87M 78M sleep 59 0 0:00:29 0.0% baplie_merge_v2/1
    9122 topx 86M 78M sleep 59 0 0:00:28 0.0% interface_eih/1
    9115 topx 88M 77M sleep 59 0 0:00:33 0.0% intf_batch_even/1
    9114 topx 88M 77M sleep 59 0 0:00:39 0.0% intf_event/1
    9117 topx 87M 78M sleep 59 0 0:00:32 0.0% interface_bxy/1
    9118 topx 86M 78M sleep 59 0 0:00:29 0.0% interface_ppm/1
    NPROC USERNAME SWAP RSS MEMORY TIME CPU

    41 topx 334M 2237M 3.4% 0:30:17 0.3%
    43 root 186M 195M 0.3% 0:14:45 0.0%
    1 noaccess 152M 136M 0.2% 0:04:00 0.0%
    1 smmsp 3688K 11M 0.0% 0:00:04 0.0%
    1 lp 1208K 3880K 0.0% 0:00:00 0.0%
    6 daemon 7264K 9944K 0.0% 0:00:26 0.0%
    Total: 93 processes, 259 lwps, load averages: 0.40, 0.39, 0.39

Prstat -mL

Code:
bash-3.00# prstat -mL
    PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/LWPID
    9202 topx 0.9 0.0 0.0 0.0 0.0 0.0 99 0.0 2 0 5 0 integrity_ch/1
    9080 topx 0.4 0.0 0.0 0.0 0.0 0.0 100 0.0 5 0 20 0 intf_resp_lo/1
    9114 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 13 0 66 0 intf_event/1
    17282 root 0.1 0.2 0.0 0.0 0.0 0.0 100 0.0 20 1 324 0 prstat/1
    9117 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 1 70 0 interface_bx/1
    9118 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 1 70 0 interface_pp/1
    9115 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 15 0 80 0 intf_batch_e/1
    9121 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 0 70 0 interface_tc/1
    9125 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 0 70 0 interface_vs/1
    9120 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 0 70 0 interface_rc/1
    9123 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 0 70 0 interface_vs/1
    9124 topx 0.2 0.0 0.0 0.0 0.0 0.0 100 0.0 10 1 70 0 baplie_merge/1
    9122 topx 0.1 0.0 0.0 0.0 0.0 0.0 100 0.0 8 0 56 0 interface_ei/1
    9119 topx 0.1 0.0 0.0 0.0 0.0 0.0 100 0.0 8 0 56 0 interface_cp/1
    9204 topx 0.1 0.0 0.0 0.0 0.0 0.0 100 0.0 15 1 1K 0 sys_check/1
    259 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 13 2 98 7 in.mpathd/1
    1 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 3 0 99 3 init/1
    1337 noaccess 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 100 0 100 0 java/14
    140 root 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 19 0 114 0 nscd/851
    Total: 90 processes, 256 lwps, load averages: 0.40, 0.40, 0.39

Vmstat

Code:
bash-3.00# vmstat 1 10
    kthr memory page disk faults cpu
    r b w swap free re mf pi po fr de sr m0 m1 m2 m3 in sy cs us sy id
    0 0 0 87720712 60562856 173 645 199 44 44 0 0 0 0 0 0 886 2819 525 0 1 99
    1 0 0 87563976 59743040 73 327 0 15 15 0 0 0 0 0 0 747 2831 439 0 1 99
    0 0 0 87563656 59742808 61 306 0 15 15 0 0 0 0 0 0 770 2684 427 0 1 99
    1 0 0 87563656 59742824 0 0 0 0 0 0 0 0 0 0 0 886 1302 525 0 1 99
    1 0 0 87563656 59742816 62 305 0 15 15 0 0 0 0 0 0 722 2652 420 3 1 96
    1 0 0 87563656 59742840 554 2777 0 139 139 0 0 0 0 0 0 959 9736 749 1 1 97
    0 0 0 87563456 59742672 1155 5735 0 292 292 0 0 0 0 0 0 1113 17489 1145 1 3 96
    1 0 0 87563656 59742840 0 0 0 0 0 0 0 0 0 0 0 845 1613 654 0 1 99
    0 0 0 87563656 59742856 62 305 0 15 15 0 0 0 0 0 0 900 2647 526 0 1 99
    0 0 0 87563656 59742856 62 304 0 15 15 0 0 0 0 0 0 683 2661 396 0 1 99

Mpstat

Code:
CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl
    0 0 0 130 261 1 121 0 13 1 0 0 0 1 0 99
    1 0 0 35 133 2 143 0 9 6 0 64 0 0 0 100
    2 0 0 5 20 2 17 0 3 0 0 29 1 0 0 99
    3 0 0 0 3 0 2 0 0 0 0 0 0 0 0 100
    4 0 0 3 14 0 15 0 2 2 0 13 0 0 0 100
    73 0 0 474330 77 3 0 7 0 2 0 2596 67 33 0 0
    73 0 0 487701 37 0 0 7 0 8 0 2670 66 34 0 0
    73 0 0 476377 14 0 0 8 0 1 0 2610 66 34 0 0
    78 0 0 473950 11 1 0 7 0 13 0 2593 67 33 0 0
    78 0 0 480562 9 0 0 7 0 4 0 2630 66 34 0 0
    79 0 0 320989 14 1 0 8 0 0 0 1766 77 23 0 0
    79 0 0 479148 9 0 0 7 0 8 0 2622 66 34 0 0
    79 0 0 479426 13 0 0 7 0 2 0 2622 67 33 0 0
    79 0 0 488949 9 0 0 8 0 13 0 2678 66 34 0 0
    64 0 0 482229 15 2 0 8 0 7 0 2638 65 35 0 0
    77 0 0 475571 14 1 0 7 0 8 0 2602 67 33 0 0
    69 0 0 467795 22 1 0 7 0 40 0 2563 66 34 0 0
    69 0 0 318623 9 1 0 7 0 8 0 1751 77 23 0 0
    66 0 0 465179 92 1 2 8 0 62 0 2545 66 34 0 0

When my application running, some of CPU have IDLE state = 0 and hold the CPU while another CPU have IDLE state 100. I think this is my bottleneck but I cannot find any solution for solve that.

Netstat

Code:
bash-3.00# netstat -i
    Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue
    lo0 8232 loopback localhost 244022 0 244022 0 0 0
    nxge0 1500 TOPX-01-DUMMY1 TOPX-01-DUMMY1 3594863 0 5801650 0 0 0
    nxge1 1500 TOPX-01-DUMMY2 TOPX-01-DUMMY2 160954 0 151230 0 0 0

iostat

Code:
bash-3.00# iostat -xn
    extended device statistics

    r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    0.1 0.1 1.5 30.8 0.0 0.0 1.8 9.0 0 0 md/d0
    0.0 0.1 0.8 30.8 0.0 0.0 0.0 9.9 0 0 md/d1
    0.0 0.1 0.8 30.8 0.0 0.0 0.0 9.8 0 0 md/d2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 15.2 0 0 md/d3
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 10.9 0 0 md/d4
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 19.4 0 0 md/d5
    0.0 0.0 2.2 1.5 0.0 0.0 0.8 7.4 0 0 md/d6
    0.0 0.0 1.1 0.7 0.0 0.0 0.0 8.5 0 0 md/d7
    0.0 0.0 1.1 0.8 0.0 0.0 0.0 8.2 0 0 md/d8
    0.0 0.1 0.4 0.6 0.0 0.0 7.6 10.0 0 0 md/d9
    0.0 0.1 0.2 0.6 0.0 0.0 0.0 9.1 0 0 md/d10
    0.0 0.1 0.2 0.6 0.0 0.0 0.0 7.8 0 0 md/d11
    0.0 0.0 0.0 0.0 0.0 0.0 0.1 10.4 0 0 md/d12
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 8.5 0 0 md/d13
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 8.4 0 0 md/d14
    0.1 0.5 2.1 32.3 0.0 0.0 0.0 9.9 0 0 c0t0d0
    0.1 0.5 2.1 32.3 0.0 0.0 0.0 9.8 0 0 c0t1d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t0d0
    0.9 0.9 125.3 123.1 0.0 0.1 0.0 34.2 0 2 c4t600144F0CC00C29A00004D1E456F0003d0
    0.3 0.7 20.7 53.9 0.0 0.0 0.2 8.2 0 0 c4t600144F0CC00C29A00004D1E43880001d0
    0.1 0.2 50.7 111.5 0.0 0.0 0.0 59.1 0 1 c4t600144F0CC00C29A00004D1E453E0002d0

Please help me find the bottleneck of my T5440 server?Thanks so much for all your help.
# 2  
Old 06-13-2012
Niagara CPUs are best suited for massive parallel loads, like webservers. Your machine can run 256 threads in parallel, giving each thread 1/256th of the overall computing power. But the single thread power is quite low, especially on early Niagara type CPUs, like the T1 and T2.

It seems, your application is composed of a few single threaded programs where each programm runs on a single virtual CPU, leaving most of the computing power unused. Such applications are not very well suited for a Niagara CPU server.

Your application runs well on the M3000, because the SPARC64 CPUs are designed for only few threads with a lot of computing power per thread.

Sun's Rock CPU was supposed to close the gap between those two architectures, but was never brought to market. The new T4 CPU seems to come close to this goal.
This User Gave Thanks to hergp For This Post:
# 3  
Old 06-13-2012
Have any solution to migrate thread power to some thread on the same cpu.
For example: i have 4 cpu and 64 thread on each cpu. i offline 56 thread on each cpu and my system have 32 thread online on 4 cpu. or tell me some solution for this?
Many thanks for your help.
# 4  
Old 06-13-2012
You can take processors offline with the psradm command, look at the manpage to get the details.

But that would not speed up your program, switching off unused cores does not make the remaining ones faster. You have to change your programs instead, if possible, to utilize more threads.
# 5  
Old 06-13-2012
My application is a third-party application so i can't change, please help me find any process, thread or commands of OS in my application using a lot of computing power on thread . I can request my software vendors to review it.
Many thanks for your help.

Last edited by insatiable1610; 06-13-2012 at 09:54 PM..
# 6  
Old 06-13-2012
I am not aware of any way to combine the computing power of several cores for one single thread, it's just how the CPU is designed.
This User Gave Thanks to hergp For This Post:
# 7  
Old 06-13-2012
Dear hergp,

Thank you so much.
Login or Register to Ask a Question

Previous Thread | Next Thread

7 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Start /SYS on SUN SPARC does not start machine [SUN SPARC ENTERPRISE T-5240]

-> start /SYS Are you sure you want to start /SYS (y/n)? y Starting /SYS ]-> show HOST /HOST Targets: bootmode diag domain Properties: autorestart = reset autorunonerror = false bootfailrecovery = poweroff ... (29 Replies)
Discussion started by: z_haseeb
29 Replies

2. Emergency UNIX and Linux Support

disk replacment, SUN M3000

we have a SUN M3000 server. setup as only 1 domain. disk c0t0d0 and c0t1d0 and setup as SVM mirrors. a few days ago disk T1 failed. new we have replaced the disk, but can's see the disk in format. have done cfgadm and devfsadm. still can't access the new disk in format. the output... (6 Replies)
Discussion started by: robsonde
6 Replies

3. Solaris

Usb driver update on Sun T5440

I need a help to find the patch for solaris to fix USB drivers issue on Solaris 10. The server has a PCIE card which is not recognised by the OS. I run #fmadm faulty command which report issue and recommandation to apply the new patch. I got a reference to //sun.com/msg/PCIEX-8000-6D but... (0 Replies)
Discussion started by: espengongo
0 Replies

4. Solaris

Adding new hard drives on Sun SPARC T5440 server with Solaris 10

I need to add two new hard drives of 300 GB capacity to a SunSPARC T5440 server. The server currently has two hard drives of 146 GB each. How do I add the new drives to the existing UFS? What are the procedures involved in setting this server with the new hard drives? I am very new to Solaris... (6 Replies)
Discussion started by: ramatnmcc
6 Replies

5. Solaris

SUN T5240 vs M3000

Hi, We are planning to buy new server for our data center. Sun T5240 or M3000 which one have better performance, we are going to create many dt sessions in this server. So, i need your suggestions. RJS (4 Replies)
Discussion started by: rajasekg
4 Replies

6. High Performance Computing

Normal (not crossover) cable for Sun cluster interconnect..

Hi grus, has anybody tried for Sun cluster interconnect normal cable ,, I mean not interconnect .. What do u think ..does it support this ? Not long ago I tried Veritas cluster and its interconnections work great with normal cable .. I wonder what will Sun cluster say to it :)) (3 Replies)
Discussion started by: samar
3 Replies

7. Solaris

Difference between sun sparc 32-bit and sun sparc 64-bit

Hi , could you please clarify me the difference between sun sparc 32-bit and sun sparc 64-bit? means on which processors these are supporting (pentium processors, sun sparc processors) Regards, Rajesh (1 Reply)
Discussion started by: pmrajesh21
1 Replies
Login or Register to Ask a Question