I have identical M5000 machines that are needing to transfer very large amounts of data between them. These are fully loaded machines, and I've already checked IO, memory usage, etc... I get poor network performance even when the machines are idle or copying via loopback. The 10 GB NICs are setup in aggregates, so we should be seeing some serious performance out of these. However, I'm only seeing about 30 MB/sec. These are brand new Solaris 11.1 installs, and fully patched.
I'm getting 30 MB/sec no matter which connection I use, whether it's 1 GB and 10 GB connections, physical, VLAN, etc...
Code:
richardc@SERVERX:/kernel/drv$ dladm show-phys
LINK MEDIA STATE SPEED DUPLEX DEVICE
net1 Ethernet unknown 0 unknown bge1
net6 Ethernet up 10000 full ixgbe0
net0 Ethernet unknown 0 unknown bge0
net4 Ethernet up 1000 full nxge2
net2 Ethernet unknown 0 unknown nxge0
net5 Ethernet unknown 0 unknown nxge3
net7 Ethernet up 10000 full ixgbe1
net3 Ethernet up 1000 full nxge1
richardc@SERVERX:/kernel/drv$ dladm
LINK CLASS MTU STATE OVER
net1 phys 1500 unknown --
net6 phys 1500 up --
net0 phys 1500 unknown --
net4 phys 1500 up --
net2 phys 1500 unknown --
net5 phys 1500 unknown --
net7 phys 1500 up --
net3 phys 1500 up --
prim1 aggr 1500 up net3 net4
bkup1 aggr 1500 up net6 net7
rep1 vlan 1500 up bkup1
richardc@SERVERX:/kernel/drv$ ipadm
NAME CLASS/TYPE STATE UNDER ADDR
bkup1 ip ok -- --
bkup1/v4 static ok -- XXX.XX.235.4/22
lo0 loopback ok -- --
lo0/v4 static ok -- 127.0.0.1/8
prim1 ip ok -- --
prim1/v4 static ok -- XXX.XX.225.244/22
rep1 ip ok -- --
rep1/v4 static ok -- XXX.XX.198.176/25
sppp0 ip ok -- --
richardc@SERVERX:/kernel/drv$ ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
prim1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet XXX.XX.225.244 netmask fffffc00 broadcast XXX.XX.227.255
bkup1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet XXX.XX.235.4 netmask fffffc00 broadcast XXX.XX.235.255
rep1: flags=201000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,CoS> mtu 1500 index 4
inet XXX.XX.198.176 netmask ffffff80 broadcast XXX.XX.198.255
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU> mtu 1500 index 5
inet 10.1.1.3 --> 10.1.1.1 netmask ff000000
lo0: flags=2002000848<LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu 8252 index 1
inet6 ::/128
prim1: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 2
inet6 ::/0
bkup1: flags=20002000840<RUNNING,MULTICAST,IPv6> mtu 1500 index 3
inet6 ::/0
rep1: flags=20202000840<RUNNING,MULTICAST,IPv6,CoS> mtu 1500 index 4
inet6 ::/0
richardc@SERVERX:/kernel/drv$
For example, when doing an SCP of a 200 MB file it takes 8 seconds and runs at 30 MB/sec:
I figured maybe it was something to do with the network, but I see this slow speed no matter which interface I use -- even localhost. I tried doing a local SCP back to the same machine, and it's taking 8 seconds there too even though it isn't actually hitting the network cards. If I do it as a regular CP it copies the file in less than a second.
Does anyone have any ideas on how to improve the performance? I figure I can't be the only one who has run into this, but my Google searches haven't turned up anything.
We have an egrep search in a while loop.
egrep -w "$key" ${PICKUP_DIR}/new_update >> ${PICKUP_DIR}/update_record_new
${PICKUP_DIR}/new_update is 210 MB file
In each iteration, the egrep on an average takes around 50-60 seconds to search. Ther'es nothing significant in the loop other... (7 Replies)
Hi all
We have got issues with copying a 2.6 GB file from one folder to another folder.
Well, this is not the first issue we are having on the box currently, i will try to explain everything we have done from the past 2 days.
We got a message 2 days back saying that our Production is 98%... (3 Replies)
hi guys
We are seeing weird issues on my Linux Suse 10, it has lotus 8.5
and 1 filesystem for OS and another for Lotus Database.
the issue is when the Lotus service starts wait on top is very high about 25% percent and in general CPU usage is very high
we found that when this happens if we... (0 Replies)
Please, I need help tuning my script. It works but it's too slow.
The code reads an acivity log file with 50.000 - 100.000 lines and filters error messages from it. The data in the actlog file look similar to this:
02/08/2011 00:25:01,ANR2034E QUERY MOUNT: No match found using this criteria.... (5 Replies)
There is a big problem with the server (VPS based on OpenVZ, CentOS 5, 3GB RAM). The problem is the following. The first 15-20 minutes after starting the server is operating normally, the load average is less than or about 1.0, but then begins to increase sharply% wa, then hovers around 95-99%.... (2 Replies)
My code
Hi All,
I am having redhat linux 5.3 (Tikanga) with GFS file system and its very very slow for executing ls -ls command also.Please see the below for 2minits 12 second takes.
Please help me to fix the issue.
$ sudo time ls -la BadFiles |wc -l
0.01user 0.26system... (3 Replies)
Hi,
I have 2 machines in production environment:
1. redhat machine for application
2. DB machine (oracle)
The application doing a lot of small read&writes from and to the DB machine.
The problem is that after some few hours the network from the application to the DB becomes very slow and... (4 Replies)