Sponsored Content
Top Forums UNIX for Beginners Questions & Answers 10gbe ixgbe slow at 420mbyte/s max, p2p lan cat8 1m, tried ethtool and ifconfig options to no avail Post 303045707 by tinfoil3d on Thursday 9th of April 2020 10:52:43 PM
Old 04-09-2020
[SOLVED] 10gbe ixgbe slow at 420mbyte/s max, p2p lan cat8 1m, tried ethtool and ifconfig options

Hey guys, first time dealing with 10gbe.
I have two boxes, one is older with i7 2.8ghz from 2009 generation, pcie 2.0 and I just put the Intel X550T there to make use of absolutely similar but builtin adapter on my newer box with AMD EPYC gen 1.
I'm dling file from tmpfs on either box and never go above 415mbyte/s which is about half of potential bandwidth there, right?
I tried most stuff from kernel . org/doc/ols/2009/ols2009-pages-169-184.pdf with a little improvement from 393mbyte/s to current 415.
that's the result of me setting mtu 9000, txqueuelen 10000.
Also tried this to no improvements over current result: darksideclouds wordpress com/2016/10/10/tuning-10gb-nics-highway-to-hell/
And a few other pages out there on google, most stating same options.
I don't see a 100% load burst on i7 box when I dl file, it's around 10% across the cores. So CPU shouldn't be a bottleneck, cable(it's 1meter cat8, with a nice thick shielding) shouldn't be a bottleneck and ram speed and pcie 2.0 speed shouldn't be.
iperf also reports same speed.
Tried these tests in both directions.
There's gotta be something obvious that I'm missing, right?

Pretty sure it doesn't matter but it's lfs on i7 and debian 10 on epyc, both sharing same sysctl settings and ethtool-controlled options.
lspci -n for it on both computers:
Code:
62:00.0 0200: 8086:1563 (rev 01)

so it's similar device. I bought this particular one for compatibility but didn't expect these issues.

Maybe someone here would suggest something, I'm out of ideas.
TIA
--
solved see my own response

Last edited by tinfoil3d; 04-11-2020 at 08:48 AM.. Reason: solved, it's PCIe lane count, usb3, sata3, 6 disks, one GPU downgraded to x8 and no more space to have x4 on pcie 2.0 mb with max 16 lanes i7 CPU
 

9 More Discussions You Might Find Interesting

1. Solaris

How to configure private LAN and coporate LAN on the same machine-Solaris10

Hi , I am trying to configure a private LAN and corporate LAN on the same machien on Solaris 10. How can I achieve this? Thanks (1 Reply)
Discussion started by: deedee
1 Replies

2. Shell Programming and Scripting

ethtool : command not found

Hi, I am uising Suse linux on my system and i tried : man ethtool and i got the manual for ethtool but when i tried to use the command as: ethtool -a eth0 it says Command not found. also, i tried rpm -qa|grep ethtool i got: ethtool-3-15.2 so,it means my system has ethtool server or... (4 Replies)
Discussion started by: salil2012
4 Replies

3. Programming

ethtool problem

Hi, I'm using ethtool to change network settings: Just for the experiment, i want to change speed to 10, instead of 100, I type: ethtool -s eth0 speed 10 duplex full autoneg off and to view settings i type: ethtool eth0, and get this: Settings for eth0: Supported ports: ... (4 Replies)
Discussion started by: alex889
4 Replies

4. IP Networking

Slow LAN FTP Transfers with FreeBSD...

Hello, I've got a FreeBSD 6 server running on my network that I used as a file server for my main computer. However I've really become frustrated with the slow up/down speeds I'm getting, about 100KB/s, which is about as bad as if I was using a "real" server. My setup is as follows: ISP |... (4 Replies)
Discussion started by: CyberCat
4 Replies

5. IP Networking

Local Lan, no-ip directed DNS forward, surf within lan

Hi, We have a website running on a local centos 5.4 surfer, static IP. The domain.com uses no-ip.com to take care of the DNS, it forwards all to my server. My router receives the port 80 call, routes it to my server and the world can see domain.com perfectly fine. However, we cannot see... (3 Replies)
Discussion started by: lawstudent
3 Replies

6. Shell Programming and Scripting

cut, sed, awk too slow to retrieve line - other options?

Hi, I have a script that, basically, has two input files of this type: file1 key1=value1_1_1 key2=value1_2_1 key4=value1_4_1 ... file2 key2=value2_2_1 key2=value2_2_2 key3=value2_3_1 key4=value2_4_1 ... My files are 10k lines big each (approx). The keys are strings that don't... (7 Replies)
Discussion started by: fzd
7 Replies

7. Red Hat

Ethtool script not running on boot.

Hi, there is an incompatibility between my NIC and my Western Digital Powerline adapters which appears to cause autonegotiation to fail intermittently, most often following a reboot. Running the following Ethtool script will establish a connection immediately: #!/bin/sh # chkconfig: 345 85... (12 Replies)
Discussion started by: torley
12 Replies

8. UNIX and Linux Applications

VSFTPD slow download outside of LAN

Hello all. I have VSFTP up and ruining on ubuntu 10.04. It works great inside LAN, however when it comes to using it outside i have some problems. If i try to connect from filezila i get the listing and if i try to download it shows it will take forever to receive a file which is larger then 1mb,... (2 Replies)
Discussion started by: mantas1
2 Replies

9. UNIX for Beginners Questions & Answers

Ethtool command not working

Hi All, I am trying to find my NIC card speed in centos 5. But i am unable to get the info using ethtool command. # ethtool eth0 Settings for eth0: Link detected: yes and even tried to look for the file # cd /sys/class/net/eth0/speed files which is also not available.... (8 Replies)
Discussion started by: sravani25
8 Replies
EM(4)							   BSD Kernel Interfaces Manual 						     EM(4)

NAME
em -- Intel(R) PRO/1000 Gigabit Ethernet adapter driver SYNOPSIS
To compile this driver into the kernel, place the following line in your kernel configuration file: device em Alternatively, to load the driver as a module at boot time, place the following line in loader.conf(5): if_em_load="YES" DESCRIPTION
The em driver provides support for PCI Gigabit Ethernet adapters based on the Intel 82540, 82541ER, 82541PI, 82542, 82543, 82544, 82545, 82546, 82546EB, 82546GB, 82547, 82571, 81572, 82573, and 82574 Ethernet controller chips. The driver supports Transmit/Receive checksum off- load and Jumbo Frames on all but 82542-based adapters. Furthermore it supports TCP segmentation offload (TSO) on all adapters but those based on the 82543, 82544 and 82547 controller chips. The identification LEDs of the adapters supported by the em driver can be controlled via the led(4) API for localization purposes. For further hardware information, see the README included with the driver. For questions related to hardware requirements, refer to the documentation supplied with your Intel PRO/1000 adapter. All hardware require- ments listed apply to use with FreeBSD. Support for Jumbo Frames is provided via the interface MTU setting. Selecting an MTU larger than 1500 bytes with the ifconfig(8) utility configures the adapter to receive and transmit Jumbo Frames. The maximum MTU size for Jumbo Frames is 16114. This driver version supports VLANs. The em driver supports the following media types: autoselect Enables auto-negotiation for speed and duplex. 10baseT/UTP Sets 10Mbps operation. Use the mediaopt option to select full-duplex mode. 100baseTX Sets 100Mbps operation. Use the mediaopt option to select full-duplex mode. 1000baseSX Sets 1000Mbps operation. Only full-duplex mode is supported at this speed. 1000baseTX Sets 1000Mbps operation. Only full-duplex mode is supported at this speed. The em driver supports the following media options: full-duplex Forces full-duplex operation half-duplex Forces half-duplex operation. Only use mediaopt to set the driver to full-duplex. If mediaopt is not specified, the driver defaults to half-duplex. For more information on configuring this device, see ifconfig(8). HARDWARE
The em driver supports Gigabit Ethernet adapters based on the Intel 82540, 82541ER, 82541PI, 82542, 82543, 82544, 82545, 82546, 82546EB, 82546GB, 82547, 82571, 82572, 82573, and 82574 controller chips: o Intel PRO/1000 CT Network Connection (82547) o Intel PRO/1000 F Server Adapter (82543) o Intel PRO/1000 Gigabit Server Adapter (82542) o Intel PRO/1000 GT Desktop Adapter (82541PI) o Intel PRO/1000 MF Dual Port Server Adapter (82546) o Intel PRO/1000 MF Server Adapter (82545) o Intel PRO/1000 MF Server Adapter (LX) (82545) o Intel PRO/1000 MT Desktop Adapter (82540) o Intel PRO/1000 MT Desktop Adapter (82541) o Intel PRO/1000 MT Dual Port Server Adapter (82546) o Intel PRO/1000 MT Quad Port Server Adapter (82546EB) o Intel PRO/1000 MT Server Adapter (82545) o Intel PRO/1000 PF Dual Port Server Adapter (82571) o Intel PRO/1000 PF Quad Port Server Adapter (82571) o Intel PRO/1000 PF Server Adapter (82572) o Intel PRO/1000 PT Desktop Adapter (82572) o Intel PRO/1000 PT Dual Port Server Adapter (82571) o Intel PRO/1000 PT Quad Port Server Adapter (82571) o Intel PRO/1000 PT Server Adapter (82572) o Intel PRO/1000 T Desktop Adapter (82544) o Intel PRO/1000 T Server Adapter (82543) o Intel PRO/1000 XF Server Adapter (82544) o Intel PRO/1000 XT Server Adapter (82544) LOADER TUNABLES
Tunables can be set at the loader(8) prompt before booting the kernel or stored in loader.conf(5). hw.em.rxd Number of receive descriptors allocated by the driver. The default value is 1024 for adapters newer than 82547, and 256 for older ones. The 82542 and 82543-based adapters can handle up to 256 descriptors, while others can have up to 4096. hw.em.txd Number of transmit descriptors allocated by the driver. The default value is 1024 for adapters newer than 82547, and 256 for older ones. The 82542 and 82543-based adapters can handle up to 256 descriptors, while others can have up to 4096. hw.em.rx_int_delay This value delays the generation of receive interrupts in units of 1.024 microseconds. The default value is 0, since adapters may hang with this feature being enabled. hw.em.rx_abs_int_delay If hw.em.rx_int_delay is non-zero, this tunable limits the maximum delay in which a receive interrupt is generated. hw.em.tx_int_delay This value delays the generation of transmit interrupts in units of 1.024 microseconds. The default value is 64. hw.em.tx_abs_int_delay If hw.em.tx_int_delay is non-zero, this tunable limits the maximum delay in which a transmit interrupt is generated. FILES
/dev/led/em* identification LED device nodes EXAMPLES
Make the identification LED of em0 blink: echo f2 > /dev/led/em0 Turn the identification LED of em0 off again: echo 0 > /dev/led/em0 DIAGNOSTICS
em%d: Unable to allocate bus resource: memory A fatal initialization error has occurred. em%d: Unable to allocate bus resource: interrupt A fatal initialization error has occurred. em%d: watchdog timeout -- resetting The device has stopped responding to the network, or there is a problem with the network connection (cable). SUPPORT
For general information and support, go to the Intel support website at: http://support.intel.com. If an issue is identified with the released source code on the supported kernel with a supported adapter, email the specific information related to the issue to <freebsd@intel.com>. SEE ALSO
altq(4), arp(4), igb(4), led(4), netintro(4), ng_ether(4), polling(4), vlan(4), ifconfig(8) HISTORY
The em device driver first appeared in FreeBSD 4.4. AUTHORS
The em driver was written by Intel Corporation <freebsd@intel.com>. BUGS
Hardware-assisted VLAN processing is disabled by default. You can enable it on an em interface using ifconfig(8). BSD
October 11, 2011 BSD
All times are GMT -4. The time now is 09:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy