Improve your iSCSI performance using jumbo frames

What are jumbo frames? Jumbo frames are ethernet frames with more than 1500 bytes of payload. Conventionally, jumbo frames can carry up to 9000 bytes of payload, but variations exist and some care must be taken when using the term.

Why use jumbo frames? Enabling them on your network equipment and on your NICs you will experiment a performance boost specially with iSCSI protocol that works over a standard ethernet network.

Implementation of Jumbo frames must be done with some rules:

  • Same MTU for all server present in the network
  • Network card must support a MTU over 1500
  • Switch must support a MTU over 1500
  • Switch must support a MTU over 1500 on a VLAN

How to enable jumbo frames on RHEL/CentOS? Enabling jumbo frames on linux is really simple: edit the NIC configuration and append MTU=9000.

Don’t forget to enable them also on your switch/router!

# vi /etc/sysconfig/network-script/ifcfg-<your_nic> # ex. eth0

MTU=9000

Then restart the single interface…

ifdown eth0; ifup eth0

…or the entire network service

service network restart

After all verify that the new configuration has been correctly applied:

# ifconfig eth0

If the configuration is ok you will see a response like this:

eth0      Link encap:Ethernet  HWaddr xx:xx:xx:xx:xx:xx
inet addr:x.x.x.x  Bcast:x.x.x.x  Mask:x.x.x.x
UP BROADCAST RUNNING MULTICAST  MTU:9000 Metric:1

If you’re using bonding, you need to enable jumbo frames only on bond device configuration:

# vi /etc/sysconfig/network-script/ifcfg-bond0

MTU=9000

2 thoughts on “Improve your iSCSI performance using jumbo frames

  1. To avoid MTU hinting you may check your dmesg:

    # grep jumbo /var/log/dmesg
    r8169 0000:03:00.0: eth2: jumbo features [frames: 7152 bytes, tx checksumming: ok]
    r8169 0000:08:00.0: eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko]

  2. The “jumbo frame” is link-dependant. You may need to set lower values:
    – wi-fi supports about 7935

    You can try some good values on your interfaces and check for errors, then set the value on the bonded interface.

    #ping supports MTU hinting. Reading the number in parentheses
    $ ping 10.0.0.1 -M do
    PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data
    ^^^^^^
    The ICMP payload was 56, and the total payload 84 bytes. So to hit an MTU=1500 we have to send a pkt with a payload of:
    1500-84+56=1472

    The highest unfragmentable payload is tested with
    $ ping 10.0.0.1 -M do -s 1472
    PING 10.0.0.1 (10.0.0.1) 1472(1500) bytes of data

    Higher payloads cause a nice error:
    Frag needed and DF set (mtu = 1500)

Lascia un commento