slow network throughput, how to improve?

[SilverNodashi]

Hi all,

 

I would like some input on this one please.

 

Two CentOS 5.5 XEN servers, with 1GB NIC's, connected to a 1GB switch

transfer files to each other at about 30MB/s between each other.

 

Both servers have the following setup:

CentOS 5.5 x64

XEN 3.0 (from xm info: xen_caps : xen-3.0-x86_64

xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64)

kernel 2.6.18-194.11.3.el5xen

1GB NIC's

7200rpm SATA HDD's

 

The hardware configuration can't change, I need to use these servers

as they are. They are both used in production with a few xen domU's

virtual machines running on them.

I want to connect them both to a SAN, with gigabit connectivity and

would like to know how I can increase network performance a bit, as

is.

The upstream datacentre only supplies 100MB network connection, so in

the internet side of it isn't much of a problem. If I do manage to

reach 100MB that will be my limit in any case.

 

 

root@zaxen02.securehosting.co.za:/vm/xen/template/centos-5-x64-cpanel

root@zaxen02.securehosting.co.za

root@zaxen02.securehosting.co.za's password:

centos-5-x64-cpanel.tar.gz

100% 1163MB

29.1MB/s 00:40

 

 

iperf indicates that the network throughput is about 930MB though:

 

root@zaxen01:[~]$ iperf -s

------------------------------------------------------------

Server listening on TCP port 5001

TCP window size: 85.3 KByte (default)

------------------------------------------------------------

[ 4] local 196.34.x.x port 5001 connected with 196.34.x.x port 45453

[ ID] Interval Transfer Bandwidth

[ 4] 0.0-10.0 sec 1.09 GBytes 935 Mbits/sec

 

root@zaxen02:[~]$ iperf -c zaxen01

------------------------------------------------------------

Client connecting to zaxen01, TCP port 5001

TCP window size: 16.0 KByte (default)

------------------------------------------------------------

[ 3] local 196.34.x.x port 45453 connected with 196.34.x.x port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 1.09 GBytes 936 Mbits/sec

 

 

Is iperf really that accurate, or reliable in this instance, since the

packet size is so small that it probably goes straight to memory,

instead of HDD? But at the same time, changing the packet size to

10MB, 100MB and 1000MB respectively doesn't seem to degrade

performance much either:

 

root@zaxen02:[~]$ iperf -w 10M -c zaxen01

------------------------------------------------------------

Client connecting to zaxen01, TCP port 5001

TCP window size: 256 KByte (WARNING: requested 10.0 MByte)

------------------------------------------------------------

[ 3] local 196.34.x.x port 36756 connected with 196.34.x.x port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 1.07 GBytes 921 Mbits/sec

root@zaxen02:[~]$ iperf -w 100M -c zaxen01

------------------------------------------------------------

Client connecting to zaxen01, TCP port 5001

TCP window size: 256 KByte (WARNING: requested 100 MByte)

------------------------------------------------------------

[ 3] local 196.34.x.x port 36757 connected with 196.34.x.x9 port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 1.08 GBytes 927 Mbits/sec

root@zaxen02:[~]$ iperf -w 1000M -c zaxen01

------------------------------------------------------------

Client connecting to zaxen01, TCP port 5001

TCP window size: 256 KByte (WARNING: requested 1000 MByte)

------------------------------------------------------------

[ 3] local 196.34.x.x port 36758 connected with 196.34.x.x port 5001

[ ID] Interval Transfer Bandwidth

[ 3] 0.0-10.0 sec 1.04 GBytes 895 Mbits/sec