How to check ethernet bandwidth of the Parallella

A forum for documenting the Parallella and Epiphany . (Not Q&A!)

How to check ethernet bandwidth of the Parallella

Postby aolofsson » Fri Aug 22, 2014 9:08 pm

Step 1: Start iperf on any machine your network:
Code: Select all
iperf -s


Step 2: Start an iperf client on a parallella board:
Code: Select all
iperf -c <ip-address-of machine that-ran-"iperf-s">


iperf run on a Parallella board with the latest Linux 3.14.12 kernel reports 500-600Mb/s. What does your board report?

Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: How to check ethernet bandwidth of the Parallella

Postby ajtravis » Fri Aug 22, 2014 11:39 pm

aolofsson wrote:Step 1: Start iperf on any machine your network:
Code: Select all
iperf -s

[...]
iperf run on a Parallella board with the latest Linux 3.14.12 kernel reports 500-600Mb/s. What does your board report?

Hi, Andreas.

Here are the results for my 4@SKU101020 cluster:
Code: Select all
root@node4:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.0.4 port 5001 connected with 192.168.0.1 port 38569
[  5] local 192.168.0.4 port 5001 connected with 192.168.0.2 port 33695
[  6] local 192.168.0.4 port 5001 connected with 192.168.0.3 port 58190
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.1 sec   175 MBytes   146 Mbits/sec
[  5]  0.0-10.1 sec   157 MBytes   131 Mbits/sec
[  6]  0.0-10.0 sec   183 MBytes   153 Mbits/sec

I built a kernel from the GitHub sources and I'm using the default rootfs:
Code: Select all
node1:~> dsh -Ma -- uname -a
node1: Linux node1 3.12.0-parallella-g71330d8 #1 SMP PREEMPT Wed Aug 13 01:45:12 UTC 2014 armv7l armv7l armv7l GNU/Linux
node2: Linux node2 3.12.0-parallella-g71330d8 #1 SMP PREEMPT Wed Aug 13 01:45:12 UTC 2014 armv7l armv7l armv7l GNU/Linux
node3: Linux node3 3.12.0-parallella-g71330d8 #1 SMP PREEMPT Wed Aug 13 01:45:12 UTC 2014 armv7l armv7l armv7l GNU/Linux
node4: Linux node4 3.12.0-parallella-g71330d8 #1 SMP PREEMPT Wed Aug 13 01:45:12 UTC 2014 armv7l armv7l armv7l GNU/Linux
node1:~>

Interconnected using a Netgear GS105 Gbit switch.

Bye,

Tony.
Minke Informatics Limited, Registered in Scotland - Company No. SC419028
User avatar
ajtravis
 
Posts: 166
Joined: Fri Jul 18, 2014 12:54 pm
Location: Scotland (UK)

Re: How to check ethernet bandwidth of the Parallella

Postby xilman » Sat Aug 23, 2014 8:31 am

Code: Select all
root@par2:/etc# iperf -c 192.168.1.201
------------------------------------------------------------
Client connecting to 192.168.1.201, TCP port 5001
TCP window size: 20.7 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.202 port 42986 connected with 192.168.1.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   242 MBytes   202 Mbits/sec
root@par2:/etc# uname -a
Linux par2.home.brnikat.com 3.12.0-g0bc9c3a-dirty #6 SMP PREEMPT Wed Jan 29 10:25:25 CET 2014 armv7l armv7l armv7l GNU/Linux
root@par2:/etc#

Interconnect is a Netgear GS108.

Had a nasty shock at first --- the switch is a new replacement for an old 100M hub that was lying around so the ethernet interfaces were set up as 100M duplex. Accordingly the bandwidth was only 22 Mbits/sec.

Added in edit:

First I built iperf on a AMD64 Gentoo box (192.168.1.3) then fired up the server there and ran the client on par2 (192.168.1.202) as above. The results are interesting, and not entirely repeatable:
Code: Select all
root@par2:/etc# iperf -c 192.168.1.3
------------------------------------------------------------
Client connecting to 192.168.1.3, TCP port 5001
TCP window size: 20.7 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.202 port 54699 connected with 192.168.1.3 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   343 MBytes   288 Mbits/sec
root@par2:/etc# iperf -c 192.168.1.201
------------------------------------------------------------
Client connecting to 192.168.1.201, TCP port 5001
TCP window size: 20.7 KByte (default)
------------------------------------------------------------
[  3] local 192.168.1.202 port 42997 connected with 192.168.1.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   279 MBytes   234 Mbits/sec
root@par2:/etc#

The inter-Parallella bandwidth is now 50% greater.

The bandwidth to the Gentoo box is markedly greater than to either the other two experiments, despite the bits having to travel over many metres of cable and through two switches. My house has cat-5e cabling to most every room, all sockets of which feed to a Zyxel GS1019-24 switch. I suspect that the raw cpu power behind iperf is playing a significant role. It might be interesting to run a tightly optimized Gentoo on the Parallellas but that's a project for another time.
xilman
 
Posts: 80
Joined: Sat May 10, 2014 8:10 pm
Location: UK

Re: How to check ethernet bandwidth of the Parallella

Postby tajuma » Sat Aug 23, 2014 3:09 pm

aolofsson wrote:iperf run on a Parallella board with the latest Linux 3.14.12 kernel reports 500-600Mb/s. What does your board report?


Also running 3.14.12 kernel from the new and fresh parallella-linux git repo but only getting 245 Mbits/sec over ZyXEL GS108B switch (a laptop in the same switch gets 926 Mbits/s).
tajuma
 
Posts: 7
Joined: Tue Jun 10, 2014 4:22 pm

Re: How to check ethernet bandwidth of the Parallella

Postby greytery » Sat Aug 23, 2014 5:07 pm

Andreas,
aolofsson wrote:iperf run on a Parallella board with the latest Linux 3.14.12 kernel reports 500-600Mb/s.

Running 1 to 1 on (my guess, probably) a very quiet network. Looks like very little contention.
As a 'benchmark', it might be worth confirming the result on a 'quiet' network, or as 'point to point' as it can be, e.g. on the same switch. And also checking the bandwith in the other direction, i.e. into the parallella. (I just need a Round Tuit).

Tony,
ajtravis wrote:
Code: Select all
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.1 sec   175 MBytes   146 Mbits/sec
[  5]  0.0-10.1 sec   157 MBytes   131 Mbits/sec
[  6]  0.0-10.0 sec   183 MBytes   153 Mbits/sec


That looks like running multiple iperf clients, 3 into 1, i.e. it's a busy loaded network. Traffic is effectively local to the Netgear GS105 Gbit switch and doesn't 'get out' onto the rest of the network, or vice-versa. But the iperf traffic will queue against the input port of node 4. Each of the 3 nodes gets less than 1/3 of the contested bandwidth into node 4, which is ~fair. Wonder, if you tried something like 1-2 and 3-4 concurrently....? :roll:

Cheers,
tery
User avatar
greytery
 
Posts: 205
Joined: Sat Dec 07, 2013 12:19 pm
Location: ^Wycombe, UK

Re: How to check ethernet bandwidth of the Parallella

Postby aolofsson » Sun Aug 24, 2014 6:06 pm

tery, tony, xilman:

Thanks for the feedback! Three variables have changed: new defconfig, updated linux kernel, and using a newer linaro tree. I hope to have a new beta headless release version up this week for some brave souls to try.

Right now, it does look like the new configuration has possibly boosted ethernet performance from ~200Mb/s to 500Mb/s. I haven't found the root cause for this nice bump, so fingers cross that it's actually "true". :D

Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: How to check ethernet bandwidth of the Parallella

Postby Morgaine » Mon Aug 25, 2014 9:54 am

Alternatively use the venerable nuttcp utility and you can compare your throughput against the nice little table of SBC Network Throughputs that we created at Element14 for the most common current ARM boards, plus a few others for comparison.

Details of running nuttcp are given in the linked thread, but in a nutshell it's simply to run "nuttcp -S" on some server "X" and then run "nuttcp -t X" and "nuttcp -r X" as transmit and receive tests to and from the server. Parallella can play the server or client role, whichever is more convenient. (iperf should of course give similar results, although comparing against our table is most fairly done using the same utility we used.)

The maximum theoretical TCP throughput over gigabit Ethernet is 941.482 Mbps with TCP TimeStamps enabled, or 949.285 Mbps without. All modern x86/amd64 equipment reaches that limiting throughput without any difficulty at all, but the gigabit-capable ARM boards we tested were nowhere near the theoretical maximum by quite some distance. Some boards couldn't even fully utilize a 100 Mbps link.

Hopefully this year will see newer ARM devices putting up a better showing. :-)

Morgaine.
Morgaine
 
Posts: 42
Joined: Tue Jul 02, 2013 8:29 pm

Re: How to check ethernet bandwidth of the Parallella

Postby ajtravis » Mon Aug 25, 2014 10:45 pm

greytery wrote:[...]
ajtravis wrote:
Code: Select all
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.1 sec   175 MBytes   146 Mbits/sec
[  5]  0.0-10.1 sec   157 MBytes   131 Mbits/sec
[  6]  0.0-10.0 sec   183 MBytes   153 Mbits/sec


That looks like running multiple iperf clients, 3 into 1, i.e. it's a busy loaded network. Traffic is effectively local to the Netgear GS105 Gbit switch and doesn't 'get out' onto the rest of the network, or vice-versa. But the iperf traffic will queue against the input port of node 4. Each of the 3 nodes gets less than 1/3 of the contested bandwidth into node 4, which is ~fair. Wonder, if you tried something like 1-2 and 3-4 concurrently....? :roll:
Cheers,

Hi, Tery.

No, "iperf -s" just keeps a log of all the connections made: I ran the tests on node[123] ->node4 one node at at time. Nothing else was using the network at the time, so I think it is the genuine point-to-point throughput.

Bye,

Tony.
Minke Informatics Limited, Registered in Scotland - Company No. SC419028
User avatar
ajtravis
 
Posts: 166
Joined: Fri Jul 18, 2014 12:54 pm
Location: Scotland (UK)

Re: How to check ethernet bandwidth of the Parallella

Postby greytery » Tue Aug 26, 2014 9:27 am

Hi Tony,
Thanks for the extra info.
aolofsson wrote: Right now, it does look like the new configuration has possibly boosted ethernet performance from ~200Mb/s to 500Mb/s.

ajtravis wrote:I ran the tests on node[123] ->node4 one node at at time. Nothing else was using the network at the time, so I think it is the genuine point-to-point throughput.

So, still flapping about ... you were using the current kernel, etc, which seems to have throughput issues.
IMHO If you ran concurrent iperf clients then it's likely that the throughput per Paralella would be lower.
The variation in your results per client is also very interesting. Noise..? (Red Herring Warning:Related to the USB fragility..??)

Something I've been concerned about is the level of dropouts on pings to/from the Parallella. Pi on the same cables etc, is fine.
I wonder if the latest fixes to the config and kernel will help. Can't wait to have a go!!

Cheers,
tery
User avatar
greytery
 
Posts: 205
Joined: Sat Dec 07, 2013 12:19 pm
Location: ^Wycombe, UK

Re: How to check ethernet bandwidth of the Parallella

Postby tajuma » Tue Aug 26, 2014 12:16 pm

greytery wrote:Something I've been concerned about is the level of dropouts on pings to/from the Parallella. Pi on the same cables etc, is fine.
I wonder if the latest fixes to the config and kernel will help. Can't wait to have a go!!

I noticed that with Linux-3.14.12-parallella-gaa3549 (i.e. the very latest DT, config and kernel) I also get lots of dropped packets and after some size (~100 bytes) limit nothing goes out of the box anymore when U-boot had touched the ethernet first. No ethernet in U-boot and the network works just fine in Linux. Some more details in http://forums.parallella.org/viewtopic.php?f=48&t=1667#p10370.
tajuma
 
Posts: 7
Joined: Tue Jun 10, 2014 4:22 pm

Next

Return to Quick Start Guides , Documentation, Examples (Start Here!)

Who is online

Users browsing this forum: No registered users and 1 guest

cron