localhost on the Parallella. Although I'm not recording them because they don't measure Ethernet throughput, they do give some idea of the overheads in the protocol stack when the communications hardware is eliminated from the path, and so they're a worthwhile sanity check to perform. If these numbers observed for localhost were close to those measured over Ethernet then the latter would be suspect, since throughput would be limited by the protocol stack instead of by the external path.
I think these last results mean that your measurements are safe, but still affected by whatever is causing your variability.
Obvious interfering candidates such as known running processes should be eliminated first when performing measurements. The big elephant in the room is the browser, since in today's Javascript-infested web both the CPU and the network will be in continual use unless Javascript has been turned off or blockers like NoScript are used well to control it. The security-conscious won't be allowing Javascript to run anyway, but when performing measurements involving the local machine it's best to go the whole way and terminate the browser completely. Even better is to use bare machines without desktops for the test.
If using headless machines over ssh then be aware that this dual use of the link means that keepalive packets may add a small amount of variability to the results, and any output sent over ssh during measurement will cause process rescheduling. In your one-liner loops, add a short sleep after nuttcp termination to add confidence that the result line isn't being sent over the link at the same time as the next iteration is running another nuttcp.
No firewall should be running anywhere in the path since that naturally adds overhead to communications. Using an in-service firewall machine directly as one endpoint for the tests would be especially inappropriate unless one is trying to evaluate the impact of service traffic on purpose. (Never leave the tool around on a public facing machine anyway, for obvious security reasons.)
CPU governors can have an impact on measurements as well. so be sure that both ends used for a test are using the performance governor. (Check with "cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor".)
With a tool like htop showing a load average of 0.0x % and no hidden use of bandwidth such as by forwarded packets, only the third digit of precision would normally show significant variation. (Terminate htop before running the test.) Variation in the 2nd digit of precision can still give us useful ballpark figures, but variation in the 1st digit as you have seen are a sign of an uncontrolled test environment, and aren't actually measuring the Ethernet performance.Statistics: Posted by Morgaine — Thu Oct 10, 2013 7:31 pm
]]>