Page 1 of 1

Data transfer rate with different interfaces

PostPosted: Thu Sep 03, 2015 6:04 am
by krmld
Hi all,
in my project I need to read large amounts of data to RAM of parallella from any storage device. I simply allocating memory on RAM and reading file from source, speed requirement is >80MB/s. So far, I can get only ~25MB/s with different interfaces (see attachment). Kernel is compiled from Xilinx's repository.
My questions are:
  • What factors on hardware do influence to transfer speed? (So I can calculate transfer speed knowing my hardware specifications)
  • Can I achieve >80MB/s speed on reading from "any storage" to memory on parallella? And how it can be done?
With USB and SD I don't expect to get rates higher than 40MB/s.
Thanks in advance!

Re: Data transfer rate with different interfaces

PostPosted: Thu Sep 03, 2015 3:03 pm
by aolofsson
Nice table! So just to confiurm, this is based on measurements on Parallella?

Comments:

Someone else might comment on how to optimize file transfers over the network to not make it CPU bound? 80MB/s "should be close to possible...

Andreas

Re: Data transfer rate with different interfaces

PostPosted: Fri Sep 04, 2015 5:00 am
by krmld
Yes, all measurements are taken from parallella board.

Actually, we have built cluster from 5 parallella boards and wrote custom map/reduce framework for big data processing. And with speeds less than 80MB/s all our work is worthless for big data processing, compared to commodity clusters. So we are now researching on getting daughter board with SATA support which will be connected to PEC connectors.
  • Till we get SATA on parallella, it would be very helpful if someone will advice how to achieve ~80MB/s over ethernet.
  • Advices on creating daughter boards with SATA also appreciated.
  • Or what would you advice to get high speed data transfer on parallella?
Thanks for reply!

Re: Data transfer rate with different interfaces

PostPosted: Fri Sep 04, 2015 1:04 pm
by sebraa
Using the current images, I did a few measurements.

I run a direct connection between an Intel GBit Ethernet and the Parallella, and ran the following commands:
on the laptop: "dd if=/dev/zero bs=1M | pv | nc parallella 12345"
on the parallella: "nc -lp 12345 | pv >/dev/null"

I got about 40 MiB/s, with the Parallella being 75% busy (1 core netcat, 1/2 core pv), using the newest image on a stock configuration.

Re: Data transfer rate with different interfaces

PostPosted: Fri Sep 04, 2015 2:15 pm
by aolofsson
sebraa wrote:Using the current images, I did a few measurements.

I got about 40 MiB/s, with the Parallella being 75% busy (1 core netcat, 1/2 core pv), using the newest image on a stock configuration.


For reference, what are you getting for interface speed with something like iperf?

Re: Data transfer rate with different interfaces

PostPosted: Fri Sep 04, 2015 6:21 pm
by sebraa
Using iperf, I get about 520 MBit/s (laptop as server), or about 580 MBit/s (Parallella as server).
When testing both directions at once (-d parameter), I get about (35 / 187) MBit/s (laptop as server; but differing widely), or about (380 / 120) MBit/s (Parallella as server).

Again, this is on a stock configuration using the 3.14.12-parallella-xilinx-g40a90c3 kernel.

Re: Data transfer rate with different interfaces

PostPosted: Sat Sep 05, 2015 3:55 pm
by ajtravis
sebraa wrote:Using iperf, I get about 520 MBit/s (laptop as server), or about 580 MBit/s (Parallella as server).
When testing both directions at once (-d parameter), I get about (35 / 187) MBit/s (laptop as server; but differing widely), or about (380 / 120) MBit/s (Parallella as server).

Again, this is on a stock configuration using the 3.14.12-parallella-xilinx-g40a90c3 kernel.


Hi, sebraa.

I've used "lmdd" for i/o benchmarking instead of "dd": http://www.bitmover.com/lmbench/lmdd.8.html

It might be worth comparing your "dd" results with "lmdd".

Bye,

Tony.

Re: Data transfer rate with different interfaces

PostPosted: Sun Sep 06, 2015 9:49 am
by tnt
If you want to improve performance a bit :
- Use larger MTU on network
- Use more aggressive DDR timings (the default ones are _very_ conservative, way under what the components are spec'd for).
- Use a memcpy library with optimized NEON code

Re: Data transfer rate with different interfaces

PostPosted: Mon Sep 07, 2015 8:29 am
by sebraa
I only wanted to see whether the extremely low numbers reported in the first post are realistic. I don't care too much about getting higher numbers. My workloads are limited by other means, not by the network interface, and the only Epiphany cluster available to us uses a 100 MBit/s switch (don't ask me, I didn't build it...).

The dd/netcat combination was chosen since it is not a benchmark, but a somewhat more realistic workload (that is, both cores working together to produce data elements).