Data transfer rate with different interfaces

Any technical questions about the Epiphany chip and Parallella HW Platform.

Moderator: aolofsson

Data transfer rate with different interfaces

Postby krmld » Thu Sep 03, 2015 6:04 am

Hi all,
in my project I need to read large amounts of data to RAM of parallella from any storage device. I simply allocating memory on RAM and reading file from source, speed requirement is >80MB/s. So far, I can get only ~25MB/s with different interfaces (see attachment). Kernel is compiled from Xilinx's repository.
My questions are:
  • What factors on hardware do influence to transfer speed? (So I can calculate transfer speed knowing my hardware specifications)
  • Can I achieve >80MB/s speed on reading from "any storage" to memory on parallella? And how it can be done?
With USB and SD I don't expect to get rates higher than 40MB/s.
Thanks in advance!
Attachments
par.png
par.png (21.65 KiB) Viewed 13768 times
Last edited by krmld on Fri Sep 04, 2015 6:30 am, edited 1 time in total.
krmld
 
Posts: 5
Joined: Wed Sep 02, 2015 12:05 pm

Re: Data transfer rate with different interfaces

Postby aolofsson » Thu Sep 03, 2015 3:03 pm

Nice table! So just to confiurm, this is based on measurements on Parallella?

Comments:

Someone else might comment on how to optimize file transfers over the network to not make it CPU bound? 80MB/s "should be close to possible...

Andreas
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: Data transfer rate with different interfaces

Postby krmld » Fri Sep 04, 2015 5:00 am

Yes, all measurements are taken from parallella board.

Actually, we have built cluster from 5 parallella boards and wrote custom map/reduce framework for big data processing. And with speeds less than 80MB/s all our work is worthless for big data processing, compared to commodity clusters. So we are now researching on getting daughter board with SATA support which will be connected to PEC connectors.
  • Till we get SATA on parallella, it would be very helpful if someone will advice how to achieve ~80MB/s over ethernet.
  • Advices on creating daughter boards with SATA also appreciated.
  • Or what would you advice to get high speed data transfer on parallella?
Thanks for reply!
krmld
 
Posts: 5
Joined: Wed Sep 02, 2015 12:05 pm

Re: Data transfer rate with different interfaces

Postby sebraa » Fri Sep 04, 2015 1:04 pm

Using the current images, I did a few measurements.

I run a direct connection between an Intel GBit Ethernet and the Parallella, and ran the following commands:
on the laptop: "dd if=/dev/zero bs=1M | pv | nc parallella 12345"
on the parallella: "nc -lp 12345 | pv >/dev/null"

I got about 40 MiB/s, with the Parallella being 75% busy (1 core netcat, 1/2 core pv), using the newest image on a stock configuration.
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm

Re: Data transfer rate with different interfaces

Postby aolofsson » Fri Sep 04, 2015 2:15 pm

sebraa wrote:Using the current images, I did a few measurements.

I got about 40 MiB/s, with the Parallella being 75% busy (1 core netcat, 1/2 core pv), using the newest image on a stock configuration.


For reference, what are you getting for interface speed with something like iperf?
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

Re: Data transfer rate with different interfaces

Postby sebraa » Fri Sep 04, 2015 6:21 pm

Using iperf, I get about 520 MBit/s (laptop as server), or about 580 MBit/s (Parallella as server).
When testing both directions at once (-d parameter), I get about (35 / 187) MBit/s (laptop as server; but differing widely), or about (380 / 120) MBit/s (Parallella as server).

Again, this is on a stock configuration using the 3.14.12-parallella-xilinx-g40a90c3 kernel.
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm

Re: Data transfer rate with different interfaces

Postby ajtravis » Sat Sep 05, 2015 3:55 pm

sebraa wrote:Using iperf, I get about 520 MBit/s (laptop as server), or about 580 MBit/s (Parallella as server).
When testing both directions at once (-d parameter), I get about (35 / 187) MBit/s (laptop as server; but differing widely), or about (380 / 120) MBit/s (Parallella as server).

Again, this is on a stock configuration using the 3.14.12-parallella-xilinx-g40a90c3 kernel.


Hi, sebraa.

I've used "lmdd" for i/o benchmarking instead of "dd": http://www.bitmover.com/lmbench/lmdd.8.html

It might be worth comparing your "dd" results with "lmdd".

Bye,

Tony.
Minke Informatics Limited, Registered in Scotland - Company No. SC419028
User avatar
ajtravis
 
Posts: 166
Joined: Fri Jul 18, 2014 12:54 pm
Location: Scotland (UK)

Re: Data transfer rate with different interfaces

Postby tnt » Sun Sep 06, 2015 9:49 am

If you want to improve performance a bit :
- Use larger MTU on network
- Use more aggressive DDR timings (the default ones are _very_ conservative, way under what the components are spec'd for).
- Use a memcpy library with optimized NEON code
tnt
 
Posts: 408
Joined: Mon Dec 17, 2012 3:21 am

Re: Data transfer rate with different interfaces

Postby sebraa » Mon Sep 07, 2015 8:29 am

I only wanted to see whether the extremely low numbers reported in the first post are realistic. I don't care too much about getting higher numbers. My workloads are limited by other means, not by the network interface, and the only Epiphany cluster available to us uses a 100 MBit/s switch (don't ask me, I didn't build it...).

The dd/netcat combination was chosen since it is not a benchmark, but a somewhat more realistic workload (that is, both cores working together to produce data elements).
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm


Return to Epiphany and Parallella Q & A

Who is online

Users browsing this forum: MSN [Bot] and 6 guests