Epiphany III to Zynq7010 DRAM bandwidth

Any technical questions about the Epiphany chip and Parallella HW Platform.

Moderator: aolofsson

Epiphany III to Zynq7010 DRAM bandwidth

Postby jamesh » Thu Dec 11, 2014 10:07 pm

After a variety of tests (including with both fast.ldf and internal.ldf), I have observed that the memory bus bandwidth between the Parallella DRAM and the Epiphany III tops out at about 120 MB/s (regardless of the number of cores engaged in transfers...).

Basically this is equivalent to the throughput of Gigabit Ethernet, and it seems rather disappointing, given that the Epiphany chip itself is supposedly capable of 8 GB/s off-chip bandwidth (which would make it competitive with PCIe-based graphics cards...).

Has anyone been able to transfer between the Epiphany and DRAM at speeds greater than 120 MB/s? (I have mostly focused on testing the DMA, by the way.) Or is this the maximal throughput speed of that bus?
jamesh
 
Posts: 2
Joined: Thu Dec 11, 2014 9:54 pm

Re: Epiphany III to Zynq7010 DRAM bandwidth

Postby FHuettig » Thu Dec 11, 2014 10:37 pm

Hi Jamesh,

jamesh wrote:Basically this is equivalent to the throughput of Gigabit Ethernet, and it seems rather disappointing, given that the Epiphany chip itself is supposedly capable of 8 GB/s off-chip bandwidth (which would make it competitive with PCIe-based graphics cards...).


The 8GB/sec capability requires all 4 ports (north/south/east/west) to be running at 1GB/sec in both TX and RX directions simultaneously. For Parallella we only have a single port (FPGA is "east" of the Epiphany) and because of limitations of the Zynq we can't run more than 600MB/s. Worse, the existing eLink FPGA code limits the timing to 1/2 rate from Epiphany to FPGA. So if you are only testing writes from the Epiphany to the ARM (one direction on one port) the max possible is 300MB/s and 120MB/s sounds about right. From the Arm to the Epiphany the limitation is the AXI interface logic which can only handle 160MB/s max.

The good news is that I have completely redesigned the FPGA logic from the ground up to improve performance and usabiliby and will be releasing a new bitstream. The bad news is I've spent this week trying to track down a bug that's causing problems with the streaming data transfers (the ones with best performance, using DMA). Not sure when it will be out but sure hope it will be very soon. The changes do require an update to the SDK but we'll have that too.
-- Fred -- Hardware Guy --
FHuettig
 
Posts: 142
Joined: Wed Jan 29, 2014 8:30 pm
Location: Lexington, MA, USA

Re: Epiphany III to Zynq7010 DRAM bandwidth

Postby jamesh » Sat Dec 13, 2014 1:42 am

Ok, thanks for the info!

I was testing DMA reads by the Epiphany (so ARM -> Epiphany), which is the context in which I'm topping out at 120 MB/s...

Basically, my test algorithm is:
----------------------------------------------------------------------------
1. Host fills up the upper half (0x01000000-0x01FFFFFF, 16MB) of the shared DRAM.
2. Load cores.
3. Start OS timer.
4. Run cores which do the following:
Code: Select all
Repeat (for X iterations -- you know, maybe a couple thousand):
        Repeat (for 4 * 16 = 64 iterations):
                Request DMA read from 16K block of DRAM to banks 1 and 2 of SRAM
                Increment block pointer by 256K (= 16K * 16) to reach "next" block destined for this core.

[When core is done, notify host.]
5. When all are finished, stop OS timer.
----------------------------------------------------------------------------
So Core 0 uses DMA to read the following regions from shared DRAM into its local SRAM:
0x01000000-0x01003FFF (in shared mem)
0x01040000-0x01043FFF
0x01080000-0x01083FFF
0x010C0000-0x010C3FFF
etc.

Core 1 reads:
0x01004000-0x01007FFF
0x01044000-0x01047FFF
0x01084000-0x01087FFF
etc., etc.

Thus, each core performs 4 * 16 = 64 reads from the Zynq DRAM (between all 16 cores, I am reading the entire 16MB upper shared region), and then it goes back to the "beginning" and does it again, etc.

So, given the current hardware setup, what's the maximum theoretical transfer rate from the ARM to the Epiphany? 160 MB/s? (even with a perfect SDK?)

-James
jamesh
 
Posts: 2
Joined: Thu Dec 11, 2014 9:54 pm

Re: Epiphany III to Zynq7010 DRAM bandwidth

Postby notzed » Wed Dec 17, 2014 1:16 am

FHuettig wrote:The changes do require an update to the SDK but we'll have that too.


Is there anything specific on what is changing from the software side? Is it some addresses or initiialisation/setup, etc?
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia


Return to Epiphany and Parallella Q & A

Who is online

Users browsing this forum: No registered users and 4 guests

cron