Parallella Community

Hi,

My initial project for the $99 board was going to be a low power, high speed network packet processor, starting with GbE. I've since learned that the built in GbE port is not capable of even half GbE speeds.

So I was wondering if it would be reasonable to expect that the epiphany could be made to talk to a GbE port wired to the GPIO pins either directly, or through the FPGA (I'm assuming the GPIO and other pins are wired through the FPGA), and whether it would work at full GbE speeds or not?

I'm expecting to need some FPGA glue to handle the physical and Ethernet layers, but can then feed the data directly through to the epiphany at full line speed (800-950mbps). Is that a realistic assumption?

My current idea is to have a single epiphany core dispatch to the rest from a ring buffer fed by the fpga. That may not be ideal, so I may have to add some extra logic so each core can either directly read from the ring buffer, or each core gets its own buffer, then have to share a common memory area to collect the information. I would prefer the former single core to dispatch, so I don't have to worry about properly managing shared memory between 16 separate epiphany cores and the ARM. Handling tcp would be interesting, may need to do packet re-ordering before it gets to the epiphany cores doing the main work, just to simplify the main epiphany code.

Thanks for reading, if anyone has some ideas or insight, I'd love to hear it

Interesting. Thanks for the information.

Looking at that pdf you linked, it seems that the block RAMs are fairly small, but could be used as a small temporary buffer (560KB assuming that the portion of the fpga that is occupied by parallela functions doesn't use any), but the PL (programmable logic aka dsp+fpga) section seems to support 1866 Mb/s DDR3 via its i/o pins, which would mean a high speed buffer could be implemented, assuming the necessary pins are available. I also notice that the Zynq has 4 separate DDR2/3 ram interfaces, but two are dedicated to the PL section, assuming those links are full speed, and the right pins are available that could be used to install buffer/work-ram as well.

It does seem like it may be possible to redirect one (or both) of the GbE interfaces directly to the PL section. The three questions that pop up in my mind are, does this data have to pass back and forth between the PS section and the PL section over the AMBA bus? I wan't to say no, since the Zynq has a pin multiplexer so it should be able to remap the pins directly to the fpga. Second, are there pins broken out that can be mapped to that ethernet module (I'll have to do some more research on which banks the zync eth1 phy signals are mapped to, and which pins can be assigned to the phy, if someone already knows this, please post!), as well as can we easily access the Zynq config bits to re-route the ethernet peripheral to the PL block (board specific driver, or firmware devicetree perhaps?). And third, is there enough room left in the FPGA to implement a MAC, and maybe even some basic TCP handling (packet reordering, and merging). 85k logic cells may not be enough. Of course some of that work can be dumped onto the Epiphany, and even the arm if necessary.

Feel free to stop reading this wall of text here if you don't care about my reasons for even thinking about this at all. And thank you for even getting this far

In case anyone is wondering why I'd even bother, and why I don't just use the ARM cpu, well I've already written some fairly performant C code that runs on linux, and tries its best to process packets as fast as possible (as I can get it in userland). I have a quad core Xeon E3 (3.2Ghz) based server here and the main packet processing thread ends up using a significant amount of cpu at full GbE speeds, and the secondary thread that just pulls the packet data off the raw socket actually uses a fair amount of cpu as well. The ARM cores in the parallella board will likely not be able to handle even 100mbps. Also, I seem to like picking the hardest possible way of learning new things (I'm totally new to fpgas).

I would not expect any PL ethernet implementation to be any faster than the built-in ones.

I'm not sure where you get this statement from "I've since learned that the built in GbE port is not capable of even half GbE speeds".
But my guess is that it's using the built-in linux ARM stack. I don't see why you couldn't bypass the ARM all together, you'll just need to connect to the AMBA interconnect and implement your own custom control driver.

You seem to be confused .... the PHYs are external in all cases, you can't implement a PHY in the FPGA fabric.

What's in the Zynq are two ethernet MAC and for those you can either connect to an external PHY through the MIO dedicated pins, or redirect those signals to the PL (EMIO) and route them to an external PHY through normal FPGA pins.

And on the other side those MACs are connected to the internal AMBA interconnect and you can access this interconnect from PL.

Finally the thread you referenced is with the full linux stack and user-space access so I wouldn't be surprised if the throughput limitation is just due to the software rather than any HW. The interconnect port the MAC is connected to isn't the fastest (3.2Gbits, but should still allow more than 300Mbis)