Ideas for daughter cards?

Sub forum for Parallella daughter cards and accessories

Moderator: Folknology

Re: Ideas for daughter cards?

Postby 9600 » Thu Mar 20, 2014 4:43 pm

shodruk wrote:By the way, Is this feasible?
How is the inter-chip routing configured?


I think so, broadly speaking. Although I'm not sure how big an eLink network, in terms of chips, has been tested, and I'm guessing you'd need a lot of PCB layers to route all those signals! Getting rid of 8 watts per board could be a challenge also and particularly given there wouldn't be much space in between boards. So, if it would work electrically and as a system, you'd likely need a very powerful fan or two, else some other method of getting rid of the heat in a compact space.

In terms of routing, I think you simply connector the eLink from one chip to another, and configure the offset within the address space. With the Parallella board this is done thus (from the Reference Manual, pp. 23):

"The ROWID and COLID can be individually set on boards through the PEC_POWER connector enabling direct board to board connection through the PEC_NORTH and PEC_SOUTH connectors."

Any particular topology in mind? :)

Cheers,

Andrew
Andrew Back (a.k.a. 9600 / carrierdetect)
User avatar
9600
 
Posts: 997
Joined: Mon Dec 17, 2012 3:25 am

Re: Ideas for daughter cards?

Postby timpart » Thu Mar 20, 2014 9:46 pm

I think the E16 only supports grid style topologies as it doesn't have the *MESHROUTE registers feature.

The Parallella only provides north and south eLink extension and there are only 16 possible row addresses for the E16 version. So you can't put more than 16 boards in a line, and there is no way to go sideways. With a daughter board like the one shown above, if the boards arrange the chips in a column you could have three of them plus the Parallella.

Alternatively for a bit more cost you could arrange the four epiphanies in a row rather than a column. Then you would have to provide north south connectors for each of them if you want free reigning communications. Plus some way of attaching the ordinary Parallella to one of the connections. You could get 15 * 4 + 1 chips in such an arrangement. A big power supply and board feeds needed too, doubt you could feed that much current through the Parallella board! The Parallella would only be able to directly talk to one column of chips

Tim
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Re: Ideas for daughter cards?

Postby 9600 » Thu Mar 20, 2014 10:46 pm

timpart wrote:A big power supply and board feeds needed too, doubt you could feed that much current through the Parallella board!


J15 soldered on the Parallella, and Epiphany expansion cards powered via their mounting hole pads and metal hex spacers used as busbars. Obviously you'd apply power to the latter and not the barrel jack :)

Cheers,

Andrew
Andrew Back (a.k.a. 9600 / carrierdetect)
User avatar
9600
 
Posts: 997
Joined: Mon Dec 17, 2012 3:25 am

Re: Ideas for daughter cards?

Postby shodruk » Sat Mar 22, 2014 4:57 pm

Thanks Tim, Andrew,
I understand that my design has many problems. :mrgreen:
I'll reconsider it.
Shodruky
shodruk
 
Posts: 464
Joined: Mon Apr 08, 2013 7:03 pm

Re: Ideas for daughter cards?

Postby mhonman » Sat Mar 29, 2014 10:41 pm

Catching up a bit here... but obviously interested in hooking up more Epiphanies as it means More RAM... I came across an earlier post by Tim that sums up the transaction routing challenge.

So... from a parallel processing expansion point of view a board with 3( or 4?) Epiphanies arranged in a North-South column might be a good idea. The only connectors needed would be North and South - to mate with two different Parallellas - and a power connector. Then one could configure chains of processors along the following lines:

Code: Select all
E      \
E       > 3 Epiphanies on expansion board
E      /
E - Z  Parallella Epiphany and Zynq
E      \
E       > 3 Epiphanies on expansion board
E      /
E - Z  Parallella Epiphany and Zynq
E      \
E       > 3 Epiphanies on expansion board
E      /


The reasoning behind this is perhaps unnecessary, but here goes:

With transactions being routed first in the east-west direction, and e-link East connected to the Zynq, the Parallella arrangement means that the Epiphany will route transactions to addresses associated with any core East of itself to the Zynq.

When multiple Parallellas are hooked up via the North and South links, an Epiphany cannot normally access the external RAM of another Parallella. Not necessarily a bad thing, as the totality of addressable external RAM increases with the number of Parallellas - it could be more than the 4GB architectural limit, though keeping track of what is where would make for an interesting programming problem!

The "cannot normally" condition comes about because a DMA engine on a Parallella's on-board Epiphany could be used by another Epiphany to the North or South - then the transactions to the Parallella's external RAM originate from the on-board Epiphany. The mesh read penalty would come into play because however you slice it the proxy DMA engine is going to have to reading either from the Zynq or the remote Epiphany - which means that the number of hops to the remote Epiphany becomes important.

It seems to me that there is another tricky aspect to this configuration, in that while any host ARM processor can write to any core in the North-South array, without the suborned-DMA-engine trick it cannot read an off-board Epiphany (well the read requests get there, but the replies go to that Epiphany's directly attached Zynq).

The Epiphany-strips serve a dual function, because in a RAM-intensive application one might want to just use the Parallella boards' on-board Epiphanies for the application code, with the strips merely providing a cluster backplane that has the added bonus of 1.5MB of mesh-addressable fast RAM. On the other other hand the application could run on the strip-Epiphanies, and the Parallella Epiphanies could serve their I/O needs.
mhonman
 
Posts: 112
Joined: Thu Apr 25, 2013 2:22 pm

Re: Ideas for daughter cards?

Postby shodruk » Sun Mar 30, 2014 10:34 am

Thanks mhonman,
Yes, this is like a puzzle game. :)
I don't know whether it is possible, I thought of one more eLink topology.

elink01.png
elink01.png (33.34 KiB) Viewed 22274 times


The "Glue board" is for figuring out the physical layout problem.
I think the Epiphanies on the add-on boards and the Zynq can communicate with each other,
but the Epiphany on the Parallella board cannot communicate with add-on boards directly.

Is it possible to implement a eLink using GPIO port?
What problems are there with this idea?
Last edited by shodruk on Sun Mar 30, 2014 11:50 am, edited 1 time in total.
Shodruky
shodruk
 
Posts: 464
Joined: Mon Apr 08, 2013 7:03 pm

Re: Ideas for daughter cards?

Postby mhonman » Sun Mar 30, 2014 7:49 pm

Not being a hardware guy, I don't know whether the GPIO could be used as another eLink, but it's a very good idea! If so, the "new" link could also be an East or West link.

The advantage of the Zynq being on an east-west e-Link is that its RAM can be exposed as a contiguous address range in Epiphany address space (that was the "lightbulb moment" that struck me in reading Tim's post).

The other really useful bit of info is the relative cost of Epiphany chips and the connectors needed for the high-speed eLink signals - IIRC 3 or 4 of those connectors cost the same as a 16-core Epiphany.

So I think the goal for Parallella expansion should be very limited and always with "hobbyist" pricing in mind.

For parallel-processing expansion it would have been nice for the Parallella's two exposed eLinks to be N and W - then the on-board Epiphany could occupy the SE corner of an arbitrarily-sized matrix of Epiphanies. Such a system couldn't have multiple Parallellas in it, to do that one would need connectors carrying the N, S, and W links.

A recent attack of megalomania prompted the idea of "Parallella Plus", being a bigger Zynq (484 or 485 pin package) with 3 links into the East edge of a 3x3 matrix of Epiphanies - all on one board. But I guess that would be too expensive for anyone to buy just to have a play.

Mark
mhonman
 
Posts: 112
Joined: Thu Apr 25, 2013 2:22 pm

Re: Ideas for daughter cards?

Postby 9600 » Mon Mar 31, 2014 8:49 am

shodruk wrote:Is it possible to implement a eLink using GPIO port?


I'm pretty sure this is possible, and perhaps even quickly tested by building a new bitstream where the I/O pin assignments for the default eLink are changed so that this is routed to PEC_FPGA instead of the on-board Epiphany.

Cheers,

Andrew
Andrew Back (a.k.a. 9600 / carrierdetect)
User avatar
9600
 
Posts: 997
Joined: Mon Dec 17, 2012 3:25 am

Re: Ideas for daughter cards?

Postby timpart » Mon Mar 31, 2014 12:18 pm

shodruk wrote:The "Glue board" is for figuring out the physical layout problem.
I think the Epiphanies on the add-on boards and the Zynq can communicate with each other,
but the Epiphany on the Parallella board cannot communicate with add-on boards directly.

I don't see any problem with directly connecting the North of the Parallella with the South of the Western column, providing the column address is set to the same value as that column and the row address is lower. The Parallella would be able to send data to any chip in that column. Any chip in the grid would be able to send to the Parallella as the data would move horizontally before going down the column. However the Parallella wouldn't be able to talk to chips that aren't in the same column without extra help as the data will go East first into the FPGA.

shodruk wrote:Is it possible to implement a eLink using GPIO port?
What problems are there with this idea?


I don't see why not. Other GPIO pins are used to form the eLink to the East of the Parallella's Epiphany. The difficulty might be in creating two eLinks in the FPGA at once. I've no idea how modular the design is. There might be variable name clashes.

If the Parallella Epiphany were connected to the westmost column, the GPIO could be used to connect to the next column to the East and route in any data being sent eastward (that wasn't destined for the DRAM), sending it northward into the array. There would only need to be a link one way to the array as returning data would be routed directly to the Parallella Epiphany as I mentioned above.

Tim
timpart
 
Posts: 302
Joined: Mon Dec 17, 2012 3:25 am
Location: UK

Re: Ideas for daughter cards?

Postby aolofsson » Mon Mar 31, 2014 1:26 pm

There is no problem in creating a second eLink in the FPGA logic and connecting it to the North/South links (or directly to other FPGAs!). In fact this was always part of our "master plan" :D
More on this soon..
User avatar
aolofsson
 
Posts: 1005
Joined: Tue Dec 11, 2012 6:59 pm
Location: Lexington, Massachusetts,USA

PreviousNext

Return to Daughter Cards & Accessories

Who is online

Users browsing this forum: No registered users and 3 guests

cron