Connecting Multiple Parallella Boards

[aviv_ohad]

Hi All!

We have spent the last few months playing with the Parallella and trying to make several boards work together. We came across several problems that we could use some help solving.

At first we followed this post: https://parallella.org/forums/viewtopic ... r&start=10
with the old HDMI Build, and connected our boards this way: (link attached in case you cannot view the image)


https://drive.google.com/file/d/0B5uBXg ... sp=sharing

Then we made our own new PCB connector:


https://drive.google.com/file/d/0B5uBXg ... sp=sharing

The problems we came across are the following:

1. Hardwiring the core-id using a jumper does not seem to work for the headless builds it seems. When ever we try to run the Hello_World example after changing the hdf and xml file the board crashes and has to be rebooted.
Any clues as to why this happens or how to fix it?

2. We have been trying to use an interrupts based mailbox program to have communication between the boards up and running, and it only works with esdk.2015.1. When trying to use the new library with the HDMI build nothing seems to work at all. We read here : viewtopic.php?f=13&t=3073 that there are some missing features on the old FPGA build, but we would like to hear some more details about it.

3. changing core ID and Speed: As we plan to use our connector, we need to change these features without the comfortable pins that come with the Porcupine. We are able to write to the FPGA system Registers, but they don't seem to affect these two issues at the moment on both the headless and HDMI versions. Any ideas on how to tackle the problem,
or details on progress with the FPGA registers?

Thanks a lot!

[aolofsson]

[aviv_ohad]

[peteasa]

When I tried communicating between boards I had some strange effects but I never had to re-write the sd card! That seems unrelated to the epiphany connection.

I found that the highest clock speed for the 2015.1 fpga worked. If you want to try to change the speed look at https://github.com/adapteva/epiphany-li ... al.c#L1115 and https://github.com/adapteva/epiphany-li ... al.c#L1133 and https://github.com/adapteva/epiphany-li ... al.c#L1139. These are the places where the txcfg and rxcfg registers are setup for the full speed elink.

My advice would be to first get this working with the 2015.1 fpga and matching epiphany sdk before attempting with the latest fpga and software. In my testing viewtopic.php?f=23&t=3207&hilit=jumper#p15984 the North Parallella could load each of the 32 cores in the large workgroup. So I used this to prove that the South Parallella could write to the North Parallella. The epiphany code makes each of the 32 cores write its Id to all the other cores in the group. The North Parallella could then read the North cores and see Id's written from the South Parallella and the South Parallella could read the South cores and see the Id's written from the North Parallella.

The next bit of advice is to bring the system up gradually. For example first configure the hdf file for only one chip and run the hello world program for the 908 core (single chip with the appropriate hardware link) and then with the second board configure the hdf file and run the hello world program for the A08 core (single chip with different link). Check that this works as expected before trying to extend to multiple chips. Once working patch the epiphany-libs code to enable the North / South links and try again with single chip to confirm that your version of the epiphany-libs code still works. Then try enable the North / South links and see if you can create a group with 32 cores.

Notice that with the very latest fpga and matching kernel driver the disable of the elinks and the setting of the elink speed is all done in the kernel driver see https://github.com/parallella/parallell ... any.c#L482 so to replicate the 2015.1 fpga tests with the 2016.3 fpga https://www.parallella.org/2016/04/11/m ... se-2016-3/ will likely require a patch to the kernel driver as well as a patch to the epiphany-libs. Also because the 2016.3 fpga has higher elink speeds then it is likely that you will have to drop the speed of the elink to get it to work between boards. Thus it will not be easy to replicate the old inter board communications with the new fpga.

Peter.

[aviv_ohad]

Hello everybody,
We are glad to tell you that we succeeded! Thanks to a lot of help from Peter and others, and a lot of time and effort,
We managed to build our very own Parallella Epiphany level cluster!
We wrote a user manual, some example code, and a technical report- all available in the link:
We also added the design files to the same folder.
Hope this helps some of you! please let us know if you use it or have any issues!

CAM00434.jpg (717.43 KiB) Viewed 21800 times

Cheers