PEC_NORTH/SOUTH Clustering Guidance Needed

PEC_NORTH/SOUTH Clustering Guidance Needed

Postby eric » Thu Mar 23, 2017 8:32 pm

I have been trying to get two (then more) Parallellas communicating via PEC_NORTH/PEC_SOUTH for some time, but keep hitting roadblocks. I am using the Porcupine boards with ribbon cables. The first hurdle is changing one of the Epiphany ROW/COL to (36,8).

I have edited the platform.hdf and changed CHIP_ROW to 36.
I have a jumper on the Porcupine, pulling YID0 up to 1.8V.

After reboot, running any software that accesses the Epiphany gives me:
e_open(): ECORE[0,0] MEM mmap failure.
And then a segmentation fault.

According to this thread (https://parallella.org/forums/viewtopic.php?f=23&t=3659), this combo has worked in the past. However their code does not run on the latest SDK. Any ideas out there?
eric
 
Posts: 2
Joined: Thu Mar 23, 2017 8:16 pm

Re: PEC_NORTH/SOUTH Clustering Guidance Needed

Postby olajep » Sat Mar 25, 2017 3:02 pm

I don't think we currently support directly accessing another board's Epiphany chip from the ARM.
But communicating directly between Epiphany chips should work.

This should work (not tested):
1. Make sure the epiphany module is loaded with the 'nopm' (no power management) option.
Either manually via command line:
Code: Select all
sudo rmmod epiphany
sudo modprobe epiphany nopm

or modprobe.d
https://wiki.archlinux.org/index.php/ke ... probe.d.2F


2. Don't change platform.hdf. Pretend CHIP_ROW is still 32.
3.
Create a epiphany program that writes to 0x90804000 (row=36 col=8 offset=0x4000)
Run it on the north board (unmodified CHIP_ROW).
4. On the south board you should be able to read out the value you wrote with the e-read command.
e-read 0 0 0x4000

// Ola
_start = 266470723;
olajep
 
Posts: 92
Joined: Mon Dec 17, 2012 3:24 am
Location: Sweden

Re: PEC_NORTH/SOUTH Clustering Guidance Needed

Postby eric » Mon Mar 27, 2017 2:13 pm

Thanks for the response Ola, I will follow your advice and post a follow-up.
-Eric
eric
 
Posts: 2
Joined: Thu Mar 23, 2017 8:16 pm

Re: PEC_NORTH/SOUTH Clustering Guidance Needed

Postby peteasa » Tue Mar 28, 2017 2:46 pm

Hi,

See viewtopic.php?f=23&t=3659. It has also been proven with ribbon cables. Mailbox should allow communication with local arm core on both epiphany chips I think. None of this is proven with the latest elink / sdk but the steps that were taken to get it to work previously should still work. Expect to have to modify kernel driver and sdk code.

Peter.
User avatar
peteasa
 
Posts: 114
Joined: Fri Nov 21, 2014 7:04 pm

Re: PEC_NORTH/SOUTH Clustering Guidance Needed

Postby olajep » Fri Mar 31, 2017 7:37 pm

eric wrote:Thanks for the response Ola, I will follow your advice and post a follow-up.
-Eric

Eric,

Had another look at the driver code.
The 'nopm' parameter wasn't respected everywhere (or rather the behavior wasn't 100% consistent).

Fixed in this commit
https://github.com/parallella/parallell ... 1a16a4965d
Code: Select all
epiphany: Don't disable disconnected chip elinks in nopm mode

In epiphany_reset():
Don't enable clock gating and don't try to disable disconnected elinks
when power management is disabled.


Attached new kernel module w/ fix
epiphany.ko.zip
(98.52 KiB) Downloaded 14 times

Code: Select all
sudo mv /lib/modules/4.6.0+/kernel/drivers/misc/epiphany.ko  /lib/modules/4.6.0+/kernel/drivers/misc/epiphany.ko.bak
unzip epiphay.ko.zip
sudo mv epiphany.ko /lib/modules/4.6.0+/kernel/drivers/misc/epiphany.ko
sudo rmmod epiphany
sudo modprobe epiphany nopm
_start = 266470723;
olajep
 
Posts: 92
Joined: Mon Dec 17, 2012 3:24 am
Location: Sweden


Return to Clustering

Who is online

Users browsing this forum: No registered users and 1 guest