Two Parallellas eLink cluster (interconnected)

Any technical questions about the Epiphany chip and Parallella HW Platform.

Moderator: aolofsson

Two Parallellas eLink cluster (interconnected)

Postby claudio4parallella » Wed Sep 13, 2017 9:18 am

Hi!
I'm trying to change on my SLAVE Parallella in the

/opt/adapteva/esdk/bsps/parallella_E16G3_1GB/parallella_E16G3_1GB.hdf

CHIP_ROW 36 (previous 32)

in order to have at the end MASTER-SLAVE connected with 32 cores and I'v noticed as follow:

on Ubuntu ubuntu-14.04-hdmi-z7010-20140611.img with the following set-up the Epiphany DOES WORK with CHIP_ROW 36 and hard wired pinout (Porcupine)

Code: Select all
// Platform description for the
// Parallella/1GB/E16G3
PLATFORM_VERSION   PARALLELLA1601
ESYS_REGS_BASE         0x808f0f00

NUM_CHIPS                       1
CHIP                      E16G301
CHIP_ROW                       36
CHIP_COL                        8

NUM_EXT_MEMS                    1
EMEM                     ext-DRAM
EMEM_BASE_ADDRESS      0x3e000000
EMEM_EPI_BASE          0x8e000000
EMEM_SIZE              0x02000000
EMEM_TYPE                    RDWR


on Ubuntu parabuntu-2016.11.1-hdmi-z7010.img that is the last realease with the following set-up the Epiphany DOES NOT WORK with CHIP_ROW 36 and hard wired pinout (Porcupine)

Code: Select all
// Platform description for the
// Parallella/1GB/E16G3
PLATFORM_VERSION   PARALLELLA1601

NUM_CHIPS                       1
CHIP                      E16G301
CHIP_ROW                       36
CHIP_COL                        8

NUM_EXT_MEMS                    1
EMEM                     ext-DRAM
EMEM_BASE_ADDRESS      0x3e000000
EMEM_EPI_BASE          0x8e000000
EMEM_SIZE              0x02000000
EMEM_TYPE                    RDWR


and I've noticed that in the last realese parallella_E16G3_1GB.hdf DOES NOT HAVE

Code: Select all
ESYS_REGS_BASE         0x808f0f00


Could anybody help me in understanding the following questions:

1. Why last realease does not have ESYS_REGS_BASE 0x808f0f00

2. Why in the the last release the Epiphany does not work with the change I made


Thanks a lot in advance
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby jar » Wed Sep 13, 2017 1:37 pm

These are probably questions for Ola, directly.

Regarding #1, I can only suggest that it looks like the functionality was not needed in newer versions. Ola left comments to that effect:
https://github.com/adapteva/epiphany-li ... geLog#L561
User avatar
jar
 
Posts: 293
Joined: Mon Dec 17, 2012 3:27 am

Re: Two Parallellas eLink cluster (interconnected)

Postby claudio4parallella » Wed Sep 13, 2017 8:11 pm

Thank you very much jar for your reply.
I'm realizing that my questions were badly asked.

1. it seems that the nnew release doesn't need any more that address initialization.
Could anybody confirm that this value has nothing to do with my attempt?

2. any suggestion about how to change 32 to 36 as needed for the two elink interconnected parallellas ?
peteasa and others have written many things here in the far past but I have not been so smart to understing every step or repeat these steps with the last image and sdk relase

thank for any help, my best
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby olajep » Sat Sep 16, 2017 1:42 pm

Recent Parabuntu images doesn't support clustering 'out-of-the-box'
Having said that it shouldn't be that hard to get it to work again.

1. Recompile kernel module (or use attached one in thread below)
2. Disable power management in the epiphany driver
3. Fix devicetree allowed address range
4. Adjust HDF file?

Follow instructions in this thread:
viewtopic.php?f=32&t=4014&p=18831

You also need to change this line in the device tree
https://github.com/parallella/parallell ... .dtsi#L114
From:
Code: Select all
   reg = <0x81000000 0x100000>, <0x80000000 0x10000000>;

To:
Code: Select all
   reg = <0x81000000 0x100000>, <0x80000000 0x40000000>;


Recipe (not tested):
Code: Select all
git clone --depth 0 https://github.com/parallella/parallella-linux
cd parallella-linux
vim arch/arm/boot/dts/zynq-parallella-headless.dtb
[make changes per above]
make ARCH=arm LOADADDR=0x8000 parallella_defconfig
make ARCH=arm LOADADDR=0x8000 drivers/misc/epiphany.ko
make ARCH=arm zynq-parallella-headless.dtb
sudo mount /dev/mmcblk0p1 /mnt
sudo mv -i /mnt/devicetree.dtb /mnt/devicetree.dtb.old
sudo cp -i arch/arm/boot/dts/zynq-parallella-headless.dtb /mnt/devicetree.dtb
sync
sudo umount /mnt

reboot

Code: Select all
cd parallella-linux
sudo rmmod epiphany
sudo insmod drivers/misc/epiphany.ko nopm


you might also need to pass the unsafe_access parameter to the epiphany module if you need to access the FPGA elink registers
_start = 266470723;
olajep
 
Posts: 139
Joined: Mon Dec 17, 2012 3:24 am
Location: Sweden

Re: Two Parallellas eLink cluster (interconnected)

Postby claudio4parallella » Sun Sep 17, 2017 10:30 am

Thank you so much olajep, I'm testing your pathway and linked instructions you pointed out.

My observations after having downloaded parallella-linux:

Code: Select all
cd parallella-linux
vim arch/arm/boot/dts/zynq-parallella-headless.dtb


zynq-parallella-headless.dtb doesn't exist there: it has been created after

Code: Select all
sudo make ARCH=arm zynq-parallella-headless.dtb


I've only

zynq-parallella-headless-ad9361-fmcomms2-3.dts
zynq-parallella-headless-noelink.dts
zynq-parallella-headless.dts
zynq-parallella-microserver.dts
zynq-parallella.dts
zynq-parallella1.dtsi

I've modified << zynq-parallella1.dtsi >> with
Code: Select all
reg = <0x81000000 0x100000>, <0x80000000 0x40000000>;


I've executed the make and cp from your instructions.

When I run:

Code: Select all
sudo insmod drivers/misc/epiphany.ko nopm


The driver doesn't exist there. So I've executed the instructions from https://parallella.org/forums/viewtopic.php?f=32&t=4014&p=18831, downloading the epiphany.ko.zip

Modification of HDF and/or of xml and/or hard wire porcupine pinout <<CTRL: YIDO - V1.8 >> doesn't effect the change of address of the slave epiphany.

Now I'll try to use elink interconnecting two Parallella and then tryning to send commands to and from the twos.

I'm sure step by step to reach the final expected result of an elink cluster.

I've noticed into zynq-parallella1.dtsi

Code: Select all
        /* No connection between elink coreid pins and the Epiphany chip */
        adapteva,no-coreid-pinout;


Does it matter with the hard wire connection of porcupine <<CTRL: YIDO - V1.8 >> pinout?

Thanks
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby claudio4parallella » Sat Sep 23, 2017 6:56 am

It does work !
with esdk.2015.1
Now I'd like to port it to esdk.2016.1
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby dejan » Fri Nov 10, 2017 5:52 pm

Where did you buy eLink (cables)?? I was looking for it everywhere! I have my old parallella kit (4 boards) and I would like to connect them using eLink, but have no idea where to buy the cables!??
dejan
 
Posts: 3
Joined: Mon Jun 16, 2014 3:37 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby claudio4parallella » Sun Nov 12, 2017 8:30 pm

@dejan

Have a look at this post, I've explained my eLink cluster project starting from BillOfMaterial and links where to find all the items.
All according to an experience that was documented years ago using 2014 ParaUbuntu Image.
You could help me trying to port the cluster to last ParaUbuntu image 2017.11 .... will you?

http://parallella.org/forums/viewtopic.php?f=32&t=4222&sid=3586d092d4044efc7c74d40e1fa7b36f

Well, about cables you have two alternatives:
1 - purchase the Parallella eLink cables here already done (18 Pounds plus shipment)
https://groundelectronics.com/products/parallella-elink-cable-pair
2 - purchase >40 flat cable (of 1mm, not standard 1.27mm) and the 40 pin female plugs. It's easy to press the flat cable to grip the contacts

Let me know about your progress, keep us in touch!
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby dejan » Mon Nov 13, 2017 2:57 pm

I do not have the porcupine board... :(
dejan
 
Posts: 3
Joined: Mon Jun 16, 2014 3:37 pm

Re: Two Parallellas eLink cluster (interconnected)

Postby claudio4parallella » Mon Nov 13, 2017 5:55 pm

without a Porcupine (http://www.digikey.com @ 35$), you need this DIY...
Attachments
eLink.png
eLink.png (280.99 KiB) Viewed 9409 times
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm


Return to Epiphany and Parallella Q & A

Who is online

Users browsing this forum: No registered users and 7 guests