eLink Cluster porting from 2015 to 2016.11 eSDK

Any technical questions about the Epiphany chip and Parallella HW Platform.

Moderator: aolofsson

eLink Cluster porting from 2015 to 2016.11 eSDK

Postby claudio4parallella » Wed Nov 22, 2017 9:54 pm

Hi all,
I am going forward with my eLink Cluster project for implemnting a multicore Neutral Network, and some other scientific applications: Orbital Mechanics and Lattice....

Please, after many attempts I'm still using the old version of "PUBUNTU 14.04-esdk2015.1" as suggested by "Parallella Epiphany Cluster Technical Report", written at Parallel Systems lab.

Any attemp to port the https://github.com/parallella/pubuntu/releases/tag/pubuntu-%2014.04-esdk.2015.1-20150130 to the last eSDK 2016.11 Ubuntu image was a failure.

Question and request for help:
I've noticed that one of the important upgrades from old images to last ones was the settings of FULL SPEED to the eLink instead of the initial partial timing.
I'm suspecting that this is the reason why the Cluster described is not well working with last image.

Could anybody support me to change time clock, eLink TIME SPEED for eLink in order to have a slower clock and try the eLink Epiphany Cluster using a short flat cable between Porcupines with the last Ubuntu and eSDK available with HDMI ?
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby jar » Mon Nov 27, 2017 7:25 pm

Your assumptions could be right.

If you make modifications, you can submit a pull request to the original repository. I doubt this is a high priority item. You should probably coordinate with Ola directly if you have particular questions.
User avatar
jar
 
Posts: 293
Joined: Mon Dec 17, 2012 3:27 am

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby claudio4parallella » Mon Nov 27, 2017 8:19 pm

Thank you jar for your reply.
I'm still investigating studying the documentation and the sdk code.
At present I'm developing applications over 64 cores (4 parallellas) and the environment works like a juice!
I'm still using pubuntu-14.04-esdk2015.1-20150130 without HDMI , headless.
Scientific calculation is welcome, graphics is possible only using libpng.
At least the Master Parallella should have lxsession to watch results ....
Each core could individually move one of this http://cs.stanford.edu/people/karpathy/convnetjs/demo/rldemo.html? ( I'm working on porting this from javascript to C).
I'll notify the request, thanks
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby peteasa » Tue Nov 28, 2017 8:32 pm

Hi,

claudio4parallella wrote:Hi all,
[/b]I've noticed that one of the important upgrades from old images to last ones was the settings of FULL SPEED to the eLink instead of the initial partial timing.
I'm suspecting that this is the reason why the Cluster described is not well working with last image.

Could anybody support me to change time clock, eLink TIME SPEED for eLink in order to have a slower clock and try the eLink Epiphany Cluster using a short flat cable between Porcupines with the last Ubuntu and eSDK available with HDMI ?

You might find this is of some help.. https://github.com/parallella/oh/blob/m ... c/e-main.c

The basic idea with this software is to trial a number of different idelay values. Once a good idelay is found then this can be used in https://github.com/parallella/parallell ... any.c#L699. You will see that the e-main.c code sets E_REG_LINKCFG = 0xf0300 (https://github.com/adapteva/epiphany-li ... ata.h#L190) with a divide cclk value 0 == divide by 2 (see e16g301_datasheet_14.03.11.pdf Table 2). Then loops over a selection of possible values for the idelay. If the link connection is good then the e-main.c code responds, if not then corruption on the link results in a failure. It seems from the elink_reset() code that a value of 0xaaaaaaaa has been selected for the cclk divide by 2. The design of the chip is such that this type of trial and error test is the best way to characterise the actual timing delays in the real silicon. You could of course just plug in the old values from the old code into the new code but I would expect running e-main idelay test would give you more confidence in the values chosen.

So your first attempt at getting this to work would be to modify the E_REG_LINKCFG setting in e-main.c to a clock of your choice, perhaps one that matches the equivalent setting in the code that works, then compile / run e-main.c and see if you can get a good value for idelay... then update the kernel driver to use this slower speed.

Good luck!

Peter.
User avatar
peteasa
 
Posts: 117
Joined: Fri Nov 21, 2014 7:04 pm

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby claudio4parallella » Wed Nov 29, 2017 11:21 am

Thanks a lot "peteasa" for you suggestion.

I red you were in trouble https://parallella.org/forums/viewtopic.php?f=23&t=3659 with clock and other issues during your first attempts.

So I'll go deeple into tests with eLink speed.

What do you think if I'll test this eLink-Cluster system:

MASTER: I'll use last Ubuntu image 2016.11 that should have eLink at FULL SPEED
SlAVE: I'll use the Ubuntu old 2015 image that should have eLink at partial SPEED.

I'll adjust/devide eLink clock on MASTER so, this system I think will point out how to go back to previously working eLink feature via cable.

Thanks a lot, I'll let you know.
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby claudio4parallella » Wed Nov 29, 2017 7:44 pm

Bad news till now, all my attempts.
With MASTER 2016.11 esdk parallella, and a modified ".hdf" for 2 chips 32 and 36, every time I run an e-write to a mem address 908xxx on the SLAVE parallella it's raised a Segmentation failure.

It does work when used MASTER and SLAVE 2015 esdk.
So by this way I cannot test the oh/src/elink/sw/idelay.
May be I've to recompile some more than only the "idelay".
uhm!!
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby claudio4parallella » Wed Nov 29, 2017 8:13 pm

Please,
has this code section from /epiphany-examples/apps/e-init
Code: Select all
 if(stage>3){
    //Configuring clock divider(EAST==2,7)
    row=2;
    if ((dev.type == E_E64G401)){
      col=7;
    }
    else{
      col=3;
    }

    txcfg.fields.ctrlmode = 0x5;
    ee_write_esys(E_SYS_CFGTX, txcfg.reg);

    //Change clock divider to solve FPGA receiver speed path
    data = 0x1;
    e_write(&dev, row, col, 0xf0300, &data, sizeof(int));
    //Up the current drive on the wait signal
    //data = 0x02000000;
    //e_write(&dev, 0, 0, 0xf0304, &data, sizeof(int));

    //Return to normal mode
    txcfg.fields.ctrlmode = 0;
    ee_write_esys(E_SYS_CFGTX, txcfg.reg);

  }
  if(stage>4){
    //Enable clock gating
    for (i=0; i<platform.rows; i++) {
      for (j=0; j<platform.cols; j++) {
        //eCore clock gating
        data=0x00400000;
        e_write(&dev, i, j, 0xf0400, &data, sizeof(data));
        //eMesh clock gating
        data=0x00000002;
        e_write(&dev, i, j, 0xf0700, &data, sizeof(data));
      }
    }
  }


Something to do with changing eLink Speed for flat cable interconnected parallellas?
thanks
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm

Re: eLink Cluster porting from 2015 to 2016.11 eSDK

Postby peteasa » Sun Dec 03, 2017 4:22 pm

Hi,

claudio4parallella wrote:May be I've to recompile some more than only the "idelay".
uhm!!


Not sure what the problem is and I have not tried this but my tips are not likely to be 100% because I have not touched code this recently. First place to look is are you having problems with a conflict with the epiphany.c driver that you are using... actually the first place might be to check that you can build the epiphany driver for your kernel. If changing the elink speed works then you will want to build the driver with your new chosen cclk divider values so getting familar with the epiphany kernel driver is not a bad idea.. Once you have got familiar with the epiphany kernel driver and the idelay code you might find that the idelay code needs to be modified to work with the epiphany driver you are using. I used idelay with a home grown epiphany kernel driver a couple of years ago now so things will have changed.

Oh yes and trying mixed speed links might not work by design.. I may be wrong but the off chip clock rate might be the same for all N,S,E,W links.. so if you connect two epiphany chips with different off chip clock speeds it might not work well. First job would be to prove the new elink speed with one epiphany then try joining two epiphany together.

Good luck,

Peter.
User avatar
peteasa
 
Posts: 117
Joined: Fri Nov 21, 2014 7:04 pm


Return to Epiphany and Parallella Q & A

Who is online

Users browsing this forum: No registered users and 7 guests