Synchronization between the ARM and the Epiphany cores

[sakalisc]

Hello,

I have a question regarding some synchronization issues between the ARM cores and the eCores. I know from that the recommended method of synchronization for now is to constantly poll a flag variable. However, given the weak memory model, how do we know if any of the other data we need from the eCore(s) have been already written to the memory when the write to the flag variable becomes globally visible? Is there some kind of hardware memory barrier that can guaranty that?

At the same time, is it possible to use this method for the eCore(s) to wait for the ARM processor? An example on why we might need this is if the ARM core handles network connections and we need the eCores to wait for some data.

Thank you.

Best regards,
Chris

[timpart]

You raise a very valid point.

We know from the behaviour of the eMesh that writes from an individual Epiphany core to the external shared DRAM will update the DRAM in the same order as they were sent. However if multiple cores are writing to external DRAM we don't know in which sequence the DRAM will be changed.

Three relatively easy solutions come to mind:

1) Give each core a separate flag in the DRAM, and make the ARM wait until they have all been set.

2) Use the software barrier function e_barrier() in the SDK to make sure all the cores have finished then write to a single flag. This might be time consuming though.

3) Use the to make sure all the cores have finished then write to a single flag. This only works if every core is involved. Need to ensure interrupts are enabled, then align to 8 byte boundary then execute WAND then IDLE then write to flag. Edit: Have just remembered need an interrupt handler for the WAND interrupt and to reset the WAND bit in the status register as well. Perhaps this option is heading away from "easy" to do from scratch.

Tim

[notzed]

I've been using one comms channel per core because anything else requires "costly" and complex synchronisation. I use a system whereby the writer or reader can poll by only reading a local value, so any forced busy-waiting doesn't flood the mesh.

Actually barriers aren't too bad if used sparingly - probably a few hundred cycles.

For external I/O i've used a trap instruction. It's got a lot of overhead but what are you gonna do? At least it simplifies the synchronisation and moves almost all of the code to the ARM which saves previous on-core bytes (or can, just not the way the sdk does things). I have a thread running on the arm which polls the status register of each core in a timed loop and if they're stopped it then reads the PC and recovers the trap code and register/stack values and so on. After running the remote rpc it then restarts the cpu.

There really needs to be some "hardware" mailboxes or other synchronisation primitives in the FPGA - with interrupt capability (in both directions); hopefully that will be added at some point.

Not sure what the apparently-imminent "sdk update" will include or add. If anything.