Following on from my previous post (), I decided to implement the algorithm using 2 workgroups that can communicate between each other with local memory writes, interrupts, and mutex calls. In order to do this, I had to write my own slightly modifed versions the eSDK functions for e_mutex_init(), e_mutex_lock(), e_mutex_unlock(), e_irq_set() and e_get_global_address() to allow them to access any core on the Epiphany, not just the cores in the caller core's workgroup.
I've deployed the ML algorithm on a 4 x 3 workgroup (which I'll call the network) with another 1 x 1 workgroup acting as an agent for the ARM. I have a dataset that needs to be processed by the network. The ARM reads in this dataset and writes it to shared memory. It then initialises and starts both workgroups.
After some basic initialisation, the cores in the network enter an IDLE state. The agent reads the first data sample from shared memory and copies it to the local memory of each core in the network. It then issues an E_SYNC interrupt call to each core to signal them to start processing the sample and then enters an IDLE state while waiting for the network to finish. During the process, the cores in the network sync using a number of e_barrier() calls. They also exchange information with each other using e_dma_copy(). I time how long the process takes on each core with the system timers (including the time spent in IDLE waiting to be started by the agent).
Once they have finished processing the sample, each core writes the timer value to the agent's local memory and increments a shared value located in the using a mutex (also located in the agent's local memory). The last core to increment this shared value (i.e. the last core to finish processing) issues an E_SYNC interrupt call to the agent to wake it up. The agent then collects and writes the totalled timer values of the network for each step to shared memory, and starts the whole process again for the next data sample.
The ARM polls the benchmark values for each sample in shared memory and then averages them and uses printf() to output them to Terminal.
My problem is this: the number of clock cycles that it takes for each core in the network to process each data sample is recorded as around ~100,000 cycles. However, the number of clock cycles recorded on the agent between the time it sends an E_SYNC interrupt call to each core to start processing and the time it is woken up again by an E_SYNC interrupt call from the last node to finish in the network is in the region of ~140,000,000 clock cycles. How can this possibly be? Even if each of the 12 cores in the network executed their processes one after each other (which is impossible, given that they sync with barriers), the total number of clock cycles for the agent to be woken up again should be nowhere near ~140,000,000. I've checked my code 100 times and I can't see any issues. The overall implementation works, just slowly. I'm using the internal.ldf linker script and all memory is allocated correctly as per e-objdump.
Am I going crazy or am I overlooking something glaringly obvious?
