Why does Oh! Library AXI convert between eMesh and Packets

Using Zynq Programmable Logic and Xilinx tools to create custom board configurations

Why does Oh! Library AXI convert between eMesh and Packets

Postby ninlar » Wed Jun 21, 2017 11:57 pm

So first off. I am an FPGA noob. I have not touched HDL (and we used VHDL) since college. I wanted a project board and was having trouble deciding between an FPGA development board and something for embedded software development. But my other passion is massive parallelism. So when I saw the Parallella board on Amazon, I was so excited. 16-cores + 2 ARM for me to see how much extra throughput I can get from my algorithms using different synchronization constructs, or lock-free approaches, awesome! But at the same time I really regretted not learning more about hardware development, since I get very interested when I hear about stories like Microsoft using FPGAs to accelerate network cards on their hypervisors in the Azure cloud.

Ok enough with the digression. So I'm trying to understand the accelerator demo. I want to do something similar. See if I can implement an AXI slave that accelerates something like SHA256 hashing. What I cannot seem to understand is why the simple accelerator has instantiations (is that the correct term?) of modules that do conversion from eMesh to "packet" and vice versa? Once again, I've just been perusing the HDL and source, and I could be way off. Is it because the DMA controller is also implemented by Adapteva, and is it because the memory management module requires the the messages to be in "eMesh" format to determine if they get routed to DRAM or the Epiphany co-processor? If so then what exactly does AXI provide when you've layered your own protocol on top of it?
ninlar
 
Posts: 16
Joined: Wed Jun 21, 2017 11:40 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby ninlar » Thu Jun 22, 2017 2:48 am

I think I get why now. I noticed that the driver calls mmap on /dev/epiphany not /dev/mmap as mentioned in the comment which through me off. So it is using the eMesh? eLink? protocol via the driver rather than just some general memory mapped I/O that writes memory to some fixed address. I'm guessing the benefit of going this route is some that there is logic to make it transactional or add a small layer of abstraction that makes it easier or more developer friendly?

If so, what would be the correct approach in Verilog to do a SHA256 hash accelerator? Finite state machine?
ninlar
 
Posts: 16
Joined: Wed Jun 21, 2017 11:40 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby sebraa » Thu Jun 22, 2017 3:28 pm

Please differentiate between three things:
(a) Xilinx Zynq Processing System (PS) - this is a dual-core ARM processor
(b) Xilinx Zynq Programmable Logic (PL) - this is an FPGA fabric
(c) Adapteva Epiphany - this is a 16-core manycore chip.

The AXI interface is used to communicate between the PS and the PL, i.e. between the ARM cores and the FPGA logic. The eLink interface is the native hardware interface of the Epiphany chip, and is implemented on the Parallella within the PL (FPGA logic). Having a Linux device driver (like /dev/epiphany or /dev/uio) makes the system more robust, since giving an application full access to /dev/mem implies giving it access to the whole system, which is a huge security problem by itself.


You do not need to use any eLink stuff if you want to write your own hardware within the FPGA fabric and access it from within the Linux system. In that case, you would lose access to the Epiphany (and it's 16 cores), and basically have an FPGA development board. Otherwise, you can add your accelerator to the FPGA, and keep both. It depends on what you want to do.
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby ninlar » Thu Jun 22, 2017 6:25 pm

Thanks!

I definitely don't want to lose eLink and access to the Epiphany co-processor. In addition, I value using least privilege access, so that is a good point right there between /dev/epiphany and /dev/mem. Does any of the literature or documentation dive into eLink.
ninlar
 
Posts: 16
Joined: Wed Jun 21, 2017 11:40 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby sebraa » Fri Jun 23, 2017 10:18 am

Hi,

I don't know what you are trying to do. You need the eLink interface if and only if you want to talk to the Epiphany chip, and you do not need to touch any HDL or FPGA logic to do so. (If you want to create your own FPGA bitstream, you need to keep the eLink, though). If you want to start doing hardware design, you don't necessarily need the Epiphany chip; I am not sure if the current eLink design could allow any direct communication between the Epiphany chip and an FPGA accelerator. It wouldn't be much faster than the memory access to the host anyway.

I have used both, but not at the same time - I have a bitstream with my FPGA logic (which uses the Linux UIO framework to talk to my design), and the original bitstream with the eLink interface for Epiphany work.

Since eLink is an Adapteva design, you won't find much documentation of it outside of Adapteva and academic work. However, you have access to the code.

Best of luck,
sebraa
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby ninlar » Fri Jun 23, 2017 10:47 pm

sebraa wrote:Hi,

I don't know what you are trying to do. You need the eLink interface if and only if you want to talk to the Epiphany chip, and you do not need to touch any HDL or FPGA logic to do so. (If you want to create your own FPGA bitstream, you need to keep the eLink, though). If you want to start doing hardware design, you don't necessarily need the Epiphany chip; I am not sure if the current eLink design could allow any direct communication between the Epiphany chip and an FPGA accelerator. It wouldn't be much faster than the memory access to the host anyway.

I have used both, but not at the same time - I have a bitstream with my FPGA logic (which uses the Linux UIO framework to talk to my design), and the original bitstream with the eLink interface for Epiphany work.

Since eLink is an Adapteva design, you won't find much documentation of it outside of Adapteva and academic work. However, you have access to the code.

Best of luck,
sebraa



Thanks for the insight. The reason I like this board so much is because I was torn between an FPGA board like the Digilent ones or the Numata one that is an FPGA on a PCIe card, or between a GPU development board like the Nvidia Jetson for massive parallel computing on the CUDA cores. But when I saw the Parallela board, I knew that was the board I wanted. Even though the PL on the FPGA is a lot smaller than the spartans or whatever, it is enough for me to get back into VHDL or learn Verilog. At the same time, I love trying to maximize throughput using new parallel patterns or lock-free algorithms.

Since the Parallella board allows me to do both, I want to do both. So my first hobby project would be to port something like CPUMiner over to the Parallela board and get all 16 cores doing hashes for mining. But then at the same time, I'd also like to create an accelerator in the FPGA that can do a SHA256. So my port of CPUMiner would have 16 epiphany cores doing hashes and then have the host spawn two threads since the ARM portion is dual-core and have one thread use the SHA256 acelerator and have the other thread manage communication with a mining pool and consolidating the work from the eipiphany cores. Before you say it, I know I won't make money off it. I know there are ASICs that can kick its ass. I just want to do it for fun to see how many hashes per second I can get the board to pull off. Similar to the Jack the Ripper example which has an FPGA portion but it looks like using the FPGA portion removes the eLink stuff. I want epiphany to remain and eLink too, since it rocks and is the main feature of the board, but at the same time, if there is some space in the PL for some SHA256 accelerators, then I want to add them too :)
ninlar
 
Posts: 16
Joined: Wed Jun 21, 2017 11:40 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby sebraa » Sat Jun 24, 2017 9:12 am

That sounds like a fun project. I only see two important things to keep in mind: The Epiphany will not be able to communicate to your SHA256 accelerator directly (unless you pull some tricks), and your host code needs to poll the Epiphany for its state (no interrupt / blocking), which reduces the available CPU time on the host.

I have never integrated a fully working eLink with custom FPGA logic, but I think others have. Good luck.
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby ninlar » Sat Jun 24, 2017 10:46 am

So I plan to have the eCores do SHA256 hashes. The accelerator will be used by the host program. So I hope that goes smooth. Why would the eCores not be able to communicate with the accelerator? The accelerator example in Oh! seems to use eLink since there is an instantiation of e2p for elink to packets and p2e for packets to elink. The accelerator example uses some mapped memory, so I don't see why the eCores could not use e_write or e_read to manipulate that memory or does e-lib's API restrict the calls to cores on mesh.
ninlar
 
Posts: 16
Joined: Wed Jun 21, 2017 11:40 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby sebraa » Sat Jun 24, 2017 12:52 pm

It depends on whether the eLink/AXI interface can communicate to the AXI master (Zynq PS) only, or if it can talk to an AXI slave (your accelerator). Just having an address doesn't mean that you can communicate. Just keep in mind that it might not work, or might require additional tricks.

Also, you need to get the addressing right, otherwise the Epiphany chip will route requests to the wrong port.
sebraa
 
Posts: 495
Joined: Mon Jul 21, 2014 7:54 pm

Re: Why does Oh! Library AXI convert between eMesh and Packe

Postby olajep » Fri Jun 30, 2017 2:05 pm

ninlar wrote:So I plan to have the eCores do SHA256 hashes. The accelerator will be used by the host program. So I hope that goes smooth. Why would the eCores not be able to communicate with the accelerator? The accelerator example in Oh! seems to use eLink since there is an instantiation of e2p for elink to packets and p2e for packets to elink. The accelerator example uses some mapped memory, so I don't see why the eCores could not use e_write or e_read to manipulate that memory or does e-lib's API restrict the calls to cores on mesh.

You need to add a crossbar if more than one AXI master should have access to the same AXI slave.

Template block design that could work:
vivado-xbar.png
vivado-xbar.png (106.83 KiB) Viewed 1182 times


Address mappings:
vivado-xbar2.png
vivado-xbar2.png (31.57 KiB) Viewed 1182 times


From the Epiphany side the accelerator can be accessed at base address 0x8C000000
For this to work the remapping in the FPGA elink must also needs to be changed
These two lines in the epiphany kernel driver:
https://github.com/parallella/parallell ... #L742-L743
should be changed to:
Code: Select all
   rxcfg.remap_sel = 0xfc0;
   rxcfg.remap_pattern = 0x3c0;


NOT TESTED!

// Ola
_start = 266470723;
olajep
 
Posts: 121
Joined: Mon Dec 17, 2012 3:24 am
Location: Sweden


Return to FPGA Design

Who is online

Users browsing this forum: No registered users and 2 guests

cron