Parallella Community

Well, if you do get that far and stop it certainly sounds you will have broken the back of the problem!

Am I right in thinking that a working solution would require some interface board other than the Porcupine, else a modfied one?

Cheers,

Andrew

Yeah, I hope this will get someone else interested and do the software part properly (i.e. a good V4L2 driver and not some hack), so the camera can be used out of the box in all the V4L applications out of the box.

I figured there might be more people with the required SW skills to do that than there is people with the FPGA design experience to do the interface and the equipment to debug it (everyone probably doesn't have a 500MHz scope handy and a couple of active probes ...)

For the HW, I'm using some hacked up interface (I don't have a porcupine) that has essentially the same wiring as the porcupine for the data lanes. However I have also some added circuit to detect 'LP mode', but this is only 4 resistors and could be modded onto a porcupine if needed.

I might be able to get rid of that requirement altogether since the early tests I did showed that doing a match requiring 0x00 0xb8 sequence on both lanes followed by a 4 bytes header with a valid ECC is discriminant enough. So I should be able to do a blind sync acquisition with that criteria and then just 'folow'the packet. At the end of it, I start another blind search, but the chance of a coincidental match during the inter-packet gap is really low.

So it might just work with just the porcupine ... (well, the v2 one that has the camera connector mounted of course ...)

Sylvain,

Amazing, I didn't realize what your tweeted picture was about until reading the forum post.

Thought I posted this already Saturday, but apparently it got lost! Upload whatever FPGA sources you have. As soon as I or someone else can check that we can reproduce it, you definitely have the $1000 bounty!! (if you wan it ). I am updating the blog post to provide another $1000 bounty for capturing frames in OpenCV.

Andreas

It's a bit too "raw" at the moment for anyone else to reproduce.

Before that happens I want to finish the steps I outlined above. I already wrote and tested the packet parsing module and connected it to DMA to validate the output, so now I receive only valid packets and not a raw data stream with garbage interleaved with valid data. I also made progress on a simple bilinear debayering core, I tested the math in python and validated it produces "good enough" images and I have its general architecture drawn on paper too.

I also want to test on the actual porcupine to validate it works there too. Not sure how long it will take to get here (you sent one right ?).

Andreas,

BTW, I would suggest making a V4L driver (and sub-device for sensor) a part of the bounty requirement. Just because getting data into a single application can be done my horrible ways (like the way I do now, just mapping /dev/mem and piloting DMA from userspace).

While a V4L driver will provide :
- Usage from other application
- Abstraction between the FPGA part and the sensor part so you could re-use with other sensor modules if needed. (assuming a CSI-2 2 lanes interface of course).

Sylvain,

Good point! Thanks for the feedback, agree with you. This is similar to what ADI did with the ad9361 if I understand you correctly. I will make a clarification.

Many times when governments do these kinds of bounties specs are written behind closed doors set up so that one team will win.

Transparancy ftw!

Andreas

A small update.

I coded what's necessary to convert the raw CSI packets into proper RGB video stream that I can feed to the AXI Video DMA core.
This works nicely and it streams 30 fps video right into the upper 32M of the RAM.

Unfortunately, having the ARM read it from there is a bit of an issue.
1) Using /dev/mem to map those 32M yields horrible performance because caching and things like that are all disabled for this zone.
2) Even copying from normal memory to normal memory yields pretty bad performance. It takes 32 ms to copy 8 Mbytes on my rev1. On another zynq board it takes 25 ms. So first something is wrong in the DDR config of the parallella, it should be faster. And second, even when that first issue is fixed, the copying is still pretty slow, 25 ms just to copy a full HD frame in RAW once mean you really can't do much at that rate.

All in all, I don't think it's possible at all to get faster frame rate from purely userspace ... will need a kernel driver that properly controls the DMA.

Here's a small video where it's captured at about 2 fps :
http://people.osmocom.org/~tnt/parallel ... 150621.mp4

The tearing effect is because I can't even re-read one frame buffer fast enough before it starts writing it again N frames later ...

I should get a porcupine next week, once I ported my stuff to it and tested it works, I'll publish what I have ATM.

Hi Sylvain,

Again, this is awesome!! I'll ask Ola to look into the DDR performance. Still, even with the 30% seems like we are missing 10x as you stated. Next step is definitely to a proper data mover driver for Parallella.
Look forward to trying it out next week.

Andreas

Hi there,

Andreas, what a great idea to include RPi Camera connector on Porcupine 2.0 and to start a bounty for its integration! Sylvain, what an impressive job so far! I'm keeping fingers crossed for you! For a moment I got an idea to try to help with achieving the goal, but I got a bit disappointed seeing unavailability of Porcupine board at Digikey... making one (or maybe a simple one - with just an RPi Camera connector) on my own would take me unnegligible amount of effort. How long would it take until it's available again? Or maybe you have some spare boards available and I could get one directly from you?

Best regards,
Michał