Accessing HDMI (and other interfaces)

Any technical questions about the Epiphany chip and Parallella HW Platform.

Moderator: aolofsson

Accessing HDMI (and other interfaces)

Postby EclecticMonk » Sun Sep 07, 2014 6:35 pm

Hi there,

although I've been playing with a board for a couple of months and silently reading and collecting feedback from the forum, this is my first post and so I'd like to take the chance to say hi everyone.

After getting familiar with the internals of the Epiphany chip and the SDK, I decided to start playing with the interfaces to make lights blink and show shining colors in the screen. I've been trying to reuse code from the Mandelbrot example to understand how to output images to the screen, but I've been unable to get all the details. As a start, I just tried to replace the Mandelbrot output by a constant number, so I could get a fixed color in the screen; but even that simple step has turned to be more complicated than expected. Depending on where I modify the value, I get nothing on the screen, runtime errors, or hang the complete system and needed a couple of reboots to get it back.

I've to say that I'm using a brute force approach just trying different combinations and seeing what happens, but this is because I haven't found any details on how to correctly access the interfaces. Is there any documentation specifying ho to access the interfaces? What are their memory addresses, required format of the output data, synchronization issues, etc. ? Are there related drivers? I've skimmed through the documents and the forum and haven't found anything related, other that trying to reverse-engineer the examples. Am I overlooking or missing some document/guide?

My next step is to try some video processing tasks, and then continue checking the other interfaces for more IO functionality. We plan to use it for teaching purposes later, so it's important to have these steps clear. Any pointer would be greatly appreciated.

Cheers
EclecticMonk
 
Posts: 3
Joined: Sun Sep 07, 2014 5:56 pm

Re: Accessing HDMI (and other interfaces)

Postby notzed » Mon Sep 08, 2014 3:47 am

All this information is readily available but I will give you an outline and some code to play with. I don't really have anything better to do right now (that i care to ...) so this is a pretty long post. I'm assuming you know very little about unix/linux but even if you do this detail might be useful for someone else. Once you know a couple of the details it all clicks into place and makes sense and is quite simple.

On the parallella all graphics access is via a "frame buffer" (i.e. a buffer which holds a video frame). The framebuffer is just a rectangular grid of values which represent the colour pixels - in the i'm sure you're aware - RGBx format. i.e. 4 bytes, for easier access but only 3 provide useful information here. It's considered a "dumb" framebuffer as it doesn't do anything else but show an array of memory on the screen; no acceleration etc.

The framebuffer is accessed via the framebuffer device which is accessed via /dev/fb0 (on parallella) - this is most certainly documented as part of the kernel. There are two parts to how you use it. There are some man pages but they're not very good and i think there's some documentation in the kernel sources (yes here: https://www.kernel.org/doc/Documentatio ... buffer.txt you should read it)

First you can memory-map the framebuffer array itself which allows the process to write (and read) pixels directly to the screen via a plain pointer (aka "array"). "man mmap" documents this command. You don't really need to know the specifics of how this is implemented but basically it adds the memory range for the framebuffer to the current process.

The second part is what is called an "ioctl" - input/output control - which is basically a device-specific function call directly into the kernel driver and is a linux/unix mechanism for programs to talk to the kernel, it can be used to query or control the framebuffer device. The interface is very simple - you give it an open file, a number specifying the command and a single value which contains the command parameters - input and/or output, usually a C structure. The ioctls available for the framebuffer device are defined in the linux header file "/usr/include/linux/fb.h" and allow you to do things like determine the size and format of the framebuffer (needed to mmap it and use it properly), it's physical address in memory - required for screen capture software or accessing it from the epiphany directly or accessing any other features or facilities the framebuffer provides. The parallella one is pretty basic so it really just lets you access the pixels.

AFAIK there are no vsync/multi-buffer facilities available with this particular framebuffer device which is a bit sucky but unfortunately pretty standard. So you just have to blat your stuff in there and live with all the horrible screen tearing and other visual artefacts this causes.

Another way of course is to use a GUI toolkit which accesses the X Window System to display output and copy your data to the screen using whatever mechanism (pixmaps/images/textures) it uses. This is a more useful approach if you want to make an "end-user" application because it will play nice with the rest of the system but comes at a performance and complexity cost.

Ok so some code on how to access it. I'm going to step through writing it from scratch here for this post - i don't think it could really be any simpler and still work so the steps are pretty straightforward.

The framebuffer like all unix devices is accessed via a file, in this case /dev/fb0 and like all files it needs to be opened first. You use the open().

This is all in C because I wouldn't use anything else. All the following code I place in the public domain.

Code: Select all
$ man open
OPEN(2)                    Linux Programmer's Manual                   OPEN(2)

NAME
       open, creat - open and possibly create a file or device

SYNOPSIS
       #include <sys/types.h>
       #include <sys/stat.h>
       #include <fcntl.h>

       int open(const char *pathname, int flags);
       int open(const char *pathname, int flags, mode_t mode);

       int creat(const char *pathname, mode_t mode);

DESCRIPTION
       Given a pathname for a file, open() returns a file descriptor, a small,
... (more that tells you what the args mean)


(use 'q' to quit the pager)

Here we see you need to include a few things before using open - just copy this into the file using your mouse.

While looking at man pages you know you'll also need ioctl and mmap so man those as well and grab the includes they need. Oh and I forgot close().

Code: Select all
$ man mmap
...
SYNOPSIS
       #include <sys/mman.h>

       void *mmap(void *addr, size_t length, int prot, int flags,
                  int fd, off_t offset);
       int munmap(void *addr, size_t length);
...
$ man ioctl
...
NAME
       ioctl - control device

SYNOPSIS
       #include <sys/ioctl.h>

       int ioctl(int d, int request, ...);
...

$ man close
...
SYNOPSIS
       #include <unistd.h>

       int close(int fd);
..


So to start the code you open the framebuffer to get the 'file descriptor' (handle/key/whatever) from which everything else is accessed. From the man page open returns -1 on failure so check that.

Code: Select all
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <stdio.h>
#include <linux/fb.h>

#define FB "/dev/fb0"

static int fb;

int main(int argc, char **argv) {
   fb = open(FB, O_RDWR);

   if (fb == -1) {
      perror("Unable to open fb " FB);
      return 1;
   }

   // rest here

   close(fb);
}


Running it at this point should produce no errors (or other output):

Code: Select all
notzed@minized:~$ gcc -o fb fb.c
notzed@minized:~$ ./fb
notzed@minized:~$


So the first you need to do is find out about the framebuffer information using ioctl.

Having a look at the fb device header gets you the info you need. The names of everything isn't entirely consistent to match the relevant info but the comments help line everything up.

Code: Select all
$ less /usr/include/linux/fb.h
...
#define FBIOGET_VSCREENINFO     0x4600
#define FBIOPUT_VSCREENINFO     0x4601
#define FBIOGET_FSCREENINFO     0x4602
...
struct fb_fix_screeninfo {
        char id[16];                    /* identification string eg "TT Builtin"
 */
        unsigned long smem_start;       /* Start of frame buffer mem */
                                        /* (physical address) */
        __u32 smem_len;                 /* Length of frame buffer mem */
        __u32 type;                     /* see FB_TYPE_*                */
        __u32 type_aux;                 /* Interleave for interleaved Planes */
        __u32 visual;                   /* see FB_VISUAL_*              */
        __u16 xpanstep;                 /* zero if no hardware panning  */
        __u16 ypanstep;                 /* zero if no hardware panning  */
        __u16 ywrapstep;                /* zero if no hardware ywrap    */
        __u32 line_length;              /* length of a line in bytes    */
        unsigned long mmio_start;       /* Start of Memory Mapped I/O   */
                                        /* (physical address) */
        __u32 mmio_len;                 /* Length of Memory Mapped I/O  */
        __u32 accel;                    /* Indicate to driver which     */
                                        /*  specific chip/card we have  */
        __u16 capabilities;             /* see FB_CAP_*                 */
        __u16 reserved[2];              /* Reserved for future compatibility */
};

...
struct fb_var_screeninfo {
        __u32 xres;                     /* visible resolution           */
        __u32 yres;
        __u32 xres_virtual;             /* virtual resolution           */
        __u32 yres_virtual;
        __u32 xoffset;                  /* offset from virtual to visible */
        __u32 yoffset;                  /* resolution                   */

        __u32 bits_per_pixel;           /* guess what                   */
        __u32 grayscale;                /* 0 = color, 1 = grayscale,    */

... and more


The fix screeninfo givs you the size of the memory you need to map (smem_len) as well as the pysical address which is needed for the epiphany (smem_start) and some other important details to do with the format.

To get the fixed screen info you use the FBIOGET_FSCREENINFO ioctl and it works just like a parameterised function call:

Code: Select all
static struct fb_fix_screeninfo fix;
...
   int res;

        // the open code here

   res = ioctl(fb, FBIOGET_FSCREENINFO, &fix);
   if (res != 0)
      perror("getscreeninfo failed");

   printf("framebuffer size %d @ %08x\n",
          fix.smem_len, fix.smem_start);

Running it now (this is on my workstation, not the parallella)
Code: Select all
$ ./fb
framebuffer size 5242880 @ c0000000


At this point you can now mmap the memory and use it but you don't know the geometry of the screen which can be different (for example my parallella outputs 1280x1024) so do that first just.

This is where the variable screen info comes into play. It's variable because you can write to it although i don't think the parallella framebuffer allows you to change it after boot.

Code: Select all
static struct fb_var_screeninfo var;

   res = ioctl(fb, FBIOGET_VSCREENINFO, &var);
   if (res != 0)
      perror("getscreeninfo failed");

   printf("size %dx%d @ %d bits per pixel\n",
          var.xres_virtual, var.yres_virtual, var.bits_per_pixel);


One uses the virtual resolution because that defines the "physical" (or in-memory) size of the screen. It is possible this is larger than the viewable area but that's not very important here.

Code: Select all
$ ./fb
framebuffer size 5242880 @ c0000000
size 1280x1024 @ 32 bits per pixel


There is also information there about the pixel format and so on but for simplicity and performance I just assume it's RGBA (or ABGR, or ARGB, i never remember so i just twiddle till it works too - i think its ARGB when encoded as a little-endian integer) on the parallella. I don't know if it supports other formats and if broke the code well you add support for it later.

So the last bit is just to mmap it. Because you have the device file you just mmap it with the size and it handles all the rest and gives you a new pointer to the process-local address of the start of the framebuffer.

So putting it all together and adding some error handling.
Code: Select all
// By Michael Zucchi.
// This code i place in the public domain or equivalent.
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <stdio.h>
#include <linux/fb.h>

#define FB "/dev/fb0"

static int fb;
static struct fb_fix_screeninfo fix;
static struct fb_var_screeninfo var;

int main(int argc, char **argv) {
   int res;
   unsigned int *fp;
   int x, y;

   fb = open(FB, O_RDWR);

   if (fb == -1) {
      perror("Unable to open fb " FB);
      return 1;
   }

   // rest here
   res = ioctl(fb, FBIOGET_FSCREENINFO, &fix);
   if (res != 0) {
      perror("getscreeninfo failed");
      goto fail;
   }

   printf("framebuffer size %d @ %08x\n",
          fix.smem_len, fix.smem_start);

   res = ioctl(fb, FBIOGET_VSCREENINFO, &var);
   if (res != 0) {
      perror("getscreeninfo failed");
      goto fail;
   }

   printf("size %dx%d @ %d bits per pixel\n",
          var.xres_virtual, var.yres_virtual, var.bits_per_pixel);

   fp = mmap(NULL, fix.smem_len, O_RDWR, MAP_SHARED, fb, 0);
   if (fp == (void *)-1) {
      perror("mmap failed");
      goto fail;
   }

   printf("virtual @ %p\n", fp);

   int stride = var.xres_virtual;

   for (y=0;y<512;y++)
      for (x=0;x<512;x++)
         fp[x+y*stride] = 0x00ff0000;

   close(fb);

   return 0;

 fail:
   close(fb);

   return 1;
}


The only assumption it makes is that the screen is 32 bits per pixel and ARGB format which is fine for prototyping/mucking about.

I normally wrap this in a couple of function calls so other code can use it. You don't really need to care about closing it because the os cleans up when the process terminates, but it's just good programming practice and allows it to be reusable.

This is from the ARM side, to access it from the epiphany cores you need to write directly to the physical address of the framebuffer - because of the way the hardware works it's not guaranteed that the epiphany can even see that memory which would force you to write to the shared memory first and then use the arm to copy it to the framebuffer but thankfully the way it currently works it can so I presume they will retain this ability.

You just need to pass the address "fix.smem_start" to the epiphany code and use that as the "fp" pointer instead. The mandelbrot example already does this as far as i know.

I would recommend not accessing it using bytes (char *) as that is going to be super-slow; actually you need to use DMA for performance but that's another topic.

(at this time) The epiphany has unprotected access to most of the physical memory on the parallella - so you need to be careful. Code bugs can easily crash the whole system as you've already discovered. It should be highly unlikely to cause any physical damage this way though.
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia

Re: Accessing HDMI (and other interfaces)

Postby EclecticMonk » Mon Sep 08, 2014 5:09 pm

Hi notzed,

that's the most outstanding answer I've ever received in a forum. I'm glad that you didn't had anything better to do by the time you wrote it :) .

It's crystal clear now. I was basically overlooking a huge detail: I've been working and debugging FPGA prototypes and bare metal embedded processors for so long, that I've developed a tendency to try to understand every single detail on the interfaces, and then I was just overlooking the benefits of having a complete OS running on the ARM. I was looking for info at the hardware level, but didn't think about looking at the kernel level. So thanks for the wake up call, I guess everything is going to be a little easier now.

I'm going to check your example and the provided links. I own you a beer ;)
EclecticMonk
 
Posts: 3
Joined: Sun Sep 07, 2014 5:56 pm

Re: Accessing HDMI (and other interfaces)

Postby claudio4parallella » Wed Sep 12, 2018 8:44 pm

That's exactly what I need for my CLUSTER of 64 cores (4 parallellas).

The fact is that the IMG ubuntu-14.04-headless-z7010-20150130.1.img to be used for the eLink Cluster have not any Frambuffer or HDMI support.

Any solution please? thanks
claudio4parallella
 
Posts: 60
Joined: Thu Aug 10, 2017 3:48 pm


Return to Epiphany and Parallella Q & A

Who is online

Users browsing this forum: Google [Bot], Majestic-12 [Bot] and 6 guests