OS4E

Re: OS4E

Postby mrgs » Wed Jan 16, 2013 10:17 am

@fmotta: Thank You for Your note. I will check OS-9, as another OS(s) to pick up nice ideas. But as I have written: I do not like to port an OS, I would like to build one from 'scratch'. --- My pleasure to see You here and additionally: You are talking about an OS not a 'bare-code' solution. ;) ---

@ysapir: Thank You for Your answers. Well, "To set to count clock cycles for TIMER" is just the first step for a nice RTC. ;) Additionally we need more: - freq_of_CPU >> freq_of_TIMER; - nice (efficient/quick/well designed) IT handler routine, - OS needs to remain minimum time at IT masked state ... OK. Forget it. Thank You again. Regards, Gabor.
| OS4E : A preemptive, multiprocessing, microkernel based OS for Epiphany ARCH |
User avatar
mrgs
 
Posts: 63
Joined: Mon Dec 17, 2012 3:22 am
Location: Hungary

Re: OS4E

Postby mrgs » Thu Jan 17, 2013 2:39 pm

Hi, Attached please find an ASCII logo for OS4E. Well, it is strange, but I need an expressive project code, logo at the beginning. So, attached please a zipped header file which describes these. (Project Code : "054E")

What do You think? What about the variations?

Code: Select all
 
    _____  _____        . ______
   |     ||_____ |_____|.|______
   |_____|______|      |.|______
                        .  .  .  .

 


Have a nice day! Gabor
Attachments
logo.h.zip
OS4E ASCII logo
(1.38 KiB) Downloaded 548 times
| OS4E : A preemptive, multiprocessing, microkernel based OS for Epiphany ARCH |
User avatar
mrgs
 
Posts: 63
Joined: Mon Dec 17, 2012 3:22 am
Location: Hungary

Re: OS4E

Postby Oneill » Fri Jan 18, 2013 6:38 pm

Hi, Looks good. I would vote for the first one.
It is simple and elegant.
I hope that will be the reaction for the OS4E design and implementation.
Regards,
Oneill
User avatar
Oneill
 
Posts: 9
Joined: Mon Dec 17, 2012 1:49 pm

Re: OS4E

Postby mrgs » Wed Jan 23, 2013 9:03 am

Dear Oneill, Thank You for Your positive feedback. 'Cool' 8-) logo is just a very beginning of OS4E. I have just almost finalized all 'functions' so, I am very close to share my SW repository access. Here You can follow the developing process. ;) Nothing working yet, 'just' a skeleton now. Regards, Gabor
| OS4E : A preemptive, multiprocessing, microkernel based OS for Epiphany ARCH |
User avatar
mrgs
 
Posts: 63
Joined: Mon Dec 17, 2012 3:22 am
Location: Hungary

Re: OS4E

Postby notzed » Mon Feb 04, 2013 8:30 am

fmotta wrote:The term "Acceptable overhead" is always relative... and the only good answer is none. The only realistic answer is as little as possible. To quantify it I would need to assess an application and determine if I have any headroom.


Well the whole point of a "time sharing" multitasking operating system is to utilise all resources more efficiently - so the overheads they entail are not simply a benefit-free cost. A good answer can certainly be more than none. You do gain potentially a great deal for their overheads although the magnitude and acceptability of those always depends on the specific application. With any problem which doesn't fit into the 32K data+code space one would need to write specific overlay/code+data virtualisation code anyway, and that will entail overheads. If a cpu core is sitting idle because it is waiting on another one, that is an inefficiency that is also an overhead. Or if the whole accelerator is tied up with one task only and can't be utilised by other applications even if it's just waiting on data - that's a pretty big overhead. So, as an 'accelerator resource' for a general purpose computer (and not just some fixed-function process which happens to fit exactly), some mechanism of re-use and arbitration will be required, no matter what you want to call it.

@mrgs: I suggested cooperative multitasking as then all registers needn't be saved which would reduce the cost of context switches. If it was implemented as a call-back (which is possibly too much pain to work with) or kernel mechanism you could get away with saving very little state indeed. Cooperative mt doesn't require any extra code as the only point you're interested in switching is for any i/o (or here, memory) delays and that only happens at specific points in library calls.

Virtual memory wont be possible for C (or similar) as there is insufficient hardware support. At least not without putting it into the compiler somehow. Not that you need an MMU or virtual memory for multitasking. Although you'll be able to make the "code pages" read-only if that is desirable.

One idea I saw from some CELL BE developers (Insomniac) was to use what they called "shaders" - which today we'd call kernels - which can fit in LDS completely and can be executed on any available SPU. The 'operating system' then just becomes a scheduling system - how to get the kernel and it's data together at the same time and run them all in the correct order. This way it is possible to hide memory latency (potentially 100%) by interleaving code/data loading and execution. But this is more like an opencl runtime than an operating system. Ideally even this idea would need position independent code, although one could always work around that for a modest cost (e.g. staging buffers for DMA, or multiple compiles for each kernel). OpenCL could potentially do this ok but has a more limited execution model than is possible on epiphany, i.e. no persistent processes.
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia

Re: OS4E

Postby mrgs » Mon Feb 04, 2013 1:21 pm

@notzed: First of all, thank You for Your comment/suggestion. Well, 10 years ago I have designed/implemented a little cooperative multitasking RT system for a uC. It was a 'precise miracle' and a very nice 'trip' for me. Without any HW support: 2xUART, I2C, RTC, E2PROM log, RS485 bus protocol on 4K, ... How can I say?! I have to check: what can I achieve with a preemptive OS.

Well, as I see the Epiphany CPU/Parallella Platform at this moment, the key question is not the context switch, rather than memory management, as You marked the 'current' weakest points as: no MMU. Well, maybe I am wrong, but at now I 'believe' that I can build up a 'nice' memory management system without a CPU based MMU. How?!

Well, start to the easiest point. As, You mentioned: - I will protect the page(s) dynamically with the RO flags which provide by the CPU, and there will be a flat memory map. Not so interest. But IMHO: possible to build up a multitasking system without MMU.

MMU: So, what is the root of the problem: We have to 'generate' each addresses on runtime. Where are the 'compiled addresses'? At the 'file system', then the 'DDR RAM'. How can we deliver the code and the data into the Epiphany? Via FPGA and CPU DMA. So, 'theoretically' we can use the FPGA like an external MMU. Oh... what a 'crazy' idea?! :D

A few days ago I just start to wondering: What if: Put an illegal, but definite instruction code before all memory address instruction, to support the memory management. (NON exist MMU case). Then generate the new address with the exception handler?! What do You think? ...

I am going to check CELL BE-s solution. Thank You. Regards, Gabor --- let me have Your feedbacks(!) thank You(!) ---
| OS4E : A preemptive, multiprocessing, microkernel based OS for Epiphany ARCH |
User avatar
mrgs
 
Posts: 63
Joined: Mon Dec 17, 2012 3:22 am
Location: Hungary

Re: OS4E

Postby notzed » Tue Feb 05, 2013 4:09 am

mrgs wrote:@notzed:
MMU: So, what is the root of the problem: We have to 'generate' each addresses on runtime. Where are the 'compiled addresses'? At the 'file system', then the 'DDR RAM'. How can we deliver the code and the data into the Epiphany? Via FPGA and CPU DMA. So, 'theoretically' we can use the FPGA like an external MMU. Oh... what a 'crazy' idea?! :D


On the CELL BE a mmu sits on the DMA engine (iirc), so any global addresses are translated to match the current process. The SPU code must manually (or compiler generated) move data to buffers in the LDS using DMA as it can only access the local address space directly.

So the idea doesn't sound so crazy, although i presume it would have to treat all cores as one in terms of memory.

A few days ago I just start to wondering: What if: Put an illegal, but definite instruction code before all memory address instruction, to support the memory management. (NON exist MMU case). Then generate the new address with the exception handler?! What do You think? ...


The only option seems to be a trap - and that has to be handled by the arm side which could be rather expensive (i presume, perhaps it could be put into the fpga). And that would require compiler work - in which case you could just add a function call anyway. Actually I guess it would just check a local cache and use that if available, and if not then do the trap.

First idea sounds more efficient/practical, although tbh i'm not sure yet how the external memory stuff is setup to start with, and if/how the epiphany can communicate with off-chip resources on it's own.

The manual talks about 'external accesses' but the memory map appears full, presumably it means any address access beyond the on-chip core count. In that case I guess one could put a basic MMU into the FPGA and wouldn't need software support. But in that case you're limited to all cores on the E chip having a shared virtual address space and not independent which is seems to be what you're after.
notzed
 
Posts: 331
Joined: Mon Dec 17, 2012 12:28 am
Location: Australia

Re: OS4E

Postby mrgs » Tue Feb 05, 2013 8:41 am

@notzed : Wow. :) My pleasure to 'talk' with You. Well, after this comment I will check CELL BE. My MMU 'vision' is:

Plan A

(1) MMU 'HW part' functions should be implemented into programmable logic at Zynq, not ARM part,
(2) The compiler should be able to generate a proper code itself,
(3) MMU should use Epiphany DMA to deliver the proper --- 'yet dynamically readdressed' --- pages.

So the idea doesn't sound so crazy, although i presume it would have to treat all cores as one in terms of memory.
Well, I think, better approach if I separate all cores with own MMU and grant an efficient method for 'IPC'. I should to start the design/implementation with one core at first. Based on my current model I imagine an MMU-DMA combo like an 'intelligent code/data injector'. As I see now Epiphany is ready to use the internal RAM on the mesh, but I should to extend this area via an 'injection' method. Could I? Could it work?


Plan B

The only option seems to be a trap - and that has to be handled by the arm side...
Well, ... no?! Let me clarify myself: As far as I understand Epiphany now: If CPU runs into an invalid instruction code, then it generates a software exception, so from the IT/Exception handler --- part of the OS --- I can manage the dynamically addresses problem. (Like a soft MMU for the applications) ... I should to write a test code for it. ...


SUMMA

First idea sounds more efficient/practical, although tbh i'm not sure yet how the external memory stuff is setup to start with, and if/how the epiphany can communicate with off-chip resources on it's own.
Yes, 'of course' the first is 'my current best tip' now. Well, as far as I understand, "Epiphany can not able handle the external resources as its own." We have to 'inject' the code/data.

... but the memory map appears full ... on the E chip having a shared virtual address space and not independent ...
Yes it is. This is way I would like to 'extend' the physical address space with a virtual one with a hard or soft?! MMU. Well, this is not a 'real/classical' extended address space rather than a dynamically forced 'moving' processing on the code/data which made for 'classical' processor. ( 'compiler' will affected too )


What do You think? Regards, Gabor. --- sorry for the long answer ---
| OS4E : A preemptive, multiprocessing, microkernel based OS for Epiphany ARCH |
User avatar
mrgs
 
Posts: 63
Joined: Mon Dec 17, 2012 3:22 am
Location: Hungary

Re: OS4E

Postby fredd » Tue Feb 05, 2013 6:01 pm

mrgs: I also wonder lot what the best way to utilize some kind of "dynamical addressing" + DMA for managing the combination of SRAM and DRAM as a heap. Even though my main interest is for object models and dynamical languages (like Python, Java, for bigger applications not fitting in 32k), rather than a multi-tasking OS, many issues will be the same. I never thought of using the FPGA for something like this, thanks for that idea! (As a complete software person, I have a lot of learning to do what a FPGA could do for me and how)

My concern is that we don't want to wrap every single load/fetch of heap memory in MMU/caching logic. Rather I think memory should be divided into functional chunks (like functions, C/C++ structs, arrays, Java/Python/etc objects) so that if a function wants to access a chunk repeately, it only needs to call the MMU logic once, to ensure the chunk is in SRAM. Preferably most of the heavy lifting should be done by the compiler rather than the programmer. For instance,
Code: Select all
mystruct* s = get_pointer_from_somewhere();
int i = s->some_field;
s->some_other_field = j;
// do more stuff with s
return;

will be translated to something like, in pseudo C-code:
Code: Select all
 
// could be pointer to chunk in DRAM
LONGPTR mystruct* s = get_pointer_from_somewhere();
LOCALPTR mystruct* s_local = MMU_ensure_in_sram(s);
int i = s_local->some_field; // fast sram access
// do more stuff with s_local
MMU_unref(s_local); // tell MMU we're done with s
return;

I think this will work well for core-private memory and read-only shared memory (code). For core-to-core communication one could avoid cache-coherency issues by using explicit message-passing, which could transfer ownership of heap chunks.

Another concern: we _really_ want DMA to work simultaneously with the CPU, and not CPU waiting for DMA. The compiler should be able to figure out the most obvious cases for prefetch, but the programmer might want to give hints in some cases. Anyway, one of the big advantages of a software/fpga memory management rather than hardware caches and MMUs is that its behavior could (and should) remain transparent and programmable to the application programmer, rather than needing to guess/measure/optimize how a black-box hardware cache will behave with your code. :)
fredd
 
Posts: 12
Joined: Wed Dec 19, 2012 1:09 pm

Re: OS4E

Postby mrgs » Tue Feb 05, 2013 8:14 pm

@fredd : My pleasure to read Your comment. Well, step by step:

(#1) “dynamical addressing” : Tell the true, I was very close to put THIS key question for SW guys whois going to port Erlang, Python, ..., another interpreter / virtual machine OR 'just' libraries. How these clever guys will deliver enough 'RAM' for their SWs. So, thanks for You, at now I am sure we have the same 'challenge'.

(#2) “OS v.s. dynamic language” : At the beginning I have had a 'doubt': an OS or virtual machines for a nice language. BUT(!) day by day I realized: (a) at finally 'a language' wants a same environment like just an OS can deliver (memory/scheduling/IO...) (b) which language?! (c) I really want to try out myself to write an OS. ('sorry' about this).

(#3) “FPGA” : You, as a sw person win(!) an another language: VHDL. Welcome on the board 'de' Parallella. ;)

(#4) “chunk” : I can accept that, You are thinking about chunk/slice. Well, I have not finalized low/high memory management system of OS4E. I agree (a main point), compiler should support 'us' its own way, but I do not want so much as You written at Your sample.

(#5) “core-private, read-only, core-to-core, message-passing...” : In my point of view these also the area of the OS not an 'in-compiled function' in an application. But I think, Your idea will work well too.

(#6) “DMA” : Should work 'without' CPU. This is why we call it DMA. - There was an another issue/topic, as I understand it, it talks about the 'remote SDRAM' access. 'Solution' is the DMA(!) -

(#7) “extended memory management” : It will bring not 'just' the 'transparency', but a very good opportunity for a flexible, 'all language' supported system. - It is necessary but not sufficient condition for a 'real' OS on Epiphany. - I think, we should share our results 'soon'.

Regards, Gabor
| OS4E : A preemptive, multiprocessing, microkernel based OS for Epiphany ARCH |
User avatar
mrgs
 
Posts: 63
Joined: Mon Dec 17, 2012 3:22 am
Location: Hungary

PreviousNext

Return to Epiphany Operating System

Who is online

Users browsing this forum: No registered users and 1 guest

cron