OS4E

[piotr5]

just a word about language vs os: long time I have thought of ways to make a better programming-language, especially with parallel processing in mind. I realized several things:

the difference between os and language is that in addition to what an os does do a language provides a human user interface. i.e. the ideal language would allow any input to produce any program possible. sort of like a programming language programming language. the ideal language would not be so ideal at all since humans do not like their interface having too many options. so one can drop this aspect and cope with it some centuries later.

something an os does provide but is better handled by language would be the notion of abstraction. for example linux provides an abstraction layer for the file-system, so that programmers can handle their files in an unified standard way. the reason why this would better be handled by languages is that bureaucracy is added and there's no possibility to optimize it out during compile-time. if this were part of the language then the things could be handled by inline-code. in reality however the overhead isn't that bad, and a language would need to provide non-inlined too.

the major problem with parallel processing has been introduced by the attempt to wrap it into an abstraction: threading! when I run multiple programs at the same time, the output of each is sent to one and the same console, and afterwards cannot be seperated. while debugging also it isn't visible how the different threads interoperate as they all are in the same debugging session. wouldn't it be better to run a separate os on each core, handling the code as single-core os? well, of course not, these problems can be solved at the level of programming language.

the reason why the compiler cannot automatically create threads on its own without being programmed to do so is that our concept of programming is centered around procedual execution. for example in mathematics every algorithm is a sequence of things that need to be done one after the other. there are attempts to break out of that though. for example the startup-files of my os are started according to what requirements they have. so instead of writing what sequence needs to be computed, let the software figure out in what order the commands need to be executed and if at all. I think it's obvious now, this is something the os should do, in cooperation with the programming-language. I think this is the direction os4e should be heading to. mere memory-management wont cut it! so forget threading-support, build a scheduler instead!

[mrgs]

@piotr5 : Thank You for Your comment! I try to reply for Your paragraphs:

topic : As 'language v.s. OS' : Well, first of all, I think, this is not a fight, this is a cooperation between the OS and 'the language'. With the separation of 'this two phenomena' I would like to partition the 'problems'/tasks.

ideal language : Well, tell the true I do not understand Your idea about the 'ideal language'. But(!) Let me note: In my model, the language is 'just' a tool to describe 'something'. An 'ideal language' is 'nice' / 'expressive' / 'natural' / 'consistent' / ... 'human lovable'

'better handled by language' : Well, I think we are free as special at the abstractions. A 'poor OS' try to deliver some services for You on a standard way. If You do not like this way, then write an own file system - 'independently from the OS' - with Your language, and You get as level abstraction as You can imagine.

threading : Well, I think, the threads are our best friends! I can imagine a TASK easily, where EVERY THREADS has own outputs/console. Additionally IMHO, the 'multi-core issue' is 'semi-independent' of programming languages.

'procedural execution' : Well, - hardest part -. We create the threads, 'poor compiler' just compiles 'our thoughts' for our machines. As far as I know at this moment we have procedural devices. So our software (does not mater it is an OS or application) MUST be procedural BUT(!) it could/should be runs in concurrent/parallel environment. So, until we have not got non procedural devices we have to use what we have. So, - I am sorry - I have to keep the memory-management, threading, scheduler, ... models for a 'present' OS. But(!) we can use any part of it for another non OS or semi OS implementation.

Regards, Gabor. - p.s. : I am really interesting: do You thinking about 'quantum computers?!'

[piotr5]

guess I must explain the differences between threading and what I have in mind:

usually a threaded program is still one monolithic piece of code and the os makes sure that huge program is split into small chunks. parallella doesn't have the memory to store the huge programs, so this is a bad idea. I think instead the programmer should split the program into small chunks, label each chunk with what it does provide and what it does require, and an os should load the chunks onto the cores and execute them so that dependencies are met. of course each chunk then still is procedual, but only at a per-procedure level. i.e. every call to procedures has been announced to the os beforehand, and basically calls to procedures are considered bad, in the same way as the "goto" command is being avoided. so instead of writing printf into a monolithic procedure, you write a short procedure that fulfills the requirements for another procedure that consists of that printf. debugging-messages then are a matter of adding the procedure to be debugged to the requirements of the printing-procedure, and the result should be that printing will be done in parallel on another core without much slow-down for the program to be debugged (since the program does not really require the debugging-message to be finished, only the printing itself requires other printing to be finished in sequence)...

the ideal language, as you said must be comfortable for the programmer, and since each programmer has different preferences, it must be customizable to such a degree that no matter what machine-code you want to get for output, it should be possible to produce it from any random garbage used as input when the fitting programming-environment is used (fitting in the sense it has been chosen based on that actual garbage you will use). for example tex tried to go into such a direction...

one word about quantum-computers: I've been told they still require a lot of knowledge we do not have ready. and even if quantum computers would exist, they wouldn't be all that great because only a subset of all programs that need long time to finish will be computed in short time by the quantum-computers as far as we know...

[mrgs]

[piotr5]

I understand your plans for a flat memory, but this does not contradict what I said!of course you can have threading the way you want it and the compile-time threading I suggest, both in the same os. my point is that my idea takes up less space and less effort, not to mention the speed-advantage when a compiler takes over the work you want to do in run-time...

as for the ideal programming language, once upon a time I too thought there would be a programming-language for every purpose, each having an own character and whoever wants something that isn't a language could just create one. but then I realized the problem is not the language but what ideas the language has implemented. for example I like the idea of static oop, therefore I hate java and love c++. and the reason I like the idea of static oop is that it allows to implement even newer ideas for what a language could do, without slowing down run-time. choosing a language really isn't about personal preferences, it's about choosing the newest technology one has deemed useful. that's why programming language does not matter, with the right preprocessor even assembler could become the right platform-independent language. it's only the libs that matter, the os and the services it provides

what you are talking about is an os for parallella, an os for epiphany should run the epiphany cores in a native mode, allowing to make use of what already is present, and it shouldn't try to emulate some other kind of system. there's lots of libs to choose from for a posix-compatible os, so I can understand you want os4e to be posix-compatible including posix threads. however, I think since all those useful libs have nice source-code, it does not make sense to solve everything at run-time, and the other way around it neither makes sense to rely on the programmers to take care of threading as opposed to handling that during runtime where it is known what threads the other cores are occupied with. we complain how little 32k is, but on a truely multicore epiphany of the future, 4k cores with a total of 128mb on-chip memory, there'll be plenty of space for an os, but no user-friendly application with current threading technology will ever manage to keep all these cores occupied!imho 16-core epiphany does not need an os, but once the chip has more cores than your programs have manually prepared threads in the sourcecode, then some os would be needed to actually create many more threads! as I said, imho abstraction is the only thing an os shouldn't do because it better fits a programming-language. even though my suggestion sounds a bit like a scripting-language, it needs to be done by an os and not by a compiler alone, it is far away from being an actual abstraction, and requires run-time info...

[mrgs]

Closer & Closer : (1) There is no contradiction. (2) Both of us like an OS?! (3) Runtime vs Compile-time : Well, as far as I can imagine our future 'massive parallel processing' system, I 'see' a flexible, adaptive, dynamic … system. I see more problems which arise at the runtime, not at compile time. I give more weight to runtime than compile time. I think 'this is just' an another strategy. I think we are face to an engineering problem and we choose from 'own' shelf/mind some tools. Luckily we have overlapped functions what we achieved with our OS.

“... I realized the problem is not the language but what ideas the language has implemented. ...” : I agree.
“... . it's only the libs that matter, the os and the services it provides … “ : On a proper level of abstraction with 'a precise knowledge' how far this level (libs, OS) from the real processing system.

Parallella vs Epiphany : You are right, actually we have to support Epiphany with a 'little' additional hardware. But a question arise: Does Epiphany start to work (boot) at 'classical' native mode. I think, she does not. We need an 'external agent' (better to call a 'little angel') to 'boot Epiphany'. So, some external HW MMU is just for fun. I would not like to emulate another system I 'just' like to bring out the best from the 'current' ARCH. (Parallella)

'POISIX - compatible' and compile-time runtime 2. : POSIX is not a 'must have' stuff, It 'just' show a standard way of an OS, so, I am not sure at now. Back to the compile-time vs runtime again: As You mentioned the two endpoint: The answer is: to find the right balance(!), well, IMHO a 'little' more 'force' should to go to the runtime.

Cores / 'S'RAM : Well, it is a hard question. Will I more happy with more RAM inside on the chip?! What a crazy question?! Of course Yes, … OR NOT(!) I guess, I could be more happy with a 'light' MMU, and native, stand alone starting. But these do not matter in really. I think, 'our major measure is GFOPS/W' now.

'OS for 16-core Epiphany' : Yes, 'why' not?! I am going to use 'only 2 local memory banks permanently' for OS4E. (!) It does not mean that the size of OS is 16K.

"...even though my suggestion sounds a bit like a scripting-language, it needs to be done by an os and not by a compiler alone, it is far away from being an actual abstraction, and requires run-time info..." : I think, this is a good 'join point' : OS will deliver nice run-time info ...

SUMMARIZE : Later or soon I have to show something. I think the best way, which is really represents my ideas, is some source code. 'Let me go!' Regards, Gabor

[SamJBarney]

I'm all for a Parallela Operating system. While the board does currently have it's limitations, it's better to start with something small and work up to the larger systems later.
For instance, I want to build a distributed OS so that I can use the PEC_NORTH and PEC_SOUTH links to connect multiple boards and use them as a single computer. Even though I will be getting two boards when they start shipping them to the lower-tier backers, I'm not going to start by connecting them together and praying it works. Gonna start small and work up to it.
As for the soft MMU concept, I do not know how well that will work, but I will spend some time mulling it over before I make a full comment on it.
My concept of how to implement multi-threading is built around using one of the ARM cores as a thread/process scheduler. Any thoughts?

[mrgs]

Hi and welcome on the board SamJBarney, Well, I am quite optimistic about the OS4E, as You could see at my previous notes I am going to build up an OS from scratch - step by step -. I see the huge possibility in the ARCH despite "her limits". - Everything has limits, and additionally "we" make these.-. So, we do the best. Good Luck. Tell the true, "I do not understand" the ARM managed approaches. Regards, Gabor.

[SamJBarney]

Thanks for the welcome, mrgs! By the way, I have a quick question: have you already built your bootstrap OS compiler for the actual development yet? If so, was there anything specific you had to do to compile it? The reason I ask is due to the fact that while I have built a few x84 OS dev compilers I wasn't sure if there was something epiphany-specific to the toolchain compiler.
Also, what do you mean by "ARM managed approaches"?

[mrgs]

- "bootstrap OS compiler" : Well, as I understand You, ... : I am going to use the official epiphany-elf-* compiler and binutils for OS4E development. "I have a toolchain" without any additional trick.
- "ARM managed approaches" : Where the ARM CPU is the master and the E16 cores are slaves. A kind of co-processor approaches. - In my "vision" the E16G301 will be the main, "stand alone" CPU/mesh with own OS. Regards, Gabor