The Evolution of OS Design
Introduction
The Object Operating System, OOS (pronounced "ooze") is an attempt to create a new operating system architecture. It's designed to use the filesystem to achieve a large number of tasks which would normally be done through different mechanisms. Much of the design philosophy was inspired by Unix and Plan 9, but OOS attempts to do things in its own way, trading compatibility for a simpler design. The most significant departure from its predecessors is a new type of filesystem in which files, directories, and libraries have been replaced by a unified entity known as a container. In addition to being unified, containers are also event driven, so that operations like reading, writing, copying, and deleting files are considered events whose default actions can be overridden. This allows a container to perform arbitrary operations on files as they are copied in and out of the container, or to provide virtual namespaces.
The filesystem
When you log onto an OOS machine, you are essentially logging onto the network, and your workstation is simply a node connected to it. As such, the filesystem is structured so that the root directory contains a list of the machines on the local network. It's also possible for a user to mount remote machines or even networks here. The root directory is multiplexed; that is, any changes a user makes to the directory are visible to only that user. Other users will be unaffected. Any directory/container can be multiplexed, and doing so is a convenient way to provide a private namespace to each login.
Because files and directories in OOS are the same thing, you are able to cat a directory and cd into a file. Being able to embed objects inside a file makes them act somewhat like resource forks on OSes such as MacOS. In this way, meta-information like icons can be embedded directly inside a file. However, OOS files are much more powerful, since the inside of a file can contain a full directory hierarchy. Because files and directories in OOS can contain data inside them, they are referred to as containers.
A container has three principle parts: A file part, a directory part, and a functional part. The file part acts just like a file in traditional operating systems. You can open it, read and write to it, seek within it, etc. The directory part also works like traditional directories. It allows a container to hold other containers within itself. The functional part works somewhat like a library, but there are a number of subtle differences. A program is able to link to one or more containers' functional code, making them act just like shared objects, but unlike shared objects, individual functions can be made to run with elevated privileges over the program linking them. This level of granularity virtually eliminates the need for setuid binaries, as are common on Unix systems. It also means hardware drivers can largely be written as libraries instead of kernel modules, reducing the burden placed on the kernel.
Containers also possess event-driven properties. Events include things like copying a file into or out of a container, creating and deleting containers, modifying the file or functional part of a container, and listing a container's contents. The default behavior for these events are the traditional behaviors seen in other OSes. For example, copying a file into a container will make a copy of the file inside the container's namespace. But because events can be overridden, it's possible to perform actions on the file as it's being copied. For example, it's possible to create a container that automatically compresses files as they are copied in. Events can also be used to provide virtual namespaces. When a user attempts to list the files inside a container, they can instead see something else. This is how OOS provides information on processes. It has a container called cpu, whose files are actually running processes, much like the proc namespace in Plan 9 and some Unices. I'll discuss the cpu container in more detail later.
Each container has an inode in which things like filenames, ownerships, permissions, and sizes are stored. Also included is a flag which determines if the container's contents will be multiplexed. By multiplexing a container, every user session will have its own private namespace within the container. Consider the previously mentioned container that compresses all files copied into it. If that container is multiplexed, several users can make use of it simultaneously without getting in each others' ways. Each user will think they have exclusive control of the container, when in reality they are sharing it. Because the namespace of a multiplexed container is connected to a specific session, when the session ends (i.e. the user logs out), the contents of that namespace are lost. Therefore, any important information stored in one of these multiplexed containers should be copied out before ending the session.
Each container is also able to specify to the kernel what type of filesystem it should contain and where it should look to read and write files. Most of the time, you'll want your containers to use the default OOS filesystem and use the local hard drive for your files. However, in some cases, you may want to store files in RAM for extra speed, or mount a foreign filesystem. Unless otherwise specified, all new containers use oosfs for the filesystem and /<host>/dev/disk for storage.
The kernel
There's nothing really revolutionary in the kernel. It follows the idea of a microkernel, with the core doing as little as possible, leaving the bulk of the functionality to external modules. Hardware abstraction in OOS is largely delegated to userspace libraries, leaving just the critical parts like multitasking, IPC, file manipulation, and memory management to the kernel. Generally, drivers are only placed in the kernel if they need special attention that can't be provided from a userspace entity (like particular timing requirements).
The GUI
The GUI is a perfect example of hardware abstraction delegated to userspace containers. A standard OOS system provides a container called gfx, which acts as the graphics driver. As such, it exports a set of privileged functions for manipulating the graphics card directly. The various functions provided include primitives like plotting points, lines, circles, and polygons, moving and scaling bitmaps, filling patterns, etc. Higher level functionality (like OpenGL) is also provided. The gfx container actually inherits its functionality from a generic gfx object which performs all its actions through software. The gfx driver overloads the functions for which it can provide hardware acceleration, higher resolution, more colors, etc.
As is normally the case with hardware drivers, containers which provide an implementation of the hardware are placed inside the driver. This gives them easy access to the driver's exported functions (in object oriented terms, the container inherits the functions from its superclass). Inside the gfx container, we find the screen container, which provides the implementation for a GUI.
Suppose we want to create a simple dialog and display it on the screen. We can create a window container using the "new" command, which creates a new object by copying it from a directory of templates. Once the window object is created, we can copy the widgets into it to fill in its content. Once that's done, displaying it is as simple as moving it into the /<host>/gfx/screen container. So it would work something like this:
new window w1 cd w1 new vpanel v1 cd v1 new textbox t1 new button b1 echo "Hello World!" >t1/text echo "OK" >b1/text cd ../.. mv w1 /localhost/gfx/screen
As GUI elements are naturally hierarchical in nature, it makes logical sense to arrange them in the filesystem's hierarchy. It also means that you would be able to use a file manager to generate graphical applications. Notice how attributes are set in the button and textbox widgets. There is a file within them called text, which, as the name implies, holds the text for these widgets. Each widget has a number of standard attribute files within it for setting colors, style, text, callback functions, etc. It is also possible to insert code directly into a widget for execution when it is activated; this is especially useful when network latency is high and a quick response is required, such as with mouseover events.
Suppose we also want to display this window on a remote machine. All we have to do is copy the window to the remote machine, e.g.:
cp /localhost/gfx/screen/w1 /titan/gfx/screen
This would put a copy of our window on a remote system named titan. Because the window and each of its widgets know how to draw themselves, the window on titan can be moved, resized, iconified, etc., and the widgets manipulated in any way, all without requiring any further network traffic. Contrast this with X, with which every action on a remote display means resending instructions on how to redraw the window and its contents. Anyone who has tried running remote X applications through a modem knows how painful this can be.
Because executable code can be inserted into the widgets, it's possible to send an entire program inside a window. Programs packaged this way are much like Java in that they are transferred in whole initially, then run entirely on the client side. In the other extreme, only the widgets themselves are transmitted through the network, leaving the originating machine to handle any computation. This scenario is similar to HTML with CGI scripts running on the server. A third way of handling remote displays is to send the program in a piecewise manner; the program is broken into pieces, and each piece is sent only when needed. Once a piece has been sent, it can be cached on the remote display server so it doesn't have to be sent again. For many tasks, this is the most efficient method.
Audio
Audio works much like video. There is a container called sfx which contains the audio driver. It exports privileged functions for playing waveforms, setting the volume, mixing channels, etc. Within the sfx container are containers for various types of implementations. A container called mp3 will play any mp3 file copied into it. Similar containers exist for other sound formats. Within the functional part of each of these containers are functions for identifying the audio type. For example, the wav container has a function to determine if an audio stream is in wav format. It also has a function to convert the wav stream into a raw audio stream, so that it can be played. All audio containers have these two functions, which can in turn be used to identify and play any known audio format.
Devices
Unix users will be happy to find that traditional device files still exist, and that they are stored in the usual /<host>/dev directory. The dev directory is generated dynamically to accurately reflect the hardware in the machine (a la devfs). Each device is represented as a hierarchy in which the top level container represents the whole device and lower levels represent components of the device. For example, a hard drive pool is represented as /<host>/dev/disk. Multiple hard drives are accumulated into a single pool of space, like LVM. Cat this file to see the raw contents of the pool, or cd into it to see the physical drives, represented as numbers (1, 2, 3, etc).
The CPU interface
Processes on OOS are presented in the /<host>/cpu container. Similar in concept to the /proc filesystem in Plan 9, but more capable, the cpu container holds a list of running processes on the given machine. An executable can be run by copying it into the cpu container. Once running, it will appear inside the container as a file whose name is its process ID. Cating this process ID file will reveal a memory image of the running process, whereas cding into it will provide a more easily disseminated collection of attributes, including a list of open files, its environment, its process-specific load average, its running state, etc. Deleting a process file will kill the process.
Starting a process through the cpu container causes it to start in its own session. This means its stdin, stdout, and stderr will be redirected to/from never-never land, which is good for daemonizing a process but bad for running interactive terminal programs. Processes will, however, be run with the same environment (environment awvariables, etc.) as your current shell, so if they create a GUI, that will still work (your local display path is stored in your environment, so running processes will know where to open their windows). If you need to run a process in your current session, you have to use the more conventional methods like fork, which under OOS is able to fork a process to a different machine. It's also possible to copy a running process from one machine's cpu container to another, provided both machines have identical CPUs.
Pcode
Throughout this paper, we've talked about running processes on remote machines without giving much thought to the incompatibilities between different CPU types. A program compiled for Intel will simply not run on a Sparc, for example. To get around this problem, we use a concept known as pcode. Pcode is a mostly-compiled form of an executable. In this form, it remains portable across CPU types, but because a large part of the compilation has already been done, it can be translated into a machine-specific executable quickly. This machine-specific portion is cached on the machine on which the process runs, allowing this translation phase to be skipped if the executable is run a second time. This idea is borrowed from Plan 9, but is similar in concept to Java with a JIT compiler.
Examples
Backups
Suppose there is a machine on the network called backup01 which contains a tape drive. If you want to backup a directory on your local machine, you would do so like this:
cp /localhost/directory_to_be_archived /backup01/dev/tape
Restoring the directory from tape is as simple as copying it back:
cp /backup01/dev/tape /localhost/directory_to_be_restored
It works essentially like creating images of, say, floppies under Unix, except here I can read and write the images to devices on the other side of the network. Unix would normally accomplish the same thing by piping the output of tar through a rsh (or something similar), which in turn would copy it to the remote device.
Debugging
A running program exports everything needed to debug it in the cpu directory. Due to its network transparency, you can attach to a program, even if it's running on a separate machine. Simply point your debugger to the proper cpu directory where the process is running, and it will be able to directly manipulate the executable's memory structures through its exported image.
Testing changes
Create a directory and copy some data into it. Now, make the directory multiplexed and cd into it. You will see the contents you copied in. At this point, any changes you make to the contents of this directory can be rolled back by simply logging out (or opening a new terminal window). Other users who access this directory will see its initial contents, regardless of any changes you have made (since it's multiplexed). If you do this, just remember to make a copy of any changes you want to keep, or they will be lost when you log out.
The magic pot
Remember how directories can perform arbitrary actions on files copied into them? We've already seen how this is useful for graphics, audio, and CPU information. Here are a few more ideas on how this can be used: Imagine a directory which will send email. Naturally, the email files would have to have full headers so the mailer would know where to send them, but it could be done. Want to send a message? Just compose it in your favorite text editor and copy it into the mailer directory. Printing can work in the same way. Why have separate commands for printing when you can just copy your files into a printer directory? Tired of using clumsy FTP clients to transfer files? Just make a directory that understands the FTP protocol and map it into a filesystem. Need to transfer a file? Just copy it into (or out of) the ftp directory for your remote host (Plan 9 does this too, by the way). If you wanted to, you could even make a compiler directory. Copy your source files in, then copy the final executable out after the compilation is finished.
Conclusion
Unix introduced the "everything is a file" philosophy more than 30 years ago. This philosophy has served it well over that time, allowing its core functionality to survive the years. However, as new technologies came into existence, many were added without regard to the philosophy which gave Unix one of its greatest strengths. Plan 9 was later introduced as a successor, pulling many of these technologies back into the realm of files and directories. Plan 9 allows applications and drivers to export their own filesystem, allowing external programs to interact with them through a consistent API. OOS hopes to extend this paradigm by extending the abilities of a file to include event-driven properties and multiplexing, and by delivering varied services through a consistent file-centric interface.
wow
I don't know that I understand all of this...but
it sounds cool as crap:)
If you build it, they will come...
Talk is cheap but programming is hard.
Wow, So how can we help?
Well, i was thinking of doing something like this. Maybe i should help this project instead of starting another.
Great Idea... maybe Implement all of that in linux ?
The basics that you describe in this fake :( os are EXCELENT, but I think that instead of going trough the process of creating a whole new OS, it would be better and easier to modify an existing one and drag the changes along the way, something like we have seen on ipv6 and other stuff on linux, as we will keep this for compatibility's sake but it will go on the next stable release.
I really LOVE, (not like) the idea of implementations of everything as containers, specially the part of CPU debugging, since it could really help programmers to be able to debug processes on several machines at once working remotely, * sounds like utopia *
also the SendmailFS and the AudioFS are extremly interesting... of course it would be interesting for instance on the AUDIOFS to se how to handle Threads (fifo maybe ? ) well I guess I'm just divagating but this idea of a super extension of plan 9 is extremly good and should be looked by people like alan cox or linus.... and maybe start with the implementation of /dev/snd/mp3 /dev/snd/wav ... come on we can cat .au files to /dev/dsp I think i wont be all that hard to make this new devices ;)
I even would like to give out this code for a simple mp3 player:
#!/bin/bash
ls | grep mp3 > /tmp/mp3files
cat /tmp/mp3files > /dev/dsp/mp3
#eof
:P :P :P :)
chuby
Re: If you build it, they will come...
> Talk is cheap but programming is hard.
Exactly. But the worst thing, if this does happen, we will end up with YAFLD, or worse, more than one.
Great concept though, but writing code for this will be the most difficult thing. And please, don't pick C++, that will really limit number of possible developers contributing to the code.
What about security?
The one thing overlooked in academic projects (okay so lets not just pick on academics.. commercial software and *gasp* even OSS) is security. Sure a *nix OS beats the crap out of a win32 platform in security but jee-whiz, thats a 30 year old security model. DAC (Discretionary Access Controls) are obsolete. Why in this day and age, are we not in a rush to implement MAC (Mandatory Access Controls)? Why is it that the only _really_ secure OS is a trusted OS (trusted solaris, trusted bsd, hp's virtual vault and argus systems line of pitbull products)? With trusted OS abilities in consumer level OS's and applications the worry of widespread damage from viruses, trojans, buffer overflows, protocol flaws... all the flaws in current software that plague us today would vanish (okay so in reality they would be isolated.. this is called damage-containment, you can't eliminate the problem but you can limit the damage.. to say a single user account, to a single file, to a single application). Anyways, applying the concept of OO to an entire OS is interesting, probably what OS'es should move toward (plan9 & inferno are very cool things) But I would hope that the authors of such a project would have some bloody foresight and not design a modern OS with archiac security conventions.
Great Idea but already Microsoft Exchange Server has OO concept a little bit
Good Article, and Great Idea.
But OOS idea is implemented a little bit in Microsoft Exchange Server. When user access file system, according to the access method the file response diffrently.
When we access from web, it shows html way.
When we access from explorer, or command line, it shows usual file data.
Directory of Folder response diffrently according to access method, too.
Because Exchange is not OS, you may say OOS is different, But I think OOS is not new.
Re: If you build it, they will come...
> Great concept though, but writing code
> for this will be the most difficult
> thing. And please, don't pick C++, that
> will really limit number of possible
> developers contributing to the code.
What do you suggest instead of C++?
I would have thought that C++ would be the ideal language for something like this, relativly well known and object orientated. I'm curious as to what viable alternatives exist.
High time to break free from hierarchical views in filesystems
<P>I mostly agree with the concepts and like the
ideas expressed in this article.</P>
<P>Only I think it's about time to quit the
hierarchical structure of filesystems. Relations
between data is seldom strictly hierarchical. Some
linking mechanism (a-la symlink) can help, but
it's just a kludge.</P>
<P>What we really want is some RDBMS-like file
organizations, where the views are not simply
folder-like, multi-hierarchical containers, but
can be generated by queries; add some friendly
interface to that, and you get the idea.</P>
<P>Examples?</P>
<TT>
cd /documents/images/modified/yesterday/ == cd
/modified/yesterday/documents/images<BR>
cd /documents/TeX/bysize/30K-100K/<BR>
cd /etc/program_x/ == cd /program_x/etc/ <BR>
</TT>
sorry
darn mozilla... :-)
Re: If you build it, they will come...
> What do you suggest instead of C++?
> I would have thought that C++ would be
> the ideal language for something like
> this, relativly well known and object
> orientated. I'm curious as to what
> viable alternatives exist.
Well, there is nothing wrong with C. If you don't believe me, name me some OS's with kernel/system libraries written in C++, that are widely used today. None that *I* can think of. Also, just because most colleges these days teach Java or C++ doesn't mean people have to use or program in these two languages.
containers =^ translators?
Sounds very much like what the HURD already does. http://www.hurd.gnu.org/
Re: High time to break free from hierarchical views in filesystems
> I mostly agree with the
> concepts and like the
> ideas expressed in this
> article.
> Only I think it's about time
> to quit the
> hierarchical structure of filesystems.
> Relations
> between data is seldom strictly
> hierarchical. Some
> linking mechanism (a-la symlink) can
> help, but
> it's just a kludge.
>
> What we really want is some
> RDBMS-like file
> organizations, where the views are not
> simply
> folder-like, multi-hierarchical
> containers, but
> can be generated by queries; add some
> friendly
> interface to that, and you get the
> idea.
I know an example of the ReiserFS future vision. (http://namesys.com )
There is the same suggestions such as a hybride of keyword-based and hierarchical filesystems.
Some queries may look like:
mail/[to/Sandra state/sent body/love]
Re: Great Idea... maybe Implement all of that in linux ?
> I even would like to give out this
> code for a simple mp3 player:
>
> #!/bin/bash
> ls | grep mp3 > /tmp/mp3files
> cat /tmp/mp3files > /dev/dsp/mp3
> #eof
you are talking about something like that ?
mknod mp3 p
cat mp3 | mpg123 - 2>/dev/null &
cat *.mp3 > mp3
if you want to pipe the filename rather than the
files itself:
mknod mp3list p
cat mp3list | mpg123 -@ - 2>/dev/null &
ls *.mp3 > mp3list
reading the manual spares redesigning the os
oh, but i like the idea of such an os..
Re: If you build it, they will come...
>
> If you don't believe me, name me some
> OS's with kernel/system libraries
> written in C++, that are widely used
> today. None that *I* can think of.
> Also, just because most colleges these
> days teach Java or C++ doesn't mean
> people have to use or program in these
> two languages.
That is ridiculous. An OO language would most definetly be best suited for this type of project.
If the system were truely modeled as an OO system this would be nice. No OS today is really object oriented at heart, most are just a set of APIs, lists of functions or requirements that are met. They were not ever visualized as an object model - and that is fine, not everything needs to be object oriented - but its a design choice, there is not reason NOT to create and OO OS.
There is nothing inherentily wrong with any language and no reason not to use something like
C++. If it means older developers don't contribute because they dont want to use a newer language - oh well.
The only real advantage a lower level language like C would give is a speed - and that is only if the developers did not take the time to optimize the C++ code (which would be able to achive similar performance). A nice language that supports OO features allows a greater degree of freedom to programmers dealing with desgin issues. You could do it in assembly language - but why.
Re: If you build it, they will come...
%Well, there is nothing wrong with C.
> If you don't believe me, name me some
> OS's with kernel/system libraries
> written in C++, that are widely used
> today. None that *I* can think of.
> Also, just because most colleges these
> days teach Java or C++ doesn't mean
> people have to use or program in these
> two languages.
You're right, there is nothing wrong with C, and I can't think of any OS's written in C++, but I also can't think of any OS's which are inherently object orientated. Why would using C++ severly limit the number of developers? Surely a system such as OOS would be far easier to program in C++ as opposed to C.
OpenDoc
isnt this a lot like Apple's old OpenDoc Technology that never took off? (were every could be contained w/in anything else) Just this project's scale is much larger b/c it goes down all the way to the OS level instead of ending at a document level.
why not plan9?
What's wrong with already existing Plan9.
It does everything you propose with much
simpler (i.e. better) interface except OO buzzwords included.
Re: What about security?
> Sure a *nix OS beats the crap out of a win32
> platform in security but jee-whiz, thats
> a 30 year old security model.
The way I look at it, security is VERY important depending on what this OS is going to be designed to do. Linux is becoming a very general OS, and Windows always has been. If OOS is going to end up being a server/www os, then security should be the first thing in the mind of the designers. If its going to be a desktop OS, screw security. I do like the idea of chmod 777 / on a desktop system. Look at Windows98, I mean if its screwed up, its the users fault right? Linux security seams to always be out of my way, and I only have to chmod www directorys usually, so more or less both systems work, only one as a multi-user enviroment.
Regardless, security depends on what the OS is going to be designed to do. BSD and Linux work great for server OS's, but fall way short in the GUI aspect of computing. We don't need another server OS unless it has a MAJOR advantage in performance to the ones currently in place. I know I wouldn't invest the time and money in upgrading unless it provided a way to boost speed and reliability.
Executables/libraries/etc
I don't know why containers would require a special function chunk. Why not just have a
directory node which is the functions contained by the container?
Executables could do the same thing. Consider a file where the directory nodes contain things like: main, my_malloc, my_printf, etc.
Of course, it looks alot like Multics in some respects.
Re: If you build it, they will come...
> Why would using C++ severly
> limit the number of developers?
Because no self-respecting developer would ever program in the language mess called C++.
Re: If you build it, they will come...
>
> %Well, there is nothing wrong with C.
>
> % If you don't believe me, name me
> some
> % OS's with kernel/system libraries
> % written in C++, that are widely
> used
> % today. None that *I* can think of.
>
> You're right, there is nothing wrong
> with C, and I can't think of any OS's
> written in C++, but I also can't think
> of any OS's which are inherently object
> orientated.
what about BeOS fully object oriented and programmed in C++.
A great concept ...
I agree that there is currently a lack of consistent
philosophie in popular OSs. Your OOS is a nice
way to complete the work begun by Unix and Plan 9.
This concept includes everything in a hierarchical
structure, as you said. Others contributed that a
RDBMS-like structure could also be useful. Why not
combine these way's to arrange the data stored on a
computer/network?
Compared to oo-languages your OOS is in some
way equivalent to the strictly hierarchical class-tree
of languages without multiple inheritance.
Introducing hard-links or soft-links extends this
conecpt to the analogon of a class tree with multiple
inheritance.
So what about the middle-way using interfaces (as
done in Java). In OOS interfaces could relate to
comon events and a set of necessary
sub-containers.
OOS should IMHO additionally provide a fast
search mechanism to retrieve containers
implementing specific interfaces. This would enable
a RDBMS-like view of the data stored in the whole
file/container tree.
Let's clarify this by an example:
You mentioned the mail file system. Let's define a
Mail interface, which says: every container marked
to be a Mail should have a sub-containers called
Subject, From, To and Body. This way you can'
create a mail by:
mkdir mymail
cd mymail
touch From
touch Subject
touch To
touch Body
cd ..
< mark mymail as implementing the Mail interface >
Now we have a common structure for mails. And can
just type:
mv mymail /<host>/mail/
But we can also use the above mentioned search
engine to find any containers implementing the Mail
interface and we'll get a kind of table view to all
stored mails.
Of course there should be additional criterias :
- regular expressions over the data part of a
container or a some sub-containers
- user attributes
- date
- physical location
- etc...
All these criterias should be composed to a
standardized query language (perhaps a mixture of
regexp, SQL and XPath)
Re: If you build it, they will come...
and remember that OOP help reusing and
distributing, which is one of the most important
issues here. we are talking about free software,
aren't we?? :-)
Re: If you build it, they will come...
What about KDE!
(now, please don't be picky about the
object-orientedness... :-)
Re: If you build it, they will come...
> What about
> HREF="http://www.kde.org"KDE!
> (now, please don't be picky about
> the
> object-orientedness... :-)
>
I'm sorry to say but, KDE is not an OS.
Re: Great Idea... maybe Implement all of that in linux ?
> The basics that you describe in this
> fake :( os are EXCELENT, but I think
> that instead of going trough the process
> of creating a whole new OS, it would be
> better and easier to modify an existing
> one and drag the changes along the way,
This viewpoint was something I had hoped for when I wrote the article. I knew that even if someone writes this OS, getting it accepted as mainstream would be difficult, and if we want to see these ideas in real use, it might be best to add on to an existing OS, like Linux. The only drawback to this would be that much of the simplicity of the design will be lost in the process, as LInux would then have to support multiple models (not that this is necessarily a bad thing).
At one time, I had the idea of writing this OS by taking the Linux kernel, adding a new filesystem to support the ideas mentioned above, and moving some of the drivers to userspace. I quickly realized that I'm an applications programmer, and building an OS was more than I was really capable of doing. Hence this article. My hope is that someone will take some of my ideas and run with them.
Re: What about security?
> The one thing overlooked in academic
> projects (okay so lets not just pick on
> academics.. commercial software and
> *gasp* even OSS) is security.
Security is something I had definitely thought about when writing this article. I didn't really mention it because there are a lot of ways of approaching it. Being able to overload the ls command, for example, makes it realtively simple to hide files. Maybe the administrator account should be able to look at the raw files. Another thing to think about: suppose I am an ordinary user in OOS, and overload the ls command to run some malicious code. If root (or whoever the administrator might be) decides to do a ls in that container, should the code run as root or as the user who created it? In this case, you would want to run it as the lowly user, but the more general case would be the other way around. There's a lot of new security concerns created by OOS's model that should carefully considered before these ideas are put into production. I'll leave the topic open for discussion, since I'm sure there are people out there smarter than me on this issue.
Re: Executables/libraries/etc
> I don't know why containers would
> require a special function chunk. Why
> not just have a
> directory node which is the functions
> contained by the container?
It makes it a little cleaner for implementing event-driven properties and drivers.
> Of course, it looks alot like Multics in some respects.
I've never seen multics in action. Guess it was before my time ;-)
A few notes....
P-code has been around for decades. I remember
a computer made that ran P-code natively in the
70's, and I actually learned to program on an Apple
///, which ran P-code almost exclusively.
As far as the design, there are only two difficulties
I can see. The first is the protocol problem - when
one wants to connect via a non-OOS
computer via ftp, how could it be accomplished
without breaking the underlying metaphor? What if
I want to use rsync -e ssh sometimes?
The second problem is one of speed. It will simply
take an incredible number of cycles to put a button
on the screen using the concept of containers. I
imagine that the process would go:
1. create the button, maybe in a tmp
container.
2. move the button into its container, which is
likely a "box," gtk-style
3. write to it, putting the title on the front
4. somehow link the action of pressing a
button with the appropriate code. In order to keep
code bits from flying everywhere, one would either
have to link in semaphored code to the button
container or have the button open and readable.
The former would be unmaintainable, the latter
would require an inordinate number of open fds to
contemplate even a simple application
5. moving it into the window container, which
outputs a large amount of instructions to the
screen container, which executes the code that
displays the button.
Re: If you build it, they will come...
> If you don't believe me, name me some
> OS's with kernel/system libraries
> written in C++, that are widely used
> today. None that *I* can think of.
Windows NT comes to mind.
Re: If you build it, they will come...
>
> % Great concept though, but writing
> code
> % for this will be the most
> difficult
> % thing. And please, don't pick C++,
> that
> % will really limit number of
> possible
> % developers contributing to the
> code.
>
> What do you suggest instead of C++?
> I would have thought that C++ would be
> the ideal language for something like
> this, relativly well known and object
> orientated. I'm curious as to what
> viable alternatives exist.
Nothing wrong with c++. its just that c++ is soo
difficult to get right, so you will indeed limit the
number of developers, and you will have a difficulty
debugging
try EROS then
EROS is a pure-capabilities OS, which allows the same model to be used for high levels of both mandatory and discretionary security, and is very much object-oriented. In fact, it doesn't even have a filesystem, only objects in a persistent database that is checkpointed every few minutes. It has no files, no root, no "users" to speak of, but a unix implementation on top of EROS, called Dionysix, is in the works. You can find it at http://www.eros-os.org
Also, it's not hard to write an OS -- the concepts are well known and widely taught. It *is* hard to write device drivers. Drivers are the wall of reality you will hit -- they are ugly because hardware is ugly, and it's very hard to keep this ugliness from infecting the rest of your system.
Finally, C++ may not be the best choice of language to implement an object-oriented OS. C++ has various problems with its OO model, mostly all of it stemming from a lack of a standard ABI across platforms. Parametric polymorphism is vitally important, but C++'s implementation in templates strews unreadable (and unportable) garbage all over the symbol table for your trouble. I would personally use Java, despite all its failings as a language, with as many parts compiled to native code as possible, with perhaps a more lightweight bridge than JNI for the parts you need to run native. If that's too much, then there's also Ada or Eiffel, both of which are far better specified than C++ ... when you're doing low-level bit banging as in an OS, you do not want surprises. Finally, you can still do OO in C, check the VFS layer in Solaris (NOT in linux, which uses a union of all FS types) for an example. It's not terribly fun, but it will not surprise you when the linker step comes if you went and did something like reimplement a base class.
Coding, etc.
What codepages and Languages will be supported? This very superficial...
Re: High time to break free from hierarchical views in filesystems
> Only I think it's about time
> to quit the
> hierarchical structure of filesystems.
> Relations
> between data is seldom strictly
> hierarchical. Some
> linking mechanism (a-la symlink) can
> help, but
> it's just a kludge.</P>
>
> <P>What we really want is some
> RDBMS-like file
> organizations
The problem with this is that it becomes increasingly difficult for the end user to navigate such a filesystem. Unfortunately, most end users have a hard enough time dealing with a hiearchical filesystem much less one where they would have to do queries to navigate the thing. On the other hand, it would provide a cool programers interface.
Actually, if I recall the AS/400 does some of this, but not in its directory structure. Essentially a native file under OS/400 is a table in and SQL database, and, I could be wrong but I think that it embeds the querying functionality into the kernel itself.
Re: If you build it, they will come...
> what about BeOS fully object oriented and programmed in C++.
I did mention that it has to be widely used. I would definitely not say that about BEOS.
Re: If you build it, they will come...
%Windows NT comes to mind.
WIN32 API is not in C++. If you want to write software for Microsoft Windows you are in no way limited to using C++. Infact, you can write ANY kind of Windows program only using C. By writing the kernel and system libraries in C++ you limit development ONLY to C++. That's the biggest problem.
Re: If you build it, they will come...
>
> % Why would using C++ severly
> % limit the number of developers?
>
> Because no self-respecting developer
> would ever program in the language mess
> called C++.
>
And what is your basis for this?
Are you saying that in order to be a "self-respecting" programmer that you must use a 4th Generation Language? Why? Some good advice I got once was that the design of what you are doing should determine what language you will use.. need some intensive memory operations, use C. Need to use Object Orientated, choose one of the OO languages. The corralary (sp?) to that is that if you do not have the developers who know the language you want, change...
There is no language that is better than another. They are all good all the time.. it all depend on the developer.
Choose a proper language (Re: If you build it, they will come...)
> What do you suggest instead of C++?
> I would have thought that C++ would be
> the ideal language for something like
> this, relativly well known and object
> orientated. I'm curious as to what
> viable alternatives exist.
C++ is a draft rather than a good language.
Consider using Caml instead, for the library level and higher (not the hardware drivers, I guess). It is a very clean language, very well designed, simple yet powerful, extensible, efficient to program and good performance to run, now well interfaced with the host OSes.
For an quick yet impressive introduction (functional, imperative and object-oriented styles, true polymorphism and the like) see http://caml.inria.fr/FAQ/general-eng.html
http://caml.inria.fr/books-eng.html#ocaml-oreilly (translation of this book is underway)
Re: wow
> I don't know that I understand all of
> this...but
> it sounds cool as crap:)
Ah, come on. I hope it sounds cooler than that :-)
Re: If you build it, they will come...
>
> %Well, there is nothing wrong with C.
>
> % If you don't believe me, name me
> some
> % OS's with kernel/system libraries
> % written in C++, that are widely
> used
> % today. None that *I* can think of.
>
> % Also, just because most colleges
> these
> % days teach Java or C++ doesn't
> mean
> % people have to use or program in
> these
> % two languages.
>
> You're right, there is nothing wrong
> with C, and I can't think of any OS's
> written in C++, but I also can't think
> of any OS's which are inherently object
> orientated. Why would using C++ severly
> limit the number of developers? Surely a
> system such as OOS would be far easier
> to program in C++ as opposed to C.
>
I think that C++ is not simpler than C for a good programmer, and can a beginner contribute to the development of an operating system?
Re: Great Idea but already Microsoft Exchange Server has OO concept a little bit
> When we access from web, it shows html
> way.
> When we access from explorer, or
> command line, it shows usual file data.
I was going to explain what's going on to this
poor soul....but I figured he'd never understand
anyway:-\
Probably interesting, but really needed?
Actually, the idea behind all that might be
interesting, but it somehow seems like an
experimental toy project to me. I cannot really see
why a design like the one described in this article is
really needed. It's probably a nice utopian vision,
but such visions do seldome come true, and even if
they do, then mostly in a modified form that often
differes very much from the original idea...
Re: Choose a proper language (Re: If you build it, they will come...)
>
> % What do you suggest instead of
> C++?
>
> C++ is a draft rather than a good
> language.
>
> Consider using Caml instead, for the
> library level and higher (not the
> hardware drivers, I guess). It is a very
> clean language [...]
I don't know how you can call Caml a clean language. Never in my life have I seen a language with more syntactic sugar.
Some of the syntax and features in C++ aren't necessarily clean either. Then again, I don't have to type a '.' after every operator and a 'let' before every assignment. Caml fails the KISS principle in that regard. What do you suppose the odds of me (and every other programmer) adopting a language like Caml when almost every other programming language in existence uses a much simpler operator syntax?
Is C++ the best we can do?
%And please, don't pick C++, that
> will really limit number of possible
> developers contributing to the code.
Don't use C++? You're right in that implementing
OOS in C++ would limit the number of developers
able to contribute, but C++ might be the only
rational option. Correct me if I'm wrong, but Ada
is more difficult and less popular than C++. Java
requires a VM. C was meant for procedural
programming and might not be suited for OOS.
Python is reputed to be a newbie-friendly
language, but it requires an interpreter.
C++ might be the only sane option, despite the
fact that it cuts off many developers.
what the...
What does this model have to do with evolution?
Seems as though the "native" programming language used to display a text box with "hello world" is interpeted thus slow.
Re: If you build it, they will come...
>
> %Windows NT comes to mind.
>
> WIN32 API is not in C++. If you want
> to write software for Microsoft Windows
> you are in no way limited to using C++.
> Infact, you can write ANY kind of
> Windows program only using C. By
> writing the kernel and system libraries
> in C++ you limit development ONLY to
> C++. That's the biggest problem.
Okay, but pick a language other than assembly and
high level assembly (sometimes called 'C') that
does not have difficulty being called from other
languages.
C++ has a lot of good support for exporting stuff
as C, and that is how people seem to like to use it.
An acceptably machochistic person could write a
set of C wrappers to the BeOS API and make it
programmable in C. Some pretty decent programmers
in Redmond have already done that with NT. For
them it was a bit easier since it was thought
about ahead of time, but they have done it.
Anyhow, your basic statement is flawed: C++ can be
called from several languages. Ada and Haskell
have mechanisms for calling C++ class methods for
example.
Re: If you build it, they will come...
I really don't understand why people are confused about this issue. C++, in my humble opinion, is perfectly suited to the design set forth in this editorial. However, there is a good reason why major OSes do not use C++. It isn't lack of speed, and it isn't because nitpicky developers might take exception to the details of the language. The reason is that C++ *does not* have a standard ABI. It is a draft standard, and as such is not quite the solid development platform that an operating system requires. Every compiler on every platform has a different ABI. Don't believe me? Witness the G++ spectacle with RedHat 7.0. G++ 2.95.2, "2.96," and "2.97" all have different ABIs. Try mixing compilers and versions, and you will soon realize that C++ isn't yet cut out for OSes. One day, when ANSI sets a standard, it will be great, but until then, look elsewhere. Most people seem to choose C, but your mileage may vary.
ObjC?
What about Objective-C?
Count me in. Can we do it to Linux? When can we start?
Observations:
1. You're nuts to try to redo OS's at this late stage. Nobody will go along.
2. I have been fiddling with this concept for the last four years. The fundamental observation is that you are making explicit what was formerly implicit: the "meaning" of data in a file is defined by the functional portion, instead of implied "ascii characters". You have pulled it together in a way that I never have. Yours is much better. It solves a lot of problems. I have serious reservations about the P-code, but nothing better comes to mind, except Java, maybe.
3. At this point, in my poor opinion, the most effective strategy would be a graft onto Linux, and grow from there. If this is as powerful and user/programmer friendly as I think it is, it will catch on with a significant number of users, and from there, there are lots of options.
4. Please count me in. I want to help.
5. I have noticed that every great project has a benevolent dictator at the top. What's-his-name Torvalds being a case in point. Do you have such a person in mind? Are you such a person? Can you find such a person? Long term insanity is probably a necessity here.
6. I am adopting this as my personally preferred programming paradigm as of right now. Do I owe you anything for copyright use?
Re: High time to break free from hierarchical views in filesystems
Linux has facility for mounting arbitrary userfses; why not write, or
at least fully specify, your idea?
Apparently Freshmeat will, based on the fact of this article, publish unfinished and vapor conversation pieces: why not submit projects that are simply in spec form?
>
> Examples?
>
>
> cd
> /documents/images/modified/yesterday/
> cd
> /documents/TeX/bysize/30K-100K/
find /documents -name '*.gif' -mtime -2 -ls
find /documents -name '*.tex' -size +30k -size -100k
and if you need these things indexed, writing a periodic thing
that makes you an index of them ahead of time is not that
tough.
Great concept, but please do not reinvent the wheel.
For example, you could maybe reuse the work done at the HURD project. I think that the concept of a translator (http://www.gnu.org/software/hurd/whatis/translator.html) implemented in this system might well be something interesting to you : http://www.gnu.org/software/hurd/whatis/translator.html (http://www.gnu.org/software/hurd/whatis/translator.html). In addition, you would eventually benefit a micro kernel foundation.
Wrappers of all kind would make possible the building of your new namespace by allowing to use existing software by talking to them with your new way. Then, one distant day in a hazy future, when enough developers will have toyed with the wrapper implementation of your concept, maybe they will begin to contribute implementations of the lower levels of your model.
I particularly love the idea of a universal namespace, but if you really want to make it happen, you stand better chances by using existing foundations. I think the core value added of your idea resides in the namespace. You don’t actually need to implement anything else to make it valuable : just design the best namespace in the world, make it usable by building wrappers to control other things with that language, and you will soon find out that people will build things that natively understand your new way to name things.
Re: Count me in. Can we do it to Linux? When can we start?
1. Heheh. I'm not actually writing this OS. I simply presented it as a hypothetical possibility so that others might take ideas from it.
2. Java with a JIT compiler is pretty much the same thing as pcode. I just don't want to limit users to using Java.
3. I someday hope to see some of these ideas grafted onto Linux, as that's the OS I use at home. Hell, I think it would be cool if I could just export my /dev directory across NFS and have it work the way it logically should. I've tried this, but it didn't work. Maybe another networked filesystem would work here.
4. I'm not sure there's much to count you in on at this point.
5. Oh dear god! You're not seriously thinking about making me a benevolent dictator, are you? I have a really bad habit about not finishing what I start. I have enough unfinished projects to keep me busy for a lifetime. And I'm not sure who I would choose to do it for me.
6. You're free to use this article however you want. I wrote it to generate ideas, not for money. Consider it in the public domain.
Re: If you build it, they will come...
>
> %
> % % Why would using C++ severly
> % % limit the number of developers?
> %
> % Because no self-respecting
> developer
> % would ever program in the language
> mess
> % called C++.
> %
>
>
> And what is your basis for this?
> Are you saying that in order to be a
> "self-respecting" programmer
> that you must use a 4th Generation
> Language? Why? Some good advice I got
> once was that the design of what you are
> doing should determine what language you
> will use.. need some intensive memory
> operations, use C. Need to use Object
> Orientated, choose one of the OO
> languages. The corralary (sp?) to that
> is that if you do not have the
> developers who know the language you
> want, change...
>
> There is no language that is better
> than another. They are all good all the
> time.. it all depend on the developer.
There is actually a good reason why C++ isn't necessarily a good choice; it's not very efficient. I personally don't think that any high-level language should be used to write an OS. Since everything else to be performed by the computer must be done in intimate conjunction with the OS, it should be as efficient as possible; even C isn't quite up to par for this sort of operation. IMHO, asm is the only real choice.
Now, having said that, there's really no reason why development can't begin with whatever programming language you want, provided it makes what you're doing for the module in question easier. Once the system is together, you can optimize to your heart's content by looking at the system statistics and moving on from there; in my experience, the only way to get a really true idea of how well a system will work is to make it, then watch it in action. Even the best theorist misses things.
Hrm... also, I don't know that you need to use an object-oriented language to make an object-oriented program. Object-orientation is a development idiom, not a hard-and-fast thing; you can write an object-oriented datastructure (which is what the OOS would be, primarily) in any language, from assembly on up. The program just has to have mannerisms for instantiation, inheritance, and so forth; this isn't really all that hard to achieve, and using something like C++ or Java to give you their limited views on the subject is fine for budgetted application development, but when designing an entirely new OS, you have to have the control to approach the situation from whatever angle you so choose. C++ doesn't give you the kind of flexibility you need; Java is ten times worse.
So anyway. Above any beyond that... I really love the OS concept; I've been looking at the possibility for a while, myself. For other neat resources, look at Unununium, on www.sourceforge.org; it heralds itself as an 'organic OS', with no kernel whatsoever, and with every cell dynamically (un)loadable. AND it's written entirely in asm. It might be a great thing to look at, to underpin this project. It also reminds me of the KDE/GNOME sort of implementation, based on OMG's CORBA specs.
Anyway. It's a great idea, and I'd love to develop it further. If anyone's interested, give me an e-mail.
Re: High time to break free from hierarchical views in filesystems
Okay. So here's what I've been bouncing my head off of for the last few months:
Do users ever need to encounter a file system not of native design?
*duck* Okay, here goes. So what is the filesystem *for*? To help the user/administrator find the information they need. So let's look at the OS as though every aspect of it is a dynamically loadable/unloadable module, kind of like Unununium; however, we should set it up so that each is also an object, in a polymorphic inheritance tree (single- or multi-, doesn't matter). One of these modules should probably be a browser of sorts, either console or GUI, allowing you to run around the object tree to see what you need to see, right from the root object (System? World? Depends, really). Make it dynamically reparentable, so you can have the whole thing revolve around the Network object, or around the System object, or some other similarly important object (the point being that the actual physical constraints of the computer are controlled independently by a tree on the heirarchy). In order to do this right, we'd have to twist the concept of object orientation immeasurably, which I'm now in the process of trying to do (with great difficulty; OO works well). But the basic need here is that we have a decentralized internal model of our computer, allowing us to use it as though all of the computers on our subnet were one computer, or as though our one computer were many (parallel VMs, each with an independant environment. Like Java). Also, we'd probably have to import the instantiation model so it's actually part of the inheritance tree (so an instance of an object is viewed as no different than a subclass of an object; it's really not that grand a change).
Hrm... I hope I made that clear enough to follow. I'm still a little rough around the edges about it, because it really exists only in my mind (so far as I know). Anyway, with this, we can now apply some of our preserved OO concepts to make the system more dynamic. Let's say we make File a subclass of BlockDevice, HardDrive subclasses of File. Finally, allow File to contain zero or more references to File objects. Voila! We have a situation (if I got it right, that is) where any file can contain any other file, regardless of what the underlying hardware seems to be; it also makes a file format really no different from a file *system*, allowing you to drop a file, intact, onto a raw hard drive, and use the whole hard drive with that access method (probably improving access times for databases, among other things); you could also enter the ISO for a cd you ripped the other day, and use it as-is.
Anyway, to finish up... this removes the need for a visible file system to contain binaries, because the OS would take care of those as modules in itself (which is really the way it should be, *I* think). The OS would have its own little secret place for files of no interest to the average user (though still available for the developer through the system tree), and applications that deal with data can handle the data in whatever way makes sense to them; if the user wants a document from a word processor, then, she would enter the word processor's module space and grab the document from there, where there would potentially be a deep file heirarchy, if she wanted to make one. For the average user, I think this would make by far the most sense; you want a spreadsheet, open the spreadsheet's data space, you want something else, open that thing's data space. You can *gasp* actually use logic to find things on your system, instead of needle-in-a-haystack file mask searches, or wandering about in the e-desert alone with no water.
...bah. It works in my head.
> <P>I mostly agree with the
> concepts and like the
> ideas expressed in this
> article.</P>
> <P>Only I think it's about time
> to quit the
> hierarchical structure of filesystems.
> Relations
> between data is seldom strictly
> hierarchical. Some
> linking mechanism (a-la symlink) can
> help, but
> it's just a kludge.</P>
>
> <P>What we really want is some
> RDBMS-like file
> organizations, where the views are not
> simply
> folder-like, multi-hierarchical
> containers, but
> can be generated by queries; add some
> friendly
> interface to that, and you get the
> idea.</P>
>
> <P>Examples?</P>
>
> <TT>
> cd
> /documents/images/modified/yesterday/ ==
> cd
%
> /modified/yesterday/documents/images<BR>
> cd
> /documents/TeX/bysize/30K-100K/<BR>
> cd /etc/program_x/ == cd
> /program_x/etc/ <BR>
> </TT>
anything new?
I think I haven't found any completely new concept in this article. Multiplexed filesystem reminds me of Plan 9. Containers remind me of GnuHurd. P-code reminds me of Java...
I don't think it's generally a bad idea to speculate (possibly dream :) of a new OS. Just ... please, post it somewhere else. I remember last year in my school we all had to write an article like this and also implement some features of our designed OS on some microkernel.
Maybe you could make your article as a description of all interesting concepts in OS design (implemented in some system or not), not just pick one of them for each task performed by operating systems.
---
R0B0
Re: Count me in. Can we do it to Linux? When can we start?
2. A common base language is necessary for the functional portion of a container. In my poor opinion, P-code is not a first choice because it is not widely used. Java is not a first choice because it is proprietary. If C# is proprietary, it has the same drawback. HTML is not flexible enough. How about XML?
3. Linux seemed like the best place to start, because it is widely used, and the source code is available. Please pardon my ignorance: what is NFS?
5. I am not trying to make you a benevolent dictator of this topic. I was sorta hoping you would do that by yourself. :-) By the way, you would not be primarily responsible for developing much. You could develop if you wanted to, but that would not be the point.
As benevolent dictator, your position is more like that of a judge. You would be mainly responsible for approving standards and changes implemented by others, and would be a point of contact. Integrity is a big issue here. So are other things, like the ability to politely suggest to helpful sorts that their reasoning is sound, but "doing everything in base 12" is not a priority just yet.
Only enough projects to keep you busy for a lifetime? Sounds like you have a lot of free time, then. Most people in this area have enough projects for two or three lifetimes.
6. Thank you for the release.
Re: If you build it, they will come...
> What do you suggest instead of C++?
> I would have thought that C++ would be
> the ideal language for something like
> this, relativly well known and object
> orientated. I'm curious as to what
> viable alternatives exist.
Objective C - the way NeXT (and now GnuStep and MacOS X) does; there are things you can do with those OSes that would never have been feasible with C++ as the core.
And don't throw the "people don't know it" BS at me - any competent programmer can learn any sane language in days. Especially a nice, simple, coherently designed one like ObjC. And an incompetent programmer should not be writing OSes.
Breaks KISS principle
People once harassed Linus about why Linux was not a multi-threaded message passing operating system like what was in vogue when he started it in 1991. Linus always replied that it was more important to keep it simple than to throw everything into the pot.
The problem with this design is that it breaks KISS (Keep It Simple, Stupid). This design appears to be extremely complex, with no real benefits for most users. While I think there are designs that are complex and have real reasons to be complex (the Reiser filesystem, for example), this does not appear to be one of them.
Re: Breaks KISS principle
Too damn complex. If this is going to be widely
used after say like 30 years, well it's ok. Don't
forget to use TAI64 as time format so we won't have any lame
year 2k, year 2034/2036(?) problems. See this page (http://cr.yp.to/y2k.html)
for more info :)
Speaking without a Lithp
Dear Mr. Williams,
I don't know much about the design of Operating Systems (only one semester of UMass/Amherst Intro to OSs behind my belt), but what do you think about the culmination of your idea (or something similar, as it seems others have been on somewhat the same wavelength) through Common Lisp?
Basically what I mean to ask is, why is everyone still talking about Java or C/C++ -based languages for building a next-generation OS? I think that with the power of expression in Lisp (CLOS, etc.), you can accomplish your object abstractions writing a minimum amount of code and with binaries that are faster than Java and C++ (http://www-aig.jpl.nasa.gov/public/home/gat/lisp-study.html)
Any criticism to this comment is very welcome.
P.S.
I know -- Symbolics is dead...but I'd like to think
that it was because they were ahead of their time.
Re: What about security?
> Security is something I had definitely
> thought about when writing this article.
> I didn't really mention it because
> there are a lot of ways of approaching
> it. Being able to overload the ls
> command, for example, makes it
> realtively simple to hide files. Maybe
> the administrator account should be able
> to look at the raw files. Another thing
> to think about: suppose I am an ordinary
> user in OOS, and overload the ls command
> to run some malicious code. If root (or
> whoever the administrator might be)
> decides to do a ls in that container,
> should the code run as root or as the
> user who created it? In this case, you
> would want to run it as the lowly user,
> but the more general case would be the
> other way around. There's a lot of new
> security concerns created by OOS's model
> that should carefully considered before
> these ideas are put into production.
> I'll leave the topic open for
> discussion, since I'm sure there are
> people out there smarter than me on this
> issue.
I would think a good way of approaching security for this model would be to have a security container as the top-most container for a given system. As such, all containers would inherit security methods and attributes (such as usernames and associated passwords).
Then, anyone implementing a container's code would take advantage of the security container's methods as required.
I tend to look at the issue of security for something like this as a 'filter'... information must pass through the security filter before getting to the other side.
Hmm... you'd probably want another container that acts as a token .. something that may automatically answer username/password requests from the security container. This container would be created within the workspace container when you logged in (and as such, would disappear when you logged out, or the workspace container became deleted). Otherwise, you run the risk of being pestered by a lot of username/password requests.
Similarly, individual containers may have a container of other containers (perhaps a permissions container) describing users (or groups of users) permitted to work with the container. As such, before you could work with a particular container in your workspace, your username must exist within that container's permissions container.
Re: What about security?
> The way I look at it, security is VERY
> important depending on what this OS is
> going to be designed to do. Linux is
> becoming a very general OS, and Windows
> always has been. If OOS is going to end
> up being a server/www os, then security
> should be the first thing in the mind of
> the designers. If its going to be a
> desktop OS, screw security.
Think about that statement for a second. If your desktop
OS is insecure, it becomes my problem! When some
3133t d00d hacks your box and uses it to run ddos against
my website, that is a problem. When some bozo uses the
VB virus kit to create a "new" outlook worm, and the resulting
flow of emails crashes my ISPs mail servers, that is a problem.
Do you leave the front door to your house and your gun
cabinet unlocked too? When some psycho walks in, steals
your .45 and starts blowing people away at McDonalds, you're
going to realize that you probably should have thought about
security a little bit more...
Overloaded man pages
Heh, I can see the man page for 'cp' now...
NAME: cp - does everything
SYNOPSIS cp [option] source destination
DESCRIPTION : makes things go, depending on what the source and destination is. It will perform backup operations on /<host>/dev/tape, print operations on /host/dev/lp0, recompile your kernel in /host/boot, start and shut down system services in /host/etc/init.d, give you system information in /host/cpu, and copy files (unless otherwise indicated)
So I guess there should be some way to make system interfaces self-documenting (like in emacs)? There needs to be some way to let the user know what's going to happen, if the same command will make the computer do so many different things.
It'll be nice when some system like this finally comes around... I suspect it will be built on top of current POSIX systems for the time being though, somewhat like a mix between PVM and JVM. This will let people play with the system in a window while still having all their current apps and hardware supported. Then it could move to run on HURD (either on Bochs or directly on the metal) once it matures and more device support comes out for it.
This article has really gotten me excited, thanks!
Re: Overloaded man pages
I think it should most certainly be writtin in Microsoft Visual Basic.......yeah right! Seriously, I've read all of the above arguments, and I swing both ways.
Inferno/GNU-Hurd
Hi, I've read the article, and it remembers me the Inferno OS, take a look at http://www.vitanuova.com/inferno/index.html, it has most of the features described here, and many more..., yes.. I know what are you thinking about... It is not open source but it is very well designed OS, from the authors of unix and plan 9. By the way, it has it's own language (limbo), so you don't have to be discusing about the c/c++/java?/shell script/ holy god language.
The other, open source, alternative to play with to implement all the features described here is the GNU-HURD OS, it is work in progress but it is maturing fast, and is posible to extend the "everything is a file" in a very nice way using the translators concept.
I believe in this vision
Yes, I have been pondering about this very same
idea for my self too!
In my vision I had this little idea, that programs
could be more integrated and modulized so that
there actually wouldn't be invidual programs, but
just few, like <i>View, Edit,</i> etc., so that if
one person writes a module that reads .png:s, the
whole system can do that, and if one doesn't like
the default interface e.g. for View, they could
select another module for the interface..
The reason behind this would be mostly code
reusability. Thu this model for os would definetly
bring new porting issues etc.
just my 2½ pennies..
Also check out:
http://www.gzigzag.org/
device drivers created from ADT?
So how nice would it be able to feed the whitepapers for a device into an OS in some sort of OO datatype and automagically have a driver built?
Re: device drivers created from ADT?
> So how nice would it be (cut)able(/cut) to feed the
> whitepapers for a device into an OS in
> some sort of OO datatype and
> automagically have a driver built?
How nice would it be to construct a valid sentence?
Re: Probably interesting, but really needed?
> Actually, the idea behind all that might
> be
> interesting, but it somehow seems like
> an
> experimental toy project to me. I
> cannot really see
> why a design like the one described in
> this article is
> really needed. It's probably a nice
> utopian vision,
> but such visions do seldome come true,
> and even if
> they do, then mostly in a modified
> form that often
> differes very much from the original
> idea...
>
Therefore we should completely give up. *sarcastic*
FS
>>A container has three principle parts: A file part, a directory part, and a functional part. The file part acts just like a file in traditional operating systems. You can open it, read and write to it, seek within it, etc.<<
why would you want to do that. Having to make your programs read and parse configuration files is boring and so not OO. ooze should make the file part natively some new OO database, or at least XML or something, wouldn't it be so much more fun to get conf data with something like "/confdir/conffile.confvariable" than searching for confvariable= and then retreiving the variable.
regular text should be reserved only for stuff intended for human reading, no wait we can change the way we think to OO. modern literature is so procedural we need to OOify it.
well I was serious about the first paragraph. I hate parsing conf files, it seriously should natively store conf data in objects
stdin, stdout, etc
>>Starting a process through the cpu container causes it to start in its own session. This means its stdin, stdout, and stderr will be redirected to/from never-never land, which is good for daemonizing a process but
bad for running interactive terminal programs.<<
No way, you should be able to change all of a processes file descriptors whenever you feel like, because the FDs will be links within the processes directory tree. so then you could conveiniently move processes from one terminal to another or optionally to /dev/null. And the way you could keep the app from freaking out when you do this is by opening fds relationaly. So if you move /dev/pty1/out to /dev/pty2/out and then the program desides it wants to open stderr it should open ./stderr; see no confusion.
Re: High time to break free from hierarchical views in filesystems
>
> % I mostly agree with the
> % concepts and like the
> % ideas expressed in this
> % article.
> % Only I think it's about time
> % to quit the
> % hierarchical structure of
> filesystems.
> % Relations
> % between data is seldom strictly
> % hierarchical. Some
> % linking mechanism (a-la symlink)
> can
> % help, but
> % it's just a kludge.
> %
> % What we really want is some
> % RDBMS-like file
> % organizations, where the views are
> not
> % simply
> % folder-like, multi-hierarchical
> % containers, but
> % can be generated by queries; add
> some
> % friendly
> % interface to that, and you get the
> % idea.
>
>
> I know an example of the ReiserFS
> future vision. (http://namesys.com )
> There is the same suggestions such as
> a hybride of keyword-based and
> hierarchical filesystems.
> Some queries may look like:
>
> mail/[to/Sandra state/sent
> body/love]
>
>
>
>
I agree, a solid rdbms should be the root
for a new operating system.
think of :
SELECT * FROM volume1 WHERE filename LIKE '%.DOC'
or...
SELECT count() FROM volume1 WHERE filesize>666666
other advantages:
Links should be way easy to implement and maintain
(have you ever used TheBrain? check out thebrain.com, an hyperlink admin and retrieval app)
A filesystem such as this could be greatly
optimized, and journaling filesystems way easier to implement than the current linux journaling filesystem.
there may be people that says
An OS shouldn't depend on a SQL engine for its filesystem... okay... but remember, it's object oriented... so you can have a hierarchy class...
like:
filesystem
|
+---- unix_like_filesystem
|
+----- object_oriented_filesystem
|
+---- journaling_filesystem
what do you think ?
Re: Count me in. Can we do it to Linux? When can we start?
> 2. A common base language is necessary
> for the functional portion of a
> container. In my poor opinion, P-code
> is not a first choice because it is not
> widely used. Java is not a first choice
> because it is proprietary. If C# is
> proprietary, it has the same drawback.
> HTML is not flexible enough. How about
> XML?
>
> 3. Linux seemed like the best place
> to start, because it is widely used, and
> the source code is available. Please
> pardon my ignorance: what is NFS?
>
> 5. I am not trying to make you a
> benevolent dictator of this topic. I
> was sorta hoping you would do that by
> yourself. :-) By the way, you would
> not be primarily responsible for
> developing much. You could develop if
> you wanted to, but that would not be the
> point.
> As benevolent dictator, your
> position is more like that of a judge.
> You would be mainly responsible for
> approving standards and changes
> implemented by others, and would be a
> point of contact. Integrity is a big
> issue here. So are other things, like
> the ability to politely suggest to
> helpful sorts that their reasoning is
> sound, but "doing everything in base 12"
> is not a priority just yet.
> Only enough projects to keep you
> busy for a lifetime? Sounds like you
> have a lot of free time, then. Most
> people in this area have enough projects
> for two or three lifetimes.
>
> 6. Thank you for the release.
>
>
XML ?
are you joking ?
Re: If you build it, they will come...
>
> %
> % %
> % % % Why would using C++ severly
> % % % limit the number of
> developers?
> % %
> % % Because no self-respecting
> % developer
> % % would ever program in the
> language
> % mess
> % % called C++.
> % %
> %
> %
> % And what is your basis for this?
> % Are you saying that in order to be
> a
> % "self-respecting"
> programmer
> % that you must use a 4th Generation
> % Language? Why? Some good advice I
> got
> % once was that the design of what you
> are
> % doing should determine what language
> you
> % will use.. need some intensive
> memory
> % operations, use C. Need to use
> Object
> % Orientated, choose one of the OO
> % languages. The corralary (sp?) to
> that
> % is that if you do not have the
> % developers who know the language
> you
> % want, change...
> %
> % There is no language that is
> better
> % than another. They are all good all
> the
> % time.. it all depend on the
> developer.
>
>
> There is actually a good reason why
> C++ isn't necessarily a good choice;
> it's not very efficient. I personally
> don't think that any high-level language
> should be used to write an OS. Since
> everything else to be performed by the
> computer must be done in intimate
> conjunction with the OS, it should be as
> efficient as possible; even C isn't
> quite up to par for this sort of
> operation. IMHO, asm is the only real
> choice.
>
> Now, having said that, there's really
> no reason why development can't begin
> with whatever programming language you
> want, provided it makes what you're
> doing for the module in question easier.
> Once the system is together, you can
> optimize to your heart's content by
> looking at the system statistics and
> moving on from there; in my experience,
> the only way to get a really true idea
> of how well a system will work is to
> make it, then watch it in action. Even
> the best theorist misses things.
>
> Hrm... also, I don't know that you
> need to use an object-oriented language
> to make an object-oriented program.
> Object-orientation is a development
> idiom, not a hard-and-fast thing; you
> can write an object-oriented
> datastructure (which is what the OOS
> would be, primarily) in any language,
> from assembly on up. The program just
> has to have mannerisms for
> instantiation, inheritance, and so
> forth; this isn't really all that hard
> to achieve, and using something like C++
> or Java to give you their limited views
> on the subject is fine for budgetted
> application development, but when
> designing an entirely new OS, you have
> to have the control to approach the
> situation from whatever angle you so
> choose. C++ doesn't give you the kind
> of flexibility you need; Java is ten
> times worse.
>
> So anyway. Above any beyond that... I
> really love the OS concept; I've been
> looking at the possibility for a while,
> myself. For other neat resources, look
> at Unununium, on www.sourceforge.org; it
> heralds itself as an 'organic OS', with
> no kernel whatsoever, and with every
> cell dynamically (un)loadable. AND it's
> written entirely in asm. It might be a
> great thing to look at, to underpin this
> project. It also reminds me of the
> KDE/GNOME sort of implementation, based
> on OMG's CORBA specs.
>
> Anyway. It's a great idea, and I'd
> love to develop it further. If anyone's
> interested, give me an e-mail.
the fact that it is assembler-based makes it
very difficult to port...
but I will have a look at it anyway...
is there more info available ?
Some Questions
I have some questions about programming-languages:
1) Could someone give me a list of the common languages (C, Asm, Basic, etc.) sorted by speed....like:
1. Asm (Fastest/hardware nearest)
2. C
3. C++
4. Java
...
103. Basic (Slowest)
---------------------
2) Could someone give me a list of the common OS's (Linux, Windows, FreeBSD, etc.) and in which language they are programmed? Like:
Windows NT - C
Linux - Asm
Windows 98 - Basic
(i dunno if winNT really is programmed in C or if linux is programmed in Asm...this is just an example)
-------------
thx in advance for answering my questions :-)
-rokyo-
Re: Some Questions
> 1) Could someone give me a list of the
> common languages (C, Asm, Basic, etc.)
> sorted by speed?
This is a dumb question for a number of reasons :). The languages really vary in how much optimization is left to the programmer; in asm, all optimization is left to the programmer, since there's no automated optimizer. C and C++ are both optimized by the compiler and have provisions for overriding these optimizations (most of which aren't very good). A language like Java has a JVM which basically sits there and optimizes your program for you while it's running (among other things), so the programmer doesn't really have any optimization to do. This makes programming easy, but if something runs slowly, you have no choice but to change the algorithm.
Interpreted languages are completely different - since there isn't really such thing as optimizing an interpreted program at the language level, it really is the responsibility of the interpreter to keep things running quickly. Oddly enough, most interpreted lanugages (Perl, Python, Ruby, Tcl) all run at very similar, very slow speeds. Since speed-critical code is rarely run in these languages, it isn't a problem.
> 2) Could someone give me a list of the
> common OS's (Linux, Windows, FreeBSD,
> etc.) and in which language they are
> programmed?
All the Unices (Linux, BSD, Solaris, etc.) are written in C, more or less by definition. Small bits of startup code are written in ASM.
NT is, contrary to popular belief, written in C at the kernel level. However, to actually run a program, you need the Win32 interfaces, which are written in C++.
A few mostly-trivial OSs are written in C++. For example, the XOSL bootloader has a built-in kernel written in C++. (It's on my computer, that's the only reason I know about it.) Offhand, I can't think of any others.
Mac OS 9 is a bizarre mix. The bootloader is OpenBoot (interpreted Forth, I think), which starts up the main OS, written in C. But there are still bits of code in there dating to 1984 written in 68000 assembler and Pascal. User programs are in Metrowerks C mostly, but big apps are C++ (like everywhere else).
Mac OS X is an interesting case. Hardware communication on OS X goes through the Mach microkernel, which is written in C and assembler. The only "program" the Mach kernel runs is a FreeBSD-based Unix kernel, written entirely in C, that uses Mach to talk to the hardware. On top of the BSD kernel are three sets of interface libraries: Carbon (mixed C++, C and Pascal - yuck), Cocoa (Java) and Posix (C). Please correct me if I'm wrong, I haven't actually used OS X yet.
Re: High time to break free from hierarchical views in filesystems
>
> Linux has facility for mounting
> arbitrary userfses; why not write, or
> at least fully specify, your idea?
>
> Apparently Freshmeat will, based on
> the fact of this article, publish
> unfinished and vapor conversation
> pieces: why not submit projects that
> are simply in spec form?
%
sometimes ideas are just that, ideas, and tiny sparks of thought may lead to bigger ones
what you call "vapor" is what I call brainstorming
well, not exactly since it is an open fourm, I agree on that, (there was a guy who wanted to write an OS in XML because he heard it was used to port stuff.... don't laugh now)
brainstorming is not bad, who said everyone is absolutely right and that their idea is the best and it must be the one to develop ?
there would be millions of operating systems if the world worked that way.
brainstorming is great, brainstorming is good, and most of all brainstorming is fun.
Re: Some Questions
Perl is NOT an interpreted language...it is a compile-on-run language. It definately is a syntactic mess; I wouldn't recommend anyone learn on it. but again Perl is NOT an interpreted language and does NOT suffer from the defects of interpreted languages.
Future Vision
First off, although you can use C to program in OO, it doesn't allow for some of the features that are present in C++. I think that C++ is the language most suited to developing this OS. For those saying that it slower than C and ASM, may I point out that with the speed of computers increasing the way they are, the difference is insignificant. The design and implementation are much more important issues. Although the C++ standards are inconsisent at the moment the OS is that of a future vision. In a couple of years time the C++ standard will have stablised. Who nows in a years time they're maybe a new OO language. From the comments made, I feel people are getting to caught up with the present situation. The idea put foward in this article is one of the future.
Secondly, what is the problem with developing a completely new OS? I think the only way in which it can be implement properly is to develop a new OS. I don't think is possible that linux could be modified into this implementation and if it did it would no longer be linux. Linux may be great but I think this future OS is beyond any OS currently out (althought HURD and PLAN 9 do look promising I haven't look into these two OSes in detail).
Anyway, I think its an interesting topic which deserves more discussion. Is they're a newsgroup on this topic. If so, please tell me.
Re: Some Questions
> C and C++ are both optimized by the compiler and have provisions for
> overriding these optimizations (most of which aren't very good).
Depends what you define by not very good. I've seen ~5 times speed
improvement with optimisations...
> A language like Java has a JVM which basically sits there and
> optimizes your program for you while it's running (among other
> things), so the programmer doesn't really have any optimization to
> do.
LOL. That's really a good one. Java is just as optimisable as C++.
If you're going for byte-code compiled Java that's certainly the case.
You could assume that the JVM was pants, and optimise for the JVM but
that would be a tad foolish methinks. At the end of the day,
irrespective of the language, it's the programmer who is able to do
the optimisations (ignoring slightly the concept of virtual machines,
as they merely sit in the middle to make your life harder).
> Interpreted languages are completely different - since there isn't
> really such thing as optimizing an interpreted program at the
> language level, it really is the responsibility of the interpreter
> to keep things running quickly.
Oh really. Describe how you think an interpreter works. Do you think
it basically executes the lines of code you've written? If it does
then you can optimise it just fine. Some would consider machine code
to be interpreted...
Re: If you build it, they will come...
% the fact that it is assembler-based
> makes it
> very difficult to port...
>
> but I will have a look at it
> anyway...
>
> is there more info available ?
Yep. The URL is uuu.sourceforge.net. It is fairly difficult to port on a one-to-one basis, but I can't see that it would be all that hard to develop an assembly compiler that's somewhat platform-compatible. Besides, the code is simple enough that a module could be rewritten for another architecture without serious difficulty.
Nice idea, but complicated for the end user
The idea of OOS is nice, but it is difficult to use. Many users suffer from complicated and inconsistent user interfaces. The big drawback here is that the same command can do completely different things when used in different contexts. This does not make for usability, but for confusion. However, if you start creating spcial commands for every possible case you do get the same as todays UNIX console and can continue that game for eternety.
I think the direction is the right one, but not the particular way chosen: the computer has become a tool like many other tools we use (pencils, paper, etc.). But it does not behave as such, even after over 20 years of trying. It rather is a set of components (hard- and software) which do not correctly interact with each other and with the user. To fix this problem the start must be at the lowest level possible. As redefining hardware standards is insane the lowest level actually possible is the OS level. This means that the software must present itself as a whole and not primarily as simple components (programs, applications or whatever). Of cause it must consist of modules internally, but the inexperienced users must not be confused by this. Therefore the internals must not be visible directly, but not be completely hidden. There must be the possibility to
A) learn about the internal workings of the software at any time and with unlimited detail
B) access the inner workings of the OS at any time given to correct things that have gone wrong or to extend the OS itself.
There must be an UI that can do excactly that. A console isn't appropriate for this - especially the way you proposed. I have carried similar ideas in my mind and have come up with what I call ChallengeOS. And yes - I am planning to implement it. Its software is fully modular (no more programs - only threads and modules!) and fully scriptable (a compiler/VM suite is actually neccesary part of the inner workings of the OS). If you want more information feel free to visit http://challangeos.sourceforge.net. I have placed quite a bunch of information there, but none comes actually to the point, I'm afraid.
What about Oberon?
Wasn't Niklaus Wirth Oberon System an OOS? They
even build a machine for it!
Re: If you build it, they will come...
> % the fact that it is assembler-based
> % makes it
> % very difficult to port...
> %
> % but I will have a look at it
> % anyway...
> %
> % is there more info available ?
>
> Yep. The URL is uuu.sourceforge.net.
> It is fairly difficult to port on a
> one-to-one basis, but I can't see that
> it would be all that hard to develop an
> assembly compiler that's somewhat
> platform-compatible. Besides, the code
> is simple enough that a module could be
> rewritten for another architecture
> without serious difficulty.
No. Assembley is by definition non-portable, since
it is also by definition exactly what the CPU
executes. You want something low-level yet
portable? Use C.
Portable assembler is an oxymoron.
Re: If you build it, they will come...
> I really don't understand why people are
> confused about this issue. C++, in my
> humble opinion, is perfectly suited to
> the design set forth in this editorial.
> However, there is a good reason why
> major OSes do not use C++. It isn't
> lack of speed, and it isn't because
> nitpicky developers might take exception
> to the details of the language. The
> reason is that C++ *does not* have a
> standard ABI. It is a draft standard,
> and as such is not quite the solid
> development platform that an operating
> system requires. Every compiler on
> every platform has a different ABI.
> Don't believe me? Witness the G++
> spectacle with RedHat 7.0. G++ 2.95.2,
> "2.96," and "2.97" all have different
> ABIs. Try mixing compilers and
> versions, and you will soon realize that
> C++ isn't yet cut out for OSes. One
> day, when ANSI sets a standard, it will
> be great, but until then, look
> elsewhere. Most people seem to choose
> C, but your mileage may vary.
>
It is true that C++ does not have a standard ABI,
but it *is* an approved standard!
Oberon system, Oberon-2 language - that's what you need
Oberon system, Oberon-2 language - that's what you need. Niclaus Wirth did excellent work for development of Object-Oriented systems and OO languages. Modula-2 is excellent language, Oberon is superior.
I don't know what parts of his work can be qualified as "open source", but I strongly encourage you to try Oberon (as it is available for Linux nowdays)
Re: What about Oberon?
> Wasn't Niklaus Wirth Oberon System an
> OOS? They
> even build a machine for it!
And now Oberon (language) is ported to most systems - Linux, Windows, etc. So, you don't have to buy/have specific hardware to run Oberon environment.
I now using KDE as my Desktop Environment. If only were bindings for KDE in Oberon - it could be really *killing system*, both in terms of usability and program development.
Ohh, yes: KDE is written in C++.
So, it is fully Object-Oriented inside.
To late
It's a pity, that I've seen the article just now.
You'll love to see, that Askemos (http://www.askemos.org/) implements some of your design ideas.
Re: try EROS then
I have a question on EROS...
Can I use it for the base OS of my server? Does it support Apache and MySQL?
Re: ObjC?
> What about Objective-C?
Just what i was wondering...