It seems that open-source operating system projects either never quite get past the point of having a kernel that can print stuff, read/write FAT32, and run a couple of userland processes, or they attempt to implement POSIX.
There's a couple of attempts to build something different - Haiku, based on the design of BeOS; and Redox, written in Rust and implementing access to the filesystem as URLs - but where's all the actual OS research? Where's the service-oriented OS built on capabilities with a typed object store for a filesystem?
Absolutely, there's been various approaches researched in the past - and I've used Oberon before, really enjoyed it - I'm just curious as to why it doesn't seem to happen any more.
Takes academics being paid to create it basically. Most academics just focus on getting papers published and citations. The number that will try to do useful stuff, esp OS's, are usually a rarity. Btw, here's a few:
So, there you go. THey don't all do what you describe. Some cool stuff worth emulating or building on in the pages above. Especially EROS or A2 Bluebottle for security and ease-of-picking-up respectively.
> I'm just curious as to why it doesn't seem to happen any more.
my _guess_ is that it is quite hard, and the payoff not so great, which would deter folks from venturing out too far. something similar to what happens in commercial space where a huge amount of energy is expended in enticing users to click advertisements etc.
Perhaps there's something I'm missing but for the last two on your list there's no reason to build a full on O/S. "A service" these days means "something that lives on a TCP or UDP port" and URL means HTTP or HTTPD.
I've built a couple or three microkernels[1] to use for real work, and it's just not interesting past a certain
point. Once you get the equivalent of the Linux ucontext.h stuff done the rest is ... stamp collecting.
[1] one back in the dark ages on top of DOS to enable getting a project rolling in parallel with the hardware, another specifically for providing sandboxed services on top of an existing RTOS, mainly to pretend this was a different computer/virtual machine, again as leverage because of schedule.
These days, you would put up a hypervisor or just use a COTS virtual machine.
I keep hoping something will be done with BeOS but ...
While there might be no reason to build a kernel - most modern ones will let you lock down a process well enough, although it "looks weird" - building good UX around capabilities and objects instead of permissions and files requires starting from the ground up. The vast majority of applications, tools, and workflows that people use right now don't fit within that paradigm.
> "A service" these days means "something that lives on a TCP or UDP port" and URL means HTTP or HTTPD.
A service consists of an administration protocol and a usage protocol. In Linux, the administration protocol is usually some ad-hoc text format and SIGHUP, in Windows it's Powershell, and the usage protocol is almost always ad-hoc over TCP, requiring a library to interact with it.
It's possible to abstract services away from how you communicate with them, provided that you're working within an environment where everyone buys into that abstraction. Communicating with a file on your desktop, a file on a file server, a database server, and your coworker's spreadsheet window all become something you can do without having to find the relevant library which speaks the right protocol, or have ad-hoc mechanisms for sharing each of these things over a network - the relevant services self-describe how to communicate with them, in the same way that you can connect Powershell to an Exchange server and have all the tools for managing Exchange without installing anything.
You get to do away with SSHing into another box with a completely different shell environment, or trying to wrangle your local tools to work over sshfs. You could work on drafting an important email with a coworker as easily as dragging the window into your chat application, with neither the chat application nor the email client having to care how that works - the chat application just needs to be able to transport capabilities, and the window manager needs to be able to create a "window view" capability.
You can also start playing with really fancy things like dynamic constraint-based load balancing across a network, once your services don't care about how they're communicating with each other or where they are - where services get farmed out onto the network automatically when you perform resource-intensive tasks.
Plan 9 gets some of the way there, although I believe a filesystem is not discoverable enough - can I reliably get a lightweight IDE to tell me what "echo swap > /dev/mouse" does, vs "mouse.swapButtons()" where mouse is a typed variable? There's also no formalized way of doing request/response (especially out-of-order), and accessing a certain field of a message requires ad-hoc string parsing which is easy to get wrong.
This needs to be done as an operating system - a complete set of tools which enables a certain way of managing and interacting with resources, both as a developer and a user - because otherwise you keep hitting on the boundary of things you can interact with within the system and things you need to interact with outside the system. If you can interact with your text editor and chat application but not your spreadsheet tool or database server this way, that diminishes its use significantly.
I've been playing with minimal systems which do some of this. It's interesting, but hard work to build entire new user experiences from scratch instead of being able to port a libc to a new kernel and get a bunch of stuff for free.
This all looks awfully familiar, even though I'm primarily in the embedded space.
>In Linux, the administration protocol is usually some ad-hoc text format and SIGHUP, in Windows it's Powershell,
It's the same everywhere if you're liberal about the word "text".
> can I reliably get a lightweight IDE to tell me what "echo swap > /dev/mouse" does, vs "mouse.swapButtons()" where mouse is a typed variable?
So one of those is doubtless a lower primitive than the other; exchanging mappings is just that. An "IDE" just exposes the mappings in some human readable form.
If the mapping is 'C' source, that's one way. If its 'C' source that calls ioctl() it's another...
Have you ever looked at SNMP? With its ASN.1 MIBS? The metadata for the name space is specified there. Now, the BER encoded UDP PDUs are anything but ASCII but that's just a tools problem, again. Using those MIBs you can basically treat a client on the requester as a shell using ASCII names.
It's a bit like JSON the hard way. You can do more sophisticated things but that's just molecules made of atoms.
> This needs to be done as an operating system - a complete set of tools which ...
It's just a stack of shells. So meta the shells and Bob's yer oyster. It's not that simple yet it sort of is...
For a variety of reasons, I think of these as seperate appliances.
> ... hard work to build entire new user experiences from ...
I expect it is because people who want to provide something like your service-oriented OS will do it as an application layer over existing operating systems.
This also gives them the flexibility to factor the problem into smaller bits. Cap'n Proto is a service-oriented something-or-other, built on capabilities and even typed objects. But it does not give a flying -ck about storage.
The issue is that all these bits and pieces are wonderful for developers, and to a lesser extent sysadmins, but not so wonderful for users, who still get to interact with their systems using much the same paradigm as when the first desktop operating systems came out.
The closest I can find would be Sandstorm, which ends up being a frankenstein mashup of Linux and a capability system.
To be fair, Cap'n Proto is a sub-project of Sandstorm.io which is basically an operating system and cares deeply about storage. But Cap'n Proto can be used independently...
> where's all the actual OS research? Where's the service-oriented OS built on capabilities with a typed object store for a filesystem?
Once you realize there's not much a difference between Microkernels are the recent microservices architectures, they become interesting again. Then you might want to look in recent advances such as Unikernels or Anykernels.
one issue is that it can take a long time to get to just that point, and so doing further, more interesting research comes after you've already done a ton of hacking.
Fun to see this popping up on the day I'm hacking on my own litte OS, intermezzOS.
There's lots of great stuff in this series, but it also exemplifies an issue with the hobby OSdev scene: it's almost all for protected mode, with some real mode stuff, and very little long mode stuff. This one does have great real mode stuff, though.
Protected mode is a bit simpler, which is nice. Hopefully over the next few years, the long mode stuff will come along. Most of the hobby OSes in Rust are in long mode.
Would it be possible to move more parts of the operating system into hardware to realize an improvement in speed? So for example say Microsoft announced Windows 10 was the perfect operating system and needed no new features/bug fixes would it make sense and would it be possible to realize the OS software in hardware?
Main reason against doing this sort of stuff is it impacts chip performance and esp reduces flexibility. People are always inventing new ways to speed up OS's. So, we implement them in SW that maps to HW with decades of improvements built-in. Works best overall. Everyone doing high-level hardware takes a loss outside some Java and Forth processors.
Yes. It's where we've been going already since time immemorial. It's why graphics cards picked up 2d, then 3d acceleration, it's why network cards have varying amounts of power to analyze and manipulate incoming and outgoing packets, etc. A lot of these things tend to be put into hardware after they've been proven useful, usually trickling down from the server world.
In chapter 2 of the tutorial, he mentions Commodore DOS as being an OS which resides in ROM (read only memory), which means it's definitely possible. I'm just a laymen but I'd wager that there are some cases where this is still done.
But that's still just software stored in a ROM chip. I think what he meant was implementing the actual logic in silicon. Something like a hardwired CPU.
Intel i432 had some of that (capabilities, garbage collection, multitasking, etc) but was crippled by a variety of problems—perhaps most of all that it was quite slow and expensive. It was originally intended to run a Smalltalk-like language, the name of which escapes me at the moment, but got retrofitted to run Ada. Unfortunately the Ada compiler for the i432 was awful, in addition to the slow hardware.
Anybody aware of nice, complete video lectures for Operating Systems class? There were lectures from Stanford, but only sessions with profesor, no TA session.
I did have some problems when working through the broken thorns tutorials. It seems like some of the code on the site does not work verbatim. Needs a little finagling.
There's a couple of attempts to build something different - Haiku, based on the design of BeOS; and Redox, written in Rust and implementing access to the filesystem as URLs - but where's all the actual OS research? Where's the service-oriented OS built on capabilities with a typed object store for a filesystem?