Pos Design

We believe that, when it comes to operating systems, many approaches are not broken by design, but rather just poorly implemented. We do aim at a production, not a research level system. A different or exotic solution to a problem is not yet a feature. Thus, when facing the decision of whether to do a proper implementation of a "classic" design, or implementing an "exotic" solution, the "exotic" (or, "cutting edge") solution must fulfill the following conditions to be considered viable:

• it must not compromise security or performance of the system as a whole;
• it must not add inappropriate complexity for the application developer, and no complexity for the user;
• research material / documentation on it must be available to an extend that implementing the solution is not a research experiment;

Experimenting is beyond the scope of this project. If you want an experimental feature in the OS, form a project outside Pro-POS to research the feature until it can no longer be considered experimental, and we will welcome your efforts.

There have been multiple, unfortunate examples of fine projects getting a bad reputation solely due to how it was evangelized. Thus, we ask each and every contributor (just as well as any by-chance visitor):

Do not advertise this project online by posting to portals, news sites, discussion forums etc.; such advertising should be prepared instead of being done ad-hoc.

Adding our project to your link list is welcome, as long as your link list is not the entry page for thousands of users, as that would qualify as "portal advertising". (This means you, MSN! ;-) )

Do not engage in flamewars of the "my OS is better than yours" kind. It is folly to make any argumentative claims for an OS still in the making. When "Perfection" v1.0 is actually released, we will seek out the judgement of the competent. Unproven claims just reduce the signal/noise ratio.

Evangelizing Pro-POS on Linux, MacOS, AmigaOS, Windows, BeOS, or other system-specific websites or newsgroups is likely to do more harm than good. Such websites and newsgroups are system specific on purpose, and we should respect that.

At first, we will target the IA32 architecture as being the most popular and readily available. But we will, right from the start, keep cross-platform issues in mind, as our final target is full hardware independance on the application level (and probably on the driver level, too). A carefully designed API and Style Guide will help making the addition of other platforms easy.

For the driver question, we are considering the Uniform Driver Interface, Project UDI, which is source compatible across all UDI supporting platforms, and binary compatible (sic!) across all platforms with the same instruction set, making it a perfect match for our idea of HW independance. While this concept has been implemented on a number of reference platforms, Pro-POS would be one of the first OS's to implement UDI as native driver model. UDI, SNAP, and similar technologies prove that the driver issue can be solved in a cross-platform way.

The second step will be the addition of other platforms with full API compatibility, so that "porting" an application from one supported platform to another is a single compiler switch.

For the third step, we are evaluating the possibility of providing a "virtual processor" / byte code build target that is translated to native code either JIT or load-time.

Originally, we intended to write everything in full-fledged C++ - including enabling exceptions, standard library, and everything in kernel space. This would still be our preferred path, since it would allow us to use the very same semantics in kernel space as in user space.

However, this would have required quite some pioneering, since the use of full-fledged C++ in kernel space is sparsely tried and not at all documented in a way useful to us. Considering the currently available "active" manpower (one, give or take one...), that was a show-stopper.

Yet still, we would prefer the additional "syntactic suggar" that C++ would offer us, so the kernel will be written in a subset of the language.

No extra run-time support (no exceptions, no RTTI)

It was next to impossible to find precise information of what is required in terms of run-time support. Most people just accept that the support is there in user space, and leave the gory details to the compiler builders who aren't exactly known for writing tutorials.

No C++ Standard Library

Using full-fledged C++ would have required us to provide a standard library first, either ported or reimplemented. We will use a different approach. Starting with a C standard library (pdclib) only, we will implement only those parts of the C++ standard library, as we go, that directly benefit our kernel code. If we need a map, we will implement one that functions like specified by the standard, as far as required. That means that our "kernel map" might not have all the features of a standard map, but what features there are are handled like defined by the standard. (The idea is to make it easier for Pro-POS newcomers with C++ experience to understand our code.)

No external C++ interfaces

The kernel will use C++ only internally, with all interfaces defined in standard C, to ease interfacing of other popular languages in user space. (The original idea was something in the Object Broker ballpark, which we felt was a bit too esoteric to use in such a central point of the system.)

The development environment will be GNU/gcc/binutils (using the latest stable v3.x series gcc) until we can bootstrap a native SDK. We will take great care to adhere to ISO standards, avoiding special GNU features, to make the compiler used interchangeable.

Virtually every Linux distribution comes with the necessary tools. Cygwin provides a GNU/gcc/binutils environment for the Microsoft Windows OS family. Assuming that people using Linux as their primary development platform are sufficiently familiar with the toolchain, any documentation on the development environment will assume a Windows / Cygwin setup.

If someone wants to provide auxiliary documents describing the necessary steps to set up a toolchain on some other operating system (MacOS, Amiga, BeOS, ...), such a contribution would be very welcome.

We will not follow the "unadulterated doctrine" of the microkernel or monolithic kernel. It will simply be designed to suit the job at hand:

Preemptive multitasking, strong multithreading and multiprocessing support, full memory protection, strong system security (not in the TCPA sense, but in the "do not crash and do not leak my data" sense), and virtual memory with optional paging support are currently percieved as "must-have"s.

The kernel will be able to run in several "modes". (Optionally, the system might provide completely different binaries for each "mode", if that should prove necessary.) Percieved mode options are responsive, throughput, multimedia, computational, and real time (under consideration).

There will be extensive multithreading support, including lightweight user-level threads.

Exokernel technology has been evaluated and declined for the POS Design.

Case Insensitive

The file system will be strong-typed but case insensitive. Note that this is different from either Windows (weak typed) or Unix (case sensitive).

Drive Names

A physical drive (e.g. a CD-ROM, a ZIP, a hard drive) will be accessible using a "physical / drive name".

To-Do: How to handle multiple partitions on a hard drive?

A media, or a partition on a media, will be accessible using a "logical / media name".

In a fully qualified path name, a file could be addressed either by using the physical drive name, or by using the logical / media name. The former would access whatever media is currently in the drive, the latter would require the correct media to be inserted.

Specific paths (on any media) can be assigned a "virtual / assigned name", and accessed as if they were a drive themselves.

It will be possible to install "file system handlers" which map access to the file system / protocol underlying the device. (Thus enabling e.g. an FTP connection to be accessed like any other drive, by having a file system handler mapping drive name "FTP" to the FTP protocol. Other examples could include a "PRINTER" device you could copy files to, or a "VCS" device doing automatic version control on the files it holds.

The file system will also provide two "special" drives, being SYSTEM (for the OS itself) and SOFTWARE (for installed applications). These special drives will not be accessible in the traditional way. See the security concept for details.

We will use the filename extension as primary means of identifying file types in everyday work. (About the only metadata that is portable across platforms and file system formats.)

This will be backed up by an automated filetype detection system (a la "magic numbers"), which can and will be used for:

• giving a correct name for a file without a filename extension,
• fixing the extension in case opening a file returns an "unknown format" error (misnamed file).

The reasoning behind this is that filename extensions are quicker to evaluate than magic numbers in everyday work (they do not require opening every file in a directory and reading parts of it); and in our heterogenous computing world they have become a widely accepted means of telling file types apart ("accepted" as in "grudgingly", not "cheerfully").

Registry Entries:

Datatype ID

A system wide unique ID used for handles etc.

Datatype Name

A system wide, unique name for the datatype.

Datatype Description

A short description of the datatype, e.g. "8 bit image file, lossless compression" for GIF or PNG (mime-types descriptive enough for this?)

Registered Extensions

An enumeration of filename extensions associated with this datatype; checked for unambiguousness.

Preferred Extension

The preferred extension for files of this type (checked to be one of the Registered Extensions).

Registered Applications

Enumeration of application IDs that are registered to handle this datatype (including timestamp of last use and counter of times of use).

Magic Number Pattern

A pattern matching expression of "magic numbers" to identify files of this datatype.

Ambiguosity

Enumeration of datatype IDs with a magic number pattern that might resolve to "hit" on the same datatype.

We have to work out a way to store file metadata: icon image (per UI module, by type or individual file), icon position (per UI module, by individual file), last application used (by individual file). A "Verified" flag that states the filename extension has been verified / created by the filetype detection system would be useful. This flag can be unset (to enforce a check on next access), but not set manually (trust revocal).

To-do: Working on how metadata can be preserved over foreign file systems, i.e. native -> DOS/ISO9660 -> native.

While not being a design target as of now, ISO/IEC 15408 is perceived; we might strive for EAL certification at a later point of time, and our aim would be EAL 5 minimum.

On a high level, the system security concept interacts strongly with the File System concept. The OS itself (i.e., the OS core and any shared ressources) will be installed in the special drive SYSTEM, and any installed software (see Installation concept) ends up in the special drive SOFTWARE.

These two drives will not be accessible in traditional ways. You will not be able to list directories, or to read / write / edit files on these drives. Any fopen() to them, or similar function, will simply fail.

There are only three ways of getting data into the SYSTEM drive:

1. during initial installation of the OS,
2. through the OS update service,
3. through the Installation service.

There is only one way of reading data out of the SYSTEM drive - by requesting an OS service through a defined API.

This protects the OS against tampering, by accident or malicious (since the OS services can protect the OS e.g. against overwriting of important system files). It allows the OS to keep a clean reference count on shared ressources (since no application can reference a resource without the OS knowing).

The same is true, basically, for the special SOFTWARE drive, with the OS additionally protecting each installed software package against each other. Attached to each application is a list of directories it is allowed (read or read / write) access to. This makes for tightly isolated applications that, even if of malicious intent, cannot harm either the system or user's data without user consent.

This might solve several authentication issues; this idea is not yet finalized.

UI Modules

The idea is to have applications only describe the logic of their user interface; the actual representation would be rendered by the UI module configured by the user. UI modules are part of the OS, not individual applications.

This has immediate impacts:

• when accessing a system over the network (compare remote X Window sessions), the amount of messages and redraw informations that has to be transmitted is less than with X Window (which is transmitting drawing primitives);
• "skinning" is automatically system-wide and consistent across all applications, since applied to the UI module instead of the application;
• all applications automatically benefit from "skinning" applied to the UI module.

This concept would allow to replace a traditional, graphical 2D desktop UI module with a sound-based module, or a braille terminal module for the blind, a virtual reality module for the geeks, or a cranial dataport interface for the system admins of tomorrow.

Command Line Interface (CLI)

In addition to the UI modules - the term "Graphical User Interface" (GUI) doesn't really apply to the more versatile UI modules - the system will also provide a powerful CLI subsystem.

Due to the fundamental difference between the abstract UI metaphor and a command line interface, the CLU subsystem will provide "CLI modules" of its own; this could include e.g. VT100 emulation, braille terminal etc.

Cornerstones of the CLI environment include:

• plaintext, non-abbreviated commands ("copy", "delete", "rename", ...) and options ("all", "quiet", ...);
• auto-completion for commands and file names;
• strong-typed, case-insensitive ("Copy", "copy" and "COPY" are the same command / file);
• a powerful, extensible online help system;
• strong scripting capabilities, with support for interactive scripts (e.g., displaying a file requester, or requesting input with type / sanity checking).

Unix tells you that a shell command should be a minimalistic tool, doing only one thing but doing that well, and that no message is a good message. Our CLI will not follow this concept.

Instead, each command should be complete to do the job at hand. To avoid reimplementation of similar functionality in different tools, common functionality will be implemented in a central command.library, to the point where the simpler tools (like the equivalents of less, find etc.) are merely executable wrappers for command.library functions.

This principle can be generalised, making systematic that all command lines are front-ends to libs. This is a big change compared to the Unix culture as it is almost thought that the cornerstone of the system is its command lines, and that any GUI's should just be front-ends to those commands. I think that change of perspective would allow easier developments of powerfull applications. --?LionelThiry

There will be a means to allow commands to detect whether they run in interactive mode, in a pipe chain, or a script, removing the need for Unix'ish options like "-" (read from stdin).

In addition to the ability of combining command line tools with basic flow-control functions into a shell script, the system will provide a dedicated scripting language that will enable more complex processing. It will be possible to access the OS API, shared ressources, and inter-process communication by means of this language, in an easy and intuitive way.

This scripting language will be the tool of choice for application macros, plug-ins, rapid development of small applications, script-controlling applications, and application feature embedding. It will be possible to compile script files into binary or run them in interpreted mode; the system will also provide a full-fledged debugger tool for the language.

If you throw Visual Basic for Applications, Rebol, AppleScript, AmigaE and ARexx into a pot and pick only the best parts from the mix, you will get the idea.

Our OS will provide basic services, for example a text editor component, a utility for displaying basic XHTML files, basic tools for developing software. Infrastructure for modular datatype parsing, cryptography modules, language catalogs, compression algorithms, you name it. But we won't provide word processing software, industrial strength databases, graphical IDE's or a full-fledged web browser as part of the OS - only the means to implement them easily and efficiently, using what is already there.

All such components will be designed so they can be upgraded / replaced, with any software using those components automatically benefitting from such an upgrade. (E.g., replacing the text editing component will immediately benefit the IDE, web browser, e-mail client, shell interface, you name it.)

While it might look like folly to distribute a "naked" OS without software suite, we believe that too strong a "default" software suite severly discourages development of better / cheaper / easier solutions (think Word and Internet Explorer), while a too diverse distribution will clutter your hard drive with lots of redundant and confusing applications (think the average Linux distro). Moreover, both approaches would divert manpower away from working on the core OS, which is the sole focus of this project.

Of course, we will be strongly supportive of any project aiming at providing Pro-POS software, but we will keep such projects apart from the OS efforts. Moreover, we will not discriminate if there should be more than one such effort. Support and endorsement is provided for all such efforts, or not at all.

Idea One: Private configurations, like visual and application preferences, should be seperated from login identity. The same "system user" should be able to run in virtually any "customization mode". This caters e.g. for the desktop used by roommates, which don't want to set up seperate user accounts, but still want to use seperate preference settings. Think system (and application!) wide "setting themes".

Idea Two: Unix "root" is basically necessary for system configuration only. We could collect any potentially destructive configuration in a protected "pandora's box" (configuration environment), accessible by any user who can sufficiently authenticate himself. You no longer enter a password to make yourself "root", you just authenticate that you are actually allowed to change things. This can be a password, a smart card, a PGP challenge / response, whatever.

Idea Three: Don't lock users out. Welcome them on a "guest" desktop so they can "click around", and possibly authenticate themselves later (perhaps even transparently - !), instead of having them bump into an upfront "login screen" the hard way. Think generic presentation machines at an exhibition which you can turn into your accustomed workstation environment any time with a single action (given the software is installed), without compromises in system security for the exhibitor.

We are considering a concept of "portable configurations". More on this later.

The Installer subsystem will meet the following requirements:

• All files are either...

  • placed in (or under) a single application directory (application-specific files); or
  • placed in system directories holding a specific type of shared ressource (one directory for libraries, one for fonts, etc.);
• a reference count is kept, allowing to safely identify and remove unused shared ressources upon deinstalling an application;
• rollback of software installations is failsafe, even after aborted, incomplete, or broken installations;
• dependencies between applications and shared ressources can always be determined unambiguously;
• version labels for shared ressources will be consistent, required, and easily determined;
• accidently (or intentionally) breaking or damaging software installations or the system will be disabled;
• installing and uninstalling software will follow a system-wide look & feel;
• downward-compatible "legacy" versions and incompatible newer versions of the same ressource will coexist on the same system.

The system will enforce this by a very restrictive handling of read/write access to system / application disk space as well as execution rights. Consider an installation to be divided in three parts: system, applications, and data.

The system, and each application, are handled as individual black boxes, with access only allowed through OS services. That means that the system and application part of the file system can not be navigated with the usual file system services.

The only way for an application to be installed is through the OS installation service. This service handles creation of a new directory in application space, with all app-specific files copied there. Shared ressources are integrated into the system space if not already present, including a reference count and "strong versioning" of libraries etc. (see below).

An application has read/write access to its own directory only. Read/write access on data space is granted / revoked by authority of the user. (This means that an application can query for the existance of certain other applications only throgh the OS, allowing the OS to keep a correct log on dependencies.)

An executable file that has not been installed in application space (i.e., residing in the data section) can only run in a sandbox, which means it cannot access existing files, can create new files only in its own directory (or subdirectories thereof), and has no way of querying the system for any ressources beyond certain basics.

Strong Versioning

An application either...

1. always uses the latest, global library;
2. requires a specific "major" revision of a library, but accepts minor "bugfix" updates - if a new "major" release is introduced to the system, the application continues to use the latest available previous one;
3. requires a specific version of a library - if a newer one is introduced to the system, the application continues to use the old one.

Case 2) above should not require any additional meddling with the applications.

Fonts

"Hard" reference counting does not work with "reusable" ressources like fonts. Keeping a "usage count" and time/date stamp of latest use should help here. Each font should be stamped with the name(s) of the application(s) that provided it, so a licensing-sensitive user can uninstall fonts for which he does not hold / no longer holds a valid license.

The default language for the OS is US English. The default time / date format is ISO 8601 (yyyy-mm-dd hh:mm:dd, on a 24-hour clock).

If the locale subsystem should fail, or an application should come without a catalog for the system locale, the system will fall back to these defaults.

The OS will provide means that allow to keep language ressource files and settings distinct from the application binaries. Thus, i18n can be outsourced without disclosing source code, and there is only one binary to maintain and distribute. An installation can be switched from one language to another at runtime. (Hello AmigaOS!)