Concept

Operating System

Definition

An operating system (OS) is the software layer that mediates between a computer's hardware and every program that runs on it. It is not a single application but a collection of services: a kernel that directly manages the CPU, memory, and devices; a set of system calls that programs use to request those services; and a user-space environment of libraries, shells, and utilities that sit above the kernel and below user applications. Major historical and modern examples include Unix, Linux, Windows NT, macOS, Android, and iOS.

The core problem an OS solves is resource sharing under isolation. A modern machine may run dozens of processes simultaneously, each believing it has the CPU to itself and a contiguous block of memory — a polite fiction the OS maintains through scheduling and virtual memory. Without this abstraction every programmer would need to reason about the physical machine in full; with it, programs can be written as if they run alone, and the OS handles contention, protection, and fairness beneath the surface.

Operating systems also enforce security boundaries. The distinction between kernel mode (unrestricted hardware access) and user mode (restricted access mediated by system calls) is the architectural basis for why one process cannot read another's memory, and why a user-space crash does not take down the entire machine.

Why it matters

How it works

The kernel and system calls

The kernel is the trusted core that runs in privileged mode and directly manipulates hardware registers, memory-management units, and device controllers. Everything else runs in user mode and must cross the user-kernel boundary via system calls — the formal API of the OS. When a program opens a file, sends data over a network, or creates a new process, it issues a system call: a controlled transition into kernel mode, execution of a specific kernel function, and a return to user mode with a result.

This boundary is the OS's primary security mechanism. Malicious or buggy user-space code cannot directly corrupt kernel data structures; it can only make legitimate (and auditable) system-call requests. Kernel exploits are dangerous precisely because they breach this boundary and gain unrestricted hardware access.

Processes, threads, and scheduling

A process is an OS abstraction representing a running program: a private address space, a set of open file descriptors, and at least one thread of execution. Threads within a process share the same address space but each have their own stack and CPU register state, enabling concurrent work with lower overhead than spawning separate processes.

The scheduler decides which thread runs on which CPU core at each moment. Preemptive schedulers interrupt running threads on a timer — ensuring no thread monopolises the CPU — and select the next thread according to a policy (round-robin, priority-based, completely-fair scheduling, real-time deadlines). The scheduler also performs the context switch: saving the outgoing thread's register state and loading the incoming thread's, an operation that takes hundreds of nanoseconds and dominates latency in high-frequency switching.

The OS as a tower of levels

The OS is also the cleanest everyday example of a multi-level system, which is why Hofstadter reaches for it in Gödel, Escher, Bach. The same machine can be described as kernel data structures, as system calls, as process abstractions, or as a user experience — each level with its own vocabulary and its own failure modes. The classic illustration is the "35-user threshold" story: a time-sharing system that grinds to a halt once a certain number of users log on. That slowdown is stored nowhere; it is an epiphenomenon that emerges from disk, memory, and scheduler dynamics interacting. You cannot find it at the hardware level, and you cannot fix it from inside a single process — it is a property of the whole tower.

Where it goes next

Operating systems are the foundation beneath all systems programming. Understanding them unlocks reasoning about performance, concurrency, security, and portability that is otherwise opaque. Containerization (Docker, namespaces, cgroups) is OS isolation taken to an operational extreme; hypervisors run entire OS instances as processes in another OS — virtualization that depends on the same privilege-separation principles the kernel uses internally.

Continue exploring

Tags