The NT Kernel Approach: Understanding Windows Process Creation

In our journey through systems programming, we often start with the UNIX-like philosophy of process creation because it is elegant and minimal: you duplicate the current process with fork(), and optionally replace the execution context with a new binary using exec().

Windows, however, takes a fundamentally different approach. The NT Kernel was designed from the ground up for robustness, security, and a microkernel-inspired architecture (though this purity ended when Windows NT 4.0 moved the window manager and graphics subsystem, win32k.sys, into kernel mode for performance). In the Windows world, there is no native user-mode fork(). Process creation is explicitly a heavy, multi-step transaction managed through the massive CreateProcess API and coordinated by various subsystems.

In this chapter of the Systems I/O series, we will dissect the Windows process creation model, explore the role of the Client/Server Run-Time Subsystem (CSRSS), and look at C code examples using the Win32 API to spawn processes.

The NT Kernel Architecture: Subsystems

Before we look at the code, we must understand where code runs in Windows. The Windows NT architecture separates the core kernel (ntoskrnl.exe) from the environment subsystems that user-mode applications interact with.

When you run a standard Windows executable, you are typically not interacting with the NT Kernel directly. Instead, you are interacting with the Win32 Subsystem.

The Client/Server Run-Time Subsystem (csrss.exe)

In the early days of Windows NT, the system was designed to support multiple personalities: POSIX, OS/2, and Win32. To achieve this, Microsoft used a client-server model. A user-mode application (the client) would send a message over an LPC (Local Procedure Call) port—or ALPC (Advanced LPC) in modern Windows—to an environment subsystem (the server) to request services. (Note that this model is largely vestigial today; most Win32 calls go straight into ntdll.dll and win32k.sys, with only a residual set round-tripping to csrss.exe.)

The most critical of these is csrss.exe (Client/Server Run-Time Subsystem). Even though modern Windows has mostly abandoned OS/2 and POSIX (replacing it with Windows Subsystem for Linux), csrss.exe remains a vital component. It handles:

• Process and thread creation/termination coordination
• Legacy Win32 console windows (prior to Windows 7, which moved this to conhost.exe)

When you ask Windows to create a process, the Win32 API sends messages to csrss.exe to register the new process with the subsystem, alongside making the necessary system calls (via ntdll.dll) into the kernel to allocate the EPROCESS structure and virtual memory.

Why Windows Lacks fork()

If you are coming from Linux, the lack of fork() in Windows can be jarring. Why doesn’t Windows support it natively in Win32?

1. Copy-on-Write Complexity with Threads: fork() duplicates the calling process. However, modern programs are highly multithreaded. If a process with 10 threads calls fork(), does the OS duplicate all 10 threads, or just the calling thread? POSIX chose to duplicate only the calling thread, which leads to infamous deadlocks if the other threads held locks at the time of the fork. Windows sidesteps this entirely.
2. Subsystem Registration: As mentioned, a Windows process is tied to a subsystem (like Win32). A raw clone of a process in memory wouldn’t be properly registered with csrss.exe, leading to an inconsistent state where the kernel knows about the process, but the Win32 subsystem doesn’t.
3. Explicit API Design: Microsoft favors explicit initialization parameters over implicit inheritance followed by mutation. CreateProcess takes a massive structure defining exactly how the new process should start, avoiding the intermediate “cloned” state of fork(). For example, instead of forking, mutating file descriptors, and then calling exec() to redirect output, Windows explicitly passes handles by setting hStdOutput/hStdError in the STARTUPINFO block, combined with the STARTF_USESTDHANDLES flag, bInheritHandles = TRUE, and an explicit PROC_THREAD_ATTRIBUTE_HANDLE_LIST (which lives in a STARTUPINFOEX structure and requires the EXTENDED_STARTUPINFO_PRESENT creation flag) to restrict exactly which inheritable handles the child receives.

[!NOTE] The NT Kernel actually does have internal mechanisms which can clone a process (like RtlCloneUserProcess or NtCreateUserProcess with specific flags). For instance, Cygwin uses user-mode hacks to emulate fork(), while WSL1 handled it internally in kernel mode via the lxss.sys pico provider. (WSL2 runs a real Linux kernel in a VM, utilizing Linux’s own fork()). But native cloning is strictly hidden from standard Win32 applications.

The CreateProcess API

The Win32 function CreateProcess is the engine of execution in user-mode Windows. It is famously complex, taking ten parameters. Let’s look at a practical example in C.

Example: Spawning a Process in C

#include <windows.h>
#include <stdio.h>

int main() {
  // STARTUPINFO structures tell the OS how to initialize the process's main window
  STARTUPINFOW si;
  // PROCESS_INFORMATION receives the handles and IDs of the newly created process and primary thread
  PROCESS_INFORMATION pi;

  // Zero out memory for the structures
  ZeroMemory(&si, sizeof(si));
  si.cb = sizeof(si);
  ZeroMemory(&pi, sizeof(pi));

  // The command line we want to execute
  // Note: CreateProcessW can modify the command line string, so it must be in mutable memory.
  wchar_t cmdArgs[] = L"notepad.exe";

  printf("Attempting to spawn Notepad...
");

  // The monumental CreateProcess call
  BOOL success = CreateProcessW(
      NULL,           // Application name (NULL means extract from command line)
      cmdArgs,        // Command line arguments
      NULL,           // Process handle not inheritable
      NULL,           // Thread handle not inheritable
      FALSE,          // Set handle inheritance to FALSE
      0,              // No creation flags
      NULL,           // Use parent's environment block
      NULL,           // Use parent's starting directory 
      &si,            // Pointer to STARTUPINFO structure
      &pi             // Pointer to PROCESS_INFORMATION structure
  );

  if (!success) {
      printf("CreateProcess failed! Error code: %lu
", GetLastError());
      return 1;
  }

  printf("Process created successfully!
");
  printf("Process ID: %lu
", pi.dwProcessId);
  printf("Thread ID: %lu
", pi.dwThreadId);

  // Wait until child process exits
  WaitForSingleObject(pi.hProcess, INFINITE);
  printf("Notepad has been closed. Parent exiting.
");

  // Close process and thread handles to avoid memory leaks
  CloseHandle(pi.hProcess);
  CloseHandle(pi.hThread);

  return 0;
}

Breaking Down the Mechanics

Let’s dissect what happens when this C code runs:

1. Mutable Command Line: Notice that cmdArgs is declared as a mutable wide-character array. CreateProcessW (the underlying Unicode version) can temporarily modify this string during execution. Passing a string literal can cause an access violation on Windows! (The ANSI version, CreateProcessA, converts to a private Unicode copy, so it doesn’t fault, but it’s best practice to use the Unicode APIs directly).
2. Handle Management: The PROCESS_INFORMATION struct gives us handles to the new process (hProcess) and its primary thread (hThread). A handle is an opaque token — an index into this process’s private handle table, whose entry holds the kernel object’s address along with the access rights we were granted.
3. Synchronization: WaitForSingleObject(pi.hProcess, INFINITE) is the Win32 equivalent of UNIX’s waitpid(). We block the parent execution until the kernel signals that the child process object has transitioned to a “signaled” state (which happens when the process terminates).
4. Cleanup: Even though the process terminates, the process object (EPROCESS) lingers as a “zombie” to keep the PID reserved as long as the handle count and kernel reference count are non-zero. We must explicitly call CloseHandle to release our reference and let the kernel free the structure. (While the kernel closes handles automatically at process exit, doing it explicitly prevents resource leaks in long-lived programs).

Summary

The NT Kernel views process creation as a heavy, deliberate action rather than a cheap cloning operation. By using CreateProcess, we provide the OS with all the context it needs upfront, allowing the kernel and the Win32 subsystem to initialize the memory space, load the PE (Portable Executable) binary, and register the UI components in a single, coordinated transaction.