P600Virtualization
Annex I · 20 notes
Hypervisor engineering notes on CPU and memory virtualization, KVM, interrupts, device emulation, virtio, guest boot, passthrough, snapshots, security, and building a Rust VMM across x86-64, ARM64, and RISC-V.
- 20
- notes
- 489
- minutes
- 261
- references
- 52
- diagrams
cat study-map
Purpose: Root navigation map for learning hardware virtualization deeply enough to build a production-minded Rust VMM on KVM for x86-64, ARM64, and RISC-V.
Virtualization
A virtual machine monitor is a user-space program that asks the kernel (KVM) to run guest code directly on the CPU, then emulates everything the CPU cannot do safely on its own: devices, firmware tables, interrupts, clocks, and boot. Hardware does the heavy lifting for CPU and memory isolation. The VMM owns policy, devices, lifecycle, and the attack surface.
This note is the map, not the content. Use it to choose what to read next and to see how the pieces depend on each other.
The fastest path is not "read every spec, then write a VMM." It is: understand trap-and-emulate, run a few guest instructions through KVM_RUN, then grow the VMM one exit reason, one device, and one architecture at a time.
Note set
| Layer | Notes | What they are for | Exit signal |
|---|---|---|---|
| Orientation | 00 Virtualization Mastery Roadmap, 01 Virtualization Mental Model and History | Study order, project ladder, and the core trap-and-emulate model. | You can explain why classic x86 needed binary translation and what VT-x changed. |
| CPU | 02 x86-64 Architecture for Hypervisor Authors, 03 Intel VT-x and AMD-V Hardware Virtualization, 15 ARM64 Virtualization EL2 VHE and GIC, 16 RISC-V Virtualization and the Hypervisor Extension | Privilege models, paging, system registers, and the hardware virtualization extension of each architecture. | You can set up a vCPU to enter 64-bit code directly on each architecture. |
| KVM core | 04 KVM API and the vCPU Run Loop, 05 Memory Virtualization EPT NPT and Guest Memory, 06 Interrupts Timers and Interrupt Controller Virtualization | The ioctl surface, the run loop, guest memory, interrupt injection, and time. | Your VMM boots a guest that takes timer interrupts and halts cleanly. |
| Devices | 07 Device Emulation PIO MMIO and the Bus Model, 08 Virtio Devices Virtqueues and vhost, 10 Storage and Network Backends, 11 Device Passthrough VFIO IOMMU and Nested Virtualization | Emulated devices, paravirtual virtio devices, host backends, and real hardware passthrough. | The guest has a console, a root disk, and a network interface. |
| Lifecycle | 09 Booting Guests Firmware and the Linux Boot Protocol, 12 Snapshots Live Migration and Dirty Page Tracking | Getting a kernel running, and capturing or moving a running VM. | You can boot Linux in under a second and restore it from a snapshot. |
| Production | 13 Security Isolation and Attack Surface, 14 Performance Observability and Debugging | Threat model, sandboxing, exit costs, tracing, and debugging. | You can justify every device you expose and measure every exit you take. |
| Ecosystem | 17 Hypervisor Architectures Compared, 18 Rust VMM Ecosystem and Project Templates | How existing VMMs are built and which rust-vmm crates to reuse. | You can sketch your VMM's crate layout and defend each design choice. |
Adjacent vault anchors
- 01 Linux Mental Model User Space Kernel and Hardware and 13 Kernel Architecture Modules Drivers and Device Model for the host kernel side of KVM.
- 03 Memory Virtual Memory Paging Allocators and OOM before 05 Memory Virtualization EPT NPT and Guest Memory.
- 09 cgroups Namespaces Containers and Runtime Isolation for the containment around a VMM process.
- 09 Unsafe Rust and the Rust Memory Model and 14 FFI Embedded WebAssembly and Interop for the unsafe and ioctl boundaries a Rust VMM lives on.
Mental model
Rendering diagram...
Three questions frame every design decision:
- Who handles this exit? Hardware, KVM in the kernel, or your VMM in user space. Each step outward costs more time.
- What does the guest see? Every register, device, table, and clock is part of a contract you must keep stable across restore and migration.
- What can a hostile guest reach? Every byte the guest can write into your VMM is attack surface.
Study paths
| Goal | Path |
|---|---|
| First working VMM on x86 | 00, 01, 02, 04, 05, 07, 09 |
| Fast Linux microVM | add 06, 08, 10, 14 |
| Multi-architecture VMM | add 03, 15, 16 |
| Production hardening | add 11, 12, 13, 17, 18 |
Review checklist
- Every leaf note links back to this map or the roadmap.
- Architecture-specific claims name the architecture.
- Facts are checked against the Primary Sources listed in each note.
- New notes use the
NN Titlefilename pattern and the shared frontmatter.
ls virtualization/ --order=reading
- No. 1Virtualization Mastery RoadmapPurpose: A staged study plan through notes 01 to 18 and a project ladder that ends with a jailed, snapshot capable Rust VMM on KVM booting Linux on x86 64, ARM64 and RISC V, with mastery checks that expose real gaps at...24 min · 2 diagrams
- No. 2Virtualization Mental Model and HistoryPurpose: Build the mental model every later note depends on (what a virtual machine is, what makes an ISA virtualizable, how the industry got from trap and emulate to KVM) so that design decisions in your own Rust VMM...24 min · 3 diagrams
- No. 3x86-64 Architecture for Hypervisor AuthorsPurpose: give a VMM author the exact x86 64 machine model (modes, segmentation, paging, control registers, MSRs, CPUID, exceptions, reset state) needed to put a vCPU into a valid state through KVM, and contrast it with...24 min · 3 diagrams
- No. 4Intel VT-x and AMD-V Hardware VirtualizationPurpose: explain how Intel VMX and AMD SVM actually run a guest (VMCS/VMCB, controls, entry, exit, bitmaps, TSC offsetting), which parts KVM hides, and which parts still leak through KVM RUN into your VMM, with ARM EL2...25 min · 2 diagrams
- No. 5KVM API and the vCPU Run LoopPurpose: teach the KVM userspace API from /dev/kvm down to the KVM RUN loop, so you can build the core of a Rust VMM that creates VMs, configures vCPUs on x86 64, ARM64 and RISC V, and dispatches every exit correctly....28 min · 3 diagrams
- No. 6Memory Virtualization: EPT, NPT, and Guest MemoryPurpose: explain how guest memory is translated (GVA to GPA to HPA), how KVM builds second stage page tables from memslots, and how a Rust VMM allocates, maps, tracks and reclaims guest RAM on x86 64, ARM64 and RISC V....26 min · 3 diagrams
- No. 7Interrupts, Timers, and Interrupt Controller VirtualizationPurpose: explain how interrupts and timers reach a guest on x86 64 (PIC, IOAPIC, LAPIC, MSI, posted interrupts), ARM64 (GICv2/v3/v4, ITS, generic timer) and RISC V (PLIC, AIA, SBI timer, Sstc), and how a KVM based VMM...28 min · 2 diagrams
- No. 8Device Emulation: PIO, MMIO, and the Bus ModelPurpose: explain how guest accesses to device registers become VM exits, how a VMM routes those exits through address space buses to device models, and how to emulate the small set of legacy and PCI devices every real...27 min · 3 diagrams
- No. 9Virtio Devices, Virtqueues, and vhostPurpose: teach the virtio 1.x device model end to end (status and feature negotiation, split and packed virtqueues, notifications, transports, the common device types) and the fast paths that take the VMM out of the...26 min · 3 diagrams
- No. 10Booting Guests: Firmware and the Linux Boot ProtocolPurpose: explain exactly what a VMM must place in guest memory and in vCPU registers so that a Linux kernel (or firmware) starts executing correctly on x86 64, ARM64 and RISC V, and how to choose between direct kernel...30 min · 3 diagrams
- No. 11Storage and Network BackendsPurpose: explain what sits behind a virtio blk or virtio net device in the host (image formats, caching, async I/O, durability, rate limiting, tap and macvtap networking, offloads, metadata services, vsock) so your VMM...24 min · 2 diagrams
- No. 12Device Passthrough, VFIO, IOMMU, and Nested VirtualizationPurpose: explain how a VMM hands a physical device (or a slice of one) directly to a guest safely, using the IOMMU and the Linux VFIO and iommufd interfaces, and how nested virtualization lets a guest run its own...26 min · 3 diagrams
- No. 13Snapshots, Live Migration, and Dirty Page TrackingPurpose: define exactly what the state of a running VM is, how to capture it consistently, how to restore it fast (eagerly or lazily), and how to move it between hosts while it keeps running, so your VMM can pause,...28 min · 3 diagrams
- No. 14Security, Isolation, and Attack SurfacePurpose: teach how to design a VMM that survives a hostile guest, from threat model and device attack surface to jailing, seccomp, fuzzing, speculative side channels, and confidential computing, so security is a...33 min · 3 diagrams
- No. 15Performance, Observability, and DebuggingPurpose: show how to measure, explain and reduce the cost of a KVM based VMM (VM exits, halt polling, pinning, hugepages, I/O threading), how to observe it with perf, tracepoints and KVM statistics, and how to debug...25 min · 3 diagrams
- No. 16ARM64 Virtualization: EL2, VHE, and the GICPurpose: explain how Armv8/Armv9 virtualizes CPUs, memory, interrupts and timers (EL2, stage 2, VHE, vGIC, generic timer) and exactly what a KVM based Rust VMM must do differently on arm64 compared with x86 64. ARM64...25 min · 3 diagrams
- No. 17RISC-V Virtualization and the Hypervisor ExtensionPurpose: explain how the RISC V Hypervisor (H) extension virtualizes privilege modes, memory, traps, interrupts and timers, how KVM RISC V exposes it, and how to build and test a RISC V target for your Rust VMM without...22 min · 3 diagrams
- No. 18Hypervisor Architectures ComparedPurpose: Compare the production hypervisors and VMMs you will be measured against (where they put device models, what language they use, what they expose to the guest, what they boot and how fast) so you can...23 min · 2 diagrams
- No. 19Rust VMM Ecosystem and Project TemplatesPurpose: Map the rust vmm crates to the responsibilities of a VMM, show how Firecracker and Cloud Hypervisor compose them, and give a workspace layout, event loop design, testing strategy and starter skeleton for...17 min · 2 diagrams
- No. 21VirtualizationPurpose: Root navigation map for learning hardware virtualization deeply enough to build a production minded Rust VMM on KVM for x86 64, ARM64, and RISC V. Virtualization A virtual machine monitor is a user space...4 min · 1 diagram