Skip to content

RISC-V Virtualization and the Hypervisor Extension

22 min4208 words3 diagrams

Source: Victor Bona's Obsidian Compendium snapshot, Knowledge base/Virtualization/16 RISC-V Virtualization and the Hypervisor Extension.md.

Purpose: explain how the RISC-V Hypervisor (H) extension virtualizes privilege modes, memory, traps, interrupts and timers, how KVM RISC-V exposes it, and how to build and test a RISC-V target for your Rust VMM without owning hardware.

RISC-V Virtualization and the Hypervisor Extension

Related notes: 02 x86-64 Architecture for Hypervisor Authors, 03 Intel VT-x and AMD-V Hardware Virtualization, 04 KVM API and the vCPU Run Loop, 05 Memory Virtualization EPT NPT and Guest Memory, 06 Interrupts Timers and Interrupt Controller Virtualization, 09 Booting Guests Firmware and the Linux Boot Protocol, 13 Security Isolation and Attack Surface, 15 ARM64 Virtualization EL2 VHE and GIC, 17 Hypervisor Architectures Compared, 18 Rust VMM Ecosystem and Project Templates.

Why RISC-V virtualization is worth learning now

RISC-V is the only major ISA whose hypervisor support was designed after x86 and Arm had shipped theirs, and it shows: the H extension is small, orthogonal and classically virtualizable. It is also the least mature ecosystem: hardware with the H extension is only now becoming common (RVA23, ratified October 2024, makes H mandatory for application processors), and parts of the interrupt and confidential-computing stacks are still settling.

Why it matters for your VMM: supporting RISC-V forces a clean architecture abstraction. The register API, interrupt controller, firmware interface and boot protocol all differ from x86 and arm64; if your VMM supports all three, its core (memory, virtio, event loop) is genuinely portable.

Privilege modes and the virtualization mode bit

Base RISC-V has three privilege levels: Machine (M), Supervisor (S), User (U). The H extension adds a virtualization mode bit V that is orthogonal to the privilege level.

ModeVPrivilegeRunsAnalogue
M0MachineFirmware (OpenSBI), platform runtimeArm EL3, x86 SMM-ish firmware
HS0Hypervisor-extended supervisorHost kernel plus KVM (like Arm VHE: the host OS is the hypervisor)Arm EL2 with VHE, x86 VMX root ring 0
U (host)0UserHost userspace, your VMMHost ring 3 / EL0
VS1Virtual supervisorGuest kernelx86 VMX non-root ring 0, Arm EL1
VU1Virtual userGuest userspaceGuest ring 3, Arm EL0

Key design choices:

  • HS-mode is S-mode with extra CSRs. An S-mode kernel runs unmodified in HS-mode; the hypervisor-specific state lives in new h* CSRs. This is like Arm VHE from day one: there is no nVHE split.
  • VS-mode sees "normal" S-mode CSR names. When V=1, accesses to sstatus, stvec, satp and friends are transparently redirected to the vs* copies. The guest kernel needs no paravirtualization.
  • The architecture is classically virtualizable in the Popek-Goldberg sense: every sensitive instruction traps when executed in VS/VU mode (or is redirected), so trap-and-emulate works without binary translation.
Rendering diagram...

The H extension CSRs

HS-level hypervisor CSRs

CSRPurpose
hstatusHypervisor status: where the last trap came from, guest config (see field table)
hedelegWhich synchronous exceptions from VS/VU are delegated to VS-mode (guest handles them directly)
hidelegWhich interrupts are delegated to VS-mode
hie, hipHypervisor-level interrupt enable / pending (VS interrupts, SGEI)
hvipHypervisor injects virtual interrupts into VS (VSSIP, VSTIP, VSEIP bits)
hgatpG-stage (guest physical to host physical) translation root, mode and VMID
htvalFaulting guest physical address shifted right by 2 on guest-page faults
htinstTransformed (decoded and normalized) trapping instruction, so the hypervisor need not read guest memory to decode it
hcounterenWhich counters VS-mode may read
htimedelta (htimedeltah on RV32)Offset added to time when read in VS/VU mode
hgeie, hgeipGuest external interrupt enable / pending, one bit per guest interrupt file (AIA)
henvcfgEnvironment config for VS-mode (cache-block ops, Svpbmt, Sstc enable via STCE, etc.)
hvictl, hvien, hviprio1/2AIA virtual interrupt control and priorities (with Ssaia)

VS-level CSRs (the guest's supervisor state)

CSRSubstitutes for when V=1
vsstatussstatus
vsie, vsipsie, sip
vstvecstvec
vsscratchsscratch
vsepcsepc
vscausescause
vstvalstval
vsatpsatp (VS-stage page table root)
vstimecmp (Sstc)stimecmp
vsiselect, vsireg, vstopei, vstopi (AIA)siselect, sireg, stopei, stopi

On a world switch KVM saves and restores the vs* CSRs; while the guest runs, its kernel's csrr sstatus actually reads vsstatus.

hstatus fields (RV64)

FieldBitsMeaning
VSBE5VS-mode big-endian data accesses
GVA6stval holds a guest virtual address on this trap
SPV7Previous virtualization mode: 1 if the trap came from VS/VU. sret with SPV=1 enters the guest
SPVP8Privilege of the guest at trap time (VS or VU), used by HLV/HSV
HU9Allow HLV/HSV hypervisor load/store instructions from U-mode
VGEIN17:12Selects which guest external interrupt file (IMSIC) is wired to VSEIP
VTVM20Trap VS-mode satp accesses and SFENCE.VMA
VTW21Trap VS-mode WFI after a timeout (KVM uses it to deschedule idle vCPUs)
VTSR22Trap VS-mode SRET
VSXL33:32VS-mode XLEN (RV64 or RV32 guest)

M-mode also gains fields relevant to hypervisors: mstatus.MPV and mstatus.GVA, plus mtval2 and mtinst mirroring htval and htinst when traps go to M-mode.

New exception and interrupt codes

CodeTypeMeaning
10ExceptionEnvironment call from VS-mode (guest SBI call)
20ExceptionInstruction guest-page fault
21ExceptionLoad guest-page fault
22ExceptionVirtual instruction (a privileged instruction that is legal in HS but not allowed in VS/VU given current config)
23ExceptionStore/AMO guest-page fault
2InterruptVirtual supervisor software interrupt (VSSI)
6InterruptVirtual supervisor timer interrupt (VSTI)
10InterruptVirtual supervisor external interrupt (VSEI)
12InterruptSupervisor guest external interrupt (SGEI, AIA guest file signalled)

The "virtual instruction" exception is a RISC-V specialty: it distinguishes "illegal in any mode" (illegal instruction, delegated to guest) from "legal, but the hypervisor must emulate it in this configuration" (virtual instruction, goes to HS).

Hypervisor instructions

InstructionUse
HFENCE.VVMAFlush VS-stage TLB entries (guest virtual), optionally by ASID
HFENCE.GVMAFlush G-stage entries, optionally by VMID and guest physical address
HLV.*, HLVX.*, HSV.*Load/store from HS (or U with hstatus.HU) using the guest's two-stage translation, as if executed by the guest. Used to read the trapping instruction or guest memory by guest virtual address

Trap delegation: who handles what

RISC-V uses explicit delegation registers at each level. Traps go to M by default; M delegates to HS (medeleg, mideleg); HS delegates to VS (hedeleg, hideleg).

Rendering diagram...

What KVM typically delegates to the guest (so they never cause exits): instruction/load/store misaligned, breakpoints, ecall from VU (guest syscalls), and ordinary (VS-stage) page faults. What it keeps: ecall from VS (SBI), guest-page faults (G-stage), virtual instruction exceptions, and interrupts not targeted at the guest.

Compare with x86, where exception bitmap and VM-exit controls in the VMCS serve the same purpose, and Arm, where HCR_EL2 trap bits do.

G-stage translation

RISC-V calls the second stage of address translation the G-stage (guest physical to supervisor physical). The guest's own page tables (vsatp) are the VS-stage.

PropertyDetail
Root registerhgatp with fields MODE, VMID, PPN
Modes (RV64)Bare (0), Sv39x4 (8), Sv48x4 (9), Sv57x4 (10)
Mode (RV32)Sv32x4
"x4" meaningGuest physical address is 2 bits wider than the corresponding Sv mode (Sv39x4 covers a 41-bit GPA). The root table is 16 KiB (four contiguous pages) and must be 16 KiB aligned
Leaf PTE requirementsG-stage PTEs must have the U bit set (all guest accesses are treated as user accesses at G-stage)
TagVMID (up to 14 bits on RV64; implementations may support fewer, discoverable by writing all ones)
Fault reportingGuest-page fault codes 20, 21, 23; htval = GPA >> 2; stval = guest virtual address when hstatus.GVA = 1; htinst = transformed instruction or a pseudo-instruction for implicit VS-stage page table accesses
TLB maintenanceHFENCE.GVMA by VMID and/or GPA

KVM exposes the G-stage mode to userspace through the satp_mode config register for the guest's VS-stage and sizes the G-stage from host capabilities; your VMM's guest memory map must fit the guest physical width KVM supports on the host.

MMIO decoding

On a load or store guest-page fault to an address without a memslot, KVM needs to know the access width and register. It uses htinst when the hardware provides a transformed instruction; otherwise it reads the faulting instruction from guest memory with HLVX. Either way, the result reaching your VMM is a normal KVM_EXIT_MMIO with address, length and data, same as on x86 and arm64. There is no RISC-V equivalent of Arm's ISV restriction for ordinary loads and stores, though compressed and misaligned accesses add corner cases KVM handles.

SBI: the hypervisor-guest interface

The Supervisor Binary Interface (SBI) is how S-mode software calls into the next more-privileged layer. On bare metal, the host kernel calls OpenSBI in M-mode. In a VM, the guest kernel calls SBI with ecall from VS-mode, which traps to HS-mode, and KVM plays the role of the SBI implementation.

Calling convention: extension ID (EID) in a7, function ID (FID) in a6, arguments in a0 to a5; returns an error code in a0 and a value in a1.

ExtensionEIDPurposeHandled by
Base0x10Probe extensions, spec and implementation version, mvendorid/marchid/mimpidKVM (values configurable via config regs)
Legacy v0.10x00 to 0x0FOld console putchar/getchar, timer, IPIKVM; console calls forwarded to userspace
TIME0x54494D45sbi_set_timerKVM (or unused by guest when Sstc present)
IPI (sPI)0x735049Send IPI to hart maskKVM
RFENCE0x52464E43Remote fence.i, sfence.vma, hfenceKVM
HSM0x48534DHart start, stop, suspend, status: SMP bring-upKVM
SRST0x53525354System reset / shutdownKVM converts to KVM_EXIT_SYSTEM_EVENT
PMU0x504D55Performance countersKVM, backed by host perf
DBCN0x4442434EDebug console read/write buffersForwarded to userspace (KVM_EXIT_RISCV_SBI) when enabled
SUSP0x53555350System suspendKVM / userspace
STA0x535441Steal-time accounting shared memoryKVM
FWFT0x46574654Firmware features (misaligned delegation, pointer masking, landing pads, shadow stack)KVM
SSE, MPXY, DBTR(SBI v3.0)Software events, message proxy, debug triggersPartially; check KVM_RISCV_SBI_EXT_* for your kernel
Experimental / vendor ranges0x08000000 and up / 0x09000000 and upCustomForwarded to userspace

SBI v2.0 was ratified in early 2024 and SBI v3.0 on 2025-07-16 (adding SSE, MPXY, DBTR, FWFT). KVM exposes the set of SBI extensions as one-reg registers (KVM_REG_RISCV_SBI_EXT), so your VMM can disable extensions it does not want the guest to see, and must save that configuration for migration.

Handling an unhandled SBI call in the VMM

kvm-ioctls 0.25 has no dedicated VcpuExit variant for RISC-V SBI; it reports VcpuExit::Unsupported(KVM_EXIT_RISCV_SBI) and you read the riscv_sbi member of kvm_run yourself.

use kvm_bindings::KVM_EXIT_RISCV_SBI;
use kvm_ioctls::{VcpuExit, VcpuFd};

const SBI_EXT_DBCN: u64 = 0x4442_434E;
const SBI_EXT_0_1_CONSOLE_PUTCHAR: u64 = 0x01;
const SBI_SUCCESS: u64 = 0;
const SBI_ERR_NOT_SUPPORTED: u64 = (-2i64) as u64;

pub fn run_once(vcpu: &mut VcpuFd, console: &mut impl std::io::Write) {
    let reason = match vcpu.run() {
        Ok(VcpuExit::Unsupported(r)) => r,
        Ok(_other) => return, // MMIO, SystemEvent and others handled elsewhere
        Err(e) => panic!("KVM_RUN: {e}"),
    };
    if reason != KVM_EXIT_RISCV_SBI {
        panic!("unexpected exit {reason}");
    }
    let run = vcpu.get_kvm_run();
    // SAFETY: exit_reason is KVM_EXIT_RISCV_SBI, so riscv_sbi is the active union member.
    let sbi = unsafe { &mut run.__bindgen_anon_1.riscv_sbi };
    match sbi.extension_id {
        SBI_EXT_0_1_CONSOLE_PUTCHAR => {
            let _ = console.write_all(&[sbi.args[0] as u8]);
            sbi.ret[0] = SBI_SUCCESS;
        }
        SBI_EXT_DBCN => {
            // Function 0 is console_write(num_bytes, base_lo, base_hi): read from guest memory,
            // validating the range exactly as for any guest-supplied buffer.
            sbi.ret[0] = SBI_ERR_NOT_SUPPORTED;
            sbi.ret[1] = 0;
        }
        _ => {
            sbi.ret[0] = SBI_ERR_NOT_SUPPORTED;
            sbi.ret[1] = 0;
        }
    }
}

Treat every SBI argument as hostile input (see 13 Security Isolation and Attack Surface): DBCN passes guest physical buffer addresses and lengths.

KVM RISC-V from the VMM's point of view

KVM RISC-V was merged in Linux 5.16 (late 2021). It follows the arm64 style: no KVM_GET_REGS/KVM_SET_SREGS structs (they are empty on RISC-V), everything goes through one-reg.

Register ID encoding

From the KVM API documentation and arch/riscv/include/uapi/asm/kvm.h: IDs are 0x8030 0000 TT SS IIII on 64-bit hosts (0x8020... on 32-bit), where TT is the group type in bits 31:24, SS an optional subtype in bits 23:16, and the low bits an index (usually offsetof(struct, field) / sizeof(unsigned long)).

TypeGroupContents
0x01KVM_REG_RISCV_CONFIGisa (legacy bitmap), zicbom_block_size, mvendorid, marchid, mimpid, zicboz_block_size, satp_mode, zicbop_block_size
0x02KVM_REG_RISCV_COREstruct user_regs_struct (pc, ra, sp, gp, tp, t0 to t2, s0, s1, a0 to a7, s2 to s11, t3 to t6) plus mode (S or U)
0x03KVM_REG_RISCV_CSRSubtypes: general (sstatus, sie, stvec, sscratch, sepc, scause, stval, sip, satp, scounteren, senvcfg), AIA, Smstateen, Zicfiss
0x04KVM_REG_RISCV_TIMERfrequency (read-only), time, compare, state (on/off)
0x05, 0x06KVM_REG_RISCV_FP_F, _FP_DF and D register files plus fcsr
0x07KVM_REG_RISCV_ISA_EXTOne register per ISA extension (KVM_RISCV_ISA_EXT_*): 1 enabled, 0 disabled; plus multi-register bitmaps
0x08KVM_REG_RISCV_SBI_EXTOne register per SBI extension (KVM_RISCV_SBI_EXT_*)
0x09KVM_REG_RISCV_VECTORV registers
0x0aKVM_REG_RISCV_SBI_STATESBI extension state (STA shared memory, FWFT features)

Examples from the kvm-ioctls test suite: core base 0x8030_0000_0200_0000 is pc, a0 is 0x8030_0000_0200_000a.

ISA extensions use KVM's own stable numbering (not the host's internal one): A=0, C=1, D=2, F=3, H=4, I=5, M=6, SVPBMT=7, SSTC=8, SVINVAL=9, ZIHINTPAUSE=10, ZICBOM=11, ZICBOZ=12, ZBB=13, SSAIA=14, V=15, and onward (the enum keeps growing; read it from the header for your kernel). Changing ISA or SBI extension registers is only allowed before the vCPU first runs (EBUSY afterwards).

Rust: configuring and booting a RISC-V vCPU

Checked against kvm-ioctls 0.25.x (set_one_reg(u64, &[u8]), get_one_reg(u64, &mut [u8]), get_reg_list(&mut RegList) available on riscv64).

use kvm_ioctls::VcpuFd;

const KVM_REG_RISCV: u64 = 0x8000_0000_0000_0000;
const KVM_REG_SIZE_U64: u64 = 0x0030_0000_0000_0000;
const TYPE_SHIFT: u64 = 24;

const fn reg(ty: u64, idx: u64) -> u64 {
    KVM_REG_RISCV | KVM_REG_SIZE_U64 | (ty << TYPE_SHIFT) | idx
}
const CONFIG: u64 = 0x01;
const CORE: u64 = 0x02;
const ISA_EXT: u64 = 0x07;

// Indices into struct user_regs_struct.
const PC: u64 = reg(CORE, 0);
const A0: u64 = reg(CORE, 10);
const A1: u64 = reg(CORE, 11);
// Index into struct kvm_riscv_config.
const SATP_MODE: u64 = reg(CONFIG, 6);
// KVM_RISCV_ISA_EXT_* numbering from the uapi header.
const ISA_EXT_V: u64 = reg(ISA_EXT, 15);

fn set(vcpu: &VcpuFd, id: u64, val: u64) {
    vcpu.set_one_reg(id, &val.to_le_bytes()).expect("set_one_reg");
}

fn get(vcpu: &VcpuFd, id: u64) -> u64 {
    let mut b = [0u8; 8];
    vcpu.get_one_reg(id, &mut b).expect("get_one_reg");
    u64::from_le_bytes(b)
}

/// Linux RISC-V boot protocol: a0 = hart ID, a1 = DTB physical address, MMU off, S-mode.
pub fn setup_boot_hart(vcpu: &VcpuFd, hart_id: u64, kernel_entry: u64, dtb: u64) {
    // Hide V from the guest (example of shaping the CPU model); must happen before first run.
    let _ = vcpu.set_one_reg(ISA_EXT_V, &0u64.to_le_bytes());
    set(vcpu, PC, kernel_entry);
    set(vcpu, A0, hart_id);
    set(vcpu, A1, dtb);
    let satp_mode = get(vcpu, SATP_MODE); // put the matching "mmu-type" in the DT
    let _ = satp_mode;
}

SMP: KVM keeps every vCPU except the one with index 0 powered off after creation; the guest starts secondaries with SBI HSM hart_start, which KVM implements. You only set registers on the boot hart.

The device tree must match KVM's view of the CPU

RISC-V guests discover ISA extensions from the device tree (riscv,isa-base and riscv,isa-extensions, or the older riscv,isa string) and the MMU type from mmu-type (riscv,sv39, riscv,sv48, riscv,sv57). Build those properties from the one-reg values you read back (ISA_EXT registers, satp_mode), not from the host's /proc/cpuinfo. If they disagree, the guest may execute an instruction KVM hid, which traps as illegal instruction inside the guest.

Minimum DT for a Linux guest on RISC-V:

NodeKey properties
/cpustimebase-frequency (read KVM_REG_RISCV_TIMER frequency)
/cpus/cpu@Ndevice_type = "cpu", compatible = "riscv", reg = &lt;hart id&gt;, riscv,isa-base, riscv,isa-extensions, mmu-type, child interrupt-controller with compatible = "riscv,cpu-intc", #interrupt-cells = 1
/memory@...RAM base and size
Interrupt controllerPLIC (sifive,plic-1.0.0) or AIA: APLIC (riscv,aplic) and IMSIC (riscv,imsics)
/chosenbootargs, stdout-path, initrd range
UART, virtio-mmioAs on arm64, interrupt specifiers per the chosen interrupt controller
/soc or root#address-cells = 2, #size-cells = 2

The kernel Image for RISC-V has a 64-byte header like arm64; load it 2 MiB aligned on RV64. See 09 Booting Guests Firmware and the Linux Boot Protocol.

Interrupts: PLIC, AIA, IMSIC and APLIC

RISC-V has two interrupt architectures, and which one you use defines your VMM's interrupt plumbing.

ComponentRoleVirtualization story
CLINT / ACLINTPer-hart timer and software interrupts (M-level)Replaced in VMs by SBI TIME/IPI or Sstc and AIA
PLICLegacy wired interrupt controllerNot virtualized in hardware; emulate in VMM, inject with KVM_INTERRUPT
AIA IMSICPer-hart MSI controller with interrupt files: one M-level, one S-level, and N guest (VS-level) filesGuest files map directly to vCPUs: devices write MSIs straight to a guest file; hstatus.VGEIN selects which file drives VSEIP
AIA APLICWired interrupt controller that can forward wired interrupts as MSIs to IMSICs (MSI mode) or deliver directly (direct mode)KVM emulates the APLIC in kernel, converting wired lines to MSIs into guest IMSIC files
Rendering diagram...

KVM AIA device

KVM implements AIA as a device: KVM_CREATE_DEVICE with KVM_DEV_TYPE_RISCV_AIA, configured with KVM_SET_DEVICE_ATTR:

GroupAttributeMeaning
KVM_DEV_RISCV_AIA_GRP_CONFIG (0)MODEEMUL (0: trap-and-emulate IMSIC), HWACCEL (1: use hardware guest files), AUTO (2: hardware when available, otherwise emulate)
IDSNumber of MSI identities, bounded by KVM_DEV_RISCV_AIA_IDS_MIN (63) and _IDS_MAX (2048)
SRCSNumber of wired sources on the APLIC (up to 1024)
GROUP_BITS, GROUP_SHIFT, HART_BITS, GUEST_BITSIMSIC address layout
KVM_DEV_RISCV_AIA_GRP_ADDR (1)ADDR_APLIC (0), ADDR_IMSIC(vcpu) (1 + vcpu)Guest physical base of APLIC (16 KiB, 4 KiB aligned) and each vCPU's IMSIC page (4 KiB)
KVM_DEV_RISCV_AIA_GRP_CTRL (2)CTRL_INITFinalize after all vCPUs exist
KVM_DEV_RISCV_AIA_GRP_APLIC (3)APLIC register offsetSave/restore APLIC state
KVM_DEV_RISCV_AIA_GRP_IMSIC (4)`(vcpu << 12)iselect`

With the AIA device in place, wired device interrupts are injected with KVM_IRQ_LINE or irqfd (routed to APLIC sources) and MSIs with KVM_SIGNAL_MSI or MSI routes, the same shapes you use on x86 and arm64. Without AIA (older hosts, or aia=none in QEMU), you emulate a PLIC in userspace and raise or lower the guest's external interrupt with KVM_INTERRUPT (KVM_INTERRUPT_SET / KVM_INTERRUPT_UNSET), which costs an exit per claim/complete.

Hardware guest files are a finite resource: each hart's IMSIC has a fixed number of VS-level files (QEMU's aia-guests=N). When vCPUs outnumber them on a host CPU, KVM falls back to software emulation (AUTO mode) or fails to run (HWACCEL), which you see as a KVM_EXIT_FAIL_ENTRY with the KVM_EXIT_FAIL_ENTRY_NO_VSFILE reason.

Timers: SBI TIME vs Sstc

ApproachGuest actionCost
SBI TIMEGuest calls sbi_set_timer(deadline) with ecallExit to KVM for every timer program; KVM arms an hrtimer and injects VSTIP via hvip
Sstc (ratified 2021, mandatory in RVA23)Guest writes stimecmp, which maps to vstimecmp in VS-modeNo exit; hardware raises VSTIP when time + htimedelta &gt;= vstimecmp

htimedelta provides the per-VM time offset, so guest time can start at zero and survive migration. For snapshots, save and restore the KVM_REG_RISCV_TIMER registers (time, compare, state); frequency is read-only and comes from the host timebase, so migration between hosts with different timebase frequencies is not transparent (the same issue as arm64's CNTFRQ_EL0).

Hide or expose Sstc deliberately via KVM_RISCV_ISA_EXT_SSTC. Guests detect Sstc from the device tree ISA extensions and stop using SBI TIME, so the DT must reflect the one-reg configuration.

Testing without hardware: QEMU with the H extension

You can develop the whole RISC-V path on an x86 or arm64 laptop with nested emulation: QEMU TCG emulates a RISC-V machine with the H extension; inside it a Linux host loads KVM; inside that your VMM runs a guest. It is slow (tens to hundreds of times slower than native) but fully functional and deterministic enough for CI.

QEMU's virt machine enables the hypervisor extension on its default CPU and supports AIA with aia=aplic-imsic and aia-guests=N (number of VS-level IMSIC files per hart).

# Outer: emulated RISC-V host with H extension and AIA guest files.
qemu-system-riscv64 \
  -machine virt,aia=aplic-imsic,aia-guests=4 \
  -cpu rv64,h=true,sstc=true \
  -smp 4 -m 8G -nographic \
  -bios default \
  -kernel host-Image \
  -append "root=/dev/vda rw console=ttyS0" \
  -drive file=host-rootfs.img,format=raw,if=none,id=hd0 \
  -device virtio-blk-device,drive=hd0 \
  -netdev user,id=n0,hostfwd=tcp::2222-:22 -device virtio-net-device,netdev=n0

Inside the emulated host:

ls -l /dev/kvm                       # KVM RISC-V available?
dmesg | grep -i kvm                  # look for "hypervisor extension available" and AIA mode
grep -m1 isa /proc/cpuinfo           # confirm 'h' and extensions such as sstc, ssaia
./my-vmm --kernel guest-Image --cmdline "console=ttyS0 earlycon=sbi" --mem 512M

Workflow tips:

TipWhy
Cross-compile the VMM with target riscv64gc-unknown-linux-gnu (or musl) and copy it inNative compile under TCG is very slow
Use earlycon=sbi in the guest command lineGuest prints via SBI legacy console or DBCN, which you handle before your UART works
Run KVM selftests (tools/testing/selftests/kvm) in the emulated host firstSeparates host KVM problems from VMM bugs
Use kvmtool or QEMU-with-KVM inside as a reference VMMCompare DT and register setup when your guest hangs
Attach gdb to the outer QEMU (-s -S) to inspect the host kernel, and to your VMM with gdbserverTwo-level debugging
Test both aia=none (PLIC path) and aia=aplic-imsicReal hardware will vary
Spike (the ISA reference simulator) for instruction-level questions about H-extension semanticsGolden model, not a system for running KVM workloads comfortably

Ecosystem maturity (late 2026)

LayerStatus
H extension specificationRatified (Privileged spec 1.12, 2021)
RVA23 profileRatified 2024; H (via the Sha bundle) and Sstc mandatory for RVA23S64; AIA not mandated by the profile itself
AIA specificationRatified 2023
SBIv2.0 (2024) and v3.0 (2025-07-16) ratified
RISC-V IOMMU specificationRatified 2023; QEMU iommu-sys device and Linux driver exist; little hardware yet
KVM RISC-VUpstream since Linux 5.16; one-reg ISA and SBI extension control, AIA device, Sstc, SBI PMU, steal time
QEMUTCG emulation of H, AIA, IOMMU; KVM accelerator for RISC-V hosts
kvmtoolSupports RISC-V KVM guests
rust-vmm kvm-ioctls / kvm-bindingsriscv64 supported (one-reg, reg list, device attrs); no RISC-V-specific VcpuExit variants for SBI
seccompilerriscv64 target supported
Cloud HypervisorExperimental riscv64 support (verify current release notes)
FirecrackerNo RISC-V support
Hardware with HEmerging: early boards advertise H; RVA23-class SoCs (for example SpacemiT K3, with Linux enablement posted in late 2025) are the first broad wave. Check the device tree ISA string and AIA presence before buying
Confidential computing (CoVE, Supervisor Domains, Smmtt)Specifications in progress plus research implementations; no stable KVM UAPI

Expect bugs and missing features in the host stack. Pin known-good host kernel and QEMU versions in your CI and record them in your repo.

Comparison with x86-64 and arm64

Concernx86-64arm64RISC-V
Hypervisor privilegeVMX root / SVM host mode, orthogonal to ringsEL2 (VHE runs host kernel there)HS-mode (S-mode plus H CSRs), V bit selects guest
Guest state containerVMCS / VMCB in memorySystem registers switched by KVMvs* CSRs switched by KVM
Second stageEPT / NPTStage-2, VTTBR_EL2G-stage, hgatp (Sv39x4 and friends)
Trap configurationVMCS controls, exception bitmapHCR_EL2 and friendshedeleg, hideleg, hstatus bits
MMIO decode helpInstruction emulator in KVMESR ISS (ISV)htinst or HLVX read
Firmware ABIACPI, CPUID, MSRs, hypercallsPSCI / SMCCC via HVCSBI via ecall
KVM register APIStruct ioctls (REGS, SREGS, MSRS)One-reg onlyOne-reg only
CPU model shapingKVM_SET_CPUID2Feature bits plus writable ID regsISA_EXT and SBI_EXT one-reg, config regs
Interrupt controllerLAPIC/IOAPIC in kernel, posted interruptsvGICv3/v4 deviceAIA device (APLIC plus IMSIC guest files) or userspace PLIC
TimerLAPIC, TSC deadline, kvmclockGeneric timer, CNTVOFF_EL2SBI TIME or Sstc, htimedelta
SMP bring-upINIT/SIPIPSCI CPU_ONSBI HSM hart_start
Hardware availabilityUniversalUniversalEmerging

Common pitfalls

PitfallSymptomFix
Device tree ISA string built from host /proc/cpuinfoGuest uses an extension KVM disabled: illegal instruction in guestBuild DT from ISA_EXT one-reg values
Setting ISA_EXT or SBI_EXT registers after first KVM_RUNEBUSYConfigure everything before running any vCPU
Missing timebase-frequency or wrong valueGuest clock runs at wrong speed, RCU stallsRead KVM_REG_RISCV_TIMER frequency
Starting secondaries yourselfDouble-started harts, crashesLeave them powered off; guest uses SBI HSM
Ignoring KVM_EXIT_RISCV_SBIGuest hangs on early console or custom SBI callHandle or return SBI_ERR_NOT_SUPPORTED
Using aia=aplic-imsic with too few guest files in QEMUKVM_EXIT_FAIL_ENTRY with no VS fileIncrease aia-guests or use AUTO mode
Assuming PLIC exists on AIA platformsNo external interruptsDetect and support both PLIC and AIA paths
G-stage size assumptions from x86Memslots above the supported GPA failKeep guest memory map under the host's G-stage limit
Testing only under TCGTiming-dependent bugs hidden by slownessAdd real hardware to CI when available; use fault injection
Forgetting SBI extension state in snapshotsRestored guest loses steal-time area or FWFT settingsSave all IDs from KVM_GET_REG_LIST, including SBI state
Assuming mode register is optionalGuest restarted in U-mode after restoreSave and restore KVM_REG_RISCV_CORE mode

Review checklist

  • VMM queries KVM_GET_REG_LIST and saves every register for snapshots.
  • ISA and SBI extension registers set deliberately before first run; DT generated from the read-back values.
  • Boot hart has pc, a0 = hart ID, a1 = DTB address; secondaries left to SBI HSM.
  • timebase-frequency and mmu-type match KVM's timer frequency and satp_mode.
  • Both interrupt paths considered: AIA device (APLIC plus IMSIC) and userspace PLIC with KVM_INTERRUPT.
  • AIA mode chosen consciously (AUTO for portability); IMSIC and APLIC addresses do not overlap other MMIO.
  • KVM_EXIT_RISCV_SBI handled with validated arguments; unknown extensions return SBI_ERR_NOT_SUPPORTED.
  • Timer registers saved and restored; Sstc exposure consistent with DT.
  • CI runs the RISC-V path under QEMU TCG with H and AIA, using pinned QEMU and host kernel versions.
  • seccomp filters built for the riscv64 syscall table (generic table, no legacy open).

Primary Sources

contactvictor.bona@hotmail.com
victor-bona · terminal

type help for commands · esc to close