Develop a fast, reliable and deterministic concept for a virtual communication mechanism for a real-time virtualised system.

Our client, a USA based multinational OEM, contracted Emutex to develop a proof of concept inter virtual machine communication mechanism for a real-time virtualised software system.

The primary requirements were that the system should be very fast, reliable and deterministic and use the KVM hypervisor with QEMU. We recommended that the "Nahanni" shared memory device for QEMU be used as a way for virtual machines to easily share memory using the idea of PCI memory mapped I/O which is very familiar to all operating systems. Nahanni presents itself as a virtual graphics device to the virtual machine with a mapped memory region of a configurable size. This mapped region is also directly mapped into the hypervisor memory space using mmap().

Emutex developed an IPC layer over the shared memory interface that allowed passing of binary messages of arbitrary size between hypervisor and guest OS. The IPC layer featured lock-less interactions by separating each memory region into two areas which were designated as read-only and write-only respectively by the hypervisor and with opposite designations for the guest. This meant there was no contention for memory access and therefore no locks required which added to the determinism of the solution. Status updates were made by updating 32-bit values in a header of each region to inform the peer of the presence of a new message or response. Reads and writes of 32-bit values to memory are by their nature atomic once properly aligned so again no locking was required to access these offset values.

The Nahanni driver also supports virtualised interrupts to indicate the presence of shared memory region updates but in this particular scenario this facility was not used. Instead guests polled for updates in between deterministic workloads which ensured the latency of an interrupt did not interfere with deterministic workloads.