Integration of Linux device driver software into popular open source applications.

Our client had developed a network packet processing acceleration chipset and contracted Emutex to integrate its associated Linux device driver software into popular open source applications. One key challenge we had to solve was to avoid expensive memory copy operations and to do so without modifying the open source applications.

HOur solution was to provide a pinned memory driver. Emutex developed a Linux kernel device driver to allocate non-paged non-cached physical memory on request. A user library providing a simple alloc, free, realloc interface was provided as a wrapper over this device driver. The open source application supported over-riding of the standard GNU malloc() and free() functions so it was straightforward to integrate the Emutex library without modification to the open source application.

The user library requested slab allocations of large fixed-size chunks of memory from the pinned memory driver and then mapped these into the user's virtual memory space using mmap(). Carving up the large chunks into smaller fragments as required by the user was done in the user library. As far as the open source app was concerned, virtual memory had been requested and provided as easily as if malloc() had been called. However, the key advantage of this scheme was that it allowed the user library to provide a virtual-to-physical (v2p) function for use by the hardware acceleration driver. The user library could easily compare pointers of any buffer provided to the v2p function to the regions that had been mapped using mmap(). The user library could then easily calculate the offset into the region and work out the corresponding physical memory address for use by hardware.

An additional feature of this slab allocator was that subdivision of large chunks into smaller sizes was always done on a size that was a power of two. So if, for example, a user requested 37 bytes, an allocation of 64 bytes was done. This ensured all memory chunks in a slab were the same size which eliminated memory fragmentation.