virt-df, virt-mem and bitmatch

One of the "self-evident truths" about virtualization is that in order to monitor aspects of the virtual machine such as free disk space / free memory / running processes / etc, you need to run some software inside the virtual machine. That's how the expensive proprietary virt systems such as VMWare work. The alternative to running monitoring software in the virtual machine would be to try to look at the disk image and memory image of the virtual machine and try to make sense of it, and that is usually regarded as an intractable, unimplementable nightmare.

That's not a truth any more.

A few months ago I proved with virt-df that it's perfectly possible to parse the disk image of a virtual machine to find out how much free disk space is left. And although in theory this could get inconsistent results, in practice it works well. Virt-df supports Linux, Windows NTFS, and complex partitioning schemes such as LVM as well as simple DOS-style partitions.

Now I'm proving with the virt-mem toolkit that we can snoop on the live memory image of running virtual machines and pull out such interesting details as the process list, the network configuration, the kernel messages (dmesg), and the kernel version strings.

So how is this possible?

Of course I'm writing these tools in OCaml, which allows me to express complicated ideas in a very few lines of code, yet is as fast as optimized C code. But OCaml alone isn't sufficient because all of the disk and memory images that we're parsing are made from binary structures and the base language doesn't handle binary structures very well. OCaml though does allow programmers to change and extend the base language using LISP-style macros -- these are language extensions that become one with the original language, but enable you to extend it in arbitrary and unexpected ways.

I have already extended OCaml by adding the entire, complete PostgreSQL SQL syntax through a macro, so this should be easy.

The key to being able to parse hundreds of different variations of the Linux task_struct (process table entry) so we can print process lists, was to look at a successful functional language which already supports bitstrings directly. Erlang is used in the telecom industry to parse binary network protocols and has a rich syntax for parsing and assembling bitstrings. I copied Erlang's syntax and using macros added it directly to OCaml. The result is the OCaml bitmatch project, hosted on Googlecode and with lots of examples and documentation.

Bitmatch is now a very advanced and powerful system for dealing with binary structures (more powerful than what the Erlang architype offered). As an example, how much code would you expect to write in order to take an ext3 partition, find the superblock and print out the number of free blocks from the superblock? (A poor man's 'df' command) You'll need to parse the Linux kernel header file <ext3_fs_sb.h> to get the fields, their offsets, sizes, endianness, etc. Then you'll need to load the superblock from disk and parse it using bit operators.

The answer in bitmatch is just 9 lines of code:

  (* bitmatch-import-c ext3.c > ext3.bmpp *)
  open bitmatch "ext3.bmpp"

  let () =
    let fd = Unix.openfile "/dev/sda1" [Unix.O_RDONLY] 0 in
    let bits = Bitmatch.bitstring_of_file_descr_max fd 4096 in

    bitmatch bits with
    | { :ext3_super_block } ->
        printf "free blocks = %ld\n" s_free_blocks_count
    | { _ } ->
        printf "/dev/sda1 is not an ext3 filesystem\n"