Building xen-arm-builder from source


I’m in Morocco for the first mirageos hackathon. One of the things I’ve done here is build xen-arm-builder from source and it was a big challenge. Here’s my general notes and pointers to help you get over this hump. I assume you know what mirageos/xen are.

Much MUCH thanks to Thomas Leonard and Mindy Preston for lending me a cubieboard2 and helping me with all the debugging, compiling issues; wouldn’t have been able to do it without them.

Machine Setup

I did this on Ubuntu 15 and was deploying to a cubieboard2.

Example Workflow

Assuming that you are connecting the cubieboard2 to your laptop over a serial connection then you can get a shell on the machine with

$ screen -h 10000 /dev/ttyUSB0 115200

Then say you are using mirage-skeleton, here’s an example flow.

$ git clone
$ cd mirage-skeleton
$ make configure MODE=xen
$ cd console
$ make
$ sudo xl create console.xe

Yay Unikernel on Cubieboard2!

Troublesome issues

I had many, many issues in building the code from source, here are some troubleshooting steps that might help you as well.

  1. Be sure to have fast internet because building from source including making a clone of the Linux source, ouch.
  2. OPAMVERBOSE=1 opam <anything> is incredibly useful.
  3. Ubuntu 15 will install a gcc-5.0 version of the cross-compiler, be sure to do make build CC=arm-linux-gnueabihf-gcc-4.8 instead of a plain make build.
  4. Be sure to not use the 4.02.0 compiler, the cubieboard2 is super slow and that OCaml compiler had a performance bug, use the 4.02.3 compiler instead.
  5. Check top to see some kind of xp binary, it eats the CPU and that really slows down compilation, best to kill them.
  6. Do make sure that the time on the machine is correct, otherwise you’ll have weird hanging by opam which will seem like its building but actually its just stuck on a timing issue, aka be sure to do: sudo ntpdate before doing anything with mirage/opam/aptitude.
  7. If you get odd errors like:
Parsing config from console.xl
xc: error: panic: xc_dom_core.c:185: failed to open file: No such file or directory: Internal error
libxl: error: libxl_dom.c:377:libxl__build_pv: xc_dom_kernel_file failed: No such file or directory
libxl: error: libxl_create.c:1022:domcreate_rebuild_done: cannot
(re-)build domain: -3

Then check that the libraries are correctly set in /usr/lib, check with

$ sudo strace -e open xl create console.xl

and either cp or ln them correctly.

  1. I recommend just installing all the depopts of all the mirage projects, there are some bugs in mirage opam files that don’t install other needed packages.