Combining Virtualbox and ZFS for Ubuntu & FreeBSD

The Software

The hardware you can find here: Cheap and Unusual Home PC Setup

The Ryzen 3 Host OS

The Host-OS is: Ubuntu 20.04 LTS, a minimal install on ZFS.

After install, I delete zsys, since I strongly despise the double administration, they introduced for snapshots. ZFS administrates the snapshots and Ubuntu-zsys has it own administration. That is typically an architecture begging for problems. However if you purge zsys, ZFS will work perfectly and with some very simple scripts you can manage all snapshots (create, destroy and rollback) and you can choose your own convention for snapshot names. You can even add a script for creating an additional user in the right ZFS dataset structure, which is completely missing from Ubuntu-ZFS.

The Virtual Machines (VM)

The main VMs use ext4 on top of the Host-ZFS, it is running from the SP nvme-SSD (3400/2300 MB/s), which has the following advantages:

  1. The vdi-files are compressed by ZFS (typically at a 1.8 ratio), resulting in less space, less IO operations and the ZFS memory cache (3 GB) is compressed too. Boot times of Linux Distros are between 7 and 13 seconds. Windows 10 Pro requires ~45 seconds. Memory cache (L1ARC) hit rates are ~98%
  2. I snapshot the VM dataset, thus all VMs, once/week. I can find the older VM vdi-files in the .zfs snapshot directory of that dataset. If needed I fall back on the copies on laptop or backup-server using the samba file service.
  3. Every Saturday I backup to backup-server and laptop with the incremental send|ssh receive ZFS function. Since ZFS only send the modified and compressed records, it is much faster than rsync, especially for the large Virtual Machine files. I do it, since more than one year now.

The Linux VMs, that boot fastest, are Xubuntu and Ubuntu Cinnamon, both boot in ~7 seconds.

I have almost all activated Windows releases from Windows 1 to 10, except 8 and 8.1 and I have all Ubuntu LTS releases starting with 4.10 :slight_smile:

I have many VMs as you can see from the photo. However the main Virtual Machines are:

  1. My Office VM is Ubuntu Mate 20.04 LTS. It boots in ~10 seconds, but is loaded always and it runs:
  • Transmission for downloads of torrents.
  • Evolution for email, calendar and tasks.
  • WhatsApp on ICE (Peppermint tool)
  • Browsing and rsync to my Samsung Galaxy S5 SD Card (64 GB) for my “off-site” backup with KDE Connect.
  • Libre Office and other Apps.
  1. My Banking VM Ubuntu 16.04 LTS will remain unchanged for another year out of respect for my good times with Unity. It boots in ~10 seconds from a vdi file encrypted by Virtualbox. I use it to pay with Paypall and I use it for my banking applications.

The Ubuntu Host Desktop and Virtualbox

  1. My Music VM has been moved in 2020 from Windows XP to Ubuntu Studio 20.04 LTS. Studio boots in ~10 seconds. I’ve detected a great audio file converter ‘soundKonverter’ and that one is lightning fast, because it use all cores of my CPU, instead of one core in a loop. Together with Lollypop (Web LookUp), Ex-Falso (tag changes also in “batch”), I have three great tools to clean up my ancient music collection, copies from my LPs and CDs, that I started with Windows Media Player in 2004.
    For my family videos I use OpenShot and for the photos mainly Shotwell. Ubuntu Studio installed DarkTable and I intend to start using it after a video course.

  2. My VM for experiments is Ubuntu 20.04, it boots in ~13 seconds. I tried DOSBOX and I love it. DOSBOX now starts-up in DOSSHELL and it plays my two ancient DOS games Winter and Wolfenstein, the real one, the original! The VM also plays the Linux games, like Extreme Tux Racer and SuperTuxKart, using the VBox accelerated 3D graphics.

  3. My VM with Windows 10 Pro is needed for my Dutch TV viewer. It boots in ~40 seconds.

The ones you can see in the photo, run from the nvme-SSD, the others, often old stuff, run from the striped datapool Dpool, which uses a WD Black 1TB HDD and a Seagate 500GB HDD (combined throughput ~ 250 MB/s). Boot times of VMs stored on Dpool are between 30 and 80 seconds. For system like Windows 95, that is perfect, since you do not want to spoil its true experience by booting in 5 seconds :slight_smile:

That Dpool has two datasets;

  • VM-Archives for additional often older virtual machines, like Windows for Workgroups 3.11, Windows 98, Ubuntu 4.10 and 10.04 but also Linux Mint 18.3, Peppermint 7 and 9, etc. Once/week I run the updates on all those OSes, that still receive updates .

  • Data, documentation, photos, videos, and my music collection. Also for this dataset I run an incremental backup to my backup server and laptop.

The “Data” dataset on the Ryzen is used with the property copies=2, which means that each record is stored twice and on different HDDs :slight_smile: Basically a Raid-1 subset inside Raid-0. ZFS will automatically correct any read errors and I assume, you can even loose one disk and keep that part/dataset workable.

The ‘Data’ dataset contains all private stuff like music, photos, family videos, HW manuals and documentation, software (exe, deb, iso), scripts and documentation. I do not need a SSD for that private stuff, since it did run in the past easily from a single IDE HDD.

The Backup Server

The system runs 32-bits FreeBSD 12.1 on ZFS with XFCE, XRDP and Conky and it is powered on for <1 hour/week, so it ages for say ~1 week/year :slight_smile: The screenshot is from Remmina running on my Ryzen PC, that I use to control and update the backup server.

Ubuntu and FreeBSD “send|ssh receive” are incompatible with respect to the ZFS-feature “large-dnode”, but you can compensate by using “dnodesize=legacy” on Ubuntu for the datasets you want to backup. Remember ZFS sends the physical compressed records and so I assume, it needs the same dnodesize for sender and receiver.

The Desktop of the Pentium 4 HT Back-up Server

The system is connected to a 1 Gbps network, but in practice it runs at ~20 MB/s, due to a CPU load of 95% on one one the HW threads of the Pentium 4 HT (3.0 GHz).

The HP Elitebook 8460p

The screen shows the desktop of my i5 laptop. The transfer speed is higher because there is less of a CPU bottleneck, normally it is ~70 MB/s.

The combination SSHD and ZFS is a pleasant surprise after all the bad SSHD reviews. The system boots in ~23 seconds and even the most used VMs boot in that time due to memory cache (1.5 GB, but it stores ~2.6 GB) and built-in solid state cache of the SSHD (8 GB, but it stores ~14 GB). So after a short time, you run the VMs from Memory cache or Solid State cache, resulting in instantaneous response time.

Screenshot from 2020-07-04 13-52-30
The Desktop of the Laptop.

Moving the VMs from R3 to i5 requires the following facilities:

For almost all VMs everything related to conky is run from a folder (Vbox shared library) in the Host home directory to take into account the differences in HW.

Other issues are:

  • Screen resolution, which is different on laptop and desktop, but it is almost always dealt with by Virtualbox in full screen mode.
  • Virtualbox will prompt the first time and propose to use the correct network-card-id (e.g. enps08) for the laptop.
  • Keyboard layout US or BE, which are installed both in all VMs and the choice is in the panel. For some older VMs, I have to remember the relevant key positions of the US keyboard typing the login password on the BE keyboard :frowning:
1 Like

This was a interesting run down, thanks for sharing.

Are you thinking about Qubes OS?