Using VirtualBox With Raw Disk Access

Dual booting sucks. Is irritating. Is inefficient. I remember the early days of my Linux experience when I dual booted. I had a common FAT32 partition to share files. In those days I wanted to use Linux systems but certain critical apps, such as email or MS Office, were in Windows only.

I no longer use Windows or Windows software for daily work flows. I haven’t for many years. My original need for dual booting no longer exists.

Yet access to multiple operating systems is a significant need and desire. Virtualization is the common tool for such access. Yet a traditional virtual machine (VM) with a virtual disk is not always the best approach and converting an existing physical system to virtual (P2V) is not always doable.

The virtualization magic to avoid dual booting is something called raw disk access. This is a handy feature for people wanting to use different operating systems on the same computer and not wanting to reboot.

I use raw disk access on several different systems.

While the techniques described here might work with KVM, the steps described apply only to VirtualBox.

Introduction

There are some caveats with using raw disk access rather than a virtual hard disk.

VirtualBox does not support booting Windows 7 with UEFI.
To avoid Windows license reactivation, tweaking the VirtualBox BIOS emulation is necessary.
Using a default VirtualBox MAC address likely will trigger Windows reactivation.
The Windows design is myopic and well known for disrupting existing boot loader configurations.

VirtualBox supports booting Windows 10 with UEFI, but I have not tested this. Additionally, the limitation with Windows 7 seems to be booting only.

Another potential caveat is wanting 3D acceleration or playing games. The optimal approach is using the VirtualBox pass-through feature and using a second video card for the guest system. Many folks do not need this support and the techniques described here should suffice.

Other considerations:

Typically Windows will be installed on at least two hard drive partitions.

While I have used a copy of an original MBR to boot a VM with raw disk access, now I use a GRUB boot ISO to boot those systems. While the affected disks might still use MBR, using a GRUB boot ISO avoids certain challenges.

Unlike a physical machine, using a GRUB boot ISO in a VM is painless and requires almost no action. The only time I need to remember to install the GRUB boot ISO is when I use other ISO images, such as updating the VirtualBox guest additions (GA). Some of the systems I support do not use the GA and I never need to remember the GRUB boot ISO.

In a dual boot environment all Linux systems use the same GRUB menu. In a raw disk VM each system needs its own boot defaults. Otherwise the boot loader will always boot the default system. When configuring a VM with raw disk access, typically access to other partitions is not allowed.

The VirtualBox user manual warns that concurrent access to disk partitions in this manner will corrupt data. Using raw disk access means configuring each VM to not have direct access to other partitions. Both Linux and Windows can “see” all partitions when running in a VM using raw disk access. Linux systems can read both native and NTFS. Windows can see the Linux partitions but is unable to read the contents. Nonetheless, the prudent action is not allowing any access.

The basic approach:

Ensure both operating systems boot fine using the respective boot loaders.
On the Linux side, create a VM for the Windows operating system.
Do not create a virtual hard drive.
Modify the VM BIOS emulation.
If available, copy the hard disk MBR as a file.
When necessary or desired, create a GRUB boot ISO to avoid virtual system boot problems.
Manually configure the virtual machine to use raw disk access.

Presumptions

Both operating systems are installed and dual booting works with no issues.

There must be sufficient RAM to concurrently run both the host and guest VM. Dual booting does not have this RAM limitation. I use the following simple RAM assignment criteria. For XP, assigning 512 MB to 1 GB of RAM to the VM usually is sufficient. For Windows 7, 1.5 GB or more. With Windows 10 2 GB or more. I am running 64-bit systems. Possibly the criteria is less for 32-bit guest systems.

Verify the CPU supports hardware virtualization extensions. Otherwise virtualization is painfully slow. In many BIOSes and UEFIs this option will be disabled by default. Look in the Security section of the BIOS. When enabled a Linux system will show the flags as enabled:

egrep ‘(vmx|svm)’ /proc/cpuinfo

The vmx flag is for Intel CPUs and the svm flag is for AMD CPUs.

Virtual machines are presumed to be installed at $HOME/VirtualBox VMs/, where $HOME is the full path of /home/username. This is easily modified to a shared location for multiple users, but is not part of this tutorial.

Create a Virtual Machine

Creating the VM is easy using the VirtualBox graphical user interface (GUI), but there is no GUI support for configuring raw disk access. Using raw disk access requires using the vboxmanagecommand to create a vmdk container. When using the VirtualBox GUI to create the VM, do not create a virtual hard disk.

Windows will not require any reactivation when the amount of RAM changes. Without installing more RAM, usually that means the amount of RAM Windows will see from within the VM will be about half of what is physically available.

Likewise with optical disk hardware changes.

Otherwise Windows might have a hissy fits when the underlying hardware changes. The hissy fits might mean reactivation of the license.

Modify the Virtual Machine BIOS Emulation

Before creating the vmdk container for raw disk access, be sure to emulate the hardware BIOS as much as possible. By default the BIOS emulation of a VirtualBox VM will be a generic Oracle system. With the hope of avoiding reactivation when running Windows inside the VM, spoof the machine hardware as much as possible.

A few BIOS parameters will not work with VirtualBox and must be deleted. As a test of the spoofed parameters, when the BIOS parameters are correct the VM will start to boot without errors despite not yet having any hard drive.

Use dmidecode to grab the BIOS information. Migrate that information dump into the VM’s /home/username/VirtualBox\ VMs/Name_of_the_VM.vbox configuration file.

Copy any SLIC table (/sys/firmware/acpi/tables/SLIC). To copy the table:

dd if=/sys/firmware/acpi/tables/SLIC of=$HOME/VirtualBox\ VMs/Name_of_the_VM/SLIC.bin

Edit the /home/username/VirtualBox\ VMs/Name_of_the_VM.vbox config file to reference the SLIC.bin file. For example,

<ExtraDataItem name="VBoxInternal/Devices/acpi/0/Config/CustomTable” value="/home/username/VirtualBox\ VMs/Name_of_the_VM/SLIC.bin"/>

Here is a snippet I once had when using Windows XP raw disk VM on my Thinkpad T400. The attached data is an example only. Do not copy and paste. The actual required data must be generated locally.

    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSFirmwareMajor” value="1"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSFirmwareMinor” value="1"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSReleaseDate” value="02/13/2009"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSReleaseMajor” value="2"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSReleaseMinor” value="7"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSVendor” value="LENOVO"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBIOSVersion” value="7UET61WW (2.07 )"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardProduct” value="6475MC2"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardSerial” value="VF26G93H25K"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardVendor” value="LENOVO"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardVersion” value="Not Available"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiChassisAssetTag” value="string:1204615"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiChassisSerial” value="Not Available"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiChassisType” value="0x0A"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiChassisVendor” value="LENOVO"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiChassisVersion” value="Not Available"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiProcManufacturer” value="GenuineIntel"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiProcVersion” value="Intel(R) Core(TM)2 Duo CPU     P8400  @ 2.26GHz"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemFamily” value="ThinkPad T400"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemProduct” value="6475MC2"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemSKU” value="Not Specified"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemSerial” value="L3CCB9Y"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemUuid” value="38AFFD81-4ACD-11CB-AAED-D1B8EB1C8C3F"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemVendor” value="LENOVO"/>
    <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiSystemVersion” value="ThinkPad T400"/>

Likewise with spoofing optical and hard drive firmware data. Here is a snippet for that data on the same T400:

    <ExtraDataItem name="VBoxInternal/Devices/piix3ide/0/Config/PrimaryMaster/ATAPIProductId” value="M149ACM5133"/>
    <ExtraDataItem name="VBoxInternal/Devices/piix3ide/0/Config/PrimaryMaster/ATAPIRevision” value="HX12"/>
    <ExtraDataItem name="VBoxInternal/Devices/piix3ide/0/Config/PrimaryMaster/ATAPIVendorId” value="HL-DT-ST DVDRAM GSA-U20N"/>
    <ExtraDataItem name="VBoxInternal/Devices/piix3ide/0/Config/PrimaryMaster/FirmwareRevision” value="ES1OA60W"/>
    <ExtraDataItem name="VBoxInternal/Devices/piix3ide/0/Config/PrimaryMaster/ModelNumber” value="Hitachi HTS543216A7A384"/>
    <ExtraDataItem name="VBoxInternal/Devices/piix3ide/0/Config/PrimaryMaster/SerialNumber” value="E2004240GHNT1M"/>

In the VirtualBox GUI, change the NIC MAC address to the actual physical MAC address of the computer. Do not use colons.

Long ago I wrote a shell script to automate this spoofing and merge the data into the VirtualBox config file. I never tested this script outside my own use case. Thus all standard disclaimers apply and the script comes with no warranty.

Likely the script will break for some people and will some tweaks because of my own local presumptions.

For XP users there is no reason to panic if the BIOS and MAC spoofing fail to avoid reactivation. The Microsoft folks do not want to be bothered by XP users and allow unlimited reactivations for XP.

If the VM does not need Internet access, then in the VirtualBox GUI disable the virtual network cable.

If the VM needs Internet access, then DNS resolution might pose a problem. When using the default NAT interface, use vboxmanage to use the host system for DNS resolution:

vboxmanage modifyvm "Name of the VM" --natdnshostresolver1 on

Copy the Hard Disk MBR

If a dual boot system is already configured and GRUB is being used, then the original Windows MBR will have been overwritten by the GRUB boot loader. Do not copy and use the MBR if GRUB is already configured.

If GRUB has not yet been installed to the MBR, grab a copy of the original disk MBR using dd:

dd if=/dev/sda of=/home/username/VirtualBox\ VMs/Name_of_the_VM/windows.mbr count=1 bs=512

When GRUB has already been installed to the MBR, there are two ways to boot the Windows VM:

Temporarily restore the MBR from Windows and copy the MBR to a file.
Use a GRUB boot ISO image.

A GRUB boot ISO must be used with GPT hard drives.

Create a GRUB Boot ISO

If the dual boot configuration has already wiped the original Windows disk MBR, create a GRUB boot ISO to boot the VM. Do this on the host Linux system.

    mkdir -p /root/iso/boot
    cp -a /boot/grub /root/iso/boot/

Edit /root/iso/boot/grub.cfg:

Delete all boot menuentries except the Windows entry.
Change the default boot entry to 0 (zero).
If desired, change the timeout to 0 (zero).

Create the GRUB boot ISO:

    cd /root
    grub-mkrescue -o windows-boot.iso iso

Move the windows-boot.iso ISO image to the virtual machine storage directory:

mv /root/windows-boot.iso /home/username/VirtualBox\ VMs/Name_of_the_VM/

Update the ownership and permissions of the moved file for the user account.

Use the VirtualBox GUI to add the ISO image to the Storage devices. Update the boot device order to use the Optical disk.

On Linux VMs that will be run with raw disk access, these same steps must be repeated inside that system. The GRUB boot ISO needs to be updated every time the kernel is updated.

Configure the VM to Use Raw Disk Access

When creating the vmdk container, the VirtualBox vboxmanage command supports an mbr option to point to an MBR file. This mechanism allows booting Windows as though booting from the hard drive. That option will be meaningless if the original MBR no longer exists and unable to create an MBR file.

First confirm the partitions of the system to be run in raw disk mode:

vboxmanage internalcommands listpartitions -rawdisk /dev/sda

Typically on a Windows system at least the first two partitions, /dev/sda1 and /dev/sda2, will be part of the Windows systems. Other systems might include /dev/sda3 or /dev/sda4.

vboxmanage internalcommands createrawvmdk -filename “/path/to/file.vmdk” -rawdisk /dev/sda -partitions 1[,2,3,4] -relative -mbr /file/path/to/filename.mbr

The latter part of the command is not needed if using a GRUB boot ISO.

Post Install

If the VM will be run in full desktop mode, then after successfully booting into the VM, install the Guest Additions to provide better display resolution and to support shared folders. A VirtualBox shared folder is great for shuffle files between the two operating systems. For systems that do not boot into desktop mode, the GA are not needed.

Posted: May 03, 2019 Category: Tutorial Tagged: Windows, Virtual Machines

Next: Firefox Add-on Certificate Expiration

Previous: Migrating a Business to Linux — 18